Mechanical arm registration method and mechanical arm registration device

Abstract

The invention relates to a mechanical arm registration method and a mechanical arm registration device. A registration array is arranged at the tail end of the mechanical arm; the method comprises the steps that multiple tail end poses of the mechanical arm are determined based on the moving space of the mechanical arm; under each tail end pose, the included angle between the plane where the registration array is located and the optical axis of the optical navigation equipment is greater than a preset included angle threshold value; under the tail end poses, first position information of the registration array under a reference coordinate system of the mechanical arm and second position information of the registration array under a reference coordinate system of the optical navigation equipment are obtained; and determining a target coordinate conversion relationship between the optical navigation equipment and the mechanical arm based on the first position information and the second position information under each tail end pose. When mechanical arm registration is carried out, it is ensured that the plane where the registration array is located is almost perpendicular to the optical axis of the optical navigation equipment, and the registration array is made to be in the measuring posture right facing the optical navigation equipment; the position measurement precision of the optical navigation equipment can be improved, and the registration accuracy of the mechanical arm is improved.

Description

Technical Field

[0001] The present application relates to the field of optical navigation technology, and in particular to a robot arm registration method and a robot arm registration device. Background Art

[0002] With the development of surgical robots, in the process of surgical robot-assisted surgery, an optical navigation system is needed to guide the surgical robot for navigation and positioning.

[0003] In order to enable the optical navigation system to guide the surgical robot to perform navigation and positioning, the robotic arm of the surgical robot needs to be spatially registered, that is, the spatial position relationship between the reference coordinate system of the robotic arm (that is, the base coordinate system of the robot) and the coordinate system of the optical navigation system needs to be determined.

[0004] However, the traditional method of spatial registration of the robotic arm of a surgical robot has the problem of low registration accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide a robot arm registration method, device, robot, computer-readable storage medium and computer program product that can improve the accuracy of robot arm registration in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for registering a robot arm. A registration array is provided at the end of the robot arm; the method comprises:

[0007] Based on the movement space of the robotic arm, multiple end positions of the robotic arm are determined; in each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0008] Under each end position, obtaining first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device;

[0009] Based on the first position information and the second position information at each end position, a target coordinate conversion relationship between the optical navigation device and the robotic arm is determined.

[0010] In one embodiment, based on the movement space of the robotic arm, multiple end positions of the robotic arm are determined, including:

[0011] Based on the moving space of the robot arm, determine the coordinates of multiple end positions of the robot arm;

[0012] Based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, the registration array posture is determined; under the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0013] According to the multiple end position coordinates and the registered array postures, multiple end positions of the robot arm are determined.

[0014] In one embodiment, determining the registration array posture based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm includes:

[0015] Based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, an X-axis direction vector corresponding to the registration array is determined; the angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a preset angle threshold;

[0016] Determine the direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis;

[0017] Based on the X-axis direction vector and the direction vector of the first coordinate axis, determine the direction vector of the second coordinate axis; the second coordinate axis is the other coordinate axis between the Y-axis and the Z-axis;

[0018] The registration array posture is determined based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

[0019] In one embodiment, determining multiple end positions of the robot arm according to multiple end position coordinates and registered array postures includes:

[0020] For each end position coordinate, based on the registered array posture and the end position coordinate, generate a registered array posture matrix corresponding to the end position coordinate;

[0021] According to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, the registered array posture matrix corresponding to the terminal position coordinates is rotated to obtain the terminal posture corresponding to the terminal position coordinates.

[0022] In one embodiment, obtaining second position information of the registration array in a reference coordinate system of the optical navigation device at each terminal position includes:

[0023] For each end position, obtain the target joint angle group corresponding to the end position;

[0024] Based on the target joint angle group corresponding to each end posture, the end of the surgical robot is controlled to adjust to each end posture in turn, and in each end posture, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0025] In one embodiment, obtaining a target joint angle group corresponding to the end pose includes:

[0026] Inputting the end position into a preset inverse kinematics model to obtain at least one robot arm joint angle group corresponding to the end position;

[0027] Selecting a candidate robot arm joint angle group that meets a preset collision condition from at least one robot arm joint angle group;

[0028] Based on the candidate robot arm joint angle group, determine the target joint angle group corresponding to the end pose.

[0029] In one embodiment, when the candidate robot arm joint angle groups include multiple ones, determining the target joint angle group corresponding to the end pose based on the candidate robot arm joint angle groups includes:

[0030] Calculate the bi-norm between each candidate robot arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture; arrange the multiple end postures in a preset order to instruct the surgical robot to adjust to each end posture in sequence according to the preset order;

[0031] The candidate robot arm joint angle group corresponding to the minimum square norm is used as the target joint angle group corresponding to the end posture.

[0032] In one embodiment, the method further comprises:

[0033] Acquire an original coordinate conversion relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the moving space of the robotic arm;

[0034] Constructing a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set and the second coordinate point set;

[0035] The three-dimensional space model is visualized to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0036] In one embodiment, the initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the method further comprises:

[0037] Based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the robot arm;

[0038] Calculating a spatial coverage ratio between a field of view space of the optical navigation device and a moving space of the surgical robot based on target coordinates of each coordinate point in the first coordinate point set and initial coordinates of each coordinate point in the second coordinate point set;

[0039] Based on the spatial coverage and a preset coverage threshold, a recommendation result of the optical field of view of the optical navigation device is determined; the recommendation result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

[0040] In a second aspect, the present application also provides a robot arm registration device. A registration array is provided at the end of the robot arm; the device comprises:

[0041] A first determination module is used to determine multiple end positions of the robotic arm based on the movement space of the robotic arm; in each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0042] A first acquisition module is used to acquire, at each end position, first position information of the registration array in a reference coordinate system of the manipulator and second position information of the registration array in a reference coordinate system of the optical navigation device;

[0043] The registration module is used to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information under each end posture.

[0044] In a third aspect, the present application further provides a robot, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the robot arm registration method in the first aspect when executing the computer program.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the robot arm registration method in the first aspect.

[0046] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the robot arm registration method in the first aspect are implemented.

[0047] The above-mentioned robot arm registration method, device, robot, storage medium and computer program product determine multiple terminal positions of the robot arm based on the moving space of the robot arm; wherein, at each terminal position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; at each terminal position, the first position information of the registration array in the reference coordinate system of the robot arm and the second position information of the registration array in the reference coordinate system of the optical navigation device are obtained; based on the first position information and the second position information at each terminal position, the target coordinate conversion relationship between the optical navigation device and the robot arm is determined. That is, the robot arm registration method proposed in the embodiment of the present application adopts a registration array with a planar structure, which is installed at the end of the robot arm, and when the end of the robot arm is controlled to be in different spatial positions, it is ensured that the plane where the registration array is located is nearly perpendicular to the optical axis of the optical navigation device, so that the registration array is in a measurement posture facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device and reducing the measurement error of the optical navigation device; further, based on the high-precision position measurement information, a high-precision robot arm registration result can be obtained, thereby improving the accuracy of robot arm registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram showing an application environment of a robot arm registration method in an embodiment;

[0049] Figure 2 A schematic diagram of a process flow of a robot arm registration method in one embodiment;

[0050] Figure 3 A schematic diagram of coordinate transformation for robot arm registration in one embodiment;

[0051] Figure 4 A schematic diagram of coordinate transformation of a registration array and a flange in one embodiment;

[0052] Figure 5 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0053] Figure 6 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0054] Figure 7 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0055] Figure 8 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0056] Fig. 9 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0057] Fig.10is a schematic diagram of the structure of the field of view space in one embodiment;

[0058] Fig.11 A schematic diagram of the structure of a three-dimensional space model in an embodiment;

[0059] Fig.12 A schematic diagram of a flow chart of a robot arm registration method in another embodiment;

[0060] Fig.13 A schematic diagram of structural parameters of a viewing field space in an embodiment;

[0061] Fig.14 A schematic diagram of a structure for determining whether a viewing space covers a moving space in one embodiment;

[0062] Fig.15 A schematic diagram of a process of visually adjusting the optical field of view in one embodiment;

[0063] Fig.16 is a structural block diagram of a robot arm registration device in one embodiment;

[0064] Fig.17 1 is a diagram showing the internal structure of a robot in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] In the field of surgical navigation, an optical navigation system is needed to guide surgical instruments or guide surgical robots to perform surgical positioning. In order to enable the optical navigation system to guide the surgical robot to complete surgical positioning, it is necessary to perform spatial registration of the robotic arm, that is, to calculate the spatial position relationship between the reference coordinate system of the robotic arm and the reference coordinate system of the optical navigation system.

[0067] However, the traditional robot arm registration method has the problem of low registration accuracy.

[0068] In addition, there are two complex issues in the registration process. The first is to ensure that the motion space of the robot is within the field of view of the optical navigation system, and the second is to ensure that the motion configuration generated by the robot does not block the optical path and the registered configuration does not collide with the environment.

[0069] Based on the above technical problems, this application proposes a robot arm registration method, which can improve the registration accuracy of the robot arm.

[0070] The robot arm registration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the robot 102 includes a base and a mechanical arm, and a registration array 104 is arranged at the end of the mechanical arm; a plurality of registration objects can be arranged on the registration array 104 as a plurality of registration points; illustratively, the registration array 104 can be a planar structure, and a plurality of registration objects can be arranged at different positions on the plane of the registration array 104.

[0071] The robot 102 may include but is not limited to an industrial robot, a medical robot, etc.; such as a surgical robot.

[0072] In one embodiment, Figure 2 As shown, a robot arm registration method is provided, which is applied to Figure 1 The robot in the example is used to illustrate the following steps:

[0073] Step 202: Determine multiple end positions of the robotic arm based on the movement space of the robotic arm.

[0074] Among them, in each terminal position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold.

[0075] Among them, the moving space of the robot arm can be a moving space range of the end of the robot arm that is pre-set according to the end operation of the robot arm; for example, the moving space can be a cube area, and the moving space of the end of the robot arm can be determined by the coordinates of each vertex of the cube. It should be noted that the coordinates of each vertex can be the coordinates in the reference coordinate system of the robot arm, or the coordinates in the world coordinate system; the embodiments of the present application do not impose too many restrictions on this. In addition, the reference coordinate system of the robot arm can be the robot base coordinate system, that is, the coordinate system where the robot base is located.

[0076] When the moving space of the robot arm is determined, multiple end positions of the robot arm can be determined within the moving space; wherein the end position can include the end position and the end posture; the end position determined within the moving space can be understood as the end position of the robot arm being located within the moving space.

[0077] Exemplarily, based on the multiple vertex coordinates of the moving space, multiple position coordinate points can be randomly determined in the moving space, or multiple uniformly distributed position coordinate points can be regularly determined as multiple end positions of the robotic arm.

[0078] In addition, for each end position, it is also necessary to determine the end posture corresponding to each end position, so as to generate multiple end postures of the robot arm based on each end position and its end posture. Among them, the end posture of the robot arm at each end position must be able to make the angle between the plane where the registration array at the end of the robot arm is located and the optical axis of the optical navigation device greater than the preset angle threshold. For example: the end posture of the robot arm must make the plane where the registration array installed at the end of the robot arm is located perpendicular to the optical axis of the optical navigation device; that is, the registration array must be facing the optical navigation device. In this way, in the facing posture, the position measurement error of the registration array by the optical navigation device can be reduced, thereby reducing the registration error of the robot arm and improving the registration accuracy of the robot arm.

[0079] Step 204 : acquiring, at each end position, first position information of the registration array in the reference coordinate system of the robot arm and second position information of the registration array in the reference coordinate system of the optical navigation device.

[0080] The reference coordinate system of the manipulator may be a robot base coordinate system, that is, a coordinate system where the robot base is located. The reference coordinate system of the optical navigation device may be a reference coordinate system of the optical navigation system, that is, a coordinate system where the optical navigation device performs position measurement.

[0081] When multiple end positions of the robot arm are determined as mentioned above, the end of the robot arm can be controlled to be in each end position in sequence, and in each end position, the robot can be controlled to obtain the first position information of the registration array in the reference coordinate system of the robot arm, and the optical navigation device can be controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0082] refer to Figure 3 As shown, it shows a schematic diagram of robot arm registration. Exemplarily, for a robot, when a registration array or other terminal components such as operating instruments are set at the end of the robot, the terminal components are usually installed at the end of the robot through a flange; based on this, the coordinate system corresponding to the robot can include the reference coordinate system (Link_1) of the robot arm, the flange coordinate system (FlangeFrama) of the end of the robot arm, the installation coordinate system (InstallFrame) and the registration array coordinate system (RobotArrayFrame); wherein the reference coordinate system (Link_1) of the robot arm is the motion reference of the robot arm, the flange coordinate system (FlangeFrama) is the end position of the robot arm, and the registration array is a robot arm array tool installed on the flange of the robot arm, which can be recognized by the optical navigation system.

[0083] If you want to obtain the position information of the registered array in the reference coordinate system of the robot, you need to know the coordinate transformation relationship between the registered array coordinate system and the flange coordinate system, and the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the robot. Figure 4 As shown, it shows the relationship between the registration array coordinate system and the flange coordinate system. It should be noted that after the registration array is installed at the end of the robotic arm, the relative position relationship between the registration array and the flange is fixed, that is, the coordinate conversion relationship between the registration array coordinate system and the flange coordinate system is also fixed.

[0084] In addition, for the coordinate conversion relationship between the flange coordinate system and the reference coordinate system of the robot arm, when the end of the robot arm moves to different spatial positions, the coordinate conversion relationship between the flange coordinate system and the reference coordinate system of the robot arm can be determined based on the motion information of each joint of the robot arm at this time. It should be noted that the coordinate conversion relationship between the flange coordinate system and the reference coordinate system of the robot arm is different for different end positions of the robot arm.

[0085] Then, according to the coordinate transformation relationship between the registered array coordinate system and the flange coordinate system, and the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the robot, the coordinate transformation relationship between the registered array coordinate system and the reference coordinate system of the robot can be determined, that is,

[0086] Then, at each terminal posture, the coordinate conversion relationship between the flange coordinate system and the reference coordinate system of the robot arm at the terminal posture can be first determined according to the terminal posture; then, the coordinate conversion relationship between the registered array coordinate system and the reference coordinate system of the robot arm and the coordinate conversion relationship between the registered array coordinate system and the flange coordinate system can be determined; further, according to the coordinate conversion relationship between the registered array coordinate system and the reference coordinate system of the robot arm, the position information of each registration point on the registration array in the registration array coordinate system is subjected to coordinate conversion processing, so that the first position information of the registration array in the reference coordinate system of the robot arm at the terminal posture can be obtained; wherein, the first position information includes the position information of each registration point on the registration array in the reference coordinate system of the robot arm at the terminal posture.

[0087] Continue to refer to the above Figure 3 As shown, exemplarily, for an optical navigation system, the optical navigation system is mainly composed of an optical navigation device, and a reference coordinate system of the optical navigation device, that is, a reference coordinate system of the optical navigation system, can be defined as an OpticalTrackFrame.

[0088] At each terminal position, the optical navigation device can be controlled to shoot the registration array, thereby obtaining the second position information of the registration array in the reference coordinate system of the optical navigation device; wherein the second position information includes the position information of each registration point on the registration array in the reference coordinate system of the optical navigation device at the terminal position. Then, the optical navigation device can send the second position information obtained at each terminal position to the robot.

[0089] Step 206: Determine a target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each end position.

[0090] Exemplarily, when the robot obtains the first position information of the registration array in the reference coordinate system of the robotic arm and the second position information of the registration array in the reference coordinate system of the optical navigation device at each end posture, the robot can perform coordinate transformation based on the first position information and the second position information to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm.

[0091] In the above-mentioned robot arm registration method, multiple end positions of the robot arm are determined based on the moving space of the robot arm; wherein, at each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; at each end position, the first position information of the registration array in the reference coordinate system of the robot arm and the second position information of the registration array in the reference coordinate system of the optical navigation device are obtained; based on the first position information and the second position information at each end position, the target coordinate conversion relationship between the optical navigation device and the robot arm is determined. That is, the robot arm registration method proposed in the embodiment of the present application adopts a registration array with a planar structure, which is installed at the end of the robot arm, and when the end of the robot arm is controlled to be in different spatial positions, it is ensured that the plane where the registration array is located is nearly perpendicular to the optical axis of the optical navigation device, so that the registration array is in a measurement posture facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device and reducing the measurement error of the optical navigation device; further, based on the high-precision position measurement information, a high-precision robot arm registration result can be obtained, thereby improving the accuracy of robot arm registration.

[0092] Figure 5 FIG. 1 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process of determining multiple end positions of the robot arm based on the movement space of the robot arm. Based on the above embodiment, Figure 5 As shown, the above step 202 includes:

[0093] Step 502: Determine multiple end position coordinates of the robotic arm based on the movement space of the robotic arm.

[0094] The moving space of the robot arm may be a space region surrounded by a plurality of spatial position points (ie vertices).

[0095] For example, the robot can determine multiple terminal position coordinates located in the mobile space based on multiple vertex coordinates of the mobile space; wherein each terminal position coordinate can be a three-dimensional space coordinate in the mobile space, such as P r (x r ,y r ,z r ).

[0096] Step 504: Determine the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the robotic arm.

[0097] Among them, in the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than the preset angle threshold. Since the angle range of the line-plane angle is 0 to 90 degrees, in order to ensure that the plane where the registration array is located is approximately perpendicular to the optical axis of the optical navigation device, the preset angle threshold should be an angle threshold that is less than and close to 90 degrees. For example: the angle difference between the preset angle threshold and 90 degrees can be determined based on the measurement accuracy of the optical navigation device, and the posture facing the optical navigation device can be reasonably defined based on the angle difference. Exemplarily, the angle difference can be π / 10, then the preset angle threshold can be any angle greater than or equal to 72 degrees and less than or equal to 90 degrees, that is, any angle within the angle range of [72,90].

[0098] Exemplarily, when determining the registration array posture, it can be determined based on the reference coordinate system of the optical navigation device and the registration array coordinate system; for example: Figure 3 As shown, it can be seen that for the registration array coordinate system, its X-axis is perpendicular to the plane where the registration array is located; and for the reference coordinate system of the optical navigation device, its Z-axis can be expressed as the optical axis direction of the optical navigation device; then, in the position facing the optical navigation device, the X-axis of the registration array coordinate system should be nearly parallel to the Z-axis of the reference coordinate system of the optical navigation device.

[0099] Based on this logic, the coordinate system direction of the registration array can be determined according to the reference coordinate system of the optical navigation device, thereby obtaining the registration array posture; wherein the registration array posture can be represented in the form of a matrix, such as: T = [x_vec y_vec z_vec].

[0100] Step 506 , determining multiple end positions and postures of the robot arm according to the multiple end position coordinates and the registered array postures.

[0101] Exemplarily, for each terminal position coordinate, the terminal position coordinate and the registered array posture are combined to obtain the terminal posture corresponding to the terminal position coordinate; multiple terminal postures are stored in the form of a matrix to obtain the registered array posture matrix T', which can be expressed as:

[0102]

[0103] In this embodiment, when determining multiple end position postures of the robot arm based on the moving space of the robot arm, first determine multiple end position coordinates of the robot arm based on the moving space of the robot arm; then, determine the registration array posture based on the initial coordinate conversion relationship between the optical navigation device and the robot arm; wherein, under the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; then, determine multiple end position postures of the robot arm according to the multiple end position coordinates and the registration array posture. That is, in this embodiment, by determining the registration array posture when facing the optical navigation device, and then combining the registration array posture and different end position coordinates, multiple end postures of the registration array are obtained; so that when the end of the robot arm is at different end positions, it maintains the posture facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device, and then providing high-precision position measurement data for the robot arm registration, to improve the accuracy of the robot arm registration.

[0104] Figure 6 FIG. 1 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process for determining the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the robot arm. Based on the above embodiment, Figure 6 As shown, the above step 504 includes:

[0105] Step 602: Determine the X-axis direction vector corresponding to the registration array based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm.

[0106] The angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is smaller than a preset angle threshold.

[0107] Step 604 : determining a direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis.

[0108] The first coordinate axis is any one of the Y axis and the Z axis.

[0109] Step 606: Determine the direction vector of the second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis.

[0110] The second coordinate axis is the other coordinate axis between the Y axis and the Z axis.

[0111] Step 608, determining the registration array posture based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

[0112] refer to Figure 3 As shown, when the registration array is in a position facing the optical navigation device, the X-axis of the registration array coordinate system and the Z-axis of the reference coordinate system of the optical navigation device are parallel to each other; then, based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, the Z-axis direction vector of the reference coordinate system of the optical navigation device can be converted into the direction vector of the X-axis of the reference coordinate system of the robotic arm; that is, the direction vector of the Z-axis of the reference coordinate system of the optical navigation device in the reference coordinate system Link_1 of the robotic arm is obtained, which is recorded as x_vec.

[0113] Since the purpose of registering the robotic arm is to obtain the precise coordinate conversion relationship between the optical navigation device and the robotic arm, at this time, other methods can be used to first obtain the initial coordinate conversion relationship between the optical navigation device and the robotic arm; so as to determine the position and posture facing the optical navigation system based on the initial coordinate conversion relationship.

[0114] For example, referring to the relevant contents of step 204 above, the end of the robot arm is controlled to be in a certain end position, and the first coordinate transformation relationship between the registration array and the robot base can be obtained, that is, Next, the end of the robotic arm is kept stationary, and the optical navigation device is controlled to shoot the registration array at the end position to obtain the second coordinate transformation relationship between the registration array and the optical navigation device, that is, Then, according to the first coordinate transformation relationship and the second coordinate transformation relationship, the initial coordinate transformation relationship between the optical navigation device and the robot arm is determined, that is,

[0115] in,

[0116]

[0117] Among them, inv() represents the inversion operation, that is, calculating the inverse matrix of a square matrix.

[0118] Based on this, we can convert the initial coordinates The direction vector of the Z axis in the reference coordinate system of the optical navigation device under the reference coordinate of the robot arm is obtained as the X axis of the registration array coordinate system, that is, x_vec.

[0119] Next, after the X axis of the registration array coordinate system is determined, the Y axis and the Z axis of the registration array coordinate system may be determined based on the X axis of the registration array coordinate system.

[0120] For example, the Y axis of the registration array coordinate system can be constrained first so that the angle between the Y axis of the registration array coordinate system and the bed is (0, pi / 2). For example, the Y axis constraint angle of the registration array coordinate system can be randomly determined within 0 to pi / 2, denoted as α∈(0, pi / 2). Then, based on the Y axis constraint angle, the Y axis direction vector y_vec and the Z axis direction vector z_vec of the registration array coordinate system are determined.

[0121] Exemplarily, the process of determining the Y-axis direction vector y_vec and the Z-axis direction vector z_vec may include:

[0122] a. Calculate the base vector of the Y axis; for example, the preset initial vector of the Y axis, such as the vector (0, -1, 0), can be calculated and projected onto the YZ plane of the robot registration array to obtain the base vector base_vec of the Y axis.

[0123] base_vec=(0,-1,0)·sin(arccos(x_vec·(0,-1,0)))

[0124] b. Calculate the base vector base_vec of the Y axis, and use the vector after rotating the X-axis direction vector x_vec by an angle α as the Y-axis direction vector y_vec.

[0125] According to the rodgrigues rotation formula:

[0126] y_vec=base_vec·cosα+x_vec×base_vec·sinα+(1-cosα)

[0127] ·(x_vec · base_vec) · x_vec

[0128] c. Calculate the Z-axis direction vector z_vec, that is, z_vec = x_vec × y_vec.

[0129] d. Normalize x_vec, y_vec and z_vec.

[0130] x_vec = x_vec / norm(x_vec)

[0131] y_vec = y_vec / norm(y_vec)

[0132] z_vec=z_vec / norm(z_vec)

[0133] Thus, the registered array posture T = [x_vec y_vec z_vec] is obtained.

[0134] Through the above steps, the posture of the registered array facing the optical navigation device can be determined, and then, based on the registered array posture, the end posture of the robotic arm facing the optical navigation device can be obtained, so that the robotic arm can maintain a posture facing the optical navigation device when it is in different end positions, thereby improving the position detection accuracy of the optical navigation device for the registration array, and then improving the registration accuracy of the robotic arm.

[0135] Figure 7 FIG. 1 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process of determining multiple end positions of a robot arm according to multiple end position coordinates and registered array postures. Based on the above embodiment, Figure 7 As shown, the above step 506 includes:

[0136] Step 702 : for each terminal position coordinate, based on the registered array posture and the terminal position coordinate, generate a registered array posture matrix corresponding to the terminal position coordinate.

[0137] Referring to the above step 506, based on the registered array posture and multiple end position coordinates, a registered array posture matrix T' facing the optical navigation device and satisfying the angle constraint can be obtained, that is:

[0138]

[0139] Step 704, rotating the registered array pose matrix corresponding to the terminal position coordinates according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, to obtain the terminal pose corresponding to the terminal position coordinates.

[0140] For example, after obtaining the registered array pose matrix T', the registered array pose matrix T' can also be randomly rotated around the Y axis and Z axis of the registered array coordinate system within the angle range of (-pi / 10, pi / 10); thereby obtaining a pose matrix T with pitch and yaw angles within the range of (-pi / 10, pi / 10). goal .

[0141] Exemplarily, the preset rotation angle of the Y axis can be randomly determined within the angle range of (-pi / 10, pi / 10), which is recorded as the random Euler angle random_y_theta of the Y axis; and the preset rotation angle of the Z axis can be randomly determined within the angle range of (-pi / 10, pi / 10), which is recorded as the random Euler angle random_z_theta of the Z axis.

[0142] Next, based on the random Euler angles random_y_theta and random Euler angles random_z_theta, the registered array pose matrix T' is rotated to obtain the rotated pose matrix Tgoal .Right now

[0143] T goal =T·Euler_matrix(0,random_y_theta,rndom_z_theta)

[0144] At this point, the rotated pose matrix T goal As the end pose matrix of the robot arm.

[0145] In this embodiment, for each end position coordinate, firstly, based on the registered array posture and the end position coordinate, a registered array posture matrix corresponding to the end position coordinate is generated; then, according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, the registered array posture matrix corresponding to the end position coordinate is rotated, and finally the end posture matrix of the manipulator is obtained. By adopting this method, when the end of the manipulator is in different spatial positions, it can not only ensure that the position measurement error is met in the X-axis direction facing the optical navigation device, but also ensure that the position measurement error of the optical navigation device is met in both the pitch and yaw directions; thereby obtaining the end posture of the manipulator that meets the position measurement error of the optical navigation device in three-dimensional space and faces the optical navigation device; so that under this end posture, the optical navigation device can achieve high-precision position detection of the registered array at the end of the manipulator, thereby improving the registration accuracy of the manipulator.

[0146] In one embodiment, in view of the many problems faced by traditional robot arm registration methods, during the process of robot arm registration, it is necessary not only to improve the position detection accuracy of the registration array installed at the end of the robot arm by the optical navigation device; in addition, when the robot arm is in the end position, the robot arm configuration at this time should also meet the requirement of not blocking the light path of the optical navigation device, otherwise the optical navigation device will not be able to recognize the registration array at the end of the robot arm, and will not be able to detect the position information of the registration array in the reference coordinate system of the optical navigation device under the end position. In addition, when controlling the movement of the end of the robot arm to the end position, the robot arm configuration should not have environmental collisions, otherwise the end of the robot arm may be outside the moving space of the robot arm, which may cause the optical navigation device to be unable to recognize the end of the robot arm, and thus be unable to obtain the position information of the registration array in the reference coordinate system of the optical navigation device.

[0147] Figure 8 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process of obtaining the second position information of the registration array in the reference coordinate system of the optical navigation device at each end position. On the basis of the above embodiment, Figure 8 As shown, the above step 204 includes:

[0148] Step 802: For each terminal posture, obtain the target joint angle group corresponding to the terminal posture.

[0149] The target joint angle group corresponding to the terminal posture includes the joint angles of each joint of the robot, and the target joint angle group satisfies the preset collision conditions; illustratively, the preset collision conditions include but are not limited to the collision of the robot body, the collision of the robot with other environmental components, the collision of the robot with the surface of the mobile space, and other collision conditions; in the case where the robot is a surgical robot, other environmental components may include a surgical cart, a bed, and other surgical related equipment. In addition, the collision between the robot and the mobile space may include the collision between the robot body and / or the end of the robot and at least one surface of the mobile space.

[0150] Exemplarily, when calculating the target joint angle group of the robotic arm according to the end posture, the inverse kinematics principle can be used to input the end posture into a preset inverse kinematics model to obtain at least one robotic arm joint angle group corresponding to the end posture; if there is only one robotic arm joint angle group at this time, it is determined whether the robotic arm joint angle group meets the preset collision condition; if so, the robotic arm joint angle group is used as the target joint angle group corresponding to the end posture; if not, the preset constraint angle and / or preset rotation angle can be updated for the robotic arm end position coordinates corresponding to the end posture, and the above steps of determining the registered array posture and the end posture are re-executed to obtain a new end posture corresponding to the end position coordinates; and then, based on the new end posture, the corresponding robotic arm joint angle group is re-determined, and then the target joint angle group corresponding to the new end posture is determined.

[0151] It should be noted that after the inverse kinematics calculation, if there is no robot arm joint angle group corresponding to the end posture, then the new end posture can be determined by updating the preset constraint angle and / or the preset rotation angle, and then the target joint angle group corresponding to the new end posture can be re-determined.

[0152] Exemplarily, if there are multiple robot arm joint angle groups corresponding to the end posture, a candidate robot arm joint angle group that meets the preset collision condition can be screened out from the multiple robot arm joint angle groups; and based on the candidate robot arm joint angle group, the target joint angle group corresponding to the end posture is determined.

[0153] Exemplarily, if there is one candidate robot arm joint angle group that satisfies the preset collision condition, then the candidate robot arm joint angle group that satisfies the preset collision condition can be directly used as the target joint angle group corresponding to the end posture. Exemplarily, if there is no candidate robot arm joint angle group that satisfies the preset collision condition after collision screening, then the above-mentioned step of determining the end posture can also be returned to determine a new end posture by updating the preset constraint angle and / or the preset rotation angle, and then the target joint angle group corresponding to the new end posture is re-determined.

[0154] Exemplarily, if there are multiple candidate robot arm joint angle groups that meet the preset collision conditions, any one candidate robot arm joint angle group can be determined from the multiple candidate robot arm joint angle groups as the target joint angle group corresponding to the end posture; or the target joint angle group can be determined from the multiple candidate robot arm joint angle groups according to a preset determination rule.

[0155] Exemplarily, multiple end postures of the robotic arm can be arranged in a preset order to instruct the robot to adjust to each end posture in sequence according to the preset order; based on this, when there are multiple candidate robotic arm joint angle groups corresponding to the current end posture, the target joint angle group corresponding to the previous end posture adjacent to the current end posture can be obtained, and the binary norm between each candidate robotic arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture is calculated; then, the candidate robotic arm joint angle group corresponding to the minimum binary norm is used as the target joint angle group corresponding to the current end posture.

[0156] It should be noted that when the current end posture is the first end posture among multiple end postures, the candidate robotic arm joint angle groups corresponding to the first end posture can be compared with the initialized joint angle group of the robotic arm, so as to select the one with the smallest two-norm as the corresponding target joint angle group of the first end posture.

[0157] Assume that the candidate robot arm joint angle groups corresponding to the current end pose are defined as j1, j2, j3, ..., j n , the target joint angle group corresponding to the previous end position adjacent to the current end position or the initialization joint angle group of the robot arm is defined as j cur , then the two norms can be expressed as diff k =‖j k -j cur ‖ 2 , where k = 1,...,n. Next, diff k The minimum value of j kThe target joint angle group corresponding to the current end posture is added to the joint angle pool of the robotic arm; wherein the joint angle pool of the robotic arm is used to sequentially store the target joint angle groups corresponding to multiple end postures of the robotic arm.

[0158] Exemplarily, when calculating the target joint angle groups corresponding to each terminal posture, it is possible to determine whether the number of target joint angle groups in the joint angle pool is equal to a preset number. If so, the loop is exited to complete the calculation of the solution pool; if not, the calculation of the target joint angle group for the next terminal posture is continued in a loop; wherein the preset number of target joint angle groups in the joint angle pool may be less than or equal to the number of terminal postures. That is, based on the above-mentioned method for determining the target joint angle groups, the target joint angle groups corresponding to each terminal posture can be determined in a loop in sequence.

[0159] By calculating the binary norm between the candidate robot arm joint angle group corresponding to the current end position and the target joint angle group corresponding to the previous end position, and selecting the candidate robot arm joint angle group corresponding to the minimum binary norm as the target joint angle group corresponding to the current end position; the movement degree of the robot arm end can be minimized when it moves from the previous end position to the current end position; it is not only conducive to the efficient movement of the robot arm, but also can improve the movement efficiency of the robot arm, thereby improving the registration efficiency of the robot arm.

[0160] Step 804, based on the target joint angle group corresponding to each end posture, control the end of the surgical robot to adjust to each end posture in sequence, and at each end posture, control the optical navigation device to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0161] In this embodiment, for each terminal posture, firstly based on the preset collision condition, the target joint angle group corresponding to the terminal posture that satisfies the preset collision condition is obtained; then, based on the target joint angle group corresponding to each terminal posture, the terminal of the surgical robot is sequentially controlled to adjust to each terminal posture, and in each terminal posture, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device. That is, by adopting the method in this embodiment, when registering the robot arm, it can be ensured that when the end of the robot arm moves in the mobile space, the configuration of the robot arm will not block the optical path of the optical navigation device, so that the optical navigation device can accurately detect the position of the registration array installed at the end of the robot arm; and the configuration of the robot arm will not cause body collision, collision with the mobile space, and collision with the surrounding environment, so that any problems caused by collision can be solved; the efficiency and accuracy of robot arm registration are improved.

[0162] In one embodiment, during the process of registering the robotic arm, the traditional method also faces a problem, that is, it is necessary to ensure that the movement space of the robotic arm is within the field of view of the optical navigation system, so as to prevent the optical navigation system from being unable to identify the registration array at the end of the robotic arm, resulting in poor accuracy of robotic arm registration, or even failure to complete robotic arm registration. Based on this, before registering the robotic arm, the optical field of view of the optical navigation device can be adjusted so that the optical field of view of the optical navigation device can completely cover the moving space of the robotic arm; when the end of the robotic arm moves in the moving space, the optical navigation device can always track and identify the end of the robotic arm, thereby navigating and positioning the operating components installed at the end.

[0163] Fig. 9 FIG. 1 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process of dynamically visually adjusting the optical field of view of an optical navigation device. Based on the above embodiment, Fig. 9 As shown, the above method also includes:

[0164] Step 902: Acquire the original coordinate conversion relationship between the optical navigation device and the mechanical arm, the first coordinate point set corresponding to the field of view space of the optical navigation device, and the second coordinate point set corresponding to the movement space of the mechanical arm.

[0165] Among them, the original coordinate conversion relationship between the optical navigation device and the robotic arm can be the coordinate conversion relationship between the two determined after the optical navigation device and the robotic arm are initialized based on the position state of the optical navigation device and the position state of the robotic arm after initialization.

[0166] For example, after initialization, the robot can calculate the coordinate transformation relationship between the registration array at the end of the robot arm and the robot base through the forward kinematics model based on the current joint information of the robot arm, that is, At the same time, the optical navigation device is controlled to shoot the registration array to obtain the coordinate conversion relationship between the registration array and the optical navigation device, that is, Next, based on the coordinate transformation relationship between the registration array and the robot base and the coordinate transformation relationship between the registration array and the optical navigation device Calculate the original coordinate transformation relationship between the optical navigation device and the robotic arm The specific calculation method can refer to the above Figure 6 The description of the relevant contents in the illustrated embodiment will not be repeated here.

[0167] In addition, the first coordinate point set corresponding to the field of view space of the optical navigation device may be a spatial point set determined based on the reference coordinate system of the optical navigation device and used to characterize the field of view space of the optical navigation device, and the initial coordinates of each coordinate point in the first coordinate point set may be the coordinates in the reference coordinate system of the optical navigation device. Fig.10 As shown, 8 points can be used to represent the field of view space of the optical navigation device, namely ots_pt1, ots_pt2, ots_pt3, ots_pt4, ots_pt5, ots_pt6, ots_pt7 and ots_pt8.

[0168] Furthermore, the second coordinate point set corresponding to the moving space of the robot arm may be a spatial point set determined based on the reference coordinate system of the robot arm to characterize the moving space of the robot arm, and the initial coordinates of each coordinate point in the second coordinate point set may be the coordinates in the reference coordinate system of the robot arm. Exemplarily, the moving space of the robot arm may also be represented by 8 points.

[0169] Step 904: construct a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set and the second coordinate point set.

[0170] Exemplarily, based on the original coordinate transformation relationship, the initial coordinates of each coordinate point in the first coordinate point set can be converted into target coordinates in the reference coordinate system of the robotic arm; then, the field of view space of the optical navigation device can be constructed based on the target coordinates of each coordinate point in the first coordinate point set, and the moving space of the robotic arm can be constructed based on the initial coordinates of each coordinate point in the second coordinate point set; and then, a three-dimensional space model can be constructed based on the field of view space of the optical navigation device and the moving space of the surgical robot.

[0171] Exemplarily, the three-dimensional space model may also include a three-dimensional model of the robot and a three-dimensional model of the optical navigation device; the three-dimensional space model takes the reference coordinate system of the robot arm as the reference coordinate system, and fills the three-dimensional model of the robot, the moving space of the robot arm, the three-dimensional model of the optical navigation device, and the field of view space of the optical navigation device in the three-dimensional space model; thereby realizing the three-dimensional simulation of the real scene, and building a three-dimensional space model corresponding to the real scene, with reference to Fig.11 shown.

[0172] For example, in a surgical scenario, in addition to a surgical robot and an optical navigation device, a surgical cart system is also required; Figure 3 When registering the robotic arm, the coordinate conversion relationship between the surgical cart system and the optical navigation device and the coordinate conversion relationship between the surgical cart system and the robotic arm can also be registered.

[0173] The surgical trolley system may include a trolley body and a trolley body array (also referred to as a trolley array), and the coordinate system corresponding to the surgical trolley system may include a trolley body coordinate system (TrolleyFrame) and a trolley array coordinate system (TrolleyArrayFrame).

[0174] Exemplarily, during the robot arm registration process, the optical navigation device can simultaneously identify the position information of the registration array at the end of the robot arm and the position information of the trolley array on the surgical trolley system, thereby obtaining the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device, and the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device. Furthermore, the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the trolley array coordinate system (TrolleyArrayFrame) can also be determined based on the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device, and the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device.

[0175] Based on this, the coordinate transformation relationship between the cart array coordinate system (TrolleyArrayFrame) and the robot's reference coordinate system (Link_1) can also be determined based on the coordinate transformation relationship between the registered array coordinate system (RobotArrayFrame) and the cart array coordinate system (TrolleyArrayFrame), and the coordinate transformation relationship between the registered array coordinate system (RobotArrayFrame) and the robot's reference coordinate system (Link_1). Furthermore, based on the coordinate transformation relationship between the cart array coordinate system (TrolleyArrayFrame) and the robot's reference coordinate system (Link_1), the three-dimensional model of the surgical cart system can be added to the above three-dimensional space model.

[0176] Of course, using the above method, the operating table, the surgical object, and other equipment in the surgical environment can also be added to the three-dimensional space model, so as to obtain a more complete three-dimensional space model corresponding to the surgical scene.

[0177] Step 906 : visualize the three-dimensional space model to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0178] Exemplarily, after obtaining the three-dimensional space model, the three-dimensional space model can be visualized; including but not limited to adding color, logo, mark, transparency and other attribute information to each device, component, field of view space, mobile space, etc., to improve the display effect of the three-dimensional model.

[0179] Furthermore, the three-dimensional space model is visualized and outputted, so that the user can adjust the optical field of view of the optical navigation device based on the visualized three-dimensional space model. Through the three-dimensional space model, the user can intuitively see whether the optical field of view of the optical navigation device covers the moving space of the robot arm, and adaptively adjust the position of the optical navigation device.

[0180] In this embodiment, before registering the robotic arm, the original coordinate conversion relationship between the optical navigation device and the robotic arm, the first coordinate point set corresponding to the field of view space of the optical navigation device, and the second coordinate point set corresponding to the mobile space of the robotic arm are first obtained; and a three-dimensional space model is constructed based on the original coordinate conversion relationship, the first coordinate point set, and the second coordinate point set; the three-dimensional space model is visualized to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model. In this way, before registering the robotic arm, the optical field of view of the optical navigation device can be visually and dynamically adjusted to ensure that when registering the robotic arm, the optical field of view of the optical navigation device can completely cover the mobile space of the robotic arm, thereby improving the accuracy of the registration of the robotic arm.

[0181] Fig.12 FIG. 2 is a flow chart of a robot arm registration method in another embodiment. This embodiment relates to an optional implementation process of automatically determining whether the optical field of view of the optical navigation device can cover the moving space of the robot arm. Based on the above embodiment, Fig.12 As shown, the above method also includes:

[0182] Step 1202 , based on the original coordinate conversion relationship, convert the initial coordinates of each coordinate point in the first coordinate point set into target coordinates in the reference coordinate system of the robot arm.

[0183] Step 1204 , based on the target coordinates of each coordinate point in the first coordinate point set and the initial coordinates of each coordinate point in the second coordinate point set, calculate the spatial coverage between the field of view space of the optical navigation device and the moving space of the robot arm.

[0184] Step 1206: Determine a recommendation result of the optical field of view of the optical navigation device based on the spatial coverage and a preset coverage threshold.

[0185] Among them, the recommendation result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the robotic arm.

[0186] That is to say, in this embodiment, first, the coordinates of each coordinate point in the first coordinate point set and the coordinates of each coordinate point in the second coordinate point set are unified into coordinates under one coordinate system; then, under the coordinate system, the coverage ratio between the field of view space and the moving space is calculated to obtain the spatial coverage ratio of the field of view space covering the moving space; then, a preset coverage ratio threshold is used to determine whether the spatial coverage ratio meets the preset spatial coverage requirement; for example: when the spatial coverage ratio is greater than or equal to the preset coverage ratio threshold, the recommended result of the optical field of view of the optical navigation device can be determined as recommended, which indicates that the optical field of view at this time can cover the moving space of the robot arm.

[0187] Exemplarily, the spatial coverage may be determined by the quotient of the volume of the overlapping area between the moving space and the field of view space and the volume of the moving space.

[0188] In another implementation, the robot may also convert the initial coordinates of each coordinate point in the second coordinate point set into target coordinates in the reference coordinate system of the optical navigation device based on the original coordinate conversion relationship.

[0189] Right now

[0190]

[0191] Next, based on the initial coordinates of each coordinate point in the first coordinate point set and the target coordinates of each coordinate point in the second coordinate point set, a recommendation result of the optical field of view of the optical navigation device is determined.

[0192] Assume that the parameters of the field of view space of the optical navigation device are as follows: Fig.13 As shown. For example, reference Fig.14 When the initial coordinates of each coordinate point in the second coordinate point set are converted into the target coordinates in the reference coordinate system of the optical navigation device, any coordinate point in the moving space of the robot arm is taken, such as pt(x p ,y p ,z p ); Take the Z coordinate of the point, Z = z p , intercept the xy cross section; if the point is in the field of view space, the corresponding (x p ,y p ) will fall into the extended rectangle (such as Fig.14 The corresponding mathematical expression is established.

[0193] |x p |<=224+x_board and |y p |<=240+y_board established

[0194] Where x_board=(|zp |-950)*k x ,y_board=(|z p |-950)*k y ;k x and k y is the proportionality coefficient; such as k x =0.2979, k y =0.3745.

[0195] Exemplarily, referring to the above data expression, if the eight vertices of the moving space are all within the field of view space of the optical navigation device, it can be determined that the recommendation result of the optical field of view of the optical navigation device is recommended.

[0196] In this embodiment, based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the manipulator; based on the target coordinates of each coordinate point in the first coordinate point set and the initial coordinates of each coordinate point in the second coordinate point set, the spatial coverage between the field of view space of the optical navigation device and the moving space of the manipulator is calculated; based on the spatial coverage and the preset coverage threshold, the recommended result of the optical field of view of the optical navigation device is determined; the recommended result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the manipulator. By adopting the method in this embodiment, the robot, the moving space of the manipulator of the robot, the optical navigation device and the field of view space of the optical navigation device in the real scene can be visualized in three dimensions; compared with the traditional method of adjusting the field of view by displaying the two-dimensional image of the robot captured by the optical navigation device on the display interface, simulating the real scene through the three-dimensional space model can more intuitively show the relative position relationship between the field of view space and the moving space, so as to facilitate the user to efficiently adjust the optical field of view of the optical navigation device; improve the adjustment efficiency of the optical field of view.

[0197] In one embodiment, a schematic diagram of a process for adjusting the optical field of view is provided. Fig.15 As shown, the process of adjusting the field of view may include the following steps:

[0198] Step 1: Initialize the robot and read the configuration parameters.

[0199] The configuration parameters include the position of the robot's moving space (move_space), the coordinate transformation matrix of the registered array under the flange The field of view spatial coordinate position of the optical navigation device (which is equipped with an optical telemetry system, Optical Telemetry System, referred to as OTS), etc.

[0200] Coordinate transformation matrix of the registered array under the flange The structural diagram can be found in Figure 4 As shown. The field of view of OTS can be referred to Fig.10 and Fig.13 shown.

[0201] The position of the moving space (move_space) can be adaptively configured according to the parameters of the robot and the surgical operation area. After configuration, the position coordinates of the 8 vertices of move_space in the reference coordinate system of the robot can be obtained.

[0202] Step 2, visualize the current field of view, mainly including: 1. Registration array position, 2. Surgical cart array position, 3. OTS field of view, 4. OTS position, 5. Robot arm move_space area, 6. Patient and operating table position, etc. The schematic diagram can be referred to Fig.11 shown.

[0203] Among them, visualizing the OTS field of view may include the following steps:

[0204] 1. Read the coordinate transformation matrix from the registered array coordinate system to the robot flange coordinate system;

[0205] 2. Obtain the coordinate transformation matrix from the registered array coordinate system to the OTS reference coordinate system;

[0206] 3. According to the coordinate conversion matrix from the registered array coordinate system to the robot flange coordinate system, and the coordinate conversion matrix from the robot flange coordinate system to the robot reference coordinate system, the coordinate conversion matrix from the registered array coordinate system to the robot reference coordinate system is calculated; then, according to the coordinate conversion matrix from the registered array coordinate system to the robot reference coordinate system, and the coordinate conversion matrix from the registered array coordinate system to the OTS reference coordinate system, the coordinate conversion matrix from the OTS to the robot reference coordinate system is calculated;

[0207] 4. Based on the coordinate conversion matrix from OTS to the reference coordinate system of the robot arm, the position and field of view of the OTS in the reference coordinate system of the OTS are converted into the position and field of view of the OTS in the reference coordinate system of the robot arm; and the position and field of view of the OTS are displayed in the reference coordinate system of the robot arm.

[0208] Step 3: Calculate the coverage of the field of view space and the moving space to obtain the recommendation result.

[0209] The process of calculating the coverage of the field of view space and the moving space can be referred to above. Fig.12 The description of the relevant contents in the illustrated embodiment will not be repeated here.

[0210] By adopting the above optical field of view adjustment process and performing three-dimensional space simulation on the optical navigation device and the robotic arm, the adjustment efficiency of the optical field of view can be greatly improved.

[0211] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0212] Based on the same inventive concept, the embodiment of the present application also provides a mechanical arm registration device for implementing the mechanical arm registration method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more mechanical arm registration device embodiments provided below can refer to the limitations on the mechanical arm registration method above, and will not be repeated here.

[0213] In one embodiment, Fig.16 As shown, a mechanical arm registration device is provided, and a registration array is provided at the end of the mechanical arm; the device includes: a first determination module 1602, a first acquisition module 1604 and a registration module 1606, wherein:

[0214] The first determination module 1602 is used to determine multiple end positions of the robotic arm based on the moving space of the robotic arm; in each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold.

[0215] The first acquisition module 1604 is used to acquire, at each end position, first position information of the registration array in the reference coordinate system of the robot arm and second position information of the registration array in the reference coordinate system of the optical navigation device.

[0216] The registration module 1606 is used to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each end posture.

[0217] In one embodiment, the first determining module 1602 includes:

[0218] A first determination submodule is used to determine multiple end position coordinates of the robotic arm based on the movement space of the robotic arm;

[0219] A second determination submodule is used to determine the registration array posture based on the initial coordinate conversion relationship between the optical navigation device and the manipulator; under the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0220] The third determination submodule is used to determine multiple end positions and postures of the robot arm according to multiple end position coordinates and registered array postures.

[0221] In one embodiment, the second determining submodule includes:

[0222] A first determining unit is used to determine an X-axis direction vector corresponding to the registration array based on an initial coordinate conversion relationship between the optical navigation device and the robotic arm; an angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a preset angle threshold;

[0223] A second determining unit is used to determine a direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis;

[0224] A third determining unit is used to determine a direction vector of a second coordinate axis based on the direction vector of the X axis and the direction vector of the first coordinate axis; the second coordinate axis is the other coordinate axis of the Y axis and the Z axis;

[0225] The fourth determining unit is used to determine the registration array posture based on the X-axis direction vector, the direction vector of the first coordinate axis and the direction vector of the second coordinate axis.

[0226] In one embodiment, the third determining submodule includes:

[0227] A generating unit, for generating, for each end position coordinate, a registered array pose matrix corresponding to the end position coordinate based on the registered array pose and the end position coordinate;

[0228] The processing unit is used to rotate the registered array posture matrix corresponding to the terminal position coordinates according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis to obtain the terminal posture corresponding to the terminal position coordinates.

[0229] In one embodiment, the first acquisition module 1604 includes:

[0230] The acquisition submodule is used to obtain the target joint angle group corresponding to each end posture for each end posture;

[0231] The control submodule is used to control the end of the surgical robot to adjust to each end posture in sequence based on the target joint angle group corresponding to each end posture, and control the optical navigation device to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device under each end posture.

[0232] In one embodiment, obtaining a submodule includes:

[0233] An acquisition unit, used for inputting the end posture into a preset inverse kinematics model to obtain at least one robot arm joint angle group corresponding to the end posture;

[0234] A screening unit, used to screen out candidate robot arm joint angle groups that meet preset collision conditions from at least one robot arm joint angle group;

[0235] The determination unit is used to determine the target joint angle group corresponding to the end posture based on the candidate robot arm joint angle group.

[0236] In one of the embodiments, when there are multiple candidate robot arm joint angle groups, a determination unit is used to calculate the binary norm between each candidate robot arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture; wherein the multiple end postures are arranged in a preset order to instruct the surgical robot to adjust to each end posture in sequence according to the preset order; then, the candidate robot arm joint angle group corresponding to the minimum binary norm is used as the target joint angle group corresponding to the end posture.

[0237] In one embodiment, the device further comprises:

[0238] A second acquisition module is used to acquire an original coordinate conversion relationship between the optical navigation device and the mechanical arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the moving space of the mechanical arm;

[0239] A construction module, used to construct a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set and the second coordinate point set;

[0240] The visualization processing module is used to visualize the three-dimensional space model to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0241] In one embodiment, the initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the device further comprises:

[0242] A coordinate conversion module, for converting the initial coordinates of each coordinate point in the first coordinate point set into target coordinates in a reference coordinate system of the robot arm based on an original coordinate conversion relationship;

[0243] A calculation module, used for calculating the spatial coverage between the field of view space of the optical navigation device and the moving space of the surgical robot based on the target coordinates of each coordinate point in the first coordinate point set and the initial coordinates of each coordinate point in the second coordinate point set;

[0244] The second determination module is used to determine the recommended result of the optical field of view of the optical navigation device based on the spatial coverage and the preset coverage threshold; the recommended result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

[0245] Each module in the above-mentioned robot arm registration device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0246] In one embodiment, a robot is provided, whose internal structure diagram can be shown as follows: Fig.17 As shown. The robot includes a processor, a memory and a communication interface connected by a system bus. Among them, the processor of the robot is used to provide computing and control capabilities. The memory of the robot includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the robot is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for registering a robotic arm is implemented.

[0247] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0248] In one embodiment, a robot is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the robot arm registration method in any of the above embodiments are implemented.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the robot arm registration method in any of the above embodiments are implemented.

[0250] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the robot arm registration method in any of the above embodiments when executed by a processor.

[0251] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0252] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0253] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A robot arm registration method, characterized in that: The end of the mechanical arm is provided with a registration array; the method comprises: Based on the moving space of the robotic arm, a plurality of end positions of the robotic arm are determined; in each of the end positions, an angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; Under each of the end positions, obtaining first position information of the registration array in a reference coordinate system of the robotic arm and second position information of the registration array in a reference coordinate system of the optical navigation device; Based on the first position information and the second position information at each of the end positions, a target coordinate conversion relationship between the optical navigation device and the robotic arm is determined.

2. The method according to claim 1, characterized in that The step of determining a plurality of end positions of the robotic arm based on the moving space of the robotic arm comprises: Determining a plurality of end position coordinates of the robotic arm based on the movement space of the robotic arm; Based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, a registration array posture is determined; under the registration array posture, an angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; A plurality of end position postures of the robotic arm are determined according to the plurality of end position coordinates and the registered array postures.

3. The method according to claim 2, characterized in that The step of determining the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the mechanical arm comprises: Based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, an X-axis direction vector corresponding to the registration array is determined; the angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a preset angle threshold; Determine the direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis; Determine a direction vector of a second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis; the second coordinate axis is the other of the Y-axis and the Z-axis; The registration array posture is determined based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

4. The method according to claim 3, characterized in that Determining a plurality of end positions and postures of the robotic arm according to the plurality of end position coordinates and the registered array postures includes: For each of the terminal position coordinates, based on the registered array posture and the terminal position coordinates, generating a registered array posture matrix corresponding to the terminal position coordinate; According to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, the registered array posture matrix corresponding to the terminal position coordinates is rotated to obtain the terminal posture corresponding to the terminal position coordinates.

5. The method according to any one of claims 1 to 4, characterized in that: The step of acquiring the second position information of the registration array in the reference coordinate system of the optical navigation device under each of the terminal positions includes: For each of the terminal postures, obtaining a target joint angle group corresponding to the terminal posture; Based on the target joint angle groups corresponding to the end postures, the end of the surgical robot is controlled to adjust to the end postures in sequence, and at each end posture, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

6. The method according to claim 5, characterized in that The step of obtaining a target joint angle group corresponding to the end position posture includes: Inputting the end position into a preset inverse kinematics model to obtain at least one robot arm joint angle group corresponding to the end position; Filtering out a candidate robot arm joint angle group that meets a preset collision condition from the at least one robot arm joint angle group; Based on the candidate robot arm joint angle group, a target joint angle group corresponding to the end position is determined.

7. The method according to claim 6, characterized in that If the candidate robot arm joint angle groups include multiple ones, then determining the target joint angle group corresponding to the end position based on the candidate robot arm joint angle groups includes: Calculating the bi-norm between each of the candidate robot arm joint angle groups and the target joint angle group corresponding to the previous end posture of the end posture; the multiple end postures are arranged in a preset order to instruct the surgical robot to adjust to each of the end postures in sequence according to the preset order; The candidate robot arm joint angle group corresponding to the minimum square norm is used as the target joint angle group corresponding to the end posture.

8. The method according to claim 1, characterized in that The method further comprises: Acquire an original coordinate conversion relationship between the optical navigation device and the mechanical arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the moving space of the mechanical arm; Constructing a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set and the second coordinate point set; The three-dimensional space model is visualized to instruct a user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

9. The method according to claim 8, characterized in that The initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the method further includes: Based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the robotic arm; Calculating a spatial coverage ratio between a field of view space of the optical navigation device and a moving space of the surgical robot based on target coordinates of each coordinate point in the first coordinate point set and initial coordinates of each coordinate point in the second coordinate point set; Based on the spatial coverage and a preset coverage threshold, a recommendation result of the optical field of view of the optical navigation device is determined; the recommendation result is used to indicate whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

10. A robot arm registration device, characterized in that: The end of the mechanical arm is provided with a registration array; the device comprises: A first determination module is used to determine a plurality of end positions of the robotic arm based on the movement space of the robotic arm; in each of the end positions, an angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; A first acquisition module, used for acquiring, at each of the end positions, first position information of the registration array in a reference coordinate system of the robotic arm and second position information of the registration array in a reference coordinate system of the optical navigation device; A registration module is used to determine a target coordinate transformation relationship between the optical navigation device and the robotic arm based on the first position information and the second position information under each of the end postures.

Images