Robotic arm registration method and robotic arm registration device

Abstract

The present application relates to a method and device for registering a robotic arm. A registration array is provided at the end of the robotic arm; the method comprises: determining multiple end positions of the robotic arm based on the moving space of the robotic arm; at each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; at each end position, obtaining the first position information of the registration array in the reference coordinate system of the robotic arm, and the second position information of the registration array in the reference coordinate system of the optical navigation device; based on the first position information and the second position information at each end position, determining the target coordinate conversion relationship between the optical navigation device and the robotic arm. When registering the robotic arm, ensure that the plane where the registration array is located is nearly perpendicular to the optical axis of the optical navigation device, so that the registration array is in a measurement posture facing the optical navigation device; this can improve the position measurement accuracy of the optical navigation device and improve the accuracy of the robotic arm registration.

Description

Technical Field

[0001] The present application relates to the field of optical navigation technology, and in particular to a robotic arm registration method and a robotic arm registration device. Background Art

[0002] With the development of surgical robots, in the process of performing surgical operations assisted by surgical robots, an optical navigation system is needed to guide the surgical robots for navigation and positioning.

[0003] In order to enable the optical navigation system to guide the surgical robot for navigation and positioning, the surgical robot's robotic arm needs to be spatially registered, that is, the spatial position relationship between the robotic arm's reference coordinate system (that is, the robot's base coordinate system) and the optical navigation system's coordinate system needs to be determined.

[0004] However, the traditional method of spatial registration of the robotic arm of a surgical robot has the problem of low registration accuracy. Summary of the Invention

[0005] Based on this, it is necessary to provide a robotic arm registration method, device, robot, computer-readable storage medium and computer program product that can improve the accuracy of robotic arm registration in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for registering a robotic arm. A registration array is provided at the end of the robotic arm; the method comprises:

[0007] Determine multiple end-point poses of the robotic arm based on the movement space of the robotic arm; in each end-point pose, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0008] At each end position, obtaining first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device;

[0009] Based on the first position information and the second position information at each end position, a target coordinate conversion relationship between the optical navigation device and the robotic arm is determined.

[0010] In one embodiment, determining multiple end-point poses of the robotic arm based on the movement space of the robotic arm includes:

[0011] Determine multiple end position coordinates of the robotic arm based on the movement space of the robotic arm;

[0012] Determine the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the robotic arm; in the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold;

[0013] Determine multiple end position poses of the robot arm based on multiple end position coordinates and registered array poses.

[0014] In one embodiment, determining the registration array pose based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm includes:

[0015] Determine an X-axis direction vector corresponding to the registration array based on an initial coordinate conversion relationship between the optical navigation device and the robotic arm; the angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a preset angle threshold;

[0016] Determining a direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis;

[0017] Determine a direction vector of a second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis; the second coordinate axis is the other of the Y-axis and the Z-axis;

[0018] The registration array posture is determined based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

[0019] In one embodiment, determining multiple end position poses of a robotic arm based on multiple end position coordinates and registered array poses includes:

[0020] For each end position coordinate, based on the registered array posture and the end position coordinate, generate the registered array posture matrix corresponding to the end position coordinate;

[0021] According to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, the registered array pose matrix corresponding to the end position coordinate is rotated to obtain the end pose corresponding to the end position coordinate.

[0022] In one embodiment, obtaining second position information of the registration array in a reference coordinate system of the optical navigation device at each terminal position includes:

[0023] For each end-position, obtain the target joint angle group corresponding to the end-position;

[0024] Based on the target joint angle group corresponding to each end posture, the end of the surgical robot is controlled to adjust to each end posture in turn, and in each end posture, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0025] In one embodiment, obtaining a target joint angle group corresponding to the end pose includes:

[0026] Inputting the end-position into a preset inverse kinematics model to obtain at least one robotic arm joint angle group corresponding to the end-position;

[0027] Selecting a candidate robotic arm joint angle group that meets a preset collision condition from at least one robotic arm joint angle group;

[0028] Based on the candidate robot arm joint angle group, determine the target joint angle group corresponding to the end pose.

[0029] In one embodiment, when the candidate manipulator joint angle groups include multiple groups, determining the target joint angle group corresponding to the end position pose based on the candidate manipulator joint angle groups includes:

[0030] Calculating the binormal between each candidate robotic arm joint angle group and the target joint angle group corresponding to the previous end pose of the end pose; arranging the multiple end poses in a preset order to instruct the surgical robot to adjust to each end pose in sequence according to the preset order;

[0031] The candidate robotic arm joint angle group corresponding to the minimum square norm is used as the target joint angle group corresponding to the end pose.

[0032] In one embodiment, the method further comprises:

[0033] Acquire an original coordinate conversion relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the movement space of the robotic arm;

[0034] Constructing a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set, and the second coordinate point set;

[0035] The three-dimensional space model is visualized to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0036] In one embodiment, the initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the method further comprises:

[0037] Based on the original coordinate transformation relationship, the initial coordinates of each coordinate point in the first coordinate point set are transformed into target coordinates in the reference coordinate system of the manipulator;

[0038] Calculating a spatial coverage ratio between a field of view space of the optical navigation device and a moving space of the surgical robot based on target coordinates of each coordinate point in the first coordinate point set and initial coordinates of each coordinate point in the second coordinate point set;

[0039] Based on the spatial coverage and the preset coverage threshold, a recommendation result of the optical field of view of the optical navigation device is determined; the recommendation result is used to indicate whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

[0040] In a second aspect, the present application further provides a robotic arm registration device. A registration array is provided at the end of the robotic arm; the device comprises:

[0041] A first determination module is configured to determine, based on a movement space of the robotic arm, a plurality of end position poses of the robotic arm; in each end position pose, an angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a preset angle threshold;

[0042] A first acquisition module is used to acquire, at each end position, first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device;

[0043] The registration module is used to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information under each end position.

[0044] In a third aspect, the present application further provides a robot comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the robot arm registration method in the first aspect when executing the computer program.

[0045] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the robot arm registration method in the first aspect.

[0046] In a fifth aspect, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the robot arm registration method in the first aspect.

[0047] The above-mentioned robot arm registration method, device, robot, storage medium and computer program product determine multiple end-point postures of the robot arm based on the movement space of the robot arm; wherein, at each end-point posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; at each end-point posture, the first position information of the registration array in the reference coordinate system of the robot arm and the second position information of the registration array in the reference coordinate system of the optical navigation device are obtained; based on the first position information and the second position information at each end-point posture, the target coordinate conversion relationship between the optical navigation device and the robot arm is determined. That is, the robot arm registration method proposed in the embodiment of the present application adopts a registration array with a planar structure, which is installed at the end of the robot arm, and when the end of the robot arm is controlled to be in different spatial positions, it is ensured that the plane where the registration array is located is nearly perpendicular to the optical axis of the optical navigation device, so that the registration array is in a measurement posture facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device and reducing the measurement error of the optical navigation device; further, based on the high-precision position measurement information, a high-precision robot arm registration result can be obtained, thereby improving the accuracy of the robot arm registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A diagram illustrating an application environment of a robotic arm registration method in one embodiment;

[0049] Figure 2 Schematic diagram of a process of a robotic arm registration method in one embodiment;

[0050] Figure 3 A schematic diagram of coordinate transformation for robot arm registration in one embodiment;

[0051] Figure 4 A schematic diagram of coordinate transformation between a registration array and a flange in one embodiment;

[0052] Figure 5 is a flowchart of a method for registering a robotic arm in another embodiment;

[0053] Figure 6 is a flowchart of a method for registering a robotic arm in another embodiment;

[0054] Figure 7 is a flowchart of a method for registering a robotic arm in another embodiment;

[0055] Figure 8 is a flowchart of a method for registering a robotic arm in another embodiment;

[0056] Figure 9 is a flowchart of a method for registering a robotic arm in another embodiment;

[0057] Figure 10Schematic diagram of the structure of the viewing field space in one embodiment;

[0058] Figure 11 is a schematic structural diagram of a three-dimensional space model in one embodiment;

[0059] Figure 12 is a flowchart of a method for registering a robotic arm in another embodiment;

[0060] Figure 13 Schematic diagram of structural parameters of the field of view space in one embodiment;

[0061] Figure 14 A schematic diagram of a structure for determining whether a viewing space covers a moving space in one embodiment;

[0062] Figure 15 1. A schematic diagram of a process for visually adjusting the optical field of view in one embodiment;

[0063] Figure 16 is a structural block diagram of a robotic arm registration device in one embodiment;

[0064] Figure 17 1 is a diagram of the internal structure of a robot in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0066] In the field of surgical navigation, optical navigation systems are needed to guide surgical instruments or guide surgical robots for surgical positioning. In order for the optical navigation system to guide the surgical robot to complete the surgical positioning, the spatial registration of the robotic arm is required. This means calculating the spatial position relationship between the reference coordinate system of the robotic arm and the reference coordinate system of the optical navigation system.

[0067] However, traditional robotic arm registration methods have the problem of low registration accuracy.

[0068] In addition, there are two complex issues during the registration process. The first is to ensure that the robot's motion space is within the field of view of the optical navigation system, and the second is to ensure that the motion configuration generated by the robot does not block the optical path and that the registered configuration does not collide with the environment.

[0069] Based on the above technical problems, this application proposes a robotic arm registration method that can improve the registration accuracy of the robotic arm.

[0070] The robot arm registration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the robot 102 includes a base and a robotic arm, with a registration array 104 disposed at the end of the robotic arm. Multiple registration objects can be disposed on the registration array 104 as multiple registration points. For example, the registration array 104 can be a planar structure, and the multiple registration objects can be disposed at different positions on the plane of the registration array 104.

[0071] The robot 102 may include but is not limited to industrial robots, medical robots, etc.; such as surgical robots.

[0072] In one embodiment, Figure 2 As shown, a robot arm registration method is provided, which is applied to Figure 1 The robot in the example is used to illustrate the following steps:

[0073] Step 202: Determine multiple end position poses of the robotic arm based on the movement space of the robotic arm.

[0074] Among them, in each terminal posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold.

[0075] The movement space of the manipulator can be a range of movement space of the end of the manipulator that is pre-set according to the end operation of the manipulator; for example, the movement space can be a cubic area, and the movement space of the end of the manipulator can be determined by the coordinates of each vertex of the cube. It should be noted that the coordinates of each vertex can be coordinates in the reference coordinate system of the manipulator or in the world coordinate system; the embodiments of the present application do not impose too many restrictions on this. In addition, the reference coordinate system of the manipulator can be the robot base coordinate system, that is, the coordinate system where the robot base is located.

[0076] When the moving space of the robotic arm is determined, multiple end postures of the robotic arm can be determined within the moving space; wherein, the end posture may include the end position and the end posture; the end posture determined within the moving space can be understood as the end position of the robotic arm being located within the moving space.

[0077] Exemplarily, based on the multiple vertex coordinates of the moving space, multiple position coordinate points can be randomly determined in the moving space, or multiple uniformly distributed position coordinate points can be regularly determined as multiple end positions of the robotic arm.

[0078] In addition, for each end position, it is also necessary to determine the end posture corresponding to each end position, so as to generate multiple end postures of the robotic arm based on each end position and its end posture. Among them, the end posture of the robotic arm at each end position must be able to make the angle between the plane where the registration array at the end of the robotic arm is located and the optical axis of the optical navigation device greater than the preset angle threshold. For example: the end posture of the robotic arm must make the plane where the registration array installed at the end of the robotic arm is located perpendicular to the optical axis of the optical navigation device; that is, the registration array must be facing the optical navigation device. In this way, in the facing posture, the position measurement error of the registration array by the optical navigation device can be reduced, thereby reducing the registration error of the robotic arm and improving the registration accuracy of the robotic arm.

[0079] Step 204 : Under each end position, obtain first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device.

[0080] The reference coordinate system of the manipulator can be the robot base coordinate system, that is, the coordinate system where the robot base is located. The reference coordinate system of the optical navigation device can be the reference coordinate system of the optical navigation system, that is, the coordinate system used by the optical navigation device for position measurement.

[0081] In the case where multiple end positions of the robotic arm are determined as described above, the end of the robotic arm can be controlled to be in each end position in sequence, and in each end position, the robot can be controlled to obtain the first position information of the registration array in the reference coordinate system of the robotic arm, and the optical navigation device can be controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0082] refer to Figure 3 As shown in the figure, it shows a schematic diagram of the registration of the robot arm. For example, for a robot, when a registration array or other end components such as operating instruments are set at the end of the robot, the end components are usually installed at the end of the robot through a flange; based on this, the coordinate system corresponding to the robot can include the reference coordinate system (Link_1) of the robot arm, the flange coordinate system (FlangeFrama) of the end of the robot arm, the installation coordinate system (InstallFrame) and the registration array coordinate system (RobotArrayFrame); among them, the reference coordinate system (Link_1) of the robot arm is the motion reference of the robot arm, the flange coordinate system (FlangeFrama) is the end position of the robot arm, and the registration array is a robot arm array tool installed on the flange of the robot arm, which can be recognized by the optical navigation system.

[0083] If you want to obtain the position information of the registered array in the reference coordinate system of the robot, you need to know the coordinate transformation relationship between the registered array coordinate system and the flange coordinate system, as well as the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the robot. Figure 4 As shown in the figure, it shows the relationship between the registration array coordinate system and the flange coordinate system. It should be noted that after the registration array is installed at the end of the robotic arm, the relative position relationship between the registration array and the flange is fixed, that is, the coordinate transformation relationship between the registration array coordinate system and the flange coordinate system is also fixed.

[0084] In addition, regarding the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the manipulator, when the end of the manipulator moves to different spatial positions, the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the manipulator can be determined based on the motion information of each joint of the manipulator at that time. It should be noted that the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the manipulator will be different depending on the end position of the manipulator.

[0085] Then, according to the coordinate transformation relationship between the registered array coordinate system and the flange coordinate system, and the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the manipulator, the coordinate transformation relationship between the registered array coordinate system and the reference coordinate system of the manipulator can be determined, that is,

[0086] Then, at each end posture, the coordinate transformation relationship between the flange coordinate system and the reference coordinate system of the robot arm at the end posture can be determined first according to the end posture; then, the coordinate transformation relationship between the registered array coordinate system and the reference coordinate system of the robot arm, and the coordinate transformation relationship between the registered array coordinate system and the flange coordinate system can be determined; further, according to the coordinate transformation relationship between the registered array coordinate system and the reference coordinate system of the robot arm, the position information of each registration point on the registration array in the registration array coordinate system is subjected to coordinate transformation processing, and the first position information of the registration array in the reference coordinate system of the robot arm at the end posture can be obtained; wherein, the first position information includes the position information of each registration point on the registration array in the reference coordinate system of the robot arm at the end posture.

[0087] Continue to refer to the above Figure 3 As shown, exemplarily, for an optical navigation system, the optical navigation system is mainly composed of an optical navigation device, and a reference coordinate system of the optical navigation device, that is, a reference coordinate system of the optical navigation system, can be defined as an OpticalTrackFrame.

[0088] At each end-point position, the optical navigation device can be controlled to capture the registration array, thereby obtaining second position information of the registration array in the optical navigation device's reference coordinate system. This second position information includes the position information of each registration point on the registration array in the optical navigation device's reference coordinate system at that end-point position. The optical navigation device can then transmit the acquired second position information for each end-point position to the robot.

[0089] Step 206 : Determine a target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each end position.

[0090] For example, when the robot obtains the first position information of the registration array in the reference coordinate system of the robotic arm and the second position information in the reference coordinate system of the optical navigation device at each end posture, the robot can perform coordinate transformation based on the first position information and the second position information to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm.

[0091] In the above-mentioned robot arm registration method, multiple end position postures of the robot arm are determined based on the movement space of the robot arm; wherein, at each end position, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; at each end position, the first position information of the registration array in the reference coordinate system of the robot arm and the second position information of the registration array in the reference coordinate system of the optical navigation device are obtained; based on the first position information and the second position information at each end position, the target coordinate conversion relationship between the optical navigation device and the robot arm is determined. That is, the robot arm registration method proposed in the embodiment of the present application adopts a registration array with a planar structure, which is installed at the end of the robot arm, and when the end of the robot arm is controlled to be in different spatial positions, it is ensured that the plane where the registration array is located is nearly perpendicular to the optical axis of the optical navigation device, so that the registration array is in a measurement posture facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device and reducing the measurement error of the optical navigation device; furthermore, based on the high-precision position measurement information, a high-precision robot arm registration result can be obtained, thereby improving the accuracy of the robot arm registration.

[0092] Figure 5 This embodiment is a flow chart of a method for registering a robot arm in another embodiment. This embodiment involves an optional implementation process for determining multiple end positions of a robot arm based on the movement space of the robot arm. On the basis of the above embodiment, Figure 5 As shown, the above step 202 includes:

[0093] Step 502 : determining multiple end position coordinates of the robotic arm based on the movement space of the robotic arm.

[0094] The moving space of the robotic arm may be a spatial region surrounded by a plurality of spatial position points (ie, vertices).

[0095] For example, the robot can determine multiple end position coordinates in the moving space based on multiple vertex coordinates of the moving space; wherein each end position coordinate can be a three-dimensional space coordinate in the moving space, such as P r (x r ,y r ,z r ).

[0096] Step 504 : Determine the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the robotic arm.

[0097] Among them, in the registration array posture, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than the preset angle threshold. Since the angle range of the line-plane angle is 0 to 90 degrees, in order to ensure that the plane where the registration array is located is approximately perpendicular to the optical axis of the optical navigation device, the preset angle threshold should be an angle threshold that is less than and close to 90 degrees. For example: the angle difference between the preset angle threshold and 90 degrees can be determined based on the measurement accuracy of the optical navigation device, and the posture facing the optical navigation device can be reasonably defined based on the angle difference. Exemplarily, the angle difference can be π / 10, then the preset angle threshold can be any angle greater than or equal to 72 degrees and less than or equal to 90 degrees, that is, any angle within the angle range of [72,90].

[0098] For example, when determining the registration array posture, it can be determined based on the reference coordinate system of the optical navigation device and the registration array coordinate system; for example: Figure 3 As shown, it can be seen that for the registration array coordinate system, its X-axis is perpendicular to the plane where the registration array is located; and for the reference coordinate system of the optical navigation device, its Z-axis can be expressed as the optical axis direction of the optical navigation device; then, in the posture facing the optical navigation device, the X-axis of the registration array coordinate system should be nearly parallel to the Z-axis of the reference coordinate system of the optical navigation device.

[0099] Based on this logic, the coordinate system direction of the registration array can be determined according to the reference coordinate system of the optical navigation device, thereby obtaining the registration array posture; wherein, the registration array posture can be expressed in the form of a matrix, such as: T = [x_vec y_vec z_vec].

[0100] Step 506 : Determine multiple end position poses of the robotic arm based on the multiple end position coordinates and the registered array poses.

[0101] For example, for each end position coordinate, the end position coordinate and the registered array posture are combined to obtain the end posture corresponding to the end position coordinate; the multiple end postures are stored in the form of a matrix to obtain the registered array posture matrix T', which can be expressed as:

[0102]

[0103] In this embodiment, when determining multiple end position poses of the manipulator based on the movement space of the manipulator, the multiple end position coordinates of the manipulator are first determined based on the movement space of the manipulator; then, based on the initial coordinate conversion relationship between the optical navigation device and the manipulator, the registration array pose is determined; wherein, in the registration array pose, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold; and then, based on the multiple end position coordinates and the registration array pose, the multiple end position poses of the manipulator are determined. That is, in this embodiment, by determining the registration array pose when facing the optical navigation device, and then combining the registration array pose with different end position coordinates, the multiple end position poses of the registration array are obtained; so that when the end of the manipulator is at different end positions, it maintains the pose facing the optical navigation device; thereby improving the position measurement accuracy of the optical navigation device, and providing high-precision position measurement data for the manipulator registration, thereby improving the accuracy of the manipulator registration.

[0104] Figure 6 This embodiment is a flow chart of a method for registering a manipulator in another embodiment. This embodiment involves an optional implementation process for determining the registration array posture based on the initial coordinate transformation relationship between the optical navigation device and the manipulator. On the basis of the above embodiment, Figure 6 As shown, the above step 504 includes:

[0105] Step 602: Determine the X-axis direction vector corresponding to the registration array based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm.

[0106] The angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is smaller than a preset angle threshold.

[0107] Step 604 : Determine the direction vector of the first coordinate axis based on the preset initial vector of the first coordinate axis and the preset constraint angle of the first coordinate axis.

[0108] The first coordinate axis is any one of the Y axis and the Z axis.

[0109] Step 606: Determine the direction vector of the second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis.

[0110] The second coordinate axis is the other of the Y axis and the Z axis.

[0111] Step 608 : Determine the registration array posture based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

[0112] refer to Figure 3 As shown, when the registration array is in a position facing the optical navigation device, the X-axis of the registration array coordinate system and the Z-axis of the reference coordinate system of the optical navigation device are parallel to each other; then, based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm, the Z-axis direction vector of the reference coordinate system of the optical navigation device can be converted into the direction vector of the X-axis of the reference coordinate system of the robotic arm; that is, the direction vector of the Z-axis of the reference coordinate system of the optical navigation device in the reference coordinate system Link_1 of the robotic arm is obtained, which is recorded as x_vec.

[0113] Since the purpose of registering the robotic arm is to obtain the precise coordinate conversion relationship between the optical navigation device and the robotic arm, other methods can be used to first obtain the initial coordinate conversion relationship between the optical navigation device and the robotic arm; so as to determine the position and posture facing the optical navigation system based on the initial coordinate conversion relationship.

[0114] For example, referring to the relevant content of step 204 above, the end of the robot arm is controlled to be in a certain end posture, and the first coordinate transformation relationship between the registration array and the robot base can be obtained, that is, Next, the end of the robotic arm is kept stationary, and the optical navigation device is controlled to shoot the registration array at the end position to obtain the second coordinate transformation relationship between the registration array and the optical navigation device, that is, Then, according to the first coordinate transformation relationship and the second coordinate transformation relationship, the initial coordinate transformation relationship between the optical navigation device and the robotic arm is determined, that is,

[0115] in,

[0116]

[0117] Among them, inv() represents the inversion operation, that is, calculating the inverse matrix of a square matrix.

[0118] Based on this, we can convert the initial coordinates Get the Z axis in the reference coordinate system of the optical navigation device and the direction vector in the reference coordinate of the manipulator as the X axis of the registration array coordinate system, that is, x_vec.

[0119] Next, after the X axis of the registration array coordinate system is determined, the Y axis and the Z axis of the registration array coordinate system may be determined based on the X axis of the registration array coordinate system.

[0120] For example, the Y-axis of the registration array coordinate system can be constrained so that the angle between the Y-axis of the registration array coordinate system and the bed is (0, pi / 2). For example, the Y-axis constraint angle of the registration array coordinate system can be randomly determined within the range of 0 to pi / 2, denoted as α∈(0, pi / 2). Next, based on the Y-axis constraint angle, the Y-axis direction vector y_vec and the Z-axis direction vector z_vec of the registration array coordinate system are determined.

[0121] Exemplarily, the process of determining the Y-axis direction vector y_vec and the Z-axis direction vector z_vec may include:

[0122] a. Calculate the base vector of the Y axis; for example, the preset initial vector of the Y axis, such as the vector (0, -1, 0), can be calculated and projected onto the YZ plane of the robot registration array to obtain the base vector base_vec of the Y axis.

[0123] base_vec=(0,-1,0)·sin(arccos(x_vec·(0,-1,0)))

[0124] b. Calculate the base vector base_vec of the Y axis, and use the vector after rotating the X axis direction vector x_vec by an angle α as the Y axis direction vector y_vec.

[0125] According to the Rodgrigues rotation formula:

[0126] y_vec=base_vec·cosα+x_vec×base_vec·sinα+(1-cosα)

[0127] (x_vec base_vec) x_vec

[0128] c. Calculate the Z-axis direction vector z_vec, that is, z_vec = x_vec × y_vec.

[0129] d. Normalize x_vec, y_vec, and z_vec.

[0130] x_vec = x_vec / norm(x_vec)

[0131] y_vec = y_vec / norm(y_vec)

[0132] z_vec = z_vec / norm(z_vec)

[0133] Thus, the registered array posture T = [x_vec y_vec z_vec] is obtained.

[0134] Through the above steps, the posture of the registered array facing the optical navigation device can be determined, and then, based on the posture of the registered array, the end posture of the robotic arm facing the optical navigation device can be obtained, so that the robotic arm can maintain the posture facing the optical navigation device at different end positions, thereby improving the position detection accuracy of the optical navigation device for the registration array, and thus improving the registration accuracy of the robotic arm.

[0135] Figure 7 This embodiment is a flow chart of a method for registering a robot arm in another embodiment. This embodiment involves an optional implementation process for determining multiple end positions of a robot arm based on multiple end position coordinates and registered array postures. Based on the above embodiment, Figure 7 As shown, the above step 506 includes:

[0136] Step 702 : For each terminal position coordinate, based on the registered array posture and the terminal position coordinate, generate a registered array pose matrix corresponding to the terminal position coordinate.

[0137] Referring to step 506 above, based on the registered array posture and multiple end position coordinates, a registered array posture matrix T' facing the optical navigation device and satisfying the angle constraint can be obtained, that is:

[0138]

[0139] Step 704 : Rotate the registered array pose matrix corresponding to the terminal position coordinates according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis to obtain the terminal pose corresponding to the terminal position coordinates.

[0140] For example, after obtaining the registration array pose matrix T', the registration array pose matrix T' can be randomly rotated around the Y axis and Z axis of the registration array coordinate system within the angle range of (-pi / 10, pi / 10); thereby obtaining a pose matrix T with pitch and yaw angles within the range of (-pi / 10, pi / 10) goal .

[0141] Exemplarily, the preset rotation angle of the Y axis can be randomly determined within the angle range of (-pi / 10, pi / 10), which is recorded as the random Euler angle random_y_theta of the Y axis; and the preset rotation angle of the Z axis can be randomly determined within the angle range of (-pi / 10, pi / 10), which is recorded as the random Euler angle random_z_theta of the Z axis.

[0142] Next, based on the random Euler angles random_y_theta and random Euler angles random_z_theta, the registered array pose matrix T' is rotated to obtain the rotated pose matrix Tgoal .Right now

[0143] T goal =T·Euler_matrix(0,random_y_theta,rndom_z_theta)

[0144] At this point, the rotated pose matrix T goal As the end pose matrix of the robotic arm.

[0145] In this embodiment, for each end position coordinate, a registration array pose matrix corresponding to the end position coordinate is first generated based on the registration array posture and the end position coordinate; then, the registration array pose matrix corresponding to the end position coordinate is rotated according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, and finally the end pose matrix of the manipulator is obtained. By adopting this method, when the end of the manipulator is in different spatial positions, it is not only possible to ensure that the position measurement error is met in the X-axis direction facing the optical navigation device, but also that the position measurement error of the optical navigation device is met in both the pitch and yaw directions; thereby obtaining the end pose of the manipulator that meets the position measurement error of the optical navigation device in three-dimensional space and faces the optical navigation device; so that under this end pose, the optical navigation device can achieve high-precision position detection of the registration array at the end of the manipulator, thereby improving the registration accuracy of the manipulator.

[0146] In one embodiment, in response to the many problems faced by traditional robotic arm registration methods, during the robotic arm registration process, it is necessary not only to improve the position detection accuracy of the registration array installed at the end of the robotic arm by the optical navigation device; in addition, when the robotic arm is in the end position, the robotic arm configuration at this time should also meet the requirements of not blocking the light path of the optical navigation device, otherwise the optical navigation device will not be able to recognize the registration array at the end of the robotic arm, nor will it be able to detect the position information of the registration array in the reference coordinate system of the optical navigation device under the end position. In addition, when controlling the movement of the robotic arm end to the end position, the robotic arm configuration should not have any environmental collision, otherwise the robotic arm end may be outside the moving space of the robotic arm, which may cause the optical navigation device to be unable to recognize the robotic arm end, and thus be unable to obtain the position information of the registration array in the reference coordinate system of the optical navigation device.

[0147] Figure 8 This embodiment is a flow chart of a method for registering a manipulator in another embodiment. This embodiment involves an optional implementation process of obtaining the second position information of the registration array in the reference coordinate system of the optical navigation device at each end position. Based on the above embodiment, Figure 8 As shown, the above step 204 includes:

[0148] Step 802: For each end-point posture, obtain the target joint angle group corresponding to the end-point posture.

[0149] The target joint angle group corresponding to the end-point posture includes the joint angles of each joint of the manipulator, and the target joint angle group satisfies preset collision conditions. Exemplarily, the preset collision conditions include, but are not limited to, collisions with the manipulator body, collisions with other environmental components, and collisions with the surface of the mobile space. If the robot is a surgical robot, other environmental components may include a surgical cart, a bed, and other surgical-related equipment. Furthermore, collisions between the manipulator and the mobile space may include collisions between the manipulator body and / or the end of the manipulator and at least one surface of the mobile space.

[0150] Exemplarily, when calculating the target joint angle group of the robotic arm based on the end posture, the inverse kinematics principle can be used to input the end posture into a preset inverse kinematics model to obtain at least one robotic arm joint angle group corresponding to the end posture; if there is only one robotic arm joint angle group at this time, it is determined whether the robotic arm joint angle group meets the preset collision condition; if so, the robotic arm joint angle group is used as the target joint angle group corresponding to the end posture; if not, the preset constraint angle and / or preset rotation angle can be updated for the robotic arm end position coordinates corresponding to the end posture, and the above-mentioned steps of determining the registered array posture and end posture are re-executed to obtain a new end posture corresponding to the end position coordinates; and then, based on the new end posture, the corresponding robotic arm joint angle group is re-determined, and then the target joint angle group corresponding to the new end posture is determined.

[0151] It should be noted that after the inverse kinematics calculation, if there is no robot arm joint angle group corresponding to the end posture, then the new end posture can be determined by updating the preset constraint angle and / or preset rotation angle, and then the target joint angle group corresponding to the new end posture can be re-determined.

[0152] Exemplarily, if there are multiple robot arm joint angle groups corresponding to the end posture, a candidate robot arm joint angle group that meets the preset collision conditions can be screened out from the multiple robot arm joint angle groups; and based on the candidate robot arm joint angle group, the target joint angle group corresponding to the end posture is determined.

[0153] For example, if there is one candidate robotic arm joint angle group that satisfies the preset collision condition, then the candidate robotic arm joint angle group that satisfies the preset collision condition can be directly used as the target joint angle group corresponding to the end-point posture. For example, if after collision screening, no candidate robotic arm joint angle group that satisfies the preset collision condition exists, then the process can return to the above-mentioned step of determining the end-point posture, determine a new end-point posture by updating the preset constraint angles and / or the preset rotation angles, and then re-determine the target joint angle group corresponding to the new end-point posture.

[0154] Exemplarily, if there are multiple candidate robotic arm joint angle groups that meet the preset collision conditions, any one candidate robotic arm joint angle group can be determined from the multiple candidate robotic arm joint angle groups as the target joint angle group corresponding to the end posture; or the target joint angle group can be determined from the multiple candidate robotic arm joint angle groups according to a preset determination rule.

[0155] Exemplarily, multiple end postures of the robotic arm can be arranged in a preset order to instruct the robot to adjust to each end posture in sequence according to the preset order; based on this, when there are multiple candidate robotic arm joint angle groups corresponding to the current end posture, the target joint angle group corresponding to the previous end posture adjacent to the current end posture can be obtained, and the binary norm between each candidate robotic arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture can be calculated; then, the candidate robotic arm joint angle group corresponding to the minimum binary norm is used as the target joint angle group corresponding to the current end posture.

[0156] It should be noted that when the current end posture is the first end posture among multiple end postures, the candidate robotic arm joint angle groups corresponding to the first end posture can be compared with the initialized joint angle group of the robotic arm, so as to select the one with the smallest two-norm as the target joint angle group corresponding to the first end posture.

[0157] Assume that the candidate robot arm joint angle groups corresponding to the current end pose are defined as j1, j2, j3, ..., j n , the target joint angle group corresponding to the previous end pose adjacent to the current end pose or the initialization joint angle group of the manipulator is defined as j cur , then the two norms can be expressed as diff k =‖j k -j cur ‖ 2 , where k = 1, ..., n. Then, diff k The minimum value of j kThe target joint angle group corresponding to the current end posture is added to the joint angle pool of the robotic arm; wherein the joint angle pool of the robotic arm is used to store the target joint angle groups corresponding to multiple end postures of the robotic arm in sequence.

[0158] For example, when calculating the target joint angle groups corresponding to each end-point pose, the number of target joint angle groups in the joint angle pool can be determined to be equal to a preset number. If so, the loop is exited to complete the calculation of the solution pool. If not, the loop continues to calculate the target joint angle group for the next end-point pose. The preset number of target joint angle groups in the joint angle pool can be less than or equal to the number of end-point poses. That is, based on the above-mentioned method for determining the target joint angle groups, the target joint angle group corresponding to each end-point pose can be determined in a sequential and cyclical manner.

[0159] By calculating the binary norm between the candidate robotic arm joint angle group corresponding to the current end position and the target joint angle group corresponding to the previous end position, and selecting the candidate robotic arm joint angle group corresponding to the minimum binary norm as the target joint angle group corresponding to the current end position; the degree of movement of the robotic arm end can be minimized when it moves from the previous end position to the current end position; it is not only conducive to the efficient movement of the robotic arm, but also can improve the movement efficiency of the robotic arm, thereby improving the registration efficiency of the robotic arm.

[0160] Step 804, based on the target joint angle group corresponding to each end posture, control the end of the surgical robot to adjust to each end posture in sequence, and at each end posture, control the optical navigation device to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

[0161] In this embodiment, for each end-position, the target joint angle group corresponding to the end-position that satisfies the preset collision conditions is first obtained based on the preset collision conditions. Then, based on the target joint angle group corresponding to each end-position, the end of the surgical robot is sequentially controlled to adjust to each end-position, and at each end-position, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device. That is, using the method in this embodiment, when performing robotic arm registration, it can be ensured that when the robotic arm end moves within the mobile space, its robotic arm configuration does not block the optical path of the optical navigation device, allowing the optical navigation device to accurately detect the position of the registration array mounted on the robotic arm end. Furthermore, the robotic arm configuration will not cause collisions with the body, the mobile space, or the surrounding environment, thereby resolving any problems caused by collisions and improving the efficiency and accuracy of robotic arm registration.

[0162] In one embodiment, during the process of registering the robotic arm, traditional methods also face a problem, namely, the need to ensure that the robotic arm's motion space is within the field of view of the optical navigation system, so that the optical navigation system will not be unable to recognize the registration array at the end of the robotic arm, resulting in poor robotic arm registration accuracy or even failure to complete robotic arm registration. Based on this, before registering the robotic arm, the optical field of view of the optical navigation device can be adjusted so that the optical field of view of the optical navigation device can completely cover the movement space of the robotic arm; so that when the robotic arm end moves within the movement space, the optical navigation device can always track and identify the robotic arm end, thereby navigating and positioning the operating components installed at the end.

[0163] Figure 9 This embodiment relates to an optional implementation process of dynamically visually adjusting the optical field of view of an optical navigation device. Based on the above embodiment, Figure 9 As shown, the above method also includes:

[0164] Step 902 : Acquire the original coordinate conversion relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the movement space of the robotic arm.

[0165] Among them, the original coordinate conversion relationship between the optical navigation device and the robotic arm can be the coordinate conversion relationship between the two determined after the optical navigation device and the robotic arm are initialized, based on the position state of the initialized optical navigation device and the position state of the robotic arm.

[0166] For example, after initialization, the robot can calculate the coordinate transformation relationship between the registration array at the end of the manipulator and the robot base through the forward kinematics model based on the current joint information of the manipulator, that is, At the same time, the optical navigation device is controlled to shoot the registration array to obtain the coordinate conversion relationship between the registration array and the optical navigation device, that is, Then, based on the coordinate transformation relationship between the registration array and the robot base And the coordinate transformation relationship between the registration array and the optical navigation device Calculate the original coordinate transformation relationship between the optical navigation device and the robotic arm The specific calculation method can refer to the above Figure 6 The description of the relevant contents in the illustrated embodiment will not be repeated here.

[0167] In addition, the first coordinate point set corresponding to the field of view space of the optical navigation device may be a spatial point set determined based on the reference coordinate system of the optical navigation device and used to characterize the field of view space of the optical navigation device. The initial coordinates of each coordinate point in the first coordinate point set may be the coordinates in the reference coordinate system of the optical navigation device. Figure 10 As shown, 8 points can be used to represent the field of view space of the optical navigation device, namely ots_pt1, ots_pt2, ots_pt3, ots_pt4, ots_pt5, ots_pt6, ots_pt7 and ots_pt8.

[0168] Furthermore, the second coordinate point set corresponding to the robot's movement space may be a set of spatial points determined based on the robot's reference coordinate system to represent the robot's movement space. The initial coordinates of each coordinate point in the second coordinate point set may be the coordinates in the robot's reference coordinate system. For example, the robot's movement space may be represented by eight points.

[0169] Step 904 : constructing a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set, and the second coordinate point set.

[0170] Exemplarily, based on the original coordinate transformation relationship, the initial coordinates of each coordinate point in the first coordinate point set can be converted into target coordinates in the reference coordinate system of the robotic arm; then, the field of view space of the optical navigation device can be constructed based on the target coordinates of each coordinate point in the first coordinate point set, and the moving space of the robotic arm can be constructed based on the initial coordinates of each coordinate point in the second coordinate point set; and then, a three-dimensional space model can be constructed based on the field of view space of the optical navigation device and the moving space of the surgical robot.

[0171] Exemplarily, the three-dimensional space model can also include a three-dimensional model of the robot and a three-dimensional model of the optical navigation device; the three-dimensional space model takes the reference coordinate system of the manipulator as the reference coordinate, and fills the three-dimensional model of the robot, the movement space of the manipulator, the three-dimensional model of the optical navigation device, and the field of view space of the optical navigation device in the three-dimensional space model; thereby achieving a three-dimensional simulation of the real scene and building a three-dimensional space model corresponding to the real scene, with reference to Figure 11 shown.

[0172] For example, in a surgical scenario, in addition to a surgical robot and an optical navigation device, a surgical cart system is also required; Figure 3 When registering the robotic arm, the coordinate conversion relationship between the surgical cart system and the optical navigation device, as well as the coordinate conversion relationship between the surgical cart system and the robotic arm can also be registered.

[0173] The surgical trolley system may include a trolley body and a trolley body array (also referred to as a trolley array), and the coordinate system corresponding to the surgical trolley system may include a trolley body coordinate system (TrolleyFrame) and a trolley array coordinate system (TrolleyArrayFrame).

[0174] For example, during the robotic arm registration process, the optical navigation device can simultaneously identify the position information of the registration array at the end of the robotic arm and the position information of the trolley array on the surgical trolley system, thereby obtaining the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device, and the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device. Furthermore, based on the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device, and the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the reference coordinate system (OpticalTrackFrame) of the optical navigation device, the coordinate transformation relationship between the registration array coordinate system (RobotArrayFrame) and the trolley array coordinate system (TrolleyArrayFrame) can be determined.

[0175] Based on this, the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the robotic arm's reference coordinate system (Link_1) can be determined based on the coordinate transformation relationship between the registered array coordinate system (RobotArrayFrame) and the trolley array coordinate system (TrolleyArrayFrame), as well as the coordinate transformation relationship between the registered array coordinate system (RobotArrayFrame) and the robotic arm's reference coordinate system (Link_1). Furthermore, based on the coordinate transformation relationship between the trolley array coordinate system (TrolleyArrayFrame) and the robotic arm's reference coordinate system (Link_1), the 3D model of the surgical cart system can be added to the aforementioned 3D spatial model.

[0176] Of course, using the above method, the operating table, the surgical object, and other equipment in the surgical environment can also be added to the three-dimensional space model, thereby obtaining a more complete three-dimensional space model corresponding to the surgical scene.

[0177] Step 906 : Visualize the three-dimensional space model to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0178] For example, after obtaining the three-dimensional space model, the three-dimensional space model can be visualized; including but not limited to adding color, logo, mark, transparency and other attribute information to each device, component, field of view space, mobile space, etc., to improve the display effect of the three-dimensional model.

[0179] The 3D spatial model is then visually displayed, allowing the user to adjust the optical field of view of the optical navigation device based on the visual 3D spatial model. This 3D spatial model allows the user to intuitively determine whether the optical field of view of the optical navigation device covers the robot's movement space and to adaptively adjust the position of the optical navigation device.

[0180] In this embodiment, before registering the robotic arm, the original coordinate transformation relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the optical navigation device's field of view, and a second coordinate point set corresponding to the robotic arm's movement space are obtained; a three-dimensional space model is constructed based on the original coordinate transformation relationship, the first coordinate point set, and the second coordinate point set; and the three-dimensional space model is visualized to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model. This allows for visual and dynamic adjustment of the optical field of view of the optical navigation device before registering the robotic arm, ensuring that the optical field of view of the optical navigation device fully covers the robotic arm's movement space during robotic arm registration, thereby improving the accuracy of robotic arm registration.

[0181] Figure 12 This embodiment is a flow chart of a method for registering a robot arm in another embodiment. This embodiment involves an optional implementation process of automatically determining whether the optical field of view of the optical navigation device can cover the moving space of the robot arm. Based on the above embodiment, Figure 12 As shown, the above method also includes:

[0182] Step 1202 : Based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the robot arm.

[0183] Step 1204 : Calculate the spatial coverage between the field of view of the optical navigation device and the moving space of the robotic arm based on the target coordinates of each coordinate point in the first coordinate point set and the initial coordinates of each coordinate point in the second coordinate point set.

[0184] Step 1206: Determine a recommendation result of the optical field of view of the optical navigation device based on the spatial coverage and a preset coverage threshold.

[0185] The recommendation result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the robotic arm.

[0186] That is to say, in this embodiment, first, the coordinates of each coordinate point in the first coordinate point set and the coordinates of each coordinate point in the second coordinate point set are unified into coordinates under one coordinate system; then, under the coordinate system, the coverage ratio between the field of view space and the moving space is calculated to obtain the spatial coverage ratio of the field of view space covering the moving space; then, a preset coverage ratio threshold is used to determine whether the spatial coverage ratio meets the preset spatial coverage requirement; for example: when the spatial coverage ratio is greater than or equal to the preset coverage ratio threshold, the recommended result of the optical field of view of the optical navigation device can be determined as recommended, which indicates that the optical field of view at this time can cover the moving space of the robotic arm.

[0187] Exemplarily, the spatial coverage may be determined by the quotient of the volume of the overlapping area between the moving space and the field of view space and the volume of the moving space.

[0188] In another implementation, the robot may also convert the initial coordinates of each coordinate point in the second coordinate point set into target coordinates in the reference coordinate system of the optical navigation device based on the original coordinate conversion relationship.

[0189] Right now

[0190]

[0191] Next, based on the initial coordinates of each coordinate point in the first coordinate point set and the target coordinates of each coordinate point in the second coordinate point set, a recommendation result of the optical field of view of the optical navigation device is determined.

[0192] Assume that the parameters of the field of view space of the optical navigation device are as follows Figure 13 As shown. For example, reference Figure 14 When the initial coordinates of each coordinate point in the second coordinate point set are converted into the target coordinates in the reference coordinate system of the optical navigation device, any coordinate point in the moving space of the manipulator is taken, such as pt(x p ,y p ,z p ); Take the Z-axis coordinate of the point, Z=z p , intercept the xy cross section; if the point is in the field of view space, the corresponding (x p ,y p ) will fall within the expanded rectangle (such as Figure 14 The following mathematical expression is valid.

[0193] |x p |<=224+x_board and |y p |<=240+y_board established

[0194] Where x_board=(|zp |-950)*k x ,y_board=(|z p |-950)*k y ;k x and k y is the proportional coefficient; such as k x =0.2979, k y =0.3745.

[0195] Exemplarily, referring to the above data expression, if all eight vertices of the moving space are within the field of view space of the optical navigation device, it can be determined that the recommendation result of the optical field of view of the optical navigation device is a recommendation.

[0196] In this embodiment, based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the manipulator; based on the target coordinates of each coordinate point in the first coordinate point set and the initial coordinates of each coordinate point in the second coordinate point set, the spatial coverage between the field of view space of the optical navigation device and the moving space of the manipulator is calculated; based on the spatial coverage and a preset coverage threshold, a recommendation result of the optical field of view of the optical navigation device is determined; the recommendation result is used to characterize whether the optical field of view of the optical navigation device covers the moving space of the manipulator. Using the method in this embodiment, a three-dimensional visualization of a robot, the moving space of the robot's manipulator, the optical navigation device, and the field of view space of the optical navigation device in a real scene can be achieved; compared to the traditional method of adjusting the field of view by displaying a two-dimensional image of the robot captured by the optical navigation device on a display interface, simulating the real scene through a three-dimensional spatial model can more intuitively show the relative positional relationship between the field of view space and the moving space, thereby facilitating the user to efficiently adjust the optical field of view of the optical navigation device, thereby improving the adjustment efficiency of the optical field of view.

[0197] In one embodiment, a flow chart of optical field adjustment is provided. Figure 15 As shown, the process of adjusting the field of view may include the following steps:

[0198] Step 1: Initialize the robot and read the configuration parameters.

[0199] Among them, the configuration parameters include the position of the robot's moving space (move_space), the coordinate transformation matrix of the registration array under the flange The field of view spatial coordinate position of the optical navigation device (which is equipped with an optical telemetry system, Optical Telemetry System, referred to as OTS), etc.

[0200] Coordinate transformation matrix of the registered array under the flange The structural diagram can be referred to Figure 4 As shown. The field of view of OTS can be referred to Figure 10 and Figure 13 shown.

[0201] The position of the moving space (move_space) can be adaptively configured according to the parameters of the robotic arm and the surgical operation area. After configuration, the position coordinates of the 8 vertices of move_space in the reference coordinate system of the robotic arm can be obtained.

[0202] Step 2: Visualize the current field of view, which mainly includes: 1. Registration array position, 2. Surgical cart array position, 3. OTS field of view, 4. OTS position, 5. Robotic arm move_space area, 6. Patient and operating table position, etc. The schematic diagram can be referred to Figure 11 shown.

[0203] The visualization of the OTS field of view may include the following steps:

[0204] 1. Read the coordinate transformation matrix from the registered array coordinate system to the robot flange coordinate system;

[0205] 2. Obtain the coordinate transformation matrix from the registered array coordinate system to the OTS reference coordinate system;

[0206] 3. Based on the coordinate conversion matrix from the registered array coordinate system to the robot flange coordinate system and the coordinate conversion matrix from the robot flange coordinate system to the robot reference coordinate system, the coordinate conversion matrix from the registered array coordinate system to the robot reference coordinate system is calculated; then, based on the coordinate conversion matrix from the registered array coordinate system to the robot reference coordinate system and the coordinate conversion matrix from the registered array coordinate system to the OTS reference coordinate system, the coordinate conversion matrix from the OTS to the robot reference coordinate system is calculated;

[0207] 4. Based on the coordinate conversion matrix from the OTS to the reference coordinate system of the robotic arm, the position and field of view of the OTS in the reference coordinate system of the OTS are converted to the position and field of view of the OTS in the reference coordinate system of the robotic arm; and the position and field of view of the OTS are displayed in the reference coordinate system of the robotic arm.

[0208] Step 3: Calculate the coverage of the field of view space and the moving space to obtain the recommendation result.

[0209] The process of calculating the coverage of the field of view space and the moving space can be referred to above. Figure 12 The description of the relevant contents in the illustrated embodiment will not be repeated here.

[0210] Using the above optical field of view adjustment process to perform three-dimensional spatial simulation of the optical navigation equipment and the robotic arm can greatly improve the adjustment efficiency of the optical field of view.

[0211] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0212] Based on the same inventive concept, embodiments of the present application also provide a robotic arm registration device for implementing the aforementioned robotic arm registration method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations in one or more robotic arm registration device embodiments provided below can be found in the limitations of the robotic arm registration method above and will not be further elaborated here.

[0213] In one embodiment, Figure 16 As shown, a robotic arm registration device is provided, wherein a registration array is provided at the end of the robotic arm; the device comprises: a first determination module 1602, a first acquisition module 1604 and a registration module 1606, wherein:

[0214] The first determination module 1602 is used to determine multiple end position poses of the robotic arm based on the movement space of the robotic arm; in each end position pose, the angle between the plane where the registration array is located and the optical axis of the optical navigation device is greater than a preset angle threshold.

[0215] The first acquisition module 1604 is used to acquire, at each end position, first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device.

[0216] The registration module 1606 is used to determine the target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each end position.

[0217] In one embodiment, the first determining module 1602 includes:

[0218] A first determining submodule is used to determine a plurality of end position coordinates of the robotic arm based on the movement space of the robotic arm;

[0219] A second determination submodule is configured to determine a registration array posture based on an initial coordinate conversion relationship between the optical navigation device and the robotic arm; in the registration array posture, an angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a preset angle threshold;

[0220] The third determination submodule is used to determine multiple end position poses of the robotic arm according to the multiple end position coordinates and the registered array posture.

[0221] In one embodiment, the second determining submodule includes:

[0222] a first determining unit, configured to determine an X-axis direction vector corresponding to the registration array based on an initial coordinate conversion relationship between the optical navigation device and the robotic arm; wherein an angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a preset angle threshold;

[0223] a second determining unit, configured to determine a direction vector of the first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis being any one of the Y axis and the Z axis;

[0224] a third determining unit, configured to determine a direction vector of a second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis; the second coordinate axis being the other of the Y-axis and the Z-axis;

[0225] The fourth determining unit is configured to determine the registration array posture based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

[0226] In one embodiment, the third determining submodule includes:

[0227] A generating unit, configured to generate, for each end position coordinate, a registered array pose matrix corresponding to the end position coordinate based on the registered array pose and the end position coordinate;

[0228] The processing unit is used to rotate the registered array pose matrix corresponding to the terminal position coordinates according to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis to obtain the terminal pose corresponding to the terminal position coordinates.

[0229] In one embodiment, the first acquisition module 1604 includes:

[0230] The acquisition submodule is used to obtain the target joint angle group corresponding to each end pose;

[0231] The control submodule is used to control the end of the surgical robot to adjust to each end posture in sequence based on the target joint angle group corresponding to each end posture, and control the optical navigation device to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device under each end posture.

[0232] In one embodiment, obtaining the submodule includes:

[0233] An acquisition unit, configured to input the end-position into a preset inverse kinematics model to obtain at least one robotic arm joint angle group corresponding to the end-position;

[0234] a screening unit, configured to screen out a candidate robotic arm joint angle group that satisfies a preset collision condition from at least one robotic arm joint angle group;

[0235] The determination unit is used to determine the target joint angle group corresponding to the end position based on the candidate robot arm joint angle group.

[0236] In one embodiment, when there are multiple candidate robotic arm joint angle groups, a determination unit is used to calculate the two-norm between each candidate robotic arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture; wherein the multiple end postures are arranged in a preset order to instruct the surgical robot to adjust to each end posture in sequence according to the preset order; then, the candidate robotic arm joint angle group corresponding to the minimum two-norm is used as the target joint angle group corresponding to the end posture.

[0237] In one embodiment, the apparatus further comprises:

[0238] A second acquisition module is used to acquire an original coordinate conversion relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the movement space of the robotic arm;

[0239] A construction module, configured to construct a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set, and the second coordinate point set;

[0240] The visualization processing module is used to visualize the three-dimensional space model to instruct the user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

[0241] In one embodiment, the initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the apparatus further comprises:

[0242] A coordinate conversion module, configured to convert the initial coordinates of each coordinate point in the first coordinate point set into target coordinates in a reference coordinate system of the robotic arm based on an original coordinate conversion relationship;

[0243] a calculation module, configured to calculate a spatial coverage ratio between a field of view space of the optical navigation device and a moving space of the surgical robot based on target coordinates of each coordinate point in the first coordinate point set and initial coordinates of each coordinate point in the second coordinate point set;

[0244] The second determination module is used to determine the recommendation result of the optical field of view of the optical navigation device based on the spatial coverage and the preset coverage threshold; the recommendation result is used to indicate whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

[0245] Each module in the aforementioned robotic arm registration device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0246] In one embodiment, a robot is provided, whose internal structure diagram can be shown as follows: Figure 17 As shown. The robot includes a processor, a memory and a communication interface connected via a system bus. The processor of the robot is used to provide computing and control capabilities. The memory of the robot includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the robot is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for registering a robotic arm is implemented.

[0247] Those skilled in the art will understand that Figure 16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0248] In one embodiment, a robot is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps of the robot arm registration method in any of the above embodiments are implemented.

[0249] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the robot arm registration method in any of the above embodiments are implemented.

[0250] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the robot arm registration method in any of the above embodiments when executed by a processor.

[0251] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0252] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0253] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A robotic arm registration method, characterized in that: The end of the robotic arm is provided with a registration array; the method comprises: Determining a plurality of end position poses of the robotic arm based on a movement space of the robotic arm; in each of the end position poses, a first angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a first preset angle threshold; Under each of the end positions, obtaining first position information of the registration array in the reference coordinate system of the manipulator and second position information of the registration array in the reference coordinate system of the optical navigation device; Determining a target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each of the end position postures; The determining of a plurality of end position poses of the robotic arm based on the movement space of the robotic arm comprises: Determining a plurality of end position coordinates of the robotic arm based on the movement space of the robotic arm; Determining a registration array posture based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm; in the registration array posture, a first angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a first preset angle threshold; Determining a plurality of end position poses of the robotic arm according to the plurality of end position coordinates and the registered array pose; The determining of the registration array posture based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm includes: Determining an X-axis direction vector corresponding to the registration array based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm; wherein a second angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a second preset angle threshold, and the X-axis direction vector is perpendicular to the plane of the registration array; Determining a direction vector of a first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis; Determine a direction vector of a second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis; the second coordinate axis is the other of the Y-axis and the Z-axis; The registration array posture is determined based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

2. The method according to claim 1, characterized in that The step of determining a plurality of end position coordinates of the robotic arm and the registered array postures includes: For each of the terminal position coordinates, generating a registration array pose matrix corresponding to the terminal position coordinate based on the registration array pose and the terminal position coordinate; According to the preset rotation angle corresponding to the first coordinate axis and the preset rotation angle corresponding to the second coordinate axis, the registered array pose matrix corresponding to the end position coordinate is rotated to obtain the end pose corresponding to the end position coordinate.

3. The method according to claim 1 or 2, characterized in that The acquiring, at each of the terminal positions, second position information of the registration array in a reference coordinate system of the optical navigation device comprises: For each of the end-point postures, obtaining a target joint angle group corresponding to the end-point posture; Based on the target joint angle group corresponding to each of the end postures, the end of the surgical robot to which the robotic arm belongs is controlled to adjust to each of the end postures in turn, and at each of the end postures, the optical navigation device is controlled to obtain the second position information of the registration array in the reference coordinate system of the optical navigation device.

4. The method according to claim 3, characterized in that The obtaining of the target joint angle group corresponding to the end position pose includes: Inputting the end position into a preset inverse kinematics model to obtain at least one robotic arm joint angle group corresponding to the end position; Filtering a candidate robotic arm joint angle group that meets a preset collision condition from the at least one robotic arm joint angle group; Based on the candidate robot arm joint angle group, a target joint angle group corresponding to the end position is determined.

5. The method according to claim 4, characterized in that If the candidate manipulator joint angle groups include multiple ones, then determining the target joint angle group corresponding to the end position based on the candidate manipulator joint angle groups includes: Calculating the binormal between each candidate robotic arm joint angle group and the target joint angle group corresponding to the previous end posture of the end posture; arranging the multiple end postures in a preset order to instruct the surgical robot to adjust to each end posture in sequence according to the preset order; The candidate manipulator joint angle group corresponding to the minimum square norm is used as the target joint angle group corresponding to the end position.

6. The method according to claim 1, characterized in that The method further comprises: Acquire an original coordinate conversion relationship between the optical navigation device and the robotic arm, a first coordinate point set corresponding to the field of view space of the optical navigation device, and a second coordinate point set corresponding to the movement space of the robotic arm; constructing a three-dimensional space model based on the original coordinate transformation relationship, the first coordinate point set, and the second coordinate point set; The three-dimensional space model is visualized to instruct a user to adjust the optical field of view of the optical navigation device based on the three-dimensional space model.

7. The method according to claim 6, characterized in that The initial coordinates of each coordinate point in the first coordinate point set are coordinates in a reference coordinate system of the optical navigation device, and the initial coordinates of each coordinate point in the second coordinate point set are coordinates in a reference coordinate system of the robotic arm; the method further includes: Based on the original coordinate conversion relationship, the initial coordinates of each coordinate point in the first coordinate point set are converted into target coordinates in the reference coordinate system of the robotic arm; Calculating a spatial coverage ratio between a field of view of the optical navigation device and a moving space of the surgical robot to which the robotic arm belongs based on target coordinates of each coordinate point in the first coordinate point set and initial coordinates of each coordinate point in the second coordinate point set; Based on the spatial coverage and a preset coverage threshold, a recommendation result of the optical field of view of the optical navigation device is determined; the recommendation result is used to indicate whether the optical field of view of the optical navigation device covers the moving space of the surgical robot.

8. A robotic arm registration device, characterized in that: The end of the robotic arm is provided with a registration array; the device comprises: a first determining module, configured to determine, based on a movement space of the robotic arm, a plurality of end position poses of the robotic arm; wherein, in each of the end position poses, a first angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a first preset angle threshold; a first acquisition module, configured to acquire, at each of the end postures, first position information of the registration array in a reference coordinate system of the manipulator and second position information of the registration array in a reference coordinate system of the optical navigation device; a registration module, configured to determine a target coordinate conversion relationship between the optical navigation device and the robotic arm based on the first position information and the second position information at each of the end postures; The determining of a plurality of end position poses of the robotic arm based on the movement space of the robotic arm comprises: Determining a plurality of end position coordinates of the robotic arm based on the movement space of the robotic arm; Determining a registration array posture based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm; in the registration array posture, a first angle between a plane where the registration array is located and an optical axis of the optical navigation device is greater than a first preset angle threshold; Determining a plurality of end position poses of the robotic arm according to the plurality of end position coordinates and the registered array pose; The determining of the registration array posture based on the initial coordinate conversion relationship between the optical navigation device and the robotic arm includes: Determining an X-axis direction vector corresponding to the registration array based on an initial coordinate transformation relationship between the optical navigation device and the robotic arm; wherein a second angle between the X-axis direction vector corresponding to the registration array and the optical axis of the optical navigation device is less than a second preset angle threshold, and the X-axis direction vector is perpendicular to the plane of the registration array; Determining a direction vector of a first coordinate axis based on a preset initial vector of the first coordinate axis and a preset constraint angle of the first coordinate axis; the first coordinate axis is any one of the Y axis and the Z axis; Determine a direction vector of a second coordinate axis based on the X-axis direction vector and the direction vector of the first coordinate axis; the second coordinate axis is the other of the Y-axis and the Z-axis; The registration array posture is determined based on the X-axis direction vector, the direction vector of the first coordinate axis, and the direction vector of the second coordinate axis.

Images