Mechanical arm control method and device, surgical robot and computer equipment

Abstract

The invention relates to a mechanical arm control method and device, a surgical robot and computer equipment. The method comprises the steps that external traction force acting on the mechanical arm and tail end pose information of the mechanical arm are obtained; the tail end pose information comprises tail end position information and tail end pose information; under the condition that it is determined that the mechanical arm is located in a preset position constraint area according to the tail end position information, the mechanical arm is controlled to move to the target position according to the external traction force; and under the condition that it is determined that the mechanical arm is located in the preset posture constraint area according to the tail end posture information, the mechanical arm is controlled to move to the target posture according to the external traction force. According to the method, a position constraint area and a posture constraint area are arranged, and external traction force is combined, so that the position and posture of the mechanical arm are adjusted in the motion constraint area; according to the mechanical arm, collision between the mechanical arm and the surrounding environment can be avoided, movement limiting and guiding can be provided, the control complexity of the mechanical arm is simplified, and the movement stability of the mechanical arm is improved.

Description

Technical Field

[0001] The present application relates to the technical field of robots, and particularly to a robotic arm control method, apparatus, surgical robot, and computer device. Background Art

[0002] With the rapid development of the cross - application of robot technology and medical science, the application of surgical robots has received extensive attention. In orthopedic surgery, the precise positioning of surgical robots can avoid the fatigue of surgeons during long - term surgeries, which is of great significance for improving the accuracy, stability, and safety of surgeries.

[0003] In an orthopedic surgical robot, the needle insertion point and target point of a surgical instrument are determined through preoperative planning, and the end of the robotic arm is controlled to move to the corresponding target pose through robotic arm motion planning.

[0004] During the process of planning the motion path of the robotic arm, it is necessary to consider the collision problem between the robotic arm and the surgical area and automatically perform obstacle avoidance processing, which makes the above - mentioned robotic arm control method extremely complex. Summary of the Invention

[0005] Based on this, in view of the above - mentioned technical problems, it is necessary to provide a robotic arm control method, apparatus, surgical robot, computer device, computer - readable storage medium, and computer program product that can simplify the control method of the robotic arm, improve the motion stability of the robotic arm, and perform comprehensive obstacle avoidance during the motion of the robotic arm.

[0006] In a first aspect, the present application provides a robotic arm control method, including:

[0007] Obtaining an external traction force acting on the robotic arm and obtaining the end pose information of the robotic arm; the end pose information includes end position information and end attitude information;

[0008] When it is determined according to the end position information that the robotic arm is within a preset position constraint region, controlling the robotic arm to move to a target position according to the external traction force;

[0009] When it is determined according to the end attitude information that the robotic arm is within a preset attitude constraint region, controlling the robotic arm to move to a target attitude according to the external traction force.

[0010] In one embodiment, the method further includes:

[0011] Determining a first central axis of the position constraint region according to the target position;

[0012] Determining a position constraint curve according to the target position and preset boundary points;

[0013] Rotate around the first central axis with the position constraint curve as the generatrix for one full rotation to obtain the position constraint region.

[0014] In one embodiment, the method further includes:

[0015] Determine the second central axis of the pose constraint region according to the target position and the target pose;

[0016] Determine the pose constraint curve according to the target position and the preset offset angle;

[0017] Rotate around the second central axis with the pose constraint curve as the generatrix for one full rotation to obtain the pose constraint region.

[0018] In one embodiment, an end effector is installed at the end of the robotic arm. The end effector includes a first end and a second end. Obtaining the end pose information of the robotic arm includes:

[0019] Obtain the position information of the first end of the end effector, and determine the end position information of the robotic arm according to the position information of the first end;

[0020] Obtain the position information of the second end of the end effector, and determine the end pose information of the robotic arm according to the position information of the first end and the position information of the second end.

[0021] In one embodiment, when it is determined according to the end position information that the end of the robotic arm is at the edge of the position constraint region or the edge of the pose constraint region, the method further includes:

[0022] Determine the direction of the first component force of the external traction force in the normal direction of the edge;

[0023] If the direction of the first component force points to the inside of the position constraint region or the inside of the pose constraint region, control the robotic arm to move along the direction of the external traction force;

[0024] If the direction of the first component force points to the outside of the position constraint region or the outside of the pose constraint region, control the robotic arm to move along the direction of the second component force of the external traction force on the tangent plane of the edge.

[0025] In one embodiment, when it is determined according to the end position information that the robotic arm is within the position constraint region, controlling the robotic arm to move to the target position according to the external traction force includes:

[0026] When it is determined according to the end position information that the end of the robotic arm is within the position constraint area but not within the proximity area of the target position, the movement of the end of the robotic arm is controlled to the proximity area of the target position according to the external traction force; the proximity area of the target position is within the position constraint area, and the distance between any position within the proximity area of the target position and the target position is less than or equal to the first preset distance threshold;

[0027] When it is determined according to the end position information that the end of the robotic arm is within the proximity area of the target position, the movement of the end of the robotic arm is controlled to the target position.

[0028] In one embodiment, controlling the movement of the end of the robotic arm to the proximity area of the target position according to the external traction force includes:

[0029] Determining the movement speed of the robotic arm according to the external traction force and the first mapping relationship, and determining the joint angle set of the robotic arm according to the movement speed of the robotic arm and the first Jacobian relationship;

[0030] Controlling the movement of the end of the robotic arm to the proximity area of the target position according to the joint angle set of the robotic arm.

[0031] In one embodiment, when it is determined according to the end pose information that the robotic arm is within the pose constraint area, controlling the movement of the robotic arm to the target pose according to the external traction force includes:

[0032] When it is determined according to the end pose information that the end of the robotic arm is within the pose constraint area but not within the proximity area of the target pose, the movement of the end of the robotic arm is controlled to the proximity area of the target pose according to the external traction force; the proximity area of the target pose is within the pose constraint area, and the distance between any position within the proximity area of the target pose and the target pose is less than or equal to the second preset distance threshold;

[0033] When it is determined according to the end pose information that the robotic arm is within the proximity area of the target pose, the movement of the robotic arm is controlled to the target pose.

[0034] In one embodiment, controlling the movement of the robotic arm to the proximity area of the target pose according to the external traction force includes:

[0035] Determining the movement angular velocity of the robotic arm according to the external traction force and the second mapping relationship, and determining the joint angle set of the robotic arm according to the movement angular velocity of the robotic arm and the second Jacobian relationship;

[0036] Controlling the movement of the robotic arm to the proximity area of the target pose according to the joint angle set of the robotic arm.

[0037] In one embodiment, the method further includes:

[0038] Obtain the entry point position information and the target point position information of the operation object;

[0039] Determine the target position and the target pose according to the entry point position information and the target point position information.

[0040] In a second aspect, the present application further provides a robotic arm control device, including:

[0041] A first acquisition module, configured to acquire the external traction force acting on the robotic arm and acquire the end pose information of the robotic arm; the end pose information includes end position information and end pose information;

[0042] A first control module, configured to control the robotic arm to move to the target position according to the external traction force when it is determined according to the end position information that the robotic arm is within a preset position constraint area;

[0043] A second control module, configured to control the robotic arm to move to the target pose according to the external traction force when it is determined according to the end pose information that the robotic arm is within a preset pose constraint area.

[0044] In a third aspect, the present application further provides a surgical robot, including the robotic arm control device in the second aspect and a robotic arm, and the robotic arm is configured to be controlled by the robotic arm control device and perform movements.

[0045] In a fourth aspect, the present application further provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the robotic arm control method in the first aspect are implemented.

[0046] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the robotic arm control method in the first aspect are implemented.

[0047] In a sixth aspect, the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the robotic arm control method in the first aspect are implemented.

[0048] The above robotic arm control method, device, surgical robot, computer device, storage medium, and computer program product obtain the external traction force acting on the robotic arm and the end pose information of the robotic arm; where the end pose information includes end position information and end attitude information; then, when it is determined according to the end position information that the robotic arm is within the preset position constraint area, the robotic arm is controlled to move to the target position according to the external traction force; and when it is determined according to the end attitude information that the robotic arm is within the preset attitude constraint area, the robotic arm is controlled to move to the target attitude. That is, the robotic arm control method proposed in the embodiments of the present application sets the position constraint area and attitude constraint area during the movement of the robotic arm, and combines the external traction force, so that the robotic arm can adjust its position and attitude within the movement constraint area; by setting the movement constraint area, not only can the collision between the robotic arm and the surrounding environment be avoided, but also movement restrictions and guidance can be provided, so that the robotic arm can move to the target pose quickly and accurately; compared with the traditional automatic motion planning method based on collision detection, the control method of the present application can not only simplify the control complexity of the robotic arm movement, improve the movement stability of the robotic arm, but also improve the control accuracy and control efficiency of the robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0050] Figure 1 It is an application environment diagram of the robotic arm control method in an embodiment;

[0051] Figure 2 It is a flowchart of the robotic arm control method in an embodiment;

[0052] Figure 3 It is a flowchart of the robotic arm control method in another embodiment;

[0053] Figure 4 It is a schematic diagram of the formation of the position constraint area in an embodiment;

[0054] Figure 5 It is a flowchart of the robotic arm control method in another embodiment;

[0055] Figure 6 It is a flowchart of the robotic arm control method in another embodiment;

[0056] Figure 7Schematic flowchart of the robotic arm control method in another embodiment;

[0057] Figure 8 Schematic flowchart of the robotic arm control method in another embodiment;

[0058] Figure 9 Schematic flowchart of the robotic arm control method in another embodiment;

[0059] Figure 10 Schematic flowchart of the robotic arm control method in another embodiment;

[0060] Figure 11 Schematic flowchart of the robotic arm control method in another embodiment;

[0061] Figure 12 Schematic structural diagram of the spinal surgery robot system in one embodiment;

[0062] Figure 13(a) is a schematic diagram of the position constraint curve in one embodiment;

[0063] Figure 13(b) is a schematic diagram of the attitude constraint curve in one embodiment;

[0064] Figure 13(c) is a schematic diagram of the target pose of the robotic arm in one embodiment;

[0065] Figure 14 Flowchart of the motion control of the spinal surgery robot in one embodiment;

[0066] Figure 15 Block diagram of the structure of the robotic arm control device in one embodiment;

[0067] Figure 16 Internal structure diagram of a computer device in one embodiment. Detailed implementation manners

[0068] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] With the rapid development of the cross-application of robot technology and medical science, the application of surgical robots has received extensive attention. In orthopedic surgeries, the precise positioning of surgical robots can avoid the fatigue of surgeons due to long-term surgeries, which is of great significance for improving the precision, stability and safety of surgeries.

[0070] In an orthopedic navigation surgical robot, the precise execution and navigation positioning of the robotic arm are one of its core technologies. Before the operation, the doctor can plan the needle insertion point and target point of the surgical instrument on the preoperative image, and through the robotic arm motion planning technology, the surgical instrument can reach the target surgical pose accurately and safely, which can effectively save the operation time and reduce the radiation impact on the doctor and the patient during the operation.

[0071] In related technologies, when the robot performs motion planning based on the needle insertion point and the target point, it usually considers the collision detection between the robotic arm and the target surgical area, including collision monitoring based on force sensors and collision detection based on cameras, etc.; traditionally, when considering collision detection for motion planning, in complex scenarios of the surgical area, it will lead to a relatively high complexity of collision detection, and further result in a relatively high complexity of the robotic arm control; in addition, due to the influence of the complex environment, the perception ability of the sensing device for the complex environment is limited, which will also have a certain impact on the accuracy of motion planning, increasing the possibility of collision between the robotic arm and the surgical area; furthermore, in traditional collision detection schemes, only the collision between the robotic arm and the surgical area is considered, and the collision between the robotic arm and the patient and surrounding devices is not considered. Therefore, there is still a collision risk of the robotic arm in traditional collision detection schemes, resulting in relatively low motion stability and reliability of the robotic arm, and the introduction of the environmental perception device will also increase the complexity and cost of the entire robotic system.

[0072] Based on this, in the embodiments of the present application, a control method for a robotic arm is proposed. Through the way of human-computer interaction, the motion planning of the robotic arm navigation positioning is realized, which can reduce the dependence on environmental perception, and can realize a safe and adaptive motion planning strategy, improving the safety and stability of the robotic system; in addition, through the constraint control of position and posture, the curve problem of the interaction and learning of the robotic system can be solved, improving the usability and accuracy of the robotic system.

[0073] The robotic arm control method provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the surgical robot 102 includes a base and a robotic arm, and different operating instruments can be clamped at the end of the robotic arm to perform different operations. Exemplarily, the surgical robot can be any type of auxiliary surgical robot in the medical field; for example, the surgical robot can be a spinal surgical robot.

[0074] In an exemplary embodiment, as Figure 2 shown, a robotic arm control method is provided. Taking the method applied to the surgical robot in Figure 1 as an example, it includes the following steps 202 to 206.

[0075] Among them:

[0076] Step 202, obtain the external traction force acting on the robotic arm and obtain the end pose information of the robotic arm; the end pose information includes end position information and end attitude information.

[0077] Exemplarily, when the surgical robot is used in an assisted operation scenario, it is usually necessary for the end of the surgical robot to move to the target pose. Among them, the target pose may include the target position and the target attitude at the target position. In this example, a method for controlling a robotic arm with human-robot coexistence is provided. By dragging the robotic arm by a user or a doctor (hereinafter referred to as the user), the end of the robotic arm is moved to the target pose, so that the robotic arm performs corresponding assisted operations at the target pose.

[0078] During the movement of the robotic arm, the movement of each joint on the robotic arm can be controlled according to the external traction force applied by the user to the robotic arm, so that the end pose of the robotic arm changes under the action of the external traction force. Exemplarily, a force sensor can be set on the robotic arm to detect the external traction force acting on the robotic arm; among them, the detected external traction force can include the magnitude and direction of the force. In a three-dimensional space, the external traction force can be a six-dimensional force including the magnitude and direction of the force.

[0079] As an example, a force sensor can be set at the end of the robotic arm. The user drags the end of the robotic arm to drive the movement of each joint on the robotic arm, thereby changing the pose of the end of the robotic arm.

[0080] Since the movement of the robotic arm is controlled in a human-robot coexistence manner in this example, during the movement of the robotic arm, collisions between the robotic arm and any object in the surrounding environment can be avoided in all directions. Compared with the related solutions for robotic arm motion planning based on collision detection in the traditional sense, during the movement of the robotic arm, there is no need to perform collision detection. Correspondingly, there is also no need to set up an environment perception setting to assist the surgical robot in performing collision detection, which can reduce the complexity of robotic arm control and the cost of robotic arm movement.

[0081] In addition, in this example, in order to facilitate the user to efficiently move the end of the robotic arm to the target pose, a motion constraint area of the robotic arm is set. That is to say, when the end of the robotic arm moves within this motion constraint area, it can move to the target pose quickly and accurately; in this way, during the movement of the end of the robotic arm, it is necessary to detect the end pose of the robotic arm in real time to ensure that the end of the robotic arm can remain within the motion constraint area for movement.

[0082] Among them, the motion constraint region may include a position constraint region and an attitude constraint region. The position constraint region may include the motion region of the end of the robotic arm from the initial position to the target position. The attitude constraint region may include the motion region of the joints of the end of the robotic arm when the end of the robotic arm is at the target position.

[0083] Exemplarily, the end pose information of the robotic arm can be obtained through a sensor, or the end pose information of the robotic arm can be determined according to the angles of the joints of the robotic arm through forward kinematics, etc. In the embodiments of the present application, the method for obtaining the end pose information of the robotic arm is not specifically limited.

[0084] Step 204, when it is determined that the robotic arm is within the preset position constraint region according to the end position information, control the robotic arm to move to the target position according to the external traction force.

[0085] Exemplarily, before the robotic arm moves, the position constraint region can be determined according to the target position of the robotic arm and the preset position constraint conditions; it should be noted that when the target position of the end of the robotic arm is different, the corresponding position constraint regions can be different; of course, they can also be the same. Based on this, when the preset position constraint region corresponding to the target position is obtained, during the movement of the robotic arm, the robotic arm can judge whether the end of the robotic arm is within the preset position constraint region according to the end position information of the robotic arm.

[0086] If the end of the robotic arm is within the preset position constraint region, it indicates that the robotic arm can move within the preset position constraint region. At this time, the robotic arm can be controlled to move towards the target position according to the detected external traction force acting on the robotic arm.

[0087] In addition, it should be noted that for the end position information of the above-mentioned robotic arm, it can be the end position information of the robotic arm before the current external traction force acts; it can also be the end position information of the robotic arm after the current external traction force acts.

[0088] As an example, in the case where the end position information of the robotic arm is the end position information of the robotic arm before the current external traction force acts, there is a situation where, before the current external traction force acts, the end position information of the robotic arm indicates that the end of the robotic arm is within the preset position constraint area, but after the current external traction force acts, the end of the robotic arm moves outside the preset position constraint area; for this situation, when applying an external traction force to the robotic arm next time, since the end of the robotic arm is not within the preset position constraint area, in order to enable the end of the robotic arm to move to the target position, it is possible to control the robotic arm to move into the preset position area when the external traction force applied next time meets certain constraint conditions; or, after the current external traction force acts, if it is detected that the end of the robotic arm moves outside the preset position constraint area, the robotic arm is automatically controlled to move into the preset position constraint area to facilitate the normal movement of the robotic arm within the preset position constraint area.

[0089] As another example, in the case where the end position information of the robotic arm is the end position information of the robotic arm after the current external traction force acts, since the end position information of the robotic arm at this time is the predicted end position information of the robotic arm based on the current external traction force, therefore, in the case where it is determined that the predicted end position information of the robotic arm may be outside the preset position constraint area, it is possible to prohibit controlling the movement of the robotic arm according to the current external traction force, that is, the situation where the end of the robotic arm moves outside the preset position constraint area can be avoided.

[0090] Step 206, when it is determined that the robotic arm is within the preset pose constraint area according to the end pose information, control the robotic arm to move to the target pose according to the external traction force.

[0091] Exemplarily, in the case of controlling the end of the robotic arm to move to the target position based on the end position information of the robotic arm and the external traction force, at this time, the pose of the end joint of the robotic arm can be adjusted so that the end joint of the robotic arm moves to the target pose; thus, the end of the robotic arm achieves the effect of moving to the target position and pose.

[0092] Exemplarily, for the pose of the end joint of the robotic arm, a pose constraint area can be set; for example: before performing the pose adjustment, the pose constraint area can be determined according to the target pose of the robotic arm and the preset pose constraint conditions; so that when the user adjusts the pose of the end joint of the robotic arm, the range of its pose adjustment can be restricted. When the surgical robot obtains the external traction force acting on the end joint of the robotic arm, it can determine whether the end joint of the robotic arm is within the preset position and pose constraint area according to the end pose information of the end joint of the robotic arm; if the end joint is within the preset position and pose constraint area, the robotic arm can be controlled to move according to the external traction force.

[0093] It should be noted that since the end position of the robotic arm remains unchanged at this time and only the end pose of the robotic arm is adjusted, the external traction force can be a three-dimensional force for adjusting the end pose of the robotic arm.

[0094] In addition, when adjusting the end pose of the robotic arm, a situation similar to the above position is also faced, that is, the obtained end pose information of the robotic arm can be the end pose information of the robotic arm before the action of the current external traction force; it can also be the end pose information of the robotic arm after the action of the current external traction force; through the above relevant analysis of the position, in the case where the end pose information is the pose information before the action of the external traction force, it may be faced with the situation that the end pose of the robotic arm is outside the preset pose constraint area, and at this time, an optimization scheme for this situation needs to be set; while in the case where the end pose information is the predicted pose information after the action of the external traction force, it can avoid the situation that the pose of the end of the robotic arm is outside the preset pose constraint area after the end of the robotic arm moves; for the end pose control methods in these two cases, the relevant content descriptions of the above position control can be referred to and will not be repeated here.

[0095] In the above robotic arm control method, by obtaining the external traction force acting on the robotic arm and the end pose information of the robotic arm; wherein, the end pose information includes end position information and end pose information; then, when it is determined according to the end position information that the robotic arm is within the preset position constraint area, the robotic arm is controlled to move to the target position according to the external traction force; and when it is determined according to the end pose information that the robotic arm is within the preset pose constraint area, the robotic arm is controlled to move to the target pose according to the external traction force. That is, the robotic arm control method proposed in the embodiments of the present application, by setting the position constraint area and the pose constraint area during the movement of the robotic arm and combining the external traction force, enables the robotic arm to adjust its position and pose within the movement constraint area; by setting the movement constraint area, not only can the collision between the robotic arm and the surrounding environment be avoided, but also movement limitation and guidance can be provided, enabling the robotic arm to move to the target pose quickly and accurately; compared with the traditional automatic motion planning method based on collision detection, adopting the control method of the present application can not only simplify the control complexity of the robotic arm movement, improve the movement stability of the robotic arm, but also improve the control accuracy and control efficiency of the robotic arm.

[0096] In an exemplary embodiment, as Figure 3 shown, the obtaining method of the position constraint area in the above step 204 may include:

[0097] Step 302, determining the first central axis of the position constraint area according to the target position.

[0098] Exemplarily, an axis passing through the target position and in the vertical direction can be determined as the first central axis of the position constraint region; alternatively, an axis passing through the target position and parallel to the target posture can be determined as the first central axis of the position constraint region; or, an axis passing through the target position and having a certain angle with the target posture can also be determined as the first central axis of the position constraint region, etc.; in this example, the determination method of the first central axis of the position constraint region is not specifically limited.

[0099] Step 304: Determine a position constraint curve according to the target position and the preset boundary points.

[0100] Among them, the target position is the position where the end of the robotic arm needs to move to. For example, in spinal surgery, the target position can be the position of the needle insertion point; the position of the needle insertion point can be the position on the patient's skin surface. Exemplarily, the target position can also be a position at a certain distance above the patient's skin surface and opposite to the needle insertion point, and the target position should be on the extension line of the target point and the needle insertion point; in this way, when the end of the robotic arm is in the target posture, by pushing the surgical needle held by the end to move, the surgical needle can be inserted from the position of the needle insertion point to the position of the target point.

[0101] In addition, the preset boundary points can be collision-free boundary points determined by recognizing the surrounding environment, or boundary points input by the user, or boundary points determined by the robotic arm through autonomous movement, or boundary points determined based on preset constraint conditions and the target position, etc.; among them, the preset constraint conditions can include the distances in the horizontal and vertical directions from the target position. For example, set the horizontal distance between the boundary point and the target position to be greater than or equal to the first preset distance threshold, and set the vertical distance between the boundary point and the target position to be greater than or equal to the second preset distance threshold. Based on this preset constraint condition, when the target position is different, the preset boundary points corresponding to the target position can be determined.

[0102] Next, in the case where the target position and the preset boundary points are determined, a position constraint curve can be generated according to the target position and the preset boundary points. Exemplarily, the position constraint curve can be a straight line passing through the target position and the preset boundary points, or a curve passing through the target position and the preset boundary points.

[0103] As an example, as Figure 4 shown, the position constraint curve can be a monotonic parabola with the target position P e1 and the preset boundary point A as endpoints, and the parabola can be a parabola convex upward. Exemplarily, a vertical plane passing through the target position P e1 and the preset boundary point A can be determined, and then, in this vertical plane, a parabola with the target position P e1A parabola t that bulges upward with the starting point as the starting point and the preset boundary point A as the ending point s , as the position constraint curve. Exemplarily, the horizontal distance l between the target position P e1 and the preset boundary point A can be greater than or equal to 200 mm, and the vertical distance h between the target position P e1 and the preset boundary point A can be greater than or equal to 300 mm.

[0104] Step 306: Rotate the position constraint curve around the first central axis for one week to obtain the position constraint region.

[0105] Exemplarily, continue to refer to Figure 4 shown. The first central axis can be an axis passing through the target position and in the vertical direction, denoted as O; with the position constraint curve, i.e., the parabola t s as the generatrix and the first central axis, i.e., the axis O, as the rotation axis for one week of rotation, a horn-shaped solid of revolution can be formed, and the upper region enclosed by the solid of revolution can be used as the position constraint region.

[0106] In this embodiment, when obtaining the position constraint region, first determine the first central axis of the position constraint region according to the target position; and determine the position constraint curve according to the target position and the preset boundary point; then, rotate the position constraint curve around the first central axis for one week to obtain the position constraint region. By using this method, a position constraint region corresponding to the target position can be adaptively generated according to different target positions, and this position constraint region can not only avoid collisions with objects around the target position, but also provide restrictions and guidance for the movement of the robotic arm, thereby improving the movement stability of the robotic arm and the movement efficiency of the robotic arm.

[0107] In an exemplary embodiment, as shown in Figure 5 , the acquisition method of the attitude constraint region in the above step 206 may include:

[0108] Step 502: Determine the second central axis of the attitude constraint region according to the target position and the target attitude.

[0109] Exemplarily, an axis passing through the target position and parallel to the target attitude can be determined as the second central axis of the attitude constraint region; or an axis passing through the target position and having a certain angle with the target attitude can be determined as the second central axis of the attitude constraint region. That is to say, the target attitude is not necessarily the central axis of the attitude constraint region.

[0110] Step 504: Determine the attitude constraint curve according to the target position and the preset offset angle.

[0111] Among them, the preset offset angle can be used to represent the offset angle from the target pose, or can be used to represent the offset angle from the second central axis. In addition, it should be noted that when the second central axis is not parallel to the target pose, the offset angle between the second central axis and the target pose should be less than the preset offset angle.

[0112] Exemplarily, the target position can be used as the starting point of the pose constraint curve. Based on the target pose, a ray with an included angle of the preset offset angle from the target pose is determined as the pose constraint curve. In another example, the target position can also be used as the starting point of the pose constraint curve. Based on the second central axis, a ray with an included angle of the preset offset angle from the second central axis is determined as the pose constraint curve.

[0113] Step 506: Rotate one week with the pose constraint curve as the generatrix and the second central axis as the rotation axis to obtain the pose constraint region.

[0114] Exemplarily, the pose constraint region can be an inverted conical region enclosed by an inverted cone. That is, when adjusting the target pose of the robotic arm, the target position of the robotic arm is kept unchanged, and the pose of the end joint of the robotic arm is adjusted within this inverted conical region.

[0115] In this embodiment, when obtaining the pose constraint region, the second central axis of the pose constraint region can be determined according to the target position and the target pose, and the pose constraint curve can be determined according to the target position and the preset offset angle; then, rotate one week with the pose constraint curve as the generatrix and the second central axis as the rotation axis to obtain the pose constraint region. By using this method, a corresponding pose constraint region can be generated based on the target position and the target pose, thereby restricting and guiding the user to adjust the end pose of the robotic arm within this pose constraint region, which can improve the motion stability of the robotic arm and improve the motion efficiency of the robotic arm.

[0116] In an exemplary embodiment, an end effector is installed at the end of the robotic arm. The end effector can include a first end and a second end; Exemplarily, the pose of the end effector can be the same as or different from the pose of the end joint of the robotic arm. That is to say, the end effector can be installed at the end of the robotic arm at a fixed angle through the end flange of the robotic arm. Exemplarily, for the end pose of the above-mentioned robotic arm, it can be expressed as the pose of the end effector installed at the end of the robotic arm. Based on this, as Figure 6 shown, "obtaining the end pose information of the robotic arm" in the above step 202 can include:

[0117] Step 602: Obtain the position information of the first end of the end effector, and determine the end position information of the robotic arm according to the position information of the first end.

[0118] Exemplarily, when the end effector is mounted at the end of the robotic arm at a certain inclination angle, the first end of the end effector can be the end of the end effector closer to the operation object, that is, the end that first contacts the operation object when performing a surgical operation on the operation object.

[0119] Exemplarily, when the end pose of the robotic arm is represented as the pose of the end effector mounted at the end of the robotic arm, the position where the first end of the end effector is located can be used as the end position of the robotic arm; based on this, when obtaining the end position information of the robotic arm, the position information of the first end of the end effector can be obtained, and the end position information of the robotic arm can be determined according to the position information of the first end.

[0120] Exemplarily, when the end effector is fixedly mounted on the end flange of the robotic arm, the coordinates of the first end of the end effector in the flange coordinate system can be obtained, and the coordinates of the flange in the robotic arm coordinate system can be obtained according to the forward kinematics of the robotic arm; then, according to the coordinates of the first end of the end effector in the flange coordinate system and the coordinates of the flange in the robotic arm coordinate system, the coordinates of the first end of the end effector in the robotic arm coordinate system can be determined, so as to obtain the end position information of the robotic arm.

[0121] Step 604, obtain the position information of the second end of the end effector, and determine the end pose information of the robotic arm according to the position information of the first end and the position information of the second end.

[0122] Wherein, the second end of the end effector can be the end far from the operation object, that is, the end that contacts the operation object later, or the end that does not contact the operation object when performing a surgical operation on the operation object.

[0123] Exemplarily, the position information of the second end of the end effector can be the coordinates of the second end of the end effector in the robotic arm coordinate system; based on this, referring to the above method for obtaining the coordinates of the first end of the end effector in the robotic arm coordinate system, the coordinates of the second end of the end effector in the robotic arm coordinate system can be obtained. Then, according to the coordinates of the first end of the end effector in the robotic arm coordinate system and the coordinates of the second end of the end effector, the end pose information of the robotic arm can be determined.

[0124] In this embodiment, by mounting an end effector at the end of the robotic arm and using the pose of the end effector in the robotic arm coordinate system as the end pose of the robotic arm, the end position information of the robotic arm is determined according to the position information of the first end of the end effector; and the end pose information of the robotic arm is determined according to the position information of the first end and the position information of the second end of the end effector. By adopting this method, the end pose of the robotic arm can be accurately obtained, so as to realize the efficient and precise control of the robotic arm.

[0125] In an exemplary embodiment, based on the above embodiment, during the process of performing position constraint control and attitude constraint control on the end pose of the robotic arm, as Figure 7 shown, when it is determined according to the end position information that the end of the robotic arm is located at the edge of the position constraint region or the edge of the attitude constraint region, the above method may further include:

[0126] Step 702, determining the direction of the first component force of the external traction force in the normal direction of the edge.

[0127] Exemplarily, the end position information of the robotic arm can be used to determine whether the end of the robotic arm is located at the edge of the position constraint region or the edge of the attitude constraint region; when the end of the robotic arm is located at the edge of the position constraint region or the edge of the attitude constraint region, it is necessary to determine whether to perform motion control on the robotic arm according to the external traction force at this time to ensure that the end of the robotic arm is always within the position constraint region or the attitude constraint region.

[0128] For the case where the end of the robotic arm is at the edge of the constraint region, the external traction force at this time can be decomposed to obtain the direction of the first component force in the normal vector direction of the current edge position and the direction of the second component force in the tangent plane of the current edge position.

[0129] It should be noted that since the edge of the constraint region is a curved surface structure, at a certain position point, the normal vector at this position point is a vector perpendicular to the tangent plane at this position point.

[0130] Step 704, if the direction of the first component force points to the inside of the position constraint region or the inside of the attitude constraint region, control the robotic arm to move along the direction of the external traction force.

[0131] That is to say, when the direction of the first component force points to the inside of the position constraint region or the inside of the attitude constraint region, it means that the user is dragging the robotic arm to move towards the inside of the constraint region at this time. In this case, the robotic arm can be controlled to move according to the actual external traction force.

[0132] Step 706, if the direction of the first component force points to the outside of the position constraint region or the outside of the attitude constraint region, control the robotic arm to move along the direction of the second component force of the external traction force on the tangent plane of the edge.

[0133] That is to say, when the first component force direction points outside the position constraint area or outside the attitude constraint area, it indicates that the user is dragging the robotic arm to move towards the outside of the constraint area at this time. For this situation, it is necessary to restrict the user's dragging action and can guide the user to slide along the edge of the constraint area; that is, the robotic arm can be controlled to move along the second component force direction of the external traction force on the tangent plane at the edge; to avoid the situation that the user drags the robotic arm to move outside the constraint area.

[0134] Exemplarily, in the case where the external traction force completely coincides with the first component force direction, that is, at this time the user is moving vertically outwards at a certain position point. For this situation, when restricting the user from dragging the robotic arm, since there is no second component force direction for this external traction force, therefore, the robotic arm can be controlled to move in any direction of the tangent plane, or the robotic arm can also be controlled to move along a preset direction of the tangent plane. For example: this preset direction can be a direction coplanar with the central axis of the constraint area.

[0135] In this embodiment, when performing position constraint and attitude constraint control on the robotic arm, for the case where the end of the robotic arm is located at the edge of the position constraint area or the edge of the attitude constraint area, the first component force direction of the external traction force in the normal direction at the edge can be determined, and when the first component force direction points inside the position constraint area or inside the attitude constraint area, the robotic arm can be controlled to move along the direction of the external traction force; while when the first component force direction points outside the position constraint area or outside the attitude constraint area, the robotic arm is controlled to move along the second component force direction of the external traction force on the tangent plane at the edge. Thus, the movement constraint of the robotic arm is realized, and the movement reliability and stability of the robotic arm are improved.

[0136] In an exemplary embodiment, as Figure 8 shown, step 204 above may include:

[0137] Step 802, when it is determined according to the end position information that the end of the robotic arm is located within the position constraint area but not within the proximity area of the target position, control the end of the robotic arm to move to the proximity area of the target position according to the external traction force.

[0138] Wherein, the proximity area of the target position is located within the position constraint area, and the distance between any position within the proximity area of the target position and the target position is less than or equal to the first preset distance threshold. For example: as Figure 4 shown, the shaded area therein can be used as the proximity area of the target position.

[0139] It should be noted that the boundary of the proximity area of the target position can be a plane, as Figure 4As shown; it can also be a curved surface, such as: a curved surface bulging upward, or a curved surface sunken downward, etc.; in the embodiments of the present application, the boundary shape of the proximity area of the target position is not specifically limited.

[0140] Step 804, when it is determined according to the end position information that the end of the robotic arm is located in the proximity area of the target position, control the end of the robotic arm to move to the target position.

[0141] That is to say, during the process of the user dragging the robotic arm to move towards the target position, at the initial movement, since the robotic arm is far from the target position, at this time, the user can drag the robotic arm to move to ensure the efficient and stable movement of the robotic arm, and during the movement of the robotic arm, operations such as active obstacle avoidance are performed; and when the robotic arm moves to the proximity area of the target position, at this time, the end of the robotic arm is close to the target position. In this case, in order to ensure that the end of the robotic arm can accurately move to the target position, an automatic movement control method of the robotic arm can be adopted; and during the movement at this stage, since the movement amplitude of the end of the robotic arm is small, the movement amplitudes of the joints of the robotic arm will also be relatively small. Therefore, no collision problem will occur during the movement process.

[0142] It should be noted that when performing position movement, the end attitude of the robotic arm can be kept unchanged, and only the end position of the robotic arm is adjusted; that is, control the end of the robotic arm to perform translational movement. Among them, the end attitude of the robotic arm remaining unchanged can be expressed as the relative position between the end instrument and the flange remaining unchanged.

[0143] In this embodiment, the position movement of the robotic arm is divided into two stages. First, the user actively operates the robotic arm to move within the position constraint area until the end of the robotic arm is dragged to the proximity area of the target position; then, within the proximity area, the robotic arm performs automatic movement adjustment so that the end of the robotic arm can accurately move to the target position; by adopting the control method in this example, in the first movement stage, through position constraint, the user can quickly and stably drag the robotic arm to move, and in the first movement stage, active obstacle avoidance can be performed based on the user's active operation to solve the collision problem during the movement process of the robotic arm; in addition, in the second movement stage, through the automatic movement control of the robotic arm, the end of the robotic arm can accurately move to the target position; and in the automatic control stage, since the movement amplitude of the robotic arm is small, while improving the movement control accuracy, no collision problem will occur during the movement process. That is, by adopting the method proposed in this embodiment, the robotic arm can be quickly controlled to move to the target position on the basis of active obstacle avoidance, which not only simplifies the movement control complexity of the robotic arm, but also improves the movement stability and movement efficiency of the robotic arm.

[0144] In an exemplary embodiment, as Figure 9 shown, the above step 802 may include:

[0145] Step 902: Determine the motion speed of the robotic arm according to the external traction force and the first mapping relationship, and determine the set of joint angles of the robotic arm according to the motion speed of the robotic arm and the first Jacobian relationship.

[0146] That is to say, first convert the external traction force into the motion speed of the robotic arm, and then convert the motion speed of the robotic arm into the joint angles of each joint on the robotic arm to obtain the set of joint angles of the robotic arm.

[0147] As an example, when dragging the robotic arm to move within the position constraint area through position constraint, the following formula (1) can be used for the motion control of the robotic arm.

[0148]

[0149] Among them, V ∈ R 6×1 represents the generalized velocity control command of the Cartesian space at the end of the robotic arm (including linear velocity and angular velocity), K1 ∈ R 6×6 represents the mapping ratio matrix (6-dimensional diagonal matrix), F i ∈ R 6×1 represents the force exerted by the human hand to drag the robotic arm, that is, the external traction force. It can be seen from formula (1) that when the end position information of the robotic arm is within the preset position constraint area, including on the boundary of the preset position constraint area, the motion speed V of the robotic arm can be determined according to the external traction force F of the robotic arm; and when the end position information of the robotic arm is not within the preset position constraint area, the motion of the robotic arm is stopped.

[0150] Next, after the surgical robot determines the generalized velocity V of the Cartesian space at the end of the robotic arm, it can determine the joint angles of each joint of the robotic arm based on the velocity V to obtain the set of joint angles of the robotic arm; as shown in formula (2).

[0151]

[0152] Among them, θ represents the joint angles of each joint on the robotic arm, and J1 represents the pose Jacobian matrix.

[0153] Step 904: Control the end of the robotic arm to move to the vicinity of the target position according to the set of joint angles of the robotic arm.

[0154] In this embodiment, when controlling the motion of the end of the robotic arm according to the external traction force, by sequentially converting the external traction force into the motion speed of the robotic arm and the set of joint angles of the robotic arm, and then controlling the robotic arm to move under the action of the external traction force based on the set of joint angles of the robotic arm; adopting this method can improve the motion efficiency and motion stability of the robotic arm.

[0155] In an exemplary embodiment, as Figure 10 shown, the above step 206 may include:

[0156] Step 1002, when it is determined according to the end attitude information that the end of the robotic arm is within the attitude constraint region but not within the proximity region of the target attitude, control the end of the robotic arm to move to the proximity region of the target attitude according to the external traction force.

[0157] Wherein, the proximity region of the target attitude is within the attitude constraint region, and the distance between any position within the proximity region of the target attitude and the target attitude is less than or equal to a second preset distance threshold. For example: as Figure 13b shown, the shaded area therein may be used as the proximity region of the target attitude; of course, the proximity region of the target attitude may also be a cylindrical region at a second preset distance threshold from the target attitude, such as: a cylindrical region, a prismatic region, etc.; the shape of the proximity region of the target attitude is not specifically limited in the embodiments of the present application.

[0158] Step 1004, when it is determined according to the end attitude information that the robotic arm is within the proximity region of the target attitude, control the robotic arm to move to the target attitude.

[0159] That is to say, when the end of the robotic arm moves to the target position, the end attitude of the robotic arm can be adjusted; at this time, the user is actually adjusting the attitude of the end instrument fixed to the end of the robotic arm, so that the attitude of the end instrument is consistent with the target attitude. Since the end instrument is fixedly arranged on the end flange of the robotic arm, when adjusting the attitude of the end instrument, it is still necessary to change the attitude of the end instrument by adjusting the joint angles of each joint of the robotic arm.

[0160] In addition, when performing attitude adjustment, the end position of the robotic arm remains unchanged, that is, the target position of the robotic arm remains unchanged; that is to say, when adjusting the attitude of the end instrument, the position of the first end of the end instrument remains unchanged, and by changing the position of the second end of the end instrument, the relative position between the first end and the second end of the end instrument is adjusted, so as to change the attitude of the end instrument; that is, by controlling the second end of the end instrument to rotate around the first end, the attitude adjustment of the end instrument is realized.

[0161] When performing attitude adjustment, two adjustment stages may also be set, that is, in the first adjustment stage, the user drags the end instrument to perform attitude adjustment, and when it is adjusted to the proximity region of the target attitude, the robotic arm can automatically perform attitude adjustment to ensure that the attitude of the end instrument is adjusted to the target attitude. When using this method to perform attitude adjustment of the robotic arm, while improving the attitude adjustment efficiency of the robotic arm, the accuracy of attitude adjustment can also be improved.

[0162] In an exemplary embodiment, as Figure 11 shown, "controlling the end of the robotic arm to move to the proximity area of the target pose according to the external traction force" in step 1002 above may include:

[0163] Step 1102, determining the angular velocity of the robotic arm's movement according to the external traction force and the second mapping relationship, and determining the set of joint angles of the robotic arm according to the angular velocity of the robotic arm's movement and the second Jacobian relationship.

[0164] Wherein, the set of joint angles of the robotic arm includes the joint angles of each joint on the robotic arm.

[0165] Exemplarily, the angular velocity of the robotic arm's movement can be determined according to the external traction force and the second mapping relationship; then, according to the angular velocity of the robotic arm's movement and the second Jacobian relationship, the joint angles of each joint on the robotic arm are determined to obtain the set of joint angles of the robotic arm; that is, first the external traction force is converted into the angular velocity of the robotic arm's movement, and then the angular velocity of the robotic arm's movement is converted into the joint angles of each joint on the robotic arm.

[0166] As an example, when the robotic arm is moved within the preset pose constraint area by dragging with pose constraints, the following formula (3) can be used for the pose adjustment of the robotic arm.

[0167]

[0168] Where, ω ∈ R 3×1 represents the Cartesian space angular velocity control command of the robotic arm end, K2 ∈ R 3×6 represents the mapping ratio matrix, F i ∈ R 3×1 represents the force exerted by the human hand to drag the robotic arm, that is, the external traction force. Similarly, it can be seen from formula (3) that when the end position information of the robotic arm is within the preset pose constraint area, including on the boundary of the preset pose constraint area, the angular velocity ω of the robotic arm can be determined according to the external traction force F of the robotic arm; and in the case where the end position information of the robotic arm is not within the preset pose constraint area, the movement of the robotic arm is stopped.

[0169] Next, after the surgical robot determines the Cartesian space angular velocity ω of the robotic arm end, the joint angles of each joint of the robotic arm can be determined based on the angular velocity ω, as shown in formula (4).

[0170]

[0171] Where, θ represents the joint angles of each joint on the robotic arm, and J2 represents the pose Jacobian matrix.

[0172] Step 1104, control the robotic arm to move to the vicinity area of the target pose according to the joint angle set of the robotic arm.

[0173] In this example, when adjusting the pose of the end of the robotic arm according to the external traction force, the external traction force is sequentially converted into the motion angular velocity of the robotic arm and the joint angle set of the robotic arm, and then the pose of the robotic arm is adjusted based on the joint angle set of the robotic arm; by adopting this method, the adjustment efficiency of the robotic arm can be improved.

[0174] In an exemplary embodiment, as Figure 12 shown, the determination method for the target position and target pose of the robotic arm may include:

[0175] Step 1202, obtain the entry point position information and target point position information of the operation object.

[0176] Exemplarily, for a surgical robot, it can insert the end effector from the surface of the operation object into the operation object. Among them, the entry point position information can be the position information on the surface of the operation object, and the target point position information can be the position information inside the operation object; for example: the location of the lesion in the operation object. Exemplarily, the operation object here can be a patient.

[0177] Exemplarily, before performing a surgical operation, a medical imaging device can be used to perform medical imaging on the lesion tissue of the patient to obtain a medical image of the lesion tissue; the user can mark the entry point and target point in this medical image, so as to obtain the entry point position information and target point position information in the image coordinate system.

[0178] Exemplarily, the target point position information in the image coordinate system can also be obtained by automatically identifying the lesion in the medical image; furthermore, based on the target point position information, the entry point position information in the image coordinate system can be determined.

[0179] Step 1204, determine the target position and target pose according to the entry point position information and target point position information.

[0180] Among them, the target position is the position information in the base coordinate system of the surgical robot, and the target pose is the pose information in the base coordinate system of the surgical robot. Alternatively, the target position can be the position information in the robotic arm coordinate system of the surgical robot, and the target pose can be the pose information in the robotic arm coordinate system of the surgical robot.

[0181] Exemplarily, since the entry point is a point on the patient's skin surface and the target point is inside the patient's skin, that is, a point inside the patient's body, when the end effector (such as a surgical needle) is advanced into the patient's body with the assistance of a robotic arm, the end of the robotic arm still needs to remain outside the patient's body; therefore, the position of the end of the robotic arm should be the position of the entry point, or a position above the entry point. Based on this, when determining the target position of the robotic arm, it should be determined based on the entry point position information.

[0182] Exemplarily, the coordinate transformation relationship between the image coordinate system and the base coordinate system of the surgical robot can be determined first; then, based on this coordinate transformation relationship, the entry point position information in the image coordinate system is transformed into the entry point position information in the base coordinate system of the surgical robot, so as to determine the target position information of the robotic arm based on the transformed entry point position information in the base coordinate system of the surgical robot, that is, to determine the target position of the robotic arm. Exemplarily, the position at a preset distance above the entry point can be determined as the target position of the robotic arm. Of course, this target position should be located on the extension line between the target point and the entry point.

[0183] Exemplarily, when determining the end pose of the robotic arm, that is, determining the pose of the end effector, since the end effector of the robotic arm needs to set the needle insertion channel for the surgical needle so as to advance the surgical needle into the patient's body based on this needle insertion channel and thus reach the target position; therefore, the end pose of the robotic arm, that is, the pose of the end effector of the robotic arm, should be consistent with the path between the entry point and the target point.

[0184] As an example, the needle insertion path can be determined first according to the entry point position information and the target point position information in the image coordinate system; then, based on the coordinate transformation relationship between the image coordinate system and the base coordinate system of the surgical robot, this needle insertion path is transformed into the needle insertion pose in the base coordinate system of the surgical robot, and the needle insertion pose in the base coordinate system of the surgical robot is used as the target pose of the robotic arm.

[0185] As another example, it is also possible to first, based on the coordinate transformation relationship between the image coordinate system and the base coordinate system of the surgical robot, transform the entry point position information and the target point position information in the image coordinate system into the entry point position information and the target point position information in the base coordinate system of the surgical robot respectively; then, according to the entry point position information and the target point position information in the base coordinate system of the surgical robot, determine the needle insertion pose in the base coordinate system of the surgical robot, and use the needle insertion pose in the base coordinate system of the surgical robot as the target pose of the robotic arm.

[0186] In this example, when determining the target position and target pose of the robotic arm, the entry point position information and target point position information of the operation object can be obtained first. Then, based on the entry point position information and target point position information, the target position and target pose are determined. By adopting this method, the entry point and target point can be marked based on the medical image of the operation object used during the surgery, and the target position and target pose of the end effector of the surgical robot can be automatically determined based on the marked entry point and target point, so as to realize the adjustment of the end pose of the surgical robot and improve the control efficiency of the surgical robot.

[0187] Next, a specific surgical application scenario is used to describe the above robotic arm control method. As an example, in the spinal surgery scenario, as shown in FIG. 13, it shows the system schematic diagram of the spinal surgery robot. Among them, the spinal surgery robot system includes a motion planning controller 1, a surgical trolley 2, a navigation device 3, a navigation device support arm 4, a surgical robotic arm 5, a force sensor 6, a surgical instrument positioning channel 7, a patient array 8, and a patient 9.

[0188] Among them, the motion planning controller 1 is used to realize surgical navigation positioning motion planning; the navigation device 3 and the patient array 8 are used to realize surgical system registration, that is, to complete the calculation of the surgical target pose in the robotic arm coordinate system; the surgical robotic arm 5 is used to realize the precise navigation positioning of the surgical instrument, that is, to move the surgical instrument positioning channel 7 at the end of the robotic arm to the surgical target pose; the force sensor 6 is used to obtain the information of the robotic arm's human-machine interaction force perception and assist in generating the motion intention; the surgical instrument channel 7 realizes the precise guidance of the surgical instrument on the basis of the precise positioning of the robotic arm, such as pushing the surgical needle from the entry point to the target point.

[0189] Before the surgery, the doctor can plan the entry point Pe and target point Pt of the orthopedic surgical instrument on the preoperative image of the patient (as Figure 13c shown), and obtain the entry point position and target point position in the robot base coordinate system through coordinate transformation. Thus, according to the entry point position and target point position in the robot base coordinate system, the target position of the robotic arm (such as Pt1 in Figure 13c ) and the target pose (such as Pe2-Pt1 in Figure 13c ) are determined; at the same time, according to the entry point position and target point position in the robot base coordinate system, the position constraint curve (such as curve 1 in Figure 13a ) and the pose constraint curve (such as curve 2 in Figure 13b ) are determined, and further the position constraint region and the pose constraint region are determined.

[0190] Refer to Figure 14As shown, the robotic arm navigation and positioning motion planning process based on human-robot coexistence may include: preoperative planning, that is, determining the position information of the needle insertion point and the target point in the image coordinate system, and determining the position constraint area and the attitude constraint area based on the needle insertion point and the target point; then, based on the position constraint area, controlling the robotic arm to move its position through the external traction force applied by the user on the robotic arm; Refer to Figure 13a As shown, under the dragging action of the user, the end of the robotic arm can be moved from Pe i -Pt i to the Pe1-Pt1 position; that is, at this time, the end of the robotic arm moves to the target position Pt1. Then, when the position constraint is satisfied, that is, when the end of the robotic arm moves to the target position, enter the attitude constraint; based on the attitude constraint area, controlling the end joint of the robotic arm to adjust its attitude through the external traction force applied by the user on the robotic arm until the attitude constraint is satisfied, that is, the end attitude of the robotic arm moves to the target attitude. Refer to Figure 13b As shown, under the dragging action of the user, the end attitude of the robotic arm can be adjusted from Pe1-Pt1 to Pe2-Pt1; that is, at this time, the end attitude of the robotic arm moves to the target attitude Pe2-Pt1.

[0191] It should be noted that when adjusting the attitude constraint of the robotic arm, the position of the end of the robotic arm needs to be kept unchanged, and only the attitude of the end joint is adjusted around the axis of the end instrument. When the robotic arm surgical instrument is dragged to the target attitude, the attitude constraint dragging is completed, that is, the motion planning under the constraint is realized, so that the robotic arm can accurately and safely reach the target pose.

[0192] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0193] Based on the same inventive concept, an embodiment of the present application further provides a robotic arm control device for implementing the robotic arm control method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the robotic arm control device provided below can refer to the limitations on the robotic arm control method in the above text, and will not be elaborated here.

[0194] In an exemplary embodiment, as Figure 15 shown, a robotic arm control device is provided, including: a first acquisition module 1502, a first control module 1504, and a second control module 1506, where:

[0195] The first acquisition module 1502 is configured to acquire the external traction force acting on the robotic arm and acquire the end pose information of the robotic arm; the end pose information includes end position information and end attitude information.

[0196] The first control module 1504 is configured to control the robotic arm to move to a target position according to the external traction force when it is determined that the robotic arm is within the preset position constraint area based on the end position information.

[0197] The second control module 1506 is configured to control the robotic arm to move to a target attitude according to the external traction force when it is determined that the robotic arm is within the preset attitude constraint area based on the end attitude information.

[0198] In one of the embodiments, the device further includes:

[0199] A first determination module, configured to determine the first central axis of the position constraint area according to the target position;

[0200] A second determination module, configured to determine the position constraint curve according to the target position and the preset boundary points;

[0201] A third determination module, configured to rotate around the first central axis for one week with the position constraint curve as the generatrix to obtain the position constraint area.

[0202] In one of the embodiments, the device further includes:

[0203] A fourth determination module, configured to determine the second central axis of the attitude constraint area according to the target position and the target attitude;

[0204] A fifth determination module, configured to determine the attitude constraint curve according to the target position and the preset offset angle;

[0205] A sixth determination module, configured to rotate around the second central axis for one week with the attitude constraint curve as the generatrix to obtain the attitude constraint area.

[0206] In one embodiment, an end effector is mounted at the end of the robotic arm. The end effector includes a first end and a second end. The first acquisition module 1502 includes:

[0207] A first acquisition unit, configured to acquire the position information of the first end of the end effector, and determine the end position information of the robotic arm according to the position information of the first end;

[0208] A second acquisition unit, configured to acquire the position information of the second end of the end effector, and determine the end attitude information of the robotic arm according to the position information of the first end and the position information of the second end.

[0209] In one embodiment, when it is determined according to the end position information that the end of the robotic arm is located at the edge of the position constraint area or the edge of the attitude constraint area, the device further includes:

[0210] A seventh determination module, configured to determine the direction of the first component force of the external traction force in the normal direction of the edge;

[0211] A third control module, configured to control the robotic arm to move along the direction of the external traction force when the direction of the first component force points to the inside of the position constraint area or the inside of the attitude constraint area;

[0212] A fourth control module, configured to control the robotic arm to move along the direction of the second component force of the external traction force on the tangent plane of the edge when the direction of the first component force points to the outside of the position constraint area or the outside of the attitude constraint area.

[0213] In one embodiment, the first control module 1504 includes:

[0214] A first control unit, configured to, when it is determined according to the end position information that the end of the robotic arm is located within the position constraint area but not within the proximity area of the target position, control the end of the robotic arm to move to the proximity area of the target position according to the external traction force; the proximity area of the target position is located within the position constraint area, and the distance between any position within the proximity area of the target position and the target position is less than or equal to a first preset distance threshold;

[0215] A second control unit, configured to control the end of the robotic arm to move to the target position when it is determined according to the end position information that the end of the robotic arm is located within the proximity area of the target position.

[0216] In one embodiment, the first control unit is configured to determine the movement speed of the robotic arm according to the external traction force and a first mapping relationship, and determine the set of joint angles of the robotic arm according to the movement speed of the robotic arm and a first Jacobian relationship; and control the end of the robotic arm to move to the proximity area of the target position according to the set of joint angles of the robotic arm.

[0217] In one embodiment, the second control module 1506 includes:

[0218] A third control unit, configured to, when it is determined according to the end attitude information that the end of the robotic arm is within the attitude constraint region but not within the proximity region of the target attitude, control the end of the robotic arm to move to the proximity region of the target attitude according to the external traction force; the proximity region of the target attitude is within the attitude constraint region, and the distance between any position within the proximity region of the target attitude and the target attitude is less than or equal to a second preset distance threshold;

[0219] A fourth control unit, configured to, when it is determined according to the end attitude information that the robotic arm is within the proximity region of the target attitude, control the robotic arm to move to the target attitude.

[0220] In one embodiment, the third control unit is configured to determine the movement angular velocity of the robotic arm according to the external traction force and the second mapping relationship, and determine the joint angle set of the robotic arm according to the movement angular velocity of the robotic arm and the second Jacobian relationship; and control the robotic arm to move to the proximity region of the target attitude according to the joint angle set of the robotic arm.

[0221] In one embodiment, the device further includes:

[0222] A second acquisition module, configured to acquire the entry point position information and the target point position information of the operation object;

[0223] An eighth determination module, configured to determine the target position and the target attitude according to the entry point position information and the target point position information.

[0224] Each module in the above robotic arm control device can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in the form of hardware or independent of it, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0225] In an exemplary embodiment, a surgical robot is provided. The surgical robot may include the above-mentioned robotic arm control device and a robotic arm, wherein the robotic arm is configured to be controlled by the above-mentioned robotic arm control device and perform movements. Its control method can refer to the relevant content descriptions of the robotic arm control method in the above-mentioned embodiments, and will not be repeated here.

[0226] In an exemplary embodiment, a computer device is provided, and its internal structure diagram can be as Figure 16As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as the target position and target posture of the robotic arm, as well as the preset position constraint area and preset posture constraint area. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external devices through a network connection. When the computer program is executed by the processor, it implements a robotic arm control method.

[0227] Those skilled in the art can understand that Figure 16 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0228] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, it implements the steps of the robotic arm control method in any of the above embodiments.

[0229] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, it implements the steps of the robotic arm control method in any of the above embodiments.

[0230] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, it implements the steps of the robotic arm control method in any of the above embodiments.

[0231] Those of ordinary skill in the art can understand that all or part of the processes in the above-described embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-described method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0232] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0233] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A manipulator control method, characterized in that, The method includes: Obtaining an external traction force acting on the robotic arm and obtaining the end pose information of the robotic arm; the end pose information includes end position information and end attitude information; When it is determined according to the end position information that the robotic arm is within the position constraint region, controlling the robotic arm to move to a target position according to the external traction force; When it is determined according to the end attitude information that the robotic arm is within the attitude constraint region, controlling the robotic arm to move to a target attitude according to the external traction force.

2. The method according to claim 1, wherein The method further includes: Determining a first central axis of the position constraint region according to the target position; Determining a position constraint curve according to the target position and a preset boundary point; Rotating once around the first central axis with the position constraint curve as the generatrix to obtain the position constraint region.

3. The method according to claim 1, wherein The method further includes: Determining a second central axis of the attitude constraint region according to the target position and the target attitude; Determining an attitude constraint curve according to the target position and a preset offset angle; Rotating once around the second central axis with the attitude constraint curve as the generatrix to obtain the attitude constraint region.

4. The method according to claim 1, wherein An end instrument is installed at the end of the robotic arm, and the end instrument includes a first end and a second end. The obtaining of the end pose information of the robotic arm includes: Obtaining the position information of the first end of the end instrument and determining the end position information of the robotic arm according to the position information of the first end; Obtaining the position information of the second end of the end instrument and determining the end attitude information of the robotic arm according to the position information of the first end and the position information of the second end.

5. The method according to claim 1, wherein When it is determined according to the end position information that the end of the robotic arm is at the edge of the position constraint region or the edge of the attitude constraint region, the method further includes: Determining the direction of a first component force of the external traction force in the normal direction of the edge; If the direction of the first component force points to the inside of the position constraint region or the inside of the attitude constraint region, controlling the robotic arm to move along the direction of the external traction force; If the direction of the first component force points to the outside of the position constraint region or the outside of the attitude constraint region, controlling the robotic arm to move along the direction of a second component force of the external traction force on the tangent plane of the edge.

6. The method according to any one of claims 1-5, characterized in that, The controlling the robotic arm to move to a target position according to the external traction force when it is determined according to the end position information that the robotic arm is within the position constraint region includes: When it is determined according to the end position information that the end of the robotic arm is within the position constraint region but not within the proximity region of the target position, controlling the end of the robotic arm to move to the proximity region of the target position according to the external traction force; the proximity region of the target position is within the position constraint region, and the distance between any position within the proximity region of the target position and the target position is less than or equal to a first preset distance threshold; When it is determined, based on the end position information, that the end of the robotic arm is located in the proximity area of the target position, control the end of the robotic arm to move to the target position.

7. The method according to claim 6, wherein The controlling the end of the robotic arm to move to the proximity area of the target position according to the external traction force includes: Determine the movement speed of the robotic arm according to the external traction force and a first mapping relationship, and determine the set of joint angles of the robotic arm according to the movement speed of the robotic arm and a first Jacobian relationship; Control the end of the robotic arm to move to the proximity area of the target position according to the set of joint angles of the robotic arm.

8. The method according to any one of claims 1-5, characterized in that When it is determined, based on the end pose information, that the robotic arm is located within the pose constraint area, controlling the robotic arm to move to the target pose according to the external traction force includes: When it is determined, based on the end pose information, that the end of the robotic arm is located within the pose constraint area but not in the proximity area of the target pose, control the end of the robotic arm to move to the proximity area of the target pose according to the external traction force; the proximity area of the target pose is located within the pose constraint area, and the distance between any position within the proximity area of the target pose and the target pose is less than or equal to a second preset distance threshold; When it is determined, based on the end pose information, that the robotic arm is located in the proximity area of the target pose, control the robotic arm to move to the target pose.

9. The method according to claim 8, wherein The controlling the robotic arm to move to the proximity area of the target pose according to the external traction force includes: Determine the movement angular velocity of the robotic arm according to the external traction force and a second mapping relationship, and determine the set of joint angles of the robotic arm according to the movement angular velocity of the robotic arm and a second Jacobian relationship; Control the robotic arm to move to the proximity area of the target pose according to the set of joint angles of the robotic arm.

10. A robotic arm control device, characterized in that, The device includes: A first acquisition module, configured to acquire the external traction force acting on the robotic arm and acquire the end pose information of the robotic arm; the end pose information includes end position information and end pose information; A first control module, configured to control the robotic arm to move to the target position according to the external traction force when it is determined, based on the end position information, that the robotic arm is located within the position constraint area; A second control module, configured to control the robotic arm to move to the target pose according to the external traction force when it is determined, based on the end pose information, that the robotic arm is located within the pose constraint area.

11. A surgical robot, characterized in that, Including the robotic arm control device as claimed in claim 10 and a robotic arm, the robotic arm being configured to be controlled by the robotic arm control device and perform movements.

12. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method as claimed in any one of claims 1 to 9 are implemented.

Images