Mechanical arm control method and device and surgical robot system

Abstract

The invention relates to a mechanical arm control method and device and a surgical robot system. The method comprises the following steps: acquiring a target path planning result of the mechanical arm; the target path planning result comprises the detection distance between the mechanical arm and the target object under each path point, it can be ensured that the mechanical arm does not collide with an obstacle in the moving process, and therefore the safety of movement of the mechanical arm is ensured; processing the target path planning result to obtain a tail end speed planning result of the mechanical arm as a basis for controlling the movement of the mechanical arm, so as to realize the variable speed adjustment of the mechanical arm; and based on the tail end speed planning result, the mechanical arm is controlled to move, and the effects that the joint movement speed of the mechanical arm is reduced when approaching the target object and is increased when being far away from the target object can be achieved. By adopting the method, the movement efficiency of the mechanical arm and the adjustability of mechanical arm control can be improved.

Description

Technical Field

[0001] The present application relates to the field of medical device technology, and in particular to a robotic arm control method, device and surgical robot system. Background Art

[0002] With the development of surgical robot technology, the robot arm is usually kept in constant motion during its autonomous motion. If the robot arm moves at a constant speed too fast, it will increase the safety risk during the operation. If the constant speed is too slow, it will prolong the operation time. The above-mentioned robot arm movement speed adjustment method is relatively simple, which is easy to affect the safety of the operation process or increase unnecessary operation time.

[0003] However, the current robotic arm control methods or traditional methods have the problem of poor motion adjustability. Summary of the invention

[0004] Based on this, it is necessary to provide an adjustable robotic arm control method, device and surgical robot system to address the above technical problems.

[0005] In a first aspect, the present application provides a method for controlling a robotic arm, the method comprising:

[0006] Obtain the target path planning result of the robot arm; the target path planning result includes the detection distance between the robot arm and the target object at each path point;

[0007] Process the target path planning results to obtain the end velocity planning results of the robot arm;

[0008] Based on the terminal velocity planning results, the robot arm movement is controlled.

[0009] In one embodiment, obtaining a target path planning result of a robotic arm includes:

[0010] Perform path planning for the robotic arm to obtain the initial path planning result of the robotic arm;

[0011] Based on the initial path planning results, collision detection is performed on the robotic arm to obtain the target path planning results.

[0012] In one embodiment, based on the terminal velocity planning result, controlling the movement of the robot arm includes:

[0013] Based on the terminal velocity planning result, the joint velocity planning result of the robot arm is obtained;

[0014] Control the movement of the robotic arm based on the target path planning results and joint speed planning results.

[0015] In one embodiment, processing the target path planning result to obtain the end velocity planning result of the robot arm includes:

[0016] The preset model is used to process each detection distance to obtain the terminal speed planning result; the terminal speed planning result includes the terminal speed level of the manipulator at each path point; the preset model includes the mapping relationship between the detection distance and the terminal speed level.

[0017] In one embodiment, the detection distance includes a minimum distance between each joint of the robot arm and the target object.

[0018] In one embodiment, the robot arm is provided with an end tool; a gripping portion of the end tool is provided with a sensor; and the method further comprises:

[0019] Obtain the sensing signal output by the sensor;

[0020] According to the sensing signal, the robot arm is controlled to switch between force control mode and position control mode.

[0021] In one embodiment, controlling the robot arm to switch between the force control mode and the position control mode according to the sensing signal includes:

[0022] If the sensing signal meets the first threshold range, the robot arm is controlled to enter the force control mode; the first threshold range is used to indicate that the end tool is in contact with the target state;

[0023] If the sensing signal meets the second threshold range, the robot arm is controlled to enter the position control mode; the second threshold range is used to indicate that the end tool is in a non-contact target state.

[0024] In a second aspect, the present application provides a robot arm control device, the device comprising:

[0025] The target path planning module is used to obtain the target path planning result of the robot arm; the target path planning result includes the detection distance between the robot arm and the target object at each path point;

[0026] The terminal speed planning module is used to process the target path planning results and obtain the terminal speed planning results of the robot arm;

[0027] The robot arm motion control module is used to control the robot arm motion based on the terminal velocity planning result.

[0028] In a third aspect, the present application provides a surgical robot system, which includes a robotic arm; the system also includes a memory and a processor, the memory stores a computer program, the processor is connected to the robotic arm, and the processor implements the steps of the above method when executing the computer program.

[0029] In a fourth aspect, the present application provides a surgical robot system, which includes a robotic arm, the robotic arm is provided with an end tool, and the gripping part of the end tool is provided with a sensor; the system also includes a memory and a processor, the memory stores a computer program, the processor is connected to the sensor and the robotic arm, and the processor implements the steps of the above method when executing the computer program.

[0030] The above-mentioned robot arm control method, device and surgical robot system, the method obtains the target path planning result of the robot arm, the target path planning result includes the detection distance between the robot arm and the target object at each path point, which can ensure that the robot arm will not collide with obstacles during the movement, thereby ensuring the safety of the robot arm movement; processing the target path planning result to obtain the terminal speed planning result of the robot arm as a basis for controlling the movement of the robot arm, thereby realizing the speed adjustment of the robot arm; based on the terminal speed planning result, the movement of the robot arm is controlled, which can achieve the effect of reducing the joint movement speed of the robot arm when approaching the target object and increasing the joint movement speed of the robot arm when away from the target object, thereby improving the movement efficiency of the robot arm and the adjustability of the robot arm control. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a flow chart of a method for controlling a robotic arm in one embodiment;

[0032] Figure 2 A schematic flow chart of a robot arm control step in one embodiment;

[0033] Figure 3 A schematic flow chart of a robot arm control step in another embodiment;

[0034] Figure 4 Schematic diagram of the mapping relationship between the distance level of the minimum distance and the terminal speed level in one embodiment;

[0035] Figure 5 is a flowchart of the robot arm control steps in another example;

[0036] Figure 6 A schematic diagram of a flow chart of a robot arm control method in another embodiment;

[0037] Figure 7 is a schematic diagram of a registration probe in one embodiment;

[0038] Figure 8 A schematic diagram of a flow chart of a robot arm control step in yet another embodiment;

[0039] Fig. 9 The figure is a flowchart of the robot arm control steps in an example;

[0040] Fig.10is a structural block diagram of a robot arm control device in one embodiment;

[0041] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The robotic arm control method provided in the present application can be applied to a processor in a surgical robot system; the surgical robot system also includes a robotic arm, wherein the processor can be connected to the robotic arm to control the movement of the robotic arm.

[0044] In one embodiment, if Figure 1 As shown, the method includes:

[0045] Step 110, obtaining a target path planning result of the robot arm; the target path planning result includes a detection distance between the robot arm and the target object at each path point;

[0046] Specifically, the robot arm can be equipped with sensors to sense the surrounding environment, for example, at least one of a laser radar and a visual sensor, to provide information about the distance, position, shape, etc. between the end of the robot arm and the surrounding objects. Further, the surrounding environment model of the robot arm can be established through the data obtained by the sensor, and the surrounding environment model can include information such as the position, shape, size, etc. of obstacles. By performing path planning on the robot arm, a target path planning result can be obtained, and the target path planning result can include the detection distance between the robot arm and the target object at each path point, and then a plurality of path points that have been subjected to collision detection can be obtained. If the end of the robot arm moves to each path point in turn, it can be ensured that there is no collision with obstacles in the surrounding environment. Among them, in the process of path planning for the robot arm, the detection distance between the robot arm and the target object at each path point can be obtained by performing collision detection on the robot arm based on the surrounding environment model. The target object can include obstacles in the environment, and can also include the target object of the actuator set at the end of the robot arm, for example, a patient. By performing collision detection on the robot arm during path planning for the robot arm, the target path planning result obtained may include the collision detection result. For example, a collision detection algorithm may be used to compare the planned motion trajectory of the robot arm with the position of obstacles in the surrounding environment to determine whether there is a collision risk, and obtain the collision detection results of each joint of the robot arm. The collision detection results may include the distance between each joint of the robot arm and obstacles in the surrounding environment. The detection distance between the robot arm and the target object may be determined based on the collision detection results. By obtaining the target path planning result of the robot arm, it can be ensured that the robot arm will not collide with obstacles during movement, thereby ensuring the safety of the robot arm movement.

[0047] In some examples, the collision detection algorithm may include a geometry-based method (eg, bounding box, convex hull, etc.) or a distance-based method (eg, closest point distance, minimum distance, etc.), etc.

[0048] Step 120, processing the target path planning result to obtain the end velocity planning result of the robot arm;

[0049] Specifically, the target path planning result can be processed, and the speed of the end of the robot arm can be reasonably planned based on the detection distance between the robot arm and the target object at each path point to obtain the end speed planning result of the robot arm. The end speed planning result can include the linear speed of the end of the robot arm. The end speed planning result obtained based on each detection distance in the target path planning result can be used as a basis for controlling the movement of the robot arm, thereby realizing the speed adjustment of the robot arm.

[0050] Step 130, controlling the movement of the robot arm based on the terminal velocity planning result.

[0051] Specifically, based on the terminal velocity planning results of the robot arm, the speed of each joint of the robot arm can be further adaptively adjusted to control the movement of the robot arm. This can achieve the effect of reducing the joint movement speed of the robot arm when approaching the target object and increasing the joint movement speed of the robot arm when moving away from the target object, thereby improving the movement efficiency of the robot arm and the adjustability of the robot arm control.

[0052] The embodiment of the present application obtains the target path planning result of the robotic arm, which includes the detection distance between the robotic arm and the target object at each path point, to ensure that the robotic arm will not collide with obstacles during movement, thereby ensuring the safety of the robotic arm's movement; processes the target path planning result to obtain the terminal speed planning result of the robotic arm as a basis for controlling the movement of the robotic arm, thereby achieving variable speed adjustment of the robotic arm; based on the terminal speed planning result, the movement of the robotic arm is controlled, which can achieve the effect of reducing the joint movement speed of the robotic arm when approaching the target object and increasing the joint movement speed of the robotic arm when away from the target object, thereby improving the movement efficiency of the robotic arm and the adjustability of the robotic arm control.

[0053] In one embodiment, if Figure 2 As shown, the target path planning results of the robot arm are obtained, including:

[0054] Step 210, performing path planning for the robot arm to obtain an initial path planning result of the robot arm;

[0055] Step 220: Based on the initial path planning result, collision detection is performed on the robot arm to obtain a target path planning result.

[0056] Specifically, the path planning of the robot arm can be performed. For example, the starting point and the target point of the end of the robot arm can be obtained, and the motion trajectory of the end of the robot arm can be determined through the corresponding path planning algorithm, so as to realize the movement of the end of the robot arm from the starting point to the target point. The initial path planning result of the robot arm can include multiple motion trajectories of the end of the robot arm, and each motion trajectory can correspond to a corresponding multiple path points. Based on the initial path planning result, the robot arm can be subjected to collision detection to avoid collision risks and ensure that the robot arm moves based on the obtained target path planning result without colliding with surrounding obstacles. The target path planning result can include a motion trajectory without collision, and the motion trajectory can correspond to multiple path points to indicate that the end of the robot arm moves to each path point in sequence.

[0057] In one embodiment, if Figure 3 As shown, based on the terminal velocity planning result, the robot arm movement is controlled, including:

[0058] Step 310, based on the terminal velocity planning result, obtain the joint velocity planning result of the robot arm;

[0059] Step 320, controlling the movement of the robot arm according to the target path planning result and the joint speed planning result.

[0060] Specifically, the terminal velocity planning result can be converted into the joint velocity planning result of the robot arm. For example, for any path point obtained by planning, the terminal linear velocity of the end of the robot arm moving from the current path point to the path point can be converted through a corresponding algorithm to obtain the joint value velocity required by each joint of the robot arm in this process. The joint value velocity may include the angle or position value changed by each joint of the robot arm in unit time. Further, according to the target path planning result and the joint velocity planning result, the joint value velocity corresponding to each path point obtained by planning can be sent to the robot arm to control the robot arm to move according to the planned motion trajectory and joint value velocity including each path point. By controlling the movement of the robot arm according to the target path planning result and the joint velocity planning result, the robot arm can change speed during movement to avoid obstacles, thereby improving the safety and effectiveness of the movement of the robot arm.

[0061] In one embodiment, processing the target path planning result to obtain the end velocity planning result of the robot arm includes:

[0062] The preset model is used to process each detection distance to obtain the terminal speed planning result; the terminal speed planning result includes the terminal speed level of the manipulator at each path point; the preset model includes the mapping relationship between the detection distance and the terminal speed level.

[0063] Specifically, the target path planning result may include multiple path points on at least one motion trajectory. For any path point, the distance between each joint and the target object when the end of the robot moves to the path point can be obtained, and then the detection distance between the robot and the target object can be determined. For example, the minimum distance between each joint of the robot and the target object can be determined as the detection distance. The target object can include obstacles in the environment, and can include the target object of the actuator set at the end of the robot, such as a patient; the mapping relationship between the detection distance and the terminal speed level can be determined by establishing a preset model, wherein each terminal speed level can correspond to a corresponding terminal speed percentage, and the terminal speed percentage can be used to describe the ratio of the actual movement speed of the end of the robot to the maximum movement speed of the end, and then the detection distance between the robot and the target object in the target path planning result is processed by using the established preset model, and the terminal speed level and the corresponding terminal speed percentage can be obtained, and the terminal speed percentage of the robot is used as the target speed planning result. By measuring the detection distance between the robot arm and the target object, the terminal speed of the robot arm is allocated as a percentage, so that the movement speed of the robot arm can be reduced when approaching the target object and increased when moving away from the target object. This saves time and improves efficiency while increasing the safety of the robot arm's movement.

[0064] In one embodiment, the detection distance includes a minimum distance between each joint of the robot arm and the target object.

[0065] Specifically, the target path planning result may include the distance between each joint of the manipulator and the target object at each path point, and then the detection distance may be determined based on the distance between each joint of the manipulator and the target object at each path point, and the detection distance may include the minimum distance between each joint of the manipulator and the target object. The preset model may include a mapping relationship between the minimum distance and the terminal speed level, and then the established preset model is used to process the minimum distance between the manipulator and the target object in the target path planning result, and the terminal speed level and the corresponding terminal speed percentage may be obtained, and the terminal speed percentage of the manipulator is used as the target speed planning result. By allocating the speed percentage of the terminal speed of the manipulator according to the minimum distance between the manipulator and the target object, the function of reducing the movement speed of the manipulator when approaching the target object and increasing the movement speed of the manipulator when away from the target object can be achieved, which saves time and improves efficiency while improving the safety of the manipulator's movement.

[0066] In some examples, the robotic arm may include joint 1, joint 2, joint 3, joint 4, joint 5, and joint 6. The minimum distance may be the minimum value of the distances between the joints and the patient, for example, the shortest distance between joint 1 and the patient is 60 mm; the shortest distance between joint 2 and the patient is 30 mm; the shortest distance between joint 3 and the patient is 40 mm; the shortest distance between joint 4 and the patient is 35 mm; the shortest distance between the joints and the patient is 50 mm; the shortest distance between joint 6 and the patient is 20 mm, then the minimum distance may be determined to be 20 mm.

[0067] In some examples, the preset model may include a neural network model, a fuzzy control model, etc. For the terminal speed percentage, the maximum movement speed of the end of the robot arm is Vmax, and the actual movement speed is V. Then, the terminal speed percentage can be calculated by the following formula:

[0068] Terminal speed percentage = (V / Vmax)*100%

[0069] In the control of the robot arm, the movement speed of the robot arm can be controlled by adjusting the end speed percentage. A higher end speed percentage means that the end is moving at a speed close to its maximum speed, which can achieve fast movement; a lower end speed percentage means that the end is moving at a slower speed, which can achieve more delicate and slow movement.

[0070] In some examples, taking the fuzzy control model as an example, the fuzzy control model can be obtained in the following manner: taking the minimum distance as the input variable and the terminal speed level as the output variable; establishing a fuzzy control rule between the input variable and the output variable; and establishing a fuzzy control model according to the fuzzy control rule.

[0071] Specifically, the minimum distance between the manipulator and the target object can be used as the input variable of the fuzzy control model, and the terminal speed level can be used as the output variable of the fuzzy control model, and they can be fuzzified, and the continuous value can be converted into a discrete fuzzy set. For each input variable and output variable, the shape and membership function of the fuzzy set can be determined to describe its membership degree. Fuzzy control rules between input variables and output variables can be established. Fuzzy control rules define how input variables are mapped to output variables. Fuzzy control rules can be used in a form similar to "IF-THEN", where the IF part is the condition of the input variable and the THEN part is the result of the output variable. Fuzzy control rules can be determined by expert knowledge or using training algorithms. After determining the fuzzy control rules, fuzzy reasoning can be used to determine the fuzzy results of the output variables. Fuzzy reasoning is based on rule matching and fuzzy logic operations to determine the fuzzy sets and membership of the output variables. After fuzzy reasoning, the fuzzy output can be converted into specific actual values. Defuzzification is the process of converting fuzzy results into specific output values. Commonly used defuzzification methods include maximum membership method, centroid method, etc. The membership function of the input variable, the membership function of the output variable and the fuzzy rules of the fuzzy control model can be determined by expert knowledge, experience or by using training algorithms (such as genetic algorithms, fuzzy clustering, etc.). In addition, the fuzzy control model can be adjusted and optimized according to actual conditions. Based on fuzzy control theory, the terminal speed level of the path planned by the robot arm is regulated to achieve the effect of changing the movement speed of the robot arm according to the minimum distance between the robot arm and the patient, which can reduce the operation time of the robot arm.

[0072] In some examples, the speed of the robot arm (e.g., the end linear speed) can be divided into three fuzzy sets: LS (slow), NS (medium), QS (fast), and the corresponding speed ranges are divided into five speed levels: 0, 1, 2, 3, 4. As shown in the following table:

[0073]

[0074] It should be noted that in fuzzy control, the degree of membership can be used to describe the degree to which a certain quantity belongs to a certain fuzzy linguistic variable (fuzzy set). The value range of the degree of membership is usually between 0 and 1, which indicates the strength of the relationship between the certain quantity and the fuzzy set. The larger the degree of membership, the higher the degree of matching between the certain quantity and the fuzzy set. The five speed levels corresponding to the end speed of the robot arm are the certain quantities. Through the above rule table, the speed of the above robot arm can be converted into three fuzzy sets to realize the fuzzification of the output variable. Among them, the membership degree of speed level 0 for LS (slow) fuzzy set can be 1, and the membership degrees for NS (medium) fuzzy set and QS (fast) fuzzy set can be 0; the membership degree of speed level 1 for LS (slow) fuzzy set and NS (medium) fuzzy set can be 0.5, and the membership degree for QS (fast) fuzzy set can be 0; the membership degree of speed level 2 for LS (slow) fuzzy set and QS (fast) fuzzy set can be 0, and the membership degree for NS (medium) fuzzy set can be 1; the membership degree of speed level 3 for LS (slow) fuzzy set can be 0, and the membership degree for NS (medium) fuzzy set and QS (fast) fuzzy set can be 0.5; the membership degree of speed level 4 for LS (slow) fuzzy set and NS (medium) fuzzy set can be 0, and the membership degree for QS (fast) fuzzy set can be 1. The above membership degrees can be determined based on corresponding empirical data.

[0075] According to the robot collision detection, the minimum distance between the robot body and the patient during the execution of the robot can be obtained, and the minimum distance is divided into five fuzzy sets: Very Near, Near, Normal, Far, Very Far, and the corresponding distance range is divided into five distance levels: -2, -1, 0, 1, 2. As shown in the following table:

[0076]

[0077] The five distance levels corresponding to the minimum distance are determined quantities. Through the above rule table, the above distance levels can be converted into five fuzzy sets to realize the fuzzification of input variables. Among them, the membership of distance level -2 for Very Near fuzzy set can be 1, and the membership of other fuzzy sets can be 0; the membership of distance level -1 for Near fuzzy set can be 1, and the membership of other fuzzy sets can be 0; the membership of distance level 0 for Normal fuzzy set can be 1, and the membership of other fuzzy sets can be 0; the membership of distance level 1 for Far fuzzy set can be 1, and the membership of other fuzzy sets can be 0; the membership of distance level 2 for Very Far fuzzy set can be 1, and the membership of other fuzzy sets can be 0. The above memberships can all be determined based on corresponding empirical data.

[0078] Based on empirical data, fuzzy control rules can be established and described in the following form:

[0079] (1) if Distance = Very Near then Speed ​​= LS, if the minimum distance belongs to the Very Near fuzzy set, then the terminal speed level belongs to the LS fuzzy set;

[0080] (2) if Distance = Near then Speed ​​= LS, if the minimum distance belongs to the Near fuzzy set, then the terminal speed level belongs to the LS fuzzy set;

[0081] (3) if Distance = Normal then Speed ​​= NS, if the minimum distance belongs to the Normal fuzzy set, then the terminal speed level belongs to the NS (medium speed) fuzzy set;

[0082] (4) if Distance = Far then Speed ​​= QS, if the minimum distance belongs to the Far (far) fuzzy set, then the terminal speed level belongs to the QS (fast) fuzzy set;

[0083] (5) if Distance = Very Far then Speed ​​= QS, if the minimum distance belongs to the Very Far (very far) fuzzy set, then the terminal speed level belongs to the QS (fast) fuzzy set.

[0084] The fuzzy control rule table can be obtained as shown in the following table:

[0085] IF Very Near Near Normal Far Very Far Then LS LS NS QS QS Weight 1 0.3 0.8 0.3 1

[0086] Among them, the weight in the fuzzy control rule table can be used to indicate the importance or priority of each rule. In the fuzzy reasoning process, the outputs of different rules can be weighted based on the weights to obtain the final fuzzy output. According to the above fuzzy control rules, it can be concluded that under fuzzy control, Figure 4 The mapping relationship between the distance level of the minimum distance and the terminal speed level is shown, wherein as the distance level of the minimum distance increases, the terminal speed level also increases. Further, the terminal speed planning result can be converted into a joint speed planning result, the corresponding terminal speed is obtained based on the terminal speed level, and the terminal speed is converted into a joint value speed, and the joint speed planning result can include, for any path point obtained by the plan, the joint value speed of each joint of the manipulator in the process of moving the end of the manipulator to the path point (moving from the current path point to the next path point).

[0087] In some examples, the following may be used: Figure 5 The speed regulation process of the robot arm based on fuzzy control shown includes: before the robot arm moves, the robot arm can be path planned, for example, the path planning of the robot arm can determine multiple feasible motion trajectories of the end movement of the robot arm when the robot arm reaches the target position. Further, the robot arm collision detection can be performed on the multiple feasible motion trajectories planned to ensure that the robot arm will not collide with other objects or patients during the movement based on the planned motion trajectory. By performing collision detection on the robot arm, the minimum distance between the robot arm and the patient on the motion trajectory can be solved to ensure the safety of the patient, and finally obtain the target motion trajectory without collision. The fuzzy control model can be used to process the minimum distance between the robot arm and the patient corresponding to each path point on the target motion trajectory to obtain the end speed planning result of the robot arm, and the end speed planning result can include the end speed level; based on the end speed planning result, the conversion can be performed to obtain the joint speed planning result of the robot arm, and the movement of the robot arm can be controlled based on the joint speed planning result. Among them, the fuzzy control model is based on a series of fuzzy rules, which can map input variables (for example, the minimum distance between the robot arm and the patient) to output variables (for example, the terminal speed level); determine the terminal speed of the robot arm based on the terminal speed level, and convert the terminal speed into the joint value speed of each joint of the robot arm, so as to realize the control of the joint movement. By adjusting the joint value speed, dynamic adjustment and optimization can be performed according to the real-time movement of the robot arm. After completing the path planning and the allocation of the joint value speed, each path segment and its corresponding joint value speed can be sent to the robot arm. The robot arm can perform corresponding movements according to the planned path and the corresponding joint value speed according to the received instructions. Through the above steps, the robot arm can realize accurate movement according to the requirements of the path and speed, and avoid collisions considering the safety of the patient.

[0088] In some examples, a target motion trajectory of a robot arm from the current joint angle (e.g., configuration A) to the next joint angle (e.g., configuration B) without colliding with the environment can be planned, wherein collision detection can be implemented using a robot arm model and an environment model. The RRT (Rapid-exploration Random Tree) method can be used to randomly scatter points in space, then give it a direction from the starting point, which points to the end point, and then find the next non-collision point closest to the current distance among these random points as the next path point, and so on, and the above operation is also performed at the end point to obtain the target motion trajectory. Among them, the minimum distance between each path point in the target motion trajectory and the environment model can be obtained by using a PQP (Proximity Query Package) algorithm, the spatial model can be stored in a triangular facet data structure, the PQP algorithm can be used to calculate the vertex distance between two triangles, use a KD-Tree (K-Dimensional Tree) to store these vertices, and then find the closest distance between the two groups of models through a nearest neighbor search. The robot arm model may include a joint 1 model, a joint 2 model, ... and a joint 6 model. The environment model may include a trolley model, a patient head model, and a head frame model generated by environment perception.

[0089] It should be noted that with the development of surgical robot technology, human-machine collaborative technology has emerged, and the robotic arm often needs to switch between force control mode and position mode. For example, you can switch between the force control mode and position mode of the robotic arm by clicking the operation button on the control panel. The above operations are relatively cumbersome and easily affect the smoothness of the surgical process, increasing unnecessary operation time. The embodiment of the present application provides a robotic arm control method that is convenient for switching the motion mode of the robotic arm, which can improve the efficiency of the surgical process. The robotic arm control method provided in the embodiment of the present application can be applied to a processor in a surgical robot system; the surgical robot system also includes a robotic arm, the robotic arm is provided with an end tool, and the gripping part of the end tool is provided with a sensor, wherein the processor can be connected to the robotic arm to control the movement of the robotic arm.

[0090] In one embodiment, the present application provides a method for controlling a robot arm, wherein the robot arm is provided with an end tool; a sensor is provided at a gripping portion of the end tool; Figure 6 As shown, the method includes:

[0091] Step 610, obtaining a sensing signal output by a sensor;

[0092] Specifically, the end tool of the robot arm may include a registration probe or other end tool disposed at the end of the robot arm; Figure 7 As shown, the registration probe 70 of the robot arm is an end tool for contact space registration. The registration probe 70 can be used to contact the space marking points to obtain the positions of the marking points in the base coordinate system of the robot arm during the space registration process. Usually, after obtaining the positions of these marking points in the base coordinate system of the robot arm, the positions of these marking points in the image coordinate system can be combined to obtain the spatial transformation relationship between the base coordinate system of the robot arm and the image coordinate system, wherein the image coordinate system can include a CT (Computed Tomography) image coordinate system or an MR (Magnetic Resonance) image coordinate system. The end tool of the robot arm can be provided with a sensor, for example, a tactile sensor, wherein the tactile sensor can include a strain tactile sensor, a piezoresistive tactile sensor, a capacitive tactile sensor or a piezoelectric tactile sensor, etc. The sensor is set at the holding part of the end tool. For example, according to the specific type of tactile sensor, the tactile sensor is set on the surface or inside of the end tool. The sensing signal output by the sensor can be used as a basis for setting or switching the motion mode of the robot arm. For example, it can be determined based on the sensing signal whether the end tool is currently touched by the operator. There is no need for the operator to set or switch the motion mode of the robot arm through other manual operation methods, which saves operation time and improves the efficiency of robot arm control.

[0093] Step 620: Control the robot arm to switch between the force control mode and the position control mode according to the sensing signal.

[0094] Specifically, the sensing signal can be processed, the scene that needs to switch the motion mode of the robot arm can be identified, and the motion mode of the robot arm can be controlled in time to switch between the force control mode and the position control mode. Among them, in the force control mode, the movement of the robot arm is controlled based on the external force or torque. The robot arm can sense the size and direction of the external force and adjust its own movement according to these forces. In the force control mode, the robot arm can perform tasks that interact with the external environment, such as assembly, collaborative operation, etc. In the position control mode, the movement of the robot arm is controlled based on the set target position or trajectory. The robot arm can achieve the required position control by controlling the joint angle or the position of the end of the robot arm. In the position control mode, the robot arm can perform tasks that require precise positioning and trajectory tracking, such as point-to-point operation, path planning, etc. By controlling the robot arm to switch between force control mode and position control mode according to the sensing signal, the human-machine collaboration function can be realized, which simplifies the interaction process between the operator and the robot arm. When the operator wants to control the robot arm to enter the force control mode, the operator can change the sensing signal output by the sensor by holding the end tool, so that the robot arm automatically switches the motion mode to the force control mode according to the change of the sensing signal; if the robot arm has unexpected movement in the force control mode, the operator can change the sensing signal output by the sensor by letting go, so that the robot arm can immediately stop the current movement of the robot arm according to the change of the sensing signal, thereby improving the safety of the robot arm during movement.

[0095] In some examples, the sensing signal can be used to characterize whether the end tool is currently touched by the operator. The sensing signal can be processed to detect whether the end tool is currently touched by the operator. By detecting the contact status of the end tool, the scenario where the motion mode of the robot arm needs to be switched can be identified. For example, when it is identified based on the sensing signal that the end tool is touched by the operator, the motion mode of the robot arm can be promptly converted to a force control mode controlled by the operator to improve the operational smoothness of the robot arm; when it is identified based on the sensing signal that the end tool is not touched by the operator, the robot arm can be controlled to a position control mode.

[0096] In some examples, the sensor may include Figure 7 The tactile sensor 710 shown in the figure can be a flexible tactile sensor integrated in the surface of the end tool. When the operator grasps the part of the end tool with the tactile sensor 710, it causes the change of the sensing signal output by the tactile sensor 710. The sensing signal is transmitted to the control system, and the movement mode of the robot arm can be selected to improve the operation smoothness.

[0097] In some examples, the sensor can be implemented as a force / torque sensor or a pressure sensor, which can sense the external force and convert the external force into an electrical signal, and then associate the electrical signal with the movement of the robotic arm through control algorithms such as impedance control and torque control. The robotic arm may also include a position sensor (such as an encoder). For example, the joint angle of the robotic arm or the end position of the robotic arm is obtained through the output signal of the encoder. For example, a control algorithm such as PID (Proportion Integration Differentiation) control and motion planning algorithm is used to compare the set target position or trajectory of the robotic arm with the current position of the robotic arm, and calculate the corresponding control instructions to drive the robotic arm to achieve the required position control.

[0098] In the robot arm control method of this embodiment, the robot arm is provided with an end tool; the gripping part of the end tool is provided with a sensor, and the sensing signal output by the sensor is obtained; according to the sensing signal, the robot arm is controlled to switch between the force control mode and the position control mode, so as to realize the human-machine collaboration function, simplify the interaction process between the operator and the robot arm, and do not need the operator to set or switch the movement mode of the robot arm through other manual operation methods, thus saving operation time and improving the efficiency of the robot arm control. If unexpected movement occurs in the force control mode of the robot arm, the operator can immediately stop the current movement of the robot arm by letting go, which improves the safety and adjustability of the robot arm during movement.

[0099] In one embodiment, if Figure 8 As shown, according to the sensing signal, the robot arm is controlled to switch between the force control mode and the position control mode, including:

[0100] Step 810: If the sensing signal meets the first threshold range, the robot arm is controlled to enter the force control mode; the first threshold range is used to indicate that the end tool is in contact with the target;

[0101] Step 820: If the sensing signal meets the second threshold range, the robot arm is controlled to enter the position control mode; the second threshold range is used to indicate that the end tool is in a non-contact target state.

[0102] Specifically, if the sensing signal meets the first threshold range, it can be indicated that the end tool is currently in a contact target state, and the motion mode of the robot arm can be set to a force control mode, wherein the contact target state can represent that the end tool is in contact with the operator, for example, the operator can hold or grasp the gripping part of the end tool, and then the sensing signal output by the sensor can be used to detect that the contact state of the end tool is a contact target state, and the robot arm is controlled to enter a force control mode, and the motion of the robot arm in the force control mode can be controlled based on the force or torque applied by the outside (for example, the operator). If the sensing signal meets the second threshold range, it can be indicated that the end tool is currently in a non-contact target state, and the motion mode of the robot arm can be set to a position control mode, and the motion of the robot arm in the position control mode can be controlled based on a set target position or trajectory.

[0103] In some examples, such as Fig. 9 As shown, the surgical robot system may include a tactile sensor, a robotic arm, a hardware board connected to the tactile sensor, and a processor connecting the hardware board and the robotic arm. The control system may be integrated in the processor; the hardware board may obtain the sensing signal output by the tactile sensor and transmit it to the control system. The control system determines whether the end tool contacts the operator. If it is detected that the end tool does not contact the operator, the movement mode of the robotic arm may be adjusted to a position control mode; if it is detected that the end tool contacts the operator, the movement mode of the robotic arm may be adjusted to a force control mode, and the robotic arm may move according to the currently set movement mode.

[0104] The above process of performing velocity planning on the end of the robot arm may be performed in any mode of the robot arm, for example, a force control mode or a position control mode.

[0105] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0106] Based on the same inventive concept, the embodiment of the present application also provides a robot arm control device for implementing the robot arm control method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more robot arm control device embodiments provided below can refer to the limitations of the robot arm control method above, and will not be repeated here.

[0107] In one embodiment, Fig.10 As shown, a robot arm control device is provided; the device comprises:

[0108] The target path planning module 1010 is used to obtain the target path planning result of the robot arm; the target path planning result includes the detection distance between the robot arm and the target object at each path point;

[0109] The terminal speed planning module 1020 is used to process the target path planning result and obtain the terminal speed planning result of the robot arm;

[0110] The robot arm motion control module 1030 is used to control the robot arm motion based on the terminal velocity planning result.

[0111] In one embodiment, the target path planning module 1010 is further used to perform path planning for the robot arm to obtain an initial path planning result of the robot arm;

[0112] Based on the initial path planning results, collision detection is performed on the robotic arm to obtain the target path planning results.

[0113] In one embodiment, the robot arm motion control module 1030 is further used to obtain the joint velocity planning result of the robot arm based on the terminal velocity planning result;

[0114] Control the movement of the robotic arm based on the target path planning results and joint speed planning results.

[0115] In one embodiment, the terminal speed planning module 1020 is also used to process each detection distance using a preset model to obtain a terminal speed planning result; the terminal speed planning result includes the terminal speed level of the robotic arm at each path point; the preset model includes a mapping relationship between the detection distance and the terminal speed level.

[0116] In one embodiment, the robot arm is provided with an end tool; a gripping portion of the end tool is provided with a sensor; and the device further comprises:

[0117] A signal acquisition module is used to acquire the sensing signal output by the sensor;

[0118] The mode switching module is used to control the robot arm to switch between the force control mode and the position control mode according to the sensing signal.

[0119] In one embodiment, according to the sensing signal, the mode switching module is further used to control the robot arm to enter the force control mode if the sensing signal satisfies a first threshold range; the first threshold range is used to indicate that the end tool is in a contact target state;

[0120] If the sensing signal meets the second threshold range, the robot arm is controlled to enter the position control mode; the second threshold range is used to indicate that the end tool is in a non-contact target state.

[0121] Each module in the above-mentioned robot arm control device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0122] In one embodiment, a surgical robot system is provided, the system includes a robotic arm; the system also includes a memory and a processor, the memory stores a computer program, the processor is connected to the robotic arm, and the processor implements the steps of the above method when executing the computer program.

[0123] In one embodiment, a surgical robot system is provided, the system includes a robotic arm, the robotic arm is provided with an end tool, and the gripping part of the end tool is provided with a sensor; the system also includes a memory and a processor, the memory stores a computer program, the processor is connected to the sensor and the robotic arm, and the processor implements the steps of the above method when executing the computer program.

[0124] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for controlling a robotic arm is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0125] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0127] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above method when executed by a processor.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A robot arm control method, characterized in that: The method comprises: Obtaining a target path planning result of the robotic arm; the target path planning result includes a detection distance between the robotic arm and the target object at each path point; Processing the target path planning result to obtain the terminal speed planning result of the robotic arm; Based on the terminal velocity planning result, the movement of the robot arm is controlled.

2. The method according to claim 1, characterized in that The obtaining of the target path planning result of the robot arm includes: Performing path planning on the robotic arm to obtain an initial path planning result of the robotic arm; Based on the initial path planning result, collision detection is performed on the robotic arm to obtain the target path planning result.

3. The method according to claim 1, characterized in that: The controlling the movement of the robot arm based on the terminal velocity planning result comprises: Based on the terminal velocity planning result, obtaining the joint velocity planning result of the robotic arm; The movement of the robotic arm is controlled according to the target path planning result and the joint speed planning result.

4. The method according to claim 1, characterized in that The processing of the target path planning result to obtain the terminal speed planning result of the robot arm includes: A preset model is used to process each of the detection distances to obtain the terminal speed planning result; the terminal speed planning result includes the terminal speed level of the robotic arm at each of the path points; the preset model includes a mapping relationship between the detection distance and the terminal speed level.

5. The method according to claim 4, characterized in that The detection distance includes a minimum distance between each joint of the robot arm and the target object.

6. The method according to claim 1, characterized in that The robot arm is provided with an end tool; a gripping portion of the end tool is provided with a sensor; and the method further comprises: Acquire a sensing signal output by the sensor; According to the sensing signal, the robot arm is controlled to switch between a force control mode and a position control mode.

7. The method according to claim 6, characterized in that The step of controlling the robot arm to switch between a force control mode and a position control mode according to the sensing signal comprises: If the sensing signal satisfies a first threshold range, the robot arm is controlled to enter the force control mode; the first threshold range is used to indicate that the end tool is in a contact target state; If the sensing signal satisfies a second threshold range, the robot arm is controlled to enter the position control mode; the second threshold range is used to indicate that the end tool is in a non-contact target state.

8. A robot arm control device, characterized in that: The device comprises: A target path planning module is used to obtain a target path planning result of the robotic arm; the target path planning result includes a detection distance between the robotic arm and the target object at each path point; A terminal speed planning module is used to process the target path planning result to obtain the terminal speed planning result of the robot arm; A robot arm motion control module is used to control the movement of the robot arm based on the terminal velocity planning result.

9. A surgical robot system, characterized in that: The system includes a robotic arm; the system also includes a memory and a processor, the memory stores a computer program, the processor is connected to the robotic arm, and the processor implements the steps of any one of claims 1 to 5 when executing the computer program.

10. A surgical robot system, characterized in that: The system includes a robotic arm, the robotic arm is provided with an end tool, and the gripping part of the end tool is provided with a sensor; the system also includes a memory and a processor, the memory stores a computer program, the processor connects the sensor and the robotic arm, and when the processor executes the computer program, the steps of the method described in any one of claims 6 to 7 are implemented.

Images