Robotic arm control method, device and surgical robotic system

Abstract

The application relates to a mechanical arm control method, device and surgical robot system. The method comprises the following steps: acquiring a target path planning result of a mechanical arm; the target path planning result comprises a detection distance of the mechanical arm and a target object at each path point, can ensure that the mechanical arm does not collide with an obstacle in a movement process, and guarantees the safety of the movement of the mechanical arm; processing the target path planning result to obtain an end speed planning result of the mechanical arm as a basis for controlling the movement of the mechanical arm, and then realizing variable speed adjustment of the mechanical arm; based on the end speed planning result, the movement of the mechanical arm is controlled, the joint movement speed of the mechanical arm can be reduced when approaching the target object, and the joint movement speed of the mechanical arm can be increased when moving away from the target object. The method can improve the movement efficiency of the mechanical arm and the adjustability of the mechanical arm control.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a robotic arm control method, device and surgical robot system. Background Technology

[0002] With the development of surgical robot technology, a constant speed is typically maintained during the autonomous movement of the robotic arm. If the robotic arm moves too fast, it increases safety risks during surgery; if it moves too slowly, it prolongs the operation time. The aforementioned methods for adjusting the robotic arm's movement speed are relatively simplistic and can easily affect the safety of the surgical procedure or add unnecessary operation time.

[0003] However, current robotic arm control methods or traditional approaches suffer from poor motion adjustability. Summary of the Invention

[0004] Therefore, it is necessary to provide an adjustable robotic arm control method, device, and surgical robot system to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a robotic arm control method, the method comprising:

[0006] Obtain the target path planning results of the robotic arm; the target path planning results include the detection distance between the robotic arm and the target object at each path point;

[0007] The target path planning results are processed to obtain the end-effector velocity planning results of the robotic arm;

[0008] The movement of the robotic arm is controlled based on the end-effector velocity planning results.

[0009] In one embodiment, obtaining the target path planning result of the robotic arm includes:

[0010] Perform path planning on the robotic arm to obtain the initial path planning result;

[0011] Based on the initial path planning results, collision detection is performed on the robotic arm to obtain the target path planning results.

[0012] In one embodiment, controlling the movement of the robotic arm based on the end-effector velocity planning results includes:

[0013] Based on the end-effector velocity planning results, the joint velocity planning results of the robotic arm are obtained;

[0014] The movement of the robotic arm is controlled based on the target path planning results and joint velocity planning results.

[0015] In one embodiment, the target path planning result is processed to obtain the end-effector velocity planning result of the robotic arm, including:

[0016] The detection distances are processed using a preset model to obtain the end-effector speed planning results. The end-effector speed planning results include the end-effector speed levels of the robotic arm at each path point. The preset model includes the mapping relationship between the detection distance and the end-effector speed levels.

[0017] In one embodiment, the detection distance includes the minimum distance among the distances between each joint of the robotic arm and the target object.

[0018] In one embodiment, the robotic arm is equipped with an end effector; the gripping portion of the end effector is equipped with a sensor; the method further includes:

[0019] Acquire the sensing signal output by the sensor;

[0020] Based on the sensor signals, the robotic arm switches between force control mode and position control mode.

[0021] In one embodiment, controlling the robotic arm to switch between force control mode and position control mode based on a sensing signal includes:

[0022] If the sensing signal meets the first threshold range, the robotic arm is controlled to enter the force control mode; the first threshold range is used to characterize the end effector as being in contact with the target.

[0023] If the sensing signal meets the second threshold range, the robotic arm is controlled to enter the position control mode; the second threshold range is used to characterize the end effector as not in contact with the target.

[0024] Secondly, this application provides a robotic arm control device, the device comprising:

[0025] The target path planning module is used to obtain the target path planning results of the robotic arm; the target path planning results include the detection distance between the robotic arm and the target object at each path point;

[0026] The end-effector velocity planning module is used to process the target path planning results and obtain the end-effector velocity planning results of the robotic arm.

[0027] The robotic arm motion control module is used to control the movement of the robotic arm based on the end-effector velocity planning results.

[0028] Thirdly, this application provides a surgical robot system, which includes a robotic arm; the system also includes a memory and a processor, the memory storing a computer program, the processor being connected to the robotic arm, and the processor executing the computer program to implement the steps of the above-described method.

[0029] Fourthly, this application provides a surgical robot system, which includes a robotic arm with an end effector and a sensor at the gripping part of the end effector; the system also includes a memory and a processor, the memory storing a computer program, the processor being connected to the sensor and the robotic arm, and the processor executing the computer program to implement the steps of the above-described method.

[0030] The aforementioned robotic arm control method, device, and surgical robot system, through obtaining the target path planning results of the robotic arm, including the detection distance between the robotic arm and the target object at each path point, can ensure that the robotic arm will not collide with obstacles during movement, thereby guaranteeing the safety of the robotic arm's movement; processing the target path planning results to obtain the end-effector velocity planning results of the robotic arm, which serves as the basis for controlling the movement of the robotic arm, thereby realizing the variable speed adjustment of the robotic arm; based on the end-effector velocity planning results, controlling the movement of the robotic arm can achieve the effect of reducing the joint movement speed of the robotic arm when approaching the target object and increasing the joint movement speed of the robotic arm when moving away from the target object, thereby improving the movement efficiency and adjustability of the robotic arm control. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating a robotic arm control method in one embodiment;

[0032] Figure 2 This is a flowchart illustrating the robotic arm control steps in one embodiment;

[0033] Figure 3 This is a flowchart illustrating the robotic arm control steps in another embodiment;

[0034] Figure 4 This is a schematic diagram illustrating the mapping relationship between the minimum distance level and the end velocity level in one embodiment;

[0035] Figure 5 This is a flowchart illustrating the robotic arm control steps in yet another example;

[0036] Figure 6 This is a flowchart illustrating the robotic arm control method in another embodiment;

[0037] Figure 7 This is a schematic diagram of a probe registration in one embodiment;

[0038] Figure 8 This is a flowchart illustrating the robotic arm control steps in another embodiment;

[0039] Figure 9 This is a flowchart illustrating the control steps of a robotic arm in an example.

[0040] Figure 10This is a structural block diagram of a robotic arm control device in one embodiment;

[0041] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] The robotic arm control method provided in this application can be applied to the processor in a surgical robot system; the surgical robot system also includes a robotic arm, wherein the processor can be connected to the robotic arm to control the movement of the robotic arm.

[0044] In one embodiment, such as Figure 1 As shown, the method includes:

[0045] Step 110: Obtain the target path planning results of the robotic arm; the target path planning results include the detection distance between the robotic arm and the target object at each path point;

[0046] Specifically, the robotic arm can be equipped with sensors to perceive its surrounding environment, such as at least one of LiDAR and vision sensors, to provide information about the distance, position, and shape between the robotic arm's end effector and surrounding objects. Furthermore, the data acquired by the sensors can be used to build a model of the robotic arm's surrounding environment, which can include information such as the position, shape, and size of obstacles. Path planning for the robotic arm yields a target path planning result, which can include the detected distances between the robotic arm and the target object at each path point. This results in multiple collision-detected path points. If the robotic arm's end effector moves sequentially to each path point, it can ensure that it does not collide with obstacles in the surrounding environment. During the path planning process, the detected distances between the robotic arm and the target object at each path point can be obtained by performing collision detection on the robotic arm based on the surrounding environment model. Target objects can include obstacles in the environment, as well as the target objects of the actuators mounted on the robotic arm's end effector, such as patients. By performing collision detection on the robotic arm during path planning, the resulting target path planning can include collision detection results. For example, a collision detection algorithm can be used to compare the planned trajectory of the robotic arm with the positions of obstacles in the surrounding environment to determine if there is a risk of collision. This yields collision detection results for each joint of the robotic arm, which can include the distances between each joint and obstacles in the surrounding environment. The detection distance between the robotic arm and the target object can also be determined based on these collision detection results. By obtaining the target path planning results for the robotic arm, it can be ensured that the robotic arm will not collide with obstacles during its movement, thus guaranteeing the safety of its operation.

[0047] In some examples, collision detection algorithms may include geometry-based methods (such as bounding box, convex hull, etc.) or distance-based methods (such as nearest distance, minimum distance, etc.).

[0048] Step 120: Process the target path planning results to obtain the end-effector velocity planning results of the robotic arm;

[0049] Specifically, the target path planning results can be processed to rationally plan the end-effector velocity based on the detection distance between the robotic arm and the target object at each path point, resulting in the end-effector velocity planning result. This end-effector velocity planning result can include the linear velocity of the robotic arm's end effector. The end-effector velocity planning result obtained based on each detection distance in the target path planning result can serve as the basis for controlling the robotic arm's movement, thereby achieving variable speed adjustment of the robotic arm.

[0050] Step 130: Based on the end-effector velocity planning results, control the movement of the robotic arm.

[0051] Specifically, based on the end-effector velocity planning results, the speed of each joint of the robotic arm can be adaptively adjusted to control the movement of the robotic arm. This can achieve the effect of reducing the joint movement speed of the robotic arm when approaching the target object and increasing the joint movement speed of the robotic arm when moving away from the target object, thereby improving the movement efficiency of the robotic arm and the adjustability of the robotic arm control.

[0052] This application embodiment obtains the target path planning results of the robotic arm, including the detection distance between the robotic arm and the target object at each path point. This ensures that the robotic arm will not collide with obstacles during movement, thereby guaranteeing the safety of the robotic arm's movement. The target path planning results are processed to obtain the end-effector velocity planning results of the robotic arm, which serve as the basis for controlling the robotic arm's movement, thus achieving variable speed adjustment of the robotic arm. Based on the end-effector velocity planning results, controlling the robotic arm's movement can achieve the effect of reducing the joint movement speed of the robotic arm when approaching the target object and increasing the joint movement speed of the robotic arm when moving away from the target object, thereby improving the movement efficiency and adjustability of the robotic arm control.

[0053] In one embodiment, such as Figure 2 As shown, the target path planning results of the robotic arm are obtained, including:

[0054] Step 210: Perform path planning on the robotic arm to obtain the initial path planning result of the robotic arm;

[0055] Step 220: Based on the initial path planning results, perform collision detection on the robotic arm to obtain the target path planning results.

[0056] Specifically, path planning can be performed on the robotic arm. For example, the starting point and target point of the robotic arm's end effector can be obtained. Using a corresponding path planning algorithm, the motion trajectory of the end effector can be determined, enabling it to move from the starting point to the target point. The initial path planning result can include multiple motion trajectories for the end effector, each corresponding to multiple path points. Based on the initial path planning result, collision detection can be performed on the robotic arm to avoid collision risks, ensuring that the robotic arm moves according to the obtained target path planning result without colliding with surrounding obstacles. The target path planning result can include collision-free motion trajectories, which can correspond to multiple path points, indicating to the end effector of the robotic arm to move sequentially to each path point.

[0057] In one embodiment, such as Figure 3 As shown, based on the end-effector velocity planning results, the movement of the robotic arm is controlled, including:

[0058] Step 310: Based on the end-effector velocity planning results, obtain the joint velocity planning results of the robotic arm;

[0059] Step 320: Control the movement of the robotic arm based on the target path planning results and joint velocity planning results.

[0060] Specifically, the end-effector velocity planning results can be converted into joint velocity planning results for the robotic arm. For example, for any planned path point, an appropriate algorithm can be used to convert the end-effector's linear velocity from the current path point to that path point, obtaining the joint velocity values ​​required for each joint of the robotic arm during this process. These joint velocity values ​​can include the angle or position change of each joint per unit time. Furthermore, based on the target path planning results and joint velocity planning results, the joint velocity values ​​corresponding to each planned path point can be sent to the robotic arm to control it to move according to the planned trajectory and joint velocity values ​​for each path point. By controlling the robotic arm's movement based on the target path planning results and joint velocity planning results, the robotic arm can change speeds to avoid obstacles during movement, improving the safety and effectiveness of its movement.

[0061] In one embodiment, the target path planning result is processed to obtain the end-effector velocity planning result of the robotic arm, including:

[0062] The detection distances are processed using a preset model to obtain the end-effector speed planning results. The end-effector speed planning results include the end-effector speed levels of the robotic arm at each path point. The preset model includes the mapping relationship between the detection distance and the end-effector speed levels.

[0063] Specifically, the target path planning result can include multiple path points on at least one motion trajectory. For any path point, the distances between each joint of the robotic arm and the target object when the end effector of the robotic arm reaches that path point can be obtained. This allows for the determination of the detection distance between the robotic arm and the target object. For example, the minimum distance between each joint of the robotic arm and the target object can be determined as the detection distance. The target object can include obstacles in the environment or the target object of the actuator installed at the end effector of the robotic arm, such as a patient. By establishing a preset model, the mapping relationship between the detection distance and the end effector speed level can be determined. Each end effector speed level can correspond to a specific end effector speed percentage. The end effector speed percentage can describe the ratio of the actual movement speed of the robotic arm's end effector to its maximum movement speed. Then, by using the established preset model to process the detection distance between the robotic arm and the target object in the target path planning result, the end effector speed level and its corresponding end effector speed percentage can be obtained. The end effector speed percentage of the robotic arm can then be used as the target speed planning result. By detecting the distance between the robotic arm and the target object, and allocating a percentage of the robotic arm's end-effector velocity, the robotic arm's movement speed can be reduced when approaching the target object and increased when moving away from the target object. This saves time, increases efficiency, and improves the safety of the robotic arm's movement.

[0064] In one embodiment, the detection distance includes the minimum distance among the distances between each joint of the robotic arm and the target object.

[0065] Specifically, the target path planning result can include the distances between each joint of the robotic arm and the target object at each path point. Based on these distances, a detection distance can be determined, which can include the minimum distance among the distances between each joint of the robotic arm and the target object. A pre-defined model can include a mapping relationship between the minimum distance and the end-effector velocity level. By processing the minimum distance between the robotic arm and the target object in the target path planning result using this pre-defined model, the end-effector velocity level and its corresponding percentage can be obtained. This percentage of the end-effector velocity is then used as the target velocity planning result. By allocating a percentage of the end-effector velocity based on the minimum distance between the robotic arm and the target object, the robotic arm's movement speed can be reduced when approaching the target object and increased when moving away from it. This saves time, improves efficiency, and enhances the safety of the robotic arm's movement.

[0066] In some examples, the robotic arm may include joints 1, 2, 3, 4, 5, and 6. The minimum distance can be the minimum of the distances between each joint and the patient. For example, if the minimum distance between joint 1 and the patient is 60mm; the minimum distance between joint 2 and the patient is 30mm; the minimum distance between joint 3 and the patient is 40mm; the minimum distance between joint 4 and the patient is 35mm; the minimum distance between joint 5 and the patient is 50mm; and the minimum distance between joint 6 and the patient is 20mm, then the minimum distance can be determined to be 20mm.

[0067] In some examples, the preset model may include a neural network model, a fuzzy control model, etc. For the end-effector velocity percentage, the maximum speed of the robotic arm's end effector is Vmax, while the actual speed during movement is V. Therefore, the end-effector velocity percentage can be calculated using the following formula:

[0068] Terminal velocity percentage = (V / Vmax) * 100%

[0069] In the control of a robotic arm, the movement speed of the robotic arm can be controlled by adjusting the end-effector velocity percentage. A higher end-effector velocity percentage indicates that the end-effector is moving at a speed close to its maximum speed, which can achieve rapid movement; a lower end-effector velocity percentage indicates that the end-effector is moving at a slower speed, which can achieve more precise and slower movement.

[0070] In some examples, taking a fuzzy control model as an example, the fuzzy control model can be obtained as follows: take the minimum distance as the input variable and the end speed level as the output variable; establish fuzzy control rules between the input and output variables; and establish a fuzzy control model based on the fuzzy control rules.

[0071] Specifically, the minimum distance between the robotic arm and the target object can be used as the input variable of the fuzzy control model, and the end-effector speed level as the output variable. These variables are then fuzzified, transforming continuous values ​​into discrete fuzzy sets. For each input and output variable, the shape and membership function of its fuzzy set are determined to describe its degree of membership. Fuzzy control rules can be established between the input and output variables, defining how the input variables map to the output variables. These rules can be in the form of "IF-THEN," where the IF part represents the condition for the input variables, and the THEN part represents the result for the output variables. Fuzzy control rules can be determined through expert knowledge or training algorithms. After determining the fuzzy control rules, fuzzy inference can be used to determine the fuzzy result of the output variable. Fuzzy inference is based on rule matching and fuzzy logic operations, used to determine the fuzzy set and membership degree of the output variable. After fuzzy inference, the fuzzy output can be converted into a concrete actual value. Defuzzification is the process of converting the fuzzy result into a concrete output value. Common defuzzification methods include the maximum membership method and the centroid method. The membership functions of the input variables, the membership functions of the output variables, and the fuzzy rules of the fuzzy control model can be determined through expert knowledge, experience, or by using training algorithms (such as genetic algorithms, fuzzy clustering, etc.). Furthermore, the fuzzy control model can be adjusted and optimized according to actual conditions. Based on fuzzy control theory, the end-effector speed of the robotic arm's planned path can be controlled at different levels, achieving the effect of changing the robotic arm's movement speed according to the minimum distance between the robotic arm and the patient, thus reducing the robotic arm's operation time.

[0072] In some examples, the speed of the robotic arm (e.g., end effector linear velocity) can be divided into three fuzzy sets: LS (slow), NS (medium), and QS (fast), with corresponding speed ranges divided into five speed levels: 0, 1, 2, 3, and 4. See the table below:

[0073]

[0074] It should be noted that in fuzzy control, membership degree describes the degree to which a definite quantity belongs to a fuzzy linguistic variable (fuzzy set). The membership degree typically ranges from 0 to 1, indicating the strength of the relationship between the definite quantity and the fuzzy set. The larger the membership degree, the higher the degree of matching between the definite quantity and the fuzzy set. The five speed levels corresponding to the end effector speed of the robotic arm are definite quantities. Using the rule table mentioned above, the speed of the robotic arm can be converted into three fuzzy sets, thus achieving the fuzzification of the output variable. Specifically, speed level 0 has a membership degree of 1 for the LS (slow) fuzzy set, and a membership degree of 0 for both the NS (medium) and QS (fast) fuzzy sets; speed level 1 has a membership degree of 0.5 for both the LS (slow) and NS (medium) fuzzy sets, and a membership degree of 0 for the QS (fast) fuzzy set; speed level 2 has a membership degree of 0 for both the LS (slow) and QS (fast) fuzzy sets, and a membership degree of 1 for the NS (medium) fuzzy set; speed level 3 has a membership degree of 0 for the LS (slow) fuzzy set, and a membership degree of 0.5 for both the NS (medium) and QS (fast) fuzzy sets; speed level 4 has a membership degree of 0 for both the LS (slow) and NS (medium) fuzzy sets, and a membership degree of 1 for the QS (fast) fuzzy set. All of these membership degrees can be determined based on relevant empirical data.

[0075] Collision detection of the robotic arm can determine the minimum distance between the robotic arm and the patient during execution. This minimum distance is divided into five fuzzy sets: Very Near, Near, Normal, Far, and Very Far, with corresponding distance ranges of five levels: -2, -1, 0, 1, and 2. See the table below:

[0076]

[0077] The five distance levels corresponding to the minimum distance are fixed quantities. Using the rule table mentioned above, these distance levels can be converted into five fuzzy sets, thus fuzzifying the input variable. Specifically, distance level -2 has a membership degree of 1 for the Very Near fuzzy set, and a membership degree of 0 for all other fuzzy sets; distance level -1 has a membership degree of 1 for the Near fuzzy set, and a membership degree of 0 for all other fuzzy sets; distance level 0 has a membership degree of 1 for the Normal fuzzy set, and a membership degree of 0 for all other fuzzy sets; distance level 1 has a membership degree of 1 for the Far fuzzy set, and a membership degree of 0 for all other fuzzy sets; distance level 2 has a membership degree of 1 for the Very Far fuzzy set, and a membership degree of 0 for all other fuzzy sets. All of these membership degrees can be determined based on relevant empirical data.

[0078] Fuzzy control rules can be established based on empirical data and described in the following form:

[0079] (1) If Distance = Very Near then Speed ​​= LS, if the minimum distance belongs to the Very Near fuzzy set, then the final speed level belongs to the LS fuzzy set.

[0080] (2) If Distance = Near then Speed ​​= LS, if the minimum distance belongs to the Near fuzzy set, then the speed level of the end belongs to the LS fuzzy set;

[0081] (3) If Distance = Normal then Speed ​​= NS, if the minimum distance belongs to the Normal fuzzy set, then the end speed level belongs to the NS fuzzy set.

[0082] (4) If Distance = Far then Speed ​​= QS, if the minimum distance belongs to the Far (far) fuzzy set, then the end speed level belongs to the QS (fast) fuzzy set;

[0083] (5) If Distance = Very Far then Speed ​​= QS, if the minimum distance belongs to the Very Far fuzzy set, then the final speed level belongs to the QS fuzzy set.

[0084] The fuzzy control rule table shown below can be obtained:

[0085] Then LS LS NS QS QS Weight 1 0.3 0.8 0.3 1

[0086] The weights in the fuzzy control rule table represent the importance or priority of each rule. During fuzzy inference, the outputs of different rules can be weighted based on these weights to obtain the final fuzzy output. Based on the aforementioned fuzzy control rules, it can be concluded that under fuzzy control... Figure 4 The diagram illustrates the mapping relationship between the minimum distance level and the end-effector velocity level, where the end-effector velocity level increases as the minimum distance level increases. Furthermore, the end-effector velocity planning results can be converted into joint velocity planning results. Based on the end-effector velocity level, the corresponding end-effector velocity is obtained, and the end-effector velocity is converted into joint value velocities. The joint velocity planning results can include, for any planned path point, the joint value velocities of each joint of the robotic arm during the process of moving the end-effector to that path point (from the current path point to the next path point).

[0087] In some examples, the following can be used: Figure 5 The speed adjustment process of the robotic arm based on fuzzy control, as shown, includes: Before the robotic arm moves, path planning can be performed. For example, path planning can determine multiple feasible motion trajectories for the end effector of the robotic arm when it reaches the target position. Further, collision detection can be performed on the planned feasible motion trajectories to ensure that the robotic arm does not collide with other objects or the patient during its movement based on the planned trajectory. By performing collision detection, the minimum distance between the robotic arm and the patient on the motion trajectory can be calculated to ensure the patient's safety, ultimately obtaining the target motion trajectory that will not result in a collision. Using a fuzzy control model, the minimum distance between the robotic arm and the patient corresponding to each path point on the target motion trajectory can be processed to obtain the end effector speed planning result, which can include the end effector speed level. Based on the end effector speed planning result, the joint speed planning result of the robotic arm can be obtained, and the movement of the robotic arm can be controlled based on the joint speed planning result. The fuzzy control model, based on a series of fuzzy rules, maps input variables (e.g., the minimum distance between the robotic arm and the patient) to output variables (e.g., the end-effector velocity level). The end-effector velocity is determined based on the end-effector velocity level and converted into joint value velocities for each joint, thus controlling joint movement. By adjusting the joint value velocities, dynamic adjustment and optimization can be performed based on the real-time movement of the robotic arm. After path planning and joint value velocity allocation, each path segment and its corresponding joint value velocity are sent to the robotic arm. The robotic arm can then perform corresponding movements according to the planned path and corresponding joint value velocities based on the received instructions. Through these steps, the robotic arm can achieve accurate movement according to path and speed requirements while avoiding collisions to ensure patient safety.

[0088] In some examples, a target motion trajectory can be planned for the robotic arm to move from the current joint angle (e.g., configuration A) to the next joint angle (e.g., configuration B) without colliding with the environment. Collision detection can be implemented using both the robotic arm model and the environment model. A Rapid-exploration Random Tree (RRT) method can be used. Points are randomly scattered in space, and a direction is given from the starting point to the ending point. The next non-collision point with the closest distance to the current point is then found as the next path point, and so on. The same operation is performed at the ending point to obtain the target motion trajectory. The minimum distance between each path point in the target motion trajectory and the environment model can be determined using the Proximity Query Package (PQP) algorithm. The spatial model can be stored as a triangular facet data structure. The PQP algorithm can be used to calculate the vertex distance between two triangles. These vertices are stored using a K-Dimensional Tree (KD-Tree), and the closest distance between the two sets of models is found through nearest neighbor search. The robotic arm model can include joint 1 model, joint 2 model, ... and joint 6 model. The environment model can include trolley model, patient head model, and head frame model generated by environment perception, etc.

[0089] It should be noted that with the development of surgical robot technology, human-machine collaboration technology has emerged, and robotic arms often need to switch between force control mode and position mode. For example, this can be done by clicking an operation button on the control panel. This operation is cumbersome and can easily affect the smoothness of the surgical procedure, increasing unnecessary operation time. This application provides a robotic arm control method that facilitates switching the movement mode of the robotic arm, improving the efficiency of the surgical procedure. The robotic arm control method provided in this application can be applied to the processor in a surgical robot system; the surgical robot system also includes a robotic arm, which has an end effector. The end effector's gripping portion has sensors, and the processor can be connected to the robotic arm to control its movement.

[0090] In one embodiment, this application provides a robotic arm control method, wherein the robotic arm is equipped with an end effector; the gripping portion of the end effector is equipped with a sensor; such as Figure 6 As shown, the method includes:

[0091] Step 610: Acquire the sensing signal output by the sensor;

[0092] Specifically, the end effector of a robotic arm may include a registration probe or other end effector located at the end of the robotic arm; such as Figure 7 As shown, the registration probe 70 of the robotic arm is an end effector for contact-based spatial registration. During spatial registration, the registration probe 70 contacts spatial markers to obtain their positions in the robotic arm's base coordinate system. Typically, after obtaining the positions of these markers in the robotic arm's base coordinate system, the spatial transformation relationship between the robotic arm's base coordinate system and the image coordinate system can be obtained by combining these positions with those in the image coordinate system. The image coordinate system can include a CT (Computed Tomography) image coordinate system or an MR (Magnetic Resonance) image coordinate system. The end effector of the robotic arm can be equipped with sensors, such as tactile sensors, which can include strain gauge tactile sensors, piezoresistive tactile sensors, capacitive tactile sensors, or piezoelectric tactile sensors, etc. By placing sensors on the gripping part of the end effector, for example, depending on the specific type of tactile sensor, tactile sensors can be placed on the surface or inside the end effector. The sensor output signals can be used as the basis for setting or switching the movement mode of the robotic arm. For example, the operator can determine whether the end effector is currently in contact with the operator based on the sensing signals, eliminating the need for the operator to set or switch the movement mode of the robotic arm through other manual operations, saving operation time and improving the efficiency of robotic arm control.

[0093] Step 620: Based on the sensing signal, control the robotic arm to switch between force control mode and position control mode.

[0094] Specifically, the sensor signals can be processed to identify scenarios requiring switching of the robotic arm's motion mode, enabling timely switching between force control and position control modes. In force control mode, the robotic arm's movement is controlled based on externally applied forces or torques. The robotic arm can sense the magnitude and direction of these forces and adjust its movement accordingly. In force control mode, the robotic arm can perform tasks that interact with the external environment, such as assembly and collaborative operations. In position control mode, the robotic arm's movement is controlled based on a set target position or trajectory. The robotic arm can achieve the required position control by controlling joint angles or the position of its end effector. In position control mode, the robotic arm can perform tasks requiring precise positioning and trajectory tracking, such as point-to-point operations and path planning. By controlling the robotic arm to switch between force control and position control modes based on sensor signals, human-machine collaboration can be achieved, simplifying the interaction between the operator and the robotic arm. When the operator wants to control the robotic arm to enter force control mode, the operator can change the sensor output signal by gripping the end effector, thereby enabling the robotic arm to automatically switch its motion mode to force control mode based on the change in the sensor signal. If the robotic arm moves unexpectedly in force control mode, the operator can release the grip, changing the sensor output signal, thereby enabling the robotic arm to immediately stop its current movement based on the change in the sensor signal, improving the safety of the robotic arm during operation.

[0095] In some examples, the sensing signal can be used to characterize whether the end-effector is currently being touched by the operator. The sensing signal can be processed to detect whether the end-effector is currently being touched by the operator. By detecting the contact status of the end-effector, scenarios that require switching the movement mode of the robotic arm can be identified. For example, if the sensing signal indicates that the end-effector is being touched by the operator, the movement mode of the robotic arm can be switched to a force control mode controlled by the operator in a timely manner to improve the smoothness of the robotic arm's operation. If the sensing signal indicates that the end-effector is not being touched by the operator, the robotic arm can be controlled to a position control mode.

[0096] In some examples, the sensor may include, for example: Figure 7 The tactile sensor 710 shown can be a flexible tactile sensor integrated into the surface of the end effector. When the operator grasps the part of the end effector with the tactile sensor 710, it causes a change in the sensing signal output by the tactile sensor 710. The sensing signal is transmitted to the control system, which can then select the movement mode of the robotic arm to improve the smoothness of operation.

[0097] In some examples, the sensor can be implemented as a force / torque sensor or a pressure sensor. The sensor can detect external forces and convert them into electrical signals. Then, control algorithms such as impedance control and torque control are used to correlate the electrical signals with the movement of the robotic arm. The robotic arm can also include position sensors (such as encoders). For example, the joint angles or end effector positions of the robotic arm can be obtained through the encoder's output signals. Control algorithms such as PID (Proportion Integration Differential) control and motion planning algorithms are used to compare the current position of the robotic arm with the set target position or trajectory and calculate the corresponding control commands to drive the robotic arm to achieve the required position control.

[0098] This embodiment of the robotic arm control method includes an end effector tool on the robotic arm. A sensor is installed on the gripping part of the end effector tool to acquire the sensor's output signal. Based on the signal, the robotic arm is controlled to switch between force control mode and position control mode, enabling human-machine collaboration. This simplifies the interaction between the operator and the robotic arm, eliminating the need for manual operation to set or switch the robotic arm's movement mode, saving operation time and improving control efficiency. In force control mode, if unexpected movement occurs, the operator can immediately stop the robotic arm's current movement by releasing the grip, improving safety and adjustability during movement.

[0099] In one embodiment, such as Figure 8 As shown, based on the sensing signal, the robotic arm switches between force control mode and position control mode, including:

[0100] Step 810: If the sensing signal meets the first threshold range, control the robotic arm to enter the force control mode; the first threshold range is used to characterize the end effector as being in contact with the target.

[0101] Step 820: If the sensing signal meets the second threshold range, control the robotic arm to enter the position control mode; the second threshold range is used to characterize the end effector as not in contact with the target.

[0102] Specifically, if the sensing signal meets a first threshold range, indicating that the end effector is currently in contact with a target, the robotic arm's motion mode can be set to force control mode. Here, contact with a target indicates that the end effector is in contact with the operator; for example, the operator can hold or grasp the gripping part of the end effector. The sensor output signal detects that the end effector is in contact with a target, and the robotic arm is controlled to enter force control mode. The movement of the robotic arm in force control mode can be controlled based on externally applied forces or torques (e.g., by the operator). If the sensing signal meets a second threshold range, indicating that the end effector is currently not in contact with a target, the robotic arm's motion mode can be set to position control mode. The movement of the robotic arm in position control mode can be controlled based on a set target position or trajectory.

[0103] In some examples, such as Figure 9 As shown, the surgical robot system may include a tactile sensor, a robotic arm, a hardware board connected to the tactile sensor, and a processor connecting the hardware board and the robotic arm. The control system can be integrated into the processor. The hardware board can acquire the sensing signals output by the tactile sensor and transmit them to the control system. The control system determines whether the end effector is in contact with the operator. If the end effector is not in contact with the operator, the movement mode of the robotic arm can be adjusted to position control mode. If the end effector is in contact with the operator, the movement mode of the robotic arm can be adjusted to force control mode. The robotic arm can move according to the currently set movement mode.

[0104] The process of speed planning for the end effector of the robotic arm described above can be performed in any mode of the robotic arm, such as force control mode or position control mode.

[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] Based on the same inventive concept, this application also provides a robotic arm control device for implementing the robotic arm control method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more embodiments of the robotic arm control device provided below can be found in the limitations of the robotic arm control method described above, and will not be repeated here.

[0107] In one embodiment, such as Figure 10 As shown, a robotic arm control device is provided; the device includes:

[0108] The target path planning module 1010 is used to obtain the target path planning results of the robotic arm; the target path planning results include the detection distance between the robotic arm and the target object at each path point;

[0109] The end-effector velocity planning module 1020 is used to process the target path planning results and obtain the end-effector velocity planning results of the robotic arm.

[0110] The robotic arm motion control module 1030 is used to control the movement of the robotic arm based on the end-effector velocity planning results.

[0111] In one embodiment, the target path planning module 1010 is further used to perform path planning on the robotic arm to obtain the initial path planning result of the robotic arm.

[0112] Based on the initial path planning results, collision detection is performed on the robotic arm to obtain the target path planning results.

[0113] In one embodiment, the robotic arm motion control module 1030 is also used to obtain the joint velocity planning result of the robotic arm based on the end-effector velocity planning result;

[0114] The movement of the robotic arm is controlled based on the target path planning results and joint velocity planning results.

[0115] In one embodiment, the end-effector speed planning module 1020 is further configured to process each detection distance using a preset model to obtain end-effector speed planning results; the end-effector speed planning results include the end-effector speed level of the robotic arm at each path point; the preset model includes the mapping relationship between detection distance and end-effector speed level.

[0116] In one embodiment, the robotic arm is equipped with an end effector; the gripping portion of the end effector is equipped with a sensor; the device further includes:

[0117] The signal acquisition module is used to acquire the sensing signal output by the sensor;

[0118] The mode switching module is used to control the robotic arm to switch between force control mode and position control mode based on the sensing signal.

[0119] In one embodiment, based on the sensing signal, the mode switching module is further configured to control the robotic arm to enter a force control mode if the sensing signal meets a first threshold range; the first threshold range is used to characterize the end effector as being in contact with the target.

[0120] If the sensing signal meets the second threshold range, the robotic arm is controlled to enter the position control mode; the second threshold range is used to characterize the end effector as not in contact with the target.

[0121] Each module in the aforementioned robotic arm control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0122] In one embodiment, a surgical robot system is provided, the system including a robotic arm; the system further includes a memory and a processor, the memory storing a computer program, the processor being connected to the robotic arm, and the processor executing the computer program to implement the steps of the method described above.

[0123] In one embodiment, a surgical robot system is provided, the system including a robotic arm with an end effector and a sensor at the gripping part of the end effector; the system also includes a memory and a processor, the memory storing a computer program, the processor being connected to the sensor and the robotic arm, and the processor executing the computer program to implement the steps of the method described above.

[0124] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a robotic arm control method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0125] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0128] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robotic arm control method, characterized in that, The method includes: Obtain the target path planning result of the robotic arm; the target path planning result includes the detection distance between the robotic arm and the target object at each path point; The target path planning result is processed to obtain the end-effector velocity planning result of the robotic arm; Based on the end-effector velocity planning results, control the movement of the robotic arm; The step of processing the target path planning result to obtain the end-effector velocity planning result of the robotic arm includes: The detection distances are processed using a preset model to obtain the end-effector speed planning result; the end-effector speed planning result includes the end-effector speed level of the robotic arm at each path point; the preset model includes the mapping relationship between the detection distance and the end-effector speed level; The detection distance includes the minimum distance among the distances between each joint of the robotic arm and the target object; The preset model includes a fuzzy control model, and the method further includes: Using the minimum distance as an input variable and the end-effector speed level as an output variable; establishing fuzzy control rules between the input and output variables; establishing the fuzzy control model based on the fuzzy control rules; further including: dividing the minimum distance into multiple fuzzy sets, where each fuzzy set corresponds to multiple distance levels, dividing the speed of the robotic arm into multiple fuzzy sets, where each fuzzy set corresponds to multiple speed levels, and establishing a mapping relationship between the fuzzy control rules and weights; The step of processing the target path planning result to obtain the end-effector speed planning result of the robotic arm further includes: inputting the minimum distance corresponding to each path point in the target path planning result into the fuzzy control model, obtaining the end-effector speed level corresponding to each path point through fuzzy inference and defuzzification, and using the end-effector speed level as the end-effector speed planning result.

2. The method according to claim 1, characterized in that, The process of obtaining the target path planning result of the robotic arm includes: Path planning is performed on the robotic arm to obtain the initial path planning result of the robotic arm; Based on the initial path planning result, collision detection is performed on the robotic arm to obtain the target path planning result.

3. The method according to claim 1, characterized in that, The step of controlling the movement of the robotic arm based on the end-effector velocity planning result includes: Based on the end-effector velocity planning results, the joint velocity planning results of the robotic arm are obtained; The movement of the robotic arm is controlled based on the target path planning results and the joint velocity planning results.

4. The method according to claim 1, characterized in that, The robotic arm is equipped with an end effector; the gripping portion of the end effector is equipped with a sensor; the method further includes: Acquire the sensing signal output by the sensor; Based on the sensing signal, the robotic arm is controlled to switch between force control mode and position control mode.

5. The method according to claim 4, characterized in that, The step of controlling the robotic arm to switch between force control mode and position control mode based on the sensed signal includes: If the sensing signal meets the first threshold range, the robotic arm is controlled to enter the force control mode; the first threshold range is used to characterize the end effector as being in contact with the target. If the sensing signal meets the second threshold range, the robotic arm is controlled to enter the position control mode; the second threshold range is used to characterize the end effector as not in contact with the target.

6. A robotic arm control device, characterized in that, The device includes: The target path planning module is used to obtain the target path planning results of the robotic arm; the target path planning results include the detection distance between the robotic arm and the target object at each path point; The end-effector velocity planning module is used to process the target path planning results to obtain the end-effector velocity planning results of the robotic arm; A robotic arm motion control module is used to control the movement of the robotic arm based on the end-effector velocity planning results; The end-effector speed planning module is further configured to process each of the detection distances using a preset model to obtain the end-effector speed planning result; the end-effector speed planning result includes the end-effector speed level of the robotic arm at each of the path points; the preset model includes the mapping relationship between the detection distance and the end-effector speed level; The detection distance includes the minimum distance among the distances between each joint of the robotic arm and the target object; The preset model includes a fuzzy control model, and the device is further used for: Using the minimum distance as the input variable and the terminal velocity level as the output variable, a fuzzy control rule is established between the input variable and the output variable; based on the fuzzy control rule, the fuzzy control model is established. The device is also used to divide the minimum distance into multiple fuzzy sets, where each fuzzy set corresponds to multiple distance levels, divide the speed of the robotic arm into multiple fuzzy sets, where each fuzzy set corresponds to multiple speed levels, and establish a mapping relationship between the fuzzy control rules and weights. The end-vehicle speed planning module is further configured to input the minimum distance corresponding to each path point in the target path planning result into the fuzzy control model, and obtain the end-vehicle speed level corresponding to each path point through fuzzy inference and defuzzification, and use the end-vehicle speed level as the end-vehicle speed planning result.

7. A surgical robot system, characterized in that, The system includes a robotic arm; the system also includes a memory and a processor, the memory storing a computer program, the processor being connected to the robotic arm, and the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 3.

8. A surgical robot system, characterized in that, The system includes a robotic arm with an end effector and a sensor at the gripping part of the end effector; the system also includes a memory and a processor, the memory storing a computer program, the processor being connected to the sensor and the robotic arm, and the processor executing the computer program to implement the steps of the method according to any one of claims 4 to 5.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images