A human-computer interaction system and method based on operator intention correction

Abstract

The present invention relates to a human-computer interaction system and method based on operator intention correction, including a robotic arm, a surgical tool, a six-axis force sensor, a force sensor, and a controller; the six-axis force sensor is installed at the end of the robotic arm, the surgical tool is installed on the six-axis force sensor through a fixing device, and the force sensor is installed on the surgical tool; when the operator pulls the part of the surgical tool where the force sensor is installed to perform an operation, it is detected whether the reading of the force sensor reaches a preset threshold; if not, the robotic arm does not move, otherwise, the torque read by the six-axis force sensor is corrected according to the reading of the force sensor and the positional relationship between the six-axis force sensor and the force sensor, and the movement of the robotic arm is controlled according to the force signal and the corrected torque signal of the six-axis force sensor. Compared with the prior art, the present invention solves the problem that the operator's intention is inconsistent with the actual movement of the robotic arm during the human-machine collaborative operation using admittance control.

Description

Technical Field

[0001] The present invention relates to the field of surgical robots, and in particular, to a human-computer interaction system and method based on operator intention correction. Background Art

[0002] In the field of surgical robot technology, due to the ever-changing intraoperative situations, there is a possibility that preoperative planning may not be applicable. This requires surgeons to adjust the surgical plan based on clinical experience and directly manipulate the robotic arm for repositioning and operation. For example, in actual clinical scenarios, active surgical robots cannot fully meet the requirements of oral and maxillofacial surgery, and passive surgical robots that allow doctors to continuously adjust the robot's state during the operation are more needed.

[0003] Human-machine collaborative control realizes the cooperation between the robot and humans for operation through force-position hybrid coordinated control, combining the high-precision control of the robot with the high-dynamic decision-making ability of humans, which is a common robot control method. It mainly uses admittance control or impedance control. Generally, a force sensor is installed at the end of the robotic arm. When the operator applies a force at the end, the external force received at the end of the robotic arm is mapped to the movement of the robotic arm, and a quantitative relationship can be established between the operation output force of the human and the output movement of the robot to solve the problem of compliance in human-machine collaborative manipulation, so as to achieve the purpose of active control of the robot by the doctor and maintaining the doctor's skill level in the surgical field.

[0004] Currently, in the field of surgical robots, common interaction methods include directly pulling the surgical tool installed at the end of the robotic arm and pulling the handle of the force sensor.

[0005] For the first interaction method, such as the Yomi dental implant robot navigation system developed by the US medical company Neocis, doctors interact by directly pulling the surgical tool installed at the end of the robotic arm. However, since the working degree of freedom of the implant drill only requires 5 (the rotation of the drill does not need to be considered, only its axial direction and position need to be determined), the adjustment range of the rotational movement of the implant drill is not large, mainly for translational movement.

[0006] For the second interaction method, such as the relevant research of He Yucheng et al. in "Research on the Mechanism Design and Safety Control of a Nasal Endoscopy Surgery-Assisted Robot", the nasal endoscopy surgery-assisted robot uses two force sensors. One is installed on the surgical tool (nasal endoscope) to sense the force on the surgical tool during operation; the other is located on the operation handle for human-machine collaborative control. Doctors complete the control of the robot by pulling the operation handle.

[0007] However, the above methods all have drawbacks. In the first case, when the surgical tool needs to rotate significantly and the operator's hand is far from the sensor position, it will cause the phenomenon that the movement of the robot is different from the operator's intention. Because there is a displacement between the force application point and the sensor at this time, according to the principle of force translation, although the forces in the X, Y, and Z directions do not change, the torques received by the sensor and the force application point are different, resulting in the problem that the rotation of the robotic arm is different from the operator's intention. For the second case, since the operator only holds and operates the handle, it can be considered that the robotic arm can execute according to the operator's intention. However, the implementation of this solution requires two six-axis force sensors, which is costly, and the six-axis force sensors are relatively large in size and difficult to be installed on the handle of the surgical tool in many clinical scenarios.

[0008] In summary, the existing interaction devices and methods have defects such as poor accuracy, poor stability, and high technical costs. Therefore, a human-machine interaction solution that can achieve the movement of the robotic arm according to the operator's intention is needed. Summary of the Invention

[0009] The purpose of the present invention is to provide a human-machine interaction system and method based on operator intention correction to overcome the defects of the above-mentioned existing technologies.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] In a first aspect, the present invention discloses a human-machine interaction system based on operator intention correction, including a robotic arm, a surgical tool, a six-axis force sensor S1, a force sensor S2, and a controller;

[0012] The six-axis force sensor S1 is installed at the end of the robotic arm, the surgical tool is installed on the six-axis force sensor S1 through a fixing device, the force sensor S2 is installed on the surgical tool, and the controller is communicatively connected to the robotic arm, the six-axis force sensor S1, and the force sensor S2;

[0013] When the operator pulls and operates the part of the surgical tool where the force sensor S2 is installed, the controller detects whether the reading of the force sensor S2 reaches a preset threshold; if the reading of the force sensor S2 does not reach the preset threshold, the robotic arm does not move, otherwise, the controller corrects the torque read by the six-axis force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six-axis force sensor S1 and the force sensor S2, and controls the movement of the robotic arm according to the force signal and the corrected torque signal of the six-axis force sensor S1.

[0014] Further, the preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and firmly holds the surgical tool.

[0015] Further, the positional relationship between the six - dimensional force sensor S1 and the force sensor S2 is determined according to the mechanical dimensions of the robotic arm, the surgical tool, and the fixing device.

[0016] Further, the correction of the torque of the six - dimensional force sensor S1 according to the reading of the force sensor S2 is specifically as follows:

[0017]

[0018] is the position vector from the center O of the six - dimensional force sensor S1 S to the center O of the force sensor S2 Doc , which is determined by the positional relationship between the six - dimensional force sensor S1 and the force sensor S2. F is the force read by the six - dimensional force sensor S1, T is the torque read by the six - dimensional force sensor S1, and T S is the corrected torque of the six - dimensional force sensor S1.

[0019] Further, the controller converts the force signal and the corrected torque signal of the six - dimensional force sensor S1 into a motion signal of the robotic arm according to admittance control. The robotic arm moves according to the motion signal, and the movement of the robotic arm drives the movement of the surgical tool.

[0020] Further, the six - dimensional force sensor S1 is fixedly installed at the end of the robotic arm, the fixing device is a clamp, and the surgical tool is installed on the six - dimensional force sensor S1 through the clamp.

[0021] Further, the force sensor S2 is installed on the handle of the surgical tool, and the operator operates by pulling the handle of the surgical tool.

[0022] In a second aspect, the present invention discloses a human - machine interaction method based on operator intention correction, including the following steps:

[0023] Check the robotic arm, the surgical tool, the six - dimensional force sensor S1, the force sensor S2, and the controller. The six - dimensional force sensor S1 is installed at the end of the robotic arm, the surgical tool is installed on the six - dimensional force sensor S1 through the fixing device, the force sensor S2 is installed on the surgical tool, and the controller is communicatively connected to the robotic arm, the six - dimensional force sensor S1, and the force sensor S2;

[0024] The operator operates by pulling the part of the surgical tool where the force sensor S2 is installed, and the controller detects whether the reading of the force sensor S2 reaches a preset threshold;

[0025] If the reading of the force sensor S2 does not reach the preset threshold, the robotic arm does not move. Otherwise, based on the reading of the force sensor S2 and the determined positional relationship between the six - dimensional force sensor S1 and the force sensor S2, the torque of the six - dimensional force sensor S1 is corrected, and the movement of the robotic arm is controlled according to the force signal and the corrected torque signal of the six - dimensional force sensor S1.

[0026] Further, the preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and firmly holding the surgical tool.

[0027] Further, the positional relationship between the six - dimensional force sensor S1 and the force sensor S2 is determined according to the mechanical dimensions of the robotic arm, the surgical tool, and the fixing device.

[0028] Further, the correction of the torque of the six - dimensional force sensor S1 according to the reading of the force sensor S2 is specifically as follows:

[0029]

[0030] Let \( \overrightarrow{OO'} \) be the position vector between the center \( O \) of the six - dimensional force sensor S1 S and the center \( O' \) of the force sensor S2, Doc determined by the positional relationship between the six - dimensional force sensor S1 and the force sensor S2, \( F \) is the force read by the six - dimensional force sensor S1, \( T \) is the torque read by the six - dimensional force sensor S1, and \( T' \) S is the corrected torque of the six - dimensional force sensor S1.

[0031] Further, the controller converts the force signal and the corrected torque signal of the six - dimensional force sensor S1 into a movement signal of the robotic arm according to admittance control, and the robotic arm moves according to the movement signal, and the movement of the robotic arm drives the surgical tool to move.

[0032] Further, the six - dimensional force sensor S1 is fixedly installed at the end of the robotic arm, the fixing device is a fixture, and the surgical tool is installed on the six - dimensional force sensor S1 through the fixture.

[0033] Further, the force sensor S2 is installed on the handle of the surgical tool, and the operator operates by pulling the handle of the surgical tool.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) By setting a preset threshold, when the reading of the force sensor is less than the preset threshold, the robotic arm does not move. After reaching the preset threshold, the torque of the six - dimensional force sensor is corrected, and the movement of the robotic arm is controlled based on the corrected torque, solving the problem that the operator's intention is inconsistent with the actual movement of the robotic arm during the man - machine collaborative operation using admittance control.

[0036] (2) Install a force sensor on the surgical tool. The preset threshold is the reading of the force sensor when the operator's hand is stationary and firmly holding the surgical tool. The force received by the force sensor can be compared with the preset threshold to determine whether the operator needs to tow the robotic arm, increasing the safety of control. Only when the force received by the force sensor reaches the preset threshold can the robotic arm move according to the force situation, avoiding interference and damaging the normal movement of the robotic arm.

[0037] (3) Operate by towing the handle of the surgical tool, fully considering the operating habits of doctors during surgery, with good human-computer interaction. The robotic arm moves according to the operator's intention, reducing the learning cost of the operator, being able to precisely operate the robot, applicable to various surgical procedures, and having good versatility. Description of the Drawings

[0038] Figure 1 It is a schematic structural diagram of a human-computer interaction system based on operator intention correction;

[0039] Reference Numerals: 1, robotic arm; 2, surgical tool; S1, six-dimensional force sensor; S2, force sensor. Detailed Embodiments

[0040] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and the detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0041] In the drawings, components with the same structure are denoted by the same numerals, and components with similar structures or functions are denoted by similar numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the drawings clearer, some parts in the drawings are appropriately enlarged.

[0042] Embodiment 1:

[0043] First, the present invention discloses a human-computer interaction system based on operator intention correction, including a robotic arm 1, a surgical tool 2, a six-dimensional force sensor S1, a force sensor S2, and a controller;

[0044] As Figure 1 shown, the six-dimensional force sensor S1 is installed at the end of the robotic arm 1, the surgical tool 2 is installed on the six-dimensional force sensor S1 through a fixing device, the force sensor S2 is installed on the surgical tool 2, and the controller is communicatively connected to the robotic arm 1, the six-dimensional force sensor S1, and the force sensor S2;

[0045] When the operator operates by pulling the part of the surgical tool 2 where the force sensor S2 is installed, the controller detects whether the reading of the force sensor S2 reaches a preset threshold; if the reading of the force sensor S2 does not reach the preset threshold, the robotic arm 1 does not move, otherwise, the controller corrects the torque read by the six-axis force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six-axis force sensor S1 and the force sensor S2, and controls the movement of the robotic arm 1 according to the force signal and the corrected torque signal of the six-axis force sensor S1.

[0046] Among them, the preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and firmly holds the surgical tool 2. It can be measured multiple times before the operation, record the reading of the force sensor S2 when the operator's hand is stationary and firmly holds the surgical tool 2, and then take the average value as the preset threshold.

[0047] The correction of the torque of the six-axis force sensor S1 according to the reading of the force sensor S2 is specifically as follows:

[0048]

[0049] is the position vector of the center O of the six-axis force sensor S1 S and the center O of the force sensor S2 Doc , that is, the position of the operator's hand grip relative to the center of the six-axis sensor S1, which is determined by the positional relationship between the six-axis force sensor S1 and the force sensor S2. The robotic arm 1 and the surgical tool 2 move synchronously. Therefore, during the operation, the center O of the six-axis force sensor S1 S and the center O of the force sensor S2 Doc are relatively unchanged. is unchanged; F is the force read by the six-axis force sensor S1, which is actually the external force applied by the operator, T is the torque read by the six-axis force sensor S1, and T S is the corrected torque of the six-axis force sensor S1.

[0050] During use, the process is as follows:

[0051] (1) Install the six-axis force sensor S1 at the end of the robotic arm 1. The six-axis force sensor S1 is fixedly installed at the end of the robotic arm 1 by screws or the like;

[0052] (2) The surgical tool 2 is installed on the six-axis force sensor S1 through a fixing device. The force sensor S2 is installed on the surgical tool 2. The fixing device is a clamp. The surgical tool 2 is installed on the six-axis force sensor S1 through the clamp, which is convenient for replacing and adjusting the surgical tool 2. The force sensor S2 is installed on the handle of the surgical tool 2. The operator holds the handle of the surgical tool 2, and it should be ensured that the position of the operator's hand grip is the center of the force sensor S2;

[0053] (3) Determine the positional relationship between the six - dimensional force sensor S1 and the force sensor S2 according to the mechanical dimensions of the robotic arm 1, the surgical tool 2, and the fixing device. As Figure 1 shown, S and Doc are the coordinate systems of the six - dimensional force sensor S1 and the force sensor S2 respectively. The position of O S is the installation position of the six - dimensional force sensor S1, and the position of O Doc is the installation position of the force sensor S2;

[0054] (4) The operator pulls the part of the surgical tool 2 where the force sensor S2 is installed for operation, that is, the operator pulls the handle of the surgical tool 2 for operation;

[0055] (5) Determine whether the reading of the force sensor S2 reaches the preset threshold. If it does not reach, it means that the operator's intention is not to pull the robotic arm 1, then the robotic arm 1 does not move. Otherwise, it means that the operator's intention is to pull the robotic arm 1. The controller corrects the torque read by the six - dimensional force sensor S1 according to the reading of the force sensor S2 and the positional relationship between the six - dimensional force sensor S1 and the force sensor S2, eliminating the displacement between the force application point and the six - dimensional force sensor S1, so that the movement of the robotic arm 1 conforms to the operator's intention. Specifically, according to admittance control, the controller, based on admittance control, converts the force signal F and the corrected torque signal T of the six - dimensional force sensor S1 S into the movement signal of the robotic arm 1, and the robotic arm 1 moves according to the movement signal, and the movement of the robotic arm 1 drives the movement of the surgical tool 2.

[0056] The present invention has the following advantages:

[0057] (1) By setting a preset threshold, when the reading of the force sensor S2 is less than the preset threshold, the robotic arm 1 does not move. After reaching the preset threshold, the torque of the six - dimensional force sensor S1 is corrected, and the movement of the robotic arm 1 is controlled based on the corrected torque, solving the problem that the operator's intention is inconsistent with the actual movement of the robotic arm 1 during the man - machine collaborative operation using admittance control.

[0058] (2) Install the force sensor S2 on the surgical tool 2. The preset threshold is the reading of the force sensor S2 when the operator's hand is stationary and firmly holds the surgical tool 2. It is possible to compare the force on the force sensor S2 with the preset threshold to determine whether the operator needs to pull the robotic arm 1, increasing the safety of control. Only when the force on the force sensor S2 reaches the preset threshold can the robotic arm 1 move according to the force condition, avoiding interference and damaging the normal movement of the robotic arm 1.

[0059] (3) It is operated through the handle of the traction surgical tool 2, fully considering the operating habits of doctors during surgery, with good human-computer interaction. The robotic arm 1 moves according to the operator's intention, reducing the learning cost of the operator, being able to precisely operate the robot, applicable to various surgical procedures, and having good versatility.

[0060] In a second aspect, the present invention discloses a human-computer interaction method based on operator intention correction, including the following steps:

[0061] ① Check the robotic arm 1, the surgical tool 2, the six-axis force sensor S1, the force sensor S2, and the controller. Among them, the six-axis force sensor S1 is installed at the end of the robotic arm 1, the surgical tool 2 is installed on the six-axis force sensor S1 through a fixing device, the force sensor S2 is installed on the surgical tool 2, and the controller is communicatively connected to the robotic arm 1, the six-axis force sensor S1, and the force sensor S2;

[0062] ② The operator operates by pulling the part of the surgical tool 2 where the force sensor S2 is installed, and the controller detects whether the reading of the force sensor S2 reaches a preset threshold;

[0063] ③ If the reading of the force sensor S2 does not reach the preset threshold, the robotic arm 1 does not move. Otherwise, the torque of the six-axis force sensor S1 is corrected according to the reading of the force sensor S2 and the determined positional relationship between the six-axis force sensor S1 and the force sensor S2, and the movement of the robotic arm 1 is controlled according to the force signal and the corrected torque signal of the six-axis force sensor S1.

[0064] The human-computer interaction method based on operator intention correction provided by the present invention and the system embodiment are based on the same inventive concept. For details, please refer to the system embodiment and will not be elaborated here.

[0065] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should fall within the protection scope determined by the claims.

Claims

1. A human-computer interaction system based on operator intention correction, characterized in that, It includes a robotic arm, a surgical tool, a six-axis force sensor, a force sensor, and a controller; The six-axis force sensor is installed at the end of the robotic arm. The surgical tool is installed on the six-axis force sensor through a fixing device. The force sensor is installed on the surgical tool. The controller is communicatively connected to the robotic arm, the six-axis force sensor, and the force sensor; When the operator pulls the part of the surgical tool where the force sensor is installed to perform an operation, the controller detects whether the reading of the force sensor reaches a preset threshold; If the reading of the force sensor does not reach the preset threshold, the robotic arm does not move. Otherwise, the controller corrects the torque read by the six-axis force sensor according to the reading of the force sensor and the positional relationship between the six-axis force sensor and the force sensor, and controls the movement of the robotic arm according to the force signal and the corrected torque signal of the six-axis force sensor; The preset threshold is the reading of the force sensor when the operator's hand is stationary and firmly holds the surgical tool; Specifically, correcting the torque of the six-axis force sensor according to the reading of the force sensor is as follows: is the center of the six - dimensional force sensor and the center of the force sensor The position vector, which is determined by the positional relationship between the six - dimensional force sensor and the force sensor, is the force read by the six - dimensional force sensor, is the torque read by the six - dimensional force sensor, is the torque after calibration of the six - dimensional force sensor.

2. The human-computer interaction system based on operator intention correction according to claim 1, wherein The controller converts the force signal and the corrected torque signal of the six-axis force sensor into a motion signal of the robotic arm according to admittance control. The robotic arm moves according to the motion signal, and the movement of the robotic arm drives the surgical tool to move.

3. A human-computer interaction system based on operator intention correction according to claim 1, characterized in that The six-axis force sensor is fixedly installed at the end of the robotic arm. The fixing device is a fixture, and the surgical tool is installed on the six-axis force sensor through the fixture.

4. A human-computer interaction system based on operator intention correction according to claim 1, characterized in that The force sensor is installed on the handle of the surgical tool, and the operator pulls the handle of the surgical tool to perform an operation.

Images