Transoral Surgery Collaborative Surgical Field Exposure-Assisted Surgical Robot and Control Method

Abstract

Oral surgery collaborative surgical field exposure assistance surgical robot and control method, comprising two 6-degree-of-freedom robotic arms, two force-sensing surgical retractor assemblies and a main control computer. The force-sensing surgical retractor assembly includes a force-sensing surgical retractor, and there is a contact sensor inside the end cover plate of the force-sensing surgical retractor. The force-sensing surgical retractor is connected to the end of the robotic arm through a surgical retractor connecting piece. There are drag mode interaction and reference force adjustment interaction on the main control computer. The present invention is used to assist doctors in continuously retracting the surgical incision. The doctor only needs to place the surgical retractor at the initial position of the target area, input the reference retraction force on the interaction interface, and turn on the control system. The robot drives the force-sensing surgical retractor to safely and stably retract the surgical incision. For the convenience of doctors' operation and to ensure human-machine safety, the present invention has designed control strategies such as detailed modeling analysis, torque compensation, active retraction compensation, and safety threshold for the surgical retraction process.

Description

Technical Field

[0001] The present invention belongs to the field of assistive surgical robots, and particularly relates to a collaborative surgical field exposure assistive surgical robot for transoral surgery and a control method thereof. Background Art

[0002] The continuous exposure of the surgical field in transoral surgery is an important task in head and neck surgery, which can leave enough space for doctors to perform surgical operations and observe the surgical area. Traditional instruments are difficult to install, lack force sensing, cause great damage to patients, and doctors' assistants are prone to fatigue after long-term auxiliary traction, resulting in physiological damage to the assistants. The existing intelligent robot control technology relies on visual perception and complex modeling. However, the contact between the surgical retractor and soft tissues has serious visual occlusion and inaccurate modeling, and it cannot be applied to surgeries in open environments. Therefore, the retractor traction operation has not been well solved by robot technology.

[0003] In the current prior art, the application number is CN202210792068.2, and the patent name is a transoral access human-machine collaborative surgical robot system. This patent discloses a transoral access human-machine collaborative surgical robot system, including a navigation camera that can collect the head data information of the patient in real time; the system is signal-connected to the navigation camera and can process the data information collected by the navigation camera; the surgical robot is signal-connected to the system, and a robotic arm is provided on the surgical robot; the surgical tool is detachably arranged at the end of the robotic arm; a pinhole camera assembly is arranged on the surgical tool to observe the actions of the surgical tool and the environmental information in real time, aiming at different parts and different diseases of the oral and craniofacial regions, solving the aesthetic and safety problems brought by facial incisions while solving a large number of technical problems caused by the narrow operating space and limited field of vision in the transoral approach.

[0004] It aims at problems such as narrow access space, limited field of vision, and difficulty for surgical tools to enter the incision in transoral surgery. It uses preoperative CT scans to establish a model and real-time navigation during the operation to control a single robotic arm to hold the surgical tool and enter the surgical incision, and uses a pinhole camera to observe the tool state. This method uses functions such as preoperative modeling, registration, and real-time path planning of the robotic arm during the operation to complete the task of the surgical tool entering the incision. The collaborative surgical robot for transoral surgery proposed by the present invention aims at the continuous exposure of the surgical incision. The robot replaces the traditional retractor and the doctor's assistant to complete the auxiliary traction task, and uses an innovative control strategy with a force controller as the core to control the dual-arm robot to complete this task, without complex preoperative scan modeling, without real-time registration, navigation and other functions during the operation, and is applicable to open surgical environments. And the present invention fully considers the human-machine collaboration elements, designs a human-machine interaction function, allows doctors to freely control the traction area of the machine, adjust the traction effect online, and integrates multiple safety prevention strategies to ensure the safe completion of the task. Summary of the Invention

[0005] To solve the problems existing in the prior art, the present invention proposes a surgical robot for collaborative oral surgical field exposure assistance and a control method. It is used to assist the doctor in continuously retracting the surgical incision. The doctor only needs to place the surgical retractor at the initial position of the target area, input the reference retraction force on the interaction interface, and turn on the system. The robot drives the force-sensing surgical retractor to safely and stably retract the surgical incision. For the convenience of the doctor's operation and to ensure human-machine safety, the present invention designs control strategies such as detailed modeling analysis, torque compensation, active retraction compensation, and safety thresholds for the surgical retraction process.

[0006] The present invention provides a surgical robot for collaborative oral surgical field exposure assistance, which includes two 6-degree-of-freedom robotic arms, two force-sensing surgical retractor assemblies, and a main control computer. The force-sensing surgical retractor assembly includes a force-sensing surgical retractor, and there is a contact sensor inside the end cover plate of the force-sensing surgical retractor. The force-sensing surgical retractor is connected to the end of the robotic arm through a surgical retractor connector. The main control computer has a drag mode interaction and a reference force adjustment interaction.

[0007] The two 6-degree-of-freedom robotic arms are used to hold the force-sensing surgical retractor to complete the retraction operation of any pose in the target area in three-dimensional space.

[0008] The two force-sensing surgical retractor assemblies are used to sense the magnitude of the contact force between the retractor and the tissue in real time.

[0009] The main control computer is used to control the entire robot system, integrate control algorithms, and complete the communication between the robot and the contact sensor.

[0010] The present invention provides a control method for a surgical robot for collaborative oral surgical field exposure assistance. The control method includes the following stages:

[0011] 1) Initialization stage;

[0012] First, the doctor sends the retractor to the target area and adjusts the direction so that the retractor plane is roughly facing the retraction area.

[0013] In this stage, through the free drag mode interaction and reference force adjustment interaction of the robotic arm, the doctor sets the reference retraction force according to his own surgical needs, drags the robotic arm to the target position, and then turns on the system.

[0014] 2) Active retraction stage;

[0015] The robotic arm controls the retractor to retract the tissue until the surgical field meets the doctor's requirements. The retraction effect is quantified by the magnitude of the pulling force. The robotic arm moves along the normal direction of the end plane of the retractor until the contact force reaches the reference force. ;

[0016] The calculation principle is:

[0017] Contact sensor value and are fed into the PD controller to calculate the current control increment , where is the unified expression form of force / moment, serves as the input of the forward dynamics model of the robotic arm , where is the joint acceleration of the robotic arm, is the inertia matrix of the robotic arm, is the Jacobian matrix of the robotic arm;

[0018] The obtained through the above calculations is integrated twice to obtain the control command of the robotic arm, completing the active traction control. When the tissue is stressed equals the set reference force , the robotic arm will stabilize in this state;

[0019] During the traction process, there may be an angular deviation between the retractor plane and the tissue. Utilizing the array distribution characteristics of the one-dimensional sensor, the torque on the plane is calculated through the distribution of the four contact forces;

[0020] The calculation principle is:

[0021] , , where is the force of each contact, W is the width in the coordinate system, L is the length in the coordinate system, , also serves as an input item to compensate for the angular deviation of the traction plane;

[0022] 3) Stable traction stage;

[0023] During the operation, the doctor's operation will cause slight displacement of the tissue. At the same time, the robotic arm should yield briefly during the interference process and then quickly return to the stable traction state. In this stage, the response speed of the system is determined by the parameters of the PD controller;

[0024] Set the two parameters kp and kd of the PD controller to 0.1 and 0.3 respectively. Since the torque compensation moment response will cause the robotic arm to shake, set lower bound, if the contact torque between the retractor and the tissue is less than the lower bound, the torque will not be compensated. The safety threshold upper bound is set to . If it exceeds the upper bound, it is judged as a dangerous situation, and the robotic arm will retract to the non-contact state and issue a warning;

[0025] 4) Retraction compensation stage;

[0026] In the initialization stage, if there is a large-angle deviation between the retractor plane and the target traction area, there will be tissue damage and slippage.

[0027] The control strategy judges whether there is a large-angle deviation through the contact forces of each contact point of the contact sensor. If there is a deviation, the retractor will retreat to the non-contact state and perform large-angle deviation compensation.

[0028] As a further improvement of the control method of the present invention, the specific implementation manner of step 4) is as follows:

[0029] In the active traction stage, when the retractor contacts the patient's tissue, the forces of the four contact points are collected in real time , the retractor displacement information d, and the torque is calculated , , if the contact torque exceeds the retraction compensation threshold, the force-deformation curve of the patient's tissue will be established by polynomial fitting using the resultant force and displacement information during this process, , according to the force distribution of each point, the displacement to be compensated for each point is calculated , the compensation displacement of each point uses an optimization algorithm in its coordinate system The rotation matrix of the angle to be compensated in the local coordinate system of the retractor is calculated, and finally the large-angle compensation is completed in a closed loop by inputting the reverse torque and feedback pose information. The active retraction compensation occurs during the first active traction process. Once a large-angle deviation occurs, this stage will be triggered, and if one compensation does not meet the requirements, the compensation process will be repeated.

[0030] Compared with the prior art, the beneficial technical effects brought by the present invention are as follows:

[0031] In the present invention, doctors need to frequently switch the surgical area during transoral surgery to complete different subtasks. In the present invention, a human-computer interaction function is added. The doctor drags the robotic arm to an initial posture and activates the system to perform the traction task. When the surgical field of view does not meet the requirements after stabilization, the interactive interface allows the doctor to adjust the target traction force online. The doctor can also pause the system at any time, re-drag the robotic arm, and complete another surgical subtask. In addition, the present invention sets a safety area for the movement of the robotic arm. If the movement of the retractor exceeds this area, the system will stop and remind the doctor to check the surgical environment. The present invention also sets a maximum traction force threshold not exceeding 200% of the set reference force to ensure the safety of the surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is the overall schematic diagram of the robotic system;

[0033] Figure 2 is the schematic diagram of the control part of the robotic system;

[0034] Figure 3Schematic diagram of the force-sensing retractor kit;

[0035] Figure 4 Schematic diagram of the retractor array structure;

[0036] Figure 5 Schematic diagram of the retractor array sensing;

[0037] Figure 6 Flowchart of the control algorithm of the present invention.

[0038] Reference numerals:

[0039] 1. Robotic arm; 2. Retractor assembly; 3. Drag mode interaction; 4. Reference force adjustment interaction; 5. Surgical retractor connecting piece; 6. Force-sensing surgical retractor; 7. Contact sensor. Detailed implementation manners

[0040] The present invention will be further described in detail below in conjunction with the accompanying drawings and the detailed implementation manners:

[0041] As shown in Figures 1-5 , two retractor assemblies 2 with force-sensing retractors 6 are installed on two robotic arms 1, and a drag mode interaction 3 and a reference force adjustment interaction 4 are set on the interaction interface. Before the operation, the doctor first sets the reference force and then turns on the drag mode 3, drags the retractor assembly 2 to the target area by hand to adjust the initial position, and turns on the system. After the system is turned on, the robotic arm 1 drives the retractor assembly 2 to start pulling the tissue. Before reaching the stable state, when a large-angle deviation is detected, the system stores the displacement data and the contact force information, completes the tissue material modeling and the angle compensation calculation, and at the same time retreats to the non-contact state for angle compensation. After the compensation is completed, the active pulling task is continued. If the effect of the first compensation does not meet the requirements, the compensation is continued to be retreated. If no large-angle deviation occurs during the first active pulling, the active retreat compensation stage is skipped and the stable pulling stage is directly reached. However, when the retractor assembly 2 exceeds the safe area during the active pulling process and the contact sensor 7 is still not subjected to the set force, the robotic arm 1 will retreat to the initial state and issue a warning. If the pulling force exceeds 200% of the reference force adjustment and the robotic arm 1 still does not stop to maintain stability, it will retreat to the initial state and issue a warning. When the retractor reaches the stable pulling state, it maintains the stable pulling and responds to the disturbances such as cutting, ablation, and observation by the doctor. At the same time, if the doctor is not satisfied with the surgical field created by the current pulling, the pulling effect can be improved through the reference force adjustment 4. During this stable pulling period, once the disturbance torque exceeds , the robotic arm 1 considers that a danger has occurred, retreats to the initial state and issues a warning. During the operation, when the doctor needs to adjust the surgical operation area and has a new pulling requirement for the retractor assembly 2, the drag mode interaction 3 needs to be turned on, the retractor assembly 2 is dragged to the new target area again, and the system is turned on. The system will re-execute the above control process. The detailed flowchart of the control algorithm is as shown in Figure 6as shown

[0042] The working principle of the oral surgery autonomous surgical field exposure surgical robot is as follows:

[0043] The following factors are considered in the design of the present invention:

[0044] 1) Doctors need to frequently adjust the surgical area during the operation.

[0045] 2) The retractor needs to remain stable at all times and be able to resist interference generated by operations such as cutting and ablation.

[0046] 3) If a large perturbation occurs, the retractor should promptly adjust its pose to avoid tissue damage.

[0047] The present invention analyzes the entire process of manual traction assisting doctors and designs a human-machine collaboration method to achieve robot-assisted traction. Specifically, the present invention models the traction task into the following stages:

[0048] 1) Initialization stage. First, the doctor sends the retractor to the target area and adjusts the direction so that the retractor plane is roughly facing the traction area. In this stage, the present invention designs the mechanical arm free drag mode interaction and the reference force adjustment interaction method. The doctor sets the reference force for traction according to his own surgical needs, drags the mechanical arm to the target position, and then turns on the system.

[0049] 2) Active traction stage. The mechanical arm controls the retractor to traction the tissue until the surgical field meets the doctor's requirements. The present invention quantifies the traction effect by the magnitude of the traction force. The mechanical arm moves along the normal direction of the retractor end plane until the contact force reaches the reference force . The calculation principle is as follows: the values of the contact point sensor and are sent into the PD controller to calculate the current control increment , where is the unified expression form of force / moment, is used as the input quantity of the mechanical arm forward dynamics model , where is the joint acceleration of the mechanical arm, is the inertia matrix of the mechanical arm. is the Jacobian matrix of the mechanical arm. The obtained is integrated twice to obtain the control command of the mechanical arm, completing the active traction control. When the force on the tissue is equal to the set reference force , the mechanical arm will stabilize in this state.

[0050] During the pulling process, an angular deviation may occur between the retractor plane and the tissue. The present invention utilizes the array distribution characteristics of the one-dimensional sensor to calculate the torque on the plane by the magnitude distribution of the forces at the four contact points. The calculation principle is as follows: , , where is the force at each contact point, W is the width in the coordinate system, and L is the length in the coordinate system. , is also used as an input item to compensate for the angular deviation of the pulling plane.

[0051] 3) Stable pulling stage. During the operation, the doctor's operation may cause slight displacement of the tissue. At the same time, the robotic arm should yield briefly during the interference and then quickly return to the stable pulling state. In this stage, the response speed of the system is determined by the parameters of the PD controller. In order to balance both the response speed and the stability of the system for tissues with different material properties, the present invention sets two parameters to 0.1 and 0.3 respectively. Since the torque compensation moment response will cause the robotic arm to shake, the present invention sets the lower bound. If the contact torque between the retractor and the tissue is less than the lower bound, no torque compensation is performed. The upper bound of the safety threshold is set to . If it exceeds the upper bound, it is judged as a dangerous situation, and the robotic arm retracts to the non-contact state and issues a warning.

[0052] 4) Retraction compensation stage. In the initialization stage, if there is a large angular deviation between the retractor plane and the target pulling area, tissue damage and slippage may occur. The control strategy of the present invention determines whether there is a large angular deviation through the forces at each contact point of the contact sensor. If there is, the retractor will retract to the non-contact state and perform large angular deviation compensation. The specific implementation method is as follows: In the active pulling stage, when the retractor contacts the patient's tissue, the forces at the four contact points , the retractor displacement information d are collected in real time, and the torque is calculated. , if the contact torque exceeds the retraction compensation threshold, the force deformation curve of the patient's tissue is established by polynomial fitting of the resultant force and displacement information during this process. . According to the force distribution at each point, the displacement that should be compensated at each point is calculated. The compensation displacement at each point uses an optimization algorithm in its coordinate system to calculate the rotation matrix of the angle that should be compensated in the local coordinate system of the retractor. Finally, the large angular compensation is completed in a closed loop by inputting the reverse torque and feedback pose information. Active retraction compensation occurs during the first active pulling process. Once a large angular deviation occurs, this stage will be triggered, and if one compensation does not meet the requirements, the compensation process will be repeated.

[0053] The above are only the preferred embodiments of the present invention, and are not any other form of limitation to the present invention. Any modification or equivalent change made according to the technical essence of the present invention still belongs to the scope claimed by the present invention.

Claims

1. An oral surgery collaborative operative field exposure-assisted surgical robot, comprising two six-degree-of-freedom robotic arms (1), two force-sensing surgical retractor assemblies (2), and a main control computer. The force-sensing surgical retractor assembly (2) includes a force-sensing surgical retractor (6), and a contact sensor (7) is inside the end cover plate of the force-sensing surgical retractor (6). The force-sensing surgical retractor (6) is connected to the end of the robotic arm (1) through a surgical retractor connecting piece (5). The main control computer has a drag interaction mode (3) and a reference force adjustment interaction mode (4); The two six-degree-of-freedom robotic arms (1) are used to clamp the force-sensing surgical retractor (6) to complete the pulling operation of any pose of the target area in three-dimensional space; The two force-sensing surgical retractor assemblies (2) are used to sense the magnitude of the contact force between the retractor and the tissue in real time; The main control computer is used to control the entire robot system, integrate control algorithms, and complete the communication between the robot and the contact sensor; The control method of the oral surgery collaborative operative field exposure-assisted surgical robot includes the following stages, and is characterized in that, 1) Initialization stage; First, the doctor sends the retractor to the target area and adjusts the direction so that the retractor plane is roughly facing the pulling area; In this stage, through the free drag interaction and reference force adjustment interaction methods of the robotic arm, the doctor sets the reference force of the pulling according to his own surgical needs, drags the robotic arm to the target position, and then turns on the control system; 2) Active pulling stage; The robotic arm controls the retractor to pull the tissue until the surgical field meets the doctor's requirements, and the pulling effect is quantified by the magnitude of the pulling force. The robotic arm moves along the normal direction of the plane at the end of the retractor until the contact force reaches the set reference force ; The calculation principle is: Contact sensor value and are sent into the PD controller to calculate the current control increment where is the unified expression form of force / torque, serves as the input of the forward dynamics model of the robotic arm where is the joint acceleration of the robotic arm, is the inertia matrix of the robotic arm, is the Jacobian matrix of the robotic arm; Obtained through the above calculations The control command of the robotic arm is obtained through two integrations to complete the active traction control. When the force on the tissue equals the set reference force , the robotic arm will stabilize in this state; During the pulling process, an angular deviation may occur between the retractor plane and the tissue. Utilizing the array distribution characteristics of the one-dimensional sensor, the torque on the plane is calculated through the distribution of the magnitudes of the four contact forces; The calculation principle is: , , where is the force of each contact, W is the width in the coordinate system, and L is the length in the coordinate system, , is also used as an input item to compensate for the angular deviation of the drawing surface; 3) Stable pulling stage; During the operation, the doctor's operation will cause slight displacement of the tissue. At the same time, the robotic arm should yield briefly during the interference process and then quickly return to the stable pulling state. In this stage, the response speed of the control system is determined by the parameters of the PD controller; Set the two parameters kp and kd of the PD controller to 0.1 and 0.3 respectively. Since the torque compensation moment response will cause the shaking of the robotic arm, set the lower bound. If the contact torque between the retractor and the tissue is less than the lower bound, the torque will not be compensated. The upper bound of the safety threshold is set to . If it exceeds the upper bound, it is judged as a dangerous situation. The robotic arm retracts to a non-contact state and issues a warning; 4) Retraction compensation stage; In the initialization stage, if there is a large angular deviation between the retractor plane and the target pulling area, tissue damage and slippage will occur; The control strategy judges whether there is a large angular deviation through the contact forces of each contact of the contact sensor. If there is a deviation, the retractor will retract to the non-contact state and perform large angular deviation compensation; The specific implementation method of step 4) is as follows: During the active traction stage, the forces of the four contacts are collected in real time when the retractor contacts the patient's tissue , the displacement information d of the retractor is obtained, and the torque is calculated , , if the contact torque exceeds the retraction compensation threshold, the force-deformation curve of the patient's tissue is established by polynomial fitting of the resultant force and displacement information during this process , according to the force distribution of each point, the displacement to be compensated for each point is calculated , the compensation displacement of each point uses an optimization algorithm in its coordinate system to calculate the rotation matrix of the angle to be compensated in the local coordinate system of the retractor. Finally, the large-angle compensation is completed in a closed loop by inputting the reverse torque and feedback pose information. The active retraction compensation occurs during the first active traction process. Once a large-angle deviation occurs, this stage will be triggered. And if one compensation does not meet the requirements, the compensation process will be repeated

Images