Operation area force feedback teleoperation control method and device for flexible surgical robot

Abstract

The invention discloses a flexible surgical robot operation area force feedback teleoperation control method and device, and relates to the technical field of medical robots. The method comprises the following steps: acquiring a signal of a main end of an operator, obtaining a tail end expected attitude instruction in a space mapping mode, acquiring visual image information through an endoscope camera, obtaining tail end stress data, tail end pose estimation, tension data, an environment model and an artificial potential field model, performing offset correction and trembling correction on the tail end pose estimation, and obtaining a tail end expected attitude result. And then periodic pose disturbance correction is carried out to obtain a corrected end expected pose, a corrected end expected pose instruction is converted into a new target position instruction, and guiding and supporting of the flexible actuator in a deep zigzag path are realized. The operation intention of an operator at the master end can be remarkably and accurately mapped to the tail end of the slave end flexible robot, and the operation area contact force is fed back to the master end in real time, so that the operation safety, stability and operation precision are improved.

Description

Technical Field

[0001] This invention relates to the field of medical robot technology, and in particular to a method and device for force feedback teleoperation control of a flexible surgical robot in the surgical area. Background Technology

[0002] Natural Orifice Transluminal Endoscopic Surgery (NOTES) has garnered widespread attention due to its lack of external incisions and minimal trauma. However, the narrow, tortuous spaces and easily deformable flexible tissues of natural orifices place higher demands on the compliance, force sensing, and teleoperation control of robots. Existing interventional surgeries primarily rely on manual manipulation by surgeons, which is both technically challenging and risky. Traditional rigid surgical robots struggle to reach target sites within narrow and tortuous natural orifices and lack sufficient compliance, easily damaging the cavity walls. In recent years, flexible continuum robots have demonstrated great potential in minimally invasive natural orifice surgeries due to their high flexibility and small diameter. However, current technologies still face the following challenges: the structural design and drive control of flexible actuators struggle to balance precision and compliance; dynamic changes in the intraoperative environment (such as confinement, vibration, and physiological movements like breathing) cause path drift, lacking effective real-time compensation mechanisms; and the human-machine collaborative control mechanism is imperfect, resulting in a mismatch between the surgeon's experience and the robot's response. Therefore, there is an urgent need for a flexible interventional surgical robot system that can operate precisely and safely in dynamic and tortuous cavities. Summary of the Invention

[0003] To address the technical problems of existing technologies, such as the difficulty in balancing precision and compliance in flexible actuators; the lack of effective real-time compensation mechanisms for path drift caused by dynamic changes in the surgical environment (e.g., constraints, vibrations, breathing, etc.); and the mismatch between the surgeon's experience and the robot's response, this invention provides a method and device for force feedback teleoperation control of a flexible surgical robot in the surgical area. The technical solution is as follows:

[0004] On the one hand, a method for force feedback teleoperation control of a flexible surgical robot is provided. This method is implemented by a force feedback teleoperation control device for the surgical area of ​​the flexible surgical robot, and includes: S1: Collect signals from the operator's master end and obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector desired pose based on transformation matrix, scaling factor and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. S2: Visual image information is acquired through the endoscope camera, and image data of the surgical area, information on potential obstacles and end-effector feature positions are extracted. Preprocessing is performed to obtain end-effector force data, end-effector pose estimation, tension data, environmental model and artificial potential field model. S3: When entering the intraoperative dynamic environment, after judging the deviation state, restricted state, obstacle avoidance state, and temperature safety state, the offset is corrected to obtain the corrected end-effector desired posture. S4: When tremors are present during surgical procedures, the pose error and driving joint torque are calculated based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. S5: When there is a periodic pose disturbance of the target organization, establish a periodic displacement function model, use model predictive control or reinforcement learning strategy to optimize the path, and obtain the corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end to apply a reverse displacement in advance to counteract the periodic motion of the target. S6: Receive the corrected end-effector desired attitude command, convert the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep tortuous path.

[0005] Preferably, the signal acquired by the operator in step S1 is used to obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master-end pose to the slave-end desired end-effector pose based on a transformation matrix, scaling factor, and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link, comprising: S11: Collect signals from the operator's main end to obtain raw data from the main end. The raw data from the main end includes position information, attitude information, force / pressure signals, and zero-position attitude data. The zero-position attitude data is obtained by setting the zero-position attitude of the main end. S12: Using a hand-eye coordination calibration method, the master-slave coordinate system is calibrated to obtain a transformation matrix. The transformation matrix is ​​used to determine the transformation relationship between the master operating handle coordinate system, the slave robot base coordinate system, and / or the surgical endoscope coordinate system. S13: Set the position scaling ratio and / or speed scaling ratio according to the surgical requirements to obtain the scaling coefficient. The scaling coefficient is used to scale the master end motion increment to obtain the target displacement of the slave end actuator. S14: Collect the roll angle, yaw angle and / or pitch angle of the master end attitude, perform angle mapping using a one-to-one correspondence, and set the attitude mapping algorithm. The attitude mapping algorithm is used to calculate the desired attitude change of the end effector of the slave flexible robot. S15: Based on the original data from the master terminal, a master terminal control command is obtained through signal processing and command generation algorithm. The master terminal control command includes a position command, an attitude command, or a force control command. S16: Based on the spatial mapping method, the master control command is mapped to the desired pose of the slave end. The spatial mapping method includes mapping the master pose to the desired pose of the slave end based on the transformation matrix, the scaling factor and the attitude mapping algorithm. S17: Input the desired pose of the end point into the inverse kinematics solution algorithm or the virtual joint mapping algorithm to obtain the extension and driving current of each cable segment. The inverse kinematics solution algorithm includes analytical solution or numerical iterative solution. S18: Based on the extension and retraction of each cable segment and the driving current, the desired end-effector posture command is obtained through communication link encapsulation and transmission. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link.

[0006] Preferably, step S2 involves acquiring visual image information via an endoscopic camera, extracting image data of the surgical area, information on potential obstacles, and the location of end-effector features, performing preprocessing to obtain end-effector force data, end-effector pose estimation, tension data, an environmental model, and an artificial potential field model, including: S21: Visual image information is acquired through an endoscope camera, and the visual image information is processed using denoising and enhancement algorithms to obtain image data of the surgical area, information on potential obstacles, and the location of end-effector features. The image data includes visual features of the surgical target and anatomical structure. S22: Force information is collected by the end force sensor and force preprocessing is performed to obtain end force data. The force preprocessing includes using a low-pass filtering algorithm to eliminate motor vibration noise from the force information to obtain end force data. S23: Temperature information is collected by a temperature sensor and temperature preprocessing is performed to obtain temperature data of the surgical site or instrument surface. The temperature preprocessing includes smoothing the temperature information using a moving average algorithm to obtain temperature data. S24: Based on the robot's kinematics model, predict the end-effector position to obtain the predicted end-effector position; S25: Compare the end feature position with the predicted end position, correct the prediction deviation, and obtain the end pose estimate, which includes the spatial position and attitude angle of the end. S26: Estimate the end-effector pose and reconstruct the flexible arm shape to obtain the reconstructed flexible arm shape; S27: Acquire tension information of the cable through a pressure sensor to obtain tension data; S28: The environment is modeled using a visual 3D reconstruction algorithm to obtain an environmental model; S29: Based on the environmental model and potential obstacle information, identify prohibited access areas or important structures, and set virtual boundaries, wherein the virtual boundaries include a safe distance threshold; S210: Match the reconstructed flexible arm shape with the environmental model and the target to obtain the relative position information of the end effector in the environment; S211: Based on the virtual boundary, potential field modeling is performed to obtain an artificial potential field model. The artificial potential field model includes a gravitational potential field and a repulsive potential field and their gradients. The gravitational potential field is applied to the end effector by the surgical target, and the repulsive potential field is applied to the end effector by the obstacle. The potential field modeling includes using a gradient calculation method to calculate the position of the obstacle and the position of the robot end effector.

[0007] Preferably, in step S3, upon entering the intraoperative dynamic environment, after determining the entry deviation state, restricted state, obstacle avoidance state, and temperature safety state, offset correction is performed to obtain the corrected end-effector desired posture, including: S31: During the procedure, information is collected in real time through the endoscope camera, end force sensor and temperature sensor to obtain the current temperature, current force feedback information, current potential obstacle information and current end relative position information in the environment; S32: Based on the predetermined route, determine the relative position information of the current terminal in the environment to obtain the deviation status. When the deviation status indicates that the terminal deviates from the predetermined route, trigger the deviation adjustment strategy. S33: Compare and analyze the current relative position information of the end in the environment with the actual posture sensor deformation to obtain the deformation difference. Combine the end force data to evaluate the deformation difference and obtain the restricted state determination result. The restricted state determination result is used to indicate whether the end has entered a restricted state. S34: When the restricted state determination result indicates that the end is in a restricted state, a local obstacle avoidance strategy is triggered; S35: Compare the real-time force feedback information with a preset force threshold. When the force in a certain direction at the end approaches the preset force threshold, it is determined to be an obstacle avoidance state, and a local obstacle avoidance strategy is triggered. S36: Based on the safety threshold, the temperature information is judged to obtain the temperature safety status. When the temperature safety status indicates that the temperature exceeds the safety threshold, a speed adjustment strategy and a warning signal are triggered. The speed adjustment strategy is used to reduce the forward speed of the end or suspend the end at a safe distance. S37: When the deviation adjustment strategy, local obstacle avoidance strategy, or speed adjustment strategy is triggered, the current temperature, current force feedback information, current potential obstacle information, and current relative position information of the end in the environment are input into the artificial potential field model to obtain an updated artificial potential field model. The updated artificial potential field model includes updated obstacle position parameters and repulsion force parameters. S38: Input the updated artificial potential field model into the admittance control module to perform force-position mapping and obtain the end position offset; S39: Input the end position offset into the inverse kinematics solver to perform inverse kinematics calculations and obtain the joint angle increment or the cable length increment. S310: Based on the joint angle increment or cable length increment, perform path planning to obtain the corrected end-effector desired posture.

[0008] Preferably, in step S4, when tremors occur during surgical procedures, the pose error and driving joint torque are calculated based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture, including: S41: Extract the desired end pose based on the path update instruction, calculate the pose difference between the desired end pose and the actual pose, and obtain the pose error, which includes position deviation and attitude deviation. S42: Using a parameter adaptive strategy, visual deviation and force feedback signal are used as inputs to obtain impedance control parameter correction increment. The impedance control parameter correction increment is used to adjust the elastic coefficient and damping coefficient in real time to achieve adaptive adjustment of contact force. S43: Establish an equivalent mass-spring-damping model in Cartesian space, and based on the pose error, correct the increment based on the equivalent mass-spring-damping model and impedance control parameters to obtain the end-effector adjustment force; S44: Based on the joint space dynamics equation, the end-effector adjustment force is converted into a preliminary driving joint torque through the Jacobian matrix; S45: The static balance force of each wire cable is obtained by using a gravity model calculation; S46: Based on the velocity of motion, the dynamic compensation term is obtained through the calculation model of Coriolis force and centrifugal force; S47: The static balance forces of each cable and the dynamic compensation terms are superimposed on the preliminary drive joint torque for compensation, so as to obtain the comprehensive calculated drive joint torque; S48: The calculated driving joint torque is processed by a torque-current conversion algorithm to obtain the corrected end effector desired posture.

[0009] Preferably, in step S5, when there is a periodic pose perturbation of the target tissue, a periodic displacement function model is established, and a path optimization is performed using model predictive control or reinforcement learning strategies to obtain a corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end-effector to apply a reverse displacement in advance to counteract the periodic motion of the target, including: S51: Collect sensor data to obtain the periodic pose perturbation of the target tissue. The sensor data includes target point position data collected by the vision sensor and acceleration data collected by the internal accelerometer. The periodic pose perturbation is the displacement of the target tissue relative to a fixed reference coordinate system. S52: Based on the periodic pose perturbation of the target tissue, the periodic parameters, amplitude parameters, and phase parameters of the physiological motion are obtained through data filtering and period detection. The data filtering and period detection include estimating the main frequency through fast Fourier transform, and then estimating the amplitude and phase using adaptive filtering or least squares method. S53: Based on the periodic parameter, amplitude parameter, and phase parameter, establish a periodic displacement function model to obtain a prediction model; S54: Input the prediction model with the time parameters of future times to obtain the compensation control quantity, which is the reverse displacement correction amount that the robot end needs to apply; S55: Based on the compensation control amount, a path optimization is performed using model predictive control or reinforcement learning strategies to obtain the path optimization result. The reinforcement learning strategy includes updating the output path planning parameters or the adjustment amount of the control gain through the interaction of state, action and reward function. S56: Convert the path optimization results into position and pose commands for each motor, perform online correction on the desired trajectory, and obtain the corrected end-effector desired posture command.

[0010] Preferably, in step S6, the received and corrected end-effector desired attitude command is transformed to obtain a new target position command. This new target position command is used to guide and support the flexible actuator in a deep, tortuous path, including: S61: Based on the flexible actuator, the zero-position length of each drive wire is collected to obtain the reference length data. The zero-position length is the reference length of each drive wire when the flexible actuator is straightened. The flexible actuator includes no less than two series continuous segments. Each series continuous segment includes multiple drive wires, a guide cavity, and a built-in telescopic skeleton. S62: Receive the corrected end-effector desired attitude command, convert the corrected end-effector desired attitude command to obtain the target curvature and azimuth of each segment of the series continuum, and calculate the length change of each drive wire in each segment of the series continuum relative to the zero position length to obtain the change in drive wire length and the change in total drive wire length. S63: Based on the change in the total length of the drive wire, a new target position command is obtained after calculating the motor rotation angle increment. The new target position command is used to guide and support the flexible actuator in the deep tortuous path.

[0011] On the other hand, a force feedback teleoperation control device for the surgical area of ​​a flexible surgical robot is provided. This device is applied to a force feedback teleoperation control method for the surgical area of ​​a flexible surgical robot. The device includes: Master signal module: used to collect signals from the operator's master end, and obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector desired pose based on transformation matrix, scaling factor and attitude mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. Data acquisition module: used to acquire visual image information through the endoscope camera, extract image data of the surgical area, information on potential obstacles and end-effector feature positions, perform preprocessing, and obtain end-effector force data, end-effector pose estimation, tension data, environmental model and artificial potential field model; Dynamic Environment Module: When entering the intraoperative dynamic environment, it determines the deviation state, restricted state, obstacle avoidance state, and temperature safety state, and then performs offset correction to obtain the corrected end-effector desired posture. Tremor module: When tremor is present during surgical procedures, it calculates the pose error and drive joint torque based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. Periodic pose perturbation module: When there is periodic pose perturbation of the target tissue, it establishes a periodic displacement function model, uses model predictive control or reinforcement learning strategy to optimize the path, and obtains the corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end to apply a reverse displacement in advance to counteract the periodic motion of the target. End-effector desired attitude module: used to receive the corrected end-effector desired attitude command, convert the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep tortuous path.

[0012] On the other hand, a flexible surgical robot surgical area force feedback teleoperation control device is provided, the flexible surgical robot surgical area force feedback teleoperation control device includes: a processor; a memory, the memory storing computer-readable instructions, the computer-readable instructions being executed by the processor to implement the method described in any of the above-described flexible surgical robot surgical area force feedback teleoperation control methods.

[0013] On the other hand, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores program code, which can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

[0014] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention provides a force feedback teleoperation control system and method for a flexible surgical robot via natural orifices; it accurately maps to the slave flexible actuator to achieve real-time tactile reproduction; through a physiological motion compensation model, it effectively reduces trajectory drift caused by restriction, tremors, breathing and pulsation; and it can significantly improve surgical safety, operational stability and clinical feasibility. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a force feedback teleoperation control method for a flexible surgical robot in the surgical area, provided by an embodiment of the present invention. Figure 2 This is a flowchart of a control method for a flexible actuator provided in an embodiment of the present invention; Figure 3 This is a block diagram of a force feedback teleoperation control device for the surgical area of ​​a flexible surgical robot provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a flexible surgical robot surgical area force feedback teleoperation control device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0022] This invention provides a method for force feedback teleoperation control of a flexible surgical robot's surgical area. This method can be implemented using a force feedback teleoperation control device for the flexible surgical robot's surgical area, which can be a terminal or a server. Figure 1 The flowchart shown is for a force feedback teleoperation control method for the surgical area of ​​a flexible surgical robot. The processing flow of this method may include the following steps:

[0023] The operator collects signals from the master end and obtains the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector posture based on transformation matrix, scaling factor and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. Preferably, signals from the operator's master end are acquired, and the desired end-effector posture command is obtained through spatial mapping. The spatial mapping method includes mapping the master end pose to the desired end-effector posture based on a transformation matrix, scaling factor, and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link, comprising: The operator collects signals from the main end to obtain the main end raw data. The main end raw data includes position information, attitude information, force / pressure signals, and zero-position attitude data. The zero-position attitude data is obtained by setting the main end zero-position attitude. A hand-eye coordination calibration method is used to calibrate the master-slave coordinate system and obtain a transformation matrix. The transformation matrix is ​​used to determine the transformation relationship between the master operating handle coordinate system, the slave robot base coordinate system and / or the surgical endoscope coordinate system. According to the surgical requirements, the position scaling ratio and / or velocity scaling ratio are set to obtain the scaling coefficient. The scaling coefficient is used to scale the master end motion increment to obtain the target displacement of the slave end actuator. The roll angle, yaw angle and / or pitch angle of the master end are collected, and the angles are mapped using a one-to-one correspondence. An attitude mapping algorithm is set, which is used to calculate the desired attitude change of the end effector of the slave flexible robot. Based on the raw data from the master terminal, signal processing and instruction generation algorithms are used to obtain master terminal control instructions, which include position instructions, attitude instructions, or force control instructions. Based on the spatial mapping method, the master control command is mapped to the desired pose of the slave end. The spatial mapping method includes mapping the master pose to the desired pose of the slave end based on the transformation matrix, the scaling factor and the attitude mapping algorithm. The desired pose of the end point is input into the inverse kinematics algorithm or the virtual joint mapping algorithm to obtain the extension and drive current of each cable segment. The inverse kinematics algorithm includes analytical solution or numerical iterative solution. Based on the extension and retraction of each cable segment and the driving current, the desired end-effector posture command is obtained through encapsulation and transmission via a communication link. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link.

[0024] In some embodiments, the main end is a force feedback operating handle with pose acquisition and force feedback output capabilities, used to acquire the operator's hand pose increment in real time and output interactive force.

[0025] The flexible actuator consists of at least two series continuous segments; each segment contains 3–4 equally distributed drive wires and a hollow guide cavity. The wires are bent via independent drive / tension measurement. Axial extension and retraction are achieved by a linear module or an internal telescopic skeleton. The material uses a superelastic alloy and low-friction bushings to reduce hysteresis and wire-hole friction. Adjustable stiffness joints are installed between segments to allow for on-demand adjustment of distal-end stiffness, thus maintaining sufficient guidance and support in deep, tortuous paths. To ensure accurate correspondence between the master-end motion and the slave-end motion, hand-eye coordination calibration is required: that is, determining the transformation relationship between the master control handle coordinate system, the slave robot base coordinate system, and the surgical endoscope (camera) coordinate system. A transformation matrix is ​​obtained through calibration to transform the position vector of the master control handle to the robot base coordinate system. Typically, a "zero-position" attitude can be set at the master end, and its corresponding position and attitude in the slave coordinate system can be recorded as a reference. Subsequently, every minute increment of motion at the master end is scaled and rotated using a mapping matrix to obtain the target displacement of the slave actuator. During the scaling and mapping process, the operator can set a scaling factor according to the surgical needs. For example, a 1 cm movement of the master end corresponds to a 1 mm movement of the slave end, thus achieving precise operation. The master end's posture (rotation around each axis) is mapped to the desired posture changes of the slave end-effector according to a one-to-one correspondence. For example, the roll, yaw, and pitch angles of the master end handle correspond to the roll, yaw, and pitch rotations of the slave end-effector. To ensure the intuitiveness of the mapping, the system adjusts the axial direction of the master end to be consistent with the coordinate system of the endoscope's field of view through hand-eye calibration—for example, when the operator pushes the handle forward, the robot end-effector in the endoscope also moves forward. The master end's posture, after hand-eye calibration and scaling, is mapped to the desired posture of the slave end-effector. The contact force generated by the interaction between the end-effector and the tissue is acquired, processed, and fed back to the master end handle, thus forming a closed-loop force feedback teleoperation. The signal flow of this master-slave spatial mapping is as follows: the operator applies a handle movement command, which is then transformed by the master-slave mapping module through coordinate and scale transformations, outputting the target posture command signal of the slave end. This command is then sent to the lower controller to drive the flexible robot's movement. The master device is also equipped with a force feedback driver to transmit the force from the end environment to the operator's hand when needed, forming a closed two-way interactive control loop.

[0026] It should be noted that a spatial mapping method using non-uniform scaling + rotation alignment + offset correction is used to map the master end pose to the slave end desired pose; the end desired pose is converted into the cable extension and drive current of each segment through inverse kinematics / virtual joints; the master end force feedback is generated by the superposition of impedance adjustment force and virtual boundary guiding force, which helps the operator maintain fine operation and reduce fatigue.

[0027] Preferably, the master and slave devices use protocols such as UDP for transmission, while the driver and sensor communicate via CAN bus or high-speed serial communication. The software architecture uses the ROS operating system to improve development speed and modular integration.

[0028] Visual image information is acquired through an endoscope camera, and image data of the surgical area, information on potential obstacles, and the location of end-effector features are extracted. After preprocessing, end-effector force data, end-effector pose estimation, tension data, environmental model, and artificial potential field model are obtained. Preferably, visual image information is acquired through an endoscopic camera, and image data of the surgical area, information on potential obstacles, and the location of end-effector features are extracted. Preprocessing is then performed to obtain end-effector force data, end-effector pose estimation, tension data, an environmental model, and an artificial potential field model, including: Visual image information is acquired through an endoscope camera, and denoising and enhancement algorithms are used to process the visual image information to obtain image data of the surgical area, information on potential obstacles, and the location of end-effector features. The image data includes visual features of the surgical target and anatomical structures. Force information is collected by an end force sensor and force preprocessing is performed to obtain end force data. The force preprocessing includes using a low-pass filtering algorithm to eliminate motor vibration noise from the force information to obtain end force data. Temperature information is collected by a temperature sensor and preprocessed to obtain temperature data of the surgical site or instrument surface. The temperature preprocessing includes smoothing the temperature information using a moving average algorithm to obtain temperature data. Based on the robot's kinematics model, the end-effector position is predicted, and the predicted end-effector position is obtained. The end feature position is compared with the predicted end position, and the prediction deviation is corrected to obtain the end pose estimation, which includes the spatial position and attitude angle of the end. The end-effector pose is estimated, and the flexible arm shape is reconstructed to obtain the reconstructed flexible arm shape. Tension data is obtained by collecting cable tension information through a pressure sensor; The environment is modeled using a visual 3D reconstruction algorithm to obtain an environmental model; Based on environmental models and potential obstacle information, prohibited access areas or important structures are identified, and virtual boundaries are set, including safe distance thresholds. The reconstructed flexible arm shape is matched with the environmental model and the target to obtain the relative position information of the end effector in the environment; Based on the virtual boundary, potential field modeling is performed to obtain an artificial potential field model. The artificial potential field model includes a gravitational potential field and a repulsive potential field and their gradients. The gravitational potential field is applied to the end effector by the surgical target, and the repulsive potential field is applied to the end effector by the obstacle. The potential field modeling includes using a gradient calculation method to calculate the position of the obstacle and the position of the robot end effector.

[0029] In some embodiments, virtual boundaries are set for prohibited access areas / important structures during the operation; the navigation perception module calculates the repulsive potential field and its gradient, and the admittance control maps the virtual repulsive force to the end position offset. The joint / cable increment is obtained through inverse kinematics to achieve smooth obstacle avoidance and path fine-tuning, thus realizing "soft constraint" safety protection.

[0030] It should be noted that a mass-spring-damping equivalent model is introduced at the end to convert the expected / actual pose error into adjustment force, and then the adjustment torque / current of each drive is obtained through the inverse Jacobian. Combined with dynamic compensation (gravity / friction / Coriolis term), a torque level closed loop is established to limit the contact load on the tissue and improve compliance and safety.

[0031] It should be further explained that the flexible robot requires the use of various sensors to navigate in confined surgical environments: these include a front-end endoscope camera providing visual images, an end-effector force sensor providing tactile feedback (see Example 3 for details), and a temperature sensor to monitor temperature changes at the surgical site or on the instrument surface. Additionally, there are attitude sensors (such as an IMU) to estimate the bending shape of the robot segments, and pressure sensors to monitor cable tension, etc. This data from various sources is collected by a high-speed communication bus and sent to the main control computer.

[0032] When the fusion status indicates that the force in a certain direction at the end is approaching a threshold, it is determined to be a touch or imminent collision. In this case, the system triggers a local obstacle avoidance strategy, temporarily modifying the direction of movement or stopping forward.

[0033] The vision module identifies surgical targets (e.g., lesion locations) and key anatomical structures from endoscopic images. The fusion result provides the relative position of the robot's end effector to the target. If a deviation from the predetermined path is detected, the navigation module calculates a corrected path to guide the end effector toward the target. During navigation, the vision module also identifies new potential obstacles (e.g., bleeding points, unmarked organ structures), and the system updates the position and repulsive force parameters of these obstacles in the artificial potential field model accordingly.

[0034] Temperature sensors can detect abnormal temperatures in the surgical area, such as a sharp rise in local temperature when using electrocautery instruments. Once the fusion module detects that the temperature exceeds a safe threshold, the navigation system will assume there is a heat source obstruction and issue a warning.

[0035] Multimodal fusion can also infer whether a robot has entered a restricted state. For example, by comparing the expected end-effector movement with the actual IMU deformation, if it is found that the end-effector movement is restricted and the force is increased, it can be determined that the robot may be stuck.

[0036] The above actions are executed in each control cycle based on the latest fused perception results, ensuring that the robot path responds promptly to environmental changes. The system continuously answers questions such as "Where is the end effector now? What's around it? Which way is safer to go next?" and translates the answers into tiny path update instructions, which are then sent to the underlying control execution.

[0037] Once the intraoperative dynamic environment is entered, the deviation state, restricted state, obstacle avoidance state, and temperature safety state are determined, and the offset is corrected to obtain the corrected end effector desired posture. Preferably, upon entering the intraoperative dynamic environment, after determining the entry deviation state, restricted state, obstacle avoidance state, and temperature safety state, offset correction is performed to obtain the corrected end-effector desired posture, including: During the procedure, information is collected in real time using an endoscopic camera, end-effector force sensor, and temperature sensor to obtain the current temperature, current force feedback information, current potential obstacle information, and current relative position of the end-effector in the environment. According to the predetermined route, the relative position information of the current terminal in the environment is judged to obtain the deviation status. When the deviation status indicates that the terminal deviates from the predetermined route, the deviation adjustment strategy is triggered. The relative position information of the current end effector in the environment is compared and analyzed with the actual posture sensor deformation to obtain the deformation difference. Combined with the force data of the end effector, the deformation difference is evaluated to obtain the restricted state determination result. The restricted state determination result is used to indicate whether the end effector has entered the restricted state. When the restricted state determination result indicates that the end is in a restricted state, a local obstacle avoidance strategy is triggered; The real-time force feedback information is compared with a preset force threshold. When the force in a certain direction at the end approaches the preset force threshold, it is determined to be an obstacle avoidance state, and a local obstacle avoidance strategy is triggered. Based on the safety threshold, the temperature information is judged to obtain the temperature safety status. When the temperature safety status indicates that the temperature exceeds the safety threshold, a speed adjustment strategy and a warning signal are triggered. The speed adjustment strategy is used to reduce the forward speed of the end effector or suspend the end effector at a safe distance. When the deviation adjustment strategy, local obstacle avoidance strategy, or speed adjustment strategy is triggered, the current temperature, current force feedback information, current potential obstacle information, and current relative position information of the end in the environment are input into the artificial potential field model to obtain an updated artificial potential field model. The updated artificial potential field model includes updated obstacle position parameters and repulsion force parameters. The updated artificial potential field model is input into the admittance control module to perform force-position mapping and obtain the end position offset. The end position offset is input into the inverse kinematics solver to perform inverse kinematics calculations and obtain the joint angle increment or the cable length increment. Based on the joint angle increment or cable length increment, path planning is performed to obtain the corrected end-effector desired posture.

[0038] In some embodiments, an active obstacle avoidance mechanism combining an artificial potential field method with admittance control is employed. When a flexible robot moves within a narrow cavity, it needs to avoid collisions with surrounding fragile tissue walls. The system first models the potential field of the surgical cavity environment: a "gravitational field" is applied to the robot at the target point, while surrounding obstacles (such as organ walls) are treated as charged particles applying a "repulsive field." When there are no obstacles, the end effector moves in a straight line toward the target; when approaching an obstacle, a lateral component is generated to push the end effector away from the obstacle, thus achieving obstacle avoidance.

[0039] To translate this virtual force into motion commands, an admittance-based obstacle avoidance control strategy is introduced. Admittance control maps force to position / velocity offsets, essentially treating a mass-damped system as a displacement driven by an external force. In software implementation, the obstacle avoidance module operates in a high-frequency loop (e.g., 1kHz), finely adjusting the trajectory, resulting in smooth robot movement that doesn't suddenly deviate from its intended path, as perceived by the operator. Because of the admittance control concept, the operator can always apply larger manual commands to replace the obstacle avoidance force (within safety limits). In other words, the system doesn't absolutely restrict the robot's freedom of movement but provides a flexible constraint, ensuring safety while maintaining the flexibility of manual control.

[0040] When tremors are present during surgical procedures, the pose error and driving joint torque are calculated based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. Preferably, when tremors are present during the surgical procedure, the pose error and driving joint torque are calculated based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture, including: The desired pose of the end point is extracted based on the path update instruction. The pose difference between the desired pose and the actual pose is calculated to obtain the pose error, which includes position deviation and attitude deviation. A parameter adaptive strategy is adopted, taking visual deviation and force feedback signal as inputs to obtain impedance control parameter correction increment. The impedance control parameter correction increment is used to adjust the elastic coefficient and damping coefficient in real time to achieve adaptive adjustment of contact force. An equivalent mass-spring-damping model is established in Cartesian space. Based on the pose error, the increment is corrected according to the equivalent mass-spring-damping model and impedance control parameters to obtain the end-effector adjustment force. Based on the joint space dynamics equations, the end-effector adjustment force is converted into a preliminary driving joint torque using the Jacobian matrix; The static balance forces of each wire were calculated using a gravity model. Based on the velocity of motion, the dynamic compensation term is obtained through the calculation model using Coriolis force and centrifugal force. The static balance forces of each cable and the dynamic compensation terms are superimposed on the preliminary drive joint torque for compensation, resulting in the comprehensively calculated drive joint torque; The calculated driving joint torque is then processed through a torque-current conversion algorithm to obtain the corrected end effector desired posture.

[0041] When there is periodic pose disturbance of the target organization, a periodic displacement function model is established, and path optimization is performed by model predictive control or reinforcement learning strategy to obtain the corrected end-effector expected posture command. The corrected end-effector expected posture command causes the robot end to apply reverse displacement in advance to counteract the periodic motion of the target. Preferably, when there is a periodic pose perturbation of the target tissue, a periodic displacement function model is established, and path optimization is performed using model predictive control or reinforcement learning strategies to obtain a corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end-effector to apply a reverse displacement in advance to counteract the periodic motion of the target, including: Sensor data is collected to obtain the periodic pose perturbation of the target tissue. The sensor data includes target point position data collected by a vision sensor and acceleration data collected by an internal accelerometer. The periodic pose perturbation is the displacement of the target tissue relative to a fixed reference coordinate system. Based on the periodic pose perturbation of the target tissue, the periodic parameters, amplitude parameters, and phase parameters of the physiological motion are obtained through data filtering and period detection. The data filtering and period detection include estimating the main frequency through fast Fourier transform, and then estimating the amplitude and phase using adaptive filtering or least squares method. Based on the periodic parameters, amplitude parameters, and phase parameters, a periodic displacement function model is established to obtain the prediction model; By inputting the time parameters of future moments into the prediction model, the compensation control quantity is obtained. The compensation control quantity is the reverse displacement correction amount that needs to be applied by the robot end effector. Based on the compensation control amount, path optimization is performed using model predictive control or reinforcement learning strategies to obtain path optimization results. The reinforcement learning strategy includes updating the output path planning parameters or the adjustment amount of the control gain through the interaction of state, action and reward function. The path optimization results are converted into position and pose commands for each motor, and the desired trajectory is corrected online to obtain the corrected end-effector desired posture command.

[0042] In some embodiments, a physiological motion compensation mechanism is provided to address the periodic displacement of the target tissue caused by physiological movements such as breathing and heartbeat. In certain minimally invasive surgeries, the target tissue undergoes periodic movements in response to the patient's breathing and heartbeat. For example, during lung surgery, the tumor moves up and down with respiration; during heart surgery, the heart wall contracts and relaxes with the heartbeat. If this periodic path disturbance is not addressed, it can make it difficult for the robotic end effector to accurately maintain its position on the target.

[0043] In one implementation, the compensation mechanism employs model predictive control to perform rolling optimization of the compensation amount for several future steps; in another implementation, a learning-based optimization strategy is used to adjust the compensation parameters online based on historical errors. The compensation process may include: data acquisition, filtering and cycle detection, model updating, compensation amount calculation, execution, and error feedback. First, at the beginning of the surgery, the robot records the target's trajectory relative to a fixed reference coordinate system using vision and sensors. Data over a period of time (e.g., several respiratory cycles) can be used to analyze motion patterns. After obtaining the motion model, the robot controller will activate the motion compensation mode: based on the model's prediction of the target's position change in the future, a reverse displacement correction is applied to the robot's end effector before the actual motion occurs. For example, if it is predicted that the target will move upward by 5mm in the next second, the system will control the robot's end effector to move downward by 5mm in advance, so that the relative displacement cancels out, thereby keeping the end effector stationary relative to the target. This feedforward compensation can be achieved through model predictive control (MPC): in each control cycle, the MPC optimizer sets the reference trajectory that the robot's end effector needs to track in the next few cycles as the inverse of the target's predicted trajectory, and solves for the future control input sequence to minimize the relative error between the end effector and the target. The actuator then applies the control input at the current moment and continuously optimizes the next step. MPC can take into account actuator constraints (such as end effector speed limits) and prediction errors, optimizing control over multiple steps, thus exhibiting good robustness and real-time performance.

[0044] Another approach is reinforcement learning (RL) compensation: physiological motion is treated as an environmental disturbance, which is counteracted by an agent (robot control strategy). During training, a simulated environment can be used for repeated trials, with a reward function encouraging the end effector to maintain alignment with the target. Through deep reinforcement learning algorithms, the agent gradually learns to output lead control when observing a certain respiratory phase or recent motion trend, offsetting any deviation in the next step. The advantage of RL is its ability to automatically adapt to complex nonlinear motion patterns and even learn patient-specific respiratory rhythms online within a certain range. If RL is used, the adaptive module will continuously adjust the strategy parameters during surgery to ensure that the compensation effect is optimized according to possible changes in respiratory patterns. Regardless of whether MPC or RL is used, the core is advance prediction and reverse correction: that is, making the robot move to "follow" the rhythm of the target, but in the opposite direction. Furthermore, compared to simple passive following, active compensation can significantly improve positioning accuracy and operational stability. Taking respiratory compensation as an example, without compensation, the target may move back and forth by ±5mm, causing deviations in surgical operations; after compensation is activated, the end effector can control this relative motion to a smaller range, making the surgeon feel as if the target is stationary, greatly reducing the difficulty of the surgery.

[0045] It is important to note that while physiological movements such as breathing are regular, their amplitude and phase can drift slowly, such as when a patient's breathing deepens or pauses. Therefore, the system still requires operator supervision. When a prediction error continuously exceeds a threshold, an alarm will sound, allowing the operator to recalibrate or pause the compensation mode. Overall, by integrating intelligent predictive control into a master-slave control chain, this invention significantly improves the adaptability of the flexible surgical robot in dynamic human environments, ensuring the accuracy and safety of the surgical procedure.

[0046] like Figure 2 As shown, the modified end-effector desired attitude command is received, and the end-effector desired attitude command is converted to obtain a new target position command. The new target position command is used to guide and support the flexible actuator in a deep, tortuous path.

[0047] Preferably, the process involves receiving a modified end-effector desired attitude command, converting the end-effector desired attitude command to obtain a new target position command, the new target position command being used to guide and support the flexible actuator in a deep, tortuous path, including: Based on the flexible actuator, the zero-position length of each drive wire is collected to obtain the reference length data. The zero-position length is the reference length of each drive wire when the flexible actuator is straightened. The flexible actuator includes no less than two series continuous segments. Each series continuous segment includes multiple drive wires, guide cavities, and built-in telescopic skeletons. The corrected end-effector desired attitude command is received, and the corrected end-effector desired attitude command is converted to obtain the target curvature and azimuth of each segment of the series continuum. The length of each drive wire in each segment of the series continuum is calculated relative to the zero-position length to obtain the change in drive wire length and the change in total drive wire length. Based on the change in the total length of the drive wire, a new target position command is obtained by calculating the motor rotation angle increment. The new target position command is used to guide and support the flexible actuator in a deep, tortuous path.

[0048] In some embodiments, the flexible actuator module adopts a multi-segment series continuum structure, each segment including a guide frame, a hollow guide cavity, and multiple drive wires (preferably 3-4) distributed circumferentially. For example, it consists of several series-connected flexible joint segments, each capable of bending and slightly twisting around its own axis in a two-dimensional plane, thus achieving a serpentine three-dimensional bending effect. Each flexible joint is controlled by a cable drive mechanism, which works by arranging several (3-4) steel wire ropes or elastic push rods as tendons around the joint segment, passing through the joint segment and fixed at its end. When a cable is shortened by a motor, it pulls the joint on that side to contract, causing the joint segment to bend in that direction; through the coordinated control of multiple cables, bending and a certain degree of torsion of the joint in any plane can be achieved.

[0049] Furthermore, to improve drive efficiency and prevent cable slack, each cable is constructed as a closed-loop circuit in the drive unit, and a spring tensioning slider mechanism is added to automatically adjust the slack, ensuring the cable remains taut at all times. The drive motor uses a small DC motor, which outputs traction force and displacement to the cable through a reduction gear or roller winding mechanism. The drive motor control logic can use a position control mode: calculating the target rotation angle of the motor based on the desired cable winding length, allowing the motor to rotate precisely to that position; or, if necessary, a torque control mode: calculating the motor current based on the required output force to apply a specific tension. Typically, during the calibration phase of the flexible continuum robot, the "zero-position length" (i.e., the reference length of each drive cable when the robot is fully extended) of each cable is measured, and the control system records this reference value.

[0050] In terms of control, the system records the zero-position length of each drive wire during the calibration phase. During operation, the slave controller converts the desired end-effector posture into the target curvature and azimuth of each segment, and further converts this into the length change of each drive wire relative to the zero position. This length change can be converted into motor rotation angle or motor displacement commands based on the reel geometry and sent to the driver, achieving closed-loop tracking of the end-effector posture. Specifically, during operation, any desired end-effector posture is converted into the target curvature and rotation angle of each joint segment, and further converted into the length change of each drive wire relative to the zero position. For example, under the constant curvature assumption, if a joint segment needs to bend at a certain angle, the outermost wire on the bending plane needs to be shortened, while the relatively inner wires are lengthened. In terms of specific calculations, assuming a segment has four evenly distributed drive wires, the relationship between the wire length and the joint bending angle can be derived based on geometric relationships. and azimuth The functional relationship between them. The controller will perform this inverse kinematics calculation segment by segment: for each joint segment The system takes the desired bending angle (pitch) and (yaw direction) as input, and calculates the required length change for each cable segment based on the model. The total length change calculated for each cable segment is summed to form the overall control requirement for the entire cable. The system then converts the length change into incremental motor rotation commands. .in Where is the diameter of the motor winding reel; for example, when the radius of the winding reel is... At that time, the length is adjusted by rotating one arc. The required motor angle change is equal to the change in line length divided by meters. Calculations yielded Then, by superimposing the current position of the motor, a new target position command can be obtained and sent to the motor driver.

[0051] It should be noted that the pose error is calculated based on the desired pose and the actual pose of the end effector, and an equivalent mass-spring-damping model is established in Cartesian space to obtain the end effector adjustment amount, which is used to reflect the desired compliance and damping characteristics.

[0052] When the actual position of the end effector deviates from the desired position, the controller calculates an adjustment force based on the error. For example, let the desired position of the end be... The actual location is Then the position error Given impedance model parameters The end-effector adjustment force can be calculated: This force is equivalent to the corrective force exerted by a virtual spring-damped system on the end effector: when the end effector lags behind the target, it generates a pulling force; when it leads, it generates a dragging force; and when there is a speed mismatch, a damping force corrects the speed difference. Then, this force in Cartesian space needs to be converted into force or torque commands on each drive joint / cable. This requires the use of the robot's dynamic parameter model.

[0053] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0054] Figure 3 This is a block diagram illustrating a force feedback teleoperation control device for the surgical area of ​​a flexible surgical robot according to an exemplary embodiment. The device is used in a force feedback teleoperation control method for the surgical area of ​​a flexible surgical robot. (Refer to...) Figure 3 The device includes a master signal module, a data acquisition module, a dynamic environment module, a jitter module, a periodic pose perturbation module, and an end-effector desired pose module.

[0055] Master signal module: used to collect signals from the operator's master end, and obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector desired pose based on transformation matrix, scaling factor and attitude mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. Data acquisition module: used to acquire visual image information through the endoscope camera, extract image data of the surgical area, information on potential obstacles and end-effector feature positions, perform preprocessing, and obtain end-effector force data, end-effector pose estimation, tension data, environmental model and artificial potential field model; Dynamic Environment Module: When entering the intraoperative dynamic environment, it determines the deviation state, restricted state, obstacle avoidance state, and temperature safety state, and then performs offset correction to obtain the corrected end-effector desired posture. Tremor module: When tremor is present during surgical procedures, it calculates the pose error and drive joint torque based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. Periodic pose perturbation module: When there is periodic pose perturbation of the target tissue, it establishes a periodic displacement function model, uses model predictive control or reinforcement learning strategy to optimize the path, and obtains the corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end to apply a reverse displacement in advance to counteract the periodic motion of the target. End-effector desired attitude module: used to receive the corrected end-effector desired attitude command, convert the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep tortuous path.

[0056] A flexible surgical robot surgical area force feedback teleoperation control device includes: a processor; and a memory storing computer-readable instructions. When the computer-readable instructions are executed by the processor, they implement the method described in any of the above-described flexible surgical robot surgical area force feedback teleoperation control methods.

[0057] A computer-readable storage medium, characterized in that the computer-readable storage medium stores program code, the program code being invoked by a processor to execute the method as described in any one of claims 1 to 7.

[0058] Figure 4 This is a schematic diagram of the structure of a flexible surgical robot surgical area force feedback teleoperation control device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the flexible surgical robot surgical area force feedback teleoperation control device may include the above-mentioned Figure 3 The illustrated flexible surgical robot surgical area force feedback teleoperation control device. Optionally, the flexible surgical robot surgical area force feedback teleoperation control device 410 may include a first processor 2001.

[0059] Optionally, the flexible surgical robot surgical area force feedback teleoperation control device 410 may also include a memory 2002 and a transceiver 2003.

[0060] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0061] The following is combined Figure 4 The components of the flexible surgical robot surgical area force feedback teleoperation control device 410 are described in detail below: The first processor 2001 is the control center of the flexible surgical robot surgical area force feedback teleoperation control device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0062] Optionally, the first processor 2001 can perform various functions of the flexible surgical robot surgical area force feedback teleoperation control device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0063] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0064] In a specific implementation, as one example, the flexible surgical robot surgical area force feedback teleoperation control device 410 may also include multiple processors, such as... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0065] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0066] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be controlled via the interface circuit of the flexible surgical robot surgical area force feedback teleoperation control device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0067] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0068] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0069] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be controlled via the interface circuit of the flexible surgical robot surgical area force feedback teleoperation control device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0070] It should be noted that, Figure 4 The structure of the flexible surgical robot surgical area force feedback teleoperation control device 410 shown does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0071] Furthermore, the technical effects of the flexible surgical robot surgical area force feedback teleoperation control device 410 can be referred to the technical effects of the flexible surgical robot surgical area force feedback teleoperation control method described in the above method embodiments, and will not be repeated here.

[0072] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0073] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0074] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0075] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0076] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0077] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0078] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0080] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0081] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0083] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for force feedback teleoperation control of a flexible surgical robot in the surgical area, characterized in that, The method includes: S1: Collect signals from the operator's master end and obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector desired pose based on transformation matrix, scaling factor and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. S2: Visual image information is acquired through the endoscope camera, and image data of the surgical area, information on potential obstacles and end-effector feature positions are extracted. Preprocessing is performed to obtain end-effector force data, end-effector pose estimation, tension data, environmental model and artificial potential field model. S3: When entering the intraoperative dynamic environment, after judging the deviation state, restricted state, obstacle avoidance state, and temperature safety state, the offset is corrected to obtain the corrected end-effector desired posture. S4: When tremors are present during surgical procedures, the pose error and driving joint torque are calculated based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. S5: When there is a periodic pose disturbance of the target organization, establish a periodic displacement function model, use model predictive control or reinforcement learning strategy to optimize the path, and obtain the corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end to apply a reverse displacement in advance to counteract the periodic motion of the target. S6: Receive the corrected end-effector desired attitude command, convert the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep tortuous path.

2. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, The signal acquired by the operator's master end in S1 is used to obtain the desired end-effector posture command through spatial mapping. This spatial mapping method includes mapping the master end pose to the desired end-effector posture based on a transformation matrix, scaling factor, and posture mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link, comprising: S11: Collect signals from the operator's main end to obtain raw data from the main end. The raw data from the main end includes position information, attitude information, force / pressure signals, and zero-position attitude data. The zero-position attitude data is obtained by setting the zero-position attitude of the main end. S12: Using a hand-eye coordination calibration method, the master-slave coordinate system is calibrated to obtain a transformation matrix. The transformation matrix is ​​used to determine the transformation relationship between the master operating handle coordinate system, the slave robot base coordinate system, and / or the surgical endoscope coordinate system. S13: Set the position scaling ratio and / or speed scaling ratio according to the surgical requirements to obtain the scaling coefficient. The scaling coefficient is used to scale the master end motion increment to obtain the target displacement of the slave end actuator. S14: Collect the roll angle, yaw angle and / or pitch angle of the master end attitude, perform angle mapping using a one-to-one correspondence, and set the attitude mapping algorithm. The attitude mapping algorithm is used to calculate the desired attitude change of the end effector of the slave flexible robot. S15: Based on the original data from the master terminal, a master terminal control command is obtained through signal processing and command generation algorithm. The master terminal control command includes a position command, an attitude command, or a force control command. S16: Based on the spatial mapping method, the master control command is mapped to the desired pose of the slave end. The spatial mapping method includes mapping the master pose to the desired pose of the slave end based on the transformation matrix, the scaling factor and the attitude mapping algorithm. S17: Input the desired pose of the end point into the inverse kinematics solution algorithm or the virtual joint mapping algorithm to obtain the extension and driving current of each cable segment. The inverse kinematics solution algorithm includes analytical solution or numerical iterative solution. S18: Based on the extension and retraction of each cable segment and the driving current, the desired end-effector posture command is obtained through communication link encapsulation and transmission. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link.

3. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, The S2 process acquires visual image information via an endoscopic camera, extracts image data of the surgical area, information on potential obstacles, and the location of end-effector features, performs preprocessing, and obtains end-effector force data, end-effector pose estimation, tension data, environmental model, and artificial potential field model, including: S21: Visual image information is acquired through an endoscope camera, and the visual image information is processed using denoising and enhancement algorithms to obtain image data of the surgical area, information on potential obstacles, and the location of end-effector features. The image data includes visual features of the surgical target and anatomical structure. S22: Force information is collected by the end force sensor and force preprocessing is performed to obtain end force data. The force preprocessing includes using a low-pass filtering algorithm to eliminate motor vibration noise from the force information to obtain end force data. S23: Temperature information is collected by a temperature sensor and temperature preprocessing is performed to obtain temperature data of the surgical site or instrument surface. The temperature preprocessing includes smoothing the temperature information using a moving average algorithm to obtain temperature data. S24: Based on the robot's kinematics model, predict the end-effector position to obtain the predicted end-effector position; S25: Compare the end feature position with the predicted end position, correct the prediction deviation, and obtain the end pose estimate, which includes the spatial position and attitude angle of the end. S26: Estimate the end-effector pose and reconstruct the flexible arm shape to obtain the reconstructed flexible arm shape; S27: Acquire tension information of the cable through a pressure sensor to obtain tension data; S28: The environment is modeled using a visual 3D reconstruction algorithm to obtain an environmental model; S29: Based on the environmental model and potential obstacle information, identify prohibited access areas or important structures, and set virtual boundaries, wherein the virtual boundaries include a safe distance threshold; S210: Match the reconstructed flexible arm shape with the environmental model and the target to obtain the relative position information of the end effector in the environment; S211: Based on the virtual boundary, potential field modeling is performed to obtain an artificial potential field model. The artificial potential field model includes a gravitational potential field and a repulsive potential field and their gradients. The gravitational potential field is applied to the end effector by the surgical target, and the repulsive potential field is applied to the end effector by the obstacle. The potential field modeling includes using a gradient calculation method to calculate the position of the obstacle and the position of the robot end effector.

4. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, When S3 enters the intraoperative dynamic environment, it determines the entry deviation state, restricted state, obstacle avoidance state, and temperature safety state, and then performs offset correction to obtain the corrected end effector desired posture, including: S31: During the procedure, information is collected in real time through the endoscope camera, end force sensor and temperature sensor to obtain the current temperature, current force feedback information, current potential obstacle information and current end relative position information in the environment; S32: Based on the predetermined route, determine the relative position information of the current terminal in the environment to obtain the deviation status. When the deviation status indicates that the terminal deviates from the predetermined route, trigger the deviation adjustment strategy. S33: Compare and analyze the current relative position information of the end in the environment with the actual posture sensor deformation to obtain the deformation difference. Combine the end force data to evaluate the deformation difference and obtain the restricted state determination result. The restricted state determination result is used to indicate whether the end has entered a restricted state. S34: When the restricted state determination result indicates that the end is in a restricted state, a local obstacle avoidance strategy is triggered; S35: Compare the real-time force feedback information with a preset force threshold. When the force in a certain direction at the end approaches the preset force threshold, it is determined to be an obstacle avoidance state, and a local obstacle avoidance strategy is triggered. S36: Based on the safety threshold, the temperature information is judged to obtain the temperature safety status. When the temperature safety status indicates that the temperature exceeds the safety threshold, a speed adjustment strategy and a warning signal are triggered. The speed adjustment strategy is used to reduce the forward speed of the end or suspend the end at a safe distance. S37: When the deviation adjustment strategy, local obstacle avoidance strategy, or speed adjustment strategy is triggered, the current temperature, current force feedback information, current potential obstacle information, and current relative position information of the end in the environment are input into the artificial potential field model to obtain an updated artificial potential field model. The updated artificial potential field model includes updated obstacle position parameters and repulsion force parameters. S38: Input the updated artificial potential field model into the admittance control module to perform force-position mapping and obtain the end position offset; S39: Input the end position offset into the inverse kinematics solver to perform inverse kinematics calculations and obtain the joint angle increment or the cable length increment. S310: Based on the joint angle increment or cable length increment, perform path planning to obtain the corrected end-effector desired posture.

5. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, When tremors are present during the surgical procedure, S4 calculates the pose error and driving joint torque based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture, including: S41: Extract the desired end pose based on the path update instruction, calculate the pose difference between the desired end pose and the actual pose, and obtain the pose error, which includes position deviation and attitude deviation. S42: Using a parameter adaptive strategy, visual deviation and force feedback signal are used as inputs to obtain impedance control parameter correction increment. The impedance control parameter correction increment is used to adjust the elastic coefficient and damping coefficient in real time to achieve adaptive adjustment of contact force. S43: Establish an equivalent mass-spring-damping model in Cartesian space, and based on the pose error, correct the increment based on the equivalent mass-spring-damping model and impedance control parameters to obtain the end-effector adjustment force; S44: Based on the joint space dynamics equation, the end-effector adjustment force is converted into a preliminary driving joint torque through the Jacobian matrix; S45: The static balance force of each wire cable is obtained by using a gravity model calculation; S46: Based on the velocity of motion, the dynamic compensation term is obtained through the calculation model of Coriolis force and centrifugal force; S47: The static balance forces of each cable and the dynamic compensation terms are superimposed on the preliminary drive joint torque for compensation, so as to obtain the comprehensive calculated drive joint torque; S48: The calculated driving joint torque is processed by a torque-current conversion algorithm to obtain the corrected end effector desired posture.

6. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, In step S5, when there is a periodic pose disturbance of the target tissue, a periodic displacement function model is established, and path optimization is performed using model predictive control or reinforcement learning strategies to obtain a corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end-effector to apply a reverse displacement in advance to counteract the periodic motion of the target, including: S51: Collect sensor data to obtain the periodic pose perturbation of the target tissue. The sensor data includes target point position data collected by the vision sensor and acceleration data collected by the internal accelerometer. The periodic pose perturbation is the displacement of the target tissue relative to a fixed reference coordinate system. S52: Based on the periodic pose perturbation of the target tissue, the periodic parameters, amplitude parameters, and phase parameters of the physiological motion are obtained through data filtering and period detection. The data filtering and period detection include estimating the main frequency through fast Fourier transform, and then estimating the amplitude and phase using adaptive filtering or least squares method. S53: Based on the periodic parameter, amplitude parameter, and phase parameter, establish a periodic displacement function model to obtain a prediction model; S54: Input the prediction model with the time parameters of future times to obtain the compensation control quantity, which is the reverse displacement correction amount that the robot end needs to apply; S55: Based on the compensation control amount, a path optimization is performed using model predictive control or reinforcement learning strategies to obtain the path optimization result. The reinforcement learning strategy includes updating the output path planning parameters or the adjustment amount of the control gain through the interaction of state, action and reward function. S56: Convert the path optimization results into position and pose commands for each motor, perform online correction on the desired trajectory, and obtain the corrected end-effector desired posture command.

7. The method for force feedback teleoperation control of a flexible surgical robot in the surgical area according to claim 1, characterized in that, The S6 receives the corrected end-effector desired attitude command, transforms the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep, tortuous path, including: S61: Based on the flexible actuator, the zero-position length of each drive wire is collected to obtain the reference length data. The zero-position length is the reference length of each drive wire when the flexible actuator is straightened. The flexible actuator includes no less than two series continuous segments. Each series continuous segment includes multiple drive wires, a guide cavity, and a built-in telescopic skeleton. S62: Receive the corrected end-effector desired attitude command, convert the corrected end-effector desired attitude command to obtain the target curvature and azimuth of each segment of the series continuum, and calculate the length change of each drive wire in each segment of the series continuum relative to the zero position length to obtain the change in drive wire length and the change in total drive wire length. S63: Based on the change in the total length of the drive wire, a new target position command is obtained after calculating the motor rotation angle increment. The new target position command is used to guide and support the flexible actuator in the deep tortuous path.

8. A flexible surgical robot surgical area force feedback teleoperation control device, wherein the flexible surgical robot surgical area force feedback teleoperation control device is used to implement the flexible surgical robot surgical area force feedback teleoperation control method as described in any one of claims 1-7, characterized in that, The device includes: Master signal module: used to collect signals from the operator's master end, and obtain the desired end-effector posture command through spatial mapping. The spatial mapping method includes mapping the master end pose to the slave end-effector desired pose based on transformation matrix, scaling factor and attitude mapping algorithm. The desired end-effector posture command is used to drive the flexible actuator. The communication link includes a wired communication link or a wireless communication link. Data acquisition module: used to acquire visual image information through the endoscope camera, extract image data of the surgical area, information on potential obstacles and end-effector feature positions, perform preprocessing, and obtain end-effector force data, end-effector pose estimation, tension data, environmental model and artificial potential field model; Dynamic Environment Module: When entering the intraoperative dynamic environment, it determines the deviation state, restricted state, obstacle avoidance state, and temperature safety state, and then performs offset correction to obtain the corrected end-effector desired posture. Tremor module: When tremor is present during surgical procedures, it calculates the pose error and drive joint torque based on visual deviation and force feedback signals to obtain the corrected end-effector desired posture. Periodic pose perturbation module: When there is periodic pose perturbation of the target tissue, it establishes a periodic displacement function model, uses model predictive control or reinforcement learning strategy to optimize the path, and obtains the corrected end-effector desired posture command. The corrected end-effector desired posture command causes the robot end to apply a reverse displacement in advance to counteract the periodic motion of the target. End-effector desired attitude module: used to receive the corrected end-effector desired attitude command, convert the end-effector desired attitude command to obtain a new target position command, the new target position command is used to guide and support the flexible actuator in a deep tortuous path.

9. A flexible surgical robot surgical area force feedback teleoperation control device, characterized in that, The flexible surgical robot surgical area force feedback teleoperation control processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images