A method and system for estimating distal contact force of a flexible endoscope based on proximal torque

Abstract

This invention discloses a method for estimating the distal contact force of a flexible endoscope based on proximal torque. The method includes: acquiring the position, angular velocity, and baseline torque of the proximal motor in free space to construct a temporal state vector containing historical trajectories; constructing a deep neural network based on the Transformer architecture and training it offline to accurately fit the inherent transmission torque caused by nonlinear friction and hysteresis in the linear drive transmission mechanism; inputting proximal kinematic data in real time to predict the current inherent transmission torque, and subtracting it from the total torque measured by the proximal sensor to decouple and extract the distal contact torque; finally, mapping this torque to the teleoperation master controller to achieve force-tactile feedback; and providing an estimation system for executing the above method. This invention eliminates the need for a miniature force sensor installed at the distal end, effectively eliminating the inherent resistance of the system and accurately estimating the distal contact force using only proximal data, significantly reducing surgical risks and improving the surgeon's situational awareness during operation.

Description

Technical Field

[0001] This invention belongs to the field of medical robots and teleoperation control, specifically relating to a method and system for estimating the distal contact force of a flexible endoscope based on proximal torque. Background Technology

[0002] In minimally invasive interventional surgeries such as robot-assisted flexible bronchoscopy, the physical interaction force between the distal endoscope and the luminal tissue is crucial for ensuring surgical safety and preventing iatrogenic injury. However, due to the small size of flexible endoscopes and the strict anatomical constraints of their working environment, directly integrating high-precision micro-force sensors at the distal end of the endoscope faces engineering bottlenecks such as high cost, wiring difficulties, and biocompatibility issues. Current alternatives mostly rely on kinematic or dynamic models to indirectly derive the distal force. However, the transmission mechanism of wire-driven flexible endoscopes has inherent compliance, and its bending and movement are accompanied by highly nonlinear friction and severe hysteresis, resulting in extremely low accuracy of traditional analytical models.

[0003] Therefore, there is an urgent need for an intelligent method that can accurately decouple and estimate the remote interaction torque using only the proximal drive data without the need for a remote force sensor, so as to provide a reliable basis for haptic rendering for teleoperation. Summary of the Invention

[0004] This invention aims to address the problems of existing flexible endoscopes lacking distal tactile feedback and traditional dynamic models being unable to overcome nonlinear friction and hysteresis effects, and provides a method and system for estimating the distal contact force of a flexible endoscope based on proximal torque.

[0005] To achieve the above objectives, this application provides the following technical solution:

[0006] A method for estimating the distal contact force of a flexible endoscope based on proximal torque, comprising the following steps:

[0007] S1. In the free space where the distal end of the endoscope is not in contact with the environment, drive the endoscope to move and simultaneously record the real-time position of the proximal motor. angular velocity and baseline torque measured by the near-end sensor ;

[0008] S2, near the motor position angular velocity The state vector is constructed by converting it into sine and cosine components, and a length of is introduced. A sliding window is used to acquire historical trajectories, and a time-series input sequence is constructed to characterize the hysteresis dynamics of the linear drive transmission. ;

[0009] S3. Construct a Transformer neural network that includes an encoder and a self-attention mechanism to process the time-series input sequence. Mapped to baseline torque, the inherent transmission torque caused by structural bending and internal friction can be predicted online through offline training. The model;

[0010] S4. Real-time acquisition of near-end measurement torque coupled with external load. The real-time kinematic data is input into the model trained in step S3, and the predicted inherent transmission torque is output. By subtracting the measured torque at the near end from the predicted inherent transmission torque, the torque generated by the far end contact is extracted through decoupling. ;

[0011] S5, the torque generated by the decoupled distal contact Mapped to the proximal master controller to generate force tactile feedback.

[0012] Furthermore, in step S2, the timing input sequence Build it as follows:

[0013] Near-end motor position angular velocity Converting to sine and cosine components forms the state vector, which represents the current time step. state vector :

[0014] ;

[0015] The introduced length is A sliding window concatenates the state vectors of consecutive time steps into a time-series input sequence. .

[0016] Furthermore, in step S3, the Transformer neural network uses root mean square error as the loss function during offline training.

[0017] Furthermore, the distal contact torque in step S4 Calculate using the following formula:

[0018] .

[0019] The present invention also provides a flexible endoscope distal contact force estimation system based on proximal torque for performing the above method, the system comprising:

[0020] The baseline data acquisition module is used to synchronously acquire the real-time position of the proximal motor during free-space movement of the distal endoscope without environmental contact. angular velocity and baseline torque measured by the near-end sensor ;

[0021] The timing feature construction module is used to construct the near-end motor position. angular velocity The data is converted into sine and cosine components to form a state vector. A sliding window is then introduced to obtain historical trajectories, constructing a time-series input sequence that characterizes the hysteresis dynamics of the linear drive transmission. ;

[0022] Inherent transmission torque prediction module: Built-in offline trained Transformer neural network model for receiving time-series input sequences. It also outputs the predicted inherent transmission torque in real time. ;

[0023] Remote contact torque decoupling module: used to acquire the near-end measured torque coupled with an external load during actual physical interaction. and compare it with the predicted inherent transmission torque. Difference and decoupling are used to extract the far-end contact torque. ;

[0024] Force haptic feedback output module: used to convert the decoupled distal contact torque After being mapped proportionally, the data is sent to the remote master controller to generate corresponding force and tactile feedback.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. This invention eliminates the need to install any force sensors at the distal end of the flexible endoscope. It can achieve distal force sensing solely based on proximal motor data and sensor data, significantly reducing instrument costs and facilitating the miniaturization of the endoscope.

[0027] 2. This invention adopts a sliding window and Transformer network architecture, which effectively captures the complex temporal dynamics and hysteresis characteristics in the line-driven mechanism. The model has high accuracy and can accurately isolate the inherent resistance of the system.

[0028] 3. The decoupled distal contact torque of the present invention It has a clear physical meaning and can be used as a reference quantity for force and tactile rendering input of the teleoperation master terminal, which significantly improves the doctor's ability to perceive the situation in complex lumens and reduces surgical risks. Attached Figure Description

[0029] Figure 1This is a flowchart of the method for estimating the distal contact force of a flexible endoscope based on proximal torque in this invention;

[0030] Figure 2 This is a module architecture diagram of the flexible endoscope distal contact force estimation system based on proximal torque in this invention. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] This embodiment uses the remote operation of a robot-assisted flexible bronchoscope as an example to provide a method and system for estimating the distal contact force of a flexible endoscope based on proximal torque. Figure 2 As shown, the overall architecture of the system mainly includes a near-end drive device, a core algorithm module, and a physical feedback module. The specific implementation process includes, for example: Figure 1 As shown, the detailed steps are as follows:

[0033] Step 1: Offline baseline data acquisition and feature engineering under no load

[0034] Before surgery or during offline calibration, the flexible endoscope is controlled to perform multiple bending movements in free space without contact with the lumen environment. Combined with... Figure 2 As can be seen from the core algorithm module, the baseline data acquisition module collects the actual position parameters of the near-end motor in real time. and motor angular velocity parameters Simultaneously, the baseline torque (i.e., the measured torque) is acquired by the near-end torque sensor.

[0035] To ensure the continuity and smoothness of the angular features input to the neural network and avoid numerical jumps caused by angular transitions, the system converts the motor position and angular velocity into sine and cosine components to form the current moment's data. state vector :

[0036] ;

[0037] Meanwhile, considering the severe nonlinear hysteresis effect of the linearly driven flexible transmission mechanism, its instantaneous torque is highly dependent on the historical motion trajectory. The system introduces a length of... A sliding window is used to extract time series data, constructing a time series input sequence that includes historical dynamic features. .

[0038] Step 2: Offline training based on the Transformer model

[0039] like Figure 1 As shown in the "Offline Training of Prediction Model" flowchart on the left, the system constructs a Transformer neural network that includes a multi-layer encoder and a multi-head attention mechanism. The time-series input sequence constructed in step one is then used... After linear mapping and the addition of positional encoding, the data is input into the network. The network does not rely on explicit physical mathematical equations, but instead learns directly from massive amounts of data and fits the nonlinear mapping relationship between motor kinematics data and transmission mechanism friction and hysteresis resistance through self-attention computation.

[0040] The root mean square error is used as the loss function, and the network weights are continuously updated through an optimizer until the model converges, resulting in a trained model that can accurately predict inherent transmission torque.

[0041] Step 3: Decoupling and Extraction of Force Sensory Perception During Online Physical Interaction

[0042] like Figure 1 As shown in the "Online Estimation of Predictive Model" flowchart on the right, during actual teleoperative interventional surgery, the distal endoscope enters the patient's lumen. When the endoscope tip physically collides with the inner wall of the lumen, an external contact load is generated. At this time, the proximal sensor collects the total torque in real time. It includes the inherent transmission resistance of the instrument and the force caused by external contact.

[0043] During operation, the system collects motor kinematic data in real time and converts it into a time-series sequence, which is then input into the deployed Transformer model. The system outputs the predicted inherent transmission torque of the endoscope at the current moment online. Subsequently, the total torque was measured from the proximal end using a linear subtraction method. By stripping away the inherent transmission resistance, the pure contact torque caused by distal tissue collision is decoupled. :

[0044] .

[0045] Step 4: Force and Haptic Rendering Feedback

[0046] like Figure 2 As shown in the physical feedback process, the extracted distal contact torque This directly reflects the intensity of interaction between the remote device and the tissue. The system multiplies this value by a preset scaling factor and maps it to the drive torque of the remote master controller. The operator can then feel the corresponding reverse physical resistance when performing master control. This implementation method effectively enhances the operator's sense of presence and prevents iatrogenic tissue damage caused by excessive force, without relying on a remote micro-force sensor.

[0047] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in this application, based on the technical solution and concept of this application, should be included within the scope of protection of this application.

Claims

1. A method for estimating the distal contact force of a flexible endoscope based on proximal torque, characterized in that the steps include... include: S1. In the free space where the distal end of the endoscope is not in contact with the environment, drive the endoscope to move and simultaneously record the real-time position of the proximal motor. angular velocity and baseline torque measured by the near-end sensor ; S2, near the motor position angular velocity The state vector is constructed by converting it into sine and cosine components, and a length of is introduced. A sliding window is used to acquire historical trajectories, and a time-series input sequence is constructed to characterize the hysteresis dynamics of the linear drive transmission. ; S3. Construct a Transformer neural network that includes an encoder and a self-attention mechanism to process the time-series input sequence. Mapped to baseline torque, the inherent transmission torque generated by the combined effects of structural bending and internal friction is obtained through offline training and can be predicted online. The model; S4. Real-time acquisition of near-end measurement torque coupled with external load. The real-time kinematic data is input into the model trained in step S3, and the predicted inherent transmission torque is output. The far-end contact torque is extracted by subtracting the near-end measured torque from the predicted inherent transmission torque and decoupling. ; S5. The decoupled far-end contact torque Mapped to the proximal master controller to generate force tactile feedback.

2. The method for estimating the distal contact force of a flexible endoscope based on proximal torque according to claim 1, characterized in that, The timing input sequence in step S2 Build it as follows: Near-end motor position angular velocity Converting to sine and cosine components forms the state vector, which represents the current time step. state vector : ； The introduced length is A sliding window concatenates the state vectors of consecutive time steps into a time-series input sequence. .

3. The method for estimating the distal contact force of a flexible endoscope based on proximal torque according to claim 1, characterized in that, In step S3, the Transformer neural network uses root mean square error as the loss function during offline training.

4. The method for estimating the distal contact force of a flexible endoscope based on proximal torque according to claim 1, characterized in that, The distal contact torque in step S4 Calculate using the following formula: 。 5. A flexible endoscope distal contact force estimation system based on proximal torque, used to perform the method as described in any one of claims 1 to 4, characterized in that, The system includes: The baseline data acquisition module is used to synchronously acquire the real-time position of the proximal motor during free-space movement of the distal endoscope without environmental contact. angular velocity and baseline torque measured by the near-end sensor ; The timing feature construction module is used to construct the near-end motor position. angular velocity The data is converted into sine and cosine components to form a state vector. A sliding window is then introduced to obtain historical trajectories, constructing a time-series input sequence that characterizes the hysteresis dynamics of the linear drive transmission. ; Inherent transmission torque prediction module: Built-in offline trained Transformer neural network model for receiving time-series input sequences. It also outputs the predicted inherent transmission torque in real time. ; Remote contact torque decoupling module: used to acquire the near-end measured torque coupled with an external load during actual physical interaction. and compare it with the predicted inherent transmission torque. Difference and decoupling are used to extract the far-end contact torque. ; Force haptic feedback output module: used to convert the decoupled distal contact torque After being mapped proportionally, the data is sent to the remote master controller to generate corresponding force and tactile feedback.

Images