Variable-curvature endoscope, minimally invasive surgery robot and control method of variable-curvature endoscope and minimally invasive surgery robot

Abstract

The invention discloses a variable-curvature endoscope, a minimally invasive surgery robot and a control method of the variable-curvature endoscope and the minimally invasive surgery robot. The invention discloses an endoscope, a surgical robot and a control method thereof. The endoscope is provided with a bent section (12) and an execution end part (11); a rigid sleeve (2) is arranged in the bent section; a sleeve control mechanism is arranged aiming at the rigid sleeve, and the sleeve control mechanism is used for controlling the rigid sleeve to move back and forth in the bending section; the endoscope is adopted by the minimally invasive surgery robot; the control method comprises the following steps: establishing a virtual environment, and obtaining a bronchial model in the virtual environment for subsequent model training; a reinforcement learning model is established, the reinforcement learning model is established according to actual operation requirements, and a reasonable relation between the robot and the environment is established; and carrying out model training and virtual-reality migration adaptation processing. The endoscope, the surgical robot and the control method of the surgical robot have the advantages of being good in operation flexibility and high in surgical operation accuracy and safety.

Description

Technical Field

[0001] The present invention relates to an endoscope and a surgical robot technology, and in particular to an endoscope with variable curvature, a minimally invasive surgical robot and a control method thereof. Background Art

[0002] In minimally invasive bronchoscopic surgery, the current surgical robot system operation part mainly adopts a design in which a multi-degree-of-freedom robotic arm and a catheter are hinged. The catheter is embedded with an endoscope channel and instrument channels of different specifications. During the operation, the doctor navigates according to the image provided by the instrument endoscope and the pre-planned path, guides the instrument to move in the airway, and performs a series of operations on the terminal instrument through the operating handle. The terminal instrument can usually be bent to provide the doctor with a better field of view and operating space, thereby ensuring the flexibility and accuracy of the surgical operation.

[0003] Currently, the end curvature of common instruments is often constant. The restricted bending movement also limits the doctor's operation in an airway environment with many bends. It is easy to collide with the inner wall during movement, causing discomfort to the patient or even damaging normal tissue.

[0004] Existing surgical robots still rely heavily on the doctor's operating skills. During the navigation process, doctors need to pay close attention to controlling the instrument to avoid touching the inner wall of the airway and causing unnecessary damage. This requires doctors to have certain operating experience, and there is a certain learning threshold for doctors who have little contact with the device or novice doctors.

[0005] In addition, although the robot system has completed reconstruction, registration and path planning before surgery, the navigation effect during surgery may still be affected by the patient's breathing movement, and the actual organ tissue may drift, causing the pre-planned path to be inaccurate. At the same time, factors such as unconscious shaking and misoperation common in manual operations also increase the risk of instrument damage to organs.

[0006] To sum up, the current problems are:

[0007] The existing endoscopes, including bronchoscopes, are designed with a constant curvature at the end. As a result, the flexibility of the end instrument is limited, making it difficult to operate in narrow and tortuous environments, and affecting the accuracy and safety of the surgery. Summary of the invention

[0008] The object of the present invention is to provide an endoscope with variable curvature, a minimally invasive surgical robot and a control method thereof, which have the advantages of good operational flexibility, high surgical operation accuracy and high safety.

[0009] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0010] A variable curvature endoscope having a bending section and an actuating end;

[0011] A rigid sleeve is arranged inside the curved section, and the rigid sleeve can reciprocate in the curved section.

[0012] Furthermore, a sleeve control mechanism is provided for the rigid sleeve, and the sleeve control mechanism is used to control the reciprocating movement of the rigid sleeve in the curved section.

[0013] Furthermore, the specific structural implementation of the sleeve control mechanism includes:

[0014] A return spring is provided for the rigid sleeve, and the return spring can apply a force toward the distal end to the rigid sleeve, so that the rigid sleeve can move toward the distal end and finally abut against the execution end;

[0015] A pulling rope is arranged inside the catheter of the endoscope, and the pulling rope can pull the rigid sleeve toward the proximal end.

[0016] Furthermore, the bending section adopts a serpentine joint structure, and a steering drive system is provided for the serpentine joint structure.

[0017] Furthermore, the endoscope is a bronchoscope.

[0018] A minimally invasive surgical robot adopts the endoscope as mentioned above.

[0019] A method for controlling a minimally invasive surgical robot, wherein the surgical robot is the above-mentioned minimally invasive surgical robot, and the control method comprises:

[0020] S1, establishment of a virtual environment, obtaining a bronchial model in the virtual environment for subsequent model training;

[0021] S2, reinforcement learning model establishment, based on actual surgical needs, build a reinforcement learning model to establish a reasonable connection between the robot and the environment;

[0022] S3, model training, virtual-reality migration adaptation processing.

[0023] Further, step S3 includes:

[0024] S31, initialization: policy network π(a|s), value network V θ (s), experience replay pool, setting hyperparameters and learning rate;

[0025] S32, data acquisition;

[0026] S33, calculate advantage function;

[0027] S34, calculate the target value and update the strategy;

[0028] S35, Update strategy and value network;

[0029] S36, loop iteration, continuously repeating steps S32 to S35 until convergence or reaching a preset maximum number of rounds.

[0030] Further, step S32 includes:

[0031] S321, initialize the environment state s 0 ;

[0032] S322, at each time step t, based on the current strategy π θ (a|s) Select an action a t ;

[0033] S323, execute action a t , observe the reward r returned by the environment t and the next state s t+1 ;

[0034] S324, determine whether the termination condition is met, if so, done = true; otherwise done = false; (s t ,a t ,r t ,s t+1 , done) is stored in the experience pool.

[0035] Compared with the prior art, the endoscope, surgical robot and control method thereof of the present invention have the following beneficial effects:

[0036] 1) A rigid sleeve with a controllable movable position is set in the curved section of the endoscope to control the position of the rigid sleeve in the curved section, so as to achieve the purpose of "controlling a certain part in the curved section to remain straight". In this way, the minimally invasive surgical robot using an endoscope can realize real-time and flexible adjustment of the posture of the endoscope end, especially in a narrow environment where a large angle of bending is required, the posture can be easily adjusted to reach the lesion area.

[0037] 2) Artificial intelligence is introduced into the control method of minimally invasive surgical robots, which can effectively assist the operator in controlling the endoscope, reduce the risk of damage to the inner wall of the human cavity caused by human factors, and ensure the safety of surgical operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 and Figure 2 is a schematic structural diagram of an endoscope with a variable curvature according to the present invention,

[0039] in,

[0040] Figure 1 The rigid sleeve in the curved section is shown in FIG.

[0041] Figure 2 A driving wire and a pulling rope for controlling the bending of the bending segment are shown;

[0042] Figure 3 is a schematic diagram of the rigid sleeve moving in the curved section;

[0043] Figure 4 System closed-loop control block diagram for model training and virtual-reality migration adaptation processing. DETAILED DESCRIPTION

[0044] The present invention will be further described below with specific embodiments:

[0045] See also Figure 1 and Figure 2 This embodiment provides an endoscope with a variable curvature. The endoscope 1 is used as a bronchoscope. This bronchoscope is used to be assembled on a minimally invasive surgical robot to achieve minimally invasive surgery on the bronchi.

[0046] The endoscope 1 of this embodiment has an appearance and structure similar to that of a conventional endoscope, and has an execution end 11 at its distal end. The execution end 11 is provided with a camera (a conventional camera or a fiber optic head) and a lighting source for observing and illuminating structures in the body. An instrument working channel is also provided in the catheter of the endoscope 1 for inserting surgical instruments or performing other operations.

[0047] A section of the endoscope 1 near the distal end is used as a bending section 12, which adopts a serpentine joint structure, and a steering drive system is provided for the serpentine joint structure, specifically a four-way drive wire 4, and a corresponding drive motor is provided for the drive wire 4. In this way, the bending of the bending section 12 can be controlled by manipulating the drive wire 4, so that the distal end (execution end 11) is turned.

[0048] The above structural features belong to the structural features of the prior art, and their specific implementation is common knowledge known to those skilled in the art, so they will not be described in detail.

[0049] The main innovations of the endoscope of this embodiment are:

[0050] A rigid sleeve 2 is disposed inside the bending section 12 . The rigid sleeve 2 is embedded inside the bending section 12 , and the rigid sleeve 2 can reciprocate along the central axis of the bending section 12 within the range of the bending section 12 .

[0051] In the catheter of the endoscope 1, a return spring (not shown in the figure) is also provided for the rigid sleeve 2, one end of which is provided with a hook, which is hooked with the farthest end of the curved section 12, and the other end of the return spring abuts against the proximal end of the rigid sleeve 2. In this way, the return spring can exert a pulling force toward the far end on the rigid sleeve 2, so that the rigid sleeve 2 can move toward the far end and finally abut against the execution end 11; that is, in the absence of external force, the rigid sleeve 2 is at the far end of the curved section 12.

[0052] In the catheter of the endoscope 1, a pulling rope 3 is also provided for the rigid sleeve 2, and the distal end of the pulling rope 3 is connected to the rigid sleeve 2 by a conventional mechanical fastening method. The pulling rope 3 is used to pull the rigid sleeve 2 to overcome the pulling force of the return spring and move toward the proximal end.

[0053] With the cooperation of the pulling rope 3 and the return spring, the rigid sleeve 2 can be controlled to move within the curved section 12 .

[0054] A driving motor is also provided for the pulling rope 3 at the proximal end of the endoscope 1. By controlling the driving motor, the rigid sleeve 2 can be controlled to move back and forth along the central axis of the curved section 12 within the range of the curved section 12, or in other words, the position of the rigid sleeve 2 within the curved section 12 can be controlled.

[0055] For the convenience of description, the driving motor is defined as "sleeve pulling motor".

[0056] The combination of the reset spring, the pulling rope 3 and the sleeve pulling motor described above actually constitutes a control mechanism for the rigid sleeve 2, which can control the rigid sleeve 2 to move back and forth in the curved section 12. For the convenience of description, the control mechanism is defined as a "sleeve control mechanism".

[0057] In the curved section 12, when the rigid sleeve 2 moves to a certain position, since the rigid sleeve 2 is rigid, the position will not bend; that is, the position of the rigid sleeve 2 in the curved section 12 is controlled to achieve the purpose of "controlling a certain part of the curved section 12 to remain straight".

[0058] The endoscope of this embodiment is provided with a rigid sleeve 2, which has the following significance:

[0059] When the minimally invasive surgical robot uses the endoscope 1, by flexibly controlling the different lengths of the traction rope 3 and the four-way drive wire 4, the posture of the end of the instrument can be adjusted in real time and flexibly, especially in the case of multi-level airway narrowness and the need for large-angle bending, the posture of the end of the instrument can be adjusted more conveniently to reach the lesion area. In this way, the problem that traditional instruments are difficult to adjust the posture in a complex airway environment and the instrument control is not flexible enough can be solved, thereby achieving the good effect of the instrument quickly and safely reaching the lesion area, reducing the safety risks caused by human factors, and reducing the learning threshold of the operator.

[0060] This embodiment also provides a minimally invasive surgical robot, which uses the aforementioned endoscope.

[0061] It should be noted that the endoscope is specifically installed on the robotic arm of the minimally invasive surgical robot, and its specific installation structure belongs to the existing technology, so it will not be described in detail.

[0062] In this embodiment, a minimally invasive surgical robot control method is also provided for the minimally invasive surgical robot. The control method can reduce the cognitive burden of the operator, intelligently assist the operator during the operation, achieve flexible control of the distal posture of the endoscope 1, and effectively assist the operator to operate the instrument stably and safely, thereby reducing surgical risks.

[0063] The control method specifically includes steps S1 to S3:

[0064] S1, establishment of a virtual environment, obtaining a bronchial model in the virtual environment for subsequent model training,

[0065] Firstly, the image segmentation algorithm based on the U-Net network model is used to extract the mask of the initial airway model from the preoperative chest CT image, and the 3D reconstruction is performed based on the segmented image to obtain the initial point cloud.

[0066] Secondly, the mask is expanded at three angles, namely the coronal plane, the transverse plane and the sagittal plane, and the original mask is subtracted to obtain an airway model with a hollow structure; the model surface is rendered using a suitable texture, and finally a three-dimensional hollow airway model in a virtual environment can be obtained.

[0067] The VMTK toolkit is used to extract the centerline of the airway model, and the Rapidly-exploring RandomTree path planning algorithm and the artificial potential field trajectory optimization algorithm are used to generate a smooth and uniform center trajectory as a reference trajectory.

[0068] S2, reinforcement learning model establishment, based on actual surgical needs, builds a reinforcement learning model to establish a reasonable connection between the robot and the environment.

[0069] First, the observation space of the agent is defined as:

[0070] S=<I,c>

[0071] Where I is the real-time RGB image captured by the endoscope 1, with a size of 224*224*3, and c is the discrete command given by the doctor on the operating handle, including forward, up, down, left, and right. Next, the action space of the agent is defined as the steering and bending parameters of the curved segment 12: A = <θ 1 ,d,θ 2 >, where θ 1 ,θ 2 are the arc angles corresponding to the two flexible sections of the distal bending section 12, and d is the displacement of the rigid sleeve 2 between the two bending sections relative to the starting point of the distal flexible bending section 12, such as Figure 3 shown.

[0072] Finally, the reward function is designed to expect the agent to keep the curved segment 12 as close to the central reference path as possible, which is specifically expressed as the Fréchet distance between the central reference path and the curved segment 12, which is equal to the distal curved segment 12:

[0073] r=d 0 -d Frechet (C ref ,C real )

[0074] Among them, C ref is the currently selected central reference trajectory that is equal in length to the curved segment 12 and closest to the curved segment 12, and is represented by a combination of the spatial three-dimensional positions of several sampling points. real is the spatial three-dimensional position of a number of sampling points on the curved segment 12, the number of sampling points is consistent and matches each other. 0 Represents C ref The sum of the distances from each sampling point to the nearest airway inner wall.

[0075] S3, model training, virtual-reality migration adaptation processing.

[0076] The agent is trained in a virtual environment. The network of the agent is designed as a stacked structure of a convolutional neural network and a multi-layer perceptron. The PPO algorithm is used for training. The training steps include steps S31 to S36:

[0077] S31, initialization: policy network π(a|s), value network V θ (s), experience replay pool. Set hyperparameters and learning rate.

[0078] S32, data collection (environment interaction): in each round, steps S321 to S324 are executed;

[0079] S321, initialize the environment state s 0 ;

[0080] S322, at each time step t, based on the current strategy π θ (a|s) Select an action a t ;

[0081] S323, execute action a t , observe the reward r returned by the environment t and the next state s t+1 ;

[0082] S324, determine whether the termination condition is met, if so, done = true; otherwise done = false. t ,a t ,r t ,s t+1 , done) is stored in the experience pool.

[0083] S33, calculate the advantage function:

[0084]

[0085] The δ t represents the temporal difference error, which is used to measure the error between the reward in the current state plus the estimated value of the next state and the estimated value of the current state;

[0086] Formula δ t =r t +γV(s t+1 ;θ)-V(s t ; θ),

[0087] s t+1 ,s t Represent the states at time t+1 and time t respectively,

[0088] V(·; θ) represents a neural network determined by the parameter θ, which represents the mapping from a state to its corresponding value, that is, V(s t+1 ;θ) and V(s t ; θ) represents the value of the state at time t+1 and time t respectively;

[0089] γ is a parameter, i.e., a discount factor, which is used to weigh the impact of future rewards and is generally taken as 0.99;

[0090] r t Represents the reward value at time t.

[0091] The A t Represents the advantage function, which is used to measure the pros and cons of the current action and is also used to calculate the subsequent objective function L clip (θ), Formula A t =δ t +(γλ)δ t+1 +(γλ) 2 δ t+2 +…, λ is a smoothing factor parameter used to balance the variance and bias of the estimate A t The influence of , generally can be taken as 0.95.

[0092] S34, calculate the target value and update the strategy. The objective function is:

[0093] L clip (θ) = E t [min(r t (θ)A t ,clip(r t (θ),1-ε,1+ε)A t ]

[0094] in,

[0095] r t (θ) represents the importance sampling ratio: Used to measure the new strategy π θ Relative to the old strategy π θold The change in probability of performing the same action in the same state,

[0096] L clip (θ) represents the objective function that the algorithm wants to optimize, the purpose is to encourage the policy network π θ To optimize in a better direction, the clipping mechanism clip (r t (θ), 1-ε, 1+ε), ε is usually between 0.1 and 0.2, the purpose is to prevent the importance sampling ratio from deviating too far from 1, resulting in a decrease in optimization performance.

[0097] S35, update the policy and value networks. Update the policy network and value network by maximizing the objective function. Use the gradient ascent method for optimization and the Adam optimizer.

[0098] S36, loop iteration, repeating 3.2-2.5 until convergence or reaching the preset maximum number of rounds.

[0099] During the training process, considering the gap between the virtual environment and the real environment, a variety of randomization methods are used to change the virtual environment to avoid overfitting of the model. For example, human discrete commands are added to noise, and factors such as light intensity, photo saturation, hue, and contrast in the virtual environment are randomly adjusted.

[0100] The overall system closed-loop control is as follows Figure 4 shown.

[0101] The control method provided in this embodiment has the following advantages:

[0102] The introduction of artificial intelligence methods can effectively assist the operator in controlling the instrument, further reducing the risk of damage to the inner wall of the airway due to human factors during the passage of the instrument into the bronchus, and effectively lowering the threshold for use of the instrument. Even if the operator is inexperienced or makes many mistakes or human jitters, the instrument can automatically optimize the operation effect based on the images acquired by the current instrument and the operator's operating intentions to ensure the safety of the operation.

[0103] It should be noted that, in the present embodiment, the driving motors configured for the driving wire 4 and the pulling rope 3 are all linear motors.

[0104] The above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An endoscope with a variable curvature, the endoscope having a bending section (12) and an execution end (11); Features: A rigid sleeve (2) is arranged inside the curved section (12), and the rigid sleeve (2) is capable of reciprocating in the curved section (12).

2. The variable curvature endoscope according to claim 1, characterized in that: A sleeve control mechanism is provided for the rigid sleeve (2), and the sleeve control mechanism is used to control the rigid sleeve (2) to move back and forth within the curved section (12).

3. The variable curvature endoscope according to claim 2, characterized in that: The specific structural implementation of the sleeve control mechanism includes: A return spring is provided for the rigid sleeve (2), and the return spring can apply a force toward the distal end to the rigid sleeve (2), so that the rigid sleeve (2) can move toward the distal end and finally abut against the execution end (11); A pulling rope (3) is arranged inside the catheter of the endoscope (1), and the pulling rope (3) can pull the rigid sleeve (2) to move toward the proximal end.

4. The variable curvature endoscope according to claim 1, characterized in that: The bending section (12) adopts a serpentine joint structure, and a steering drive system is provided for the serpentine joint structure.

5. The variable curvature endoscope according to claim 1, characterized in that: The endoscope is a bronchoscope.

6. A minimally invasive surgical robot, characterized in that: The minimally invasive surgical robot adopts the endoscope as described in claim 1.

7. A method for controlling a minimally invasive surgical robot, wherein the surgical robot is the minimally invasive surgical robot according to claim 6, characterized in that: The control method comprises: S1, establishment of a virtual environment, obtaining a bronchial model in the virtual environment for subsequent model training; S2, reinforcement learning model establishment, based on actual surgical needs, build a reinforcement learning model to establish a reasonable connection between the robot and the environment; S3, model training, virtual-reality migration adaptation processing.

8. The minimally invasive surgical robot according to claim 7, characterized in that: Step S3 includes: S31, initialization: policy network π(a|s), value network V θ (s), experience replay pool, setting hyperparameters and learning rate; S32, data acquisition; S33, calculate advantage function; S34, calculate the target value and update the strategy; S35, Update strategy and value network; S36, loop iteration, continuously repeating steps S32 to S35 until convergence or reaching a preset maximum number of rounds.

9. The minimally invasive surgical robot according to claim 8, characterized in that: Step S32 includes: S321, initialize the environment state s0; S322, at each time step t, based on the current strategy π θ (a|s) Select an action a t ; S323, execute action a t , observe the reward r returned by the environment t and the next state s t+1 ; S324, determine whether the termination condition is met, if so, done = true; otherwise done = false; (s t ,a t ,r t ,s t+1 , done) is stored in the experience pool.

Images