Bronchoscope intervention simulation method and system

Abstract

The application discloses a bronchoscope intervention simulation method and system, which comprises the following steps: a simulation platform for simulating a bronchoscope intervention process is constructed in advance, the simulation platform is connected with a handle rocker and a controller, a trachea three-dimensional model and a bronchoscope tip three-dimensional model are constructed and input into the simulation platform, physical parameters and collision constraint parameters are set for the trachea three-dimensional model and the bronchoscope tip three-dimensional model respectively, so as to construct a virtual trachea and a virtual bronchoscope tip which can simulate collision, an electric signal output by the handle rocker is acquired to obtain a real target posture of a real bronchoscope tip, the controller determines a first driving amount for controlling the virtual bronchoscope tip according to a current virtual posture of the virtual bronchoscope tip and the real target posture of the real bronchoscope tip, and the simulation platform controls the virtual bronchoscope tip to move in the virtual trachea according to the first driving amount. The application can be applied to bronchoscope intervention simulation training and verification of controller precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, and in particular to a bronchoscope intervention simulation method and system. BACKGROUND

[0002] Minimally invasive surgery occupies an increasingly important position in modern surgical procedures due to its small trauma, fast recovery and other advantages. However, the operation precision and proficiency of the doctor are extremely high in minimally invasive surgery. Before the doctor actually operates the bronchoscope robot to implement minimally invasive surgery, the doctor needs to adapt to the remote control operation process of the bronchoscope robot through repeated practice. In addition, with the continuous development of surgical robot technology, surgical path planning and bronchoscope control algorithm have become the key to realizing automatic navigation and describing lung nodules. The process of verifying these algorithms also consumes a large amount of experimental resources, increasing the research and development cost.

[0003] There are two mainstream ways for surgical robot algorithm verification and minimally invasive surgery training. One is the traditional animal or human model experiment on a real platform, which has problems such as high cost, limited resources, and inability to repeat training. Limited by the non-reusable medical devices (bronchoscope, biopsy needle, ablation needle) and the scarcity of the laboratory, this training process requires a large amount of human resources and funds.

[0004] The other is a simulation experiment based on the ROS simulation environment and MATLAB / Simulink, which verifies the surgical robot algorithm and conducts minimally invasive surgery training with reproducibility and the ability to repeat training. However, the existing MATLAB-based controller design method cannot provide a real physical engine and has limitations, resulting in that the controller designed by the soft robot model cannot fully consider the disturbance and non-linear soft modeling complexity in the real environment, so that the control effect in the ideal situation deviates greatly from the real effect, and cannot achieve good display effect.

[0005] In addition, in the current surgical robot algorithm verification process, the convergence speed and overshoot of the algorithm need to be tested through the bending test of the bronchoscope tip, which will greatly reduce the service life of the bronchoscope. Moreover, the process of verifying these algorithms consumes a large amount of experimental resources, increasing the research and development cost. Even because of the use and loss of human models and bronchoscope consumables, the training cost is much higher than the traditional method, so it is not adopted.

[0006] The disclosure of the above background art content is only used to assist in understanding the inventive concept and technical solutions of the present application, which does not necessarily belong to the prior art of the present patent application, nor does it necessarily give technical teaching; in the absence of explicit evidence that the above content has been disclosed before the filing date of the present patent application, the above background art should not be used to evaluate the novelty and inventiveness of the present application. Summary of the Invention

[0007] The purpose of the present invention is to provide a bronchoscopic intervention simulation method and system, which can be applied to bronchoscopic intervention simulation training and verification of controller accuracy.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A bronchoscopic intervention simulation method comprises the following steps:

[0010] A simulation platform for simulating a bronchoscope interventional procedure is pre-built, the simulation platform being connected to a joystick and a controller, respectively. The joystick is configured to output an electrical signal corresponding to a target posture of the bronchoscope tip, and the controller is configured to determine a driving amount for controlling the bronchoscope tip to reach the target posture based on the current posture and the target posture of the bronchoscope tip;

[0011] Constructing a three-dimensional tracheal model based on the structure of a real trachea, and establishing a three-dimensional model of a bronchoscope tip based on the structure of a real bronchoscope tip, inputting the three-dimensional tracheal model and the three-dimensional model of the bronchoscope tip into the simulation platform, and setting physical parameters and collision constraint parameters for the three-dimensional tracheal model and the three-dimensional model of the bronchoscope tip, respectively, to construct a virtual trachea and a virtual bronchoscope tip in the simulation platform that can simulate collisions;

[0012] acquiring an electrical signal output by the handle rocker to obtain a real target posture of the real bronchoscope terminal, and determining, by the controller, a first driving amount for controlling the virtual bronchoscope terminal according to the current virtual posture of the virtual bronchoscope terminal and the real target posture of the real bronchoscope terminal;

[0013] The simulation platform controls the movement of the virtual bronchoscope tip in the virtual trachea according to the first driving amount.

[0014] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the controller determines a first driving amount for controlling the virtual bronchoscope terminal according to the current virtual posture of the virtual bronchoscope terminal and the real target posture of the real bronchoscope terminal, comprising the following steps:

[0015] Determining the coordinate system of the virtual bronchoscope end as a virtual coordinate system, determining the coordinate system of the real bronchoscope end as a real coordinate system, and determining a mapping relationship between the virtual coordinate system and the real coordinate system;

[0016] mapping the real target pose of the real bronchoscope tip to the virtual coordinate system according to the mapping relationship to obtain a first virtual target pose of the virtual bronchoscope tip in the virtual coordinate system;

[0017] The controller determines the first driving amount according to the current virtual pose of the virtual bronchoscope tip in the virtual coordinate system and the first virtual target pose.

[0018] Alternatively,

[0019] The controller determines the first driving amount for controlling the virtual bronchoscope tip according to the current virtual pose of the virtual bronchoscope tip and the real target pose of the real bronchoscope tip, and includes the following steps:

[0020] determining that the coordinate system of the virtual bronchoscope tip is a virtual coordinate system, determining that the coordinate system of the real bronchoscope tip is a real coordinate system, and determining a mapping relationship between the virtual coordinate system and the real coordinate system;

[0021] mapping the current virtual pose of the virtual bronchoscope tip to the real coordinate system according to the mapping relationship to obtain a current real pose of the real bronchoscope tip in the real coordinate system;

[0022] The controller determines the first driving amount according to the current real pose of the real bronchoscope tip in the real coordinate system and the real target pose.

[0023] Further, any of the foregoing technical solutions or a combination of multiple technical solutions further includes the following steps:

[0024] Based on the first driving amount, the simulation platform controls the virtual bronchoscope tip to move to a first pose in the virtual bronchus;

[0025] determining the control accuracy of the controller according to the first pose of the virtual bronchoscope tip and the first virtual target pose, or mapping the first pose to the real coordinate system according to the mapping relationship to obtain a first real pose of the real bronchoscope tip in the real coordinate system, and determining the control accuracy of the controller according to the first real pose of the real bronchoscope tip and the first real target pose.

[0026] Further, any of the foregoing technical solutions or a combination of multiple technical solutions further includes the following steps:

[0027] determining the current pose of the virtual bronchoscope tip and a second virtual target pose of the virtual bronchoscope tip;

[0028] The controller determines a second driving amount according to the current posture of the virtual bronchoscope tip and the second virtual target posture;

[0029] Based on the second driving amount, the simulation platform controls the distal end of the virtual bronchoscope to move to a second posture in the virtual trachea;

[0030] Determining the simulation accuracy of the simulation platform according to the second posture of the virtual bronchoscope tip and the second virtual target posture;

[0031] If the simulation accuracy does not meet the preset requirements, the simulation platform is optimized until the simulation accuracy meets the preset requirements; if the simulation accuracy meets the preset requirements, the simulation platform is applied to simulate bronchoscopic intervention.

[0032] Furthermore, any one of the above technical solutions or a combination of multiple technical solutions further includes:

[0033] For the virtual bronchoscope tip, a continuum dynamics model MΔv=dt(f(x,t)) is constructed, where M is the mass matrix of the virtual bronchoscope tip, x is the position of the virtual bronchoscope tip, Δv is the velocity change of the virtual bronchoscope tip, and f(x,t) represents the external force function of the virtual bronchoscope tip;

[0034] The simulation platform is based on the continuum dynamics model MΔv=dt(f(x,t)) and utilizes a forward and inverse differential equation solver or a linear solver to simulate the movement process and bending effect of the virtual bronchoscope tip in the virtual trachea.

[0035] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the controller is configured with an automatic target aiming function, and the automatic target aiming function includes: when the distance between the bronchoscope tip and the target is not greater than a preset distance threshold, the controller controls the bronchoscope tip to automatically aim at the target;

[0036] The automatic aiming target function is simulated by using the simulation platform in the following manner:

[0037] Determining a virtual target point on the virtual trachea and determining the coordinates of the virtual target point;

[0038] Determining the coordinates of the virtual bronchoscope end, and determining the distance from the virtual bronchoscope end to the virtual target point according to the coordinates of the virtual bronchoscope end and the coordinates of the virtual target point;

[0039] When the distance is not greater than a preset distance threshold, the controller determines a third driving amount based on the coordinates of the virtual bronchoscope end, the current posture and the coordinates of the virtual target, and the simulation platform controls the movement of the virtual bronchoscope end according to the third driving amount.

[0040] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the first driving amount is the driving amount of the guide wire, and the guide wire is configured to control the posture of the end of the real bronchoscope;

[0041] Based on the discrete PID controller of linear stepper motor control, the first driving quantity [u1,u2,u3,u4] T Determined by the following formula:

[0042] e i (k)=sd i -s i (k)

[0043] u i (k) = K p *e i (k)+K i *(e i (k)+e i (k-1))+K d *(e i (k)-e i (k-1))

[0044] Among them, u i (k) = [u1,u2,u3,u4] T , i is an integer from 1 to 4, e i (k) represents the guide wire state error at time k, sd i Indicates the guidewire state corresponding to the desired posture, s i (k) represents the guide wire state corresponding to the current posture, e i (k-1) represents the guide wire state error at time k-1, K p is the scale parameter, K i is the integration parameter, K d is the differential parameter.

[0045] Furthermore, according to any one of the above technical solutions or a combination of multiple technical solutions, the first driving amount [u1, u2, u3, u4] is determined. T The following steps are also included:

[0046] The internal force of the guidewire at each step is calculated by the following formula: f(x j )≈f(x j-1 )+K(x j-1)dx, wherein f(x j ) is the internal stiffness force of the guide wire at position x j ) is the internal stiffness force of the guide wire at position x j-1 ) is the internal stiffness force of the guide wire at position x j-1 ) is the internal stiffness force of the guide wire at position x j ) is the internal stiffness force of the guide wire at position x j-1 ) is the internal stiffness force of the guide wire at position x j-1 ) is the tangent stiffness matrix of the guide wire at position x j-1 ) is the tangent stiffness matrix of the guide wire at position x j-1 ) is the tangent stiffness matrix of the guide wire at position x j-1 ) is the tangent stiffness matrix of the guide wire at position x

[0047] The guide wire satisfies static equilibrium in each step by the following formula: -K(x j-1 )dx = p + f(x j-1 ) + J T λ, wherein p represents external force suffered by the guide wire, the external force including gravity, J T represents the transpose of the Jacobian matrix, and λ represents the Lagrange multiplier;

[0048] The finite element model of the guide wire is projected into the constraint space, including:

[0049] The free displacement amount x j-1 of the guide wire and the vector between the actual posture and the expected posture of the virtual bronchoscope tip are found by setting λ = 0 and solving the equation -K(x j-1 )dx = p + f(x T ) + J free λ.

[0050] Mechanics is projected into the constraint space to obtain the minimum possible projection space of the inverse problem: δ e represents the vector based on the position of the constrained virtual bronchoscope tip, J a T represents the transpose of the Jacobian matrix of the driving guide wire, and λ a represents the Lagrange multiplier of the driving guide wire;

[0051] At the end of the time step, the final configuration is corrected using the value of the constraint response, so as to obtain the deformation amount x of the virtual bronchoscope tip and the cable length relationship of the guide wire: x = x free + K -1 J a T λ a , x being the virtual bronchoscope tip position.

[0052] Further, any of the technical solutions or combinations of the technical solutions described above, the simulation platform is configured with a first perspective display module and a global perspective display module; wherein,

[0053] The first perspective display module is configured to display the movement of the virtual bronchoscope tip in the virtual trachea along the direction of movement of the virtual bronchoscope tip in the virtual trachea;

[0054] The global perspective display module is configured to display the virtual trachea in perspective and globally display the movement of the virtual bronchoscope tip in the virtual trachea.

[0055] Further, any of the technical solutions or combinations of the technical solutions described above, further comprising constructing the simulation platform using SOFA, and configuring the trachea three-dimensional model and bronchoscope tip three-dimensional model as soft structures in the SOFA;

[0056] The physical parameters and collision constraint parameters of the trachea three-dimensional model and bronchoscope tip three-dimensional model are set respectively, wherein the physical parameters of the trachea three-dimensional model include one or more of the mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient in the trachea cavity, and elastic modulus of the trachea wall; the physical parameters of the bronchoscope tip include the mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient of the bronchoscope tip; the collision constraint parameters include that the maximum driving speed cannot exceed 15.7 mm / s, and the maximum traction length cannot exceed 14 mm.

[0057] Further, any of the technical solutions or combinations of the technical solutions described above, further comprising constructing the bronchoscope tip three-dimensional model by:

[0058] Constructing a first bronchoscope tip three-dimensional model of the bronchoscope tip using three-dimensional software;

[0059] Grid processing the first bronchoscope tip three-dimensional model to obtain a second bronchoscope tip three-dimensional model that can be recognized by the SOFA, and the second bronchoscope tip three-dimensional model is configured as the bronchoscope tip three-dimensional model;

[0060] And / or,

[0061] Further comprising constructing the trachea three-dimensional model by:

[0062] Pre-acquiring CT images including the pulmonary airway;

[0063] Segmenting the tracheal structure in the CT images using image segmentation technology, and three-dimensionally reconstructing the tracheal structure to obtain a first trachea three-dimensional model;

[0064] performing meshing processing on the first trachea three-dimensional model to obtain a second trachea three-dimensional model that can be recognized by the SOFA, wherein the second trachea three-dimensional model is configured as the trachea three-dimensional model;

[0065] and / or,

[0066] An effector is set at the end of the virtual bronchoscope, and the effector is a sampling point for reading the coordinates and direction of the end of the virtual bronchoscope in real time. The coordinates p of the end of the virtual bronchoscope are read by the effector. t =[x t ,y t ,z t ] and direction d t =[α, β, γ], and determining the current virtual posture of the virtual bronchoscope tip according to the coordinates and direction of the virtual bronchoscope tip.

[0067] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions mentioned above, the control accuracy of the controller is verified using the simulation platform. If the control accuracy of the controller meets the preset control accuracy requirements, the controller is applied to the bronchoscope robot.

[0068] According to another aspect of the present invention, a bronchoscope intervention simulation system is provided, comprising a pre-built simulation platform, the simulation platform being connected to a joystick and a controller, the joystick being configured to output an electrical signal corresponding to a target posture of the bronchoscope tip, the controller being configured to determine a driving amount for controlling the posture of the bronchoscope tip based on the current posture of the bronchoscope tip and the target posture;

[0069] The bronchoscopic intervention simulation system is configured to perform bronchoscopic intervention simulation using the bronchoscopic intervention simulation method described in any one of the above technical solutions or a combination of multiple technical solutions.

[0070] The beneficial effects brought about by the technical solution provided by the present invention are as follows:

[0071] a. This invention uses a pre-established simulation platform to create a highly realistic simulation environment, capable of accurately simulating the visual and collision effects of real interventional procedures. This replaces traditional animal and human model experiments, reduces the consumption of experimental resources, and significantly lowers training costs. Furthermore, the automated setup function of the simulation platform system significantly shortens operational preparation time, improves training efficiency, and achieves low-cost and high-efficiency training.

[0072] b. The bronchoscopic intervention simulation method provided by the present invention can not only be used for simulation training of bronchoscopic intervention procedures, but can also be used to verify and improve the accuracy of the controller in the bronchoscopic robot system, and can also be used to verify and improve the simulation accuracy of the simulation platform itself;

[0073] c. The bronchoscopic intervention simulation method provided by the present invention connects the simulation platform with the bronchoscopic robot remote control device, joystick, and controller. This not only improves the integration of the robot product but also enables multi-degree-of-freedom operation, meeting the user's needs for precise training of complex surgical techniques.

[0074] d. The bronchoscopic intervention simulation method provided by the present invention provides the possibility of repeated practice: doctors can practice repeatedly in the same operation scenario until they fully master the bronchoscopic operation skills. This simulation-based training system breaks through the one-time operation limitation of traditional training methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0076] Figure 1 A schematic diagram of a flow chart of a bronchoscopic intervention simulation method provided by an exemplary embodiment of the present invention;

[0077] Figure 2 A schematic structural diagram of a virtual trachea provided by an exemplary embodiment of the present invention;

[0078] Figure 3 A schematic structural diagram of a virtual bronchoscope tip provided as an exemplary embodiment of the present invention;

[0079] Figure 4 A comparison diagram of a real bronchoscope tip bending posture and a virtual bronchoscope tip bending 45° provided for an exemplary embodiment of the present invention;

[0080] Figure 5 A comparison diagram of a real bronchoscope tip bending posture and a virtual bronchoscope tip bending 90° provided for an exemplary embodiment of the present invention;

[0081] Figure 6 A schematic diagram of a simulation interface of a first perspective and a global perspective provided for an exemplary embodiment of the present invention;

[0082] Figure 7A simulation schematic diagram of an exemplary embodiment of the present invention showing the distal end of a virtual bronchoscope positioned adjacent to a virtual target. DETAILED DESCRIPTION

[0083] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0084] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0085] In addition to the above-mentioned deficiencies in the existing technologies, although existing interventional surgery simulators can provide certain training functions, they also have obvious deficiencies in simulation and operational feel, including: (1) Most bronchoscope controller simulation test schemes cannot reflect the elastic deformation of the bronchoscope during bending; (2) Some lung bronchoscopic intervention simulations have no collision settings, which makes it easy for the bronchoscope to penetrate the lung model; (3) In some simulation schemes, the bronchoscope lung intervention simulation completely follows the tracheal centerline trajectory, and the trajectory is too ideal to reflect the real tracheal navigation scene, which greatly reduces the training effect.

[0086] Therefore, the present invention aims to provide a highly simulated, low-cost, and highly efficient bronchoscope controller testing scenario to solve the problem of large wear and tear on the bronchoscope itself during bronchoscope controller testing. In addition, the present invention provides surgeons with a safe and efficient training platform by constructing a highly realistic surgical environment and simulating the collision and visual effects between the bronchoscope and the tracheal wall during real surgery, thereby improving the success rate of surgery and ensuring the safety of patients during surgery.

[0087] The symbols designed in this invention are defined as follows:

[0088] (1) Bending angle of the bronchoscope tip: θ;

[0089] (2) Deflection angle of the bronchoscope tip:

[0090] (3) Target bending angle of the bronchoscope tip: θ d ;

[0091] (4) Target deflection angle of the bronchoscope tip:

[0092] (5) Spatial coordinates of the end of the bronchoscope: p t =[x t ,y t ,z t ];

[0093] (6) Bronchoscope tip direction vector: d t =[α,β,γ];

[0094] (7) The state quantities of the four guide wires corresponding to the actual posture: s = [s1, s2, s3, s4];

[0095] (8) The state quantities of the four guide wires corresponding to the desired posture: sd = [sd1, sd2, sd3, sd4];

[0096] (9) Arc length of the end of the bronchoscope: l;

[0097] (10) Cross-sectional radius of the bronchoscope tip: R1;

[0098] (11) Plane coordinates of the joystick: p joystick =[x joystick ,y joystick ];

[0099] (12) Bronchoscope robot execution end coordinates: p tip =[x tip ,y tip ,z tip ].

[0100] In one embodiment of the present invention, a bronchoscopic intervention simulation method is provided. Figure 1 , the method comprises the following steps:

[0101] A simulation platform for simulating a bronchoscope interventional procedure is pre-built, the simulation platform being connected to a joystick and a controller, respectively, the joystick being configured to output an electrical signal corresponding to a target posture of the bronchoscope tip, and the controller being configured to determine a driving amount for controlling the posture of the bronchoscope tip based on the current posture of the bronchoscope tip and the target posture;

[0102] A three-dimensional trachea model is constructed based on the structure of a real trachea, and a three-dimensional model of the bronchoscope end is constructed based on the structure of a real bronchoscope end. The three-dimensional trachea model and the three-dimensional model of the bronchoscope end are input into the simulation platform, and physical parameters and collision constraint parameters are set for the three-dimensional trachea model and the three-dimensional model of the bronchoscope end, respectively, so as to construct a virtual trachea and a virtual bronchoscope end in the simulation platform that can simulate collisions. It should be noted that the virtual trachea is not limited to a virtual trachea constructed based on the structure of the human trachea, but also includes a virtual trachea constructed based on the structure of the animal trachea.

[0103] acquiring an electrical signal output by the handle rocker to obtain a real target posture of the real bronchoscope terminal, and determining, by the controller, a first driving amount for controlling the virtual bronchoscope terminal according to the current virtual posture of the virtual bronchoscope terminal and the real target posture of the real bronchoscope terminal;

[0104] The simulation platform controls the movement of the virtual bronchoscope tip in the virtual trachea according to the first driving amount.

[0105] It should be noted that the handle rocker and the controller are both pre-set components of the bronchoscope robot system for controlling the end of a real bronchoscope. In order to be able to realistically simulate the bronchoscope intervention process and verify the accuracy and performance of the bronchoscope robot system, including the accuracy of the controller. The simulation platform in this application is electrically connected to the handle rocker through an I / O interface to obtain the electrical signal output by the handle rocker. The controller can be electrically connected to the simulation platform through another I / O interface, or the controller can be integrated into the simulation platform as a module.

[0106] In this embodiment, there are two ways for the controller to determine the first driving amount for controlling the virtual bronchoscope tip according to the current virtual posture of the virtual bronchoscope tip and the real target posture of the real bronchoscope tip.

[0107] One method is: determining the coordinate system of the virtual bronchoscope end as a virtual coordinate system, determining the coordinate system of the real bronchoscope end as a real coordinate system, and determining the mapping relationship between the virtual coordinate system and the real coordinate system; according to the mapping relationship, mapping the real target posture of the real bronchoscope end to the virtual coordinate system to obtain the first virtual target posture of the virtual bronchoscope end in the virtual coordinate system; the controller determines the first driving amount according to the current virtual posture of the virtual bronchoscope end in the virtual coordinate system and the first virtual target posture.

[0108] After obtaining the first driving amount in this way, based on the first driving amount, the simulation platform controls the virtual bronchoscope end to move to a first posture in the virtual trachea. The control accuracy of the controller is determined based on the first posture of the virtual bronchoscope end and the first virtual target posture. If the control accuracy of the controller meets the preset control accuracy requirements, it is determined that the controller can be applied to the bronchoscopic surgery system. If the control accuracy of the controller does not meet the preset control accuracy requirements, it is necessary to optimize and improve the controller, including optimizing algorithms, optimizing parameters, etc. until the control accuracy of the controller meets the preset control accuracy requirements, and then apply the controller to the bronchoscopic surgery system.

[0109] Another method is: determining the coordinate system of the virtual bronchoscope end as a virtual coordinate system, determining the coordinate system of the real bronchoscope end as a real coordinate system, and determining the mapping relationship between the virtual coordinate system and the real coordinate system; according to the mapping relationship, mapping the current virtual posture of the virtual bronchoscope end to the real coordinate system to obtain the current real posture of the real bronchoscope end in the real coordinate system; the controller determines the first driving amount according to the current real posture of the real bronchoscope end in the real coordinate system and the real target posture.

[0110] After obtaining the first driving amount in this way, based on the first driving amount, the simulation platform controls the virtual bronchoscope end to move to a first posture in the virtual trachea. According to the mapping relationship, the first posture is mapped to the real coordinate system to obtain the first real posture of the real bronchoscope end in the real coordinate system, and the control accuracy of the controller is determined based on the first real posture and the first real target posture of the real bronchoscope end. If the control accuracy of the controller does not meet the preset control accuracy requirements, it is necessary to optimize and improve the controller, including optimizing algorithms, optimizing parameters, etc. until the control accuracy of the controller meets the preset control accuracy requirements, and then apply the controller to the bronchoscopic surgery system.

[0111] In one embodiment of the present invention, the bronchoscopic intervention simulation method further comprises determining and improving the simulation accuracy of the simulation platform by:

[0112] determining a current posture of the virtual bronchoscope tip and a second virtual target posture of the virtual bronchoscope tip;

[0113] The controller determines a second driving amount according to the current posture of the virtual bronchoscope tip and the second virtual target posture;

[0114] based on the second driving amount, the simulation platform controls the virtual bronchoscope tip to move to a second pose in the virtual airway;

[0115] determine the simulation accuracy of the simulation platform according to the second pose of the virtual bronchoscope tip and the second virtual target pose;

[0116] If the simulation accuracy does not meet the preset requirements, the simulation platform is optimized until the simulation accuracy meets the preset requirements; if the simulation accuracy meets the preset requirements, the simulation platform is applied to simulate the bronchoscope intervention process.

[0117] In an embodiment of the present application, the construction process of the simulation platform, virtual airway and virtual bronchoscope tip is as follows.

[0118] The present application adopts open source C++ library SOFA physical engine, OpenGL computer graphics rendering technology and the like to build the simulation platform, and constructs a highly realistic bronchoscope surgery simulation environment. The system can realistically restore the bronchoscope surgery scene and provide a virtual environment for bronchoscope controller testing. The SOFA physical engine is used to realize bending, deformation and collision effect of the virtual bronchoscope, and the OpenGL is used to render the lung environment. In addition, multiple IO interfaces are designed for connection of different controllers and handle joysticks to simulate and test, so as to avoid excessive use of physical bronchoscopes and cause waste.

[0119] It should be noted that in other embodiments, a simulation system based on Unity3D or Unreal Engine engine can be used instead of SOFA physical engine. Unity and Unreal engine can provide similar 3D scene construction and real-time feedback functions through physical plug-ins, although the accuracy of these two engines in simulating soft robot motion is not as good as SOFA, but in some simple scenarios, they can be used as an alternative solution.

[0120] The bronchoscope tip three-dimensional model is constructed in the following way:

[0121] A first bronchoscope tip three-dimensional model of the bronchoscope tip is constructed by using three-dimensional software, for example, according to the design drawing of the bronchoscope, a three-dimensional structure model of the bronchoscope tip is created by using CAD / SolidWorks and the like, it should be noted that the three-dimensional structure model of the bronchoscope tip includes the soft structure of the bronchoscope tip and a plurality of guide wires arranged inside the soft structure for pulling the bronchoscope tip to change the pose of the bronchoscope tip, usually 4 guide wires;

[0122] griding the first bronchoscope tip three-dimensional model to obtain a second bronchoscope tip three-dimensional model in obj data format that can be recognized by the SOFA, the second bronchoscope tip three-dimensional model being configured as the bronchoscope tip three-dimensional model. In one specific embodiment of the present application, the bronchoscope tip three-dimensional model is as shown in Figure 3

[0123] The bronchial three-dimensional model is constructed in the following manner:

[0124] CT images including lung airways are obtained in advance;

[0125] The bronchial structure including lung airways and bronchial three-dimensional surface model in the CT images is segmented by image segmentation technology, and the bronchial structure is three-dimensionally reconstructed to obtain a first bronchial three-dimensional model;

[0126] The first bronchial three-dimensional model is grided by triangular mesh and tetrahedral mesh to obtain a second bronchial three-dimensional model in obj data format that can be recognized by the SOFA, the second bronchial three-dimensional model being configured as the bronchial three-dimensional model. In one specific embodiment of the present application, the bronchial three-dimensional model is as shown in Figure 2

[0127] In this embodiment, the bronchial three-dimensional model and the bronchoscope tip three-dimensional model are configured as soft structure in the SOFA, and physical parameters and collision constraint parameters are set for the bronchial three-dimensional model and the bronchoscope tip three-dimensional model, respectively.

[0128] To simplify the bending calculation of the bronchoscope tip, the first bronchoscope tip three-dimensional model is grided by triangular mesh and tetrahedral mesh by QTetraMesher, so that the continuum structure is converted into a finite grid model to obtain the second bronchoscope tip three-dimensional model, and bending driving is performed by setting a stretchable guide wire. Using finite element method to split and discretize the soft structure can obtain more accurate deformation and position data.

[0129] Meanwhile, the physical parameters of the second bronchoscope tip three-dimensional model are set by using the soft structure calculation characteristics of the SOFA physical engine, including bronchoscope tip mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient, etc. Four deformable guide wires are set as bronchoscope bending driving, and model Young's modulus, Poisson's ratio and other parameters are set to realize control of model deformation performance (in one specific embodiment of the present application, the Young's modulus and Poisson's ratio of the guide wire are set to 206Gpa and 0.28), and external force is set to simulate gravity and disturbance, and based on the above design, the bending state of the bronchoscope in the real environment can be basically restored in the computer. As shown in Figure 3 ​​and Figure 4 As shown in the simulation platform, the virtual bronchoscope tip provided by the application accurately simulates the changes of the real bronchoscope tip, and the simulation accuracy of the simulation platform is very high.

[0130] The control target to be achieved based on the constructed kinematic model is the bending angle θ and the deflection angle The desired length change of the guide wire in the robot and the posture of the bronchoscope tip satisfy a constant curvature model relationship:

[0131]

[0132] Wherein, s1, s2, s3, s4 are state quantities of 4 guide wires for driving the posture change of the bronchoscope tip, κ represents the curvature of the bronchoscope tip, l represents the arc length of the bronchoscope tip, R1 represents the cross-sectional radius of the bronchoscope tip, represents the bending angle.

[0133] An effector is arranged at the virtual bronchoscope tip, the effector is a sampling point for reading the position and direction of the virtual bronchoscope tip in real time, and the effector is configured to obtain the coordinates p t =[x t ,y t ,z t ] and the direction d t =[α,β,γ] of the virtual bronchoscope tip in real time, wherein x t ,y t ,z t respectively represent the coordinates of the corresponding sampling point of the effector in the virtual coordinate system, and α, β, γ represent Euler angles. According to the data, the real-time bending angle θ and the deflection angle Here, set the reference direction vector z = [0, 0, 1], then the calculation formula of the bending angle θ and the deflection angle is as follows:

[0134]

[0135] Wherein, the physical parameters of the tracheal three-dimensional model include one or more of the mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient in the tracheal cavity, and the elastic modulus of the tracheal wall. The collision constraint parameters include that the maximum driving speed cannot exceed 15.7mm / s, and the maximum traction length cannot exceed 14mm.

[0136] The bending angle θ and the deflection angle (i.e. current virtual posture) input the controller, the controller according to the current virtual bronchoscope tip bending angle θ and deflection angle and the target bending angle θ corresponding to the target virtual posture of the virtual bronchoscope tip d and the target deflection angle Calculate the driving amount required to make the virtual bronchoscope tip reach the desired posture, i.e. the target virtual posture, and input it to the virtual bronchoscope tip.

[0137] Wherein the target bending angle θ corresponding to the target virtual posture d and the target deflection angle Determined according to the electrical signal output by the joystick. In an embodiment of the present application, the joystick is accessed through the IO interface provided by the pygame development library to realize the input of the electrical signal output by the joystick into the simulation platform. Based on the mapping relationship between the joystick and the target posture of the bronchoscope tip, it is ensured that the joystick and the button can realize the corresponding motion effect. The buttons on the side of the handle are used to control the advancement and contraction of the bronchoscope, and the joystick controls the bending deformation of the bronchoscope robot tip. The maximum bending angle of the bronchoscope tip is limited to 90 degrees, and the mapping relationship between the joystick and the bending angle obtained is as follows:

[0138]

[0139]

[0140] Wherein, x joystick and y joystick are the output values of the joystick, x joystick and y joystick have a corresponding relationship with the horizontal and vertical coordinates of the joystick, when the absolute value of the output value of the joystick is less than or equal to 0.1, directly set θ = 0, This setting is used to set the dead zone of the joystick.

[0141] For the virtual bronchoscope tip, a continuum dynamics model MΔv = dt(f(x, t)) is constructed, where M is the mass matrix of the virtual bronchoscope tip, x is the degree of freedom of the virtual bronchoscope tip, Δv is the velocity change of the virtual bronchoscope tip, and f(x, t) represents the external force function of the virtual bronchoscope tip;

[0142] The simulation platform simulates the motion process and bending effect of the virtual bronchoscope tip in the virtual trachea based on the continuum dynamics model MΔv = dt(f(x, t)) using a forward and inverse differential equation solver or a linear solver. Specifically, the corresponding forward and inverse differential equation solver or linear solver of the SOFA software simulation platform is used to calculate the motion process and bending effect of the virtual bronchoscope tip in the lung.

[0143] The simulation platform is configured with a first perspective display module and a global perspective display module. The first perspective display module is configured to display the movement of the virtual bronchoscope tip in the virtual airway along the direction of movement of the virtual bronchoscope tip in the virtual airway. The global perspective display module is configured to display the virtual airway in perspective and globally display the movement of the virtual bronchoscope tip in the virtual airway. As Figure 6 The left side illustrates the advancement of the bronchoscope in the pulmonary airway based on SOFA displayed by the first perspective display module, as Figure 6 The right side illustrates the global state of the bronchoscope in the pulmonary airway based on SOFA displayed by the second perspective display module.

[0144] In an embodiment of the present application, the controller is configured with an automatic target aiming function, which includes: when the distance between the bronchoscope tip and the target is not greater than a preset distance threshold, the controller controls the bronchoscope tip to automatically aim at the target. When the bronchoscope tip approaches the target, manual remote control input is terminated.

[0145] In this embodiment, the automatic target aiming function is simulated by the simulation platform in the following manner:

[0146] A virtual target is determined on the virtual airway, and the coordinates of the virtual target are determined;

[0147] The coordinates of the virtual bronchoscope tip are determined, and the distance between the virtual bronchoscope tip and the virtual target is determined according to the coordinates of the virtual bronchoscope tip and the coordinates of the virtual target;

[0148] When the distance is not greater than a preset distance threshold, the controller determines a third driving amount according to the coordinates of the virtual bronchoscope tip, the current posture, and the coordinates of the virtual target, and the simulation platform controls the movement of the virtual bronchoscope tip according to the third driving amount.

[0149] According to the coordinates of the virtual target (target lung nodule) in the initial lung model and the coordinates of the virtual bronchoscope tip [x tip ,y tip ,z tip ] read in real time in the software, the expected value of the target posture of the software robot is solved using a geometric method and the corresponding expected length change amount [s1, s2, s3, s4] T . Finally, the automatic aiming function of the bronchoscope tip is realized using an inverse solver or a PID or model predictive control algorithm.

[0150] In this embodiment, the process of using a general quadratic programming inverse solver and a PID controller to solve the desired length change and driving amount of the guide wire is as follows.

[0151] The finite element model (FEM) of the guidewire is projected into the constraint space. The finite element model of the guidewire is a set of elements connected only at nodes, transmitting forces only at nodes, and constrained only at nodes. The nonlinear geometric deformation is converted into an iterative linearization problem. In each step of the simulation, the linearized internal force of the guidewire is calculated:

[0152] f(x j )≈f(x j-1 )+K(x j-1 )dx, where f(x j ) is the position x of the guide wire j The internal stiffness force at j-1 ) is the position x of the guide wire j-1 Internal stiffness force at x j and x j-1 are two adjacent nodes on the guidewire, K(x j-1 ) is the tangent stiffness matrix that depends on the actual position of the node, K(x j-1 ) is the position x of the guide wire j-1 The corresponding tangent stiffness matrix at .

[0153] The guidewire is kept in static equilibrium at each step by the following formula:

[0154] -K(x j-1 )dx=p+f(x j-1 )+J T λ, where p represents the external force on the guidewire, including gravity, and J T represents the transpose of the Jacobian matrix, and λ represents the Lagrange multiplier.

[0155] Projecting the finite element model of the guidewire into the constraint space includes:

[0156] (1) By setting λ = 0 and solving the equation -K(x j-1 )dx=p+f(x j-1 )+J T λ to find the free displacement x of the guidewire free , that is, how the system would move in the absence of constraints, and the vector between the actual and desired positions of the effector is

[0157] (2) Project the mechanics into the constraint space to obtain the minimum possible projection space of the inverse problem: δ e represents a vector based on the position of the virtual bronchoscope tip after constraint, J a T represents the transpose of the Jacobian matrix of the driving guide wire, λ a represents the Lagrange multiplier of the driving guide wire;

[0158] (3) At the end of the time step, the final configuration is corrected using the value of the constraint response, so that the deformation of the virtual bronchoscope tip and the relationship between the guide wire and the pull length are obtained: x = x free + K -1 J a T λ a , x is the position of the virtual bronchoscope tip.

[0159] wherein step (2) is the core step in the inverse solver, J e K -1 J a T is in the form of a quadratic convex function, which involves an optimization method to solve the minimum value. The general form of the solution of the quadratic programming problem is:

[0160]

[0161] The traditional motor control algorithm is based on the control of the traction length based on the input torque. The output of the linear stepper motor and the controllable object of the simulation platform are the change amount and the change rate of the guide wire. In actual operation, considering the limitation of the motor speed mode, the position output mode is preferably adopted.

[0162] The driving amount of the guide wire (including the first driving amount and the second driving amount) is expressed by the following formula based on the discrete PID controller of the linear stepper motor control:

[0163] e i (k) = sd i -s i (k)

[0164] u i (k) = K p *e i (k) + K i *(e i (k) + e i (k-1)) + K d *(e i (k) - e i (k-1))

[0165] wherein e i (k) represents the guide wire state error at time k, sd irepresents the guide wire state quantity corresponding to the desired pose, s i (k) represents the guide wire state quantity corresponding to the current pose, e i (k-1) represents the guide wire state quantity error at the k-1 time, K p K is a proportional parameter, i K is an integral parameter, d is a differential parameter.

[0166] Finally, whether the driving quantity input into the virtual bronchoscope tip can drive the virtual bronchoscope tip to reach the target virtual pose is observed. Figure 4 is a comparison diagram of the bending pose of the real bronchoscope tip and the bending of the virtual bronchoscope tip by 45°, Figure 5 is a comparison diagram of the bending pose of the real bronchoscope tip and the bending of the virtual bronchoscope tip by 90°, and it can be seen that the virtual bronchoscope tip can basically restore the bending effect of the real bronchoscope tip, and therefore the simulation platform can be used for testing the controller.

[0167] The bronchoscope intervention simulation method provided by the application not only has high simulation, can reliably and accurately simulate the bronchoscope surgery process, is used for the training of bronchoscope surgery, but also can test the performance of the bronchoscope controller, can reduce the test times of the bronchoscope entity, and reduce the bronchoscope loss. As described in the above embodiment, the virtual bronchoscope bending state is estimated by the virtual bronchoscope tip position coordinates and the direction, the real-time coordinates and the direction can be directly obtained by setting a tracking point in the virtual bronchoscope tip in the Sofa physical engine. And a controller interface (such as a PID controller, a fuzzy logic controller) is provided, the driving quantity required to change the pose of the bronchoscope tip is calculated by using the controller, and then the driving quantity is input into the virtual bronchoscope tip to drive the virtual guide wire, and the deformation state of the virtual bronchoscope tip is updated, so that the control effect of the controller can be obtained without using the real object, and the loss of the real bronchoscope is greatly reduced.

[0168] Based on the positioning of the bronchoscope tip coordinates, the virtual bronchoscope bending state is estimated, the bronchoscope driving quantity (the length of the wire required by the guide wire) is updated by using the controller, the virtual bronchoscope is input, and the bronchoscope bending state is updated, so that the performance test of the bronchoscope bending controller without using the entity bronchoscope is realized; by importing the human lung model and controlling the external joystick, the bronchoscope is controlled to advance, retreat and bend in the collision lung model in the simulation environment. The bronchoscope first perspective and the lung perspective global perspective are provided in the interactive interface, which assists the doctor to complete the exploration and training of each branch of the lung.

[0169] In addition, when the bronchoscope tip is about to reach the nodule position, i.e., the target point, the automatic aiming simulation mode can be switched to, the bronchoscope tip will automatically aim at the target point, and the aiming accuracy depends on the performance of the controller used for testing, and in this case, a PID controller is used. Based on the above scheme, a safe and efficient training platform is provided for surgeons to help them master the operation skills of the bronchoscope robot surgery system.

[0170] The bronchoscope intervention simulation method provided by the application has the following advantages:

[0171] Highly simulated simulation environment: a highly realistic surgical environment is constructed through computer graphics, topology, etc., which can highly simulate the visual effect, collision effect, etc. in real surgery;

[0172] Multi-degree-of-freedom remote control device: a simulation platform interface is provided for the bronchoscope surgery robot remote control device, which not only improves the integration of the robot product, but also realizes multi-degree-of-freedom operation and accurate training of complex surgical skills;

[0173] Low-cost and efficient training: the simulation system replaces the traditional animal experiment and human model experiment, reduces the consumption of experimental resources, and greatly reduces the training cost; at the same time, the automatic setting function of the system greatly shortens the operation preparation time and improves the training efficiency;

[0174] Possibility of multiple repeated practice: the doctor can repeatedly practice in the same operation scene until he completely masters the bronchoscope operation skill. This simulation-based training system breaks through the one-time operation limitation in the traditional training method.

[0175] In one embodiment of the application, a bronchoscope intervention simulation system is provided, comprising a pre-constructed simulation platform, the simulation platform is connected with a handle rocker and a controller, the handle rocker is configured to output an electrical signal corresponding to a target pose of the bronchoscope tip, and the controller is configured to determine a driving amount for controlling the pose of the bronchoscope tip according to the current pose and the target pose of the bronchoscope tip.

[0176] In this embodiment, the bronchoscope intervention simulation system is configured to use the bronchoscope intervention simulation method as described in any one of the above embodiments or a combination of multiple embodiments for bronchoscope intervention simulation.

[0177] It should be noted that the bronchoscope intervention simulation system embodiment and the bronchoscope intervention simulation method embodiment are based on the same inventive concept, and the entire content of the bronchoscope intervention simulation method is incorporated into the bronchoscope intervention simulation system embodiment by reference.

[0178] It is to be noted that, as used in this specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component" can include a combination of two or more components, and the term "an element" can include comparable reference to a plurality of elements. Also, as used in this specification and the appended claims, the term "or" as used in the context of "A / B or C" means any of the following: A; B; or C. Also, the term "comprising" as used in the claims should not be interpreted as being restricted to the means listed thereafter; it does not exclude other elements or steps. It is thus to be interpreted as specifying the presence of the stated features, integers, steps or components as referred to, but does not preclude the presence or addition of one or more other features, integers, steps or components, or groups thereof. Moreover, it is to be understood that the application can be carried out by specifically designed hardware- and / or software-configured modules, by a suitably programmed computer, or by a combination of hardware and software modules.

[0179] The foregoing is considered only as a specific implementation of the present application, and it is understood that numerous changes and modifications can be made by those skilled in the art without departing from the spirit and principles of the application, and the scope of which is to be measured only by the claims that follow.

Claims

1. A bronchoscopic intervention simulation method, characterized in that: The following steps are involved: A simulation platform for simulating a bronchoscope interventional procedure is pre-built, the simulation platform being connected to a joystick and a controller, respectively. The joystick is configured to output an electrical signal corresponding to a target posture of the bronchoscope tip, and the controller is configured to determine a driving amount for controlling the bronchoscope tip to reach the target posture based on the current posture and the target posture of the bronchoscope tip; Constructing a three-dimensional tracheal model based on the structure of a real trachea, and establishing a three-dimensional model of a bronchoscope tip based on the structure of a real bronchoscope tip, inputting the three-dimensional tracheal model and the three-dimensional model of the bronchoscope tip into the simulation platform, and setting physical parameters and collision constraint parameters for the three-dimensional tracheal model and the three-dimensional model of the bronchoscope tip, respectively, to construct a virtual trachea and a virtual bronchoscope tip in the simulation platform that can simulate collisions; acquiring an electrical signal output by the handle rocker to obtain a real target posture of the real bronchoscope terminal, and determining, by the controller, a first driving amount for controlling the virtual bronchoscope terminal according to the current virtual posture of the virtual bronchoscope terminal and the real target posture of the real bronchoscope terminal; The simulation platform controls the movement of the virtual bronchoscope tip in the virtual trachea according to the first driving amount; The controller is configured with an automatic target aiming function, and the automatic target aiming function includes: when the distance between the end of the bronchoscope and the target is not greater than a preset distance threshold, the controller controls the end of the bronchoscope to automatically aim at the target; The automatic aiming target function is simulated by using the simulation platform in the following manner: Determining a virtual target point on the virtual trachea and determining the coordinates of the virtual target point; Determining the coordinates of the virtual bronchoscope end, and determining the distance from the virtual bronchoscope end to the virtual target point according to the coordinates of the virtual bronchoscope end and the coordinates of the virtual target point; When the distance is not greater than a preset distance threshold, the controller determines a third driving amount based on the coordinates of the virtual bronchoscope end, the current posture and the coordinates of the virtual target, and the simulation platform controls the movement of the virtual bronchoscope end according to the third driving amount.

2. The bronchoscopic intervention simulation method according to claim 1, characterized in that: The controller determines a first driving amount for controlling the virtual bronchoscope tip according to the current virtual posture of the virtual bronchoscope tip and the real target posture of the real bronchoscope tip, comprising the following steps: Determining the coordinate system of the virtual bronchoscope end as a virtual coordinate system, determining the coordinate system of the real bronchoscope end as a real coordinate system, and determining a mapping relationship between the virtual coordinate system and the real coordinate system; According to the mapping relationship, the real target posture of the real bronchoscope terminal is mapped to the virtual coordinate system to obtain a first virtual target posture of the virtual bronchoscope terminal in the virtual coordinate system; The controller determines the first driving amount according to the current virtual posture of the virtual bronchoscope tip in the virtual coordinate system and the first virtual target posture; or, The controller determines a first driving amount for controlling the virtual bronchoscope tip according to the current virtual posture of the virtual bronchoscope tip and the real target posture of the real bronchoscope tip, comprising the following steps: Determining the coordinate system of the virtual bronchoscope end as a virtual coordinate system, determining the coordinate system of the real bronchoscope end as a real coordinate system, and determining a mapping relationship between the virtual coordinate system and the real coordinate system; According to the mapping relationship, mapping the current virtual posture of the virtual bronchoscope tip to the real coordinate system to obtain the current real posture of the real bronchoscope tip in the real coordinate system; The controller determines the first driving amount according to the current real posture of the real bronchoscope tip in the real coordinate system and the real target posture.

3. The bronchoscopic intervention simulation method according to claim 2, characterized in that: The following steps are also included: Based on the first driving amount, the simulation platform controls the distal end of the virtual bronchoscope to move to a first posture in the virtual trachea; The control accuracy of the controller is determined based on the first posture of the virtual bronchoscope end and the first virtual target posture; or, based on the mapping relationship, the first posture is mapped to the real coordinate system to obtain the first real posture of the real bronchoscope end in the real coordinate system, and the control accuracy of the controller is determined based on the first real posture and the first real target posture of the real bronchoscope end.

4. The bronchoscopic intervention simulation method according to claim 1, characterized in that: The following steps are also included: determining a current posture of the virtual bronchoscope tip and a second virtual target posture of the virtual bronchoscope tip; The controller determines a second driving amount according to the current posture of the virtual bronchoscope tip and the second virtual target posture; Based on the second driving amount, the simulation platform controls the distal end of the virtual bronchoscope to move to a second posture in the virtual trachea; Determining the simulation accuracy of the simulation platform according to the second posture of the virtual bronchoscope tip and the second virtual target posture; If the simulation accuracy does not meet the preset requirements, optimizing the simulation platform until the simulation accuracy meets the preset requirements; If the simulation accuracy meets the preset requirements, the simulation platform is applied to simulate the bronchoscopic intervention process.

5. The bronchoscopic intervention simulation method according to claim 1, characterized in that: Also includes: Construct a continuum dynamics model for the virtual bronchoscope end , where M is the mass matrix of the virtual bronchoscope end, x is the position of the virtual bronchoscope end, is the velocity change of the end of the virtual bronchoscope, represents the external force function of the distal end of the virtual bronchoscope; The simulation platform is based on the continuum dynamics model , using a forward and inverse differential equation solver or a linear solver to simulate the movement process and bending effect of the virtual bronchoscope tip in the virtual trachea.

6. The bronchoscopic intervention simulation method according to claim 1, characterized in that: The first driving amount is the driving amount of the guide wire, and the guide wire is configured to control the posture of the end of the real bronchoscope; Based on the discrete PID controller of linear stepper motor control, the first driving quantity Determined by the following formula: ； ； in, , i is an integer from 1 to 4, express The guide wire state error at the moment, Indicates the guidewire state quantity corresponding to the desired posture, Indicates the guide wire state corresponding to the current posture, express The guide wire state error at the moment, K p is the scale parameter, K i is the integration parameter, K d is the differential parameter.

7. The bronchoscopic intervention simulation method according to claim 6, characterized in that: Determine the first driving amount The following steps are also included: The internal force of the guidewire at each step is calculated using the following formula: ,in, Position of the guidewire The internal stiffness force at Position of the guidewire The internal stiffness force at and are two adjacent nodes on the guide wire, is the tangent stiffness matrix that depends on the actual position of the nodes, yes Position of the guidewire The corresponding tangent stiffness matrix at ; The guidewire is kept in static equilibrium at each step by the following formula: , where p represents the external force on the guidewire, and the external force includes gravity, represents the transpose of the Jacobian matrix, represents the Lagrange multiplier; Projecting the finite element model of the guidewire into the constraint space includes: By setting And solve the equation To find the free displacement of the guide wire , and the vector between the actual posture and the desired posture of the virtual bronchoscope tip is ; Project the mechanics into the constraint space to obtain the smallest possible projection space for the inverse problem: , The vector representing the position of the virtual bronchoscope tip based on the constraint, represents the transpose of the Jacobian matrix driving the guide wire, represents the Lagrange multiplier driving the guidewire; At the end of the time step, the value of the constraint response is used to correct the final configuration, thereby obtaining the relationship between the deformation amount of the virtual bronchoscope tip and the pull wire length of the guidewire: , x is the end position of the virtual bronchoscope.

8. The bronchoscopic intervention simulation method according to claim 1, characterized in that: The simulation platform is configured with a first-view display module and a global-view display module; wherein, The first viewing angle display module is configured to display a moving picture of the virtual bronchoscope tip in the virtual trachea along the moving direction of the virtual bronchoscope tip in the virtual trachea; The global viewing angle display module is configured to display the virtual trachea in a perspective manner and to globally display a moving picture of the virtual bronchoscope tip in the virtual trachea.

9. The bronchoscopic intervention simulation method according to claim 1, characterized in that: The method also includes constructing the simulation platform using SOFA, and configuring the trachea three-dimensional model and the bronchoscope end three-dimensional model as a software structure in the SOFA; Physical parameters and collision constraint parameters are set for the three-dimensional model of the trachea and the three-dimensional model of the bronchoscope end, respectively, wherein the physical parameters of the three-dimensional model of the trachea include one or more of the mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient in the tracheal cavity, and elastic modulus of the inner wall of the trachea; the physical parameters of the bronchoscope end include the mass, density, Young's modulus, Poisson's ratio, friction coefficient and damping coefficient of the bronchoscope end; and the collision constraint parameters include that the maximum driving speed cannot exceed 15.7 mm / s and the maximum traction length cannot exceed 14 mm.

10. The bronchoscopic intervention simulation method according to claim 9, characterized in that: The invention also includes constructing the three-dimensional model of the bronchoscope tip by the following methods: constructing a first bronchoscope tip three-dimensional model of the bronchoscope tip using three-dimensional software; performing meshing processing on the first bronchoscope terminal three-dimensional model to obtain a second bronchoscope terminal three-dimensional model that can be recognized by the SOFA, wherein the second bronchoscope terminal three-dimensional model is configured as the bronchoscope terminal three-dimensional model; and / or, The invention also includes constructing the trachea three-dimensional model by the following methods: Pre-acquire CT images including the lung airways; Segmenting the tracheal structure in the CT image using an image segmentation technique, and performing three-dimensional reconstruction on the tracheal structure to obtain a first three-dimensional tracheal model; performing meshing processing on the first trachea three-dimensional model to obtain a second trachea three-dimensional model that can be recognized by the SOFA, wherein the second trachea three-dimensional model is configured as the trachea three-dimensional model; and / or, An effector is set at the end of the virtual bronchoscope, and the effector is a sampling point for reading the coordinates and direction of the end of the virtual bronchoscope in real time. The coordinates of the end of the virtual bronchoscope are read by the effector. and direction , and determining the current virtual posture of the virtual bronchoscope end according to the coordinates and direction of the virtual bronchoscope end.

11. The bronchoscopic intervention simulation method according to any one of claims 1 to 10, characterized in that: The control accuracy of the controller is verified using the simulation platform. If the control accuracy of the controller meets the preset control accuracy requirements, the controller is applied to the bronchoscope robot.

12. A bronchoscopic intervention simulation system, characterized in that: The system comprises a pre-built simulation platform, the simulation platform being connected to a joystick and a controller, respectively, the joystick being configured to output an electrical signal corresponding to a target posture of the bronchoscope tip, and the controller being configured to determine a driving amount for controlling the posture of the bronchoscope tip according to the current posture of the bronchoscope tip and the target posture; The bronchoscopic intervention simulation system is configured to perform bronchoscopic intervention simulation using the bronchoscopic intervention simulation method according to any one of claims 1 to 11.

Images