A practical training teaching system and medical trolley
The practical training system, which combines a three-dimensional digital airway model with sensor signals, solves the problems of high cost and safety in bronchoscopy operation training, achieves low-cost and diversified training results, and improves operational safety and realism.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UE MEDICAL
- Filing Date
- 2023-09-08
- Publication Date
- 2026-06-19
AI Technical Summary
Existing bronchoscopy training devices are costly and cannot simulate diverse clinical training. Furthermore, excessive bending angles at the tip of the endoscope insertion section can cause patient injury, and current technology struggles to identify such situations.
The endoscope insertion path is determined by combining a three-dimensional digital airway model with sensor signals, and the collision detection between the endoscope and the airway wall is realized. The endoscope front end is rendered in real time. The model is trained through a virtual model and does not rely on a physical model.
It enables low-cost training of multiple airway models, improves the realism and safety of training, reduces the potential risk of harm to patients, and enhances training effectiveness.
Smart Images
Figure CN117218915B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical teaching equipment technology, and in particular to a practical training system and a medical trolley. Background Technology
[0002] While interns may possess sufficient theoretical knowledge, they often lack practical skills and have limited mastery of medical equipment. Practicing directly on actual patients carries significant risks and could lead to unnecessary harm. Therefore, pre-operative training is essential for surgical interns to transform their knowledge and skills into practical abilities, enabling them to become qualified surgeons. The demand for practical training and pre-operative patient familiarization among medical students is increasing, necessitating a training operating system / teaching system to meet the needs of both doctors and students. This is particularly relevant for complex tissues within the body, such as the bronchi. Bronchoscopy, an instrument inserted through the mouth or nose into the lower respiratory tract, is used for observing lesions in lobes, segments, and subsegments of the bronchi, for biopsy sampling, bacteriological and cytological examinations, research on bronchial and lung diseases, and post-operative examinations. The bronchial tree, with its complex branching structure, requires particular training and practical instruction.
[0003] Some existing technologies use devices for simulating bronchoscopy training. These devices typically establish a physical bronchial model to provide hands-on training. However, such devices have the following drawbacks: due to the complexity of the bronchial system, the manufacturing cost of the physical model is high; and without replacing the physical model, they can only simulate operations under one type of airway morphology, failing to meet diverse clinical training needs.
[0004] Furthermore, during actual bronchoscopy procedures using an endoscope, the tip of the endoscope may touch the bronchial wall due to excessive bending angle of the flexible portion at the insertion point, potentially causing harm to the patient. Therefore, it is necessary to identify this situation. Identifying this situation without relying on a physical bronchial model is a challenge in simulated bronchoscopy training. Summary of the Invention
[0005] To address the aforementioned problems, the present invention aims to provide a practical training system and a medical trolley.
[0006] A practical training system includes:
[0007] The airway model providing module provides a selected 3D digital airway model and displays the currently selected airway model on the display.
[0008] The path planning module is used to generate a navigation path from the airway inlet to the target lesion based on the target lesion in the currently selected airway model, and to display the navigation path on the airway model presented on the current display.
[0009] The lens pose calculation module is used to determine the current pose information of the lens in the three-dimensional space where the currently selected airway model is located, based on the insertion length of the endoscope tip, the bending angle of the endoscope tip, and the pose of the endoscope tip. The pose information includes the position and orientation information in the three-dimensional space.
[0010] The collision detection module is used to determine whether the endoscope tip collides with the airway wall based on the current pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model.
[0011] The lens pose calculation module is also used to determine the pose information of the lens after it moves with the endoscope by combining the detection results of the collision detection module.
[0012] The real-time rendering module is used to determine the image captured by the lens based on the current pose information and the pose information after movement of the end lens of the end lens, and to render and present the image in real time through the display.
[0013] Preferably, the system includes:
[0014] The CT reconstruction module is used to import CT medical images and perform three-dimensional reconstruction based on the imported CT medical images to obtain a three-dimensional numerical airway model.
[0015] The database is used to store pre-set three-dimensional numerical airway models, as well as three-dimensional numerical airway models obtained by the CT reconstruction module.
[0016] According to claim 1, a practical training system is characterized in that it includes an instrument insertion channel, wherein the instrument insertion channel is provided with an instrument insertion inlet as the airway inlet, and the tip of the endoscope is inserted through the instrument insertion inlet; the instrument insertion channel is provided with a length detection device for detecting the insertion length of the endoscope tip lens.
[0017] Preferably, the instrument insertion channel is set on a human head model, and the human head model has instrument insertion entrances at the mouth and nose respectively; a laser sensor is set at the instrument entrance, and the laser sensor is used to identify whether an instrument has entered the instrument insertion channel and to distinguish whether the current instrument enters from the mouth or nose.
[0018] Preferably, the endoscope includes a handheld part, which is always located outside the airway inlet; the handheld part is provided with a rotation detection device for detecting the bending angle of the endoscope tip and an attitude sensor for detecting the endoscope's posture.
[0019] Preferably, the handheld part is provided with a swing arm mechanism, which controls the bending of the curved part at the tip of the endoscope; the rotation detection device includes a sliding potentiometer, and the movement of the swing arm mechanism drives the sliding potentiometer to move synchronously; the correspondence between the resistance value returned by the sliding potentiometer and the rotation angle of the swing arm mechanism is obtained in advance, the rotation angle of the swing arm mechanism is determined according to the current resistance value returned by the sliding potentiometer, and then the bending angle of the tip of the endoscope is determined according to the rotation angle.
[0020] Preferably, the attitude sensor includes a gyroscope and an accelerometer, and the lens attitude calculation module calculates the attitude angle of the endoscope tip lens in the geographic spatial coordinate system based on the detection signal of the attitude sensor; the selected three-dimensional numerical airway model also represents its position information in the geographic spatial coordinate system.
[0021] Preferably, the determination of the lens's pose information after the endoscope moves, based on the detection results of the collision detection module, specifically includes:
[0022] The position and orientation of the lens are represented by a vector V in three-dimensional space. After a collision is detected, the triangular face T where the collision point is located can be determined. The angle between V and T is the collision angle between the lens and the airway wall. After the collision, the lens vector V will rotate in the direction with a larger angle. The angle V1 after rotation is the forward direction of the lens. Combined with the insertion length of the endoscope tip, the position and orientation of the lens after movement are calculated.
[0023] Preferably, the determination of whether the endoscope tip collides with the airway wall based on the pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model specifically includes:
[0024] The three-dimensional digital airway model and the endoscope tip lens are both represented using 3D mesh models;
[0025] Calculate the orientation (DV) of the endoscope tip lens;
[0026] Scan the airway grid vertices in the DV direction and construct the airway wall vertex matrix MA in the DV direction;
[0027] Calculate the Euclidean distance D between the tip of the endoscope and MA;
[0028] A collision can be determined when the distance is less than or equal to 0.
[0029] Preferably, the system includes:
[0030] The operation evaluation module is used to count the number of collisions between the endoscope tip and the airway wall during a single training session, and output the evaluation results based on the number of collisions.
[0031] A medical trolley, including a training system as described in any of the preceding items.
[0032] The practical training system provides a 3D numerical airway model for training, independent of physical airway models. It determines the endoscope insertion path based on various sensor signals and fits it to the 3D numerical airway model, enabling endoscope insertion training within the airway model. This invention, employing the above scheme, can simply and conveniently provide various airway models of different shapes for training. Furthermore, it achieves collision detection between the endoscope tip and the airway wall of the virtual airway model, and displays the image captured by the endoscope's front lens in real time. This makes the training process more realistic, and the collision detection does not rely on any external components or sensors, making it suitable for various application scenarios involving insertion training under virtual airway models. Combining collision detection with other methods provides multiple training scenarios, which helps improve training effectiveness. Attached Figure Description
[0033] Figure 1 This is a schematic diagram of the structure of the traditional Chinese medicine trolley in this application;
[0034] Figure 2 for Figure 1 A structural diagram from another angle;
[0035] Figure 3 A schematic diagram of the instrument insertion channel and its sensors;
[0036] Figure 4 This is a schematic diagram of the structure of the practical training system;
[0037] Figure 5 A schematic diagram illustrating the implementation process of simulating endoscopic insertion training in a practical training system;
[0038] Figure 6 This is a schematic diagram of the menu interface of the practical training teaching system;
[0039] Figure 7 One of the schematic diagrams of the display interface in basic exploration mode;
[0040] Figure 8 The second schematic diagram of the display interface in the basic exploration mode;
[0041] Figure 9 This is a schematic diagram of the display interface under the personalized data function mode;
[0042] Figure 10 This is a schematic diagram of the coordinate system for the attitude sensor.
[0043] Figure 11 A diagram showing the transformation relationship of the endoscope handpiece rotating around the z-axis;
[0044] Figure 12This is a schematic diagram of the collision detection process;
[0045] Figure 13 A schematic diagram of the interface for operating biopsy forceps;
[0046] Figure 14 This is one of the schematic diagrams of the interface displayed in navigation exploration mode;
[0047] Figure 15 This is the second interface displayed in the navigation exploration mode.
[0048] Figure label:
[0049] Human head model 1, instrument insertion channel 11, laser sensor 12, rotary encoder 13, display 2, endoscope 3, biopsy channel 31, biopsy channel supporting instruments 4, lower computer 51, main computer 52. Detailed Implementation
[0050] The embodiments of the present invention are described in detail below.
[0051] Example 1:
[0052] This embodiment provides a medical trolley. For example... Figure 1-3 As shown, the medical trolley includes a trolley frame, on which an instrument insertion channel 11, a processing device, and a display are provided.
[0053] Endoscope 3 and other instruments are inserted into the instrument insertion channel 11 to simulate operation in the airway model. In this embodiment, the instrument insertion channel 11 is a solid model formed on a trolley, which has an opening for instrument insertion that connects to the activity space. The instrument insertion channel 11 is equipped with a length detection device for detecting the insertion length of the endoscope tip, which determines the insertion depth of the endoscope in the airway model. In this embodiment, the length detection device is a rotary encoder 13 located at the opening of the instrument insertion channel. The rotary encoder 13 is used to identify the path length of the endoscope entry. When the endoscope enters, it causes the rotary encoder 13 to rotate, and the rotary encoder 13 transmits the number of rotations to a processing device to convert the data into length data.
[0054] In this embodiment, the instrument placement channel 11 is set on a human head model 1, which has instrument placement entrances at the mouth and nose. A laser sensor 12 is installed at each instrument entrance to detect whether an instrument has entered the placement channel and to distinguish whether the instrument has entered through the mouth or nose. When the endoscope is inserted, it cuts the laser beam. The sensor detects the laser beam being cut and transmits a signal to the processing device, which can then identify whether another instrument is being placed. By identifying which laser sensor the received signal comes from, it is possible to distinguish whether the instrument has entered through the mouth or nose. As can be seen from the above, different placement methods can be experienced by the operator, improving the applicability of the medical trolley.
[0055] The instruments used in conjunction with the medical trolley for instrument insertion training generally include an endoscope 3. The endoscope 3 includes a handheld part for manual operation, an insertion part for insertion into the human body, and a bending part that bends the endoscope tip. The endoscope tip is the tip of the insertion part; in this context, "tip" refers to the end that is first inserted into the human body. The bending part can be a flexible structure such as a keel or soft material. Since the objective lens is located at the endoscope tip, the bending part, which rotates the endoscope tip, increases the objective lens's imaging range. The bending part is driven to bend by a linkage mechanism located on the handheld part. In one embodiment, the linkage mechanism includes a swing arm mechanism that controls the bending angle and direction of the bending part; in other embodiments, the linkage mechanism may also include a lever that controls the bending angle and direction of the bending part.
[0056] The handheld unit is equipped with a rotation detection device for detecting the bending angle of the endoscope tip and an attitude sensor for detecting the posture of the endoscope operating end. The processing unit determines the position and pose information of the endoscope tip in the intestinal model based on the detection signals from the length detection device, rotation detection device, and attitude sensor, thereby determining the current image and angle captured by the lens.
[0057] The length detection signal reflects the length of the distal endoscope inserted into the airway; the rotation detection signal reflects the rotation angle of the endoscope tip caused by the bending part; and the posture sensor's detection signal reflects the posture change of the endoscope tip caused by the posture change of the handheld part itself. Since all of the above information leads to changes in the posture of the endoscope tip, combining this information can determine the insertion position of the endoscope in the intestinal model and the changes in the image captured by the objective lens. Therefore, in this embodiment, the movement of the endoscope after insertion into the airway and the images captured can be simulated based on the above multiple detection devices, without relying on a physical airway model. In other words, it is not necessary to set up a physical airway model in the activity space connected to the instrument insertion opening to restrict the movement of the endoscope.
[0058] In this embodiment, the medical cart does not rely on a physical airway model to determine the pose of the endoscope. When combined with a 3D physics engine, it can provide a variety of different airway models for operation training, which is beneficial to improving the applicability of the medical cart and reducing its manufacturing cost.
[0059] Preferably, in this embodiment, the endoscope is provided with a biopsy channel 31, through which supporting instruments 4, such as syringes, puncture needles, and biopsy forceps, can enter. A detection device for detecting the insertion length of the instrument is provided at the entrance of the biopsy channel. In this embodiment, the detection device can be a resistance sensor. When the supporting instrument enters the biopsy channel, it changes the resistance value at the channel opening. Different resistance values correspond to the path distance of the instrument, thereby determining its position in the airway. In other embodiments, the detection device can also be a rotary encoder, which determines the insertion length. Optionally, a laser sensor is provided at the entrance of the biopsy channel to detect whether a supporting instrument has been inserted. Each supporting instrument has a sliding potentiometer in its operating handle. When the resistance value of the sliding potentiometer changes, a signal is transmitted to the lower-level machine to sense the usage status of the supporting instrument, indicating that the supporting instrument is triggered to perform sampling or removal of the lesion.
[0060] By setting up biopsy channels and supporting instruments, the medical trolley can train users on the use of these instruments, thus providing better training results.
[0061] Optionally, the processing device in this embodiment includes two parts: a lower-level machine 51 and a host machine 52. The lower-level machine 51 is connected to the detection devices in the instrument placement channel, the endoscope, the detection devices in the biopsy channel, and the biopsy forceps that cooperate with the biopsy channel, etc. After acquiring signals from these devices, it processes the signals, compiles them into computer language that the host machine can recognize, and then sends them to the host machine 52. The host machine 52 determines the meaning of each signal, compiles it into an image, and displays it on the display 2. Dividing the processing device into two independent parts, and making the signal acquisition and signal processing processes independent of each other, can reduce the data processing pressure on the host machine.
[0062] Example 2:
[0063] This embodiment provides a practical training system that operates on a medical trolley as shown in Embodiment 1. Figure 4 The practical training system includes:
[0064] The airway model providing module provides a selected 3D digital airway model and displays the currently selected airway model on the display.
[0065] The path planning module is used to generate a navigation path from the airway inlet to the target lesion based on the target lesion in the currently selected airway model, and to display the navigation path on the airway model presented on the current display.
[0066] The lens pose calculation module is used to determine the current pose information of the lens in the three-dimensional space where the currently selected airway model is located, based on the insertion length of the endoscope tip, the bending angle of the endoscope tip, and the pose of the endoscope tip. The pose information includes the position and orientation information in the three-dimensional space.
[0067] The collision detection module is used to determine whether the endoscope tip collides with the airway wall based on the current pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model.
[0068] The lens pose calculation module is also used to determine the pose information of the lens after it moves with the endoscope by combining the detection results of the collision detection module.
[0069] The real-time rendering module is used to determine the image captured by the lens based on the current pose information and the pose information after movement of the end lens of the end lens, and to render and present the image in real time through the display.
[0070] This training system uses a provided three-dimensional digital airway model for training, without relying on a physical airway model. It determines the insertion path of the endoscope based on various sensor signals and fits it with the three-dimensional numerical airway model to achieve endoscope insertion training in the airway model.
[0071] Because this system uses a virtual 3D digital airway model for training, when simulating endoscopic insertion, it is necessary to calculate and display the insertion position of the endoscope within the airway model and the image captured by its lens on the screen. Its workflow is as follows: Figure 5 As shown, after the endoscope insertion is detected by the detection device at the entrance of the instrument insertion channel, the airway model is loaded and the current endoscopic image is drawn. Then, the signals returned by the length detection device in the instrument insertion channel and the signals returned by the detection device of the endoscope handpiece are collected to perform collision detection. The lens posture is calculated based on the collision detection results. If an instrument is detected inserted in the biopsy instrument channel, the biopsy instrument is drawn. The specific implementation process is described in detail below.
[0072] The practical training system in this embodiment, such as Figure 6 As shown, after powering on, a menu page is displayed on the monitor, offering various function modes for the user to choose from, including basic exploration, basic diagnosis, advanced diagnosis, and therapeutic procedures.
[0073] Selecting the basic exploration option allows access to a 3D digital airway model with a pre-existing route. This 3D digital airway model is a pre-set route within the corresponding 3D digital airway model, or it can be a navigation path generated in real-time from the airway inlet to the target lesion based on the currently selected airway model. In a specific embodiment, after entering a specific function mode from the menu page, one or more 3D digital airway models can be provided from the database for the user to select.
[0074] like Figure 7-8 As shown, in the basic exploration mode, the display interface includes two independent display areas: a first display area and a second display area. In the first display area, the currently selected three-dimensional digital airway model and navigation path are displayed; in the second display area, the real-time images captured by the endoscope lens are displayed.
[0075] Preferably, the system includes a CT reconstruction module for importing CT medical images and performing 3D reconstruction based on the imported CT images to obtain a 3D digital airway model. The corresponding functional mode also includes a personalized data function. Under the personalized data function, the CT reconstruction module performs 3D reconstruction to obtain a personalized 3D digital airway model, and performs training or diagnostic operations based on this numerical airway model. Preferably, the 3D digital airway model obtained by the CT reconstruction module is stored in a database for later use in basic exploration and other training. Setting up a CT reconstruction module can provide personalized preoperative training based on specific patient conditions, which is beneficial to improving the success rate of surgery.
[0076] like Figure 9 As shown, in the personalized data function mode, the currently imported CT medical images are displayed on the monitor. The system includes a lesion recognition module, which identifies lesion information in the CT medical images through image recognition and displays the lesion information in a designated area on the monitor. The specific method for three-dimensional reconstruction based on CT medical images is a conventional technique in this field and will not be described in detail here.
[0077] The data settlement module in the training system of this embodiment is responsible for calculating the position and orientation of the endoscope lens in three-dimensional space, thereby determining the position of the endoscope in the current three-dimensional digital airway model and the images captured by its lens.
[0078] In this embodiment, the insertion length of the endoscope tip is obtained by a length detection device installed in the instrument insertion channel. Taking a rotary encoder as the length detection device as an example, the endoscope insertion part drives the shaft of the rotary encoder to rotate, sending 4096 pulse signals per revolution. The diameter of the encoder shaft is d mm, and the current number of received pulses is n. The bronchoscope insertion length is calculated as follows:
[0079] L = n / 4096 * d * 3.14
[0080] This allows us to determine the insertion length of the endoscope and its insertion position within the current three-dimensional digital airway model.
[0081] When the endoscope insertion section moves within the airway, the bending section causes the endoscope tip to bend in a predetermined direction to enter different branches, or to view images from different angles. Therefore, by determining the bending angle of the bending section, the bending angle of the endoscope tip can be determined, thus determining its position in the three-dimensional digital airway model. Furthermore, the bending angle of the endoscope tip is related to the direction of lens movement. In this embodiment, a rotation detection device is provided on the handheld part to determine the bending angle of the endoscope tip. In this embodiment, the handheld part of the endoscope is equipped with a swing arm mechanism. The rotation detection device includes a sliding potentiometer. The movement of the swing arm mechanism causes the sliding potentiometer to move synchronously. The correspondence between the resistance value returned by the sliding potentiometer and the rotation angle of the swing arm mechanism is obtained in advance. The rotation angle of the swing arm mechanism is determined based on the current resistance value returned by the sliding potentiometer, and then the bending angle of the endoscope tip is determined based on the rotation angle.
[0082] For example, if the swing arm mechanism has a vertical rotation range of -40° to +40°, its resistance at -40° is 'a' ohms, its resistance at +40° is 'b' ohms, and its resistance at its current position is 'c' ohms, then the angle α of the swing arm mechanism is:
[0083] α = -40 + (c – a) / ((ba) / 80)
[0084] Since the swing arm mechanism is linked with the curved part through a traction cable, the rotation angle of the endoscope tip can be determined after the angle of the swing arm mechanism is determined.
[0085] In practical applications, changes in the posture of the endoscope handle, such as rotation, will cause changes in the posture of the endoscope tip, resulting in changes in the image captured by the endoscope lens. Therefore, it is necessary to determine the changes in the endoscope tip posture caused by changes in the endoscope handle posture. In this embodiment, a posture sensor is horizontally positioned on the endoscope handle. The posture sensor includes a gyroscope and an accelerometer. The lens posture calculation module calculates the posture angle of the endoscope tip lens in the geographic spatial coordinate system based on the detection signal from the posture sensor; the selected three-dimensional numerical airway model also represents its position information using coordinates in the geographic spatial coordinate system. The following is combined with... Figure 9 and Figure 10 Please provide a detailed explanation.
[0086] like Figure 10As shown, in this embodiment, the z-axis of the attitude sensor represents the direction of gravity, the y-axis represents the direction of the operator's chest, and the x-axis represents the left and right direction of the operator, thus constructing its own coordinate system according to these directions. Combined with... Figure 10 Taking the rotation of the handle around the z-axis as an example, this illustrates how the attitude sensor calculates attitude changes. As the handle rotates around the z-axis, the z-axis remains constant; the transformations of the x and y axes are as follows: Figure 10 As shown.
[0087] Depend on Figure 11 It can be known that:
[0088] r x2 =OA + AB + BC
[0089] =OD cosα + BD sinα + BF sinα
[0090] =r x1 cosα+r Y1 sinα
[0091] r Y2 =DE-AD
[0092] =DF cosα-OD sinα
[0093] =r Y1 cosα-r x1 sinα
[0094] r z2 =r z1
[0095] Rewriting the above three equations in matrix form yields the following relationship after rotating about the z-axis by an angle α:
[0096]
[0097] The transformation matrix is as follows:
[0098]
[0099] Similarly, the rotation matrices around the x and y axes can be calculated. Following the child level (x) (relative to y), the child level (y) (relative to z), and the parent level (z), in this embodiment, the z-axis is rotated by an angle Ψ, the y-axis by an angle θ, and the x-axis by an angle γ. Matrix multiplication is then performed to calculate the attitude matrix.
[0100]
[0101] The attitude angle transformation matrix transforms the object coordinate system to the geographic coordinate system.
[0102]
[0103] g1 = 2(q1q3 - q0q2)
[0104] g2 = 2(q2q3 + q0q1)
[0105] g3=q0 2 -q1 2 -q2 2 +q3 2
[0106] g4 = 2(q1q2 + q0q3)
[0107] g5 = q0 2 +q1 2 -q2 2 -q3 2
[0108] Let q be a vector in three-dimensional space. From the above formula, we can obtain the x, y, and z axis attitude angles as follows:
[0109] γ=-arcsin(g1)
[0110] θ = arctan(g² / g³)
[0111] ψ = arctan(g4 / g5)
[0112] After the endoscope insertion section enters the airway, it bends approximately 90 degrees. Therefore, when the endoscope handle rotates around the y and z axes, the tip attitude angle is the sum of the y and z axis attitude angles. Thus, the lens attitude calculation module can determine the endoscope tip attitude using the above calculation method. Furthermore, the attitude angle transformation matrix in the above calculation method is represented using a geographic coordinate system, and the coordinates of the 3D digital airway model are also represented using a geographic coordinate system, thereby enabling rapid determination of the endoscope's insertion position within the 3D digital airway model.
[0113] However, the current pose information of the endoscope tip determined in the above manner does not take into account the restriction of the airway wall on the endoscope insertion process. Based on this current pose information, unrealistic situations may occur, such as the endoscope tip penetrating the airway wall. Therefore, to avoid this situation, this embodiment also performs collision detection on the endoscope tip, and adjusts the current pose of the endoscope tip based on the collision detection results to obtain the moved pose information.
[0114] In this embodiment, the collision between the endoscope tip and the airway wall is determined based on the pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model.
[0115] like Figure 12 As shown, the specific steps of collision detection are as follows:
[0116] The three-dimensional digital airway model and the endoscope tip lens are both represented using 3D mesh models;
[0117] Calculate the orientation (DV) of the endoscope tip lens;
[0118] Scan the airway grid vertices in the DV direction and construct the airway wall vertex matrix MA in the DV direction;
[0119] Calculate the Euclidean distance D between the endoscope tip vertex and MA through a finite number of iterations;
[0120] A collision can be determined when the distance is less than or equal to 0.
[0121] The collision detection described above does not rely on any external components or sensors, and is suitable for various virtual airway models for insertion training applications. Combining collision detection can provide a variety of training scenarios, which is beneficial to improving training results.
[0122] When the endoscope insertion section moves within the actual airway, in this embodiment, if the lens pose calculation module combines the detection results from the collision detection module to determine the lens's pose information after moving with the endoscope, specifically:
[0123] The position and orientation of the lens are represented by a vector V in three-dimensional space. After a collision is detected, the triangle T where the collision point is located can be determined. The angle between V and T is the collision angle between the lens and the airway wall. After the collision, the lens vector V will rotate in the direction with a larger angle. The angle V1 after rotation is the forward direction of the lens. Combined with the insertion length collected by the length detection device, the position and attitude of the lens after movement can be calculated.
[0124] Therefore, based on the current pose information and the pose information after movement of the endoscope tip lens, the pose information and movement of the lens in the three-dimensional space where the currently selected airway model is located can be accurately described. Based on the above information, the pose of the endoscope in the three-dimensional digital airway model and the image captured by the lens are calculated, and the image is rendered and presented in real time on the display.
[0125] Furthermore, this embodiment also evaluates the training results based on collision detection results. The system includes an operation evaluation module, used to count the number of collisions between the endoscope tip and the airway wall during a single training session, and output evaluation results based on the number of collisions. Figure 7As shown, the number of times the airway wall was touched during the operation is displayed on the screen to evaluate the operation result. Preferably, the insertion path of the endoscope is also analyzed to determine its alignment with the midline during the operation, and the training time is recorded. Since whether the insertion path of the endoscope aligns with the midline and the time taken from insertion to reaching the target lesion can both be used to evaluate the quality of an operation, parameters such as the training time, the number of times the airway wall was touched, and the time to align with the midline are output on the screen to evaluate the training result. Presenting the training results in a parameterized manner helps to intuitively reflect the training results.
[0126] In this system, selecting the therapeutic operation mode on the menu page allows for training on the use of the accompanying instruments in conjunction with the biopsy channel. For example... Figure 13 As shown, the biopsy forceps sampling is illustrated. Once the endoscope is inserted correctly, a prompt to insert the biopsy forceps is displayed. After detecting the signal returned from the detection device in the biopsy channel, the insertion of the biopsy forceps is confirmed. Based on the insertion length of the biopsy forceps and the current insertion length of the endoscope, the correct insertion length is determined, and a simulated biopsy forceps operation screen is displayed on the screen. Upon receiving the signal returned from the operating handle of the accompanying instrument, the initiation of the sampling or removal action on the lesion is confirmed, and the corresponding operation animation is played on the screen to make the simulation training more realistic. For simulated operations such as snare or syringe irrigation, the process is similar to the biopsy forceps sampling process; only the corresponding accompanying instrument needs to be displayed on the screen, and the corresponding instrument's snare cutting or irrigation operation animation needs to be played.
[0127] The system also provides navigation and exploration functions, such as Figure 14 and Figure 15 As shown, in navigation and exploration mode, the first display area on the screen displays the target image, presented in a key shape; the second display area displays the currently used 3D digital airway model; the operator controls the endoscope to reach the target location, and the real-time observation of the endoscope and the movement path of the endoscope insertion section are displayed in the first and second display areas respectively. Furthermore, in this mode, the size and color of the target image distinguish whether the target has been found. For example, after the target is found, the corresponding image is displayed on the screen as a magnified version of the target image in a different color.
[0128] The system in this embodiment enables practical training operations based on a virtual three-dimensional digital airway model, as well as endoscopic movement simulation and endoscopic visualization simulation within this model. Those skilled in the art can provide various training modes based on this. This system does not rely on a physical airway model, has strong applicability, and can provide multiple training methods.
[0129] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.
Claims
1. A practical training teaching system, characterized in that, include: The airway model providing module provides a selected 3D digital airway model and displays the currently selected airway model on the display. The path planning module is used to generate a navigation path from the airway inlet to the target lesion based on the target lesion in the currently selected airway model, and to display the navigation path on the airway model presented on the current display. The lens pose calculation module is used to determine the current pose information of the lens in the three-dimensional space where the currently selected virtual airway model is located, based on the insertion length of the endoscope tip, the bending angle of the endoscope tip, and the pose of the endoscope tip. The pose information includes the position and orientation information in the three-dimensional space. The collision detection module is used to determine whether the endoscope tip collides with the airway wall of the virtual airway model based on the current pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model. The lens pose calculation module is also used to determine the pose information of the endoscope tip after movement by combining the detection results of the collision detection module; The real-time rendering module is used to determine the image captured by the lens based on the current pose information and the pose information after movement of the end lens of the end lens, and to render and present the image in real time through the display.
2. The practical training system according to claim 1, characterized in that, include: The CT reconstruction module is used to import CT medical images and perform three-dimensional reconstruction based on the imported CT medical images to obtain a three-dimensional numerical airway model. The database is used to store pre-set three-dimensional numerical airway models, as well as three-dimensional numerical airway models obtained by the CT reconstruction module.
3. The practical training system according to claim 1, characterized in that, It includes an instrument insertion channel, which has an instrument insertion inlet serving as the airway inlet, through which the tip of the endoscope is inserted; the instrument insertion channel is equipped with a length detection device for detecting the insertion length of the endoscope tip lens.
4. The practical training system according to claim 3, characterized in that, The instrument insertion channel is set on the human head model, and the human head model has instrument insertion entrances at the mouth and nose respectively; a laser sensor is set at the instrument insertion entrance to identify whether an instrument has entered the instrument insertion channel and to distinguish whether the current instrument enters from the mouth or nose.
5. The practical training teaching system according to claim 1, characterized in that, The endoscope includes a handheld part, which is always located outside the airway inlet; the handheld part is equipped with a rotation detection device for detecting the bending angle of the endoscope tip and an attitude sensor for detecting the attitude of the endoscope handheld part.
6. The practical training system according to claim 5, characterized in that, The handheld part is equipped with a swing arm mechanism. The rotation detection device includes a sliding potentiometer. The movement of the swing arm mechanism drives the sliding potentiometer to move synchronously. The correspondence between the resistance value returned by the sliding potentiometer and the rotation angle of the swing arm mechanism is obtained in advance. The rotation angle of the swing arm mechanism is determined according to the current resistance value returned by the sliding potentiometer. Then, the bending angle of the endoscope tip is determined according to the rotation angle.
7. The practical training teaching system according to claim 1, characterized in that, The determination of the lens's pose information after the endoscope moves, based on the detection results from the collision detection module, specifically includes: The position and orientation of the lens are represented by a vector V in three-dimensional space. After a collision is detected, the triangular face T where the collision point is located can be determined. The angle between V and T is the collision angle between the lens and the airway wall. After the collision, the lens vector V will rotate in the direction with a larger angle. The angle V1 after rotation is the forward direction of the lens. Combined with the insertion length of the endoscope tip, the position and orientation of the lens after movement are calculated.
8. The practical training teaching system according to claim 1, characterized in that, The determination of whether the endoscope tip collides with the airway wall based on the pose information of the endoscope tip and the position information of the airway wall in the three-dimensional digital airway model specifically includes: The three-dimensional digital airway model and the endoscope tip lens are both represented using 3D mesh models; Calculate the orientation (DV) of the endoscope tip lens; Scan the airway grid vertices in the DV direction and construct the airway wall vertex matrix MA in the DV direction; Calculate the Euclidean distance D between the tip of the endoscope and MA; A collision can be determined when the distance is less than or equal to 0.
9. The practical training teaching system according to claim 1, characterized in that, include: The operation evaluation module is used to count the number of collisions between the endoscope tip and the airway wall during a single training session, and output the evaluation results based on the number of collisions.
10. A medical cart, characterized by, Includes a practical training system as described in any one of claims 1-9.