An interactive bronchoscope navigation system and method

Abstract

The application relates to an interactive bronchoscope navigation system and method, the navigation system comprising: a CT data acquisition module for acquiring chest CT data of a patient; a navigation data acquisition device for simulating the insertion operation of a camera at the head end of an insertion tube of a bronchoscope based on the chest CT data and obtaining insertion position data and posture data of the camera; a digital bronchial tree generation module for generating a digital bronchial tree based on the chest CT data; a navigation data mapping module for mapping the insertion position data and the posture data of the camera into the digital bronchial tree, performing motion calculation of a virtual camera under the constraint of a digital airway lumen of the digital bronchial tree, obtaining the pose of the virtual camera in the digital airway lumen, and realizing simulation operation. The simulation operation of the bronchoscope can be realized, the digital airway of the patient can be preoperatively preoperated, and the operation efficiency is improved, the operation risk is reduced, and the medical cost is saved.

Description

Technical Field

[0001] This invention belongs to the field of medical simulation technology, and relates to a bronchoscope navigation system and method, and more particularly to an interactive bronchoscope navigation system and method. Background Technology

[0002] Virtual bronchoscopy navigation can perform 3D reconstruction of the airway and blood vessels and path planning. Compared with CT hand-drawn navigation, virtual bronchoscopy navigation provides a more intuitive 3D visualization solution. However, existing virtual bronchoscopy navigation products can only display the surgical path and do not have interactive operation.

[0003] Because there is no interactive interface, it is impossible to simulate bronchoscopy. This means that, on the one hand, the surgical plan cannot be rehearsed before the procedure using the patient's digital airway, reducing surgical efficiency and increasing surgical risks; on the other hand, it prevents direct communication with the patient and their family before the procedure, hindering their understanding of their condition and the surgical plan, thus increasing the patient's psychological burden during the operation and reducing patient satisfaction.

[0004] Given the technical deficiencies of existing technologies, there is an urgent need for an interactive bronchoscopy navigation system and method. Summary of the Invention

[0005] In view of the problems existing in the prior art, the present invention provides an interactive bronchoscopy navigation system and method, which can realize the simulation operation of bronchoscopy, thereby enabling the preoperative simulation of the surgical plan for the patient's digital airway, thereby improving surgical efficiency, reducing surgical risks, and saving medical costs.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An interactive bronchoscopy navigation system, comprising:

[0008] The CT data acquisition module is used to acquire the patient's chest CT data;

[0009] A navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope based on the chest CT data, and to obtain the insertion position data and attitude data of the camera.

[0010] A digital bronchial tree generation module, which is used to generate a digital bronchial tree based on the chest CT data;

[0011] The navigation data mapping module is used to map the insertion position data and attitude data of the camera to the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, the motion calculation of the virtual camera is performed to obtain the pose of the virtual camera in the digital airway lumen, thereby realizing simulation operation.

[0012] Preferably, the navigation data acquisition device includes a bronchoscope, a transparent operating tube, and a camera located above the transparent operating tube. An angle detection sensor is installed in the lever of the bronchoscope, and the angle detection sensor is used to collect the swing angle of the lever. The surface of the insertion tube of the bronchoscope is printed with multiple coded patterns. The camera at the tip of the insertion tube is inserted into the transparent operating tube, and after the camera at the tip of the insertion tube is inserted into the transparent operating tube, the camera takes an image of the insertion tube inside the transparent operating tube, so as to obtain the insertion position data and attitude data of the camera at the tip of the insertion tube based on the coded patterns in the image and the swing angle of the lever.

[0013] Preferably, each of the coded patterns printed on the surface of the insertion tube includes a position marker from the head and a head rotation angle marker.

[0014] Preferably, each of the coded patterns includes a circumferential line arranged around the circumference of the insertion tube and four sets of numbers and horizontal lines arranged at 90° intervals around the circumference of the insertion tube.

[0015] Preferably, the coded pattern is printed on the surface of the insertion tube using printing ink mixed with fluorescent agents.

[0016] Preferably, the navigation data acquisition device further includes a polarizing mirror, and the polarizing mirror is installed between the camera and the transparent operating conduit.

[0017] Preferably, the digital bronchial tree generation module includes:

[0018] An airway segmentation submodule is used to segment the airway based on the chest CT data in order to extract a binary image of the airway.

[0019] A 3D reconstruction submodule is used to perform 3D reconstruction based on the airway binary image to generate a 3D visualization model;

[0020] The skeleton extraction submodule is used to extract airway skeleton points from the three-dimensional visualization model.

[0021] The tree structure construction submodule is used to organize the airway skeleton points into a hierarchical structure through a multi-branch tree traversal algorithm to construct a tree-structured digital bronchial tree.

[0022] Preferably, the airway segmentation submodule uses a deep learning-based segmentation method to segment the airway and extract a binary image of the airway.

[0023] Furthermore, the present invention also provides an interactive bronchoscopy navigation method, which employs the interactive bronchoscopy navigation system described above and includes the following steps:

[0024] The patient's chest CT data is acquired using the CT data acquisition module.

[0025] Based on the chest CT data, the navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope insertion tube to obtain the insertion position data and posture data of the camera.

[0026] A digital bronchial tree is generated based on the chest CT data using the digital bronchial tree generation module.

[0027] The navigation data mapping module is used to map the insertion position data and attitude data of the camera onto the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, the motion calculation of the virtual camera is performed to obtain the pose of the virtual camera in the digital airway lumen, thereby realizing the simulation operation.

[0028] Preferably, generating a digital bronchial tree based on the chest CT data using the digital bronchial tree generation module specifically includes:

[0029] Based on the chest CT data, the airway is segmented to extract a binary image of the airway;

[0030] Three-dimensional reconstruction is performed based on the binary airway image to generate a three-dimensional visualization model;

[0031] Extract the airway skeleton points from the 3D visualization model;

[0032] The airway skeleton points are organized into a hierarchical structure using a multi-branch tree traversal algorithm to construct a tree-like digital bronchial tree.

[0033] Compared with the prior art, the interactive bronchoscopy navigation system and method of the present invention have one or more of the following beneficial technical effects:

[0034] 1. This invention, through an angle detection sensor, a specially designed coded pattern on the surface of the bronchoscope insertion tube, a transparent operating catheter, and a camera, can easily simulate the insertion operation of the camera at the tip of the insertion tube and obtain the insertion position data and attitude data of the camera at the tip of the insertion tube.

[0035] 2. The insertion position and attitude data of the camera at the tip of the bronchoscope obtained by this invention can be combined with the digital bronchial tree to simulate the insertion position and angle of the camera at the tip of the bronchoscope, thereby facilitating the simulation operation of the bronchoscope. This allows for preoperative rehearsal of the surgical plan for the patient's digital airway, thereby improving surgical efficiency, reducing surgical risks, and saving medical costs.

[0036] 3. This invention can realize the simulation operation of bronchoscopy, which facilitates intuitive communication with patients and their families before the operation, enabling them to understand their own situation and surgical plan more easily, thereby reducing the psychological burden on patients during the operation and increasing patient satisfaction. Attached Figure Description

[0037] Figure 1 A schematic diagram of the interactive bronchoscopy navigation system of the present invention is shown.

[0038] Figure 2 A schematic diagram of the navigation data acquisition device of the present invention is shown.

[0039] Figure 3 A plan view of a portion of the insertion tube in this invention, after axial unfolding, is shown to better illustrate the coded pattern thereon.

[0040] Figure 4 A schematic diagram of the digital bronchial tree generation module in this invention is shown.

[0041] Figure 5 A flowchart of the interactive bronchoscopy navigation method of the present invention is shown. Detailed Implementation

[0042] Before detailing any embodiment of the invention, it should be understood that the invention, in its application, is not limited to the details of the construction and arrangement of the components set forth in the following description or illustrated in the following figures. The invention can have other embodiments and can be practiced or carried out in various ways. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered limiting. The use of “comprising” or “having” and variations thereof is intended to cover the items set forth below and their equivalents, as well as any additional items. Unless otherwise specified or limited, the terms “installation,” “connection,” “support,” and “linkage,” and variations thereof are used broadly and cover both direct and indirect installation, connection, support, and linking. Moreover, “connection” and “linkage” are not limited to physical or mechanical connections or links.

[0043] Furthermore, firstly, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention. Secondly, the term "a" should be understood as "at least one" or "one or more," that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple. The term "a" should not be construed as a limitation on the quantity.

[0044] Figure 1 A schematic diagram of the interactive bronchoscopy navigation system of the present invention is shown. Figure 1 As shown, the interactive bronchoscopy navigation system of the present invention includes:

[0045] I. CT Data Acquisition Module.

[0046] The CT data acquisition module is used to acquire the patient's chest CT data.

[0047] In this invention, chest CT data of patients can be obtained before surgery using existing CT equipment in hospitals and other facilities.

[0048] To obtain high-quality chest CT data, it is preferable to use thin-slice scanning (less than 1 mm) with a pixel pitch of 0.5-0.8 mm during CT scanning. This ensures clear visualization of details in distal small airways and avoids branch breakage due to excessive slice thickness. Furthermore, a combination of spiral CT plain scan and contrast-enhanced scan is preferred. The spiral CT plain scan is used to acquire the basic airway structure, while the contrast-enhanced scan can distinguish between the airways and pulmonary arteries and veins (which are highly similar in grayscale values ​​and easily confused).

[0049] The raw data of chest CT scans obtained after scanning includes metadata such as pixel values, slice thickness, voxel spacing, and patient position, which can be directly read during subsequent processing.

[0050] Meanwhile, in this invention, after obtaining the original data, artifact correction can be performed on the original data, such as metal artifact removal and ray hardening correction, to avoid blurring of airway edges caused by equipment noise.

[0051] Furthermore, raw chest CT data suffers from noise, uneven grayscale, and background interference. To improve the quality of chest CT data, preprocessing can be performed to enhance the distinction between the airway and the background, laying a solid foundation for subsequent digital bronchial tree generation.

[0052] In this invention, the preprocessing of raw chest CT data includes:

[0053] 1. Threshold filtering and contrast enhancement.

[0054] (1) Threshold filtering: CT values ​​are filtered to a fixed range. The CT value of the airway region is usually 500-1000 HU. By using the threshold, non-airway tissues such as bones (>200 HU) and blood vessels (50-150 HU) can be removed, thus narrowing the processing range.

[0055] (2) Contrast enhancement: Existing contrast enhancement methods such as adaptive histogram equalization or Gaussian filtering + Laplacian sharpening can be used to enhance the gray difference between the airway wall and the surrounding lung tissue, especially highlighting the edge features of the distal small airways.

[0056] 2. Noise suppression.

[0057] Gaussian filtering and other methods can be used to smooth CT images and remove high-frequency noise. After removing high-frequency noise, CT images can be used to fill in tiny holes inside the airways and repair local breaks caused by insufficient scan resolution.

[0058] II. Navigation data acquisition device.

[0059] The navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope based on the chest CT data, and to obtain the insertion position data and attitude data of the camera.

[0060] like Figure 2 As shown, the navigation data acquisition device includes a bronchoscope, a transparent operating tube 5, and a camera 6 located above the transparent operating tube 5.

[0061] Similar to existing bronchoscopes, in this invention, the bronchoscope also includes a handle 1 and an insertion tube 3 connected to the handle 1. The handle 1 is equipped with a lever 2 and a cable connector 4. A camera (not shown) is located at the tip of the insertion tube 3. The cable connector 4 is connected to an external cable to transmit the image detected by the camera to the outside.

[0062] The handle 1 and lever 2 can be used to move the insertion tube 3 and the camera at its head. Specifically, moving the handle 1 up and down can move the insertion tube 3 and the camera at its head back and forth; rotating the handle 1 can rotate the insertion tube 3 and the camera at its head; and moving the lever 2 up and down can swing the camera at the head of the insertion tube 3.

[0063] Unlike existing bronchoscopes, this invention incorporates an angle detection sensor within the lever 2 of the bronchoscope. This angle detection sensor is used to acquire the swing angle of the lever 2. Therefore, the bending angle of the camera can be obtained from the swing angle of the lever 2 acquired by the angle detection sensor.

[0064] Meanwhile, in this invention, the surface of the bronchoscope insertion tube 3 is printed with multiple coded patterns. A camera at the tip of the insertion tube 3 is inserted into the transparent operating catheter 5, and after the camera at the tip of the insertion tube 3 is inserted into the transparent operating catheter 5, the camera 6 takes images of the insertion tube 3 inside the transparent operating catheter 5.

[0065] Each of the coded patterns printed on the surface of the insertion tube 3 includes a position mark from the head and a head rotation angle mark, to indicate the length by which the camera at the head end of the insertion tube 3 has been inserted into the transparent operating conduit 5 and the rotation angle of the camera at the head end of the insertion tube 3.

[0066] like Figure 3 As shown, each of the coded patterns includes a circumferential line 301 arranged around the circumference of the insertion tube 3, and four sets of numbers 302 and horizontal lines 303 arranged at 90° intervals around the circumference of the insertion tube 3. The circumferential line 301 and the numbers 302 determine the distance at which the camera at the head of the insertion tube 3 is inserted into the transparent operating guide tube 5. The combination of the numbers 302 and the horizontal lines 303 determines the rotation angle of the camera at the head of the insertion tube 3.

[0067] Specifically, starting 1 cm from the end face of the insertion tube 3, a circumferential line 301 is set at 5 cm intervals to mark the 5 cm length. Simultaneously, the insertion tube 3 is a cylindrical object, visible within a 90-degree range (1 / 4 of a cylinder) of the image from the camera 6. Therefore, a number 302 is set at 90-degree intervals to the left and right of the circumferential line 301; this number can be between 0 and 9. The number 302 indicates the physical length, ensuring that the camera 6 can recognize valid digital information in each frame.

[0068] Meanwhile, horizontal lines 303 are set at positions corresponding to the number 302, with lengths of 8mm, 4mm, 4mm, and 8mm, respectively, as indicators of the rotation angle of the camera at the end of the insertion tube 3. The terms "8mm horizontal line (right) + number (left)" indicate 0 degrees, where the 8mm horizontal line 303 is to the right of the circumference line 301 and the number 302 is to the left of the circumference line 301. "Number (right) + 4mm horizontal line (left)" indicates 90 degrees, where the number 302 is to the right of the circumference line 301 and the 4mm horizontal line 303 is to the left of the circumference line 301. "4mm horizontal line (right) + number (left)" indicates 180 degrees, where the 4mm horizontal line 303 is to the right of the circumference line 301 and the number 302 is to the left of the circumference line 301. "Number (right) + 8mm horizontal line (left)" indicates 270 degrees, where the number 302 is to the right of the circumference line 301 and the 8mm horizontal line 303 is to the left of the circumference line 301. In this way, by combining the number 302 with horizontal lines 303 of different lengths and varying their positions, the different rotation angles of the camera at the head of the insertion tube 3 can be identified.

[0069] Preferably, the coded pattern is printed on the surface of the insertion tube 3 using printing ink mixed with fluorescent agents. Therefore, due to the presence of fluorescent agents and the use of 200-400nm wavelength illumination, coupled with a high frame rate (90 frames / second) UV camera 6, the coded pattern on the surface of the insertion tube 3 is clearly imaged with neat edges and no ghosting, ensuring a high success rate for extracting the coded pattern from the captured images.

[0070] More preferably, the outer diameter of the insertion tube 3 is 6mm and the inner diameter of the transparent operating conduit 5 is 8mm, ensuring that the insertion tube 3 can rotate, move forward and backward normally after being inserted into the transparent operating conduit 5, and constraining the insertion tube 3 to maintain a straight shape without bending or deformation.

[0071] Furthermore, the distance between the lens of the camera 6 and the transparent operating conduit 5 is 6cm, and the imaging length of the insertion tube 3 is >7cm, ensuring that there is at least one set of valid coded patterns within the imaging range.

[0072] In this invention, the swing angle of the lever 2 is acquired by the angle detection sensor, and the bending angle of the camera at the head of the insertion tube 3 is calculated based on the swing angle of the lever 2. The camera 6 captures images of the insertion tube 3 inside the transparent operating guide tube 5, and the insertion position data and rotation angle data of the camera at the head of the insertion tube 3 can be obtained based on the coded pattern in the images.

[0073] Therefore, based on the acquired CT data, the insertion position of the camera at the tip of the insertion tube 3 and the corresponding rotation angle for each insertion position can be determined. Simultaneously, by operating the lever 2, the bending angle of the camera can be adjusted at the intersection of the left and right pulmonary ducts to allow the camera to enter either the left or right pulmonary duct.

[0074] In this invention, preferably, the navigation data acquisition device further includes a polarizing filter 7, which is installed between the camera 6 and the transparent operating guide 5. The polarizing filter 7 filters out highlights and reduces abnormal noise information during camera 6 shooting.

[0075] Furthermore, in this invention, the navigation data acquisition device further includes a base 8. The transparent operating conduit 5, the camera 6, and the polarizing filter 7 are all mounted on the base 8. This facilitates the installation and fixation of the transparent operating conduit 5, the camera 6, and the polarizing filter 7.

[0076] Preferably, the base 8 is provided with two retaining clips 9. Each retaining clip 9 includes two symmetrical blocks with a circular channel in the middle of the two symmetrical blocks. The transparent operating conduit 5 passes through the circular channel and is mounted on the two retaining clips 9. Thus, the two retaining clips 9 can better clamp the transparent operating conduit 5 onto the base 8, thereby better securing the transparent operating conduit 5.

[0077] More preferably, the base 8 is further provided with support rods 10, for example, four support rods 10. A camera mounting plate 12 is provided on the top of each support rod 10. The camera 6 is mounted on the camera mounting plate 12. Furthermore, the support rods 10 are height-adjustable support rods, such as telescopic support rods, to allow adjustment of the distance between the camera 6 and the transparent operating guide tube 5.

[0078] Preferably, the support rod 10 is further provided with a polarizing mirror mounting bracket 11. The polarizing mirror 7 is mounted on the polarizing mirror mounting bracket 11. This facilitates the installation of the polarizing mirror 7.

[0079] III. Digital Bronchial Tree Generation Module.

[0080] The digital bronchial tree generation module is used to generate a digital bronchial tree based on the chest CT data.

[0081] like Figure 4 As shown, the digital bronchial tree generation module includes:

[0082] 1. Airway segmentation submodule.

[0083] The airway segmentation submodule is used to segment the airway based on the chest CT data in order to extract a binary image of the airway.

[0084] Considering the shortcomings of traditional airway segmentation methods, such as low segmentation accuracy of distal small airways and easy destruction of topological structure, the airway segmentation submodule in this invention employs a deep learning-based segmentation method to segment the airway and extract a binary airway image.

[0085] Specifically, an improved 3D U-Net model is used for airway segmentation to extract binary airway images. The improved 3D U-Net model includes an encoder, a decoder, and an attention module. The encoder employs a 5-level downsampling module, each level containing: 2-3 3×3×3 convolutional layers to capture local spatial relationships and adapt voxel-level details in the airway scene; a batch normalization (BN) layer to stabilize training and alleviate gradient vanishing; a GELU activation function to introduce non-linearity and enhance feature representation; and a max-pooling layer for downsampling to expand the receptive field. Meanwhile, the encoder also includes a residual connection layer for cross-layer splicing of input and convolutional output to solve the gradient vanishing problem in deep networks; a dilated convolutional layer, which adopts a parallel design with multiple dilation rates to expand the receptive field without reducing resolution, while capturing the global structure of the main airway and the local details of the small airway; and a multi-scale input layer, which is used to generate feature maps of different resolutions (such as 1×1×1, 1 / 2×1 / 2×1 / 2) through pyramid pooling to adapt to the multi-scale branching characteristics of the airway.

[0086] The decoder is symmetrically designed with the encoder, also featuring a 5-level upsampling module. Each level includes: an upsampling layer, employing transposed convolution or pixel shuffling to improve resolution and restore spatial location; a feature fusion layer, used for concatenation or addition operations to fuse shallow detail features of the encoder at the same scale with semantic features of the decoder; a 1×1×1 convolutional layer, used to compress the number of channels, reduce computational cost, and integrate the fused multi-dimensional features. The decoder also includes a nested connection layer, used to directly inject high-level encoder features into the corresponding decoder level to enhance semantic transmission; a boundary aggregation layer, used to directly concatenate low-level features into bi-branch features (preserving small airway edges), and dynamically match semantically related regions at higher levels; and a diffusion denoising layer, used to optimize the resolution recovery of difficult-to-segment regions (far-end small airways) through multiple noisy inferences.

[0087] The attention module includes a spatial attention submodule, which calculates weights on the spatial dimension of the feature map using a 3×3×3 convolutional layer to generate a spatial attention map and assigns weights per voxel; a channel attention submodule, which calculates weights on the channel dimension of the feature map to generate a channel attention vector, highlighting the channels corresponding to airway features; a hybrid attention submodule, which connects the spatial attention submodule and the channel attention submodule in series or in parallel, while optimizing the weight allocation of spatial location and channel features; and an airway-specific attention submodule, which injects global topological semantics of the airway into local features through trainable generalized average pooling, enhancing the distinction between small airways and main airways, calculating feature similarity, dynamically matching the associated regions of airway boundaries and background, and resolving feature confusion between main airways and pulmonary vessels.

[0088] During airway segmentation, CT data is input into the encoder and downsampled at 5 levels to generate multi-scale feature maps from shallow to deep layers while filtering out noise. The decoder gradually restores the resolution through 5 levels of upsampling, fusing features of the same scale as the encoder through skip connections at each level. Attention modules are inserted at key layers of the encoder and decoder to assign high weights to airway features and suppress interference from blood vessels and background. Finally, the decoding layer maps the multi-channel features to binary classification (airway / non-airway) results through 1×1×1 convolution, extracts the airway results to obtain a binary airway image with the same resolution as the input CT data.

[0089] 2. 3D Reconstruction Submodule.

[0090] The 3D reconstruction submodule is used to perform 3D reconstruction based on the airway binary image to generate a 3D visualization model.

[0091] In this invention, the 3D reconstruction submodule generates a 3D visualization model using surface reconstruction technology. Specifically, it first traverses all voxels of the airway binary image, determining whether each voxel contains the airway surface, and generates a 3D mesh model composed of triangular facets. The airway edge voxels are subdivided for sampling to avoid jagged surface artifacts. Then, Laplacian smoothing or bilateral filtering is performed on the generated 3D mesh model to make the surface smoother while preserving the airway topology.

[0092] 3. Skeleton Extraction Submodule.

[0093] The skeleton extraction submodule is used to extract airway skeleton points from the three-dimensional visualization model.

[0094] While the 3D visualization model is intuitive, it is difficult to directly use for quantitative analysis (such as branch length, tube diameter, bifurcation angle, etc.). Therefore, it is necessary to extract a one-dimensional skeleton, i.e., airway skeleton points. In this invention, extracting airway skeleton points from the 3D visualization model specifically includes:

[0095] (1) Preprocessing: The three-dimensional visualization model is preprocessed by first erosion and then expansion to remove surface burrs and ensure smooth boundaries.

[0096] (2) Simple point deletion: Iterative deletion of simple points. The simple point is a point that has only one foreground point (line endpoint) in the 26 neighborhood, has a change of 0 in the Euler characteristic number, and does not satisfy connectivity.

[0097] (3) Candidate point recovery: The deleted simple points are used as candidate points and divided into 8 groups (each pair is not connected to the other). The connectivity of each group of candidate points is judged in turn. Candidate points that do not meet the connectivity are remarked as skeleton points to avoid breakpoints due to excessive deletion.

[0098] In addition, after extracting the airway skeleton points, skeleton optimization can be performed, that is, the skeleton position can be adjusted by using central axis transformation to ensure that the skeleton is located on the airway centerline.

[0099] 4. Construct sub-modules using a tree structure.

[0100] The tree structure construction submodule is used to organize the airway skeleton points into a hierarchical structure using a multi-branch tree traversal algorithm to construct a tree-structured digital bronchial tree.

[0101] In this invention, the extracted skeleton points are organized into a hierarchical structure using a multi-branch tree traversal algorithm to construct a tree-like digital bronchial tree, thereby achieving orderly management and quantitative analysis of the airways. Specifically, this includes:

[0102] (1) Root node determination: The root node is the beginning of the trachea (the position where it connects to the larynx) (level 1).

[0103] (2) Neighborhood traversal: Starting from the root node, traverse the 26 neighborhoods of each skeleton node, distinguishing between connection points (2 neighborhood points), branching points (3 or more neighborhood points), and leaf nodes (1 neighborhood point).

[0104] (3) Node classification: The root node is level 1; at the bifurcation point, the current branch level is the same as the parent node, and the next level branch level is +1 (e.g., the main bronchus is level 2, the lobar bronchus is level 3, the segmental bronchus is level 4, and so on); the leaf node (the end of the distal small airway) has the same level as the parent node.

[0105] (4) Structure output: The tree structure is stored in the form of nodes-edges, and each edge records parameters such as branch length, pipe diameter, and direction angle.

[0106] IV. Navigation Data Mapping Module.

[0107] The navigation data mapping module maps the camera's insertion position and attitude data into the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, it performs motion calculations on the virtual camera (which simulates the camera at the tip of the insertion tube 3) to obtain the virtual camera's pose within the digital airway lumen, thus achieving simulation operation. In other words, based on the camera's insertion position and attitude data, the position and attitude of the virtual camera at the tip of the virtual bronchoscope's insertion tube are determined within the digital bronchial tree, enabling the depiction of a virtual image under the bronchoscope.

[0108] Figure 5 A flowchart of the interactive bronchoscopy navigation method of the present invention is shown. Figure 5 As shown, the interactive bronchoscopy navigation method of the present invention includes the following steps:

[0109] 1. CT data acquisition.

[0110] The patient's chest CT data is acquired using the CT data acquisition module. Of course, the chest CT data can be preprocessed after acquisition.

[0111] 2. Navigation data acquisition.

[0112] Based on the chest CT data, the navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope insertion tube to obtain the insertion position data and orientation data of the camera.

[0113] 3. Digital bronchial tree generation.

[0114] A digital bronchial tree is generated based on the chest CT data using the digital bronchial tree generation module. Specifically, it includes:

[0115] First, the airway is segmented based on the chest CT data to extract a binary image of the airway.

[0116] Then, a three-dimensional reconstruction is performed based on the binary airway image to generate a three-dimensional visualization model.

[0117] Next, the airway skeleton points are extracted from the three-dimensional visualization model.

[0118] Finally, the airway skeleton points are organized into a hierarchical structure using a multi-branch tree traversal algorithm to construct a tree-like digital bronchial tree.

[0119] 4. Navigation data mapping.

[0120] The navigation data mapping module is used to map the insertion position data and attitude data of the camera onto the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, the motion calculation of the virtual camera is performed to obtain the pose of the virtual camera in the digital airway lumen, thereby realizing the simulation operation.

[0121] In this invention, the patient first undergoes a preoperative chest CT scan to obtain chest CT data. Then, based on this chest CT data, routine CT data can be viewed, and automated three-dimensional segmentation and reconstruction of the trachea, bronchi, and pulmonary vessels can be performed to form a digital bronchial tree. Surgical target points can be defined, and a bronchoscopic surgical plan can be planned. Simultaneously, a navigation data acquisition device can be used to simulate camera insertion, obtaining the camera's insertion position and posture data. Based on this data, an interactive rehearsal of the surgical plan can be conducted on the digital bronchial tree. This facilitates preoperative familiarization with the surgical path, identification of intraoperative risk points, improved preoperative meeting efficiency, and reduced intraoperative risks. Furthermore, the interactive rehearsal of the surgical plan on the digital bronchial tree facilitates preoperative communication with the patient and their family, enabling them to more easily understand their condition and the surgical plan, reducing intraoperative psychological burden and improving patient satisfaction.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention based on the concept of the present invention, without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. An interactive bronchoscopy navigation system, characterized in that, include: The CT data acquisition module is used to acquire the patient's chest CT data; A navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope based on the chest CT data, and to obtain the insertion position data and attitude data of the camera. A digital bronchial tree generation module, which is used to generate a digital bronchial tree based on the chest CT data; The navigation data mapping module is used to map the insertion position data and attitude data of the camera to the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, the motion calculation of the virtual camera is performed to obtain the pose of the virtual camera in the digital airway lumen and realize the simulation operation. The navigation data acquisition device includes a bronchoscope, a transparent operating tube (5), and a camera (6) located above the transparent operating tube (5). An angle detection sensor is installed in the lever (2) of the bronchoscope. The angle detection sensor is used to collect the swing angle of the lever (2). The surface of the insertion tube (3) of the bronchoscope is printed with multiple coded patterns. The camera at the head end of the insertion tube (3) is inserted into the transparent operating tube (5). After the camera at the head end of the insertion tube (3) is inserted into the transparent operating tube (5), the camera (6) takes a picture of the insertion tube (3) inside the transparent operating tube (5). The insertion position data and attitude data of the camera at the head end of the insertion tube (3) are obtained based on the coded patterns in the picture and the swing angle of the lever (2).

2. The interactive bronchoscopy navigation system according to claim 1, characterized in that, Each of the coded patterns printed on the surface of the insertion tube (3) includes a position mark from the head and a head rotation angle mark.

3. The interactive bronchoscopy navigation system according to claim 2, characterized in that, Each of the coded patterns includes a circumferential line (301) arranged around the circumference of the insertion tube (3) and four sets of numbers (302) and horizontal lines (303) arranged at 90° intervals around the circumference of the insertion tube (3).

4. The interactive bronchoscopy navigation system according to claim 3, characterized in that, The coded pattern is printed on the surface of the insertion tube (3) using printing ink mixed with fluorescent agents.

5. The interactive bronchoscopy navigation system according to claim 1, characterized in that, The navigation data acquisition device further includes a polarizing mirror (7) and the polarizing mirror (7) is installed between the camera (6) and the transparent operating conduit (5).

6. The interactive bronchoscopy navigation system according to any one of claims 1-5, characterized in that, The digital bronchial tree generation module includes: An airway segmentation submodule is used to segment the airway based on the chest CT data in order to extract a binary image of the airway. A 3D reconstruction submodule is used to perform 3D reconstruction based on the airway binary image to generate a 3D visualization model; The skeleton extraction submodule is used to extract airway skeleton points from the three-dimensional visualization model. The tree structure construction submodule is used to organize the airway skeleton points into a hierarchical structure through a multi-branch tree traversal algorithm to construct a tree-structured digital bronchial tree.

7. The interactive bronchoscopy navigation system according to claim 6, characterized in that, The airway segmentation submodule uses a deep learning-based segmentation method to segment the airway and extract a binary image of the airway.

8. An interactive bronchoscopy navigation method, characterized in that, It employs the interactive bronchoscopy navigation system as described in any one of claims 1-7 and includes the following steps: The patient's chest CT data is acquired using the CT data acquisition module. Based on the chest CT data, the navigation data acquisition device is used to simulate the insertion operation of the camera at the tip of the bronchoscope insertion tube to obtain the insertion position data and posture data of the camera. A digital bronchial tree is generated based on the chest CT data using the digital bronchial tree generation module. The navigation data mapping module is used to map the insertion position data and attitude data of the camera onto the digital bronchial tree. Under the constraints of the digital airway lumen of the digital bronchial tree, the motion calculation of the virtual camera is performed to obtain the pose of the virtual camera in the digital airway lumen, thereby realizing the simulation operation.

9. The interactive bronchoscopy navigation method according to claim 8, characterized in that, Generating a digital bronchial tree based on the chest CT data using the digital bronchial tree generation module specifically includes: Based on the chest CT data, the airway is segmented to extract a binary image of the airway; Three-dimensional reconstruction is performed based on the binary airway image to generate a three-dimensional visualization model; Extract the airway skeleton points from the 3D visualization model; The airway skeleton points are organized into a hierarchical structure using a multi-branch tree traversal algorithm to construct a tree-like digital bronchial tree.

Images