Bronchoscope navigation operation system and method based on bronchial tree naming

Abstract

The invention provides a bronchoscope navigation operation system and method based on bronchial tree naming, and the system comprises a three-dimensional reconstruction module which is used for constructing a three-dimensional bronchial tree model; the bronchus naming and labeling module is used for naming branches at all levels of the bronchus tree; the virtual image generation module is used for generating a plurality of bronchial branch virtual images with bronchial branch data; the route generation module is used for forming a planned lens entering route taking a glottis gate as a starting point and a target bronchus as an ending point on the three-dimensional bronchus tree model; the image fusion module is used for acquiring a bronchial branching real image of each stage of bronchial tube on an actual endoscope entering path in real time and performing overlapping fusion on the bronchial branching real image and the optimal matching bronchial branching virtual image; and the auxiliary navigation module is used for assisting in completing bronchoscope operation according to the fused bronchial branching real image. The bronchoscope operation is rapidly and accurately completed through a semi-automatic auxiliary guiding scheme.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical examination, in particular to a bronchoscope navigation operation system and method based on bronchial tree naming. BACKGROUND

[0002] Bronchoscope operation is a common means of internal examination and treatment. Existing bronchoscope navigation technologies include electromagnetic navigation bronchoscopy (ENB), virtual bronchoscope navigation (VBN), ultrasonic bronchoscopy (EBUS), optical coherence tomography (OCT), and robot-assisted bronchoscopy (RAB).

[0003] However, due to the complex anatomical structure of the bronchial tree of the human body, endoscopic physicians, especially novice physicians, are difficult to quickly identify each segment, sub-segment, and even sub-sub-segment bronchus during endoscopic operation if they have not undergone professional and systematic bronchial dissection training, and thus are prone to lose direction during operation, resulting in the inability to quickly and accurately complete bronchoscope operation. SUMMARY

[0004] To solve the above problems of the prior art, the present application provides a bronchoscope navigation operation system and method based on bronchial tree naming to quickly and accurately complete bronchoscope operation.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows: In a first aspect, the present application provides a bronchoscope navigation operation system based on bronchial tree naming, comprising: a three-dimensional reconstruction module for constructing a three-dimensional bronchial tree model; a bronchial naming labeling module for naming each level of bronchial bifurcation to obtain a three-dimensional bronchial tree model with bronchial naming; a virtual image generation module for dividing the three-dimensional bronchial tree model with bronchial naming into multiple local classifiers according to the anatomical rules of the bronchial tree and bronchial bifurcation, analyzing and sampling each local classifier to generate multiple bronchial bifurcation virtual images with bronchial bifurcation data; a route generation module for obtaining a target bronchus and forming a planned bronchoscope insertion route on the three-dimensional bronchial tree model with bronchial naming, with the glottis as the starting point and the target bronchus as the endpoint; an image fusion module for obtaining real bronchial bifurcation images of each level of bronchus on the actual bronchoscope insertion route in real time when performing bronchoscope operation according to the planned bronchoscope insertion route, and superimposing and fusing the real bronchial bifurcation images with the most matching virtual bronchial bifurcation images thereof; an auxiliary navigation module for assisting in completing bronchoscope operation according to the fused real bronchial bifurcation images.

[0006] The beneficial effects of the present application are that: by constructing a three-dimensional bronchial tree model, and taking the branch opening of each bronchial branch in the three-dimensional bronchial tree model to obtain a bronchial branch virtual image with bronchial branch opening data. When performing bronchoscopy, the planned mirror route can be automatically generated to guide the doctor, and the bronchial branch opening data of the bronchial branch virtual image is compared and fused with the real bronchial branch image to assist the doctor to complete the bronchoscopy, so that the bronchoscopy is quickly and accurately completed through the semi-automatic auxiliary guidance scheme, and the operation safety is improved, and the health risk caused by misoperation is reduced.

[0007] Optionally, the local classifier is provided with eight, respectively corresponding to eight bronchial branches with anatomical recognition in the bronchial tree, and classified into a first group including each segment of the right upper lobe bronchus and each segment of the left inherent upper lobe bronchus, a second group including each segment of the right middle lobe bronchus and each segment of the left lingual lobe bronchus, a third group including each segment of the right lower lobe back segment bronchus and each segment of the left lower lobe back segment bronchus, and a fourth group including each segment of the right lower lobe basal segment bronchus and each segment of the left lower lobe basal segment bronchus.

[0008] According to the above description, the bronchial tree is grouped and segmented according to the anatomical recognition, so as to improve the recognition and matching efficiency by using the similarity of anatomical features in the group.

[0009] Optionally, the bronchial naming module specifically adopts different character rules for naming each level of bronchus according to the preset order of different groups when naming each level of bronchus of the bronchial tree. The planned mirror route is also provided with route characters, which are generated in sequence according to the bronchial naming of each bronchus passed by the planned mirror route.

[0010] According to the above description, different character rules are used for naming each level of bronchus, so that the bronchus passed by the planned mirror route can be clearly planned through the route characters by generating the route characters of the planned mirror route through the bronchial naming, so as to more accurately complete the bronchoscopy.

[0011] Optionally, the bronchus on the local classifier is a cavity type model with an internal hollow on the three-dimensional bronchial tree model, and the virtual image generation module includes the following when analyzing and photographing each local classifier respectively to generate multiple bronchial branch virtual images with bronchial branch opening data: The cavity model in each local classifier is perspective, and the bronchial branching opening data of the bronchial opening at each level of the bronchial branching is obtained, and the bronchial branching virtual image with the bronchial branching opening data is saved by taking a screenshot, wherein the bronchial branching opening data includes the number, the size of the tube diameter and the relative position of the bronchial opening at each level.

[0012] Optionally, the auxiliary navigation module is further used for: The laser detection sensor added to the bronchoscope head is used to detect the distance between the front, back, left and right directions of the bronchoscope head and the tracheal wall, and if the distance of any direction exceeds the preset distance threshold, a prompt instruction is sent out. The force sensor integrated at the operation handle of the bronchoscope is used to detect that the resistance when the bronchoscope is pushed exceeds the preset resistance threshold, and a prompt instruction is sent out.

[0013] Optionally, the bronchial naming labeling module is further used for naming the three-dimensional external space of the left bronchial tree and the right bronchial tree respectively, and obtaining a three-dimensional bronchial tree model with bronchial naming and spatial naming. The bronchial branching virtual image is also labeled with the up, down, left and right spatial names.

[0014] According to the above description, it can be known that the spatial naming can better assist the doctor to enter the correct bronchus according to the indication.

[0015] Optionally, the auxiliary navigation module is specifically used for: The bronchial branching real image with the bronchial naming after fusion is directly displayed on the endoscope host display screen, and the position of the bronchial branching real image is displayed on the four sides of the display screen, and the operation instruction is generated according to the planned bronchoscope route and the position, and the operation instruction is voice broadcasted and / or real-time displayed, so as to assist the completion of the bronchoscope operation. When the bronchoscope deviates from the planned bronchoscope route or reaches the bronchial branching intersection, the voice broadcast and / or real-time display of the prompt instruction on how to enter the planned bronchus is performed.

[0016] Optionally, the three-dimensional reconstruction module is specifically used for: The chest thin layer CT is obtained, and the three-dimensional bronchial tree model is formed by three-dimensional reconstruction of the bronchus of the cavity according to the chest thin layer CT.

[0017] Optionally, the image fusion module is used for overlapping and fusing the bronchial branching real image with the bronchial branching virtual image that is most matched with the bronchial branching real image. The bronchial branch real image is drawn a circle and the center of the circle is labeled in each bronchial lumen of each bronchial branch virtual image, the centers of the circles are connected into a center pattern in turn, the two pictures are overlapped, and the bronchial branch virtual image that is most matched with the bronchial branch real image is obtained according to the coincidence degree of the circle and the center pattern, and the bronchial branch real image is overlapped and fused with the bronchial branch virtual image that is most matched with the bronchial branch real image.

[0018] In a second aspect, the present application provides a bronchoscope operation method based on a bronchial tree naming local classifier, comprising the following steps: constructing a three-dimensional bronchial tree model; naming each level of bronchial branch to obtain a three-dimensional bronchial tree model with bronchial naming; According to the anatomical law of the bronchial tree and the bronchial branch, the three-dimensional bronchial tree model with bronchial naming is divided into a plurality of local classifiers, each local classifier is analyzed and photographed respectively, and a plurality of bronchial branch virtual images with bronchial branch opening data are generated; obtaining a target bronchus, forming a planned mirror insertion route starting from the glottis and ending at the target bronchus on the three-dimensional bronchial tree model with bronchial naming; When the bronchoscope operation is performed according to the planned mirror insertion route, the bronchial branch real image of each level of bronchus on the actual mirror insertion route is obtained in real time, and the bronchial branch real image is overlapped and fused with the bronchial branch virtual image that is most matched with the bronchial branch real image; According to the fused bronchial branch real image, the bronchoscope operation is assisted to be completed.

[0019] The technical effects corresponding to the bronchoscope operation method based on the bronchial tree naming local classifier provided in the second aspect are referred to the related description of the bronchoscope navigation operation system based on the bronchial tree naming provided in the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The structure diagram of the bronchoscope navigation operation system based on the bronchial tree naming of the embodiment of the present application; Figure 2 The structure diagram of the three-dimensional bronchial tree model divided into 4 groups and 8 parts related to the embodiment of the present application; Figure 3 The main flow diagram of the bronchoscope operation method based on the bronchial tree naming local classifier of the embodiment of the present application.

[0021] Explanation of reference signs: 1, three-dimensional reconstruction module; 2, bronchial naming labeling module; 3. a virtual image generation module; 4. a route generation module; 5. an image fusion module; 6. an auxiliary navigation module. DETAILED DESCRIPTION

[0022] In order to better understand the above technical solutions, the exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present application can be more clearly, thoroughly understood and the scope of the present application can be completely conveyed to those skilled in the art.

[0023] Embodiment one Please refer to Figure 1 , the bronchoscope navigation operation system based on bronchial tree naming includes a three-dimensional reconstruction module, a bronchial naming labeling module, a virtual image generation module, a route generation module, an image fusion module and an auxiliary navigation module. The following is a specific description of each module.

[0024] 1. The three-dimensional reconstruction module is used to construct a three-dimensional bronchial tree model.

[0025] In this embodiment, the three-dimensional reconstruction module is specifically used to construct a three-dimensional bronchial tree model when constructing a three-dimensional bronchial tree model. Obtain chest thin layer CT, and perform three-dimensional reconstruction on the bronchus of the cavity according to the chest thin layer CT to form a three-dimensional bronchial tree model.

[0026] In this embodiment, when a user needs bronchoscope operation, the chest thin layer CT of the user is first input into the system to generate a three-dimensional bronchial tree model.

[0027] Please refer to Figure 2 It can be seen that there are eight bronchial branches with anatomical recognition in the bronchial tree, which are classified into four groups: the first group A group including the branch points of each segment of the right upper lobe bronchus (RB 1+2+3 ) and the left inherent upper lobe bronchus (LB 1+2+3 ), the second group B group including the branch points of each segment of the right middle lobe bronchus (RB 4+5 ) and the left lingual lobe bronchus (LB 4+5 ), the third group C group including the branch points of each segment of the right lower lobe dorsal segment bronchus (RB 6 ) and the left lower lobe dorsal segment bronchus (LB 6 ), and the fourth group D group including the branch points of each segment of the right lower lobe basal segment bronchus (RB 7+8+9+10 ) and the left lower lobe basal segment bronchus (LB 8+9+10Group D, the fourth group of the classifiers. Name the subsequent local classifiers in the order described above.

[0028] 2. The bronchial naming and labeling module is used to name the branches of the bronchial tree at each level, resulting in a three-dimensional bronchial tree model with bronchial names.

[0029] In this embodiment, the bronchus naming and labeling module is also used to name the three-dimensional external space of the left bronchus tree and the right bronchus tree respectively, so as to obtain a three-dimensional bronchus tree model with bronchus naming and space naming.

[0030] The naming of the three-dimensional external space of the bronchial tree is as follows: The bronchial tree is divided into left and right sides. The naming of the six surfaces of the three-dimensional external space of the left bronchial tree, namely the anterior chest surface, posterior back surface, left axillary surface, right mediastinal surface, cephalic surface, and pedispheric surface, are as follows. The naming of the six surfaces of the three-dimensional external space of the right bronchial tree, namely the anterior chest surface, posterior back surface, left mediastinal surface, right axillary surface, cephalic surface, and pedispheric surface, are as follows. Reference Figure 2 As can be seen, *cranial* indicates the region is located near the head, *media* indicates the region is near the medial side, and *cranial(media)* indicates the region is located in the cephalic region or on the medial mediastinum side. *Thoraciic* describes the region as being near the anterior chest region. *Caudal* indicates the region is located near the foot. *Caudal(lateral)* indicates the region is located near the foot or lateral axilla. *Dorsal* describes the region as being near the back. *Lateral* is used to locate the region on the lateral axilla of the body.

[0031] In this embodiment, the bronchial naming and labeling module names the branches of the bronchial tree at each level by using different character rules according to the preset order of different groups.

[0032] The bronchial tree branches are named as follows: the trachea is defined as a level 0 bronchus, the bilateral main bronchi as level 1 bronchi, the lobar bronchus as level 2 bronchi, the segmental bronchus as level 3 bronchi, the subsegmental bronchus as level 4 bronchi, and so on.

[0033] Bronchi of grades 2-4 were matched with the large database of anatomical opening types of lobar and segmental bronchi according to the eight parts mentioned above, and their names were determined. Bronchi of grade 5 and above were named according to the following naming principles.

[0034] The overall rules are detailed below: the upper right lobe and the upper left lobe are named in the order of head before foot, back before chest, and inside before outside; the middle right lobe, the left lingula, and both lower lobes are named in the order of head before foot, back before chest, and outside before inside.

[0035] Specifically named characters as follows: main bronchus using RB or LB; segment bronchus using Arabic numerals (1~10) naming; sub-segment bronchus using a, b, c naming; 5 level bronchus using i, ii, iii naming; 6 level bronchus using alpha, beta, gamma naming; each level bronchus after using x, y, z naming. Finally form a complete naming system.

[0036] In this embodiment, the process of naming algorithm is described as follows: 1. Initialization: Starting from the trachea (0 level), recursively traverse each node of the bronchial tree.

[0037] 2. Naming rules: - 0 level: trachea.

[0038] - 1 level: left main bronchus (LB) or right main bronchus (RB).

[0039] - 2 level: according to the side and region naming leaf bronchus (such as right upper lobe, left proper upper lobe, etc.).

[0040] - 3 level: segment bronchus, using Arabic numerals (1-10) naming.

[0041] - 4 level: sub-segment bronchus, using letters (a, b, c) naming.

[0042] - 5 level: using Roman numerals (i, ii, iii) naming.

[0043] - 6 level: using Greek letters (alpha, beta, gamma) naming.

[0044] - 7 level and above: using x, y, z naming.

[0045] 3. Recursion traversal: recursively call the naming function for the left and right child nodes of each node until the entire bronchial tree is traversed.

[0046] 4. Match database: in the naming process, match with the large database of leaf segment bronchus anatomical opening type to ensure the accuracy of naming.

[0047] In this embodiment, the partial code of the naming algorithm is as follows: plaintext FUNCTION NameBronchus(node, level, side, region): IF node is NULL: RETURN IF level == 0: node.name = "Trachea" ELSE IF level == 1: IF side == "right": node.name = "RB" ELSE: node.name = "LB" ELSE IF level == 2: node.name = GetLobarName(side, region) ELSE IF level == 3: node.name = GetSegmentalNumber(side, region) ELSE IF level == 4: node.name = GetSubsegmentalLetter(side, region) ELSE IF level == 5: node.name = GetFifthLevelRoman(side, region) ELSE IF level == 6: node.name = GetSixthLevelGreek(side, region) ELSE: node.name = GetBeyondSixthLevel(side, region) NameBronchus(node.left, level + 1, side, region) NameBronchus(node.right, level + 1, side, region) FUNCTION GetLobarName(side, region): IF side == "right": IF region == "upper": RETURN "Right Upper Lobe" ELSE IF region == "middle": RETURN "Right Middle Lobe" ELSE: RETURN "Right Lower Lobe" ELSE: IF region == "upper": RETURN "Left Upper Lobe" ELSE IF region == "lingula": RETURN "Left Lingula" ELSE: RETURN "Left Lower Lobe" FUNCTION GetSegmentalNumber(side, region): # Return the number of segmental bronchus according to region and side RETURN segment_number FUNCTION GetSubsegmentalLetter(side, region): # Return the letter of subsegmental bronchus according to region and side RETURN subsegmental_letter FUNCTION GetFifthLevelRoman(side, region): # Return the Roman numeral of 5th level bronchus according to region and side RETURN roman_numeral FUNCTION GetSixthLevelGreek(side, region): # Return the Greek letter of 6th level bronchus according to region and side RETURN greek_letter FUNCTION GetBeyondSixthLevel(side, region): # Return the naming of bronchus beyond the 6th level according to region and side RETURN "x"# or other logic In this embodiment, the mind map of the naming algorithm is as follows: Trachea (0) ├── LB (1) │├── Left Upper Lobe (2) ││├── Segment 1 (3) │││├── a (4) ││││├── i (5) │││││├── α (6) ││││││├── x (7) ││││││└── y (7) │││││└── β (6) ││││└── ii (5) │││└── b (4) ││└── Segment 2 (3) │├── Left Lingula (2) │└── Left Lower Lobe (2) └── RB (1) ├── Right Upper Lobe (2) ├── Right Middle Lobe (2) └── Right Lower Lobe (2) Wherein the Chinese expression is as follows: Root node: trachea (0 level) - Left main bronchus (1 level, LB) - Left solid upper lobe (2 level) - Segment bronchus (3 level, 1-10) - Sub-segment bronchus (4 level, a, b, c) - 5 level bronchus (i, ii, iii) - 6 level bronchus (α, β, γ) - 7 level and above (x, y, z) - Left lingual lobe (2 level) - Segment bronchus (3 level, 1-10) - Sub-segment bronchus (4 level, a, b, c) - 5 level bronchus (i, ii, iii) - 6 level bronchus (α, β, γ) - 7 level and above (x, y, z) - Left lower lobe (2 level) - Segment bronchus (3 level, 1-10) - Subsegmental bronchi (4th order, a, b, c) - 5th order bronchi (i, ii, iii) - 6th order bronchi (a, b, g) - 7th order and above (x, y, z) - Right main bronchus (1st order, RB) - Right upper lobe (2nd order) - Segmental bronchi (3rd order, 1-10) - Subsegmental bronchi (4th order, a, b, c) - 5th order bronchi (i, ii, iii) - 6th order bronchi (a, b, g) - 7th order and above (x, y, z) - Right middle lobe (2nd order) - Segmental bronchi (3rd order, 1-10) - Subsegmental bronchi (4th order, a, b, c) - 5th order bronchi (i, ii, iii) - 6th order bronchi (a, b, g) - 7th order and above (x, y, z) - Right lower lobe (2nd order) - Segmental bronchi (3rd order, 1-10) - Subsegmental bronchi (4th order, a, b, c) - 5th order bronchi (i, ii, iii) - 6th order bronchi (a, b, g) - 7th order and above (x, y, z) 3. A virtual image generation module, configured to divide a three-dimensional bronchial tree model with bronchial naming into a plurality of local classifiers according to the anatomical rules of the bronchial tree and bronchial bifurcations, and analyze and sample each local classifier to generate a plurality of virtual images of bronchial bifurcations with bifurcation data.

[0048] In this embodiment, the local classifiers are provided with eight, which correspond to eight bronchial bifurcations with anatomical recognition in the bronchial tree. Specifically, the eight parts are divided into four groups according to the above list.

[0049] From the anatomical structure, each of the four groups contains two parts of the left and right lung symmetry or similar function and structure, wherein the right upper lobe of the first group and the left proper upper lobe, the right middle lobe of the second group and the left lingual lobe belong to the upper lobe of the lung and the similar area, and the bronchial branching form and opening rule have high homology; the double lower lobe dorsal segments of the third group and the double lower lobe basal segments of the fourth group belong to the specific functional segments of the lower lobe of the lung, and the branching structure characteristics are similar. Therefore, the individual identification of the eight bronchial branching places is retained, at the same time, the analysis logic of the complex bronchial tree is simplified through the classification of the group, so that the two parts in the same group can share similar anatomical feature analysis standards and virtual image matching parameters, avoid analysis redundancy caused by individual differences when directly divided into eight groups, thereby more efficiently supporting the operation of subsequent virtual image generation, real image comparison and fusion modules, and finally improving the overall efficiency and accuracy of the inspection system.

[0050] In the embodiment, the bronchus on the local classifier is a cavity type model reconstructed into an internal hollow on the three-dimensional bronchial tree model. Therefore, when the virtual image generation module analyzes and samples each local classifier respectively to generate multiple bronchial branching virtual images with bronchial branching opening data, it includes: Perspective of the cavity type model in each local classifier to obtain bronchial branching opening data of bronchial openings at each level of bronchial branching, and screenshot saving as bronchial branching virtual images with bronchial branching opening data.

[0051] In the embodiment, the bronchial branching opening data includes the number, size and relative position of the bronchial openings at each level.

[0052] 4. A route generation module for obtaining a target bronchus, forming a planned advancement route on the three-dimensional bronchial tree model with bronchial naming, starting from the glottis and ending at the target bronchus.

[0053] In the embodiment, the planned advancement route is also provided with route characters, which are generated in sequence according to the bronchial naming of each bronchus passed by the planned advancement route, such as RB3aiβxy, so that the route characters of the planned advancement route are generated by bronchial naming, and the bronchus passed by the planned advancement route can be clearly planned by the route characters, so as to more accurately complete the bronchoscopic operation.

[0054] The bronchial tree is a three-dimensional structure, and when advancing along the roadmap, frequent different direction turning occurs, and the plane of the lens also constantly changes with the three-dimensional space of the bronchial tree. Therefore, in the embodiment, the bronchial branching virtual image is also marked with four spatial names of up, down, left and right, so that the spatial names can better assist the doctor to enter the correct bronchus according to the indication. For example, when the lens is in the left bronchus, and the plane of the lens has the largest projection area with the axial plane of the human body, the four directions corresponding to the four sides of the axial plane, that is, the four directions of up, down, left and right on the virtual image are the front chest side, the back side, the left axillary side and the right mediastinal side respectively.

[0055] 5. An image fusion module, configured to acquire a bronchial branching real image of each level of bronchus on an actual bronchoscope insertion route in real time when performing bronchoscope operation according to a planned insertion route, and to overlap and fuse the bronchial branching real image with a bronchial branching virtual image that is most matched with the bronchial branching real image.

[0056] In the embodiment, the image fusion module is configured to overlap and fuse the bronchial branching real image with the bronchial branching virtual image that is most matched with the bronchial branching real image in the following manner: The bronchial branching real image and each bronchial lumen in each bronchial branching virtual image are drawn with a circle and marked with a circle center, the circle centers are sequentially connected into a circle center pattern, and then the two images are overlapped, the bronchial branching virtual image that is most matched with the bronchial branching real image is obtained according to the overlapping degree of the circle and the circle center pattern of each bronchial lumen, and the bronchial branching real image is overlapped and fused with the bronchial branching virtual image that is most matched with the bronchial branching real image.

[0057] The endoscope host image output end is connected with the bronchoscope operation system, the real image of each level of bronchus branching at the port on the insertion route under the bronchoscope is input to the bronchoscope operation system, is analyzed and recognized, and is compared with the database of the virtual bronchial branching image that has been named and marked in the bronchoscope operation system. The compared data includes the number, size, shape and relative position of the bronchial branching port in the bronchial tree cavity. Then, the endoscope image in the real world and the virtual image are overlapped and fused with each other, and the naming identification of each level of bronchus is consistent with the naming identification on the virtual bronchial tree.

[0058] The circle center pattern is a line segment or a polygon, for example, a line segment when there are only two bronchi, a triangle when there are three bronchi, and so on.

[0059] 6. An auxiliary navigation module, configured to assist in completing the bronchoscope operation according to the fused bronchial branching real image.

[0060] In the embodiment, the auxiliary navigation module is specifically configured to: The bronchial bifurcation real image with the bronchial name after fusion is directly displayed on the endoscope host display screen, and the position of the bronchial bifurcation real image is displayed on four sides of the display screen, and operation instructions are generated according to the planned bronchoscopic route and position, and the operation instructions are voice broadcast and / or real-time display, thereby assisting the completion of bronchoscopy operation; When the bronchoscope deviates from the planned bronchoscopic route, or reaches the bronchial bifurcation intersection, the voice broadcast and / or real-time display of the reminder instructions on how to enter the planned bronchus is generated.

[0061] The laser detection sensor installed on the bronchoscope head is used to detect the distance between the front, back, left and right directions of the bronchoscope head and the tracheal wall. If the distance of any direction exceeds the preset distance threshold, a reminder instruction is issued. The force sensor integrated at the handle of the bronchoscope is used to detect the resistance when the bronchoscope is pushed, and if the resistance exceeds the preset resistance threshold, a reminder instruction is issued.

[0062] In this embodiment, the bronchoscope head is equipped with a laser detection sensor, and the handle is equipped with a high-sensitivity force sensor with a sampling frequency of ≥100Hz. For the laser detection sensor, it is used to predict the left and right branch directions at the tracheal bifurcation intersection, and combined with the navigation information of the planned bronchoscopic route, the operation instructions for turning suggestions can be provided. If the laser detection sensor detects that the left branch distance decreases sharply and the right branch distance increases, but the planned bronchoscopic route is in the right branch, the actual bronchoscopic route is inconsistent with the planned bronchoscopic route, and the direction correction prompt is voice broadcast and / or real-time display to guide the bronchoscope into the planned bronchus. At the same time, the default centering threshold of the laser detection sensor on the tracheal / bronchial wall is that the deviation of each direction distance from the tracheal / bronchial wall is ≤20% of the current tracheal / bronchial diameter. When the laser detection sensor detects that any single-side distance of the front, back, left and right directions of the bronchoscope head from the tracheal wall is <10% or >30% of the current tracheal / bronchial diameter, it is determined that there is a deviation, and a reminder instruction can be voice broadcast and / or real-time display, or a vibrating handle. For the handle, when the pushing resistance is large, it is also considered that there is a path deviation or a distance problem, and a reminder instruction is generated.

[0063] In the present embodiment, the bronchial naming is presented in the form of text and voice broadcast, the text part of the bronchial naming is directly displayed on the endoscope host display screen, and the position of the picture is displayed on the four sides of the display screen. The system forms a visible mirror insertion route on the virtual bronchial tree, which is a one-way curve formed by the connection of the centers of the bronchial lumens at each level and each segment, the curve has a starting point and an ending point, and the distance of the lens advancing or retreating on the route can be calculated, so that the voice broadcast how to advance, retreat, turn, etc. when the doctor operates, and when the doctor deviates from the planned route or reaches the bronchial branching intersection, the operator is reminded to enter the bronchus with a planned naming identifier, and finally reaches the target bronchus.

[0064] Therefore, the present embodiment has the following advantages: (1) The bronchoscope operation is quickly and accurately completed by the semi-automatic auxiliary guided personalized scheme, the operation safety is improved, and the health risk caused by misoperation is reduced.

[0065] (2) For doctors, it helps novice doctors quickly master endoscopic operation skills and shortens the learning curve. At the same time, the quick and accurate bronchoscope operation can improve the operation experience and self-confidence of the endoscopic doctor. In addition, the present embodiment can also enhance the training of the endoscopic operator's anatomical knowledge of the bronchus, the three-dimensional spatial sense, and the distance sense of the target lesion biopsy, thereby having endoscopic training teaching value.

[0066] (3) The personalized guiding scheme can be customized according to the characteristic bronchial anatomical structure of each patient, the diversity of real-world operation can be displayed, and the database of bronchial anatomy can be further improved.

[0067] Embodiment Two Please refer to Figure 3 , the bronchoscope operation method based on the bronchial tree naming local classifier, comprising the steps of: S1, constructing a three-dimensional bronchial tree model; S2, naming each level of the bronchial tree, obtaining a three-dimensional bronchial tree model with bronchial naming; S3, according to the anatomical law of the bronchial tree and the bronchial branching, the three-dimensional bronchial tree model with bronchial naming is divided into multiple local classifiers, each local classifier is analyzed and imaged respectively, to generate multiple bronchial branching virtual images with bronchial branching data; S4, obtaining a target bronchus, forming a planned mirror insertion route on the three-dimensional bronchial tree model with bronchial naming, the route starting from the glottis and ending at the target bronchus; S5, in the bronchoscope operation according to the planned bronchoscope route, real-time acquisition of the bronchial branch real image of each level bronchus on the actual bronchoscope route, and the bronchial branch real image is overlapped and fused with the most matched bronchial branch virtual image itself; S6, according to the fused bronchial branch real image to assist in completing the bronchoscope operation.

[0068] Among them, the specific implementation and specific effect of each step refer to the description of one corresponding module of the embodiment.

[0069] Since the system / device described in the above embodiment of the application is the system / device used for implementing the method of the above embodiment of the application, the specific structure and modification of the system / device can be understood by those skilled in the art based on the method described in the above embodiment of the application, and thus will not be described here. Any system / device used in the method of the above embodiment of the application belongs to the scope of protection of the application.

[0070] Those skilled in the art should understand that the embodiments of the application can be provided as a method, device or computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0071] The application is described with reference to flowcharts and / or block diagrams of methods, devices (apparatuses) and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions.

[0072] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps not listed in the claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the claims, the word 'comprising' does not exclude the presence of other elements or steps than those listed in the claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The word 'first','second', 'third' etc. does not imply any order. The use of heading of this application does not limit the scope of the application.

[0073] Moreover, it should be noted that in the description of the application, the terms "one embodiment", "some embodiments", "an embodiment", "an example", "a specific example" or "some examples" are intended to mean that a particular feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. The illustrative appearances of the above terms in various places in the description are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples. Moreover, the terms "first", "second" and the like are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0074] Although preferred embodiments of the application have been described herein, those skilled in the art will readily devise many other variations of these embodiments that will fall within the scope of the present application. Accordingly, the claims should be interpreted as including all embodiments falling within the scope of the application and all such modifications and variations as would be apparent to one skilled in the art.

[0075] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A bronchoscopic navigation operating system based on bronchial tree nomenclature, characterized by, The method comprises the following steps: a three-dimensional reconstruction module for constructing a three-dimensional bronchial tree model; a bronchial naming labeling module for naming each level of bronchial bifurcation of the bronchial tree to obtain a three-dimensional bronchial tree model with bronchial naming; a virtual image generation module for dividing the three-dimensional bronchial tree model with bronchial naming into multiple local classifiers according to the anatomical rules of the bronchial tree and bronchial bifurcation, analyzing and taking pictures of each local classifier respectively to generate multiple bronchial bifurcation virtual images with bronchial bifurcation opening data; a route generation module for obtaining a target bronchus and forming a planned bronchoscope route on the three-dimensional bronchial tree model with bronchial naming, the planned bronchoscope route starting from the glottis and ending at the target bronchus; an image fusion module for obtaining real bronchial bifurcation images of each level of bronchus on the actual bronchoscope route in real time when the bronchoscope operation is performed according to the planned bronchoscope route, and superimposing and fusing the real bronchial bifurcation images with the bronchial bifurcation virtual images most matched with the real bronchial bifurcation images; an auxiliary navigation module for assisting the completion of the bronchoscope operation according to the fused real bronchial bifurcation images.

2. The bronchoscopy navigation operating system based on bronchial tree naming of claim 1, wherein, The local classifiers are eight in number, corresponding to eight bronchial bifurcations with anatomical recognition in the bronchial tree, and are classified into a first group including right upper lobe bronchus bifurcation and left inherent upper lobe bronchus bifurcation, a second group including right middle lobe bronchus bifurcation and left lingual lobe bronchus bifurcation, a third group including right lower lobe dorsal segment bronchus bifurcation and left lower lobe dorsal segment bronchus bifurcation, and a fourth group including right lower lobe basal segment bronchus bifurcation and left lower lobe basal segment bronchus bifurcation.

3. The bronchoscopy navigation operating system based on bronchial tree naming of claim 2, wherein, The bronchial naming labeling module specifically names each level of bronchus according to different character rules in different groups according to a preset order when naming each level of bronchial bifurcation of the bronchial tree; The planned bronchoscope route is also provided with route characters, which are generated in sequence according to the bronchial naming of each bronchus passed through by the planned bronchoscope route.

4. The bronchoscopy navigation operating system based on bronchial tree naming of claim 1, wherein, The bronchus on the local classifier is a cavity type model with an internal hollow cavity reconstructed on the three-dimensional bronchial tree model, and the virtual image generation module includes the following steps when analyzing and taking pictures of each local classifier respectively to generate multiple bronchial bifurcation virtual images with bronchial bifurcation opening data: perspective view of the cavity type model in each local classifier to obtain bronchial bifurcation opening data of each level of bronchial bifurcation opening, and taking pictures to save as bronchial bifurcation virtual images with bronchial bifurcation opening data, the bronchial bifurcation opening data including the number, size and relative position of each level of bronchial opening.

5. The bronchoscope navigation operating system based on bronchial tree naming of claim 1, wherein, The bronchial naming labeling module is also used for naming the three-dimensional external space of the left bronchial tree and the right bronchial tree respectively to obtain a three-dimensional bronchial tree model with bronchial naming and spatial naming; The bronchial bifurcation virtual image is also labeled with four spatial names of up, down, left and right.

6. The bronchoscopic navigation system based on the naming of bronchial tree according to any one of claims 1 to 5, characterized in that, The auxiliary navigation module is specifically used for: The branched bronchus real image after fusion is directly displayed on the endoscope host display screen, and the position of the branched bronchus real image is displayed on four sides of the display screen, and operation instructions are generated according to the planned bronchoscope route and the position, and the operation instructions are played by voice and / or displayed in real time, thereby assisting the bronchoscope operation; When the bronchoscope deviates from the planned bronchoscope route or reaches a branched bronchus intersection, a prompt instruction is played by voice and / or displayed in real time on how to enter the planned bronchus.

7. The bronchoscope navigation operating system based on bronchial tree naming of claim 6, wherein, The auxiliary navigation module is further configured to: Detect the distance between the front, back, left and right of the bronchoscope head and the tracheal wall through the laser detection sensor installed on the bronchoscope head, and if the distance of any direction exceeds the preset distance threshold, a prompt instruction is issued; Detect the resistance when the bronchoscope is pushed through the force sensor integrated at the handle of the bronchoscope, and if the resistance exceeds the preset resistance threshold, a prompt instruction is issued.

8. The bronchoscope navigation system based on the naming of bronchial tree according to any one of claims 1 to 5, characterized in that, The three-dimensional reconstruction module is specifically configured to: Obtain a chest thin layer CT, and perform three-dimensional reconstruction on the bronchus of the cavity according to the chest thin layer CT to form a three-dimensional bronchial tree model.

9. The bronchoscopic navigation operating system based on the naming of bronchial tree according to any one of claims 1 to 5, characterized in that, The image fusion module is configured to overlap and fuse the branched bronchus real image with the branched bronchus virtual image that is most matched with the branched bronchus real image, and the overlapping and fusion is specifically configured as follows: Draw a circle and label the center of the circle for each bronchial lumen in the branched bronchus real image and each branched bronchus virtual image, connect the centers of the circles in sequence to form a center pattern, and then overlap the two pictures, and according to the degree of coincidence of the circles and the center pattern of each bronchial lumen, obtain the branched bronchus virtual image that is most matched with the branched bronchus real image, and overlap and fuse the branched bronchus real image with the branched bronchus virtual image that is most matched with the branched bronchus real image.

10. A bronchoscopy method based on a local classifier of bronchial tree nomenclature, characterized in that, The method comprises the following steps: Constructing a three-dimensional bronchial tree model; Naming each level of branched bronchus of the bronchial tree to obtain a three-dimensional bronchial tree model with bronchial naming; According to the anatomical rules of the bronchial tree and the branched bronchus, the three-dimensional bronchial tree model with bronchial naming is divided into multiple local classifiers, each local classifier is analyzed and imaged respectively to generate multiple branched bronchus virtual images with branched bronchus opening data; Obtaining a target bronchus, and forming a planned bronchoscope route on the three-dimensional bronchial tree model with bronchial naming, the planned bronchoscope route starting from the glottis and ending at the target bronchus; When the bronchoscope is operated according to the planned bronchoscope route, the branched bronchus real image of each level of bronchus on the actual bronchoscope route is obtained in real time, and the branched bronchus real image is overlapped and fused with the branched bronchus virtual image that is most matched with the branched bronchus real image; The branched bronchus real image after fusion is used to assist the bronchoscope operation.

Images