Registration system for the pulmonary airways and electronic device

Abstract

Embodiments of the present application provide a registration system for a lung airway and an electronic device. The registration method executed by the processing device in the registration system includes: determining a movement path of a positioning sensor in a three-dimensional model of a bronchial tree based on a first set of movement path points of the positioning sensor in the lung airway, the three-dimensional model of the bronchial tree being reconstructed based on a medical image of the lung airway; determining an airway path of the three-dimensional model of the bronchial tree based on centerline information of the three-dimensional model of the bronchial tree; extracting a first set of key points of the movement path, the first set of key points including a movement path bifurcation point and a movement path end point of the movement path; determining a second set of key points corresponding to the first set of key points in the airway path; and registering the three-dimensional model of the bronchial tree and the lung airway based on the first set of key points and the second set of key points. The present scheme can effectively improve the registration accuracy between the three-dimensional model of the bronchial tree and the actual lung airway.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a registration system and electronic device for the trachea of ​​the lungs. Background Technology

[0002] With the development of computer technology and medical imaging technology, surgical navigation technology, with its advantages of precision, flexibility, and minimally invasiveness, is increasingly widely used in disease diagnosis and treatment. For example, electromagnetic navigation bronchoscopy (ENB) is commonly used to examine peripheral lung diseases. In intraoperative diagnosis and treatment of the patient's lungs using ENB, an essential step is registration. The purpose of registration is to match the actual physical space of the patient's lungs and trachea during surgery with the three-dimensional model of the tracheal tree reconstructed preoperatively. This ensures the precise positioning of surgical instruments and other equipment within the tracheal tree model during real-time surgical navigation, providing a prerequisite for the surgery.

[0003] The aforementioned 3D tracheal tree model was obtained through 3D reconstruction using medical images of the patient's lungs acquired preoperatively. Currently, during surgery, various factors such as the patient's breathing and posture often cause deformation of the tracheal shape, resulting in a discrepancy between the tracheal shape in the preoperative medical images and the actual tracheal shape. This leads to a mismatch between the intraoperative 3D tracheal tree model and the patient's actual trachea, particularly at the terminal bronchi, which can significantly reduce navigation accuracy. Summary of the Invention

[0004] In view of the above problems, this application provides a method, system and electronic device for registering lung trachea to solve the above problems or at least partially solve the above problems.

[0005] Therefore, in one embodiment of this application, a registration method for the trachea of ​​the lungs is provided. The method includes:

[0006] Acquire the first set of movement path points of the positioning sensor within the trachea of ​​the lungs;

[0007] Based on the first set of movement path points, the movement path of the positioning sensor in the 3D model of the tracheal tree is determined; wherein, the 3D model of the tracheal tree is reconstructed based on medical imaging images of the lungs and trachea;

[0008] Obtain the centerline information of the tracheal tree 3D model;

[0009] Based on the centerline information, the airway path of the tracheal tree 3D model is determined;

[0010] Extract a first set of key points from the movement path, the first set of key points including the movement path branching point and the movement path endpoint;

[0011] In the airway path, determine a second set of key points corresponding to the first set of key points;

[0012] Based on the first key point set and the second key point set, the 3D model of the tracheal tree is registered with the trachea of ​​the lungs.

[0013] In another embodiment of this application, a registration system for the trachea of ​​the lungs is also provided. The system includes:

[0014] A magnetic field generator is used to generate a positioning magnetic field; the trachea of ​​the lungs is placed in the positioning magnetic field;

[0015] A positioning sensor that generates a positioning signal by sensing the positioning magnetic field when moving within the trachea of ​​the lungs, and sends the positioning signal to a processing device so that the processing device can determine the movement path point of the positioning sensor within the trachea of ​​the lungs based on the positioning signal;

[0016] A processing device for performing the steps in the registration method for lung trachea provided in the above embodiment.

[0017] In one embodiment of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory is used to store a computer program; and the processor, coupled to the memory, is used to execute the computer program stored in the memory to implement the steps in the lung-trachea registration method provided in the above embodiment of this application.

[0018] The technical solutions provided in the embodiments of this application, based on the reconstruction of the corresponding tracheal tree three-dimensional model from medical imaging images of the lungs and trachea, determine the airway path of the tracheal tree three-dimensional model according to the centerline information of the obtained tracheal tree three-dimensional model; and determine the movement path of the positioning sensor in the tracheal tree three-dimensional model according to the first movement path point set of the positioning sensor in the lungs and trachea; then, extract the first key point set of the movement path (including the movement path bifurcation point and the movement path endpoint), and determine the second key point set corresponding to the first key point set in the airway path, and then register the tracheal tree three-dimensional model with the lungs and trachea according to the first key point set and the second key point set. The aforementioned path bifurcation points are generally the path points where the positioning sensor intersects with two different tracheas during its movement within the trachea of ​​the lungs. In other words, the positioning sensor's location within the trachea, as indicated by the path bifurcation point, is typically at one end of the actual trachea, such as the distal end. Therefore, this solution utilizes the path bifurcation points and the corresponding points on the airway path of the tracheal tree 3D model to achieve registration between the tracheal tree 3D model and the actual trachea of ​​the lungs, effectively improving the registration accuracy at the distal end of the trachea. This solution also incorporates the path endpoint and the corresponding point on the airway path during registration. The path endpoint is the point at the end of the corresponding path in the movement path, representing the positioning sensor's location within the trachea of ​​the lungs. If it is at the distal end of the trachea, this further improves the registration accuracy at the distal end; if it is within the trachea (such as in the middle), it helps to improve the overall registration accuracy between the tracheal tree 3D model and the actual trachea of ​​the lungs to a certain extent. Furthermore, the number of bifurcation points and endpoints of the aforementioned movement paths is often limited and small. Therefore, this solution can effectively reduce the amount of registration data computation while ensuring registration accuracy. In summary, the technical solution provided by the embodiments of this application can effectively improve the registration accuracy between the 3D model of the tracheal tree and the actual lung trachea, especially the registration accuracy of the tracheal terminal, which can provide a guarantee for subsequent precise guidance of surgery. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figures 1a to 1c This is a schematic diagram of the structure of a medical system for lung diagnosis and treatment according to an embodiment of the application;

[0021] Figure 2 A schematic flowchart of a registration method for the lungs and trachea provided in an embodiment of this application;

[0022] Figure 3 A three-dimensional model of the tracheal tree with centerline information, centerline bifurcation points, etc., provided in an embodiment of this application, and a frontal plan view of the hierarchical division of the three-dimensional model of the tracheal tree;

[0023] Figure 4a A frontal plan view of a 3D model of a tracheal tree with a second set of movement path points provided in an embodiment of this application;

[0024] Figure 4b A frontal plan view of a 3D model of a tracheal tree with a movement path provided in an embodiment of this application;

[0025] Figure 5 A schematic diagram of a registration system for the lungs and trachea provided in an embodiment of this application;

[0026] Figure 6 A schematic diagram of the structure of a registration device for the lungs and trachea provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0028] Figure 8 A schematic diagram of the structure of a computer program product provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the surgical registration interface provided in the embodiments of this application. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0031] In some processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. These operations may be executed out of order or in parallel. Operation numbers such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the terms "first," "second," etc., used herein are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types. The term "or / and" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A or / and B means that A can exist alone, A and B can exist simultaneously, or B can exist alone. The character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system including said element. Furthermore, the following embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0032] The following embodiments of this application involve positioning sensors, such as positioning sensors mounted on positioning catheters. These sensors employ electromagnetic navigation principles, which differ from ordinary electromagnetic navigation principles. Ordinary electromagnetic navigation relies on an external magnetic field to attract or repel a permanent magnet within the catheter, influencing the direction of a medical device entering the body. In the embodiments of this application, the positioning sensor operates primarily by outputting a current to an external control system in response to the magnetic field of its surrounding space, allowing the control system to locate the corresponding catheter. Specifically, the positioning sensor and the control system are connected via wired or wireless means. The control system includes a magnetic field generator to generate a magnetic field within a defined positioning space. The positioning sensor itself is non-magnetic; a coil within the sensor senses the magnetic field generated by the magnetic field generator. The magnetic field generator produces a changing magnetic field within the positioning space, ensuring that the magnetic field characteristics of each electrode within the positioning space are unique. The coil in the positioning sensor generates a current in the changing magnetic field, which, after being collected and used to generate a signal (i.e., the positioning signal described in the context), is transmitted to the control system. The control system analyzes the signal to determine the precise position and orientation of the corresponding catheter.

[0033] To facilitate understanding, before introducing the technical solutions provided in the embodiments of this application, a medical system for lung surgery navigation will be first introduced. Specifically,

[0034] Figure 1a and Figure 1b This is a three-dimensional structural diagram of a medical system provided as an embodiment of this application. See also... Figure 1a and Figure 1b As shown, the medical system includes: an operating console 10, a magnetic navigation device 20, a medical imaging device 30, and a positioning medical tool 40. Among them,

[0035] The magnetic navigation device 20 is used to navigate the positioning medical tool 40 to the corresponding target point (or target area) on the medical path according to the medical path.

[0036] The aforementioned medical pathway can be planned by a processing module with pathway planning capabilities. Specifically, the processing module can reconstruct a corresponding three-dimensional model based on medical imaging data (such as CT, MRI, etc.) of the patient's whole body or a specific body part, and perform pathway planning based on this planned three-dimensional model to obtain the corresponding medical pathway. For example, a three-dimensional model of the tracheal tree, including the main trachea and bronchi, can be reconstructed based on preoperative medical imaging data of the patient's lungs, and a navigation path (or planned path) from the main carina through the natural trachea of ​​the lungs to the target point can be planned based on this tracheal tree model. This navigation path is the medical pathway. The main carina refers to the bifurcation of the main trachea, which branches into the left and right bronchi.

[0037] In practical implementation, the aforementioned processing module can be installed on the magnetic navigation device 20, or it can be installed on, for example, Figure 1c The above-mentioned processing module is on the main control carriage 50 used by the doctor, as shown in the figure. This processing module can be application software, which has functions such as reconstructing 3D models, path planning, and registration. Specifically, it can be installed in control devices such as the main control carriage 50 or the magnetic navigation device 20, for example... Figure 1c The control device 511 shown is (e.g., a computer device). However, to reduce the risk of radiation exposure for doctors and cross-infection between doctors and patients, in this embodiment, the control device 511 is preferably separated from the electromagnetic navigation device 20 and mounted on the main control carriage 50, so that doctors can operate the control device outside the operating room. The main control carriage 50 can communicate with the magnetic navigation device 20 and can be regarded as an extension of the display and control functions of the magnetic navigation device 20. For example, through the display screen included in the control device 511 on the main control carriage 50, specifically the display screen showing a human-computer interaction interface, the reconstructed three-dimensional model (e.g., a tracheal tree three-dimensional model), medical path, surgical progress (e.g., registration progress, real-time navigation progress), endoscopic images, etc.; in addition, doctors can also trigger the registration operation to start by clicking the registration control displayed on the human-computer interaction interface; or, control the medical imaging device 30 to perform scanning through the human-computer interaction interface, etc.

[0038] The aforementioned medical imaging device 30 is used to acquire medical images of a patient. For example, it can acquire medical images of a patient's lungs.

[0039] In specific implementation, the aforementioned medical imaging device 30 can be, but is not limited to, CT scanners, X-ray machines, etc. Where conditions permit, it can refer to any medical imaging device used to image a patient's local or whole body, and its imaging type includes, but is not limited to, two-dimensional images.

[0040] In one example, the aforementioned medical imaging device 30 includes a C-shaped arm 311 movable relative to the operating table 10. The two ends of the C-shaped arm 311 are respectively used to house a radiation module (for emitting corresponding medical radiation, such as X-rays) and an imaging module (not shown in the figure, but may be located at the end of the C-shaped arm 311 above the operating table), with the C-shaped opening facing the operating table. The movable C-shaped arm 311 relative to the operating table means that the opening of the C-shaped arm can move along the length or width of the operating table 10, or it can mean that the opening of the C-shaped arm can rotate around the operating table 10, so that the medical imaging device 30 can capture or scan the entire operating table and its various directions, capturing medical images (i.e., medical images) at appropriate angles as needed, thus improving the user experience of the medical system.

[0041] It should be noted that the aforementioned medical imaging device 30 can be connected to the control equipment on the main control carriage 50 or the magnetic navigation device 20, so that the processing module (application software) on the control equipment can receive and process the medical images obtained by the medical imaging device 30.

[0042] The aforementioned operating table 10 can be, but is not limited to, an operating table in an operating room, or a patient treatment bed, etc. This operating table 10 allows X-ray transmission from the medical imaging device 30 without affecting the imaging effect of the medical imaging device 30. Furthermore, the aforementioned medical imaging device 30 and magnetic navigation device 20 can be respectively installed beside the operating table 10.

[0043] The medical positioning tool 40 mentioned above refers to a medical tool with positioning function, such as a biopsy or treatment tool. Specifically, it can be a lead or catheter used for positioning, an ablation catheter, etc.

[0044] The positioning medical tool 40 is connected to the magnetic navigation device 20, or to a portion thereof. For example, the magnetic navigation device 20 has a shape and structure as shown in the image. Figure 1b In the configuration shown, the magnetic navigation device 20 includes a delivery mechanism 201, which can be connected to a positioning medical tool 40. The delivery mechanism 201 guides the positioning medical tool 40 into the patient's body, such as into the airway (or trachea) of the lungs, according to a medical path, and reaches the corresponding target point (or target area) under the control of the magnetic navigation device 20. Figure 1b An example is given of delivering the positioning medical tool 40 into the patient's body via a delivery mechanism. In other embodiments, the positioning medical tool 40 can also be delivered into the patient's body by a physician's operation via a corresponding propulsion device (such as the lifting mechanism 2012 described below). The physician uses the display device corresponding to the magnetic navigation device (such as...) Figure 1cThe display device 511 shown in the figure is manually controlled to propel the positioning medical tool 40 along the medical path, thereby reaching the corresponding target point (or target area).

[0045] It should be noted that the aforementioned conveying mechanism 201 may include a robotic arm 2011 and a lifting mechanism 2012. The lifting mechanism 2012 is connected to one end of the positioning medical tool 40. By manually pushing or pulling the lifting mechanism 2012, the positioning medical tool 40 can be moved in and out of the human trachea. In addition, the electromagnetic navigation device 20 can, in addition to providing... Figure 1b Besides the shapes and structures shown, other shapes and structures are also possible, such as... Figure 1c The shape and structure shown in the figure are not specifically limited to the shape and structure of the electromagnetic navigation device 20 in this embodiment of the application.

[0046] Furthermore, the aforementioned magnetic navigation device 20 may also include a magnetic field generator 202 disposed within the aforementioned operating table 10, and a positioning sensor 203 disposed within the positioning medical tool 40. The magnetic field generator 202 is used to generate a positioning magnetic field to locate the positioning sensor 203.

[0047] In practice, the aforementioned magnetic field generator 202 is installed inside the operating table 10, which can emit a positioning magnetic field towards the bed surface. When the patient lies on the operating table 10, the corresponding body part of the patient (such as the lungs) is located in this positioning magnetic field. Since the patient is generally under general anesthesia on the operating table 10, the relative position of the corresponding body part of the patient (such as the lungs) in the positioning magnetic field is fixed. Because the positioning magnetic field has its own coordinate system, called the magnetic field coordinate system, the relative position of the corresponding body part of the patient (such as the lungs) in the magnetic field coordinate system is fixed. The position and orientation of the positioning medical tool 40 in the patient's body can be represented by the position coordinates of the head of the positioning medical tool 40 in the magnetic field coordinate system. Specifically, the positioning magnetic field can locate the positioning sensor 203 inside the positioning medical tool 40. Since the positioning sensor 203 is located at the head end of the medical tool 40, the position and orientation of the positioning medical tool 40 in the patient's body can be obtained by locating the positioning sensor 203 and obtaining its position coordinates in the magnetic field coordinate system.

[0048] The following is a detailed description of the registration method for the lungs and trachea provided in the embodiments of this application.

[0049] Figure 2 This diagram illustrates a flowchart of a lung-tracheal registration method according to an embodiment of this application. The method is executed by a corresponding processing device, such as... Figures 1a to 1c The magnetic navigation device 20 shown in the figure or Figure 1cThe control equipment 511 (such as computer equipment) on the main control carriage 50 shown is not specifically limited here. See also Figure 2 As shown, the registration method for the lungs and trachea includes the following steps:

[0050] 101. Obtain the first set of movement path points of the positioning sensor within the trachea of ​​the lung;

[0051] 102. Based on the first set of movement path points, determine the movement path of the positioning sensor in the 3D model of the tracheal tree, wherein the 3D model of the tracheal tree is reconstructed based on medical imaging images of the lungs and trachea.

[0052] 103. Obtain the centerline information of the tracheal tree three-dimensional model;

[0053] 104. Based on the centerline information, determine the airway path of the tracheal tree 3D model;

[0054] 105. Extract the first key point set of the movement path, the first key point set including the movement path branching point and the movement path endpoint;

[0055] 106. Determine a second set of key points in the airway path that corresponds to the first set of key points;

[0056] 107. Based on the first key point set and the second key point set, register the tracheal tree 3D model with the lung trachea.

[0057] In the above 101-104, the trachea of ​​the lungs refers to the actual tracheal tree of the lungs. The corresponding three-dimensional model of the tracheal tree can be obtained by using three-dimensional reconstruction technology (such as three-dimensional reconstruction software) to process multiple medical images (such as two-dimensional tomographic images (CT images) or magnetic resonance images (MRI images) collected from different angles of the patient's lungs before surgery.

[0058] For example, before surgery, multiple two-dimensional CT images of the patient's lungs can be obtained by scanning the patient's lungs with a CT tomography scan. Then, these multiple two-dimensional CT images are imported into the processing module (which can be an application software with functions such as reconstructing three-dimensional models, path planning, and registration) described in other embodiments of the above application to perform image recognition and segmentation of tissues such as the lungs and trachea, obtain three-dimensional modeling parameters of the lungs and trachea, and then construct a three-dimensional model of the tracheal tree of the patient's lungs and trachea based on the three-dimensional modeling parameters.

[0059] Based on the above example, this embodiment may further include the following implementation steps for reconstructing a corresponding 3D model of the tracheal tree for the lungs and trachea before performing step 101:

[0060] 100a. Acquire multiple medical imaging images of the lungs and trachea;

[0061] 100b. Recognize the plurality of medical images to obtain the three-dimensional modeling parameters of the lungs and trachea;

[0062] 100c. Based on the three-dimensional modeling parameters, construct the three-dimensional model of the tracheal tree.

[0063] After obtaining the 3D model of the tracheal tree, when performing steps 103-104 above, the 3D model of the tracheal tree can be skeletonized to extract the centerline information of the 3D model of the tracheal tree, providing data support for subsequent hierarchical division of the 3D model of the tracheal tree and determination of airway paths.

[0064] In practice, centerline information of the tracheal tree 3D model can be extracted using, but is not limited to, corresponding thinning algorithms, such as topological thinning algorithms. Thinning algorithms primarily extract the centerline of the target object by repeatedly eroding the surface pixels until only the skeleton remains. For specific implementation details on using thinning algorithms to extract centerline information of the tracheal tree 3D model, please refer to existing related solutions.

[0065] exist Figure 3 In the frontal view of the tracheal tree 3D model 100 shown in the figure, the relatively thin solid line in the tracheal tree 3D model 100 represents the center line information of the tracheal tree 3D model.

[0066] After extracting the centerline information of the tracheal tree 3D model, the path formed by the centerline information is used as the airway path of the tracheal tree 3D model.

[0067] Furthermore, since the human lung trachea is tree-like in shape, based on its anatomical structure, the lung trachea is usually divided into different levels from top to bottom according to the bifurcation hierarchy. For example, the main trachea located in the main trunk (level 1, a tube that connects to the larynx at one end and to the bronchus at the other end, serving as an air passage) enters the lung through the hilum and bifurcates into two main bronchi (also called lobar main bronchi, level 2): ​​the right main bronchus and the left main bronchus. Further, the two main bronchi will further bifurcate, for example, the right main bronchus further bifurcates into the right upper lobe bronchus and the right lower lobe bronchus, and so on. Based on this, for the aforementioned 3D tracheal tree model, this embodiment can further analyze the extracted centerline information of the 3D tracheal tree model to determine the centerline bifurcation points. Then, based on the centerline bifurcation points and combined with the actual anatomical structure of the lungs and trachea described above, the 3D tracheal tree model can be divided into multi-level tracheal models to provide support for subsequent processing steps. That is, the method provided in this embodiment may also include the following steps:

[0068] S11. Determine the centerline bifurcation point in the centerline information;

[0069] S12. Based on the bifurcation point of the center line, the three-dimensional model of the tracheal tree is divided into levels from top to bottom to obtain a multi-level tracheal model.

[0070] In the above, the centerline bifurcation point refers to the intersection of at least two centerline segments. For information on centerline bifurcation points in centerline information, please refer to [link to centerline information]. Figure 3 The black circle in the middle The multiple bifurcation points shown are: bifurcation point b0, bifurcation point b1, bifurcation point b2, bifurcation point b3, bifurcation point b4, bifurcation point b5, and bifurcation point b6. Since in this embodiment, the path formed by the centerline information is used as the airway path in the 3D model of the tracheal tree, the aforementioned centerline bifurcation points (i.e., bifurcation points b0~b6) are also referred to as airway path bifurcation points in the subsequent detailed description of the key points for determining the airway path.

[0071] Accordingly, see [link to relevant documentation] Figure 3 Based on the aforementioned centerline bifurcation point, the multi-level tracheal model obtained by hierarchically dividing the tracheal tree 3D model from top to bottom can include:

[0072] A primary tracheal model 1 (main tracheal model);

[0073] The two secondary tracheal models 2 that are bifurcated and connected to the primary tracheal model 1 are: right main bronchus model 21 and left main bronchus model 22.

[0074] Four tertiary tracheal models 3 (lobar bronchus models); wherein, two of the four tertiary tracheal models are connected to one of the two secondary tracheal models 2, and the other two tertiary tracheal models are connected to the other of the two secondary tracheal models 2. Specifically, the four tertiary tracheal models 3 include: a right upper lobe bronchus model 31 and a right lower lobe bronchus model 32, which are bifurcated and connected to the right main bronchus model 21; and a left upper lobe bronchus model 33 and a left lower lobe bronchus model 34, which are bifurcated and connected to the left main bronchus model 22.

[0075] Eight fourth-level tracheal models 4 (segmental bronchial models). Specifically, the eight fourth-level tracheal models 4 include: a first segmental bronchial model 41 and a second segmental bronchial model 42 connected to the right upper lobe bronchial model 31; a third segmental bronchial model 43 and a fourth segmental bronchial model 44 connected to the right lower lobe bronchial model 32; and a fifth segmental bronchial model 45 and a sixth segmental bronchial model 46 connected to the left upper lobe bronchial model 33; and a seventh segmental bronchial model 47 and an eighth segmental bronchial model 48 connected to the left lower lobe bronchial model 34.

[0076] Accordingly, the centerline information of the tracheal tree 3D model mentioned above includes the centerlines of each level of the tracheal model.

[0077] When performing step 101 above, the first set of movement path points can be from, for example... Figure 1a or Figure 1b This information is obtained from the magnetic navigation device 20 shown in the diagram. Specifically, it can be obtained from, for example, […]. Figure 1a or Figure 1b The magnetic navigation device 20 shown in the figure determines the first set of movement path points (i.e., the actual set of movement path points) generated by the movement of the positioning sensor within the trachea of ​​the lungs based on the positioning signal transmitted in real time by the positioning sensor as it moves (wanders) within the trachea of ​​the lungs, and sends the first set of movement path points to the execution subject of this application embodiment; wherein, the positioning signal is generated by the positioning sensor based on the current generated by its own internal coil in the changing magnetic field. The changing magnetic field is generated by the magnetic field generator 202 set in the operating table 10 under the control of the magnetic navigation device 20. Since the patient's lungs are under the changing magnetic field when the patient is lying supine in the operating table 10, the positioning sensor can sense the changing magnetic field and generate a corresponding current when it moves within the trachea of ​​the patient's lungs.

[0078] The first set of mobile path points contains mobile path points that represent the corresponding positioning positions of the positioning sensors within the lungs.

[0079] When performing step 102 above, the coordinates of each movement path point in the first movement path point set can be transformed according to the current spatial coordinate transformation relationship (the spatial coordinate transformation relationship between the actual physical space (magnetic space) of the lung trachea and the image space of the tracheal tree 3D model). This transforms the coordinates of each movement path point in the first movement path point set to map (or classify) the movement path points in the first movement path point set to the corresponding level of the tracheal tree 3D model, thus obtaining the second movement path point set of the positioning sensor in each level of the tracheal tree model. Then, a path search is performed on the second movement path point set in each level of the tracheal tree model to determine the movement path of the positioning sensor in the tracheal tree 3D model based on the path search results. The purpose of the above path search is to find the shortest movement path of the positioning sensor in each level of the tracheal tree model. By merging the shortest movement paths of the positioning sensor in each level of the tracheal tree model, the final movement path of the positioning sensor in the tracheal tree 3D model can be obtained. When performing the path search, the starting point can be, but is not limited to, the point in the tracheal tree 3D model corresponding to the main carina of the lung trachea.

[0080] Therefore, the tracheal tree three-dimensional model described in this embodiment includes a multi-level tracheal model, and in one feasible technical solution, the above-mentioned 102 "determining the movement path of the positioning sensor in the tracheal tree three-dimensional model based on the first movement path point set" can specifically include:

[0081] 1021. Based on the first set of movement path points, determine the second set of movement path points of the positioning sensor in each level of the tracheal model;

[0082] 1022. Determine the point in the three-dimensional model of the tracheal tree that corresponds to the main carina of the pulmonary trachea, and use it as the reference point;

[0083] 1023. Starting from the reference point, based on the second movement path point set of the positioning sensor in each level of the tracheal model, perform path search for each level of the tracheal model to obtain the shortest movement path of the positioning sensor in each level of the tracheal model.

[0084] 1024. The shortest movement paths of the positioning sensor in each level of the tracheal model are merged to obtain the movement path.

[0085] In the above 1021, considering that in the actual process of collecting data through the positioning sensor, the amount of data collected is often large and contains a lot of redundant data, that is, the above first set of movement path points contains a large number of redundant path points. Therefore, if a coordinate space transformation method is directly used to map each movement path point contained in the first set of movement path points to the corresponding level of the tracheal model in the tracheal tree 3D model, and then no processing is performed, directly obtaining the second set of movement path points of the positioning sensor in each level of the tracheal model based on the above mapping result, there will be problems such as excessive data and large data processing volume. To address this problem, this embodiment maps each movement path point contained in the first set of movement path points to the corresponding level of the tracheal model in the tracheal tree 3D model and then performs downsampling processing. Specifically, in one feasible technical solution, the above 1021 "determining the second set of movement path points of the positioning sensor in each level of the tracheal model based on the first set of movement path points" may include the following steps:

[0086] 10211. Determine the first voxel set in each level of the tracheal model that corresponds to the movement path points contained in the first movement path point set;

[0087] 10212. Determine the tracheal radius of each level of tracheal model;

[0088] 10213. Based on the tracheal radius, downsample the first voxel set in each level of the tracheal model;

[0089] 10214. The first voxel point set in each level of the tracheal model after downsampling is determined as the second movement path point set of the positioning sensor in each level of the tracheal model.

[0090] In specific implementation, step 10211 is achieved by performing spatial coordinate transformation on each movement path point in the first movement path point set, as described above, to map each movement path point in the first movement path point set to the corresponding level of the trachea model in the trachea tree 3D model, and then determining the implementation based on the mapping result. For steps 10212-10214, please refer to... Figure 3 or Figure 4a Taking a three-level tracheal model 3, such as the right upper lobe bronchus model 31, as an example, the tracheal radius R of the right upper lobe bronchus model 31 can be obtained by calculating the distance from each voxel point on the surface of the right upper lobe bronchus model 31 to its centerline and then averaging the calculated distances. Further, based on the tracheal radius R, the first voxel point set VG in the right upper lobe bronchus model 31 can be downsampled using, but is not limited to, the voxel grid method. During downsampling, the size of the grid is determined based on the tracheal radius R of the upper lobe bronchus model 31. Finally, the downsampled first voxel point set VG is the second movement path point set of the positioning sensor in the right upper lobe bronchus model 31.

[0091] It should be noted that the tracheal radius R of the right upper lobe bronchus model 31 can also be determined by other methods. For example, to reduce the amount of calculation, two voxel points on the surface of the right upper lobe bronchus model 31, corresponding to the two endpoints of the center line of the right upper lobe bronchus model 31, can be determined first. Then, using these two voxel points as the beginning and end points, a curve can be drawn on the surface of the right upper lobe bronchus model 31, and the distance from each voxel point on the curve to the center line of the right upper lobe bronchus model can be calculated. The average value of the calculated distances can then be used to obtain the tracheal radius R of the right upper lobe bronchus model 31.

[0092] For details on the implementation of data downsampling using the voxel grid method, please refer to existing related content; it will not be elaborated here. Using the voxel grid method for data downsampling can not only reduce the amount of data but also preserve the original morphological characteristics of the data.

[0093] Based on the above, if the third target-level tracheal model is one of the multi-level tracheal models, then: in a specific implementable solution, the above-mentioned "determining the tracheal radius of the third target-level tracheal model" may include:

[0094] From the centerline information, determine the second centerline of the third target level tracheal model;

[0095] Calculate the second distance from each second voxel point on the surface of the third target-level tracheal model to the second centerline;

[0096] The average of the second distances from each second voxel point to the second center line is determined as the tracheal radius of the third target-level tracheal model.

[0097] In another specific feasible solution, the aforementioned "determining the tracheal radius of the third target-level tracheal model" may include:

[0098] From the centerline information, determine the second centerline of the third target level tracheal model;

[0099] On the surface of the second target-level tracheal model, two points with the same ordinate as the two endpoints of the second centerline are determined respectively; in specific implementation, these two points can be understood as: drawing a perpendicular line from the two endpoints of the second centerline to the surface of the third target-level tracheal model, and obtaining two perpendicular points corresponding to the two endpoints of the second centerline;

[0100] Using one of the two points as the starting point and the other as the ending point, draw a curve on the surface of the third target-level tracheal model;

[0101] Calculate the third distance from each point on the curve to the second centerline;

[0102] The average of the third distances from each point on the curve to the second center line is determined as the tracheal radius of the third target-level tracheal model.

[0103] To facilitate understanding of step 1021 above, an example is given below. Specifically,

[0104] Assuming that a positioning sensor, under the doctor's control, enters a patient's actual trachea, the sensor, up to the current point in time, has traversed the main trachea, right main bronchus, right upper lobe bronchus, and a segment of bronchus connected to and slightly above the right upper lobe bronchus. After obtaining the first set of movement path points generated during the sensor's movement through the trachea, various processing steps, such as spatial coordinate transformation, are performed on the coordinates of each movement path point in this set. The final set of points corresponding to the first movement path point set is then determined in the tracheal tree 3D model. This set includes, for example, points such as... Figure 4a In the 3D model of the tracheal tree, all virtual points are shown, i.e., all virtual points indicated by virtual line L. Then: for example, the virtual line segment " " located within the right main bronchus model 21 (a secondary tracheal model) in virtual line L. The set of virtual points shown represents the second movement path point set of the positioning sensor in the primary tracheal model 1. Similarly, the second movement path point sets of the positioning sensor in other tracheal models can also be obtained. For example, the left main bronchus model 22 (a secondary tracheal model) does not contain any virtual points, indicating that the current positioning sensor has not yet traversed the tracheal region in the lung trachea corresponding to the left main bronchus model 22, and therefore has no corresponding second movement path point set in the left main bronchus model 22.

[0105] Based on the above example, the second set of movement path points of the positioning sensor in a certain level of the trachea model can be simply understood as: determined based on the movement path points generated by the positioning sensor during its movement within the lung trachea and the corresponding tracheal region in that certain level of the trachea model; wherein, the first set of movement path points includes the movement path points generated by the positioning sensor during its movement within the lung trachea and the corresponding tracheal region in that certain level of the trachea model. Furthermore, this application in... Figure 4a The frontal view of the 3D model of the tracheal tree shown in the figure illustrates the second set of movement path points of the positioning sensor in each level of the tracheal model by means of virtual points (i.e. points indicated by virtual lines L).

[0106] In the above 1022, since the main carina refers to the bifurcation of the main trachea in the lung trachea, the point corresponding to the main carina in the tracheal tree three-dimensional model can be determined based on the intersection of the center lines of the primary tracheal model and the two secondary tracheal models, and then used as the reference point.

[0107] For example, see Figure 4a or Figure 3 As shown, the intersection point b0 of the centerline of the primary tracheal model 1 (main tracheal model) and the centerlines of the two secondary tracheal models 2 (including the right main bronchus model 21 and the left main bronchus model 22) obtained from the centerline information of the tracheal tree 3D model, that is, the bifurcation point, can be determined as the reference point A.

[0108] In step 1023 above, starting from the reference point, it can be first determined which one or two levels of tracheal models should be targeted for path searching, and then path searching is performed based on the second movement path point set in the determined one or two levels of tracheal models. Specifically, in a specific implementation scheme, step 1023 above, "starting from the reference point, performing path searching for each level of tracheal model based on the second movement path point set of the positioning sensor in each level of tracheal model, to obtain the shortest movement path of the positioning sensor in each level of tracheal model," can be implemented using the following specific steps:

[0109] 10231. Starting from the aforementioned reference point, determine the second target-level tracheal model from which the current path search should be performed;

[0110] 10232. Determine the starting and ending positions of the second target-level tracheal model;

[0111] 10233. From the set of second movement path points of the positioning sensor within the second target-level tracheal model, find the movement path point with the shortest distance to the starting position and determine it as the search starting point; determine the movement path point with the shortest distance to the ending position and determine it as the search ending point.

[0112] 10234. Based on the search start point, the search end point, and the second movement path point set of the positioning sensor within the second target-level tracheal model, perform a path search to determine the shortest path from the search start point to the search end point according to the search results, which is taken as the shortest movement path of the positioning sensor within the second target-level tracheal model.

[0113] 10235. After the path search for the second target-level tracheal model is completed, determine whether there are still tracheal models in the multi-level tracheal models that need to be searched.

[0114] 10236. If there is a problem, return to the step described in 10231 above to determine the second target tracheal model for which the current path search should be performed.

[0115] For example, see Figure 4a Assuming we start from reference point A, and begin a path search in all directions based on the second set of movement path points in each level of the tracheal model using the positioning sensor, and the path search algorithm used is a coordinate-based path search algorithm (such as the CPS algorithm), then the specific search process can be as follows:

[0116] First, by comparing the shape and direction of the centerline information of the tracheal tree 3D model, and based on the coordinates of all the second movement path points in the second movement path point set of the positioning sensor in each level of the tracheal model, forward and backward searches are performed relative to the reference point to determine the second movement path point adjacent to the reference point A. Specifically, two second movement path points adjacent to the reference point A can be determined, one of which is located in the primary tracheal model 1, and the other is located in the secondary tracheal model 2 (i.e., the right main bronchus model 21), which is connected to the primary tracheal model 1 and deviated to the right. Then, based on the determined second movement path point adjacent to the reference point A, the second target level tracheal model to be searched is determined to be: the primary tracheal model 1 and the right main bronchus model 21.

[0117] For the two second-target tracheal models identified above (i.e., first-level tracheal model 1 and right main bronchus model 21), path search can be performed simultaneously.

[0118] Taking the right main bronchus model 21 as an example: First, based on the centerline information of the 3D tracheal tree model, the centerline l1 of the right main bronchus model 21 (a solid line between bifurcation point b0 and bifurcation point b1) can be determined. Then, based on the two endpoints of the centerline l1 (i.e., bifurcation point b0 and bifurcation point b1), the starting and ending positions of the right main bronchus model 21 can be determined. For example, since path searching is performed with a reference point as the starting point, the right main bronchus model 21 should be searched forward relative to the reference point. Therefore, the endpoint closest to the reference point among the two endpoints of the centerline l1 can be used as the starting position of the right main bronchus model 21, and the other endpoint as the ending position. That is, the starting and ending positions of the right main bronchus model 21 are bifurcation point b0 and bifurcation point b1, respectively. Further, a second set of movement path points (denoted as movement point set G1) can be used within the right main bronchus model 21 for positioning sensors. Figure 4a In the right main bronchus model 21, using a set of virtual points (shown as virtual lines), the search starts by finding the movement path point c0 with the shortest distance to the bifurcation point b0, and ends by finding the movement path point c1 with the shortest distance to the bifurcation point b1. Further, using the search start point (i.e., movement path point c0) as the first path point (i.e., the initial path point), a forward search is performed relative to the first path point to determine at least one first adjacent point in the movement point set G1 that is adjacent to the first path point. Then, the distances (e.g., Euclidean distances) between the first path point and each first adjacent point are calculated, and the first adjacent point with the shortest corresponding distance is selected as the second... The search process begins by identifying a second path point and adding it to the search path point set Search_G corresponding to the right main bronchus model 21. Next, a forward search is performed relative to the second path point to determine at least one second adjacent point on the movement point set G1 that is adjacent to the second path point. The second adjacent point with the shortest distance from the second path point is selected as the third path point and added to the search path point set Search_G corresponding to the right main bronchus model 21. This process continues until the search reaches its endpoint (i.e., movement path point c1), at which point the path search for the right main bronchus model 21 ends. Finally, the path points in the search path point set Search_G are connected sequentially to generate a path, which is the shortest movement path for the positioning sensor within the right main bronchus model 21.

[0119] Similarly, the above path search can be performed on the primary tracheal model 1 (main tracheal model) to determine the shortest movement path of the outgoing sensor in the primary tracheal model 1.

[0120] It should be noted that the forward search described in the above example can be understood as searching in the ascending direction of the tracheal model hierarchy within the 3D tracheal tree model. When performing a path search for the first-level tracheal model 1, a backward search is performed.

[0121] As described above, after the path search is completed for the right main bronchus model 21, a forward search can be performed relative to the movement path point c1. Based on the coordinates of all the second movement path points in the set of second movement path points in each level of tracheal model obtained by the positioning sensor, it is determined whether there is a second movement path point adjacent to movement path point c1. If there is, the corresponding level of tracheal model where the second movement path point adjacent to movement path point c1 is located is determined as the next target level tracheal model for which a path search should be performed. The path search is then performed again for the next target level tracheal model for which a path search should be performed. For the specific path search implementation process, please refer to the path search process detailed above for the right main bronchus model 21. It will not be described in detail here.

[0122] Based on the above example, a specific feasible technical solution for the above-mentioned 10232 "determining the starting and ending positions of the second target-level tracheal model" is as follows:

[0123] 102321. Determine the first centerline of the second target-level tracheal model from the centerline information;

[0124] 102322. Determine the starting and ending positions of the second target-level tracheal model based on the two endpoints of the first centerline.

[0125] In step 1024 above, the shortest movement paths of the positioning sensors in each level of the tracheal model can be merged to finally obtain the movement path of the positioning sensors in the 3D model of the tracheal tree. This merging can be achieved by connecting the beginning and end of the path.

[0126] The above finally yields the movement path of the positioning sensor in the 3D model of the tracheal tree, which can be found in [reference needed]. Figure 4b The dashed lines shown in the 3D model of the tracheal tree.

[0127] After obtaining the airway path of the tracheal tree 3D model and the movement path of the positioning sensor in the tracheal tree 3D model through steps 101-104 above, key points on the movement path and airway path can be further extracted for subsequent registration. To ensure accurate registration between the tracheal tree 3D model and the actual lung trachea, and especially to effectively avoid deformation of the actual lung trachea caused by the patient's breathing and other movements during surgery, particularly significant deformation at the tracheal ends, which can lead to poor matching and large deviations between the actual lung trachea ends and the tracheal model ends in the tracheal tree 3D model, this embodiment considers that when the positioning sensor moves within the lung trachea, if it moves to the intersection of at least two different tracheas, it will be at the end of the corresponding tracheal end. For example, if the positioning sensor moves within the lung trachea, if it moves to the intersection of at least two different tracheas, it will be at the end of the corresponding tracheal end. When the positioning sensor moves to the intersection of the main trachea and two main bronchi in the trachea of ​​the lungs, it can be considered that the positioning sensor is at the end of the main trachea. Using the path points generated by the positioning sensor at the end of the main trachea as key points for registration analysis is beneficial to improving the registration accuracy at the end of the trachea. Based on this, in this embodiment, the bifurcation points on the path will be used as key points. In addition, it is also considered that the point at the end of the path (i.e., the path endpoint) may also be a path point generated by the positioning sensor when it moves to the corresponding end of the trachea. For example, when the positioning sensor moves to a certain segment of the trachea (such as the intersection of the main trachea and the two main bronchi), it may be a path point generated by the positioning sensor when it moves to the end of the corresponding trachea. Figure 3 The path point generated at the end of the actual trachea corresponding to the first segment trachea model 41 shown in the figure is the path endpoint. Therefore, in order to further improve the registration accuracy of the trachea end, this embodiment also uses the path endpoint on the path as the key point. The key points on the airway path can be determined based on the key points on the extracted movement path.

[0128] Specifically, when performing step 105 above to extract key points (the first key points) on the movement path, the reference point mentioned above can be used as the starting point to perform a search around the movement path to extract the movement path branching points, until the end of each path included in the movement path is found, and the corresponding movement path endpoint is extracted, thereby obtaining the first key point set of the movement path.

[0129] That is, the aforementioned first set of key points includes: a path branching point and a path endpoint. Specifically, it may include at least one path branching point and at least one path endpoint. The reason for including at least one path branching point will be explained in detail in the last part of the following textual embodiment.

[0130] For example, see Figure 4b The movement path shown by the dashed line is defined as follows: The first set of key points of this movement path includes at least one path fork point. Figure 4b China and Israel The bifurcation points c0', c1', and c2' shown include at least one endpoint of the movement path, which is: Figure 4b China and Israel The path endpoints d1' and d2' are shown.

[0131] In the above 106, the second set of key points determined for the airway path corresponds to the first set of key points. Therefore, the second set of key points includes the airway path bifurcation point and the airway path endpoint corresponding to the movement path bifurcation point and the movement path endpoint, respectively.

[0132] In specific implementation, when determining airway bifurcation points, the airway bifurcation points corresponding to the movement path bifurcation points can be found on the airway path based on the target-level tracheal model where the movement path bifurcation point is located. Based on the above, the aforementioned second key point set includes airway bifurcation points; correspondingly, step 106, "determining the second key point set corresponding to the first key point set on the airway path," can specifically include the following steps:

[0133] 1061. Determine the first target-level tracheal model in which the bifurcation point of the movement path is located in the tracheal tree 3D model;

[0134] 1062. On the airway path, find the airway path bifurcation point in the first target-level airway model that is the shortest distance to the bifurcation point of the movement path.

[0135] For example, continuing from the example in step 105 above, see [link to example]. Figure 4b and combined Figure 3 Taking the bifurcation point c1' as an example, based on the obtained coordinates of bifurcation point c1', it can be determined that bifurcation point c1' is in a right-leaning secondary trachea model, namely the right main bronchus model 21. Further, by searching along the airway path, it can be determined that bifurcation point b1 on the airway path is in the right main bronchus model 21 and is closest to bifurcation point c1'. Therefore, bifurcation point b1 is the airway path bifurcation point corresponding to bifurcation point c1'. Accordingly, the airway path bifurcation points corresponding to the other bifurcation points c0' and c2' included in the first key point set can also be determined as bifurcation points b0 and b6, respectively.

[0136] Furthermore, the aforementioned second key point set also includes the airway path endpoint; and the "determining the second key point set corresponding to the first key point set in the airway path" in step 106 may further include the following steps:

[0137] 1063. Along the movement path, determine the target movement path branch point that is closest to the end point of the movement path;

[0138] 1064. Calculate the first distance between the bifurcation point of the target movement path and the end point of the movement path;

[0139] 1065. Determine the search direction based on the fork point of the target movement path and the end point of the movement path;

[0140] 1066. Determine the target airway path bifurcation point corresponding to the bifurcation point of the target movement path from the airway path;

[0141] 1067. Following the search direction, find the airway path endpoint corresponding to the endpoint of the moving path on the airway path, wherein the distance between the airway path endpoint and the target airway path bifurcation point is equal to the first distance.

[0142] For example, see continue. Figure 4b Taking the endpoint of the movement path as the endpoint d2' as an example, then:

[0143] First, following the movement path (i.e., the dotted line shown in the diagram), determine the nearest fork in the path to the endpoint d2' as fork point c2'. Then, based on the coordinates of fork point c2' and endpoint d2', use a suitable distance calculation algorithm (such as Manhattan distance) to calculate the distance between fork point c2' and endpoint d2' as the first distance Distan_0. Finally, based on fork point c2' and endpoint d2', determine the search direction for subsequent use. The search direction is as follows: Figure 4b The direction indicated by the solid arrow is the hierarchical progression of different levels of the tracheal model in the 3D tracheal tree model, more specifically, the hierarchical progression direction that is slightly to the upper right.

[0144] Then, following the airway path bifurcation point determination scheme described in steps 1061-1062 above, the target airway path bifurcation point corresponding to bifurcation point c2' can be determined as bifurcation point b6. Alternatively, steps 1061-1062 can be skipped, and the target airway path bifurcation point corresponding to bifurcation point b6 can be determined directly from the airway path determined by steps 1061-1062 above, which correspond to the bifurcation points of each movement path included in the first key information, namely bifurcation points c0', c1', and c2' respectively (i.e., bifurcation points b0, b1, and b6).

[0145] Furthermore, following the search direction determined above, search for a path point on the airway path whose distance from the bifurcation point b6 is equal to the first distance Distan_0 mentioned above, for example... Figure 4bThe path point e1 shown is the airway path endpoint corresponding to the aforementioned path endpoint d2'.

[0146] Therefore, the airway path endpoints corresponding to the other movement path endpoints (i.e., path endpoints d1') contained in the first set of key points can also be determined, such as... Figure 4b The path point e2 is shown in the diagram. In determining the airway path endpoint corresponding to the path endpoint d1', the search direction is as follows: Figure 4b The direction indicated by the dashed arrow.

[0147] Based on the examples above, the final set of second key points for the airway path may include: airway path bifurcation point b0, airway path bifurcation point b1, airway path bifurcation point b6, path point e1, and path point e2.

[0148] It should be noted that, based on the examples described above for determining the airway bifurcation point and the airway endpoint, the embodiments of this application can determine the airway endpoint after determining the airway bifurcation point, or the determination of the airway bifurcation point can be performed during the process of determining the airway endpoint; this is not limited here. For example, in the example given above for steps 1063 to 1067, in the process of determining the target airway path bifurcation point corresponding to bifurcation point c2' (a movement path bifurcation point) as bifurcation point b6, if the method adopted is to directly determine the bifurcation point from the airway path determined by executing steps 1061 to 1062 above, which corresponds to each movement path bifurcation point, namely bifurcation point c0', bifurcation point c1', and bifurcation point c2' respectively, that is, bifurcation point b0, bifurcation point b1, and bifurcation point b6, then the airway path bifurcation point is determined after the airway path bifurcation point is determined; if the method adopted is to determine the airway path bifurcation point using the method described in steps 1061 to 1062 above, then the airway path bifurcation point is determined during the process of determining the airway path end point.

[0149] In section 107 above, based on the first key point set of the movement path and the second key point set of the airway path determined above, a corresponding registration algorithm can be used to find the spatial coordinate transformation relationship (or transformation matrix) between the first key point set from the actual physical spatial coordinate system (magnetic spatial coordinate system) of the lungs and trachea and the second key point set from the image spatial coordinate system. This allows for spatial matching between the two, and the original spatial coordinate transformation relationship is updated to the newly found spatial coordinate transformation relationship. Furthermore, according to this newly found spatial coordinate transformation relationship, accurate spatial matching between the tracheal tree 3D model and the actual lungs and trachea can be achieved.

[0150] The registration algorithm mentioned above can be, but is not limited to, ICP (Iterative Closest Point), NDT (Normal Distribution Transform), and other fine registration algorithms such as point cloud registration based on deep learning. This embodiment preferably uses the ICP algorithm. For specific implementation details on using the ICP algorithm to find the spatial coordinate transformation relationship between the first and second keypoint sets, please refer to existing related content.

[0151] The technical solution provided in this embodiment, based on the medical image graphics reconstruction of the lungs and trachea to obtain the corresponding 3D model of the tracheal tree, determines the airway path of the 3D model of the tracheal tree according to the centerline information of the obtained 3D model of the tracheal tree; and determines the movement path of the positioning sensor in the 3D model of the tracheal tree according to the first movement path point set of the positioning sensor in the lungs and trachea; then, extracts the first key point set of the movement path (including the movement path bifurcation point and the movement path endpoint), and determines the second key point set corresponding to the first key point set in the airway path, and then registers the 3D model of the tracheal tree with the lungs and trachea according to the first key point set and the second key point set. The aforementioned path bifurcation points are generally the path points where the positioning sensor intersects with two different tracheas during its movement within the trachea of ​​the lungs. In other words, the positioning sensor's location within the trachea, as indicated by the path bifurcation point, is typically at one end of the actual trachea, such as the distal end. Therefore, this solution utilizes the path bifurcation points and the corresponding points on the airway path of the tracheal tree 3D model to achieve registration between the tracheal tree 3D model and the actual trachea of ​​the lungs, effectively improving the registration accuracy at the distal end of the trachea. This solution also incorporates the path endpoint and the corresponding point on the airway path during registration. The path endpoint is the point at the end of the corresponding path in the movement path, representing the positioning sensor's location within the trachea of ​​the lungs. If it is at the distal end of the trachea, this further improves the registration accuracy at the distal end; if it is within the trachea (such as in the middle), it helps to improve the overall registration accuracy between the tracheal tree 3D model and the actual trachea of ​​the lungs to a certain extent. Furthermore, the number of bifurcation points and endpoints of the aforementioned movement paths is often limited and small. Therefore, this scheme can effectively reduce the amount of registration data computation while ensuring registration accuracy. In summary, this scheme can effectively improve the registration accuracy between the 3D tracheal tree model and the actual lung trachea, especially the registration accuracy at the tracheal terminal, which can provide a guarantee for subsequent precise guidance of surgery.

[0152] Generally, point cloud registration is divided into two registration steps: coarse registration and fine registration. Coarse registration is the initial registration of point clouds, which refers to aligning point clouds in two different spaces as closely as possible using an initial spatial transformation matrix. After coarse registration, the overlapping parts of the two point clouds can be roughly aligned, but the accuracy is often far from meeting the requirements for subsequent positioning. Therefore, fine registration is needed. Fine registration refers to further calculating the approximate spatial transformation matrix of the two point clouds based on the initial registration. Based on this, before performing step 102 above, this embodiment can also use a corresponding coarse registration algorithm to first perform coarse registration between the tracheal tree 3D model and the first movement path points, so that the coarsely registered first movement path point set and tracheal tree 3D model can be used for subsequent processing to achieve the fine registration that benefits the first key point set and the second key point set. That is, further, the method provided in this embodiment also includes the following steps:

[0153] S21. Register the first set of movement path points with the 3D model of the trachea tree to obtain the registered first set of movement path points and the registered 3D model of the trachea tree.

[0154] S22. Based on the registered first moving path point set and the registered tracheal tree 3D model, trigger the execution of step 102 above.

[0155] In practice, some classic coarse registration algorithms can be used, but are not limited to, such as RANSAC (Random Sample Consensus) and 4PCS (4-Point Congruent Sets), to coarsely register the first movement path point set and the tracheal tree 3D model.

[0156] Of course, the methods used in fine registration, such as ICP and NDT, can also be used in this coarse registration process, and are not limited here. However, since coarse registration often involves a large amount of data processing, using algorithms such as ICP and NDT often results in slow calculation speed and low accuracy. Therefore, in this embodiment, the coarse registration is preferably implemented using algorithms such as RANSAC and 4PCS, with RANSAC being more preferred. For details on the specific process of using RANSAC to implement coarse registration, please refer to existing related content.

[0157] Furthermore, the method provided in this embodiment may also include the following steps:

[0158] 108. Periodically update the first set of mobile path points, and re-execute each of the above steps based on the updated first set of mobile path points.

[0159] The purpose of performing step 108 in this embodiment is to achieve real-time registration between the 3D model of the tracheal tree and the trachea of ​​the lungs. The period for updating the first movement path point set can be flexibly set according to actual conditions, such as 1 second, 2 seconds, 1 minute, etc., and is not limited here.

[0160] In summary, it is necessary to further explain that: if the positioning sensor remains stationary within the trachea, the point cloud data collected by the positioning sensor (i.e., the movement path point data mentioned above, or the position data of the positioning sensor within the trachea) is often meaningless; and, initially, the amount of point cloud data collected by the positioning sensor is often relatively small. In such cases, if the point cloud data collected by the positioning sensor is processed for registration calculations, the registration accuracy will often be low. Therefore, in this embodiment, processing of the point cloud data collected by the positioning sensor only begins when the number of point clouds collected by the positioning sensor is greater than or equal to a set threshold, and the movement range of the positioning sensor is greater than a set range threshold. This is to determine the current movement path of the positioning sensor in the tracheal tree 3D model, the first key point set of the movement path, etc.

[0161] In this embodiment, the reference point is the point in the 3D model of the tracheal tree that corresponds to the main carina of the trachea in the lungs. Figure 4a The reference point A shown in the figure is used to trigger the registration scheme provided in this embodiment only when the positioning sensor is detected to have reached the reference point. For example, in lung surgery navigation, when the physician manipulates the positioning sensor to move in the patient's lung trachea, if it is determined that the positioning sensor has reached the reference point A in the tracheal tree 3D model based on the current positioning position of the positioning sensor in the lung trachea, then the registration scheme provided in this embodiment will be triggered. Figure 9 The surgical registration interface displayed shows a prompt message such as "Positioning sensor placed at the main carina position, click the [Confirm] button to start registration." Upon clicking the "Confirm" button in the prompt message, the registration process begins automatically. The aforementioned surgical registration interface can be accessed via... Figure 1cThe display screen of the control device 511 shown in the figure is used to display the process; during the registration process, the physician can continue to manipulate the positioning sensor to move within the patient's trachea. Based on the above example, when the point cloud data collected by the positioning sensor is processed for registration, the positioning sensor reaches at least the end of the primary trachea model (main trachea model) in the trachea 3D model, which also ensures that the first key point set mentioned above contains at least one movement path bifurcation point. Furthermore, based on the above example, when setting the aforementioned setting range threshold, this application embodiment can flexibly set it according to the actual situation, as long as it ensures that when the point cloud data collected by the positioning sensor is processed, the movement range of the positioning sensor has exceeded the primary trachea model (i.e., main trachea model) area of ​​the trachea tree 3D model, that is, it ensures that the positioning sensor is not currently within the primary trachea model of the trachea tree 3D model.

[0162] In the above processing, the point cloud data collected by the positioning sensor is first preprocessed to remove some abnormal data, such as outliers and invalid data caused by signal loss. For example, if the coil inside the positioning sensor cannot sense a magnetic field due to interference, it will be unable to generate current. In this case, the positioning sensor will send a default signal to the execution subject of this embodiment. The received default signal becomes invalid data or may be outlier data.

[0163] An embodiment of this application also provides a registration system for the lungs and trachea corresponding to the above-described method embodiments. See also... Figure 5 As shown, the registration system for the lungs and trachea may include:

[0164] A magnetic field generator is used to generate a positioning magnetic field; the trachea of ​​the lungs is placed in the positioning magnetic field;

[0165] A positioning sensor that generates a positioning signal by sensing the positioning magnetic field when moving within the trachea of ​​the lungs, and sends the positioning signal to a processing device so that the processing device can determine the movement path point of the positioning sensor within the trachea of ​​the lungs based on the positioning signal;

[0166] A processing device for performing the steps in the embodiments of the registration method for lung trachea provided in this application.

[0167] For a detailed description of the aforementioned magnetic field generator and positioning sensor, please refer to the relevant content in other embodiments of the above application; further details will not be repeated here.

[0168] The aforementioned processing equipment can be a control device with data processing capabilities, such as... Figure 1c The control device 511 shown is an example of a computer device. The control device can be independently set up, for example... Figure 1c The doctor's main control console 50 shown in the image, or it can be used with, for example... Figure 1a The magnetic navigation device 20 shown is integrated into the magnetic navigation device 20, but this embodiment does not limit this.

[0169] It should be noted that the lung-tracheal registration system provided in the above embodiments may include other devices or equipment besides those described above. The details of these other devices or equipment can be found in the above-described related systems. Figures 1a to 1c This section introduces information related to the medical system.

[0170] Figure 6 A structural block diagram of a registration device for the lungs and trachea provided in one embodiment of this application is shown. Figure 6 As shown, the registration progress detection device for the lungs and trachea includes: an acquisition module 61, a determination module 62, an extraction module 63, and a registration module 64; wherein,

[0171] Acquisition module 61 is used to acquire the first set of movement path points of the positioning sensor within the trachea of ​​the lung;

[0172] The determining module 62 is used to determine the movement path of the positioning sensor in the tracheal tree three-dimensional model based on the first movement path point set, wherein the tracheal tree three-dimensional model is obtained by reconstructing medical images of the lung trachea;

[0173] The acquisition module 61 is also used to acquire the centerline information of the tracheal tree three-dimensional model;

[0174] The determining module 62 is further configured to determine the airway path of the tracheal tree three-dimensional model based on the centerline information;

[0175] Extraction module 63 is used to extract a first set of key points of the movement path, the first set of key points including the movement path branching point and the movement path endpoint;

[0176] The determining module 62 is further configured to determine a second set of key points corresponding to the first set of key points on the airway path;

[0177] The registration module 64 is used to register the tracheal tree 3D model with the lung trachea based on the first key point set and the second key point set.

[0178] Furthermore, the aforementioned tracheal tree 3D model includes a multi-level tracheal model; the second key point set includes airway path bifurcation points; and the aforementioned determining module 62, when used to determine the second key point set corresponding to the first key point set in the airway path, is specifically used to: determine the first target-level tracheal model in which the movement path bifurcation point is located in the tracheal tree 3D model; and on the airway path, find the airway path bifurcation point in the first target-level tracheal model that has the shortest distance to the movement path bifurcation point.

[0179] Further, the aforementioned second key point set includes the airway path endpoint; and the aforementioned determining module 62, when determining the second key point set corresponding to the first key point set in the airway path, is specifically used for: determining the target moving path bifurcation point closest to the moving path endpoint along the moving path; calculating the first distance between the target moving path bifurcation point and the moving path endpoint; determining the search direction based on the target moving path bifurcation point and the moving path endpoint; determining the target airway path bifurcation point corresponding to the target moving path bifurcation point on the airway path; and searching for the airway path endpoint corresponding to the moving path endpoint on the airway path according to the search direction; wherein, the distance between the airway path endpoint and the target airway path bifurcation point is equal to the first distance.

[0180] Furthermore, the aforementioned tracheal tree three-dimensional model includes multi-level tracheal models; the aforementioned determining module 62, when determining the movement path of the positioning sensor in the tracheal tree three-dimensional model based on the first movement path point set, is specifically used for: determining the second movement path point set of the positioning sensor in each level of the tracheal model based on the first movement path point set; determining the point in the tracheal tree three-dimensional model corresponding to the main carina of the lung trachea as a reference point; starting from the reference point, performing path search for each level of the tracheal model based on the second movement path point set of the positioning sensor in each level of the tracheal model to obtain the shortest movement path of the positioning sensor in each level of the tracheal model; merging the shortest movement paths of the positioning sensor in each level of the tracheal model to obtain the movement path.

[0181] Further, the aforementioned determining module 62, when used to perform path searches for each level of the trachea model based on the second movement path point set of the positioning sensor in each level of the trachea model starting from the reference point, specifically performs the following: starting from the reference point, determining the second target level trachea model for which path search should be performed; determining the starting position and ending position of the second target level trachea model; and finding the movement path point with the shortest distance to the starting position from the second movement path point set of the positioning sensor in the second target level trachea model, determining it as the search starting point and the ending position as the search endpoint. The point of least distance from the starting point to the destination is determined as the search endpoint. A path search is performed based on the search starting point, the search endpoint, and the second set of movement points of the positioning sensor within the second target-level tracheal model. The shortest path from the search starting point to the search endpoint is determined based on the search results and used as the shortest movement path of the positioning sensor within the second target-level tracheal model. After the path search for the second target-level tracheal model is completed, it is determined whether there are any tracheal models in the multi-level tracheal model that require route searching. If so, the process returns to the step of determining the second target-level tracheal model for which a path search should be performed.

[0182] Furthermore, the aforementioned determining module 62, when used to determine the starting and ending positions in the second target-level tracheal model, is specifically used to: determine the first centerline of the second target-level tracheal model from the centerline information; and determine the starting and ending positions of the second target-level tracheal model based on the two endpoints of the first centerline.

[0183] Further, the aforementioned determining module 62, when determining the second set of movement path points of the positioning sensor in each level of the tracheal model based on the first set of movement path points, specifically performs the following steps: determining the first set of voxel points in each level of the tracheal model corresponding to the movement path points contained in the first set of movement path points; determining the tracheal radius of each level of the tracheal model; performing downsampling processing on the first set of voxel points in each level of the tracheal model based on the tracheal radius; and determining the first set of voxel points in each level of the tracheal model after downsampling processing as the second set of movement path points of the positioning sensor in each level of the tracheal model.

[0184] Furthermore, the third target-level tracheal model is one of the multi-level tracheal models; and, the aforementioned determining module 62, when determining the tracheal radius of the third target-level tracheal model, is specifically used to: determine the second centerline of the third target-level tracheal model from the centerline information; calculate the second distance from each second voxel point on the surface of the third target-level tracheal model to the second centerline; and determine the average of the second distances from each second voxel point to the second centerline as the tracheal radius of the third target-level tracheal model.

[0185] Furthermore, the registration module 64 is also used to: register the first set of movement path points with the three-dimensional model of the tracheal tree to obtain the registered first set of movement path points and the registered three-dimensional model of the tracheal tree; and the device provided in this embodiment further includes: a triggering module, used to trigger the execution of the step of determining the movement path of the positioning sensor in the three-dimensional model of the tracheal tree reconstructed for the lungs and trachea based on the registered first set of movement path points and the registered three-dimensional model of the tracheal tree.

[0186] Furthermore, the apparatus provided in this embodiment may further include: an update module; the update module is used to periodically update the first movement path point set, so as to re-execute the step of registering the first movement path point set with the tracheal tree three-dimensional model according to the updated first movement path point set.

[0187] It should be noted that the registration device for the lungs and trachea provided in the above embodiments can realize the technical solutions described in the method embodiments provided in this application. The specific implementation principles of each module or unit can be found in the corresponding content of the above method embodiments, and will not be repeated here.

[0188] Figure 7 A schematic diagram of the structure of an electronic device provided according to an embodiment of this application is shown. Figure 7 As shown, the electronic device includes a memory 71 and a processor 72. The memory 71 stores a computer program; the processor, coupled to the memory, executes the computer program stored in the memory to implement the steps or functions of the lung-trachea registration method embodiments provided in this application.

[0189] The aforementioned memory 71 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0190] The aforementioned electronic devices may refer to the above-mentioned Figure 5 The processing device shown in the figure.

[0191] Furthermore, such as Figure 7 As shown, the electronic device may also include other components such as a communication component 73, a display 74, a power supply component 75, and an audio component 76. Figure 7 The diagram only shows some components and does not imply that the electronic device includes only these components. Figure 7 The components shown.

[0192] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, can implement the steps or functions in the registration progress detection method for lungs and trachea provided in the embodiments of this application.

[0193] The methods in this application can be implemented, in whole or in part, by software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented, in whole or in part, as a computer program product. Figure 8 A block diagram of a computer program product provided in this application is schematically shown. The computer program product includes a computer program / instructions 81, which, when the computer program / instructions 81 are executed by, for example... Figure 7 When the processor 72 shown is executed, it can perform all or part of the steps or functions in the lung and trachea registration methods provided in the various embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, a core network device, an OAM, or other programmable device.

[0194] The computer program or instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; or an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0195] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0197] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A registration system for the lung airways, characterized by, The method comprises the following steps: a magnetic field generator for generating a positioning magnetic field; a lung airway is located in the positioning magnetic field; a positioning sensor which can generate a positioning signal by sensing the positioning magnetic field when moving in the lung airway, and send the positioning signal to a processing device to determine a moving path point of the positioning sensor in the lung airway according to the positioning signal by the processing device; a processing device for performing the following steps in a registration method for a lung airway; The registration method for a lung airway comprises the following steps: receiving a first moving path point set sent by a magnetic navigation device; the first moving path point set represents a path formed by the movement of a positioning sensor in a lung airway; determining a moving path of the positioning sensor in a three-dimensional model of a bronchial tree according to the first moving path point set, wherein the three-dimensional model of the bronchial tree is reconstructed based on medical image images of the lung airway; obtaining center line information of the three-dimensional model of the bronchial tree; determining an airway path of the three-dimensional model of the bronchial tree according to the center line information; extracting a first key point set of the moving path, which includes a moving path bifurcation point and a moving path end point of the moving path; determining a second key point set corresponding to the first key point set in the airway path; registering the three-dimensional model of the bronchial tree and the lung airway according to the first key point set and the second key point set.

2. The registration system of claim 1, wherein, The three-dimensional model of the bronchial tree comprises a multi-level bronchial model; the second key point set comprises an airway path bifurcation point; determining a second key point set corresponding to the first key point set in the airway path comprises: determining a first target level bronchial model in which the moving path bifurcation point is located in the three-dimensional model of the bronchial tree; finding an airway path bifurcation point in the first target level bronchial model and closest to the moving path bifurcation point on the airway path.

3. The registration system of claim 1, wherein, The second key point set comprises an airway path end point; determining a second key point set corresponding to the first key point set in the airway path comprises: determining a target moving path bifurcation point closest to the moving path end point along the moving path; calculating a first distance between the target moving path bifurcation point and the moving path end point; determining a search direction according to the target moving path bifurcation point and the moving path end point; determining a target airway path bifurcation point corresponding to the target moving path bifurcation point on the airway path; finding an airway path end point corresponding to the moving path end point on the airway path in the search direction, wherein the distance between the airway path end point and the target airway path bifurcation point is equal to the first distance.

4. The registration system of claim 1, wherein, The three-dimensional model of the bronchial tree comprises a multi-level bronchial model; determining a moving path of the positioning sensor in the three-dimensional model of the bronchial tree according to the first moving path point set comprises: determining a second moving path point set of the positioning sensor in each level bronchial model according to the first moving path point set; determining a point corresponding to a main carina of the lung airway in the three-dimensional model of the bronchial tree as a reference point; The second moving path point set of the positioning sensor in each air tube model is determined according to the first moving path point set, and the second moving path point set of the positioning sensor in each air tube model is determined as follows: A first voxel point set corresponding to the moving path point set in each air tube model is determined; 5. The registration system of claim 4, wherein, The air tube radius of each air tube model is determined; The first voxel point set in each air tube model is down-sampled based on the air tube radius; The first voxel point set of each air tube model after the down-sampling processing is determined as the second moving path point set of the positioning sensor in each air tube model. The third target air tube model is one of the multi-level air tube models; The air tube radius of the third target air tube model is determined as follows: A second center line of the third target air tube model is determined from the center line information; The second distance of each second voxel point on the surface of the third target air tube model to the second center line is calculated; 6. The registration system of claim 5, wherein, The mean value of the second distance of each second voxel point to the second center line is determined as the air tube radius of the third target air tube model. The memory and the processor, wherein, The memory is configured to store a computer program.

7. The registration system of claim 4, wherein, ​ ​ ​ ​ ​ 8. The registration system of claim 7, wherein, ​ ​ ​ ​ ​ 9. An electronic device, comprising: ​ ​ ​ The processor, coupled with the memory, is configured to execute the computer program stored in the memory, so as to implement the steps in the method executed by the processing device in the registration system for the lung trachea according to any one of claims 1 to 8. The electronic device is the processing device in the registration system for the lung trachea according to any one of claims 1 to 8.

Images