Intraoperative navigation methods, devices, equipment, and media based on cone-beam CT for bronchial procedures

Abstract

This invention provides a method, device, equipment, and medium for intraoperative bronchial navigation based on cone-beam CT (CBCT), comprising: estimating the current estimated position of the interventional end in the bronchial tree model and determining the target path based on bronchoscope images and a bronchial tree model during CBCT-assisted localization; determining a first-view plane and N second-view planes based on the target path; acquiring CBCT projection images of the bronchus under the first-view plane and the N second-view planes respectively; performing forward projection on the preoperative three-dimensional medical images of the bronchus according to the forward projection angles corresponding to the first-view plane and the N second-view planes to obtain simulated projection images under the first-view plane and the N second-view planes; and registering the CBCT projection images and simulated projection images under the same view to obtain the current three-dimensional localization information of the interventional end in the three-dimensional medical image. The three-dimensional localization information of this invention is more accurate, with shorter imaging time and lower radiation dose.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a method, device, equipment and medium for intraoperative bronchial navigation based on cone-beam CT. Background Technology

[0002] CT is short for Computed Tomography. CBCT is short for Cone Beam CT.

[0003] Currently, CBCT-assisted intraoperative navigation in bronchial surgery typically employs two methods: one is cone-beam CT-3D imaging navigation. Real-time cone-beam CT-3D imaging has long imaging times and high radiation doses. Prolonged imaging may increase surgical risks and complexity, while high radiation doses can cause unnecessary radiation damage to patients. The other method is single-view 2D projection imaging. Because 2D images lack depth information, a unique matching point cannot be obtained from preoperative 3D images. Although the three-dimensional coordinates need to meet constraints within the bronchial structure, in complex structures with numerous bronchial bifurcations, different bronchial branches overlap in the depth direction, making it difficult to guarantee positioning accuracy. This can easily lead to mapping to incorrect bronchial branches, causing a mismatch between the virtual coordinates of the interventional device and its actual position, resulting in navigation failure. Therefore, single-view 2D projection imaging navigation suffers from low positioning accuracy and a high error rate in complex branch areas.

[0004] Therefore, it is necessary to propose a method, device, equipment, and medium for intraoperative bronchial navigation based on cone-beam CT to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, equipment and medium for intraoperative bronchial navigation based on cone-beam CT, in order to improve the problems of long imaging time and high radiation dose of cone-beam CT-3D image navigation, low navigation and positioning accuracy of single-view 2D projection image and high error rate at complex branches.

[0006] In a first aspect, the present invention provides a method for intraoperative navigation of bronchial surgery based on cone-beam CT, the method comprising:

[0007] S101: When performing cone-beam CT-assisted localization, based on the bronchoscopic images and the bronchial tree model, the current estimated position of the interventional device's interventional end in the bronchial tree model is estimated, and the target path is determined based on the current estimated position.

[0008] S102: Determine a first view plane and N second view planes based on the target path. The first view plane is the plane closest to the target path. The N second view planes have different angles with the first view plane, and the angles are not 0. N is a positive integer.

[0009] S103: Based on the first perspective plane and N second perspective planes, acquire cone-beam CT projection images of the bronchus under the first perspective and N second perspectives respectively;

[0010] S104: Based on the forward projection angles corresponding to the first view plane and N second view planes, perform forward projection on the three-dimensional medical images of the bronchus before surgery to obtain simulated projection images under the first view and N second view.

[0011] S105: For each of the first viewpoint and N second viewpoints, register the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image.

[0012] In one possible embodiment, for each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image, including:

[0013] For each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the deformation field corresponding to the first viewpoint and N second viewpoints. The cone-beam CT projection image is a fixed image, and the simulated projection image is a floating image.

[0014] Based on the deformation fields corresponding to the first view and N second view, and the position information of the intervention end in the cone-beam CT projection images under the first view and N second view, the position information of the intervention end in the simulated projection images under the first view and N second view is obtained.

[0015] Based on the position information of the interventional end in the simulated projection images under the first perspective and N second perspectives, the current three-dimensional positioning information of the interventional end in the three-dimensional medical image is obtained.

[0016] In one possible embodiment, for each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the deformation field corresponding to the first viewpoint and the N second viewpoints, including:

[0017] For each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the feature point pairs corresponding to the first viewpoint and N second viewpoints;

[0018] Based on the feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated.

[0019] Based on the affine transformation matrices corresponding to the first perspective and N second perspectives, the deformation fields corresponding to the first perspective and N second perspectives are obtained.

[0020] In one possible embodiment, before obtaining the deformation field corresponding to the first viewpoint and the N second viewpoints based on the affine transformation matrices corresponding to the first viewpoint and the N second viewpoints, the method further includes:

[0021] Based on the positions of feature point pairs corresponding to the first viewpoint and N second viewpoints, and the mutual information values ​​corresponding to the first viewpoint and N second viewpoints, the local elastic transformation matrices corresponding to the first viewpoint and N second viewpoints are obtained.

[0022] Based on the affine transformation matrices corresponding to the first viewpoint and N second viewpoints, the deformation fields corresponding to the first viewpoint and N second viewpoints are obtained, including:

[0023] Based on the affine transformation matrices and local elastic transformation matrices corresponding to the first perspective and N second perspectives, the deformation fields corresponding to the first perspective and N second perspectives are obtained.

[0024] In one possible embodiment, after performing feature point matching on the cone-beam CT projection image and the simulated projection image under the same viewpoint for each of the first viewpoint and N second viewpoints to obtain feature point pairs corresponding to the first viewpoint and N second viewpoints, the method further includes:

[0025] Remove erroneous feature point pairs from the feature point pairs corresponding to the first viewpoint and N second viewpoints to obtain the removed feature point pairs corresponding to the first viewpoint and N second viewpoints;

[0026] Based on the feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated, including:

[0027] Based on the culled feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated.

[0028] In one possible embodiment, determining the target path based on the current estimated location includes:

[0029] On the planned path of the bronchial tree model, a curve segment close to the current estimated position is selected as the first prompting path, and the range of M curves including the first prompting path and smaller than the entire planned path is determined as M second prompting paths, where M is a positive integer;

[0030] The target path is determined in response to the user's selection instruction, which includes one of the first suggested path, M second suggested paths, or the entire planned path.

[0031] In one possible embodiment, the method further includes step S106:

[0032] The current 3D positioning information is mapped onto the 3D medical image and the bronchial tree model for navigation reference.

[0033] In one possible embodiment, the method further includes step S107:

[0034] Find the nearest point of the current three-dimensional positioning information on the bronchial tree point cloud of the bronchial tree model, and display the nearest point on the bronchial tree model to determine whether the intervention end is currently on the planned path of the bronchial tree model.

[0035] Secondly, embodiments of the present invention also provide a bronchial intraoperative navigation device based on cone-beam CT, which includes modules / units for executing any possible design method described in the first aspect above. These modules / units can be implemented in hardware or by hardware executing corresponding software.

[0036] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory. The memory stores one or more computer programs; when the one or more computer programs stored in the memory are executed by the processor, the electronic device is able to implement any of the possible design methods described in the first aspect.

[0037] Fourthly, this invention also provides a computer-readable storage medium comprising a computer program that, when run on an electronic device, causes the electronic device to perform any of the possible designs described in the first aspect.

[0038] Fifthly, embodiments of the present invention also provide a method comprising a computer program product, which, when the computer program product is run on an electronic device, causes the electronic device to perform any possible design of any of the above aspects.

[0039] The beneficial effects of this invention are as follows: acquiring cone-beam CT projection images from at least two perspectives allows for accurate depth information; registration with simulated projection images from corresponding perspectives of three-dimensional medical images yields more accurate three-dimensional positioning information, and it also has the advantages of short imaging time and low radiation dose. Furthermore, selecting the plane closest to the target path as the first perspective plane most directly reflects the elastic deformation caused by the stress of the interventional device, improving registration accuracy and three-dimensional positioning accuracy. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the intraoperative navigation method for bronchial surgery based on cone-beam CT according to the present invention.

[0041] Figure 2 This is a magnified view of the cone-beam CT projection image from the first perspective during bronchial surgery based on cone-beam CT according to the present invention.

[0042] Figure 3a This is a schematic diagram of the bronchial tree model and the planned path located on the bronchial tree model.

[0043] Figure 3b This is a schematic diagram of a local branch of the target path in the bronchial tree model.

[0044] Figure 3c This is a schematic diagram of a local branch of the target path in the bronchial tree model from a first-view perspective.

[0045] Figure 3d This is a schematic diagram of a local branch of the target path in the bronchial tree model from a second-view perspective.

[0046] Figure 4 This is a schematic diagram of the intraoperative navigation device for bronchial surgery based on cone-beam CT according to the present invention.

[0047] Figure 5 This is a schematic diagram of the structure of the electronic device of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] The most intuitive and precise localization method currently available is to acquire and reconstruct cone-beam CT-3D images of the current moment, and then register them with preoperative images to obtain precise localization. However, this method has a long image acquisition time and a high radiation dose, so it is generally not used. Another method is to acquire 2D projection images and register them with preoperative 3D images for localization. This method can obtain relatively accurate positional information on the 2D projection plane, but it lacks depth positional information perpendicular to the 2D plane. In complex structures with many bronchial bifurcations, different bronchial branches overlap on the 2D plane, making it impossible to obtain accurate 3D localization information.

[0050] To address the problems existing in the above-mentioned technologies, embodiments of the present invention provide a method for intraoperative bronchial navigation based on cone-beam CT, applied to the bronchi of the lungs. See [link to relevant documentation]. Figure 1 The method includes:

[0051] S101: When performing cone-beam CT-assisted localization, based on the bronchoscopic images and bronchial tree model, the current estimated position of the interventional device's interventional end in the bronchial tree model is predicted, and the target path is determined based on the current estimated position.

[0052] S102: Determine a first-view plane and N second-view planes based on the target path. The first-view plane is the plane closest to the target path. The target path lies on the second-view planes. Each of the N second-view planes has a different angle with the first-view plane, and the angle is not 0. N is a positive integer. Preferably, the angle includes at least one of 30°, 45°, 60°, or 90°. The angle can be selected and set as needed, and is not limited here.

[0053] S103: Based on the first-view plane and N second-view planes, acquire cone-beam CT projection images of the bronchus under the first-view plane and N second-view planes respectively, such as... Figure 2 As shown.

[0054] S104: Based on the forward projection angles corresponding to the first perspective plane and N second perspective planes, forward projection is performed on the preoperative three-dimensional medical images of the bronchus to obtain simulated projection images under the first perspective and N second perspectives. The simulated projection images correspond one-to-one with the cone-beam CT projection images under the same perspective. Specifically, the three-dimensional medical images can be three-dimensional spiral CT or cone-beam CT images, three-dimensional magnetic resonance images, or other medical images that can reflect the anatomical structure of the bronchus.

[0055] S105: For each of the first viewpoint and N second viewpoints, register the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image.

[0056] During real-time navigation during the procedure, the current status is largely determined by bronchoscopic images. Before the interventional device (e.g., catheter) passes through the tubular structure of the bronchial tree and branches, the path determination is usually unambiguous. The operator can determine when cone-beam computed tomography (CBCT)-assisted localization is needed, such as when the catheter passes through a bronchial branch or when the bronchoscopic image is blurred due to lighting or rapid movement. In these situations, relying solely on bronchoscopic images is insufficient to accurately determine the current position of the interventional device's tip or whether it has entered the correct branch. This embodiment uses CBCT-assisted localization to determine the target path based on the estimated current position of the interventional tip, and then determines a first-view plane and N second-view planes based on the target path. CBCT projection images of the bronchus are acquired under the first-view plane and the N second-view planes. Simulated projection images of the corresponding three-dimensional medical images are obtained, and the CBCT projection images and simulated projection images under the same view are registered to obtain the current three-dimensional localization information of the interventional tip in the three-dimensional medical image. In this embodiment, by acquiring and registering multi-view planes, the position of the interventional device can be more accurately located in three-dimensional space. Compared with single-view 2D projection images, this method provides richer depth information, enabling a more accurate determination of the true position of the interventional device and reducing positioning errors. Furthermore, only cone-beam CT projection images of the corresponding viewpoints need to be captured during cone-beam CT-assisted positioning; there is no need to generate a three-dimensional cone-beam CT image, reducing imaging time and radiation dose, minimizing unnecessary radiation damage to the patient, and also reducing risks during the procedure.

[0057] In one specific embodiment, preoperatively, three-dimensional medical images of the bronchus are acquired. Voxel segmentation of the bronchial airway is performed on the three-dimensional medical images to obtain a three-dimensional model of the bronchial airway. This model is then processed into a skeleton, and a bronchial tree model with an airway tree structure is obtained by proceeding from the airway inlet to the branching end. At the navigation endpoint (i.e., the lesion location) marked on the three-dimensional medical image, the nearest bronchial tree terminal (leaf node) is found. The navigation path is then traced back along the bronchial tree structure towards the main trachea, and the planned path is marked on the bronchial tree model.

[0058] In a preferred embodiment, for each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image. This includes: for each of the first viewpoint and N second viewpoints, registering the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the deformation field corresponding to the first viewpoint and the N second viewpoints, wherein the cone-beam CT projection image is a fixed image and the simulated projection image is a floating image; based on the deformation field corresponding to the first viewpoint and the N second viewpoints, and the position information of the interventional end in the cone-beam CT projection images under the first viewpoint and the N second viewpoints, the position information of the interventional end in the simulated projection images under the first viewpoint and the N second viewpoints is obtained; based on the position information of the interventional end in the simulated projection images under the first viewpoint and the N second viewpoints, the current three-dimensional positioning information of the interventional end in the three-dimensional medical image is obtained.

[0059] In this embodiment, by registering cone-beam CT projection images and simulated projection images from different viewpoints, the deformation fields of the cone-beam CT projection images from different viewpoints are converted to the corresponding simulated projection images. Combined with the positional information of the interventional end in the cone-beam CT projection images from different viewpoints, the positional information of the interventional end in the simulated projection images from different viewpoints can be calculated separately. This allows for the calculation of the three-dimensional positioning information of the interventional end, providing precise navigation support for the surgeon. Through the mapping of positional information, it can be ensured that the position of the interventional end in the simulated projection image is consistent with its position in the cone-beam CT projection image, eliminating differences between different images. The fusion of multi-view information can reduce the errors that may exist in single-view positioning, improve the stability and reliability of positioning, and help improve the accuracy of surgical navigation.

[0060] Furthermore, based on the deformation fields corresponding to the first viewpoint and N second viewpoints, and the position information of the interventional end in the cone-beam CT projection images under the first viewpoint and N second viewpoints, the position information of the interventional end in the simulated projection images under the first viewpoint and N second viewpoints is obtained, including: using image recognition technology to identify and obtain the position information of the interventional end in the cone-beam CT projection images under the first viewpoint and N second viewpoints; based on the deformation fields corresponding to the first viewpoint and N second viewpoints, mapping the position information of the interventional end in the cone-beam CT projection images under the first viewpoint and N second viewpoints to the corresponding simulated projection images, to obtain the position information of the interventional end in the simulated projection images under the first viewpoint and N second viewpoints.

[0061] Furthermore, based on the position information of the interventional end in the simulated projection images under the first viewpoint and N second viewpoints, the current three-dimensional positioning information of the interventional end in the three-dimensional medical image is obtained, including: based on the position information of the interventional end in the simulated projection images under the first viewpoint and N second viewpoints, and the positional relationship between the first viewpoint plane and the N second viewpoint planes in three-dimensional space, the current three-dimensional positioning information of the interventional end in the three-dimensional medical image is obtained.

[0062] In a preferred embodiment, for each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the deformation field corresponding to the first viewpoint and the N second viewpoints. This includes: for each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain feature point pairs corresponding to the first viewpoint and the N second viewpoints; based on the feature point pairs corresponding to the first viewpoint and the N second viewpoints, the affine transformation matrix corresponding to the first viewpoint and the N second viewpoints is calculated; and based on the affine transformation matrix corresponding to the first viewpoint and the N second viewpoints, the deformation field corresponding to the first viewpoint and the N second viewpoints is obtained.

[0063] Preferably, for each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain feature point pairs corresponding to the first viewpoint and the N second viewpoints. This includes: for each of the first viewpoint and N second viewpoints, obtaining feature point descriptors for the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain feature point pairs corresponding to the first viewpoint and the N second viewpoints. The algorithm used for feature point matching can be Scale Invariant Feature Transform (SIFT) algorithm, Speeded Up Robust Features (SURF) algorithm, etc. Specifically, for each detected feature point, its surrounding image information is extracted to generate a descriptor. The descriptor can uniquely identify the feature point and maintain a certain stability under different viewpoints, illumination, or scale changes. By comparing the descriptors of feature points in the cone-beam CT projection image and the simulated projection image, corresponding feature point pairs are found. Usually, the similarity between descriptors (such as Euclidean distance, cosine similarity, etc.) is calculated to obtain matching feature point pairs.

[0064] In this embodiment, the feature point matching method exhibits robustness to interference factors such as image rotation, scale changes, and noise, accurately locating corresponding feature points under complex conditions. Through precise feature point matching, a precise correspondence can be established between the cone-beam CT projection image and the simulated projection image. Using the coordinate information of these feature point pairs, an affine transformation matrix is ​​calculated using optimization algorithms (such as least squares). The affine transformation matrix describes the geometric transformation relationship from the cone-beam CT projection image to the simulated projection image, including translation, rotation, and scaling. The affine transformation matrix maintains global consistency of the image, providing a correspondence of the overall structure between images, ensuring that the entire image retains its original structure and shape after transformation.

[0065] In one specific embodiment, before obtaining the deformation field corresponding to the first viewpoint and N second viewpoints based on the affine transformation matrices corresponding to the first viewpoint and N second viewpoints, the method further includes: obtaining the local elastic transformation matrix corresponding to the first viewpoint and N second viewpoints based on the positions of the feature point pairs corresponding to the first viewpoint and N second viewpoints and the mutual information values ​​corresponding to the first viewpoint and N second viewpoints.

[0066] Based on the affine transformation matrices corresponding to the first viewpoint and N second viewpoints, the deformation fields corresponding to the first viewpoint and N second viewpoints are obtained, including: based on the affine transformation matrices and local elastic transformation matrices corresponding to the first viewpoint and N second viewpoints, the deformation fields corresponding to the first viewpoint and N second viewpoints are obtained.

[0067] In this embodiment, mutual information is an indicator of the similarity between two images. During registration, the similarity between the cone-beam CT projection image and the simulated projection image can be evaluated by calculating the mutual information values ​​corresponding to the first viewpoint and N second viewpoints. The higher the mutual information value, the better the matching degree between the two images, and the higher the registration quality. In the calculation process, an initial transformation relationship is first established based on the position of feature point pairs. Then, the transformation parameters are adjusted by optimization algorithms (such as least squares method, iterative nearest point algorithm, etc.) to maximize the mutual information value between the transformed image and the reference image. The final local elastic transformation matrix describes the local deformation process from the simulated projection image to the cone-beam CT projection image. Calculating the local elastic transformation matrix by combining the position of feature point pairs and mutual information values ​​can more accurately describe the correspondence between images. Local elastic transformation allows the image to undergo non-rigid deformation in local areas to better adapt to changes in different viewpoints and shapes. By considering the global structural correspondence while further considering the local deformation, the accuracy of registration can be improved.

[0068] In one possible embodiment, for each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain feature point pairs corresponding to the first viewpoint and N second viewpoints. The method further includes: removing erroneous feature point pairs from the feature point pairs corresponding to the first viewpoint and N second viewpoints to obtain removed feature point pairs corresponding to the first viewpoint and N second viewpoints. The algorithm for removing erroneous feature point pairs can be the Random Sample Consensus (RANSAC) algorithm, a removal method based on a distance threshold, a removal method based on a ratio test, etc.

[0069] Based on the feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated, including: based on the feature point pairs corresponding to the first viewpoint and N second viewpoints after culling, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated.

[0070] In this embodiment, by eliminating erroneous matching point pairs, the error in the registration process can be significantly reduced, thereby improving the registration accuracy and making the entire registration process more robust to factors such as image quality and viewpoint changes.

[0071] In one possible embodiment, determining the target path based on the current estimated location includes: selecting a curve segment adjacent to the current estimated location as a first suggested path on the planned path of the bronchial tree model; determining M curve ranges that include the first suggested path and are smaller than the entire planned path as M second suggested paths, where M is a positive integer; and determining the target path in response to a user's selection instruction, where the selection instruction includes one of the first suggested path, the M second suggested paths, or the entire planned path. For example, the first suggested path may be the current branch segment or a curve segment on the current branch segment, and the second suggested path may be the current branch segment or the current branch segment and at least one adjacent branch segment.

[0072] In this embodiment, the interventional device undergoes elastic deformation under stress during the procedure, particularly when passing through tortuous or narrow bronchi. The plane closest to the target path is selected as the first viewing plane, where the elastic deformation of the interventional device is most pronounced, ensuring accurate registration. The physician can choose a first suggested path, a second suggested path, or, if the physician deems the entire planned path more suitable, the entire planned path.

[0073] In this embodiment, N is 1, a dual-view plane is used, and the angle between the second view plane and the first view plane is 90°, as shown below. Figure 3aThe diagram shows a bronchial tree model and a planned path on the bronchial tree model. The bold curve represents the planned path, and the dots represent lesion locations. Figure 3b The diagram shows the target path along a branch of the local bronchial tree. Figure 3c The diagram shown illustrates a local branch of the target path in the bronchial tree model from a first-view perspective. Figure 3d The diagram shows a local branch of the target path in the bronchial tree model in the second view direction, where 301 is the first view plane and 302 is the second view plane.

[0074] In one specific embodiment, the method further includes step S106: mapping the current three-dimensional positioning information onto a three-dimensional medical image and a bronchial tree model for navigation reference. In this embodiment, by mapping the current three-dimensional positioning information onto a three-dimensional medical image and a bronchial tree model, doctors can intuitively observe the current position of the interventional device on the three-dimensional medical image and the bronchial tree model, thereby helping doctors to make more accurate location judgments and navigation.

[0075] In one possible embodiment, the method further includes step S107: finding the nearest point of the current three-dimensional positioning information on the bronchial tree point cloud of the bronchial tree model, and displaying the nearest point on the bronchial tree model to determine whether the interventional end is currently on the planned path of the bronchial tree model. In this embodiment, the nearest point of the current three-dimensional positioning information is determined on the bronchial tree point cloud and displayed on the bronchial tree model. If the nearest point is located on the planned path, the interventional end is currently on the planned path; if the nearest point is not located on the planned path, the interventional end is currently not on the planned path. By displaying the nearest point and the current three-dimensional positioning information point on the bronchial tree model, doctors can quickly determine whether the interventional end is on the planned path, thereby adjusting the operation strategy in a timely manner, reducing unnecessary adjustments and attempts, and improving surgical efficiency.

[0076] In one possible embodiment, the method further includes step S108: providing navigation guidance instructions based on the mapping position information of the current three-dimensional positioning information on the three-dimensional medical image and the bronchial tree model, the position information of the nearest point on the bronchial tree model, and the planned path. In this embodiment, by comprehensively considering the current three-dimensional positioning information, the position of the nearest point, and the planned path, more accurate and reliable navigation guidance instructions can be provided to the doctor, which helps to reduce operational errors and ensures that the interventional device is accurately guided to the target location.

[0077] This invention proposes a method for intraoperative bronchial navigation based on cone-beam computed tomography (CBCT). This method involves acquiring multi-view CBCT projection images of the bronchus using a cone-beam computed tomography (CBCT) device, obtaining the current three-dimensional positioning information of the interventional end using image registration methods, and mapping this current three-dimensional positioning information onto a three-dimensional medical image and a bronchial tree model to achieve more accurate navigation. Specifically, this invention includes the following technical effects:

[0078] 1. Compared with the traditional single-view 2D-X-ray image registration scheme, the multi-view scheme of the present invention can obtain richer spatial information, especially accurate depth information, which makes the three-dimensional positioning of the intervention end more accurate, reduces positioning error, and improves the precision of the operation.

[0079] 2. Compared with the intraoperative cone-beam CT full sampling reconstruction of 3D images, the present invention avoids a long imaging process and high dose of radiation exposure, thus reducing the radiation risk to patients.

[0080] 3. Selecting the plane closest to the target path as the first-view plane can most intuitively reflect the elastic deformation caused by the stress of the interventional device, improve the registration accuracy, and thus obtain the three-dimensional positioning of the interventional end more accurately.

[0081] In addition, this invention also proposes a bronchial intraoperative navigation device based on cone-beam CT, see [link to relevant documentation]. Figure 4 The device includes: a path determination unit 401, used to estimate the current estimated position of the interventional device's insertion tip in the bronchial tree model based on bronchoscope images and the bronchial tree model during cone-beam CT-assisted localization, and determine the target path based on the current estimated position; a viewing angle determination unit 402, used to determine a first viewing angle plane and N second viewing angle planes based on the target path, wherein the first viewing angle plane is the plane closest to the target path, and the N second viewing angle planes each have different angles with the first viewing angle plane and the angles are not 0, where N is a positive integer; and an image acquisition unit 403, used to acquire images of the bronchial tubes based on the first viewing angle plane and the N second viewing angle planes. The method includes cone-beam CT projection images under a first viewpoint and N second viewpoints; image projection unit 404 is used to project the three-dimensional medical image of the bronchus before surgery according to the forward projection angles corresponding to the first viewpoint plane and the N second viewpoint planes, respectively, to obtain simulated projection images under the first viewpoint and the N second viewpoints. The simulated projection images and cone-beam CT projection images correspond one-to-one with each other under the corresponding viewpoints; image registration unit 405 is used to register the cone-beam CT projection image and the simulated projection image under the same viewpoint for each of the first viewpoint and the N second viewpoints, to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image. All relevant content of each step involved in the above method embodiment can be referred to in the functional description of the corresponding functional module, and will not be repeated here.

[0082] In other embodiments of the present invention, an electronic device is disclosed, see [link to relevant documentation]. Figure 5 The electronic device may include: one or more processors 501; a memory 502; a display 503; one or more application programs (not shown); and one or more computer programs 504. These devices may be connected via one or more communication buses 505. The one or more computer programs 504 are stored in the memory 502 and configured to be executed by the one or more processors 501. The one or more computer programs 504 include instructions that can be used to perform actions such as… Figure 1 and Figure 4 And the various steps in the corresponding embodiments.

[0083] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] In the various embodiments of this invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0086] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as set forth in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways. Unless otherwise defined, the technical or scientific terms used herein should be understood in their ordinary sense by one of ordinary skill in the art to which this invention pertains. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word encompasses the element or object listed following the word and its equivalents, but do not exclude other elements or objects.

Claims

1. A bronchial intraoperative navigation device based on cone-beam CT, which can perform a bronchial intraoperative navigation method, characterized in that, The intraoperative navigation methods for bronchial procedures include: S101: When performing cone-beam CT-assisted localization, based on the bronchoscopic images and the bronchial tree model, the current estimated position of the interventional device's interventional end in the bronchial tree model is estimated, and the target path is determined based on the current estimated position. S102: Determine a first view plane and N second view planes based on the target path. The first view plane is the plane closest to the target path. The N second view planes have different angles with the first view plane, and the angles are not 0. N is a positive integer. S103: Based on the first perspective plane and N second perspective planes, acquire cone-beam CT projection images of the bronchus under the first perspective and N second perspectives respectively; S104: Based on the forward projection angles corresponding to the first view plane and N second view planes, perform forward projection on the three-dimensional medical images of the bronchus before surgery to obtain simulated projection images under the first view and N second view. S105: For each of the first viewpoint and N second viewpoints, register the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image.

2. The intraoperative bronchial navigation device according to claim 1, characterized in that, For each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image, including: For each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the deformation field corresponding to the first viewpoint and N second viewpoints. The cone-beam CT projection image is a fixed image, and the simulated projection image is a floating image. Based on the deformation fields corresponding to the first view and N second view, and the position information of the intervention end in the cone-beam CT projection images under the first view and N second view, the position information of the intervention end in the simulated projection images under the first view and N second view is obtained. Based on the position information of the interventional end in the simulated projection images under the first perspective and N second perspectives, the current three-dimensional positioning information of the interventional end in the three-dimensional medical image is obtained.

3. The intraoperative bronchial navigation device according to claim 2, characterized in that, For each of the first viewpoint and N second viewpoints, the cone-beam CT projection image and the simulated projection image under the same viewpoint are registered to obtain the deformation field corresponding to the first viewpoint and N second viewpoints, including: For each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the feature point pairs corresponding to the first viewpoint and N second viewpoints; Based on the feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated. Based on the affine transformation matrices corresponding to the first perspective and N second perspectives, the deformation fields corresponding to the first perspective and N second perspectives are obtained.

4. The intraoperative bronchial navigation device according to claim 2, characterized in that, Before obtaining the deformation fields corresponding to the first and N second perspectives based on the affine transformation matrices of the first and N second perspectives, the process also includes: Based on the positions of feature point pairs corresponding to the first viewpoint and N second viewpoints, and the mutual information values ​​corresponding to the first viewpoint and N second viewpoints, the local elastic transformation matrices corresponding to the first viewpoint and N second viewpoints are obtained. Based on the affine transformation matrices corresponding to the first viewpoint and N second viewpoints, the deformation fields corresponding to the first viewpoint and N second viewpoints are obtained, including: Based on the affine transformation matrices and local elastic transformation matrices corresponding to the first perspective and N second perspectives, the deformation fields corresponding to the first perspective and N second perspectives are obtained.

5. The intraoperative navigation device for bronchial surgery according to claim 4, characterized in that, For each of the first viewpoint and N second viewpoints, feature point matching is performed on the cone-beam CT projection image and the simulated projection image under the same viewpoint to obtain the feature point pairs corresponding to the first viewpoint and N second viewpoints. This also includes: Remove erroneous feature point pairs from the feature point pairs corresponding to the first viewpoint and N second viewpoints to obtain the removed feature point pairs corresponding to the first viewpoint and N second viewpoints; Based on the feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated, including: Based on the culled feature point pairs corresponding to the first viewpoint and N second viewpoints, the affine transformation matrices corresponding to the first viewpoint and N second viewpoints are calculated.

6. The intraoperative navigation device for bronchial surgery according to claim 1, characterized in that, Determining the target path based on the current estimated location includes: On the planned path of the bronchial tree model, a curve segment close to the current estimated position is selected as the first prompting path, and the range of M curves including the first prompting path and smaller than the entire planned path is determined as M second prompting paths, where M is a positive integer; The target path is determined in response to the user's selection instruction, which includes one of the first suggested path, M second suggested paths, or the entire planned path.

7. The intraoperative bronchial navigation device according to claim 1, characterized in that, It also includes step S106: The current 3D positioning information is mapped onto the 3D medical image and the bronchial tree model for navigation reference.

8. The intraoperative bronchial navigation device according to claim 1, characterized in that, It also includes step S107: Find the nearest point of the current three-dimensional positioning information on the bronchial tree point cloud of the bronchial tree model, and display the nearest point on the bronchial tree model to determine whether the intervention end is currently on the planned path of the bronchial tree model.

9. The intraoperative bronchial navigation device according to any one of claims 1-8, characterized in that, The intraoperative bronchial navigation device includes: The path determination unit is used to estimate the current estimated position of the interventional device tip in the bronchial tree model based on the bronchoscope image and the bronchial tree model during cone-beam CT-assisted localization, and to determine the target path based on the current estimated position. The viewpoint determination unit determines a first viewpoint plane and N second viewpoint planes based on the target path. The first viewpoint plane is the plane closest to the target path, and the N second viewpoint planes each have a different angle with the first viewpoint plane, and the angle is not 0. N is a positive integer. The image acquisition unit is used to acquire cone-beam CT projection images of the bronchus under the first view plane and N second view planes, respectively, based on the first view plane and N second view planes. The image projection unit is used to project the three-dimensional medical image of the bronchus before surgery according to the forward projection angles corresponding to the first view plane and N second view planes, respectively, to obtain simulated projection images under the first view and N second view. The simulated projection images correspond one-to-one with the cone-beam CT projection images under the corresponding view. The image registration unit is used to register the cone-beam CT projection image and the simulated projection image under the same viewpoint for each of the first viewpoint and N second viewpoints, so as to obtain the current three-dimensional positioning information of the interventional end in the three-dimensional medical image.

10. An electronic device, characterized in that, include: Processor and memory, wherein the memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory to cause the electronic device to perform the bronchial navigation method performed by the bronchial navigation device according to any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the intrabronchial navigation method performed by the intrabronchial navigation device according to any one of claims 1 to 8.

Images