Navigation path calibration device, system, equipment and medium based on bronchial tree

Abstract

The present invention relates to the technical field of medical image processing, and provides a navigation path calibration device, system, equipment and medium based on a bronchial tree. The device is used to perform the following steps: S2, registering a first bronchial tree with first magnetic data collected by a magnetic probe, and converting the first magnetic data into a three-dimensional image coordinate system using a registration matrix to obtain a coarse registration result; determining the position of the front end of a sheath in the bronchial tree: converting second magnetic data into a three-dimensional image coordinate system to determine the position of the magnetic probe; S3, confirming the ray plane of the remaining path once every preset time period; S4, adjusting a mobile imaging device to a corresponding angle and taking a second image; reconstructing a second bronchial tree based on the second image; registering and generating a third bronchial tree based on the first and second bronchial trees; replacing the first bronchial tree with the third bronchial tree; and S5, repeating S2-S4 to obtain a fine registration result. The device is used to calibrate a navigation path for lung surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a navigation path calibration device, system, equipment and medium based on a bronchial tree. Background Art

[0002] Lung puncture and biopsy are important means of screening for lung cancer and other lesions. Current examination methods include transthoracic lung puncture biopsy, electronic bronchoscope plus biopsy, and navigation bronchoscope plus biopsy. During the biopsy process, a puncture device is required to puncture and sample the lesions in the lungs and the surrounding tissues. In order to improve the success rate of the operation, the doctor will plan the puncture path before the operation. Before the endoscopic examination, the doctor first performs a three-dimensional computed tomography (CT) scan on the patient to obtain the morphological position of the lung, bronchi and vascular tissues, as well as the morphological position of the lung nodules, and plans the surgical path based on the three-dimensional structure of the blood vessels and bronchi and the morphological position of the lesions. Then, during the examination, the doctor operates the endoscope to navigate to the desired location of the lung nodule.

[0003] However, during intraoperative navigation, the bronchi deform in real time as the patient moves and breathes, and as instruments enter the trachea. Doctors lack a true correlation between the position of the 3D CT scan and the actual endoscope tip, effectively performing a blind inspection. This blind inspection relies heavily on the doctor's experience and endoscopic technique, limiting the success rate of the procedure. Therefore, a bronchial tree-based navigation path calibration device, system, equipment, and medium are urgently needed to address these issues. Summary of the Invention

[0004] The object of the present invention is to provide a navigation path calibration device, system, equipment and medium based on the bronchial tree, which is used to calibrate the navigation path of lung surgery.

[0005] In a first aspect, the present invention provides a navigation path calibration device based on a bronchial tree, which is used for updating the bronchial tree and the real-time navigation path in real time, comprising: an acquisition unit, a processing unit and a storage unit; the acquisition unit is used to take a first image containing the bronchi by a mobile imaging device in a preoperative stage; and take a second image containing the bronchi in an intraoperative stage, collect first magnetic data, move the magnetic probe along an ideal path, and collect second magnetic data in real time; the processing unit is used to perform the following steps: S1, obtaining a first image containing the bronchi taken by a mobile imaging device in a preoperative stage, extracting a first bronchial tree based on the first image, the first bronchial tree containing spatial data of the main airway and secondary branches; S2, aligning the first bronchial tree with the first magnetic data acquired by the magnetic probe in an intraoperative stage, converting the first magnetic data into a three-dimensional image coordinate system using a registration matrix to obtain a coarse registration result; determining whether the front end of the sheath is currently in the first bronchus Position in the tree: using a point cloud-based registration method to convert the second magnetic data into the three-dimensional image coordinate system, perform real-time registration, and determine the current position of the magnetic probe; S3, based on the ideal path and the current position of the magnetic probe, confirm the ray plane of the remaining path once every preset time interval, and the ray plane is used to determine whether the magnetic probe has reached the specified position; S4, based on the current ray plane, adjust the mobile imaging device to the corresponding angle and take a second image containing the bronchus; reconstruct the second bronchial tree based on the second image; based on the first bronchial tree and the second bronchial tree, register to generate a third bronchial tree; replace the first bronchial tree with the third bronchial tree; S5, repeat S2-S4 until the magnetic probe reaches the target point to obtain a fine registration result; the storage unit is used to store the third bronchial tree and the real-time navigation path after registration; the acquisition unit and the storage unit are both electrically connected to the processing unit.

[0006] Optionally, when performing coarse registration between the first bronchial tree and the first magnetic data collected by the magnetic probe, the processing unit is further used to: confirm the transformation form of the registration matrix, the transformation form including translation and rotation; collect the data of the second magnetic data after the coarse registration transformation in real time; when executing the point cloud-based registration method to convert the second magnetic data into the three-dimensional image coordinate system, the processing unit is further used to: transform the second magnetic data using the registration matrix, the transformation form including at least one of translation, rotation and shearing, select the mean square error as the metric function, {R o ,t o}satisfy:

[0007]

[0008] Among them, R orepresents the optimal rotation and shear transformation matrix, t o represents the optimal translation transformation matrix, R represents the rotation and shear transformation matrix, t represents the translation transformation matrix, n represents the number of data collected by the magnetic probe, p i represents the i-th point in the data collected by the magnetic probe, q i represents the distance from p on the ideal path i The nearest point.

[0009] Optionally, when executing S4, the processing unit is also used to: segment the second bronchial tree based on deep learning to generate an initial lesion area; reconstruct the third bronchial tree and the reconstructed lesion area; perform three-dimensional point cloud rigid registration on the third bronchial tree and the first bronchial tree, map the third bronchial tree and the initial lesion area to the coordinate system of the first image, and replace the first bronchial tree with the mapped third bronchial tree as the tree to be registered for subsequent magnetic data; and plan and update the ideal path based on the current position of the third bronchial tree, the reconstructed lesion area and the magnetic probe.

[0010] Optionally, when executing S4, the processing unit is further used to: extract surface point sets on the first bronchial tree to obtain a first surface point set; extract surface point sets on the second bronchial tree to obtain a second surface point set; rigidly align the first surface point set and the second surface point set, update the first surface point set after transformation with the second surface point set, and generate the third bronchial tree; perform ideal path planning based on the third bronchial tree, and determine the actual position of the current magnetic probe according to the alignment of the second magnetic data with the third bronchial tree.

[0011] Optionally, the processing unit is further configured to: perform scale and sparsity consistency sampling on the first surface point set and the second surface point set according to the resolution, so that the scale and sparsity of the first surface point set and the second surface point set are consistent; align the sampling points based on the improved iterative closest point algorithm to establish a matching relationship between the first surface point set and the second surface point set. satisfy:

[0012]

[0013] Where i∈{1,2,…,m}, j∈{1,2,…,n}, m is the number of points in the first target point set, and n is the number of points in the second target point set; define the loss function C to satisfy:

[0014]

[0015] Among them, R represents the rotation transformation matrix, t represents the translation transformation matrix, and p j represents the jth point in the second target point set, qi represents the i-th point in the first target point set, α represents the matching relationship penalty parameter; optimize the matching relationship that minimizes the loss function C Rotation and translation parameters as final matching and transformation parameters.

[0016] Optionally, the mobile imaging device is configured as a C-arm machine; when executing S1, the processing unit is used to: take the sagittal plane passing through the midpoint of the x-direction of the first image as the zero-degree plane, set the sampling interval, and set the traversal angle range to -90 to 90 degrees; find the angle with the most pixels and the most angles projected onto the coronal plane by the planned ideal path, as the angle to which the C-arm machine needs to be adjusted.

[0017] In a second aspect, the present invention provides a navigation path calibration system based on the bronchial tree, comprising a device as described in any one of the first aspects, and also comprising a sheath and a guide core; the guide core is arranged on the inner side of the sheath, and the first end of the guide core is provided with a magnetic probe; the second end of the guide core is connected to the processing unit of the device.

[0018] Optionally, when the magnetic probe is set to a 5-degree-of-freedom magnetic sensor, when the magnetic probe is set to a 5-degree-of-freedom magnetic sensor, the number of the 5-degree-of-freedom magnetic sensors is set to at least 2, and the data of at least 2 5-degree-of-freedom magnetic sensors are used to synthesize the data of the 6-degree-of-freedom magnetic sensor; when the magnetic probe is set to a 6-degree-of-freedom magnetic sensor, the number of the 6-degree-of-freedom magnetic sensors is set to at least 1.

[0019] In a third aspect, the present invention provides a medical device comprising a memory and a processor, wherein the memory stores a program for running on the processor, and when the program is executed by the processor, the medical device implements the steps that can be implemented by the device described in any one of the first aspects.

[0020] In a fourth aspect, the present invention provides a readable storage medium, wherein the readable storage medium stores a program, and when the program is executed, the steps that can be implemented by the device described in any one of the first aspects are implemented.

[0021] The present invention has the following beneficial effects: By using a point cloud-based registration method to convert the second magnetic data into the three-dimensional image coordinate system, real-time registration is performed to determine the current position of the magnetic probe. Combined with the angle adjustment of the mobile imaging device, this allows for timely updates of the bronchial tree. This allows for intraoperative updates based on real-time bronchial deformation, providing a dynamic navigation reference for lung surgery and improving the success rate of lung surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A schematic structural diagram of a bronchial tree-based navigation path calibration device provided by the present invention;

[0023] Figure 2 A schematic flow chart of steps executed by a processing unit provided by the present invention;

[0024] Figure 3 A schematic diagram of a three-dimensional virtual model provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of the results of centerline extraction of a three-dimensional virtual model provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a navigation path planned based on the location of a nodule and the bronchial tree, provided in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of the structure of a magnetic probe bound to a sheath provided by an embodiment of the present invention;

[0028] Figure 7 A schematic diagram comparing the updated bronchial tree before and during surgery provided by an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of registering data collected by a magnetic probe with extracted bronchial structures according to an embodiment of the present invention;

[0030] Figure 9 This is a structural schematic diagram of a medical device provided by the present invention. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the present invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0032] In view of the problems existing in the existing technology, such as Figure 2As shown, the first embodiment provides a navigation path calibration device 10 based on the bronchial tree, which is used to update the bronchial tree and the real-time navigation path in real time, including: an acquisition unit 13, a processing unit 11 and a storage unit 12; the acquisition unit 13 is used to take a first image containing the bronchi by a mobile imaging device in the preoperative stage; and take a second image containing the bronchi in the intraoperative stage, collect first magnetic data, move the magnetic probe along the ideal path, and collect second magnetic data in real time; the processing unit 11 is used to perform the following steps: S1, in the preoperative stage, obtain a first image containing the bronchi taken by a mobile imaging device, extract a first bronchial tree based on the first image, and the first bronchial tree contains spatial data of the main airway and secondary branches; S2, in the intraoperative stage, align the first bronchial tree with the first magnetic data acquired by the magnetic probe, and use the alignment matrix to convert the first magnetic data into a three-dimensional image coordinate system to obtain a rough alignment result; determine the current position of the sheath front end At the position of the first bronchial tree: the second magnetic data is converted into the three-dimensional image coordinate system using a point cloud-based registration method, and real-time registration is performed to determine the current position of the magnetic probe; S3, based on the ideal path and the current position of the magnetic probe, the ray plane of the remaining path is confirmed once every preset time interval, and the ray plane is used to determine whether the magnetic probe has reached the specified position; S4, based on the current ray plane, the mobile imaging device is adjusted to the corresponding angle to capture a second image containing the bronchus; based on the second image, a second bronchial tree is reconstructed; based on the first bronchial tree and the second bronchial tree, a third bronchial tree is generated by registration; the first bronchial tree is replaced by the third bronchial tree; S5, S2-S4 are repeated until the magnetic probe reaches the target point to obtain a fine registration result; the storage unit 12 is used to store the third bronchial tree; the acquisition unit 13 and the storage unit 12 are both electrically connected to the processing unit 11.

[0033] Exemplarily, the acquisition unit 13 is configured as a computed tomography camera and a magnetic probe. The computed tomography camera is connected to the end of the C-arm and is used to capture a first image including the bronchi; the magnetic probe is used to collect first magnetic data and second magnetic data. The processing unit 11 is configured as a processor or a processing chip. The storage unit 12 is configured as a memory or a storage chip. The mobile imaging device is configured as a C-arm.

[0034] Specifically, S1 includes: taking the sagittal plane passing through the midpoint of the x-direction of the first image as the zero-degree plane, setting the sampling interval, and setting the traversal angle range to -90 to 90 degrees; finding the angle with the most pixels and the most angles projected onto the coronal plane by the planned ideal path as the angle to which the C-arm machine needs to be adjusted.

[0035] In some other embodiments, before executing S1, the process further includes: S0, performing a preoperative CT assessment based on the subject's lung image, generating an original bronchial tree and path planning, and selecting the optimal navigation path to the lesion as the ideal path. Specifically, the preoperative CT assessment is performed on the quality of the CT image. Exemplarily, a qualified CT image includes the tracheal inlet and the entire lung area, and has a resolution of less than 0.8.

[0036] The method comprises: S01, segmenting the original bronchial tree and reconstructing it in three dimensions to obtain a three-dimensional virtual model; S02, extracting the center line of the three-dimensional virtual model to generate a regenerated bronchial tree, and traversing the regenerated bronchial tree to generate all branches and nodes; S03, interactively identifying nodules and reconstructing the nodule positions; S04, planning the optimal navigation path according to the reconstructed nodule positions and the bronchial morphology in the regenerated bronchial tree.

[0037] Specifically, the original bronchial tree is segmented using deep learning or traditional image processing algorithms, the segmentation result is refined to obtain the bronchial centerline, and the centerline is traversed to generate the three-dimensional virtual model.

[0038] In another specific embodiment, the process of identifying nodules adopts deep learning or traditional image processing algorithms; bronchi and nodules are reconstructed using a moving cube isosurface reconstruction method, and the optimal navigation path is planned based on the generated bronchial tree and nodule position.

[0039] In another specific embodiment, when assisting navigation in bronchial surgery, cone beam computed tomography (CBCT) is used. The 3D virtual model obtained by 3D reconstruction of the bronchus is as follows: Figure 3 As shown in the figure, the results of centerline extraction of the 3D virtual model are as follows: Figure 4 As shown in the figure, the navigation path planned according to the location of the nodule and the regenerated bronchial tree is shown in the figure. Figure 5 shown.

[0040] In another specific embodiment, interactively identifying nodules and reconstructing nodule positions includes manually outlining a layer of nodule edges on the coronal / sagittal / transverse planes, or selecting a point on the nodule and performing three-dimensional nodule recognition based on the outlined or selected point.

[0041] In some embodiments, the secondary branches are configured to not include the two main branches below the main airway; when performing coarse registration between the first bronchial tree and the first magnetic data collected by the magnetic probe, the processing unit 11 is further configured to: confirm the transformation form of the registration matrix, the transformation form including translation and rotation; collect the data of the second magnetic data after the coarse registration transformation in real time; and convert the second magnetic data into the three-dimensional image coordinate system using a point cloud-based registration method, including: transforming the second magnetic data using a registration matrix, the transformation form including at least one of translation, rotation, and shearing, selecting the mean square error as the metric function, {R o ,t o}satisfy:

[0042]

[0043] Among them, R o represents the optimal rotation and shear transformation matrix, t o represents the optimal translation transformation matrix, R represents the rotation and shear transformation matrix, t represents the translation transformation matrix, n represents the number of data collected by the magnetic probe, p i represents the i-th point in the data collected by the magnetic probe, q i represents the distance from p on the ideal path i The nearest point.

[0044] In some possible embodiments, Figure 6 As shown, during S2, the magnetic probe is attached to the front end of the sheath as a guide core. The sheath and auxiliary navigation device are advanced into the main airway and traverse all two-level main branches of the bronchi. Magnetic probe data is collected and coarse registration is performed between the extracted two-level main branches of the bronchi and the magnetic probe data. The process also includes: using mean square error as a metric function for the coarse registration and employing a gradient-based optimization algorithm as an optimization method for the coarse registration. The auxiliary navigation device includes a magnetic probe sensor, an endoscope core, a biopsy needle, and biopsy forceps. In some embodiments, S4 further includes: using the sagittal plane passing through the midpoint of the CT x-axis as the zero-degree plane, traversing the angle from -90 to 90 degrees at one-degree intervals, and finding the angle at which the planned optimal path projects the most pixels onto the coronal plane, i.e., the angle to which the C-arm needs to be adjusted. The C-arm is adjusted to the corresponding angle, a new CT image is captured using the CBCT system, and a new three-dimensional bronchial tree and lesion segmentation is performed based on deep learning to reconstruct a second bronchial tree containing the lesion.

[0045] In some embodiments, S4 includes: segmenting the second bronchial tree based on deep learning to generate an initial lesion area; reconstructing the third bronchial tree and the reconstructed lesion area; performing three-dimensional point cloud rigid registration on the third bronchial tree and the first bronchial tree, mapping the third bronchial tree and the initial lesion area to the coordinate system of the first image, replacing the first bronchial tree with the mapped third bronchial tree as the tree to be registered for subsequent magnetic data; planning and updating the ideal path based on the current position of the third bronchial tree, the reconstructed lesion area and the magnetic probe.

[0046] In some embodiments, S4 also includes: performing surface point set extraction on the first bronchial tree to obtain a first surface point set; performing surface point set extraction on the second bronchial tree to obtain a second surface point set; rigidly aligning the first surface point set and the second surface point set, updating the first surface point set after transformation with the second surface point set, and generating the third bronchial tree; performing ideal path planning based on the third bronchial tree, and determining the actual position of the current magnetic probe according to the alignment of the second magnetic data with the third bronchial tree.

[0047] In some embodiments, the method further includes: performing scale and sparsity consistency sampling on the first surface point set and the second surface point set according to the resolution, so that the scale and sparsity of the first surface point set and the second surface point set are consistent; aligning the sampling points based on the improved iterative closest point algorithm to establish a matching relationship between the first surface point set and the second surface point set. satisfy:

[0048]

[0049] Where i∈{1,2,…,m}, j∈{1,2,…,n}, m is the number of points in the first target point set, and n is the number of points in the first target point set; define the loss function C to satisfy:

[0050]

[0051] Among them, R represents the rotation transformation matrix, t represents the translation transformation matrix, and p j represents the jth point in the second target point set, q i represents the i-th point in the first target point set, α represents the matching relationship penalty parameter; optimize the matching relationship that minimizes the loss function C Rotation and translation parameters as final matching and transformation parameters.

[0052] In a specific embodiment, according to the steps executed by the processing unit 11 in the navigation path calibration device 10 provided by the present invention, a comparison diagram of the first bronchial tree extracted before surgery and the third bronchial tree updated during surgery is shown as follows: Figure 7As shown, the white part R1 is the three-dimensional virtual model obtained by three-dimensional reconstruction, and the gray part R2 is the result of matching the first bronchial tree extracted by CBCT to the CT coordinate system. The result is used to update the bronchial tree used in the last iteration and plan the updated navigation path.

[0053] like Figure 8 As shown, in the steps executed by the processing unit 11 of the navigation path calibration device 10 of the embodiment of the present invention, the data collected by the magnetic probe and the extracted bronchial structure are aligned by first performing a rough alignment, and then performing a real-time alignment based on the rough alignment result, so that the real-time alignment can be performed when the deviation between the data collected by the magnetic probe and the branches of the bronchial structure extracted in real time is small, which is conducive to achieving high-precision real-time navigation and improving navigation accuracy. The present invention converts the second magnetic data into the three-dimensional image coordinate system by adopting a point cloud-based alignment method, performs real-time alignment, and determines the current position of the magnetic probe. Combined with the angle adjustment of the mobile imaging device, the bronchial tree is updated in a timely manner, and the bronchial tree can be updated according to the real-time deformation of the bronchi during the operation, thereby providing a dynamic navigation reference for lung surgery, which is conducive to improving the success rate of lung surgery.

[0054] The second embodiment provides a navigation path calibration system based on the bronchial tree, including the device 10 described in any one of the above embodiments, such as Figure 6 and Figure 1 As shown, it also includes a sheath 502 and a guide core 503; the guide core 503 is arranged on the inner side of the sheath 502, and the first end 51 of the guide core 503 is provided with a magnetic probe 501; the second end 52 of the guide core 503 is connected to the processing unit 11 of the device 10.

[0055] In some embodiments, when the magnetic probe 501 is configured as a five-degree-of-freedom magnetic sensor, the number of magnetic probes 501 is set to at least two, that is, the number of five-degree-of-freedom magnetic sensors is set to at least two, and the data of at least two five-degree-of-freedom magnetic sensors is used to synthesize the data of the six-degree-of-freedom magnetic sensor. Exemplarily, the two magnetic probes 501 are respectively tied to opposite sides of the first end 51 of the guide core 503. When the magnetic probe 501 is configured as a six-degree-of-freedom magnetic sensor, the number of magnetic probes 501 is set to at least one, that is, the number of the six-degree-of-freedom magnetic sensor is set to at least one.

[0056] like Figure 9 As shown, the third embodiment provides a medical device 20, including a memory 22 and a processor 21, wherein the memory 22 stores a program for running on the processor 21, and when the program is executed by the processor 21, the medical device 20 implements any one of the steps in the first aspect.

[0057] In a possible embodiment, the medical device 20 further includes: an output interface 23 for outputting results; a communication interface 24 for communicating and transmitting signals; and an antenna 25 for transmitting or receiving signals.

[0058] It should be noted that the processor 21 in this embodiment can be an image processing chip or an integrated circuit chip capable of processing image signals. During implementation, each step of the above-described method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-described processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device. The methods, steps, and logic block diagrams disclosed in this embodiment can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-described method.

[0059] It is understood that the memory 22 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0060] A fourth embodiment provides a readable storage medium, wherein the readable storage medium stores a program, and when the program is executed, the steps of any one of the first aspects are implemented.

[0061] It is worth noting that if the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a medical device to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memories, random access memories, magnetic disks, optical disks, and other media that can store program code.

[0062] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.

Claims

1. A navigation path calibration device based on a bronchial tree, used for real-time updating of the bronchial tree and the real-time navigation path, characterized in that: include: Acquisition unit, processing unit and storage unit; The acquisition unit is used to capture a first image containing the bronchi by using a mobile imaging device in a preoperative stage; and capturing a second image including the bronchus during an intraoperative stage, acquiring first magnetic data, and moving the magnetic probe along an ideal path to acquire second magnetic data in real time; The processing unit is configured to perform the following steps: S1, acquiring a first image containing bronchi taken by a mobile imaging device in a preoperative stage, and extracting a first bronchial tree based on the first image, wherein the first bronchial tree includes spatial data of main airways and secondary branches; S2, during the intraoperative stage, registering the first bronchial tree with the first magnetic data acquired by the magnetic probe, and converting the first magnetic data into a three-dimensional image coordinate system using a registration matrix to obtain a coarse registration result; Determining the current position of the sheath tip in the first bronchial tree: converting the second magnetic data into the three-dimensional image coordinate system using a point cloud-based registration method, performing real-time registration, and determining the current position of the magnetic probe; S3, based on the ideal path and the current position of the magnetic probe, confirming the ray plane of the remaining path once every preset time interval, the ray plane being used to determine whether the magnetic probe has reached the designated position; S4, adjusting the mobile imaging device to a corresponding angle based on the current ray plane to capture a second image containing the bronchus; and reconstructing a second bronchial tree based on the second image; Based on the first bronchial tree and the second bronchial tree, register and generate a third bronchial tree; replace the first bronchial tree with the third bronchial tree as the tree to be registered with subsequent magnetic data; S5, repeating S2-S4 until the magnetic probe reaches the target point to obtain a fine registration result; The storage unit is used to store the third bronchial tree and the registered real-time navigation path; the acquisition unit and the storage unit are both electrically connected to the processing unit.

2. The device according to claim 1, characterized in that When the processing unit performs coarse registration of the first bronchial tree with the first magnetic data collected by the magnetic probe, the processing unit is further configured to: confirm a transformation form of the registration matrix, the transformation form including translation and rotation; and collect data of the second magnetic data after the coarse registration transformation in real time; When executing the point cloud-based registration method to convert the second magnetic data into the three-dimensional image coordinate system, the processing unit is further configured to: The second magnetic data is transformed using a registration matrix, wherein the transformation form includes at least one of translation, rotation, and shearing, and the mean square error is selected as the metric function. satisfy: Among them, R o represents the optimal rotation and shear transformation matrix, t o represents the optimal translation transformation matrix, R represents the rotation and shear transformation matrix, t represents the translation transformation matrix, n represents the number of data collected by the magnetic probe, p i represents the i-th point in the data collected by the magnetic probe, q i represents the distance from p on the ideal path i The nearest point.

3. The device according to claim 1, characterized in that When executing S4, the processing unit is further configured to: Segmenting the second bronchial tree based on deep learning to generate an initial lesion area; reconstructing the third bronchial tree and reconstructing the lesion area; performing three-dimensional point cloud rigid registration on the third bronchial tree and the first bronchial tree, mapping the third bronchial tree and the initial lesion area to the coordinate system of the first image, and replacing the first bronchial tree with the mapped third bronchial tree as the tree to be subsequently registered with the magnetic data; The ideal path is planned and updated based on the third bronchial tree, the reconstructed lesion area, and the current position of the magnetic probe.

4. The device according to claim 1, characterized in that When executing S4, the processing unit is further configured to: performing surface point set extraction on the first bronchial tree to obtain a first surface point set; performing surface point set extraction on the second bronchial tree to obtain a second surface point set; rigidly registering the first surface point set with the second surface point set, and updating the first surface point set after transformation using the second surface point set to generate the third bronchial tree; An ideal path planning is performed based on the third bronchial tree, and the actual position of the current magnetic probe is determined according to the registration of the second magnetic data with the third bronchial tree.

5. The device according to claim 4, characterized in that The processing unit is further configured to: perform scale and sparsity consistency sampling on the first surface point set and the second surface point set according to the resolution, so that the scale and sparsity of the first surface point set and the second surface point set are consistent; The sampling points are registered based on the improved iterative closest point algorithm to establish the matching relationship between the first surface point set and the second surface point set. satisfy: in, , m is the number of points in the first target point set, n is the number of points in the second target point set; define the loss function C to satisfy: Among them, R represents the rotation transformation matrix, t represents the translation transformation matrix, and p j represents the jth point in the second target point set, q i represents the i-th point in the first target point set, α represents the matching relationship penalty parameter; optimize the matching relationship that minimizes the loss function C Rotation and translation parameters as final matching and transformation parameters.

6. The device according to claim 1, characterized in that The mobile imaging device is configured as a C-arm machine; and the processing unit, when executing S1, is configured to: The sagittal plane passing through the midpoint of the x-direction of the first image is set as the zero-degree plane, the sampling interval is set, and the traversal angle range is set to -90 to 90 degrees; the angle with the most pixels and the maximum angle projected onto the coronal plane of the planned ideal path is found, and this angle is used as the angle to which the C-arm machine needs to be adjusted.

7. A navigation path calibration system based on a bronchial tree, comprising the device according to any one of claims 1 to 6, characterized in that: It also includes a sheath tube and a guide core; the guide core is arranged inside the sheath tube, and a magnetic probe is arranged at the first end of the guide core; the second end of the guide core is connected to the processing unit of the device.

8. The system according to claim 7, characterized in that When the magnetic probe is set as a five-degree-of-freedom magnetic sensor, the number of the five-degree-of-freedom magnetic sensors is set to at least 2, and the data of at least 2 five-degree-of-freedom magnetic sensors are used to synthesize the data of the six-degree-of-freedom magnetic sensor; when the magnetic probe is set as a six-degree-of-freedom magnetic sensor, the number of the six-degree-of-freedom magnetic sensors is set to at least 1.

9. A medical device, characterized in that The medical device comprises a memory and a processor, wherein the memory stores a program for running on the processor. When the program is executed by the processor, the medical device implements the steps that can be implemented by the device according to any one of claims 1 to 6.

10. A readable storage medium having a program stored therein, characterized in that: When the program is executed, the steps that can be implemented by the device according to any one of claims 1 to 6 are implemented.

Images