Methods and systems for preoperative visualization of resection areas in thoracic surgeries using laparoscopic and endoluminal robotic systems

Abstract

Preoperative thoracic surgical systems and methods provide tools for visualizing and planning resection procedures. At least a lung and a lesion are segmented from radiographic images and a 3D model is created therefrom. The 3D model is displayed with interactive tools, which enable a clinician to review the 3D model and plan a resection procedure. The interactive tools may allow a clinician to display different portions of the 3D model, to define a margin of the lesion, to define parameters of a resection procedure and / or to display a virtual resection. Radiographic images obtained on different dates may also be displayed simultaneously.

Description

METHODS AND SYSTEMS FOR PREOPERATIVE VISUALIZATION OF RESECTION AREAS IN THORACIC SURGERIES USING LAPAROSCOPIC AND ENDOLUMINAL ROBOTIC SYSTEMSCROSS REFERENCE TO RELATED APPLCATIONSThis application claims priority under to U.S. Provisional Patent Application No. 63 / 816,218, filed June 2, 2025, and U.S. Provisional Patent Application 63 / 674,048, filed July 22, 2024, the entire contents of each of which are incorporated herein by reference.BACKGROUND

[0001] This disclosure relates to medical imaging and computer-assisted surgical planning. In particular, this disclosure is directed to systems and methods employing neural networks and artificial intelligence (Al) to analyze medical imaging data to detect and visualize anatomical features and abnormalities in the medical images and to assist in laparoscopic and endoscopic surgical planning.

[0002] Surgical planning often involves one or more imaging modalities including magnetic resonance imaging (MRI), positron emissions tomography (PET), computed tomography (CT), cone-beam computed tomography (CBCT), and others. These imaging modalities enable generation of volumetric renderings (e.g., 3D models) that can be analyzed and reviewed by the surgeon.

[0003] Traditional thoracic surgical planning, particularly in the context of lung resections, predominantly relies on the manual interpretation of pre-operative imaging data (e.g., CT, MR, CBCT). The imaging data provides valuable anatomical insights. But traditional rendering software may fail to reveal subtle anatomical variations that are clinically significant.

[0004] Intraoperatively, surgeons may rely on laparoscopic and endoscopic images for realtime visualization. However, these laparoscopic and endoscopic views may be limited by their narrow field of vision and potential for distortion, which complicates interpretation — especially when confronted with unexpected variations such as aberrant pulmonary venous drainage or dense pleural adhesions. In such cases, abnormalities may remain undetected even during the surgery and may increase the complexity of the procedure. Surgeons may be forced to make real-time decisions without the benefit of preoperative preparation forthose variations. The combination of incomplete preoperative data and real-time interpretive challenges may lead to elevated surgical risks and may compromise clinical outcomes.

[0005] This disclosure is directed to addressing these shortcomings of existing imaging methodologies to enhance surgical planning and performance.SUMMARY

[0006] The techniques of this disclosure generally relate to preoperative visualization of resection areas in thoracic surgical procedures involving laparoscopic or robotic systems. The methods and systems of the disclosure leverage imaging and visualization technologies to allow surgeons to plan and visualize the resected area in a comprehensive and interactive manner. By enabling surgeons to assess the surgical area before and after resection, the system aims to fill the gap in preoperative visualization for thoracic surgeries, contributing to improved patient outcomes and enhanced clinical results. The methods and systems also leverage advanced image processing and visualization techniques to enable side-by-side comparisons of patient CT datasets, thereby enhancing surgical decision-making and patient care.

[0007] In one aspect, this disclosure provides a system including a processor and a memory. The memory has stored thereon instructions, which when executed by the processor, causes the processor to: receive radiographic images of a lung and a lesion; segment the lung and the lesion in the radiographic images; define a margin around the lesion; generate a three- dimensional (3D) model based on the segmented lung and lesion, and the defined margin; display the 3D model; receive parameters of a resection procedure; and modify the 3D model to show a virtual resection based on the received parameters.

[0008] Implementations of the system may include one or more of the following features. The virtual resection may be a lobectomy, a segmentectomy, or a wedge.

[0009] The instructions may cause the processor to receive identification of a tool and modify the 3D model based on characteristics of the tool and the virtual resection. The 3D model may be modified to show the effects of the virtual resection on the tissue.

[0010] The instructions may cause the processor to receive instructions to adjust the virtual resection and modify the 3D model based on the instructions to adjust the virtual resection.

[0011] The instructions may cause the processor to display a message requesting a clinician to review and confirm the virtual resection.

[0012] In another aspect, the disclosure provides a method. The method includes receiving radiographic images of a lung and a lesion and segmenting the lung and the lesion in the radiographic images. The method also includes defining a margin around the lesion and generating a three-dimensional (3D) model based on the segmented lung and lesion, and the defined margin. The method also includes displaying the 3D model, receiving parameters of a resection procedure, and modifying the 3D model to show a virtual resection based on the received parameters.

[0013] Implementations of the method may include one or more of the following features. The virtual resection may be a lobectomy, a segmentectomy, or a wedge.

[0014] The method may include receiving identification of a tool and modifying the 3D model based on characteristics of the tool and the virtual resection. The method may include modifying the 3D model to show the effects of the virtual resection on the tissue.

[0015] The method may include receiving instructions to adjust the virtual resection and modifying the 3D model based on the instructions to adjust the virtual resection. The method may include displaying a message requesting a clinician to review and confirm the virtual resection.

[0016] The method may include simultaneously displaying at least two radiographic images captured on different dates. The method may include automatically selecting and displaying the at least two radiographic images based on the segmented lesion or the defined margin. The method may include displaying a user control which, when selected, displays the at least two radiographic images in a separate window.

[0017] The method may include displaying measurement tools enabling a user to measure features within the at least two radiographic images.

[0018] The at least two radiographic images may be displayed side-by-side or adjacent to each other.

[0019] The radiographic images may be tomographic image slices.

[0020] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF DRAWINGS

[0021] FIG. 1A is a schematic view of a robotic surgical system in accordance with the disclosure.

[0022] FIG. IB is a schematic view of a luminal network navigation system in accordance with the disclosure;

[0023] FIG. 2 depicts a deep 3D lung model including airways, blood vessels, and the pleura generated in accordance with the disclosure;

[0024] FIG. 3 depicts a deep 3D lung model including airways, blood vessels, and the pleura as well as any detected abnormalities identified in the patient in accordance with the disclosure;

[0025] FIG. 4 is a flow diagram illustrating the process for identifying and visualizing lung abnormalities using the Al-enabled system in accordance with the disclosure; and

[0026] .

[0027] FIG. 5 is a flow diagram that illustrates a method of visualizing a resection procedure.

[0028] FIG. 6 is a graphical diagram that illustrates a surgical planning user interface displaying a main anatomy screen showing a lung model and CT slices.

[0029] FIG. 7 is a graphical diagram that illustrates a surgical planning user interface of FIG. 6 displaying all generations of airways and blood vessels of the lung model.

[0030] FIG. 8 is a graphical diagram that illustrates the surgical planning user interface of FIG. 6 displaying a portion of the generations of airways and blood vessels of the lung model.

[0031] FIG. 9 is a graphical diagram that illustrates the surgical planning user interface of FIG. 6 displaying a resection planning screen for a lobectomy.

[0032] FIGS. 10 and 11 are graphical diagrams that illustrate the surgical planning user interface of FIG. 6 displaying a resection plan for a lobectomy.

[0033] FIG. 12 is a graphical diagram that illustrates another surgical planning user interface displaying a main anatomy screen showing a lung model and CT slices.

[0034] FIG. 13 is a graphical diagram that illustrates the surgical planning user interface of FIG. 12 displaying a resection planning screen for a lobectomy.

[0035] FIG. 14 is a graphical diagram that illustrates the surgical planning user interface of FIG. 12 displaying a resection plan for a lobectomy.

[0036] FIGS. 15 and 16 are graphical diagrams that illustrate a surgical planning user interface displaying CT comparison screens.

[0037] FIG. 17 is a schematic representation of a computing system in accordance with the disclosure.DETAILED DESCRIPTION

[0038] This disclosure is directed to methods and systems for generation of 3D models from preoperative images and preoperative visualization of resection areas in surgeries including those performed by laparoscopic or endoluminal robotic systems. The methods and systems provide surgeons with preoperative and advanced visualization of features within the areas to be resected. This allows for a comprehensive assessment of the surgical area before and after resection.

[0039] Leveraging various imaging and visualization technologies, the methods and systems of the disclosure offer real-time, interactive visualization capabilities, enhancing the precision and safety of thoracic surgical procedures. These methods and systems aim to optimize surgical planning and decision-making by empowering surgeons to observe and analyze the resection area preoperatively and post-operatively. The methods and systems of the disclosure contribute to improved patient outcomes and enhanced clinical results.

[0040] In accordance with one aspect of the disclosure, pre-procedural, or intra-procedural images (e.g., CT, CBCT, PET, or MR images) are automatically analyzed with an image processing application incorporating an Al model generator or neural network algorithm (referred to generally herein as neural network algorithms) to develop a deep airway tree (e.g., 3D model of the airways). The neural network algorithms enable a pixel -by-pixel segmentation identifying and distinguishing fluid, air, luminal tissue (e.g., the tissue of the airways), bone, and others. A deep airway tree refers to a 3D model of the luminal network (e.g., airways) which approaches the pleura boundary encompassing substantially all of the luminal network. Thus, 3D modeling applications employing neural networks output deeper 3D models resulting in a more complete 3D model (compared to, e.g., an inferior or incomplete 3D model, which may model only the larger airways).

[0041] A further aspect of the disclosure is directed to the use of image processing applications to detect a wide range of anatomical abnormalities and structural patterns within image data. These patterns may include surgical risk indicators, anomalousanatomical variations, and clinically relevant regions requiring heightened attention during resection planning. The neural network algorithms may be specifically trained to identify and distinguish between pulmonary structures such as airways, blood vessels, lobes, pleura, segments, tumors, and fissures. These neural network algorithms are trained using large datasets containing labeled imaging data, encompassing both normal and abnormal anatomical presentations.

[0042] The image processing application performs detailed pixel-by-pixel analysis of preprocedural or intra-procedural images (e.g., CT, CBCT, PET, or MR images) and analyzes Hounsfield unit or equivalent pixel intensity metrics to differentiate between tissues. This differentiation enables the separation of anatomical elements such as airways, vascular structures, lobular divisions, fissures, pleural boundaries, and pathologies like tumors or lesions. Pixels with common or similar values are clustered together and matched against known anatomical templates and spatial configurations. For example, tubular structures with consistent diameters and orientations are interpreted as airways or blood vessels, while denser, irregularly shaped clusters may indicate tumor masses.

[0043] In addition to direct pixel classification, the neural networks incorporate structural recognition logic, comparing detected shapes and textures to expected forms such as the trachea, carina, and branching vascular and blood vessel networks. This layered recognition strategy allows the system to differentiate between healthy tissues and those presenting abnormalities, even in complex or ambiguous regions. Through this process, the application enhances the visibility of subtle or hidden anatomical variations that may otherwise go unnoticed in traditional radiological reviews or be difficult to fully appreciate in laparoscopic and endoscopic imaging, and thereby provide useful information to support clinicians in making more informed surgical decisions.

[0044] A further aspect of the disclosure is directed to advanced surgical planning and visualization systems empowering clinicians with tools for precise resection in laparoscopic and endoluminal robotic surgical procedures. The systems provide a comprehensive platform for physicians to outline lesions and desired margins and accurately mark the resection area. The systems offer a visual representation of the resected area, thereby allowing clinicians to, for example, examine the cut and stapling region while considering the involved anatomical structures and characteristics of the surgical tools utilized.

[0045] Defining the resection margin using the segmented data of the 3D model of the lungs and the lesion may follow oncological guidelines to ensure the complete removal of malignant tissue, while preserving as much healthy lung tissue as possible. Based on current oncological guidelines, the best practices are as follows.

[0046] General guidelines for lung cancer resection margins focus on ensuring the complete removal of cancerous tissue while preserving as much healthy lung tissue as possible. Some points include margin size, resection types, and special considerations.

[0047] Regarding margin size, for non-small cell lung cancer (NSCLC), a margin of at least 2 cm or the size of the tumor may be desirable. If anatomical constraints make this difficult, the maximum feasible margin should be obtained.

[0048] For small cell lung cancer (SCLC), the margins are less clearly defined due to the nature of the disease. Thus, ensuring complete resection with negative margins (no cancer cells at the edge of the tissue) is needed. For adenocarcinoma and squamous cell carcinoma, a resection margin of at least 2 cm around the lesion or a margin equal to the size of the tumor, whichever is greater, is recommended.

[0049] The resection types may include, for example, a wedge resection, segmentectomy, lobectomy, and pneumonectomy. A wedge resection is used for small, peripheral tumors and requires careful margin assessment. A segmentectomy aims for at least 2 cm margins and is suitable for small, early-stage tumors. A lobectomy is standard for most operable NSCLC cases and ensures adequate margins in most cases. A pneumonectomy involves removal of an entire lung. A pneumonectomy is considered when a tumor is centrally located and cannot be resected with sufficient margins by other methods.

[0050] There are also other special considerations including tumor location, vital structure involvement, and preoperative imaging and staging. The tumor location, e.g., a central location versus a peripheral location, affects the approach to a resection. The involvement of vital structures (e.g., major blood vessels, bronchi) may necessitate more extensive resection. Preoperative imaging and staging (e.g., CT, CBCT, PET, or MRI) are needed for planning resection and ensuring adequate margins.

[0051] By following these guidelines and utilizing advanced imaging and 3D modeling tools, the systems of the disclosure help accurately define the resection margin is both oncologically sound and surgically feasible, optimizing patient outcomes, and ensuring effective and safe surgical outcomes. The National Comprehensive Cancer Network(NCCN) Guidelines recommend achieving clear resection margins (RO resection), which means no cancer cells at the margin of the removed tissue. Typically, a margin of at least 2 cm around the tumor or a margin equal to the size of the tumor is recommended, whichever is greater. For early-stage lung cancers, lobectomy with a systematic lymph node dissection is preferred.

[0052] The European Society for Medical Oncology (ESMO) guidelines emphasize the importance of clear margins and recommend lobectomy or pneumonectomy for larger tumors, with segmentectomy or wedge resection being considered for smaller, peripheral tumors when medically necessary. The American Society of Clinical Oncology (ASCO) Guidelines also recommend an RO resection and highlight the importance of multidisciplinary team discussions to determine the surgical plan for the individual patient’s condition. The Journal of Thoracic Oncology includes research articles and reviews on lung cancer surgery can provide detailed insights and updates on best practices. The Society of Thoracic Surgeons (STS) provides guidelines and resources for thoracic surgery, including lung cancer resection. The surgical planning methods of the disclosure are consistent with the recommended guidelines.

[0053] The surgical planning methods of this disclosure may involve loading segmented data into a computer system. This may include importing the 3D model of the lung and the segmented lesion into the surgical planning software application running. The methods may also include defining the resection margin. This may involve calculating a margin and checking anatomical constraints. Calculating the margin may include using software tools to define a resection margin of at least 2 cm around the lesion or a margin equal to the size of the lesion or tumor. This may be done by expanding the segmented lesion by a desired margin. Checking anatomical constraints may include evaluating the expanded margin against the surrounding anatomical structures to ensure the feasibility of the resection.

[0054] The surgical planning methods of this disclosure may also involve visualizing the margin. This may include software tools that generate a 3D visualization of the resection plan, showing the lesion and the proposed resection margin. Visualizing the margin may also include simulating the resection virtually to understand the spatial relationships between tissue and to plan the surgical approach.

[0055] The surgical planning methods of this disclosure may also involve refinement and validation of the resection plan. This may include features enabling interdisciplinary review.For example, the features may enable review of the resection plan by a multidisciplinary team, including thoracic surgeons, oncologists, and radiologists, to validate the resection margins. The surgical planning methods may also enable a clinician to adjust the resection plan based on feedback and further anatomical considerations.

[0056] In aspects, the methods of the disclosure may include segmenting the lung and the lesion in radiographic images using one or more software applications, which may incorporate artificial intelligence and / or machine learning techniques or algorithms. The blood vessels may also be segmented. Next, the methods and systems may provide tools enabling a clinician to create and / or adjust a margin around the lesion. Then, the methods and systems may create a 3D model based on the segmented lung and lesion, and the resection margin. The 3D model may be displayed to allow for visualization of the lung, lesion, and resection margin in 3D. The methods and systems also provide simulation features, which may include performing a virtual resection to check for feasibility and to adjust a resection plan, as necessary. The methods and systems also provide resection plan reviewing features, which include display screens and / or tools to enable a clinician to review and analyze the resection plan and present it to a multidisciplinary team for validation and final adjustments.

[0057] FIG. 1 A depicts a robotic surgical system 10 including a control tower 20, which is connected to all of the components of the robotic surgical system 10 including a surgeon console 30A and one or more mobile carts 60. Each of the mobile carts 60 includes a robotic arm 40 having a surgical instrument 50 removably coupled thereto. The robotic arms 40 also couple to the mobile carts 60. The robotic surgical system 10 may include any number of mobile carts 60 and / or robotic arms 40.

[0058] The surgical instrument 50 is configured for use during minimally invasive surgical procedures (e.g., laparoscopic) or for catheter-based intraluminal diagnostic, therapeutic, and surgical procedures (described in greater detail in connection with FIG. IB). The surgical instrument 50 may include an end effector 49 such as, for example, an electrosurgical forceps configured to seal tissue by compressing tissue between jaw members and applying electrosurgical current thereto, a surgical stapler including a pair of jaws configured to grasp and clamp tissue while deploying a plurality of tissue fasteners, e.g., staples, and cutting stapled tissue, a surgical clip applier including a pair of jaws configured apply a surgical clip onto tissue or other end effectors without departing fromthe scope of the disclosure. Alternatively, the surgical instrument 50 may include a robotic actuated catheter including an articulation mechanism (e.g., one or more pull-wires, tubes, or tendons) to alter the shape of the catheter. In addition, the robotic arm 40 may be employed to advance or retract the catheter during a laparoscopic or endoluminal (or combined) procedure.

[0059] In accordance with various embodiments of the present disclosure, one of the robotic arms 40 may include a laparoscopic camera 51 configured to capture video of the surgical site. The laparoscopic camera 51 may be, for example, a stereoscopic endoscope configured to capture two side-by-side (i.e., left and right) images of the surgical site to produce a video stream of the surgical scene. The laparoscopic camera 51 is coupled to an image processing device, which may be disposed, for example, within the control tower 20. The image processing device may be any computing device configured to receive the image feed from the laparoscopic camera 51 and output the processed images or video stream.

[0060] The surgeon console 30A includes a first screen 32, which displays a video feed of the surgical site provided by camera 51 of the surgical instrument 50 disposed on the robotic arm 40, and a second screen 34, which displays a user interface for controlling the robotic surgical system 10. The first screen 32 and second screen 34 may be touchscreens allowing for displaying various graphical user inputs.

[0061] The surgeon console 30A also includes a plurality of user interface devices, such as, for example, foot pedals 36 and a pair of hand controllers 38a and 38b which a user may use to remotely control robotic arms 40 and / or surgical instruments 50. The surgeon console 30A further includes an armrest 33 used to support clinician’ s arms while operating the hand controllers 38a and 38b. As an alternative, or in addition to surgeon console 30A, as depicted in FIG. IB, the robotic arms 40, and other elements of the system, including the surgical instrument 50 may be controlled via surgeon console 30B, including in one example, via a handheld controller 38c, which may be connected to the console 30B in a wired or wireless manner.

[0062] The control tower 20 includes a screen 23, which may be a touchscreen, and outputs various graphical user interfaces (GUIs). In accordance with various aspects of the disclosure, the control tower 20 also acts as an interface between the surgeon console 30 and one or more robotic arms 40. In particular, the control tower 20 is configured to control the robotic arms 40, such as to move the robotic arms 40 and the corresponding surgicalinstrument 50, based on a set of programmable instructions and / or input commands from the surgeon console 30, in such a way that robotic arms 40 and the surgical instrument 50 execute a desired movement sequence in response to input from the surgeon console 30 A. For example, in accordance with one embodiment, surgeon console 30A may include the foot pedals 36 and the hand controllers 38a and 38b. The foot pedals 36 may be used to enable and lock the hand controllers 38a and 38b, repositioning camera movement and surgical instrument activation / deactivation. In particular, the foot pedals 36 may be used to perform a clutching action on the hand controllers 38a and 38b. Clutching may be initiated by pressing one of the foot pedals 36, which disconnects (i.e., prevents movement inputs) the hand controllers 38a and / or 38b from the robotic arm 40 and corresponding surgical instrument 50 or camera 51 attached thereto. This allows the user to reposition the hand controllers 38a and 38b without moving the robotic arm(s) 40 and the instrument 50 and / or camera 51. This is useful when reaching control boundaries of the surgical space. In accordance with another embodiment of the present disclosure, the surgeon console 30A or 30B may communicate with and control one or more robotic arms 40 and surgical instruments 50 directly, without using the control tower 20 as an interface.

[0063] Each of the control tower 20, the surgeon console 30, and the robotic arm 40 includes a respective computer 21, 31, 41. The computers 21, 31, 41 are interconnected to each other using any suitable communication network based on wired or wireless communication protocols. The term “network,” whether plural or singular, as used herein, denotes a data network, including, but not limited to, the Internet, Intranet, a wide area network, or a local area network, and without limitation as to the full scope of the definition of communication networks as encompassed by the present disclosure. Suitable protocols include, but are not limited to, transmission control protocol / internet protocol (TCP / IP), datagram protocol / internet protocol (UDP / IP), and / or datagram congestion control protocol (DC). Wireless communication may be achieved via one or more wireless configurations, such as, e.g., radio frequency, optical, Wi-Fi, Bluetooth (an open wireless protocol for exchanging data over short distances, using short length radio waves, from fixed and mobile devices, creating personal area networks (PANs), ZigBee® (a specification for a suite of high level communication protocols using small, low-power digital radios based on the IEEE 122.15.4- 1203 standard for wireless personal area networks (WPANs)).

[0064] The computers 21, 31, 41 may include any suitable processor (not shown) operably connected to a memory (not shown), which may include one or more of volatile, nonvolatile, magnetic, optical, or electrical media, such as read-only memory (ROM), random access memory (RAM), electrically-erasable programmable ROM (EEPROM), non-volatile RAM (NVRAM), or flash memory. The processor may be any suitable processor (e.g., control circuit) adapted to perform the operations, calculations, and / or set of instructions described in the present disclosure including, but not limited to, a hardware processor, a field programmable gate array (FPGA), a digital signal processor (DSP), a central processing unit (CPU), a microprocessor, and combinations thereof. Those skilled in the art will appreciate that the processor may be substituted with any logic processor (e.g., control circuit) adapted to execute algorithms, calculations, and / or set of instructions described herein.

[0065] FIG. IB is a perspective view of an exemplary system for facilitating robotic endoluminal navigation of a medical device, e.g., a catheter, to a soft-tissue target via airways of the lungs. System 100 may be further configured to construct fluoroscopic based three-dimensional volumetric data of the target area from 2D fluoroscopic images to confirm navigation to a desired location. Other intraprocedural imaging modalities may also be employed including CBCT, ultrasound, laparoscopic cameras, and others. System 100 may be further configured to facilitate the approach of a medical device to the target area by using, for example, Electromagnetic Navigation (EMN) and for determining the location of a medical device with respect to the target. The EMN system may employ a variety of sensor technologies including without limitation air-coil sensors, tunnel magnetoresistance (TMR), and others without departing from the scope of the disclosure. Though described in connection with EMN, other systems for intraluminal and lung navigation are considered within the scope of the disclosure including shape sensing technology (e.g., Fiber-bragg gratings) which detect the shape of the distal portion of the catheter and match that shape to the shape of the luminal network in a 3D model.

[0066] With respect to both FIGS. 1 A and IB each of the various components of the system may be connected via either a wired or wireless connection, or combinations of both, without departing from the scope of the disclosure. In this manner the computers 21, 31, 41 or console 30B can control the robotic arms 40 and the surgical instruments 50 to perform the desired procedure whether endoluminal, laparoscopic, or a combination of both.

[0067] In FIG. IB, the surgical instrument 50 includes a catheter 102. The catheter 102 is inserted into the luminal network of the patient P (e.g., via the nose or mouth). The catheter 102 includes one or more sensors 104 used for determining a position and orientation of distal portion of the catheter 102 within the patient. In one example, this position and orientation may be determined with reference to a coordinate system defined by an electromagnetic (EM) field generated by a magnetic field generator 106. The magnetic field generator 106 is a board-shaped device including two or more magnetic field generating antennae that is typically placed beneath the patient. In one embodiment the catheter 102 includes imaging capabilities. The catheter 102 may also be configured to receive a locatable guide (LG), camera, biopsy, or therapy tool (not specifically shown). In one example, the locatable guide is a second catheter that may include a sensor or a camera and may be inserted into catheter 102 and locked into position. System 100 generally includes an operating table 108 configured to support a patient P for insertion of the catheter 102 through patient P’s mouth and into patient P’s airways. The camera on either the catheter 102 or the locatable guide may be operably connected to a display 110 on the console 30B to display live video and images in one or more user interfaces 112. Also connected to the console 30B are the magnetic field generator 106 and a plurality of reference sensors 114. The magnetic field generator 106 may include a plurality of fiducial markers that are observed in fluoroscopic or cone beam CT images of the patient. The console 30B includes a computing device 116 including software and / or hardware used to facilitate identification of a target, pathway planning to the target, navigation of a medical device to the target, and / or confirmation and / or determination of placement of catheter 102, or a suitable device therethrough, relative to the target. In addition, the computing device generates signals causing the magnetic field generator 106 to generate magnetic fields as well as other functions.

[0068] In accordance with aspects of the disclosure, the visualization of intra-body navigation of a medical device (e.g., a catheter, LG, biopsy, or therapy tool), towards a tumor or lesion, may be a portion of a larger workflow of a navigation system. An imaging device 118 capable of acquiring images or video of the patient P is also included in this particular aspect of system 100. The images, sequence of images, or video captured by imaging device 118 may be stored within imaging device 118 or transmitted to computing device 116 for storage, processing, and display. Additionally, imaging device 118 may moverelative to the patient P so that images may be acquired from different angles or perspectives relative to patient P to create a sequence of images, such as a fluoroscopic video. The pose of imaging device 118 relative to patient P while capturing the images may be estimated via the fiducial markers incorporated within the magnetic field generator 106. The markers are positioned under patient P, between patient P and operating table 108 and between patient P and a radiation source or a sensing unit of imaging device 118. The markers incorporated with the magnetic field generator 106 may be two separate elements which may be coupled in a fixed manner or alternatively may be manufactured as a single unit. Imaging device 118 may include a single imaging device or more than one imaging device. The imaging device 118 may be for example, a fluoroscopic imaging, an ultrasound imaging device, an intraprocedural CBCT imaging device, or an intraprocedural PET imaging device.

[0069] Computing device 116 may be any suitable computing device including a processor and storage medium, wherein the processor is capable of executing instructions stored on the storage medium. Computing device 116 may further include a database configured to store patient data, image data sets including CT images, CBCT images, fluoroscopic images and video, fluoroscopic 3D reconstruction, navigation plans, and other such image data. Although not explicitly illustrated, computing device 116 may include inputs, or may otherwise be configured to receive, image data sets and other data described herein. Additionally, computing device 122 includes a display configured to display images, 3D models, and other data in one or more graphical user interfaces. Computing device 116 may be connected to one or more networks through which one or more databases (e.g., image databases) may be accessed.

[0070] For use in a navigation phase, a suitable system for determining position and orientation of a distal portion of the catheter 102 (e.g., magnetic field generator 106 and software on the computing device 116), is utilized for performing registration of the images and the pathway for navigation with the patient’s luminal network. In one aspect of the disclosure, magnetic field generator 106 is positioned beneath patient P. The magnetic field generator 106 generates an electromagnetic field around at least a portion of the patient P within which the position of the plurality of reference sensors 114 and the sensor 104 can be determined by an application running on the computing device 116. Registration is generally performed to coordinate locations of the three-dimensional model and two-dimensional images, with the patient P’s airways as observed in the images capturedby the camera (either on the catheter 102 or locatable guide) and allow for the navigation phase to be undertaken with knowledge of the location of the sensor 104 within the body of the patient.

[0071] As depicted in FIG. IB, the catheter 102 may be navigated using functionality from the robotic arm 40 and mobile cart 60 as well as functionality in the surgical instrument 50 (e.g., rotation, advancement, and articulation). As noted above, the robotic arms 40, surgical instrument 50, and catheter 102 may be controlled via surgeon console 30B, including in one example, via a handheld controller 38c, which may be connected to the console 30B in a wired or wireless manner. Alternatively, the surgeon console 30B may include controls (e.g., track ball, buttons, etc.) to control the robotic arms 40, surgical instrument 50, and catheter 102. Still further, mobile cart 60 may include a computing device and display presenting the user interfaces described herein, as a result all of the functionality of the console 30B described above, including receiving input from handheld controller 38c may be incorporated directly on the mobile cart 60 such that a separate console 30B may not be required.

[0072] Still further, as noted above, the catheter 102 may include an endoluminal camera. The endoluminal camera captures images of the endoluminal pathway as the catheter 102 is advanced towards a target. The endoluminal camera may be a permanent feature of the catheter 102, or a removable camera (e.g., on the locatable guide) that is advanced into a working channel of the catheter 102. One or more fiber-optic filaments or light pipes may be employed to carry light to the end of the catheter 102 from a light source that remains outside of the patient. An active pixel sensor may be located on a distal portion of the catheter (e.g., a CMOS or CCD image sensor) for the formation of live images from captured light reflected by the patient’s tissues. Alternatively, both the light source and image forming components may also be positioned at a distal portion of the catheter 102, or both may remain outside of the patient with the fiber-optic filaments carrying light both from the light source and to the image forming components.

[0073] Registration of the patient P’s location on the magnetic field generator 106 may be performed by moving sensor 104 through the airways of the patient P. More specifically, data pertaining to locations of sensor 104 and the reference sensors 114 in the magnetic field generated by the magnetic field generator 106 is recorded using an application stored in the memory of the computing device 116. A shape resulting from this location data is comparedto an interior geometry of passages of a 3D model generated during a planning phase of the procedure (e.g., from pre-procedure CT images), and a location correlation between the shape and the 3D model based on the comparison is determined, e.g., utilizing the software on computing device 116. In addition, the software identifies non-tissue space (e.g., air filled cavities) in the three-dimensional model. The software aligns, or registers, an image representing a location of sensor 104 with the 3D model and / or two-dimensional images generated from the 3D model, which are based on the recorded location data and an assumption that catheter 102 remains located in non-tissue space in patient P’s airways. Alternatively, a manual registration technique may be employed by navigating the catheter 102 with the sensor 104 to pre-specified locations in the lungs of the patient P and manually correlating the detected locations (and / or images from the camera) to the 3D model.

[0074] Though described herein with respect to EMN systems using EM sensors, including, e.g., TMR sensors, the instant disclosure is not so limited and may be used in conjunction with flexible sensor, ultrasonic sensors, or without sensors. Fiber-bragg gratings may also be used. Additionally, the methods described herein may be used in conjunction with manual systems including, for example, a bronchoscope configured to receive the catheter 102 and manually inserted and navigated within the patient to a location near the target.

[0075] One aspect of the system 100 is a software component stored or accessible from (e.g., cloud) a computing device 116 and configured for processing computed image data. Though many aspects of image data processing have previously been performed at least partially manually, aspects of this disclosure are directed to automated image processing techniques and systems (e.g., using neural network algorithms). These aspects of the disclosure are described in greater detail below in connection with target identification and pathway planning. Some of the aspects described herein, particularly features displayed on a user interface, may be presented during the planning phase or during the navigation phase of a procedure where a medical device, such as a biopsy tool or treatment tool, may be inserted into catheter 102 to obtain a tissue sample from or to treat the target.

[0076] In accordance with the disclosure, a 3D model of a luminal network (e.g., the patient’s lungs) or another suitable portion of the anatomy, may be generated from previously acquired scans, such as CT, CBCT, PET or MRI scans. Tumors and lesions within the scan data are detected and pathways through the 3D model, to arrive at the tumors or lesions, are generated. The review of the previously acquired scans and the generationof the pathway plan or surgical plan may be performed on a separate (not shown) computing device hours or days prior to the procedure or immediately before or even during the procedure using, for example, the functionality of console 30B or mobile cart 60.

[0077] Once the pathway plan is generated and accepted by a clinician, that pathway plan may be utilized by a navigation system to drive a catheter or catheter-like device along the pathway plan through the anatomy and particularly the luminal network (e.g., airways) to reach the tumor or lesion. The driving of the catheter along the pathway plan may be manual or it may be robotic, or a combination of both. In a single procedure planning, registration of the pathway plan to the patient, and navigation are performed to enable a medical device, e.g., catheter 102 to be navigated along the planned path to reach the target or lesion, so that a biopsy or treatment of the target can be completed.

[0078] As described above, in connection with aspects of the disclosure, neural network algorithms are employed by computing device 116 to automatically analyze image data (e.g., CT, CBCT, PET, or MRI image data). The image data may be pre-procedure image data (e.g., acquired days or weeks before an endoluminal navigation procedure) or image data that is acquired intra-procedurally (e.g., 3D fluoroscopic or CBCT image data). One or more neural network algorithms analyze the image data for multiple purposes. In accordance with one aspect of the disclosure, a first purpose of the neural network algorithm is to identify and segment the pleura of a patient’s lungs in the image data. As will be appreciated, the location of the pleura is a procedurally significant portion of the patient’s anatomy. Generally, it is desirable to avoid piercing of the pleura as piercing of the pleura can lead to air (and blood) entering chest cavity and can lead to pneumothorax or other complications. Further, in instances where piercing of the pleura is unavoidable, prior knowledge of the pleura’s location and the probability of its being pierced during the procedure allows the clinician to plan for and be ready to mitigate the effects of the piercing the pleura.

[0079] As noted above, neural network algorithms are configured to analyze the image data on a pixel -by-pixel basis. This may be based on a variety of factors including the density of the tissue in each pixel of the image data. The harder or denser a tissue is, the brighter pixels of that tissue appear in an image. For example, bone tissue which is very dense will appear brighter (e.g., white) in the image data, having a higher Hounsfield value, than any other tissue. In contrast, air which has little density typically appears very dark (e.g., black)in image data. Soft tissues have varying levels of density and thus express varying levels of brightness ranging from dense cartilaginous tissue (e.g., trachea and central airways) to less dense and elastic tissues (e.g., the alveoli) and are depicted in the images in a range of brightnesses. Using segmenting and thresholding techniques the neural network algorithms can define the pixels of the pleura as the outermost portions of the lung tissue and separate them from the pixels that make up the muscles and bones of the rib cage.

[0080] A second purpose of the neural network algorithms is to identify tumors or lesions within the image data. As with the pleura boundary, segmentation techniques are employed to define a grouping of pixels that define the tumor or lesion. The tumor or lesion will be denser than the healthy soft tissue of the lungs and will have a shape not associated with harder tissues of the lungs. Harder tissues within the lungs are generally a result of cartilaginous tissue associated with the airways themselves. By distinguishing the grouping of pixels of tissue both on density and shape, false positive detections of a tumor or lesion can be avoided.

[0081] A third purpose of the neural network algorithms is to generate 3D models of the airways and in some instances blood vessels of the lungs. Airways and blood vessels have a density that is relatively uniform, or at least within a range. The neural network algorithms can detect these tissues and group like density tissues to form shapes that are generally tubular in nature. The tubular shapes, which are generally connected to a common source (e.g., trachea, pulmonary artery, pulmonary vein), by grouping the pixels together through multiple slices of the image data and in multiple direction (axial, coronal, sagittal) a 3D model of the airways and blood vessels can be generated. As noted above, through the use of neural network algorithms and their ability to detect minute differences in brightness of pixels, 3D models can be generated which reach deep into the airways and approach the pleura of the lungs.

[0082] A fourth purpose of the neural network algorithms is to identify pathways through the 3D model to the target so that a catheter 102 can be navigated through the airways to arrive at the target (e.g., a tumor or lesion). This pathway starts at the target and is defined in the direction of the trachea. In one example, the target center (center of the tumor) is used as the starting point of the pathway. The neural network seeks the closest airways in the 3D model to the center of the target. Proximity of an airway is not the only factor for consideration. Among the factors the neural network considers when determining apathway are an angle at which a tool (e.g., biopsy tool) exits the luminal network to reach the target. As will be appreciated, it is difficult if not impossible for most endoluminal tools to exit an airway at angles approaching a right angle to the longitudinal direction of the airway. Most endoluminal tools are straight and somewhat rigid on their distal end portions, thus incapable of being maneuvered to make such sharp turns to exit the narrow airways of the patient. Accordingly, the determination of the nearest airway must be made in conjunction with an angle of exit of the catheter 102 or tool to reach the center of the tumor. Another factor which is part of the pathway generation is the radii of the airways. For example, determining whether the airway closest to the target is large enough to receive the catheter 102. If not, an airway further away, for example, may be a better choice for navigation of the catheter, or another location at which the catheter will have to leave the airway to reach the target may be identified, one where the angle to the center of the target allows for substantially straight movement of the biopsy or therapy tool. In some instances, multiple pathways may be generated by the neural network algorithm and a suggested pathway identified for presentation to the user. Additionally, the neural network algorithm may analyze a potential pathway as it proceeds from the target back to the trachea to ensure that the airways being traversed have an increasing radius along the pathway. Other factors considered including tissue densities, mechanical properties of biopsy tools and therapeutic tools, and others may also be factors in the pathway generation.

[0083] In accordance with one aspect of the disclosure, an application on computing device 116 may present to a user, via a user interface, a number of image data sets associated with different patients, from which a selection can be made. The user selects an image data set for a patient requiring generation of a pathway plan. Once selected via the user interface, the application launches set of algorithms including neural network algorithms to perform the aspects described above including segmentation of the pleura boundaries, identification of tumors or lesions (targets) within the image data set, generation of 3D models of the airways and blood vessels, and generation of pathways from the trachea to the tumor or lesion. Following these steps, the 3D model 202 is displayed in the user interface 204 as shown in FIG. 2. The user interface 204 may be one of the user interfaces 112 presented on the display 110. The 3D model 202 includes the airway 206 tree, the blood vessels 208, the pleura boundary 210, and the tumors or lesions 212. Accordingly, the 3D model 202 provides a substantially more accurate model of the anatomy of the patient because theneural network algorithms are capable of generating 3D models of the airways much deeper than prior systems. Further the relative positions of the airways 206, blood vessels 208, pleura boundary 210, and the tumors or lesions 212 are much more accurate than prior 3D models employed in luminal navigation, and particularly lung navigation.

[0084] FIG. 2 shows an example of a deep 3D lung model 102 generated in accordance with the disclosure. The deep 3D lung model 202 serves as a foundational element of the system's interactive visualization platform and provides the anatomical modeling fidelity required for advanced thoracic surgical planning. In one aspect of the disclosure, the 3D lung model 202 is constructed from contrast-enhanced pre-procedural or intra-procedural CT scan data, which is processed by an Al or neural network-based segmentation algorithm. The segmentation algorithm performs high-precision analysis to identify and delineate anatomical structures, including the trachea, primary and secondary bronchi, pulmonary arteries and veins, lobular boundaries, interlobar fissures, and the pleura. Each structure is segmented as an individual data layer and may be visually integrated into a composite 3D rendering of the patient-specific anatomy.

[0085] The 3D model 202, however, does more than visualize isolated structures — it represents their spatial relationships, vascular patterns, and proximities to pathological regions (e.g., tumors or lesions). The 3D model 202 provides a unified view of both normal and aberrant structures, offering contextual depth that aids pre-surgical assessment and planning. The user interface 204 includes functionality to enhance interpretability, allowing surgeons to manipulate the model (e.g., rotate, zoom, isolate elements) and view the 3D model 202 from any anatomical perspective. The 3D model’s integration into the user interface empowers clinicians to determine surgical access, assess resection feasibility, and preemptively identify risks. This not only enhances visual fidelity of the 3D model 202 but also transforms the 3D model 202 into a clinically actionable tool for procedural planning.

[0086] As depicted in FIG. 2, the 3D model 202 is substantially complete and represents a typical patient-specific anatomical rendering without notable anatomical variations or abnormalities, apart from the presence of tumors or lesions 212. The 3D model 202 is integrated into a user interface 204, which allows for detailed manipulation and exploration of anatomical structures. The 3D model 202 includes several key elements labeled for clarity and reference including the airways 206, the pulmonary blood vessels 208 (differentiating arteries from veins), the pleura boundary 210, and any identified tumors or lesions 212. Thiscomprehensive anatomical framework thus provides clinicians with an understanding of the patient’s lung structure and may serve as the reference for detecting deviations in more complex cases.

[0087] FIG. 3 depicts a 3D model 202 of a patient having a variety of abnormalities identified by the image processing application (including the neural network algorithms) accordance with the disclosure. These abnormalities are visually presented in the user interface 204 and annotated directly within the 3D model 202 for clarity. In FIG. 3 the application has identified and labeled an adhesion of the pleura to the wall of the thoracic cavity — an anatomical irregularity that may complicate surgical access or resection margins. Additionally, the application has detected areas of potential emphysema located in the upper lobes of both lungs, indicated by regions of decreased tissue density and abnormal air retention. Furthermore, the system has highlighted an area of reduced vascularization in the lower lobe of the right lung, which may pose risks for oxygenation or healing following resection. Notably, the application has also flagged the absence of an expected airway in one region of the lung model. This could result from a congenital anomaly — where the patient never developed airways in that location — or from a previous surgical resection, potentially performed to remove previously diseased tissue. These annotations equip the surgeon with a detailed preoperative view of the patient’s unique anatomy, enhancing risk assessment, decision-making, and procedural planning. Other abnormalities including a potential blockage of an airway caused by a tumor 112 is also identified in the 3D model 202.

[0088] Regarding the tumors 212, the application identifies and labels the fissures of the left lung within the 3D model 202. These fissures delineate the boundaries between pulmonary segments, which serve as functionally and anatomically distinct regions of the lung. In thoracic surgery, particularly in resection planning, it is typically desirable to avoid crossing these segmental boundaries, as doing so may increase the likelihood of postoperative complications, such as pneumothorax or the creation of non-viable tissue segments with impaired airflow or blood supply. Recognizing and respecting segmental boundaries ensures better preservation of healthy tissue and supports optimal respiratory function after surgery. However, these segments can be difficult to visualize during laparoscopic procedures due to the limited field of view and absence of consistent visual markers. By mapping these segments within the 3D model and correlating them with thelocation of tumor 212, the application provides the surgeon with a detailed and spatially accurate roadmap. This enhanced visibility allows for more precise planning of incision paths and resection margins, contributing to safer procedures and improved patient outcomes.

[0089] The detected abnormalities may be superimposed onto the 3D model 202 as depicted in FIG. 3 using distinct visual markers, such as color-coded highlights, contours, shading variations, or iconographic symbols, enabling clinicians to quickly differentiate between anomaly types at a glance. Each type of abnormality — emphysema, vascular anomalies, missing airways, or pleural adhesions — may be represented with a unique visual cue, creating an intuitive and organized visual layout within the model. These visual annotations are embedded within the spatial structure of the 3D model 202, providing dynamic interactivity that allows users to isolate specific abnormalities, cross-reference them with anatomical landmarks, and understand their proximity to other critical lung features. The integration of this visual information within the 3D model 202 delivers highly contextualized insights, offering the surgeon an enhanced understanding of the lung’s condition. Consequently, these detailed representations facilitate more informed surgical planning, allow for the development of resection strategies, and reduce the likelihood of intraoperative surprises by equipping the clinician with a patient-specific map of challenges and potential risks.

[0090] FIG. 4 is a flow diagram illustrating a method 300 for identifying and visualizing lung abnormalities using the neural network or Al-enabled image processing application in accordance with the disclosure. At step 302, the method begins with the acquisition of a patient’s pre-procedural or intra-procedural images. The images may be from modalities such as high-resolution, contrast-enhanced CT image data sets, but also MRI, or CBCT image datasets. These images are accessed by the application and undergo preprocessing to normalize resolution, denoise, and format standardization enabling subsequent analysis.

[0091] As described in greater detail below, the application includes Al models or neural networks that are trained using a comprehensive dataset of annotated 3D CT images, including segmentation maps and abnormality labels. Training strategies for the Al or neural network algorithms may involve supervised learning on labeled cases, semi -supervised learning using pseudo-labeling, transfer learning from pre-trained models, and active learning with iterative feedback from thoracic surgery experts. Further, continuousimprovement of the Al or neural network algorithm ensures robust and clinically relevant 3D model generation and abnormality detection.

[0092] At step 304, the application performs segmentation on the scan data using a 3D convolutional neural network (CNN), such as U-Net or 3DVNet, to identify and label lung structures. These structures include airways, blood vessels (with distinction between arteries and veins), lobes, pleura, lung segments, tumors, and fissures. The segmentation process involves analyzing the volumetric image data to assign anatomical labels to each voxel based on its features and intensity, enabling spatial differentiation of structures with high anatomical fidelity. Each anatomical structure is encoded as a discrete component within the overall volumetric model, and the segmentation maps serve as foundational inputs for subsequent processing stages. The segmentation includes the detection of abnormalities by enabling focused analysis within specifically segmented regions, thereby improving the model’s ability to identify pathological deviations such as obstructed airways, anomalous vasculature, or abnormal tissue densities. The fine-grained, structure-specific segmentation ensures the system’s clinical precision and supports accurate, contextualized decisionmaking throughout the surgical planning process.

[0093] At step 306, the segmented scan data is used to generate a detailed, patient-specific 3D model 202 of the lungs, capturing all relevant structures in high resolution. The 3D model 202 reflects the true spatial configuration of the anatomy within the patient at the time of imaging and serves as the foundation for further analysis.

[0094] At step 308, the application employs deep learning algorithms to analyze the 3D model for specific abnormalities and structural patterns. This analysis includes identifying vascular anomalies, airway obstructions, absent or malformed anatomical features, and tissue pathologies such as emphysema or fibrosis. The abnormalities are then highlighted directly within the 3D model 202, as exemplified in FIG. 3. The segmented output may be processed by an object detection model, such as, for example, YOLOv8 or Faster R-CNN, or an integrated multitask model, to detect the abnormalities. In some instances, these abnormalities are predefined anatomical abnormalities which the neural network algorithms or Al are to search for. Alternatively, the application may identify any abnormality detected in the 3D model.

[0095] At step 310, the fully annotated 3D model — including segmented structures and identified abnormalities — is rendered and presented within an interactive user interface 204.The user interface 204 provides the surgeon with intuitive controls to rotate the model, zoom, and pan, as well as selectively hide or isolate individual layers (e.g., arterial, venous, pleura, or tumor layers). This interactivity enables the user to focus on clinically significant regions, such as a tumor 212 or a specific lung segment where intervention is planned.

[0096] Additionally, the highlighting of abnormalities may be accompanied by a textual explanation of each finding, detailing its anatomical location, type, size, shape, proximity to critical structures, and potential clinical implications. These explanations are integrated into the user interface 204 enabling the surgeon to interact directly with the abnormality markers. By clicking or hovering over a highlighted region in the 3D model 202, the user can retrieve an expanded information panel that may include diagnostic reasoning, statistical likelihood of pathology, and recommended surgical considerations. These contextual insights are part of a real-time clinical decision support system designed not only to assist with preoperative planning but also to provide intraoperative reference guidance. The application adapts its output based on anatomical variances and model training, further personalizing the output to individual patient anatomy. In scenarios where multiple abnormalities are identified, the system prioritizes alerts based on severity.

[0097] In accordance with the disclosure, the highlighting of abnormalities within the 3D model may be accompanied by a detailed and interactive explanation of each finding’s anatomical characteristics and clinical implications. By interacting with the user interface — using tools such as a mouse, touchscreen, or keyboard — the surgeon can select a highlighted abnormality to reveal a structured overlay of supplementary information. This information may include the precise location of the anomaly, its type (e.g., vascular, airway, pleura), spatial relationships to adjacent structures, and potential risks it may pose during resection. Additionally, the interface may present clinically informed alternatives in surgical approach or strategy, including different access paths or resection margins tailored to avoid or address the abnormality.

[0098] The application may also cross-reference the identified abnormalities with known laparoscopic visual cues and highlight anatomical landmarks that are likely to be visible intraoperatively. This supports real-time orientation during surgery, helping surgeons to correlate the 3D model with the live operative field. Furthermore, the system may propose additional procedural considerations or optional steps — such as reinforcing resection margins, adding auxiliary imaging, or modifying tool selection, each optional step includingcontextual data and clinical rationale. The surgeon can then accept or decline these suggestions within the interface. This level of detail transforms the 3D visualization into an actionable planning tool that anticipates intraoperative conditions and augments decisionmaking based on patient-specific anatomy and procedural context.

[0099] Tools integrated within the user interface 204 allow the surgeon to graphically define or trace proposed resection lines on the 3D model. These tools enable the selection and virtual simulation of instruments to be used during the procedure, such as staplers, vessel sealers, ligation tools, extractors, and other surgical instruments. As the surgeon defines the resection parameters, the application dynamically analyzes the resection plan in conjunction with the identified anatomy and proposed tools. Based on this analysis, the system may generate recommendations for additional procedural steps required to complete the resection safely — such as extending resection margins, reinforcing boundaries, addressing anatomical constraints, or changing tools for aspects of the procedure — and provide justifications for each suggested step. The application, via the user interface 204, may also suggest any supplemental tools or materials that may be required.

[0100] This information, including the resection lines, chosen instruments, recommended steps, and rationale, is compiled and stored as a detailed resection plan in memory. The finalized resection plan serves as a comprehensive guide and reference for the surgical team. It may be accessed preoperatively to ensure surgical staff have the correct tools and materials on hand, organized and prepared in advance within the surgical suite. The resection plan may also be linked to other systems for scheduling, resource allocation, and surgical checklist confirmation to further enhance procedural readiness and patient safety.

[0101] The resection plan may be accessed by one or more computing devices available in the surgical suite and reviewed by the surgeon as part of the preoperative workflow. This review may include a detailed walkthrough of each procedural step, with particular attention paid to abnormalities previously identified by the application, their anatomical context, and their potential influence on the surgical approach. During the actual procedure, the surgical plan remains accessible and can be actively referred to by the surgeon or other members of the surgical team. The plan may be reviewed before performing a subsequent step to ensure alignment with the intended surgical pathway. This real-time reference helps verify that key anatomical features and abnormalities are accounted for, reducing the likelihood of intraoperative surprises and ensuring adherence to the planned resection strategy.Integration of this guidance system within the surgical suite facilitates consistent execution of the procedure while enhancing identification and observance of surgical landmarks, resection boundaries, and potential complications.

[0102] As described herein, the application analyzes pre-procedural or intra-procedural imaging scans (e.g., CT, MRI, CBCT, etc.) to construct detailed and anatomically accurate 3D models of the relevant patient anatomy. While this disclosure focuses on thoracic surgery — particularly the detection and visualization of lung abnormalities in preparation for resection procedures — the described systems and methods are not limited to thoracic applications. Rather, the technology is broadly applicable across a wide range of surgical domains where accurate anatomical modeling and abnormality identification can enhance procedural planning, intraoperative guidance, and postoperative assessment.

[0103] For example, in abdominal surgeries involving the stomach, intestines (both large and small), gall bladder, spleen, liver, pancreas, and kidneys, 3D models can assist in delineating vascular structures, identifying lesions or cysts, evaluating organ boundaries, and planning incisions or resections. Similarly, in other thoracic procedures involving the heart, esophagus, or mediastinum, detailed 3D reconstructions can support identification of congenital defects, arterial anomalies, or tissue pathologies.

[0104] Thus, the methodology described herein is extensible to any clinical context in which personalized 3D modeling and abnormality detection offer value, supporting a broader transformation in precision surgical planning and anatomical navigation across surgical specialties.

[0105] Returning to the context of lung modeling, the image processing application (e.g., a trained Al or neural network application) can be configured specifically to detect and characterize a wide variety of anatomical abnormalities. For instance, the system can identify vascular anomalies such as irregular branching, unusual pulmonary venous drainage, and malformations or stenoses of arteries and veins, all of which may alter blood flow or surgical accessibility. Airway deformities — including bronchial stenosis, obstructions, collapses, or anatomical deviations — can also be recognized by the application, providing critical insight into airway patency and ventilation planning.

[0106] Beyond the vascular and airway analyses, the application is capable of detecting lobar and segmental variations, including anatomical anomalies in lung subdivision patterns or structural changes resulting from previous resections. Such variations may affectsegmental resection decisions or complicate access to target tissue. The application also identifies pleural abnormalities, such as pleural thickening, adhesions, or effusions, which could interfere with dissection planes or increase the risk of postoperative complications like pneumothorax.

[0107] Further capabilities of the application include detection of pulmonary diseases, such as emphysema, pulmonary fibrosis, and infectious processes. These conditions may influence surgical margins or contraindicate certain types of resections. The application may flag these findings and provide contextualized insights, such as recommending additional imaging or biopsy prior to resection.

[0108] As noted above, while this disclosure focuses on thoracic applications — particularly lung resections — the disclosure is not so limited. When applied to other anatomical regions, such as the abdomen, pelvis, or cardiac structures, the system can be adapted to detect organspecific abnormalities (e.g., hepatic cysts, renal artery stenosis, bowel wall thickening) and highlight their procedural relevance. Thus, the model's abnormality-detection engine is broadly adaptable, allowing it to support surgeons across a spectrum of specialties with patient-specific anatomical and pathological challenges.

[0109] Key features that may be called out by the application in identifying abnormalities include the spatial relationship of each abnormality to adjacent and functionally significant structures — such as resection lines, critical blood vessels, and the pleural boundary — helping to assess the potential clinical impact on the surgical approach. In addition, the application can measure and analyze the dimensions (e.g., length, width, volume), geometric shape (e.g., spherical, lobulated, irregular), and growth patterns (e.g., infiltrative versus expansive) of each abnormality to characterize its potential behavior and surgical risk. These parameters may be further correlated with anatomical variations that differ from expected norms, such as shifted lobe boundaries or atypical airway branching. This set of features offers clinicians a high-resolution, context-aware understanding of each anomaly’s clinical relevance and supports precise planning of resection boundaries, tool deployment, and procedural strategy.

[0110] Yet a further aspect of the disclosure is directed to the method 400 described in connection with FIG. 5. Resection planning method 400 enables clinicians to preoperatively visualize a resection area for laparoscopic and endoscopic robotic thoracic surgeries. At step 402, preoperative images of one or more critical structures and a lesion are received. Theimages may include tomographic images such as CT or CBCT images. At step 404, the one or more critical structures and the lesion are automatically segmented from the received radiographic images.[OHl] The segmentation may utilize deep-learning models to identify and segment critical structures, which may include lesions, airways, blood vessels (arteries and / or veins), pleura, lobes, and / or segments. The deep learning models may include neural networks such as convolutional neural networks. The deep learning model may train on appropriate radiographic imaging data sets, e.g., CT data sets, which may be annotated by suitable clinicians, e.g., medical students in the final stages of their education. The deep learning model may then be tested on dedicated test data sets, and model performance may be compared to human medical professionals. The segmentation by the deep learning model may happen in real-time as a clinician, e.g., a physician, is loading the radiographic imaging data into a system application.

[0112] At step 406, a margin is defined around the margin, and, at step 408, a three- dimensional (3D) model 202 is generated based on the segmented critical structures and lesion, and the defined margin. Step 308 may be implemented by a software application that uses data output from the deep learning model and the segmented critical structures (lesions, airways, blood vessels, pleura, lobes, and / or segments) to create several file types in order to visualize the anatomy correctly (e.g., mesh files, scale graphs, dist. maps). Those files create the 3D objects that form the 3D model and are presented to the clinician, e.g., physician. The 3D objects may be an accurate representation of the patient’s anatomy.

[0113] At step 310, the 3D model may be displayed. The software application may allow the clinician to manipulate the 3D model. For example, the anatomy presented may be rotated, zoomed, removed, and / or set to a particular level of transparency to enable the clinician, e.g., a physician, to examine the anatomy in a way the clinician could not do before.

[0114] FIG. 6 illustrates an example of a surgical planning user interface 204 displaying a main anatomy screen showing a 3D model 202 and CT slices 220. The critical structures of the 3D model may be colored with different colors to enable the clinician to easily distinguish between critical structures. As shown in FIG. 6, the lung lobes are colored with different colors. The main screen includes a “Views & Overlays” section 222, which provides a legend indicating the color of the critical structures and slide bar controls. Theslide bar control for the lesion enables the clinician to adjust the size of the margin associated with the lesion. The other slide bar controls enable the clinician to limit the number of generations of the airways and blood vessels that are displayed. For example, as illustrated in FIG. 6, the slide bar controls are set to display a portion of the generations of the arteries and veins in the displayed 3D lung model.

[0115] This disclosure provides layered visualization. The 3D objects are presented in a layered manner, with each layer connected to the others by a shared coordinate system. Accordingly, an action taken on an outer layer, e.g., a pleura or a lobe, affects an inner layer, e.g., a segment or a critical structure. As shown in FIG. 6, for example, the surgical planning interface includes the following “Layer” buttons: “Lobe,” “Seg.” (segment), and “Custom.” The “Layer” buttons enable the clinician to select different layers of the 3D model to highlight different 3D objects of the 3D model. For example, in the main screen of FIG. 4, the “Lobe” layer is selected, while in the screen of FIG. 5, the “Lobe” layer is unselected.

[0116] After critical structures are segmented and displayed via a 3D model 202, the clinician may begin resection planning. Since the layers of the 3D objects are connected by a shared coordinate system, a resection can be planned in several ways. The resection can be marked on the pleura, lobe, or segment to remove the critical structures connected to them. Another way is to completely remove a defined part, such as a lobe or segment, with the possibility of removing the other critical structures connected to them. Yet another way is to examine the airways and blood vessels leading to the lesion or tumor and mark on them the section that the user wants to remove (as an example, mark the second closest bifurcation of an airway to the lesion), and the application will recommend a segment or a lobe to remove.

[0117] As shown in the example main screen of FIG. 6, the surgical planning interface 204 includes the button “Create Plan,” which, upon selection, starts a planning software application for creating a resection plan. At step 412, parameters of a resection procedure are received. The planning application may display the resection planning screen of FIG. 9 for a lobectomy. As shown in FIG. 9, a message may be displayed prompting the user to select the lobe to be resected. The message may include buttons labeled “RUL” for right upper lobe, “RML” for right middle lobe, “RLL” for right lower lobe, “LUL” for left upper lobe, and “LLL” for left lower lobe. The parameters may include the tool to be used in the resection procedure. The systems may enable the clinician to choose between several toolsto perform the resection. Each tool may have different visualizations and characteristics that are taken into account while visualizing the resection, the resected area, and the resection’s effects on the tissue.

[0118] Then, before ending at step 416, the 3D model is modified to show a virtual resection based on the received parameters, at step 414. For example, when one of the lobe buttons is selected, the 3D model 202 may display the corresponding lobe as being virtually resected. FIG. 10 illustrates a resection plan screen that is displayed when the “RUL” button is selected. The right upper lobe is shown removed from the 3D model 202 and parameters for the resection procedure are displayed, e.g., three staples. The 3D model 202 may then be rotated or otherwise manipulated to review and confirm the virtual resection as illustrated in FIG.119. FIGS. 12-14 illustrate other examples of resection planning screens in which a lobe is virtually resected by selecting the lobe in the 3D model 202.

[0119] After performing a resection (e.g., a lobectomy, a segmentectomy, a wedge), the 3D model 202 may be corrected to represent the anatomy after the resection and the stapling of the tissue. This may be done by considering the procedure performed, the stapling line position, the tissue characteristics, and the tool characteristics.

[0120] The methods of this disclosure also provide for preoperative comparative visualization of patient radiographic data, e.g., CT data, for supporting decision-making in thoracic surgical procedures. For example, the methods enable a clinician to visually compare different instances of a patient’s radiographic scans, e.g., CT scans. The methods and systems provide an approach for the comparative visualization of patient CT data to assist surgeons in evaluating and comparing different instances of patient radiographic scans, e.g., CT scans, preoperatively and intraoperatively, highlighting changes over time to support better diagnostic decisions.

[0121] The systems utilize advanced image processing and visualization techniques to provide side-by-side comparisons of multiple patient radiographic imaging datasets, enabling the identification of anatomical changes, pathology progression, and treatment response overtime. This approach enhances surgical decision-making by offering clinicians, e.g., surgeons, a comprehensive and intuitive tool for evaluating and comparing patientspecific radiographic imaging data (e.g., CT or CBCT imaging data), thereby contributing to improved clinical outcomes and patient care.

[0122] The methods of visualizing the radiographic scans involve a dual scan or a multiscan comparison. The comparison may include loading and displaying two or more instances of a patient’s radiographic scan data side-by-side, adjacent to each other, or otherwise near each other to enable visualization of the differences between the radiographic scan data over time.

[0123] The preoperative visualization methods may include automatic lesion visualization. The methods involve automatically opening the relevant tomographic image slice, e.g., CT slice, which shows the lesion. The lesion’s location, size, and relevant tomographic image slice may be predetermined from earlier data. The lesion may be highlighted in one or more colors for easy identification within the tomographic image slice.

[0124] The preoperative visualization methods may also provide measurement tools to aid in visualization. The measurement tools may enable a clinician to measure various objects in the tomographic scan. For example, the measurement tools may enable a clinician to measure the lesion’s size and the lesion’s distance from other anatomical structures.

[0125] The preoperative visualization methods may also analyze changes and display results of that analysis. The analysis results may be displayed in a way that the clinician can easily see and analyze changes in the lesion over time. The analysis may include taking measurements to track the lesion’s growth or changes in its position.

[0126] The visualization methods of this disclosure provide improved diagnostics as the clinician can clearly see how the lesion has changed over time, thereby supporting more accurate diagnoses. The user interface is easy to use as it is a user-friendly interface designed for medical professionals. The visualization methods also provide precision by providing accurate measurement tools help in a detailed analysis of the radiographic image data. This intuitive flow ensures that medical professionals can efficiently and effectively use the CT Scan Viewer screen to monitor patient progress and make informed decisions.

[0127] FIGS. 15 and 16 show example screens that display two CT scans taken one year apart. The planning application can automatically recognize the lesion on each CT scan, identify the relevant slice for demonstration to the clinician, e.g., a physician, and measure the lesion. Additionally, the planning application gives the clinician access to manual measurement tools to assess further parameters (e.g., lesion size or distances between the lesion and other critical structures) based on clinical or anatomical circumstances.

[0128] Various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the techniques). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a medical device.

[0129] In one or more examples, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0130] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0131] Reference is now made to FIG. 17, which is a schematic diagram of a system 500 configured for use with the features described hereinabove. System 500 may include a workstation 501, and optionally an imaging device 515 (e.g., a CT, MRI, CBCT imager). In some embodiments, workstation 501 may be coupled with imaging device 515, directly or indirectly, e.g., by wireless communication. Workstation 501 may include a memory 502, a processor 504, a display 506 and an input device 510. Processor 504 may include one or more hardware processors. Workstation 501 may optionally include an output module 512 and a network interface 508. Memory 502 may store an application 518 and image data 514.Application 518 may include instructions executable by processor 504 for executing the methods of the disclosure (e.g., the image processing neural network algorithms described herein).

[0132] Application 518 may further include a user interface 516. Image data 514 may include the MRI, CT, or CBCT images, 3D models, and the like. Processor 504 may be coupled with memory 502, display 506, input device 510, output module 512, network interface 508 and imaging device 515. Workstation 501 may be a stationary computing device, such as a personal computer, or a portable computing device such as a tablet computer. Workstation 501 may include a plurality of computer devices. Those of skill in the art will recognize that the methods and systems described herein may be incorporated and executed on any computing device capable of receiving the image data of the luminal network.

[0133] Memory 502 may include any non-transitory computer-readable storage media for storing data and / or software including instructions that are executable by processor 504 and which control the operation of workstation 501 and, in some embodiments, may also control the operation of imaging device 515. Memory 502 may include one or more storage devices such as solid-state storage devices, e.g., flash memory chips. Alternatively, or in addition to the one or more solid-state storage devices, memory 502 may include one or more mass storage devices connected to the processor 504 through a mass storage controller (not shown) and a communications bus (not shown).

[0134] Although the description of computer-readable media contained herein refers to solid-state storage, it should be appreciated by those skilled in the art that computer-readable storage media can be any available media that can be accessed by the processor 504. That is, computer readable storage media may include non-transitory, volatile, and non-volatile, removable, and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media may include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, Blu-Ray or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium (e.g., cloud based storage) which may be used to store the desired information, and which may be accessed by workstation 501.

[0135] Application 518 may, when executed by processor 504, cause display 506 to present user interface 516. User interface 516 may be configured to present to the user with the various UI 204 described herein and others without departing from the scope of the disclosure.

[0136] Network interface 508 may be configured to connect to a network such as a local area network (LAN) consisting of a wired network and / or a wireless network, a wide area network (WAN), a wireless mobile network, a Bluetooth network, and / or the Internet. Network interface 408 may be used to connect between workstation 501 and imaging device 515. Network interface 508 may also be used to receive image data 514. Input device 510 may be any device by which a user may interact with workstation 501, such as, for example, a mouse, keyboard, foot pedal, touch screen, and / or voice interface. Output module 512 may include any connectivity port or bus, such as, for example, parallel ports, serial ports, universal serial busses (USB), or any other similar connectivity port known to those skilled in the art. From the foregoing and with reference to the various figures, those skilled in the art will appreciate that certain modifications can be made to the disclosure without departing from the scope of the disclosure.EXAMPLES

[0137] Example 1 - A system for detecting and visualizing anatomical abnormalities, including a computing device including at least one processor and memory, the memory storing therein instructions that when executed, cause the computing device to receive a patient image scan dataset, segment the image scan dataset to identify and label anatomical structures and generate a three-dimensional (3D) model, analyze the 3D model to identify abnormalities, and display in a user interface the 3D model of the anatomical structures and the identified abnormalities, wherein the abnormalities include annotated alerts.

[0138] Example 2 - The system of example 1, wherein the detected abnormalities include one or more of an aberrant vascular branching pattern, stenosed airways, obstructed airways, pleural adhesions, pleural thickening, fibrosis, emphysema, or infectious.

[0139] Example 3 - The system of examples 1 or 2, wherein the segmentation of the image scan dataset to identify anatomical structures and the detection of abnormalities are performed by an artificial intelligence or a neural network algorithm.

[0140] Example 4 - The system of any of the preceding examples, wherein the memory stores therein instructions that when executed, cause the computing device to:evaluate and rank detected abnormalities based on clinical relevance.

[0141] Example 5 - The system of any of the preceding examples, wherein the memory stores therein instructions that when executed, cause the computing device to: receive inputs to cause zoom, rotate, pan, isolation of anatomical layers, or highlighting of specific regions of interest of the displayed 3D model.

[0142] Example 6 - The system of any of the preceding examples, wherein the memory stores therein instructions that when executed, cause the computing device to: present textual summaries of detected abnormalities, each textual summary including descriptive information of relevant clinical implications of the abnormality.

[0143] Example 7 - The system of any of the preceding examples, wherein the memory stores therein instructions that when executed, cause the computing device to: overlay detected abnormalities onto corresponding anatomical structures within the 3D model; and indicate abnormality type, severity, and spatial context in relation to surrounding anatomy.

[0144] Example 8 - The system of any of the preceding examples, wherein the memory stores therein instructions that when executed, cause the computing device to: receive input via the user interface of resection lines for a resection procedure.

[0145] Example 9 - The system of example 8, wherein the memory stores therein instructions that when executed, cause the computing device to: store the 3D model, abnormalities, annotated alerts, and resection lines as a procedure plan in the memory.

[0146] Example 10 - The system of example 9, wherein the procedure plan is accessible via one or more computing devices in a surgical suite for reference during the planned procedure.

[0147] Example 11 - A method of detecting and visualizing anatomical abnormalities, including receiving a patient image scan dataset, segmenting the image scan dataset, identifying and labeling anatomical structures, generating a three-dimensional (3D) model, analyzing the 3D model to identify abnormalities, and displaying in a user interface the 3D model of the anatomical structures and the identified abnormalities, wherein the abnormalities include annotated alerts.

[0148] Example 12 - The method of example 11, wherein the detected abnormalities include one or more of an aberrant vascular branching pattern, stenosed airways, obstructed airways, pleural adhesions, pleural thickening, fibrosis, emphysema, or infectious.

[0149] Example 13 - The method of examples 11 or 12, wherein the segmentation of the image scan dataset to identify anatomical structures and the detection of abnormalities are performed by an artificial intelligence or a neural network algorithm.

[0150] Example 14- The method of any of examples 11-13, further comprising: evaluating and ranking detected abnormalities based on clinical relevance.

[0151] Example 15 - The method of any of examples 11-14, further comprising: receiving inputs to cause zoom, rotate, pan, isolation of anatomical layers, or highlighting of specific regions of interest of the displayed 3D model.

[0152] Example 16 - The method of any of examples 11-15, further comprising: presenting textual summaries of detected abnormalities, each textual summary including descriptive information of relevant clinical implications of the abnormality.

[0153] Example 17 - The method of any of examples 11-16, further comprising: overlaying detected abnormalities onto corresponding anatomical structures within the 3D model; and indicating an abnormality type, severity, and spatial context in relation to surrounding anatomy.

[0154] Example 18 - The method of any of examples 11-17, further comprising: receiving input via the user interface of resection lines for a resection procedure.

[0155] Example 19 - The method of example 18, further comprising: storing the 3D model, abnormalities, annotated alerts, and resection lines as a procedure plan in a memory of a computing device.

[0156] Example 20 - The method of example 19, wherein the procedure plan is accessible via one or more computing devices in a surgical suite for reference during the planned procedure.

[0157] Example 21 - A system including a processor; and memory having stored thereon instructions, which when executed by the processor, causes the processor to receive radiographic images of a lung and a lesion, segment the lung and the lesion in the radiographic images, define a margin around the lesion, generate a three-dimensional (3D) model based on the segmented lung and lesion, and the defined margin, display the 3Dmodel, receive parameters of a resection procedure, and modify the 3D model to show a virtual resection based on the received parameters.

[0158] Example 22 - The system of example 21, wherein the virtual resection is a lobectomy, a segmentectomy, or a wedge.

[0159] Example 23 - The system of examples 21 or 22, wherein the instructions further cause the processor to receive identification of a tool, and modify the 3D model based on characteristics of the tool and the virtual resection.

[0160] Example 24 - The system of example 23, wherein the 3D model is modified to show effects of the virtual resection on tissue.

[0161] Example 25 - The system of example 21, wherein the instructions further cause the processor to receive instructions to adjust the virtual resection, and modify the 3D model based on the instructions to adjust the virtual resection.

[0162] Example 26 - The system of example 25, wherein the instructions further cause the processor to display a message requesting a clinician to review and confirm the virtual resection.

[0163] Example 27 - A method including receiving radiographic images of a lung and a lesion, segmenting the lung and the lesion in the radiographic images, defining a margin around the lesion, generating a three-dimensional (3D) model based on the segmented lung and lesion, and the defined margin, displaying the 3D model, receiving parameters of a resection procedure, and modifying the 3D model to show a virtual resection based on the received parameters.

[0164] Example 28 - The method of example 27, wherein the virtual resection is a lobectomy, a segmentectomy, or a wedge.

[0165] Example 29 - The method of example 27 or 28, further comprising receiving identification of a tool, and modifying the 3D model based on characteristics of the tool and the virtual resection.

[0166] Example 30 - The method of example 29, further comprising modifying the 3D model to show effects of the virtual resection on tissue.

[0167] Example 31 - The method of example 27, further comprising receiving instructions to adjust the virtual resection, and modifying the 3D model based on the instructions to adjust the virtual resection.

[0168] Example 32 - The method of example 31, further comprising displaying a message requesting a clinician to review and confirm the virtual resection.

[0169] Example 33 The method of example 27, further comprising simultaneously displaying at least two radiographic images captured on different dates.

[0170] Example 34 - The method of example 33, further comprising automatically selecting and displaying the at least two radiographic images based on the segmented lesion or the defined margin.

[0171] Example 35 - The method of example 33, further comprising displaying a user control which, when selected, displays the at least two radiographic images in a separate window.

[0172] Example 36 - The method of example 33, further comprising displaying measurement tools enabling a user to measure features within the at least two radiographic images.

[0173] Example 37 - The method of claim example 33, wherein the at least two radiographic images are displayed side-by-side or adjacent to each other.

[0174] Example 38 - The method of example 27, wherein the radiographic images are tomographic image slices.

[0175] While detailed embodiments are disclosed herein, the disclosed embodiments are merely examples of the disclosure, which may be embodied in various forms and aspects. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a basis for the claims and as a representative basis for teaching one skilled in the art to variously employ the disclosure in virtually any appropriately detailed structure.

Claims

WHAT IS CLAIMED IS:

1. A system comprising: a processor; and memory having stored thereon instructions, which when executed by the processor, causes the processor to: receive radiographic images of a lung and a lesion; segment the lung and the lesion in the radiographic images; define a margin around the lesion; generate a three-dimensional (3D) model based on the segmented lung and lesion, and the defined margin; display the 3D model; receive parameters of a resection procedure; and modify the 3D model to show a virtual resection based on the received parameters.

2. The system of claim 1, wherein the virtual resection is a lobectomy, a segmentectomy, or a wedge.

3. The system of claim 1, wherein the instructions further cause the processor to: receive identification of a tool; and modify the 3D model based on characteristics of the tool and the virtual resection.

4. The system of claim 3, wherein the 3D model is modified to show effects of the virtual resection on tissue.

5. The system of claim 1, wherein the instructions further cause the processor to: receive instructions to adjust the virtual resection; and modify the 3D model based on the instructions to adjust the virtual resection.

6. The system of claim 5, wherein the instructions further cause the processor to display a message requesting a clinician to review and confirm the virtual resection.

7. A method comprising: receiving radiographic images of a lung and a lesion; segmenting the lung and the lesion in the radiographic images; defining a margin around the lesion; generating a three-dimensional (3D) model based on the segmented lung and lesion, and the defined margin; displaying the 3D model; receiving parameters of a resection procedure; and modifying the 3D model to show a virtual resection based on the received parameters.

8. The method of claim 7, wherein the virtual resection is a lobectomy, a segmentectomy, or a wedge.

9. The method of claim 7, further comprising: receiving identification of a tool; and modifying the 3D model based on characteristics of the tool and the virtual resection.

10. The method of claim 9, further comprising modifying the 3D model to show effects of the virtual resection on tissue.

11. The method of claim 7, further comprising: receiving instructions to adjust the virtual resection; and modifying the 3D model based on the instructions to adjust the virtual resection.

12. The method of claim 11, further comprising displaying a message requesting a clinician to review and confirm the virtual resection.

13. The method of claim 7, further comprising simultaneously displaying at least two radiographic images captured on different dates.

14. The method of claim 13, further comprising automatically selecting and displaying the at least two radiographic images based on the segmented lesion or the defined margin.

15. The method of claim 13, further comprising displaying a user control which, when selected, displays the at least two radiographic images in a separate window.

Images