warning and mitigating the deviation of anatomical feature orientation from previous images to real-time interrogation

Abstract

Systems, apparatuses, methods, and computer program products for identifying and mitigating image-body bias are disclosed herein. In some embodiments, a method includes receiving sensor data from a medical device while the medical device is inserted into an anatomical region of a patient and after the medical device has been registered to an anatomical model of the anatomical region, wherein the anatomical model is based on previously obtained image data of the anatomical region and includes a virtual path extending through the anatomical model to an anatomical structure of interest, and wherein the sensor data is indicative of a position of at least a portion of the medical device; comparing the sensor data to a corresponding portion of the virtual path; generating, based at least in part on the comparison, a bias ranker indicative of a bias of the anatomical region from the anatomical model; and generating a warning when the bias ranker exceeds a predetermined threshold.

Description

[0001] Cross-reference of related applications

[0002] This patent document claims priority and benefit to U.S. Provisional Patent Application No. 63 / 065,420, filed August 13, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to systems, apparatus, methods, and computer program products for identifying, warning, predicting, and / or mitigating discrepancies in anatomical features between past preoperative images and current physical orientation, particularly during minimally invasive medical procedures using medical devices. Background Technology

[0004] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects. Such techniques can be performed through natural openings in the patient's anatomy or through one or more surgical incisions. Through these natural openings or incisions, the operator can insert minimally invasive medical instruments to reach the target tissue location. Minimally invasive medical instruments include, for example, therapeutic, diagnostic, biopsy, and surgical instruments. These instruments can be inserted into anatomical pathways and navigated toward regions of interest within the patient's anatomy.

[0005] To aid in reaching target tissue locations, the orientation and movement of minimally invasive medical tools can be mapped to image data of the patient's anatomy, typically acquired prior to the medical procedure. This image data can be used to assist in navigating medical tools through natural or surgically created pathways in the anatomical system, such as the lungs, colon, intestines, kidneys, heart, and circulatory system. However, several challenges have arisen in reliably and accurately mapping images of medical tools and anatomical pathways, particularly in locating identifiable anatomical structures of interest within previously acquired image data. Summary of the Invention

[0006] This document discloses systems, apparatuses, methods, and computer program products for identifying and mitigating deviations in anatomical structures between preoperative images (e.g., previously acquired images obtained using one or more medical imaging modalities, such as computed tomography (CT) scan images) and sensor data, such as position, orientation, or shape sensor data obtained from one or more sensors within the patient's anatomical structure and / or intraoperative images obtained in real-time by the medical device. Systems, apparatuses, methods, and computer program products are also disclosed for determining deviations between anatomical structures and anatomical models and predicting their actual orientation while the medical device is querying the patient's anatomical structure in real-time.

[0007] In some embodiments, a system for determining, for example, the deviation between an anatomical region and an anatomical model of the anatomical region includes: a medical device including sensors, wherein the medical device is insertable into a patient's anatomical structure; and a computing device in communication with said medical device, wherein the computing device includes a processor and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations including: receiving data acquired by the sensors of the medical device simultaneously with the insertion of the medical device into the patient's anatomical region and after the medical device has been registered to an anatomical model of the anatomical region, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to an anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device; comparing the sensor data with corresponding portions of the virtual path; generating a deviation grader indicating the deviation between the anatomical region and the anatomical model, at least in part based on the comparison; and generating a warning when the deviation grader exceeds a predetermined threshold.

[0008] In some embodiments, for example, a non-transitory computer-readable medium may store instructions thereon that, when executed by one or more processors of a computing system, cause the computing system to perform operations including: receiving data acquired by sensors of the medical device simultaneously with the insertion of the medical device into the anatomical region of a patient and after the medical device has been registered to an anatomical model of the anatomical region, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to an anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device; comparing the sensor data with corresponding portions of the virtual path; generating a deviation grader indicating a deviation between the anatomical region and the anatomical model, at least in part based on the comparison; and generating a warning when the deviation grader exceeds a predetermined threshold. Attached Figure Description

[0009] Many aspects of this disclosure can be better understood by referring to the following accompanying drawings. The components in the drawings are not necessarily to scale. Rather, the focus is on clearly illustrating the principles of this disclosure. The drawings should not be construed as limiting this disclosure to the specific embodiments depicted, but are intended for explanation and understanding only.

[0010] Figure 1 This is a schematic diagram of a medical device system that is inserted into the patient's anatomical pathway.

[0011] Figure 2 This is a drawing showing multiple coordinate points that form a configuration according to various embodiments of the present technology. Figure 1 The point cloud of the shape of the part of the medical device system.

[0012] Figure 3 This illustrates various embodiments of the present technology for identifying and / or mitigating target anatomical structures along... Figure 1 A flowchart of a method for determining the deviation of a planned path from the predetermined orientation of a medical device system.

[0013] Figures 4A-4C This is a drawing illustrating virtual navigation images of a medical device system using an anatomical model according to various embodiments of the present technology, the anatomical model depicting deviations from the target anatomical structure.

[0014] Figure 5A and Figure 5B This is a schematic diagram of a display system according to various embodiments of the present technology.

[0015] Figure 6 This is a schematic diagram of a robot or remotely operated medical system constructed according to various embodiments of the present technology.

[0016] Figure 7 This is a schematic diagram of a manipulator assembly, medical device system, and imaging system constructed according to various embodiments of the present technology. Detailed Implementation

[0017] This disclosure relates to systems, apparatuses, methods, and computer program products for identifying and mitigating deviations between: (i) anatomical models generated from prior preoperative imaging of a patient's anatomy (e.g., obtained using one or more medical imaging modalities, such as CT scan imaging); and (ii) sensor data, such as position, orientation, and shape sensor data, and / or images acquired in real time by a medical device located within the patient's anatomy. Systems, apparatuses, methods, and computer program products for determining deviations in anatomy and predicting its actual orientation while a medical device is querying the patient's anatomy in real time are also disclosed.

[0018] In some implementations, as illustrated in the exemplary embodiments below, the disclosed systems, apparatuses, methods, and computer program products can be used to determine how a target region of the lung airway changes from a preoperative 3D map of the airway (image data collected from an imaging system, such as a CT system) constructed prior to the medical procedure to its actual orientation and conformation during the medical procedure, while a medical device (e.g., a robotic catheter) is navigating in the airway based on a 3D map.

[0019] While the disclosed embodiments are described herein primarily in the context of identifying and mitigating biases in target anatomical structures within the lung airways to facilitate understanding of the basic concepts, it should be understood that the disclosed embodiments may also include identifying and mitigating biases in target anatomical structures in other tissues, organs, and organ systems, including but not limited to the colon, intestines, kidneys, heart, urinary tract, and circulatory system. Similarly, although the disclosed embodiments relate to CT-body biases, it should be understood that other imaging modalities are also applicable to the disclosed techniques, including but not limited to magnetic resonance imaging (MRI), X-ray imaging, ultrasound imaging, and so on.

[0020] Image-body bias is a phenomenon in which the current orientation of an anatomical structure within a patient's body changes relative to its previous orientation as observed in previously acquired (or preoperative) images. This phenomenon is referred to herein as "image-body bias," and in specific imaging modalities such as CT imaging, it can be called "CT-body bias." In common clinical practice, image-body bias typically occurs when medical images are obtained prior to a medical procedure (e.g., weeks or months before the procedure). For certain anatomical structures (e.g., the lungs), image-body bias can also occur shortly before the procedure. For patients undergoing a procedure to inquire about target anatomical structures (e.g., potential tumor sites) identified in previously acquired medical images (e.g., CT images, MRI images, etc.), image-body bias of the target anatomical structure is a relatively common occurrence at the time of the procedure. This can complicate or even hinder the procedure, as it may be difficult or impossible for the physician to locate the target site using currently available medical equipment. For example, image-body bias can still occur in a patient’s lungs on the same day as preoperative imaging. This is because the patient’s breathing patterns differ between preoperative imaging (controlled breathing via mechanical ventilation) and during the medical procedure (natural breathing), as well as due to the effects of lung contraction and atelectasis. Either of these factors can produce and exacerbate image-body bias, which in turn can greatly affect the medical procedure.

[0021] Image-body deviations can result from both intrinsic and extrinsic causes. For example, lung structures can naturally change shape during medical procedures due to a variety of reasons (known as “natural deformities”). These reasons include atelectasis, inconsistencies between the patient’s body position and breathing patterns during preoperative imaging and the medical procedure (e.g., breath-holding, positive pressure ventilation versus spontaneous breathing), and / or changes in the lung anatomy itself, such as changes in the target tumor or lung since previous imaging (especially when imaging was performed weeks to months prior to the medical procedure). Similarly, lung structures can change shape due to “induced deformities.” Induced deformities can be caused by medical devices pushing against airway walls during execution and / or by the angle of intubation that alters the tilt of the airway within the patient.

[0022] For lung diagnostic procedures (e.g., for tissue biopsies within or around the airways of the lung), preoperative images of lung structures can be overlaid to generate a 3D map or model of the lung structure. This can be used to inform the medical device user (e.g., a physician) where and how to navigate the medical device through the lung structures to reach the target anatomical structure or region. Because the 3D map or model is based on preoperative images, image-body bias can cause the physician to drive the medical device to the correct orientation within the 3D map or model where the system has mapped the target anatomical structure, but the target anatomical structure is no longer in that orientation, resulting in missed diagnoses or inappropriate interventions due to failure to find or reach the target anatomical structure and / or action on incorrect tissue samples. Conventional techniques for creating 3D maps and navigation plans do not address or accurately account for the image-body bias problem. Furthermore, existing techniques for creating 3D maps and navigation plans for medical devices do not provide the medical device user with sufficient and timely notification of image-body bias regarding the target structure to be navigated to.

[0023] This technology provides techniques for identifying and determining the degree of image-body deviation and for mitigating such image-body deviation, achieved by generating warnings and / or predicting the true orientation of a target anatomical structure determined to have deviated from previously obtained images (e.g., preoperative images). For example, the disclosed technology generates an anatomical model of the anatomical region based on a preoperative image of the anatomical region. Based on the preoperative image, the disclosed technology also generates a planned path for the user to navigate a medical device through the anatomical region to the target anatomical structure identified within the preoperative image. The planned path is projected onto the anatomical model. Once the medical device is positioned within the anatomical region and registered to the anatomical model, the user can navigate the medical device toward the target anatomical structure within the anatomical region using the anatomical model and the planned path. At various points during navigation (e.g., as the medical device passes anatomical landmarks, as the medical device navigates a specified distance, as the medical device approaches the end of the planned path, as the medical device approaches the target anatomical structure, etc.), the disclosed technology is able to compare (i) one or more planned orientations (e.g., planned orientation of the tip) of one or more parts of the medical device along the planned path with (ii) one or more current orientations (e.g., current orientation of the tip) of the corresponding parts of the medical device extracted from position sensor data (e.g., shape data) acquired by one or more sensors of the medical device. The difference between the current orientation and the corresponding planned orientation can provide an indication of the magnitude and direction of the deviation between the anatomical region and the anatomical model. In some embodiments of the disclosed technology, the user is warned when the determined deviation exceeds a predetermined threshold (e.g., a predetermined deviation distance). Furthermore, because the movement of the target anatomical structure is correlated with the movement of local anatomical regions of the target anatomical structure, in some embodiments, the disclosed technology is able to (i) evaluate the difference between (one or more) current orientations and (one or more) planned orientations as the medical device approaches the target anatomical structure in order to predict the current position of the target anatomical structure and (ii) update the anatomical model to reflect the predicted position. Therefore, the current technology mitigates the effects of CT-body deviations on the anatomical model in a manner where the source or cause of the deviation is unknown, and helps to navigate the medical device to the likely current position of the target anatomical structure, thereby increasing the likelihood of a successful medical procedure.

[0024] These and other embodiments are discussed in more detail below with examples.

[0025] A. Examples of techniques for identifying and mitigating image-body bias

[0026] Figure 1This is a schematic diagram of a portion of a medical device system 604 constructed according to various embodiments of the present technology, which is inserted, for example, into an anatomical region 150 of a patient (e.g., a human lung) during a medical procedure. Figure 1 As shown in the drawing, the portion of the medical device system 604 inserted into the anatomical region includes an elongated device 131 that extends within a branching anatomical pathway 152 of the anatomical region 150. In this example, the anatomical pathway 152 includes the trachea 154 of the lung and a plurality of bronchi 156.

[0027] In some embodiments, the elongated device 131 is part of a flexible catheter or other biomedical device whose size and shape can be configured to receive a medical device and facilitate delivery of the medical device to the distal portion 138 of the elongated device 131 for various purposes. For example, the medical device of the medical device system 604 can be used for medical purposes such as investigation of anatomical pathways, surgery, biopsy, ablation, illumination, irrigation, and / or aspiration. The medical device can include position sensors, rate sensors, image acquisition probes, biopsy instruments, laser ablation fibers, and / or other surgical, diagnostic, and / or therapeutic tools. The following is in conjunction with… Figure 6 and Figure 7 Further details regarding medical device system 604 are described in more detail.

[0028] In some exemplary embodiments (such as in combination) Figure 7 (Discussed more specifically below), the elongated device 131 may include an endoscope or other biomedical device having one or more image acquisition devices 747, wherein the image acquisition devices are positioned at the distal portion 138 of the elongated device 131 (e.g., Figure 1 (as shown in the example) and / or at other locations along the elongated device 131. In these embodiments, while the elongated device 131 is within the patient's anatomical region 150, the one or more image acquisition devices 747 are capable of acquiring one or more real navigation images or videos (e.g., a sequence of one or more real navigation image frames) of the anatomical pathway and / or other real patient anatomy.

[0029] exist Figure 1In the exemplary embodiment shown, the elongated device 131 has a specific position, orientation, posture, and shape within the anatomical region 150, wherein all or part of (supplementing or replacing motion, such as rate or velocity) is capable of being acquired by position sensor data. In this or other examples, a position sensor system 608 communicating with the medical device system 604 is configured to acquire position sensor data from at least one sensor of the elongated device 131. In various embodiments of the medical device system 604, for example, at least one sensor of the elongated device 131 may include a shape sensor 133 and / or one or more position measuring devices (hereinafter referred to as such). Figure 7 (Each of which will be discussed more specifically). In some embodiments, the position sensor system 608 is capable of using the reference frame of the medical device and / or elongated device 131 (e.g., a Cartesian coordinate system reference frame (X)). M ,Y M Z M The location sensor data of the medical device system 604 within the anatomical region 150 is collected to investigate the anatomical pathway 152. The location sensor data may be at least partially recorded as a set of two-dimensional or three-dimensional coordinate points.

[0030] In the example where anatomical region 150 is a human lung, the coordinate points may represent the orientation of the distal portion 138 of the elongated device 131 and / or other portions of the elongated device 131 as it advances through the trachea 154 and bronchus 156. In these and other embodiments, the set of coordinate points may represent the shape (one or more) of the elongated device 131 as it advances through anatomical region 150. Furthermore, in these and other embodiments, the coordinate points may represent positional data for other portions of the medical device system 604.

[0031] The set of 2D and / or 3D coordinate points from recorded position sensor data can be used together to form a point cloud. For example, Figure 2 This is a drawing showing a plurality of coordinate points 262, which form a configuration according to various embodiments of the present technology. Figure 1 A point cloud 260 representing the shape of the portion of the elongated device 131. In various embodiments, for example, the point cloud 260 is generated by the union of all or a subset of coordinate points 262 recorded by the position sensor system 608 while the elongated device 131 is in a stationary position.

[0032] In some embodiments, the point cloud (e.g., point cloud 260) may include the union of all or a subset of coordinate points 608 recorded by the position sensor system 608 during a data acquisition cycle of multiple shapes, positions, orientations, and / or postures of the elongated device 131 spanning the anatomical region 150. In these embodiments, the point cloud may include coordinate points acquired by the position sensor system 608 representing multiple shapes of the elongated device 131 as the elongated device 131 advances or moves through the patient's anatomy during an image acquisition cycle. Alternatively or additionally, because the configuration (including shape and orientation) of the elongated device 131 within the patient's body changes due to anatomical movement during an image acquisition cycle, in some embodiments the point cloud may include the plurality of coordinate points 262 representing the shape of the elongated device 131 acquired by the position sensor system 608 as the elongated device 131 passively moves within the patient's body.

[0033] Point clouds of coordinate points acquired by the position sensor system 608 can be registered to different models or datasets of the patient's anatomy. For example, the point cloud of coordinate points can be registered to previously acquired (preoperative) image data of the anatomical region 150 acquired by the imaging system. In some embodiments, for example, previously acquired image data of the anatomical region 150 is used to generate an anatomical model of the anatomical region. The elongated device 131 can be registered to the anatomical model (and / or to endoscopic image data (if applicable) generated by the one or more image acquisition devices 747) based on the position sensor data generated by the position sensor system 608 to (i) map the tracking position, orientation, posture, shape, and / or movement of the medical device system 604 within the anatomical region 150 to the correct position within the anatomical model in real time, and / or (ii) determine a virtual navigation image of the virtual patient anatomy of the anatomical region 150 as viewed from the viewpoint of the medical device system 604 at an orientation within the anatomical model corresponding to the orientation of the elongated device 131 within the patient's body.

[0034] Return to reference Figure 1The anatomical region 150 includes an anatomical structure of interest 198 (also referred to as a “target anatomical structure” or “target”), such as suspected tumor tissue. In some embodiments, the target 198 is mapped to an anatomical model of the anatomical region 150 based on the physical orientation of the target 198 relative to the anatomical region 150 as observed in previously acquired (preoperative) image data. The anatomical model can further provide the user with a planned path to navigate the elongated device 131 to the target anatomical structure 198 during the medical procedure. After the medical device system 604 is registered to the anatomical model, the planned path can be updated based on the tracking position, orientation, posture, shape, and / or movement of the elongated device 131 within the anatomical region 150. The planned path can be displayed on (one or more) virtual navigation images of the virtual patient anatomy of the anatomical region 150 from the perspective of the medical device system 604, which can be represented as a line or a series of points along one or more views of the anatomical region 150 of the generated anatomical model corresponding to the orientation of the medical device system 604 within the patient. (The following is in conjunction with...) Figures 4A-4C and / or Figure 5A and Figure 5B More specifically, an example of a virtual navigation image associated with an anatomical model is discussed, showing a portion of an exemplary planned path to target 198.

[0035] However, as discussed above, due to image-body bias, the user cannot determine that the target 198 is in the same orientation during the medical procedure as it was when the previously acquired preoperative image data was obtained. Therefore, the user also cannot determine the orientation of the target 198 as depicted in the anatomical model and the virtual navigation image. Therefore, the medical device system 604 can be implemented to identify and / or mitigate potential image-body bias based on the techniques described below according to this technology.

[0036] Figure 3 This is a flowchart illustrating a method 300 according to various embodiments of the present technology, the method 300 being used to identify deviations between an anatomical region and an anatomical model of the anatomical region, the anatomical model being generated from previously obtained preoperative image data of the anatomical region. Figure 3 The flowchart further illustrates how the method 300, also according to various embodiments of the present technology, can be implemented to mitigate identified biases, for example by alerting the user of the medical device and / or predicting the actual location of the target anatomical structure, which can be used to redirect the navigation of the medical device.

[0037] Method 300 is shown as a set of operations or processes 302-312, which can optionally include processes 314-322 in various ways or combinations. All or a subset of the processes of method 300 can be implemented by a medical device combined with a computing device, such as a control system communicating with or integrated with a medical system including the medical device. Alternatively or in combination, all or a subset of the processes of method 300 can be implemented by a control system of a medical device system or apparatus, including but not limited to a robot or remote operating system 600 (as described below). Figure 6 and Figure 7 (As described more specifically) and various components or devices of any other suitable system. A computing device or control system for implementing method 300 can include one or more processors operatively coupled to a memory storing instructions that, when executed, cause the computing system to perform operations according to some or all of processes 302-310 and / or processes 312-322 of method 300. Alternatively or concurrently, all or a subset of processes of method 300 can be executed by an operator of system 600 (e.g., a doctor, user, etc.). Furthermore, any one or more processes of method 300 can be executed according to the discussion above. Reference continues below. Figure 4A-5B One or more of the processes in 302-322 will be discussed to help clarify and understand this technique.

[0038] In process 302, method 300 acquires, receives, and / or processes image data of anatomical regions of the patient from an imaging system and generates an anatomical model. In some embodiments, the imaging system is a CT imaging system or other imaging system. In some embodiments of process 302, image data can be acquired, received, and / or processed during an image acquisition cycle of the imaging system. The image acquisition cycle can correspond to the time period during which the imaging system is activated. In some embodiments, for example, the image acquisition cycle can be preoperative, such that image data is acquired, received, and / or processed before the medical device system moves into the patient's body (e.g., minutes, hours, days, weeks, months, etc.). In other embodiments, the image acquisition cycle can be intraoperative, such that image data of the patient is acquired, received, and / or processed while the medical device system is in the patient's body. In these embodiments, the medical device system can be stationary during the image acquisition cycle, can undergo commanded movement (e.g., forward or flexing commanded by an operator), and / or can undergo passive movement (e.g., uncommanded movement arising from respiratory activity, cardiac activity, or other voluntary or involuntary patient movement) during the image acquisition cycle. In some other embodiments, the image acquisition cycle can be post-operative, such that patient image data is acquired, received, and / or processed after the medical device system has been removed from the patient. In some embodiments of process 302, for example, image data can be acquired, received, and / or processed in real-time or near real-time.

[0039] The patient image data acquired, received, and / or processed can include graphical elements representing the patient's anatomical features, and in the case of intraoperative image data, the acquired, received, and / or processed image data can include graphical elements representing the medical device system. In some embodiments of process 302, for example, an anatomical model of the patient's anatomical features can be generated by segmenting and filtering the graphical elements included in the image data. For example, during the segmentation process, pixels or voxels generated from the image data can be segmented into segments or elements and / or labeled to indicate that they share certain features or computational properties, such as color, density, intensity, and texture. In some embodiments, less image data than the entire image data can be segmented and filtered. The segments or elements associated with the patient's anatomical features are then converted to an image reference frame (X). I ,Y I Z I The generated anatomical model is shown in [the document / reference]. Figure 5A and Figure 5B And this will be discussed in more detail below.

[0040] At step 304, method 300 identifies the position of one or more target anatomical structures relative to anatomical regions from image data and generates planned paths to (one or more) targets via anatomical pathways through the generated anatomical model. The planned paths provide a map to navigate medical devices (e.g., medical instruments) toward (one or more) targets via the imaged anatomical regions (e.g., via anatomical pathways such as lung airways). Figure 1 The elongated device 131 of the medical device system 604 shown. In some embodiments of process 304, for example, a planned path can be generated on a virtual navigation image (or a set of virtual navigation images) that provides a virtual map of the patient's anatomy. This will be discussed in more detail below. Figure 4A An example of a planned path 452 is shown overlaid on a virtual image of a portion of a patient’s anatomical region 150 (e.g., the lung).

[0041] At step 306, method 300 records position sensor data of a medical device system (e.g., medical device system 604) positioned within the anatomical region and performs registration between the recorded position sensor data and image data. In some embodiments of step 306, position sensor data is recorded using a position sensor system (e.g., position sensor system 608) capable of recording during a position data acquisition cycle. For example, while at least a portion of the medical device system is located within the anatomical region, the position sensor data provides position information (e.g., shape, location, orientation, posture, motion, etc.) of the medical device system. The position data acquisition cycle can correspond to the time period during which one or more other position sensors of the shape sensor and / or position sensor system are activated to collect and record position sensor data. For example, during the position data acquisition cycle, the medical device system may be stationary, may undergo commanded movement (e.g., forward or flexing as commanded by an operator), and / or may undergo passive movement (e.g., uncommanded movement arising from respiratory activity, cardiac activity, or other voluntary or involuntary patient movement). In some embodiments of step 306, the position sensor data can be at least partially recorded as a medical device reference frame (X). M ,Y M Z M One or more two-dimensional or three-dimensional coordinate points in a medical device reference system, such as in an exemplary embodiment in a surgical setting, can involve a surgical reference system (X). S ,Y S Z S In these and other implementations, for example, coordinate points corresponding to location sensor data can be associated with timestamps that can be included as part of the recorded location sensor data.

[0042] In some embodiments of process 306, registration of the medical device system based on recorded position sensor data and image data includes establishing a medical device reference frame (X). M ,Y M Z M (and / or surgical reference system (X) S ,Y S Z S Aligned with the image reference frame (X) I ,Y I Z I In some embodiments of process 306, registration includes generating a point cloud of recorded position sensor data. In various embodiments, for example, the point cloud can be generated from the union of all coordinate points or subsets of coordinate points associated with position sensor data recorded, for example, during one or more position data acquisition cycles of the position sensor system. For example, the point cloud can represent one or more shapes of the medical device system when it is stationary and / or actively or passively moving within a patient. In these and other embodiments, points can represent the orientation of one or more portions (e.g., tips) of the medical device system over time (e.g., during multiple data acquisition cycles). In various examples, the point cloud can be in a medical device reference frame (X... M ,Y M Z M It can be generated in two or three dimensions.

[0043] Medical Device Reference System (X) M ,Y M Z M It can be registered to the image reference frame (X). I ,Y I Z IThe registration is performed on the anatomical model within the point cloud. This registration can be achieved by rigidly and / or non-rigidly transforming the coordinates of the point cloud to rotate, translate, or otherwise manipulate them to align the coordinates with the anatomical model. The transformation can be a six-degree-of-freedom transformation, such that the point cloud can be translated or rotated in any or all of the X, Y, Z, pitch, roll, and yaw axes. In some implementations of the registration between the recorded position sensor data and image data at step 306, method 300 uses an Iterative Closest Point (ICP) algorithm to perform the registration. For example, method 300 is capable of (i) calculating point-to-point correspondences between coordinates in the point cloud and points within the anatomical model (e.g., on the centerline or in other orientations) and (ii) calculating the optimal transformation to minimize the Euclidean distance between the corresponding points. Registration between position sensor data recorded in an instrument reference frame and image data in an image reference frame can be achieved, for example, by using point-based ICP techniques, as described in U.S. Provisional Patent Applications Nos. 62 / 205,440 and 62 / 205,433, both of which are incorporated herein by reference in their entirety. In other embodiments of process 306, registration can be performed using other techniques.

[0044] At process 308, as the medical device system (e.g., the elongated device 131 of medical device system 604) is navigated (e.g., driven) along a planned path generated at process 304 on its way to (one or more) targets, method 300 acquires position sensor data at various moments. In some embodiments, these moments correspond to the times when the medical device system is located at anatomical landmarks, has been navigated a specified distance, is approaching the end of the planned path, is approaching (one or more) targets, and / or other specified events. For example, in various embodiments of process 308, when it is determined that the medical device system is approaching a recognizable anatomical landmark, the medical device system is navigated along a planned path generated at process 304 on its way to the target. In some examples where the anatomical region is an anatomical pathway, the specified landmark may include branching points within the anatomical pathway. In examples where the anatomical pathway is the pulmonary airway of the lung, such as... Figure 1 As shown in the anatomical region 150, designated landmarks may include ridges. For example, in such an embodiment, process 308 may include verification techniques in which the operator of the medical device system checks to ensure that the medical device system is being driven along the correct path according to the planned path (on the way to one or more targets). In some embodiments, the verification techniques may include guiding the operator to drive the medical device system to the nearest landmark along the navigation path, such as a ridge in the driven path, and acquiring one or more images using the one or more image acquisition devices 747 to compare with the expected ridge along the planned path based on one or more virtual navigation images associated with a virtual map of the patient's anatomy.

[0045] Figures 4A-4C Provided to aid in further description Figure 3 Method 300 in certain aspects. Figure 4A For example, a partial schematic diagram illustrates an example of a virtual navigation image of a medical device system based on an anatomical model 450 of an anatomical region (e.g., generated at steps 302 and / or 304 of method 300). The virtual navigation image includes a planned path 452 that navigates an anatomical pathway 152 of the anatomical model 450 toward a virtual target orientation 455. The virtual target orientation 455 is positioned relative to the anatomical model 450 at a location corresponding to the orientation of a target anatomical structure 198 relative to the anatomical region, previously determined by preoperative image data of the anatomical region. Figure 4A The diagram also shows a point cloud 260 composed of multiple location coordinate points 262, which are associated with the current or recent location (e.g., shape) of the medical device system within the anatomical region. As the medical device system is navigated through the anatomical region according to a planned path 452, one or more of the coordinate points 262 can be acquired. For example, as the medical device system approaches or reaches the end point of the planned path 452, the coordinate points 262 in the point cloud 260 can be acquired.

[0046] Continuing the example above, although the medical device system has been navigated to the location within the anatomical region corresponding to the endpoint of the planned path 452 within the anatomical model 450, the position of the point cloud 260 in the image reference frame deviates from the position of the planned path 452 in the image reference frame. That is, the anatomical region has deviated from the anatomical model 450 at least along the portion of the anatomical region corresponding to the illustrated point cloud 260. Similarly, because the position of target 198 ( Figure 1 The movement is associated with the local anatomical region of the target, so the true position of the target (in) Figure 4A The target (represented by the real target orientation 435) has also deviated from the virtual target orientation 455. Therefore, if a user attempts to navigate from the current orientation of the medical device system shown in point cloud 260 to the virtual target orientation 455, the medical device system may not encounter the target now located at the orientation corresponding to the real target orientation 435 shown in the virtual image. Thus, in this situation, the likelihood of a successful medical procedure is significantly reduced (e.g., biopsy target 198). Figure 1 ), ablation target 198, etc.

[0047] In other words, the difference between planned path 452 and point cloud 260 can provide an indication of the direction and magnitude of the deviation between the anatomical region and the anatomical model. Therefore, refer to... Figure 3 and Figure 4AIn step 300, method 300 compares acquired position sensor data (e.g., all coordinate points 262 or a subset of coordinate points 262 in point cloud 260) with planned path 452 at step 310. In an implementation of step 310, the comparison between position sensor data and planned path can match the position sensor data to one or more points along planned path 452. For example, the comparison can match one or more points along the planned path corresponding to a specified landmark (e.g., one or more ridges) to the position sensor data (e.g., one or more coordinate points 262) of the medical device system closest to the specified landmark. In some examples, the position sensor data of the medical device system (e.g., the one or more coordinate points 262) used in the comparison with the specified landmark corresponds to one or more locations of the distal portion 138 of the elongated device 131. Figure 1 Alternatively, a comparison between position sensor data and planned path 452 can match the endpoint of planned path 452 to position sensor data (e.g., coordinate point 262) corresponding to the orientation of the distal portion 138 (e.g., distal or other part) of the medical device system. In this way, process 310 can be implemented using embodiments of a medical device system that may include only a single sensor (e.g., a position sensor or rate sensor located at a known orientation relative to the size of the medical device system) and embodiments that include multiple sensors and / or shape sensors. For example, process 310 can be implemented using one or more locations determined by a single sensor, such as an electromagnetic (EM) sensor, preferably located at the distal portion 138 of the elongated device 131, to determine the offset from the planned path using a deviation vector from that location on the elongated device. However, in another example, process 310 can be implemented using one or more position data points along the body of the elongated device 131 already acquired by shape sensor 133, which can be correlated with the shape of the airway by comparison with a larger portion of the planned path. Alternatively or alternatively, for example, the comparison between the position sensor data and the planned path 452 can include sampling a subset of the position sensor data and / or points along the planned path 452, such that a first number of position sensor data points are matched to the same number of points along the planned path 452.

[0048] In some implementations, comparing the planned path 452 with position sensor data (e.g., coordinate point 262 of the point cloud 260) can include determining an offset from the planned path to the position of the medical device system indicated by the position sensor data. In one embodiment, for example, the offset can be determined using a vector (hereinafter referred to as the “offset vector”) (or vice versa) from one or more points along the planned path to the determined position of one or more parts of the medical device system indicated by one or more corresponding position sensor data points (e.g., coordinate point 262). For example, the offset vector can be represented as:

[0049] Deviation vector x,y,z = position sensor data points x,y,z – matching points x,y,z along the planned path, where there is a deviation vector x,y,z for each pair of matching points.

[0050] Figure 4B It is shown Figure 4A A schematic diagram of a virtual navigation image. (For example...) Figure 4B As shown, coordinate point 262 of point cloud 260 is matched with corresponding points along planned path 452. For example, coordinate point 262 of point cloud 260 is matched with the nearest point along planned path 452. In these and other embodiments, coordinate point 262 at the end of point cloud 260 is matched with a point at the end of planned path 452. In these and still other embodiments, one or more points along planned path 452 corresponding to one or more anatomical landmarks within anatomical model 450 are matched with the nearest coordinate point 262 of point cloud. As discussed above, if there are more points along planned path 452 than coordinate point 262 in point cloud 260 (or vice versa), points or coordinate points 262 along planned path 452 can be downsampled such that the number of points along planned path 452 matches the same number of coordinate points 262 in point cloud 260.

[0051] exist Figure 4B The deviation vector 473 is shown between the point along the planned path 452 and the corresponding coordinate point 262 in the point cloud 260. Each deviation vector 473 can be calculated using the formula provided above. Furthermore, each deviation vector 473 provides an indication (magnitude and direction) of the deviation (attribute and direction) of the anatomical region from the anatomical model 450 at the orientation corresponding to the coordinate point 262 in the point cloud 260.

[0052] As discussed in more detail below, the correlation of a given deviation vector 473 can differ from... Figure 4BThe correlation of other deviation vectors 473 shown. For example, the deviation vector 473 corresponding to the coordinate point 262 indicating the location of the distal portion (e.g., distal end) of the medical device system in point cloud 260 can have a greater correlation than the deviation vector 473 corresponding to the coordinate point 262 indicating the location of the more proximal portion of the medical device system in point cloud 260. This is because the anatomical pathway 152 is generally considered to decrease in size (e.g., diameter) distally, which is thought to make the distal portion of the anatomical pathway 152 more susceptible to CT-body deviations. Another reason why the correlation of the two deviation vectors 473 may differ is that motion of a portion of the anatomical region of the target (e.g., target 435) is thought to have a greater influence on the target's position than motion of another portion of the anatomical region farther from the target. Therefore, the deviation vector 473 corresponding to the portion of the medical device system closer to the target (e.g., target 435) is thought to provide a better indication of the target's current position than the deviation vector 473 corresponding to the portion of the medical device system farther from the target. Therefore, method 300 can take into account all deviation vectors 473 or subsets of deviation vectors 473 generated by process 310 of method 300 (e.g., a set of deviation vectors 473 closest to target 198, another anatomical landmark or distal region (e.g., distal end) of medical device system; all deviation vectors within a specified distance of target (e.g., target 435), another anatomical landmark or distal region (e.g., distal end) of medical device system; etc.), and / or method 300 can apply different weights to deviation vectors 473.

[0053] In this manner, for example, method 300 can detect clinically relevant deviations to indicate significant tissue deformations in the probed anatomical pathway, and possibly indicate positional changes of the target anatomical structure from its orientation determined by previously acquired image data. See again Figure 3 At step 312, method 300 identifies whether a deviation exists. Method 300 may terminate at step 314 if no deviation is detected, or continue to step 316 if a deviation is detected. It should be understood that method 300 can be repeated intermittently or continuously, such that deviation identification is performed (at step 312) for multiple landmarks (e.g., for all ridges or subsets of ridges encountered as the medical device system is driven along the planned path), after the medical device system has traveled a specified distance, as the medical device system reaches the end of the planned path 452, as the medical device system approaches the target, etc.

[0054] To detect significant deviations, for example, some embodiments of method 300 include a deviation grading technique implemented at process 312. For example, the deviation grading technique is capable of grading (e.g., determining, predicting, etc.) the magnitude and / or direction of deviations from a corresponding part of the anatomical model of a portion of the anatomical region (e.g., an anatomical pathway); and / or (ii) the actual orientation of the target (e.g., true target orientation 435); Figure 4A and Figure 4B From its virtual orientation within or relative to the anatomical model (e.g., virtual target orientation 455); Figure 4A and Figure 4B The magnitude and / or direction of the deviation. Deviation grading techniques may include identifying one or more matching points between the planned path (e.g., a virtual line of the airway) and position sensor data generated by one or more position sensors of the medical device system, as discussed above. After determining the matching points, the technique may include calculating one or more deviation vectors, which can be included as vectors between one or more points along the planned path and one or more corresponding position sensor data points (e.g., indicating the position and / or shape of the elongated device 131), also as discussed above. Obviously, the technique may implement other methods to identify deviations in position sensor data from preoperative image data and / or other methods to update the orientation of the target, for example by using the endpoint of the planned path to the distal region 138 corresponding to the elongated device 131. Figure 1 A simple transformation of one or more coordinate points. In an example implementation of the deviation vector, the determined deviation vector may differ from the deviation vector obtained by point-by-point vectorization of the planned path to the position sensor data. For example, linear or quadratic interpolation can be used to determine the difference between the position sensor data and the planned path. The deviation grading technique produces a quantitative value called a "deviation grader," which can be a scalar or vector representation of the deviation of the anatomical region and / or the target from the planned path (from its orientation determined in previously acquired image data) based on some exemplary embodiments of process 310.

[0055] In some embodiments of the deviation grading technique implemented at process 312, method 300 determines an offset between the portion of the virtual path and the position sensor data of the medical device, wherein the offset is a quantitative value corresponding to the amount or degree of deviation between an anatomical feature determined from previously acquired image data and the same anatomical feature detected by the insertable medical device as it is driven along the planned path. In some embodiments, the deviation grader includes a device-path distance parameter comprising an average distance between at least one point on the planned path and the one or more position data points of the medical device system (e.g., at the orientation of the distal portion 138 of the elongated device 131), which can be determined using one or more deviation vectors calculated at process 310. The device-path distance parameter can be expressed in x cm or other distance units. In some embodiments, for example, the device-path distance parameter includes a range of average distance values. However, in some embodiments, the device-path distance parameter can include the average distance weighted by deviation vectors. For example, as discussed above, the deviation vector closer to the target (e.g., at the far end of the elongated device) can be weighted more heavily than the deviation vector farther from the target.

[0056] Because the deviation grading is a quantitative value, it can be used to identify whether there is a significant deviation from the target or other parts of the anatomical region detected by the medical device system. For example, if the device-path distance parameter determined based on the deviation threshold is relatively small, such as less than 1 mm, then method 300 can determine at step 312 that there is no significant deviation and terminate method 300 at step 314 (e.g., at least for this stage of the medical procedure). Clearly, the deviation threshold can vary for different anatomical regions or for different patients. In some examples of the lung airways, the deviation threshold for determining a significant deviation from the target could be 1 cm or greater. Similarly, the deviation threshold can be a range rather than a specific value to indicate the degree of deviation. For example, the deviation threshold could include 0 mm to <5 mm in one range to indicate negligible deviation, 5 mm to <10 mm in the next range to indicate insignificant deviation, 10 mm to <20 mm in another range to indicate significant deviation, and so on.

[0057] Similarly, for example, a deviation grading system can be used to determine whether a user should be warned about a deviation based on its quantity or degree. Furthermore, this feature can be used, for example, to derive a deviation metric based on other clinical factors, or it can be used as a signal to switch between different registration technologies in a medical device registration protocol.

[0058] In some embodiments, method 300 may optionally include process 316. At process 316, method 300 triggers a deviation warning when a deviation is detected at 312. In some embodiments of process 316, method 300 generates a warning when the deviation grader exceeds a predetermined threshold. For example, in some embodiments, method 300 compares a predetermined deviation threshold of 1 cm or greater with the deviation grader to trigger a warning to the user of the medical device system that the target anatomical structure 198 has significantly deviated from the planned path. This will be discussed in more detail below. Figure 5A and Figure 5B An example of warning the user is shown in an exemplary implementation of process 316.

[0059] Method 300 may optionally include procedures 318-322, which involve updating the anatomical model based on the comparison performed at procedure 310 and / or based on the deviation detected at procedure 312. Figure 3 As shown in the flowchart, method 300 identifies at step 318 whether to update the anatomical model (e.g., the anatomical model and / or a virtual image of the target). If method 300 determines that updating the anatomical model is unnecessary, method 300 can terminate at block 320. Otherwise, method 300 can continue to execute step 322 to update the description of the corresponding portion of the anatomical region, update the planned path, and / or update the virtual target location to the predicted location of the target (e.g., shown as...). Figure 4C The anatomical model is updated using the predicted target orientation 445 (discussed in more detail below). In some implementations, process 318 may be based on a deviation grader (e.g., when the identified deviation exceeds a specified threshold) or on (e.g., provided via a system display, as described below). Figure 5A (For more specific discussion) User input determines that the anatomical model needs to be updated. It should be understood that method 300 can be repeated intermittently or continuously, such that the determination of whether to update the anatomical model is performed continuously or intermittently (at process 318).

[0060] At step 322, method 300 updates the anatomical model. In some implementations, this may include updating the corresponding portions of the anatomical regions shown in the anatomical model, updating the planned paths projected onto the anatomical model, and / or updating the virtual target position 455. Figures 4A-4C Updated to predicted target location 445 ( Figure 4C ), and it is believed that the predicted target azimuth 445 is better aligned with the actual or true target azimuth 435 ( Figures 4A-4C Alignment. For example, method 300 may perform techniques to determine the predicted orientation (e.g., predicted true orientation) of a target anatomical structure, where the target has been moved due to image-body bias.

[0061] In some implementations, for example, predictive techniques identify matching points between the planned path (e.g., a virtual line or midline of an anatomical pathway) and position sensor data (e.g., shape data) generated by sensors of the medical device system, as discussed more specifically above. After determining the matching points, the technique calculates a deviation vector, which can include one or more vectors between (one or more) points from the planned path and the one or more position data points associated with the shape of the medical device system. Obviously, the technique can implement other methods to compare the shape data with previously acquired image data and update the virtual target, for example, by using a simple transformation from the endpoint of the planned path to the target to the final piece of the catheter shape. In deviation vector embodiments, the determined deviation vector can differ from the deviation vector obtained using point-by-point vectorization of the planned path to the shape of the medical device. For example, linear or quadratic interpolation can be used to determine the difference between the position sensor data and the virtual planned path of the airway. The technique can update the virtual target position in real time as the user (e.g., a physician) moves closer to the target tissue. Similarly, once the physician is sufficiently close to the virtual target, the virtual target position can be updated (e.g., by pressing a button or selecting a software tool).

[0062] For example, in some embodiments of method 300, the method further includes updating the virtual orientation of the target anatomical structure, which can include selecting data comprising the last one or more (n) vectors associated with the point closest to the target, fitting a curve to the selected data (where the curve includes linear and quadratic curves), and extrapolating the curve to estimate a new target location of the anatomical structure at x, y, z coordinate points. Furthermore, in these and other embodiments, updating the virtual orientation further includes applying weighted values ​​to one or more of the vectors.

[0063] Figure 4C yes Figure 4A and Figure 4B A schematic diagram of the virtual navigation image. As shown, the anatomical model 450 is in Figure 4C The anatomical model 450 has been updated to align with the position sensor data (e.g., point cloud 260). In other words, the anatomical model 450 has been updated to show a deformation of the anatomical pathway 152 in the distal region of the medical device system. The planned path 452 has also been updated to better align with the position sensor data. Furthermore, Figure 4C The predicted target bearing 445 is shown, representing the updated position of the virtual target bearing 455 along the target deviation vector 463. This is achieved through... Figure 4B The target deviation vector 463 is calculated using the plurality of deviation vectors 473 shown. For example, in some embodiments, it is based on the... Figure 4BThe target deviation vector 463 is calculated by averaging all deviation vectors 473 or subsets thereof. For example, all deviation vectors 473 or subsets thereof can be weighted to determine the target deviation vector 463. In some embodiments, the weighting of deviation vectors 473 can be based on the distance between a point on the end or tip of the elongated device 131 and a point on a deviation vector extending from the elongated device 131; for example, deviation vectors closer to the end or tip are weighted more. In these and other embodiments, the weighting of deviation vectors 473 can be based on the distance between a point on the virtual target orientation 455 and a corresponding deviation vector 473; for example, deviation vectors closer to the virtual target orientation 455 are weighted more. Such distance-based weighting can be assigned in groups (e.g., based on distance) or weighted individually.

[0064] Figure 5A Figure 510 is a schematic diagram showing various embodiments of the present technology, illustrating a graphical user interface (GUI) with features indicating deviation warnings. Figure 510 can be communicated by a display system (e.g., display system 610, hereinafter referred to as display system 610) that communicates with a medical device system (e.g., medical device system 604). Figure 6 (This will be discussed in more detail) For example... Figure 5A As shown, Figure 510 includes a real navigation image 570, a synthetic virtual navigation image 591 (also referred to as "synthetic virtual image 591"), a virtual navigation image 592, and a deviation warning GUI feature 585.

[0065] Figure 5A and Figure 5B The synthesized virtual image 591 is displayed in the image reference frame (X). I Y I Z I This includes image data from the anatomical region 150 (e.g., acquired by an imaging system). Figure 1 Anatomical model 550 generated by the position sensor system 608. Figure 1 The generated coordinate point cloud (e.g., Figure 2 The point cloud 260 is registered (i.e., dynamically associated) to display the medical device system 604 (e.g., elongated device 131) within the anatomical model 550 on the patient ( Figure 1 Characterization of tracking position, shape, posture, orientation, and / or motion within the body 504. In Figure 5A In the image, the anatomical model 550 of the patient's imaged anatomical region 150 includes a target 198, which is determined by preoperative images and shown relative to a virtual anatomical pathway 552 extending from the bronchus 557 to a deeper pathway 558. Figure 5BIn the process, the anatomical model 550 of the patient's imaged anatomical region 150 includes an updated target 598, which is determined by the implementation of process 322 and is also shown relative to the virtual anatomical pathway 552 extending from the bronchus 557 to the deeper pathway 558.

[0066] Deviation warning GUI feature 585 Figure 5A The composite virtual image 591 is shown in display figure 510 in the vicinity of the image, which can be implemented, for example, as an icon, dialog box, or indicator as described in the above reference. Figure 3 Other manifestations of the deviation identification described in processes 312-316. Figure 5A In the specific example shown, deviation warning GUI feature 585 (optionally) includes an updated registration prompt, allowing the user to indicate whether they want the system to update the registration of the medical device system according to procedures 318-322. The user can choose (a) interactive GUI feature 587a to update the registration (e.g., the registration of target anatomical feature 198 and / or planned path in display figure 510) or (b) interactive GUI feature 587b to not update the registration.

[0067] Figure 5B yes Figure 5A The exemplary display diagram 510 is shown, but it shows the display diagram after the user selects the interactive GUI feature 587a, which indicates an affirmative command to update the registration of the anatomical model with the position sensor data, the orientation of the target 198 and / or the planned path in the display diagram 510 (e.g., at least in the synthetic virtual navigation image 591).

[0068] Let's refer to each other. Figure 5A and Figure 5B Both, the actual navigation image 570 shows the distal portion 138 (from the elongated device 131) Figure 1 The actual patient anatomy observed from a distally directed viewpoint (e.g., the ridge 571 marking the branching points of the two anatomical pathways 152). For example, the actual navigation image 570 can be generated by one or more image acquisition devices 747 (e.g., embodiments of the endoscopic imaging system 609, such as combined...). Figure 6 (More specifically discussed) The data is collected and provided to the display system 610 so that it can be displayed on the display diagram 510 in real time or near real time.

[0069] In some embodiments, the synthesized virtual image 591 is controlled by the control system 612 ( Figure 6 Virtual visualization systems (e.g., combined with) Figure 6 The virtual visualization system 615, discussed in more detail, generates the image. The generation of the synthetic virtual image 591 can include using an image reference frame (X). I ,Y I ZI Registration to surgical reference frame (X) S ,Y S Z S ) and / or medical device reference system (X M ,Y M Z M For example, this registration can be achieved by rigidly and / or non-rigidly transforming the coordinates of the point cloud acquired by the position sensor system 608 (e.g., Figure 2 The coordinates of the point cloud 260 (262) can be rotated, translated, or otherwise manipulated to align the coordinates with the anatomical model 550. Registration between the image and the surgical / instrument reference frame can be achieved, for example, by using ICP technology or other point cloud registration techniques.

[0070] like Figure 5A and Figure 5B As further shown, virtual navigation image 592 illustrates the real navigation image 570 acquired by image acquisition device 747. Figure 1 The virtual patient anatomical structures are located in essentially the same orientation as the real anatomical pathway 152, such as virtual protuberances 501 (corresponding to real protuberances 571) marking the branch points of the two virtual anatomical pathways 552 (corresponding to real anatomical pathways 152) of the anatomical model 550. Therefore, the virtual navigation image 592 provides a rendered estimate of the patient anatomical structures visible to the image acquisition device 747 at a given orientation within the anatomical region 150. This is because the virtual navigation image 592 is at least partially based on the point cloud generated by the position sensor system 608 and the image cloud generated by the imaging system 618. Figure 1 The registration between the acquired image data provides insights into the accuracy of the registration between the virtual navigation image 592 and the real navigation image 570.

[0071] like Figure 5A and Figure 5B As further shown, the virtual navigation image 592 may optionally include a navigation path overlay 599 (e.g., a planned path). In some embodiments, the navigation path overlay 599 is used to assist the operator in navigating the medical device system 604 through the anatomical pathways of the anatomical region 150 to a target anatomical structure 198 within the patient's body. For example, the navigation path overlay 599 may show an "optimal" path through the anatomical region for the operator to follow, thereby delivering the distal portion of the elongated device 132 to the target orientation within the patient's body. In some embodiments, the navigation path overlay 599 may be aligned with a midline or other line along the corresponding anatomical pathway (e.g., its base).

[0072] B. Robots or remote operators used to implement image-body deviation recognition and mitigation technologies during medical procedures. Examples of medical systems

[0073] Figure 6This is a schematic diagram of a robot or remotely operated medical system 600 (“Medical System 600”) constructed according to various embodiments of the present technology. As shown, the Medical System 600 includes a manipulator assembly 602, (from...) Figure 1 The medical device system 604, main component 606, and control system 612 are described. Manipulator component 602 supports the medical device system 604 and drives it in the direction of the main component 606 and / or control system 612 to perform various medical procedures on a patient 603 located on an operating table 607 in a surgical environment 601. In this regard, the main component 606 typically includes one or more control devices that can be operated by an operator 605 (e.g., a physician) to control the manipulator component 602. Alternatively or additionally, the control system 612 includes a computer processor 614 and at least one memory 616 to enable control between the medical device system 604, the main component 606, and / or other components of the medical system 600. The control system 612 may also include programming instructions (e.g., a non-transitory computer-readable medium storing instructions) to implement one or more of the methods described herein, including instructions for providing information to a display system 610 and / or processing data to register the medical device system 604 with an anatomical model of the patient 603 (as described above). The manipulator component 602 can be a remotely operated, non-remotely operated, or hybrid remotely and non-remotely operated component. Therefore, all or part of the main component 606 and / or all or part of the control system 612 can be located inside or outside the surgical environment 601.

[0074] To assist operator 605 in controlling manipulator assembly 602 and / or medical device system 604 during image-guided medical procedures, medical system 600 may further include position sensor system 608, endoscopic imaging system 609, imaging system 618, and / or virtual visualization system 615. In some embodiments, position sensor system 608 includes orientation sensor system (e.g., electromagnetic (EM) sensor system) and / or shape sensor system for acquiring position sensor data (e.g., position, orientation, velocity, rate, attitude, shape, etc.) of medical device system 604. In these and other embodiments, endoscopic imaging system 609 includes one or more image acquisition devices (not shown) that record endoscopic image data including concurrent or real-time images (e.g., video, still images, etc.) of patient anatomy. Images acquired by endoscopic imaging system 609 may be, for example, two-dimensional or three-dimensional images of patient anatomy acquired by image acquisition devices located within patient 603, and are referred to as “real-world navigation images,” for example… Figure 5A and Figure 5B The actual navigation image shown is 570.

[0075] In some embodiments, the medical device system 604 may include components of the position sensor system 608 and / or components of the endoscopic imaging system 609. For example, components of the position sensor system 608 and / or components of the endoscopic imaging system 609 may be integrally or removably coupled to the medical device system 604. Alternatively or additionally, the endoscopic imaging system 609 may include a separate endoscope (not shown) attached to a separate manipulator assembly (not shown), which can be used in conjunction with the medical device system 604 to image patient anatomy. The position sensor system 608 and / or the endoscopic imaging system 609 may be implemented as hardware, firmware, software, or a combination thereof, which interacts with or is otherwise performed by one or more computer processors (e.g., one or more computer processors 614 of the control system 612).

[0076] The imaging system 618 of the medical system 600 can be positioned near the patient 603 in the surgical environment 601 to obtain real-time and / or near-real-time images of the patient 603 before, during, and / or after the medical procedure. In some embodiments, the imaging system 618 includes a mobile C-arm cone-beam computed tomography (CT) imaging system for generating three-dimensional images. For example, the imaging system 618 can include a Siemens DynaCT imaging system or other suitable imaging systems. In these and other embodiments, the imaging system 618 can include other imaging techniques, including magnetic resonance imaging (MRI), fluoroscopy, infrared thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, etc.

[0077] When the medical device system 604 is controlled during an image-guided medical procedure, the virtual visualization system 615 of the control system 612 provides navigation and / or interaction with anatomical structures to the operator 605. As described in more detail below, virtual navigation using the virtual visualization system 615 can be based at least in part on preoperative or intraoperative datasets obtained with reference to the anatomical pathways of the patient 603 (e.g., at least in part on data generated by the position sensor system 608, the endoscopic imaging system 609, and / or the imaging system 618). In some embodiments, for example, the virtual visualization system 615 processes preoperative and / or intraoperative image data of the anatomical regions of the patient 603 acquired by the imaging system 618 to generate an anatomical model (not shown) of the anatomical regions. The virtual visualization system 615 then registers the anatomical model to position sensor data generated by the position sensor system 608 and / or to endoscopic image data generated by the endoscopic imaging system 609, so as to (i) map the tracking position, orientation, posture, shape and / or motion of the medical device system 604 in the anatomical region to the correct position within the anatomical model, and / or (ii) determine a virtual navigation image of the virtual patient anatomy of the anatomical region viewed from the viewpoint of the medical device system 604 at an orientation within the anatomical model corresponding to the orientation of the medical device system 604 within the patient 603.

[0078] Display system 610 is capable of displaying various images or representations of patient anatomy and / or medical device system 604 generated by position sensor system 608, endoscopic imaging system 609, imaging system 618 and / or virtual visualization system 615. In some embodiments, display system 610 and / or main component 606 may be oriented such that operator 605 can control manipulator component 602, medical device system 604, main component 606 and / or control system 612 in a immersive manner.

[0079] As discussed above, the manipulator assembly 602 drives the medical device system 604 in the direction of the main assembly 606 and / or the control system 612. In this regard, the manipulator assembly 602 can include selectable degrees of freedom of motion that can be motorized and / or remotely operated, as well as selectable degrees of freedom of motion that can be non-motorized and / or non-remotely operated. For example, the manipulator assembly 602 can include multiple actuators or motors (not shown) that drive inputs on the medical device system 604 in response to commands received from the control system 612. The actuators can include drive systems (not shown) that, when coupled to the medical device system 604, can advance the medical device system 604 into a natural or surgically generated anatomical opening. Other drive systems can move distal portions (not shown) of the medical device system 604 with multiple degrees of freedom, which may include three linear movements (e.g., linear movements along the X, Y, Z Cartesian axes) and three rotational movements (e.g., rotations about the X, Y, Z Cartesian axes). Alternatively or concurrently, the actuator can be used to actuate the articulated end effector of the medical device system 604 (e.g., for gripping tissue in the jaws of a biopsy device and / or similar).

[0080] Figure 7 yes Figure 6 A schematic diagram of the manipulator assembly 602, medical device system 604, and imaging system 618 within a surgical setting 601, constructed according to various embodiments of the present technology. Figure 7 As shown, the surgical environment 601 has a surgical reference frame (X). S ,Y S Z S In this system, patient 603 is positioned on operating table 607, and medical device system 604 has a medical device reference frame (X) within surgical environment 601. M ,Y M Z M During the medical procedure, patient 603 can remain still within surgical environment 601, meaning that overall patient movement can be limited by sedation, restraint, and / or other means. In these and other embodiments, circulatory anatomical movements (including respiratory and cardiac movements) of patient 603 can continue unless patient 603 is instructed to hold his or her breath to temporarily suspend respiratory movements.

[0081] The manipulator assembly 602 includes an instrument trolley 726 mounted to the insertion stage 728. In the illustrated embodiment, the insertion stage 728 is linear, while in other embodiments, the insertion stage 728 is curved or a combination of curved and linear segments. In some embodiments, the insertion stage 728 is fixed within the surgical environment 601. Alternatively, the insertion stage 728 can be movable within the surgical environment 601, although having a known orientation within the surgical environment 601 (e.g., via a tracking sensor (not shown) or other tracking device). In these alternatives, a medical device reference frame (X) is used. M ,Y M Z M ) relative to the surgical reference frame (X S ,Y S Z S () is fixed or otherwise known.

[0082] Figure 7 The medical device system 604 includes an elongated device 731 (e.g., corresponding to...). Figure 1 The device comprises at least a portion of an elongated device 131, a medical device 732, an instrument body 735, a position sensor system 608, and an endoscope imaging system 609. In some embodiments, the elongated device 731 is a flexible catheter or other biomedical device defining a channel or lumen 744. The size and shape of the channel 744 can be configured to receive the medical device 732 (e.g., via the proximal end 736 and / or instrument port (not shown) of the elongated device 731) and facilitate delivery of the medical device 732 to the distal portion 738 of the elongated device 731. The elongated device 731 is coupled to the instrument body 735, which is in turn coupled to and secured relative to the instrument trailer 726 of the manipulator assembly 602.

[0083] In operation, the manipulator assembly 602 is capable of controlling the insertion movement of the elongated device 731 into the patient 603 via a natural or surgically created anatomical opening (e.g., proximal and / or distal movement along axis A) to facilitate navigation of the elongated device 731 through anatomical pathways of the patient 603's anatomical region and / or to facilitate delivery of the distal portion 738 of the elongated device 731 to or near a target location within the patient 603. For example, the instrument trolley 726 and / or insertion stage 728 may include actuators (not shown), such as servo motors, which facilitate control of the movement of the instrument trolley 726 along the insertion stage 728. Alternatively or additionally, in some embodiments, the manipulator assembly 602 is capable of controlling the movement of the distal portion 738 of the elongated device 731 in multiple directions (including yaw, pitch, and roll rotation directions) (e.g., to navigate the patient's anatomy). For this purpose, the elongated device 731 may house or include cables, linkages, and / or other steering controls (not shown) that the manipulator assembly 602 can use to controllably bend the distal portion 738 of the elongated device 731. For example, the elongated device 731 may house at least four cables that can be used by the manipulator assembly 602 to provide (i) independent "up and down" steering to control the pitch of the distal portion 738 of the elongated device 731 and (ii) independent "left and right" steering of the elongated device 731 to control the yaw of the distal portion 738 of the elongated device 731.

[0084] Medical device 732 of medical device system 604 can be used for medical purposes, such as for anatomical pathway investigation, surgery, biopsy, ablation, illumination, irrigation, and / or aspiration. Therefore, medical device 732 can include image acquisition probes, biopsy instruments, laser ablation fibers, and / or other surgical, diagnostic, and / or therapeutic tools. For example, medical device 732 can include an endoscope or other biomedical device having one or more image acquisition devices 747, wherein the image acquisition devices are positioned at a distal portion 737 of medical device 732 and / or along other orientations of medical device 732. In these embodiments, while medical device 732 is within the anatomical region of patient 603, image acquisition devices 747 can acquire one or more real navigation images or videos (e.g., a sequence of one or more real navigation image frames) of the anatomical pathway and / or other real patient anatomy.

[0085] As discussed above, the medical device 732 can be deployed and / or delivered to a target location within the patient 603 via the channel 744 defined by the elongated device 731. In embodiments where the medical device 732 includes an endoscope or other biomedical device having an image acquisition device 747 on its distal portion 737, the image acquisition device 747 can advance to the distal portion 738 of the elongated device 731 before, during, and / or after the manipulator assembly 602 navigates the distal portion 738 of the elongated device 731 to the target location within the patient 603. In these embodiments, the medical device 732 can be used as an investigation instrument to acquire realistic navigation images of anatomical pathways and / or other real patient anatomy structures, and / or to assist an operator (not shown) in navigating the distal portion 738 of the elongated device 731 through anatomical pathways to the target location.

[0086] As another example, after the manipulator assembly 602 positions the distal portion 738 of the elongated device 731 toward a target location within the patient 603, the medical device 732 can advance beyond the distal portion 738 of the elongated device 731 to perform a medical procedure at the target location. Continuing this example, after all or part of the medical procedure has been completed at the target location, the medical device 732 can retract into the elongated device 731 and, alternatively, be removed from the proximal end 736 of the elongated device 731 or from another instrument port (not shown) along the elongated device 731.

[0087] like Figure 7 As shown, the position sensor system 608 of the medical device system 604 includes a shape sensor 733 and a position measuring device 739. In these and other embodiments, the position sensor system 608 may include other position sensors (e.g., accelerometers, rotary encoders, etc.) that supplement or replace the shape sensor 733 and / or the position measuring device 739.

[0088] The shape sensor 733 of the position sensor system 608 includes an optical fiber extending within and aligned with an elongated device 731. In one embodiment, the optical fiber of the shape sensor 733 has a diameter of approximately 200 μm. In other embodiments, the diameter of the optical fiber may be larger or smaller. The optical fiber of the shape sensor 733 forms an optical fiber bending sensor, which is used to determine the shape, orientation, and / or attitude of the elongated device 731. In some embodiments, an optical fiber with a Bragg fiber grating (FBG) can be used to provide strain measurements of the structure in one or more dimensions. Various systems and methods for monitoring the shape and relative position of optical fibers in three dimensions are further described in detail in U.S. Patent Publication No. 2006 / 0013523 (filed July 13, 2005) (disclosing an optical fiber position and shape sensing device and related methods thereof); U.S. Patent No. 7,781,724 (filed September 26, 2006) (disclosing an optical fiber position and shape sensing device and related methods thereof); U.S. Patent No. 7,772,541 (filed March 12, 2008) (disclosing optical fiber position and / or shape sensing based on Rayleigh scattering); and U.S. Patent No. 6,389,187 (filed June 17, 1998) (disclosing an optical fiber bending sensor), the entire contents of which are incorporated herein by reference. In these and other embodiments, the sensor of this technology may use other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. In these and still other embodiments, the shape of the elongated device 731 may be determined by using other techniques. For example, the history of the orientation of the distal portion 738 of the elongated device 731 can be used to reconstruct the shape of the elongated device 731 over a time interval.

[0089] In some embodiments, the shape sensor 733 is fixed at a proximal point 734 on the device body 735 of the medical device system 604. In operation, for example, the shape sensor 733 measures in a medical device reference frame (X). M ,Y M Z M The shape sensor 733 is shaped from the proximal point 734 to another point along the optical fiber (e.g., the distal portion 738 of the elongated device 731). The proximal point 734 of the shape sensor 733 may be movable with the device body 735, but the orientation of the proximal point 734 may be known (e.g., via a tracking sensor (not shown) or other tracking device).

[0090] As the device body 735 moves along the insertion axis A on the insertion stage 728 of the manipulator assembly 602, the position measuring device 739 of the position sensor system 608 provides information about the position of the device body 735. In some embodiments, the position measuring device 739 includes a resolver, an encoder, a potentiometer, and / or other sensors that determine the rotation and / or orientation of an actuator (not shown) that controls the movement of the device trailer 726 of the manipulator assembly 602 and thus the movement of the device body 735 of the medical device system 604.

[0091] C. Example

[0092] Several aspects of the present technology are illustrated in the examples below. Although several aspects of the present technology are illustrated in examples relating to systems, computer-readable media, and methods, any of these aspects of the present technology can be similarly used in other embodiments in examples relating to any system, computer-readable medium, and method.

[0093] 1. A system for determining the deviation between an anatomical region and an anatomical model of the anatomical region, the system comprising:

[0094] Medical devices including sensors, wherein the medical device can be inserted into a patient's body; and

[0095] A computing device communicating with a medical device, the computing device including a processor and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations, including:

[0096] Simultaneously with the insertion of the medical device into the patient's anatomical region and after the medical device has been registered to an anatomical model of the anatomical region, sensor data acquired by the sensors of the medical device is received, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to the anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device.

[0097] Compare the sensor data with the corresponding part of the virtual path.

[0098] Based at least in part on comparison, a deviation grading device is generated to indicate the deviation of anatomical regions from anatomical models, and

[0099] A warning is generated when the deviation grader exceeds a predetermined threshold.

[0100] 2. The system according to Example 1, wherein generating a warning includes displaying a graphical user interface on a display communicating with a computing device, wherein the graphical user interface includes one or both of graphical and textual information indicating a determination of deviation between the anatomical region and the anatomical model.

[0101] 3. The system according to Example 2, wherein the graphical user interface includes prompts to command the system to update the orientation of the anatomical region within the anatomical model.

[0102] 4. The system according to any one of Examples 1-3, wherein the operation further includes updating the virtual orientation of the anatomical region with respect to the anatomical model while the medical device is within the anatomical region.

[0103] 5. The system according to Example 4, wherein the generated deviation grader indicates the deviation between the anatomical structure of interest and the anatomical model, and wherein the operation further includes updating the virtual orientation of the anatomical structure of interest while the medical device is within the anatomical region.

[0104] 6. In the system according to Example 5, updating the virtual orientation of the anatomical structure includes:

[0105] Calculate one or more deviation vectors between the corresponding portion of the virtual path and at least one portion of the medical device.

[0106] Select one or more of the deviation vectors that are associated with one or more locations along at least a portion of the medical device that are closest to the anatomical structure.

[0107] Fit the curve to one or more selected deviation vectors, and

[0108] Extrapolation curves are used to estimate the new target location of the anatomical structure at x, y, z coordinates.

[0109] 7. The system according to Example 6, wherein updating the virtual orientation of the anatomical structure further includes applying a weighting value to at least one of the one or more deviation vectors.

[0110] 8. A system according to any one of Examples 1-7, wherein the comparison includes:

[0111] Determine the offset between the corresponding part of the virtual path and the medical device.

[0112] 9. The system according to Example 8, wherein determining the offset includes generating one or more offset vectors pointing from a corresponding portion of a virtual path to a medical device, and wherein the one or more offset vectors are point-matched.

[0113] 10. The system according to any one of Examples 1-9, wherein the operation further comprises analyzing sensor data to determine the shape of said at least a portion of the medical device.

[0114] 11. The system according to Example 10, wherein comparing sensor data with a corresponding portion of the virtual path includes comparing the shape of said at least a portion of the medical device with a portion of the virtual path.

[0115] 12. The system according to Example 11, wherein the anatomical region includes an anatomical pathway, and wherein determining the offset includes measuring the magnitude and direction of deformation of the anatomical pathway to determine a plurality of deformation vectors, and comparing the shape of said at least a portion of the medical device in real time with a virtual path of the anatomical pathway predetermined from previously acquired image data based on the deformation vectors.

[0116] 13. The system according to any one of Examples 1-12, wherein the deviation grader includes a device-path distance parameter, which includes the average distance between the last region of the portion of the virtual path and the distal portion of the medical device.

[0117] 14. A system according to any one of Examples 1-13, wherein sensor data acquired by the sensors of the medical device is associated with one or more of the position, orientation, velocity, attitude and / or shape of the medical device.

[0118] 15. The system according to any one of Examples 1-14, wherein the medical device includes a catheter, and wherein the sensor includes a shape sensor comprising an optical fiber extending within and aligned with an elongated portion of the catheter.

[0119] 16. The system according to Example 15, wherein a plurality of points associated with the shape of the medical device are determined by points sampled by the shape sensor.

[0120] 17. The system according to any one of Examples 1-16, wherein the medical device includes a catheter, and wherein the sensor includes an electromagnetic (EM) sensor located at the distal end or tip of the catheter.

[0121] 18. The system according to Example 17, wherein multiple points associated with the shape of the medical device are determined by multiple individual points at the distal or tip of the catheter measured by an EM sensor as the catheter is driven through an anatomical pathway.

[0122] 19. The system according to any one of Examples 1-18, wherein the sensors of the medical device are configured to generate one or both of position sensor data and motion sensor data during data sampling of an anatomical region of a patient, and wherein said operation further comprises:

[0123] The medical device identifies anatomical landmarks within the patient's anatomical area while navigating it.

[0124] A comparison step is performed to compare sensor data with corresponding portions of the virtual path associated with anatomical landmarks, and

[0125] The registration of medical devices is updated based at least in part on comparative sensor data and anatomical landmarks.

[0126] 20. A non-transitory computer-readable medium having instructions stored thereon, wherein when executed by one or more processors of a computing system, the instructions cause the computing system to perform operations, the operations comprising:

[0127] Simultaneously with the insertion of the medical device into the patient's anatomical region and after the medical device has been registered to an anatomical model of the anatomical region, sensor data acquired by the sensors of the medical device is received, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to the anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device.

[0128] Compare the sensor data with the corresponding part of the virtual path.

[0129] Based at least in part on comparison, a deviation grading device is generated to indicate the deviation of anatomical regions from anatomical models, and

[0130] D. Summary

[0131] The systems and methods described herein can be provided in the form of a tangible and non-transitory machine-readable medium or media (e.g., hard disk drives, hardware memory, etc.) having instructions recorded thereon that are executed by a processor or computer. The set of instructions can include various commands that instruct a computer or processor to perform specific operations, such as the methods and processes of the various embodiments described herein. The set of instructions can be in the form of a software program or application. Computer storage media can include volatile and non-volatile media, as well as removable and non-removable media, for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media can include, but is not limited to, RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, disk storage, or any other hardware medium capable of storing desired information and accessible by system components. Components of the system can communicate with each other via wired or wireless communication. Components can be separate from each other, or various combinations of components can be integrated together into a monitor or processor or housed within a workstation having standard computer hardware (e.g., processors, circuits, logic circuits, memory, etc.). The system can include processing devices such as microprocessors, microcontrollers, integrated circuits, control units, storage media, and other hardware.

[0132] Although many embodiments have been described above in the context of navigating and performing medical procedures within a patient's lungs, other applications and embodiments besides those described herein are also within the scope of this technology. For example, unless otherwise specified or obvious from the context, the devices, systems, methods, and computer program products of this technology can be used for a variety of image-guided medical procedures, such as those performed on, in, or near adjacent hollow patient anatomy structures, or to otherwise treat tissues within and / or proximal to hollow patient anatomy structures. Thus, for example, the systems, devices, methods, and computer program products of this disclosure can be used in one or more medical procedures associated with other patient anatomy structures, such as a patient's bladder, urethra, GI system, and / or heart.

[0133] As used herein, the term "operator" should be understood to include any type of person who may be performing or assisting a medical procedure, and therefore includes physicians, surgeons, doctors, nurses, medical technicians, other persons or users of the technology disclosed herein, and any combination thereof. Alternatively or additionally, the term "patient" should be considered to include human and / or non-human (e.g., animal) patients who are undergoing a medical procedure.

[0134] As will be appreciated from the foregoing, for illustrative purposes, specific embodiments of the present technology have been described herein, but well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of embodiments of the present technology. This disclosure maintains control to the extent that any material incorporated herein by reference conflicts with this disclosure. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, unless the word “or” is explicitly limited to referring to only a single item and excluding other items in a list of two or more items, its use in such a list shall be construed as including (a) any single item in the list, (b) all items in the list, or (c) any combination of items in the list. As used herein, the phrase “and / or” in “A and / or B” refers to A alone, B alone, and both A and B. Where the context permits, singular or plural terms may also include plural or singular terms, respectively. Furthermore, the terms “comprising,” “including,” “having,” and “with” throughout mean at least including the stated features, thereby not excluding any further number of the same features and / or features of other types.

[0135] Furthermore, as used herein, the term "substantially" refers to the extent or degree of completeness or near-completeness of an action, characteristic, nature, state, structure, item, or result. For example, an object "substantially" closed means that the object is either completely closed or almost completely closed. In some cases, the exact permissible deviation from absolute completeness may depend on the specific context. However, in general, the degree of near-completeness will be the same as the overall result of achieving absolute or complete completion. The use of "substantially" also applies when used in a negative sense, referring to an action, characteristic, nature, state, structure, item, or result that is not completed or is close to being incomplete.

[0136] The above detailed description of embodiments of this technology is not intended to be exhaustive or to limit the technology to the precise forms disclosed above. Although specific embodiments and examples of this technology have been described above for illustrative purposes, various equivalent modifications can be made within the scope of this technology, as will be recognized by those skilled in the art. For example, while the steps are presented in a given order, alternative embodiments may perform the steps in a different order. As another example, various components of this technology can be further divided into sub-components, and / or various components and / or functions of this technology can be combined and / or integrated. Furthermore, while advantages associated with certain embodiments of this technology have been described in the context of some embodiments, other embodiments may also present such advantages, and not all embodiments must present such advantages to fall within the scope of this technology.

[0137] It should also be noted that other embodiments besides those disclosed herein are also within the scope of this technology. For example, embodiments of this technology may have different constructions, components, and / or processes than those shown or described herein. Moreover, those skilled in the art will understand that these and other embodiments may be without certain constructions, components, and / or processes shown or described herein without departing from this technology. Therefore, this disclosure and related technologies may include other embodiments not expressly shown or described herein.

Claims

1. A system for determining the deviation of an anatomical region from an anatomical model of the anatomical region, the system comprising: A medical device including sensors, wherein the medical device is insertable into a patient's body; and A computing device communicating with the medical device, the computing device including a processor and a memory, the memory being coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations, the operations including: Simultaneously with the insertion of the medical device into the patient's anatomical region and after the medical device has been registered to an anatomical model of the anatomical region, sensor data acquired by the sensors of the medical device is received, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to the anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device. The sensor data is compared with the corresponding portion of the virtual path. Based at least in part on the comparison, a deviation grader is generated to indicate the deviation of the anatomical region from the anatomical model, wherein the deviation grader is generated based on one or more weighted deviation vectors, and A warning is generated when the deviation grader exceeds a predetermined threshold.

2. The system of claim 1, wherein generating the warning includes displaying a graphical user interface on a display in communication with the computing device, wherein the graphical user interface includes one or both of graphical and textual information indicating a determination of the deviation between the anatomical region and the anatomical model.

3. The system of claim 2, wherein the graphical user interface includes prompts to command the system to update the orientation of the anatomical region within the anatomical model.

4. The system of claim 1, wherein the operation further comprises updating the virtual orientation of the anatomical region with respect to the anatomical model while the medical device is within the anatomical region.

5. The system of claim 4, wherein the generated deviation grader indicates the deviation of the anatomical structure of interest from the anatomical model, and wherein the operation further includes updating the virtual orientation of the anatomical structure of interest while the medical device is within the anatomical region.

6. The system of claim 5, wherein updating the virtual orientation of the anatomical structure comprises: Calculate one or more deviation vectors between the corresponding portion of the virtual path and at least one portion of the medical device. Select one or more of the deviation vectors that are closest to the anatomical structure along at least a portion of the medical device. Fit the curve to one or more selected deviation vectors, and Extrapolate the curve to estimate the new target position of the anatomical structure at x, y, z coordinates.

7. The system of claim 6, wherein updating the virtual orientation of the anatomical structure further comprises applying a weighting value to at least one of the one or more deviation vectors to generate the one or more weighted deviation vectors.

8. The system of claim 1, wherein the comparison comprises: Determine the offset between the corresponding portion of the virtual path and the medical device.

9. The system of claim 8, wherein determining the offset includes generating one or more deviation vectors pointing from the corresponding portion of the virtual path to the medical device, and wherein the one or more deviation vectors are point-matched.

10. The system of claim 1, wherein the operation further comprises analyzing the sensor data to determine the shape of the at least portion of the medical device.

11. The system of claim 10, wherein comparing the sensor data with the corresponding portion of the virtual path comprises comparing the shape of the at least portion of the medical device with a portion of the virtual path.

12. The system of claim 11, wherein the anatomical region includes an anatomical pathway, and wherein comparing the sensor data with the corresponding portion of the virtual path further includes: Determine the offset between the corresponding portion of the virtual path and the medical device, wherein determining the offset includes measuring the magnitude and direction of the deformation of the anatomical pathway to determine a plurality of deviation vectors, and comparing the shape of the at least portion of the medical device in real time with the virtual path of the anatomical pathway predetermined from the previously obtained image data based on the deviation vectors.

13. The system of claim 1, wherein the deviation grader includes a device-path distance parameter comprising the average distance between the final region of the portion of the virtual path and the distal portion of the medical device.

14. The system of claim 1, wherein the sensor data acquired by the sensor of the medical device is associated with one or more of the position, orientation, velocity, attitude and / or shape of the medical device.

15. The system of claim 1, wherein the medical device comprises a catheter, and wherein the sensor comprises a shape sensor comprising an optical fiber extending within and aligned with an elongated portion of the catheter.

16. The system of claim 15, wherein a plurality of points associated with the shape of the medical device are determined by points sampled by the shape sensor.

17. The system of claim 1, wherein the medical device comprises a catheter, and wherein the sensor comprises an electromagnetic sensor, i.e., an EM sensor, located at the distal end or tip of the catheter.

18. The system of claim 17, wherein a plurality of points associated with the shape of the medical device are determined by a plurality of individual points at the distal or tip of the catheter measured by the EM sensor as the catheter is driven through an anatomical pathway.

19. The system of claim 1, wherein the sensor of the medical device is configured to generate one or both of position sensor data and motion sensor data during data sampling of the anatomical region of the patient, and wherein the operation further comprises: The medical device identifies anatomical landmarks within the patient's anatomical region while navigating within that region. The comparison step is performed to compare the sensor data with the corresponding portion of the virtual path associated with the anatomical landmark, and The registration of the medical device is updated based at least in part on the comparison of sensor data with the anatomical landmarks.

20. A non-transitory computer-readable medium having instructions stored thereon, wherein when executed by one or more processors of a computing system, the instructions cause the computing system to perform operations, the operations comprising: Simultaneously with the insertion of the medical device into the patient's anatomical region and after the medical device has been registered to an anatomical model of the anatomical region, sensor data acquired by the sensors of the medical device is received, wherein the anatomical model is based on previously acquired image data of the anatomical region and includes a virtual path extending through the anatomical model to the anatomical structure of interest, and wherein the sensor data indicates the orientation of at least a portion of the medical device. The sensor data is compared with the corresponding portion of the virtual path. Based at least in part on the comparison, a deviation grader is generated to indicate the deviation of the anatomical region from the anatomical model, wherein the deviation grader is generated based on one or more weighted deviation vectors, and A warning is generated when the deviation grader exceeds a predetermined threshold.

Images