Method and system for guiding device insertion during medical imaging

Abstract

Various methods and systems for medical imaging are provided. In one embodiment, a method for an interventional imaging procedure includes: identifying a medical device during insertion of the medical device into a subject based on real-time images of the insertion; inferring a trajectory of the medical device during the insertion in real-time based on the real-time images of the insertion; and displaying the inferred trajectory of the medical device on the real-time images.

Description

Technical Field

[0001] The embodiments of the subject matter disclosed herein relate to medical imaging, and more specifically to X-ray fluorescence imaging. Background Technology

[0002] Non-invasive imaging techniques allow for the acquisition of images of the internal structure or features of a patient or object without the need for invasive procedures. Specifically, such non-invasive imaging techniques rely on various physical principles (such as differential transmission of X-rays through a target volume or reflection of sound waves) to acquire data and construct images or otherwise represent the observed internal features of a patient or object.

[0003] For example, in fluoroscopy and other X-ray-based imaging techniques such as computed tomography (CT), X-ray radiation is directed at the subject, typically a patient in medical diagnostic applications, packaging or luggage in safety screening applications, or a manufactured part in industrial quality control or inspection applications. A portion of the radiation strikes a detector, where image data is collected and used in the image generation process. In images generated by such systems, it is possible to identify and examine internal structures and organs within a patient's body, objects within packaging or containers, or defects (e.g., cracks) within manufactured parts.

[0004] In certain situations, such as in fluoroscopic examinations used to support interventional or navigation procedures, X-rays can be acquired at high frame rates over extended periods to provide real-time image data that can be used to guide or navigate tools within the patient's body. Complementarily, cone-beam computed tomography (CBCT) can be used for interventional X-ray-guided needle procedures, preoperative three-dimensional (3D) imaging, and / or intraoperative 3D imaging. Summary of the Invention

[0005] In one aspect, a method for interventional imaging procedures includes: identifying the medical device based on real-time images of the insertion during insertion into a subject; inferring the trajectory of the medical device during insertion in real time based on the real-time images of the insertion; and displaying the inferred trajectory of the medical device on the real-time images. In this way, medical devices (e.g., needles) can be inserted with increased accuracy and reduced operator workload.

[0006] It should be understood that the above brief description is provided to introduce selected concepts further described in the detailed embodiments in a simplified form. This is not intended to identify key or essential features of the claimed subject matter, the scope of which is uniquely defined by the claims following the detailed embodiments. Furthermore, the claimed subject matter is not limited to embodiments that address any shortcomings mentioned above or in any part of this disclosure. Attached Figure Description

[0007] The invention will be better understood by referring to the following description of non-limiting embodiments, in which:

[0008] Figure 1 A drawing view of an imaging system according to one embodiment is shown;

[0009] Figure 2 A block diagram of an exemplary image processing system for segmenting and analyzing images according to one embodiment is shown;

[0010] Figure 3 This is a flowchart illustrating a method for dynamic trajectory enhancement during an interventional imaging procedure according to one embodiment;

[0011] Figure 4 A first example of displaying the device's real-time estimated trajectory during insertion is shown; and

[0012] Figure 5 A second example of a device displaying a real-time estimated trajectory during insertion is shown according to one implementation scheme. Detailed Implementation

[0013] Now refer to Figures 1 to 5 Embodiments of this disclosure are described by way of example and relate to various embodiments for providing enhanced trajectory of a medical device in real time via real-time medical imaging during insertion of a medical device. During interventional imaging procedures (such as surgical procedures monitored / assisted by fluoroscopic imaging), continuous real-time X-ray images of the patient can be displayed, allowing clinicians to monitor the movement of an inserted medical device (also referred to herein as "device") such as a needle, surgical instrument, or endoscope relative to anatomical features. The acquired X-ray images are two-dimensional (2D) conic projections.

[0014] In some procedures, a three-dimensional (3D) model can be generated and overlaid on a real-time 2D X-ray image. The 3D model can provide a guided workflow for inserting a medical device. For example, the 3D model can indicate the entry point and path for inserting the medical device into a target location (e.g., an anatomical region or structure). While a guided workflow can predict the positioning of the medical device based on the performed procedure and the inherent insertion characteristics of the medical device, it does not predict or accommodate deviations of the medical device from the proposed entry point and path. Furthermore, a guided workflow does not provide the clinician performing the insertion with real-time feedback on the actual trajectory of the medical device within the patient. For example, even with a guided workflow, precise positioning of the medical device can be challenging. Starting from the puncture site on the patient, clinicians may find it difficult to predict whether the angle taken by the medical device will reach the target location. Additionally, the patient's respiratory movements can further complicate precise positioning. This can be particularly challenging during the insertion of a flexible device that can deform and deflect during puncture.

[0015] Therefore, according to the embodiments disclosed herein, the virtual estimated trajectory of the device from the puncture site to the target location during the procedure can provide feedback and additional guidance to the clinician. For example, the medical device can be identified by automatically analyzing real-time X-ray images in real time via segmentation, and the trajectory within the patient's body can be inferred. For example, the inference can take into account the location of the distal portion of the medical device, the characteristics of the medical device (such as length, thickness, shape, stiffness, or flexibility), and the type of procedure being performed.

[0016] By providing real-time virtual tracking of medical devices, placement becomes easier and faster, reducing the time spent performing procedures and decreasing their complexity and potential complications. Consequently, clinicians can perform more procedures in a given day. Furthermore, reduced procedure time increases patient comfort. Moreover, the shorter imaging duration reduces the radiation dose delivered to the patient.

[0017] exist Figure 1 The image shows a projection imaging system that can be used to acquire medical images of a region of interest. Figure 2 An exemplary image processing system is illustrated, which can be used to determine the real-time trajectory of a device (e.g., a needle) based on real-time images acquired during device insertion. This image processing system may employ image processing algorithms to segment the device, locate the device, and, based on... Figure 3 The method shown infers the trajectory of the device. Figure 4 and Figure 5 An example is provided that displays the real-time trajectory of a needle on a real-time medical image.

[0018] Now turn to the attached diagram. Figure 1An exemplary embodiment of an imaging system 10 for acquiring and processing image data is schematically illustrated. In the illustrated embodiment, the imaging system 10 is a digital X-ray system designed to both acquire raw image data and process the image data for display. The imaging system 10 can be a fixed or mobile X-ray system. Figure 1 In the embodiments shown, imaging system 10 is depicted as a C-arm fluorescein fluoroscopic imaging system, but it will be understood that other forms of imaging and / or navigation systems may be used within the scope of this disclosure. For example, it will be understood that the techniques of this invention may also be useful when applied to images acquired using other imaging modalities, such as standard, non-fluorescein fluoroscopic X-ray imaging, tomography, etc. The inventive discussion of fluorescein fluoroscopic imaging modalities provides only as examples of suitable imaging modalities. For example, imaging system 10 may be any imaging system that acquires two-dimensional images (e.g., slices or projections) of a three-dimensional object.

[0019] Imaging system 10 can acquire X-ray attenuation data from various angles around a patient and is suitable for tomographic reconstruction. Imaging system 10 includes an X-ray source 56 fixed to a C-arm 14. Exemplarily, X-ray source 56 may be an X-ray tube, a distributed X-ray source (such as a solid-state or thermionic X-ray source), or any other X-ray radiation source suitable for acquiring medical or other images. X-ray source 56 may also be referred to as a radiation source. For example, X-ray source 56 may include an X-ray generator and an X-ray tube. X-ray source 56 emits X-ray radiation 16 from focal point 12 in the direction of subject (or object) 18. For example, subject 18 may be a patient. In the depicted embodiment, X-ray radiation 16 is emitted in a cone shape (e.g., a cone beam). The X-ray cone beam passes through the imaging volume of subject 18. An incident portion of X-ray radiation 16 (also referred to as incident X-ray) 20 passes through or surrounds subject 18 and collides (or impacts) with X-ray detector 34, which includes detector array 22. X-ray detector 34 may also be referred to as a radiation detector. In this example, the x-ray detector 34 is a digital x-ray detector and may be portable or permanently mounted to the imaging system 10. In some embodiments, the detector array 22 may convert incident x-ray photons into detected lower-energy photons. Electrical signals are generated in response to the detected photons, and these signals are processed to reconstruct images of features (e.g., anatomical features) within the subject 18. The x-ray source 56 and the x-ray detector 34 together comprise an x-ray imaging chain.

[0020] As an example, detector array 22 may include one or more complementary metal-oxide-semiconductor (CMOS) optical imager panels, each CMOS optical imager panel individually defining an array of detector elements (e.g., pixels). Each detector element generates an electrical signal representing the intensity of the X-ray beam incident at the location of the detector element when the beam illuminates detector array 22. This signal may be digitized and transmitted to a monitor / display device for display.

[0021] Exemplarily, the x-ray source 56 and the x-ray detector 34 are controlled by a system controller 24, which simultaneously provides power and control signals for the operation of the imaging system 10. The system controller 24 may control the x-ray source 56 via an x-ray controller 26, which may be a component of the system controller 24. In such embodiments, the x-ray controller 26 may be configured to provide power and timing signals to the x-ray source 56.

[0022] Exemplarily, the x-ray detector 34 is further connected to a system controller 24. The system controller 24 controls the acquisition of signals generated in the x-ray detector 34 (e.g., acquired by the detector array 22). In an exemplary embodiment, the system controller 24 uses a data acquisition system (DAS) 28 to acquire the signals generated by the detector array 22. The DAS 28 receives data collected by the readout electronics of the x-ray detector 34. The DAS 28 may receive sampled analog signals from the x-ray detector 34 and convert the data into digital signals for subsequent processing by the processor 30, which is discussed in further detail herein. Alternatively, in other embodiments, the digital-to-analog conversion may be performed by a circuitry disposed on the x-ray detector 34 itself. The system controller 24 may also perform various signal processing and filtering functions with respect to the acquired image signals, such as, but not limited to, initial adjustments for dynamic range and digital image data interleaving.

[0023] Furthermore, the x-ray detector 34 includes or communicates with a control circuitry system in or with a system controller 24, which commands the acquisition of signals generated in the detector array 22. The x-ray detector 34 may communicate with the system controller 24 via any suitable wireless communication or via cable or other mechanical connection. Alternatively, operating commands may be implemented within the x-ray detector 34 itself.

[0024] System controller 24 is further operatively connected to C-arm 14 and to stage 32 configured to support subject 18. Motor controller 36 of system controller 24 provides instructions and commands to the mechanical components of C-arm 14 and stage 32 to perform their linear and / or rotational movements. The linear and / or rotational movements of C-arm 14 enable x-ray source 56 and x-ray detector 34 to rotate one or more revolutions around subject 18, such as rotating primarily in the XY plane or at an angle relative to the subject. The distance between x-ray detector 34 and x-ray source 56 can also be adjusted. Furthermore, stage 32 supporting subject 18 can be longitudinally moved relative to the movement of C-arm 14 and / or the planned movement of C-arm 14 to position the patient within the imaging field of view of imaging system 10. Therefore, movement of the patient and / or components of the imaging system for adjusting the imaging field of view may include movement of one or both of C-arm 14 and stage 32.

[0025] Generally, system controller 24 commands the operation of imaging system 10 (such as via the operation of x-ray source 56, x-ray detector 34, and the aforementioned positioning system) to execute the examination plan and process the acquired data. For example, via the aforementioned system and controller, system controller 24 can rotate the gantry supporting x-ray source 56 and x-ray detector 34 around the region of interest or target T, so that x-ray attenuation data can be obtained at various views relative to target T. For example, the central axis 52 of x-ray radiation 16 can be focused on target T. In this example, system controller 24 may also include signal processing circuitry, associated memory circuitry for storing computer-executable programs and routines (such as routines for executing the image processing techniques described herein), configuration parameters, image data, and so on.

[0026] In the depicted embodiment, image signals acquired and processed by system controller 24 are provided to processor 30 for image reconstruction. Processor 30 may be one or more conventional microprocessors. Data collected by DAS 28 may be transferred directly to processor 30 or transferred after being stored in memory 38. Any type of memory suitable for storing data may be utilized by imaging system 10. For example, memory 38 may include one or more optical, magnetic, and / or solid-state memory storage structures. Furthermore, memory 38 may be located at the acquisition system site and / or may include a remote storage device for storing data, processing parameters, and / or routines for image reconstruction, as described below. An example of image reconstruction may include cone-beam computed tomography (CBCT), in which images acquired at multiple angles around subject 18 are projected relative to each other to form voxels representing a 3D image region. Other forms of image reconstruction may be used, including but not limited to processing image data from detector signals to produce clinically useful images.

[0027] The processor 30 can be configured to receive commands and scan parameters from an operator via an operator workstation 40, which is typically equipped with a keyboard, touchscreen, and / or other input devices. The operator can control the imaging system 10 via the operator workstation 40. Therefore, the operator can use the operator workstation 40 to view the reconstructed images and / or otherwise operate the imaging system 10. For example, a display 42 coupled to the operator workstation 40 can be used to view the reconstructed images and control the imaging process. Additionally, the images can also be printed by a printer 44, which can be coupled to the operator workstation 40.

[0028] Furthermore, the processor 30 and operator workstation 40 can be coupled to other output devices, which may include standard or dedicated computer monitors and associated processing circuitry. One or more operator workstations 40 can be further linked within the system for outputting system parameters, requesting checks, viewing images, etc. Generally, the monitors, printers, workstations, and similar devices provided within the system may be local to the data acquisition components, or may be located remotely, such as elsewhere within the institution or hospital, or in entirely different locations, linked to the image acquisition system via one or more configurable networks (such as the Internet, VPN, etc.).

[0029] It should also be noted that the operator workstation 40 can also be coupled to a Picture Archiving and Communication System (PACS) 46. The PACS 46 can then be coupled to a remote client 48, a Radiology Information System (RIS), a Hospital Information System (HIS), or an internal or external network, allowing other people in different locations to access the raw or processed image data.

[0030] While the foregoing discussion has described various exemplary components of the imaging system 10, these various components may be provided within a common platform or in an interconnected platform. For example, the processor 30, memory 38, and operator workstation 40 may be provided collectively as a general-purpose or special-purpose computer or workstation configured to operate according to various aspects of this disclosure. In such embodiments, the general-purpose or special-purpose computer may be provided as a separate component relative to the data acquisition components of the imaging system 10, or it may be provided in a common platform having such a component. Similarly, the system controller 24 may be provided as part of such a computer or workstation, or as part of a separate system dedicated to image acquisition.

[0031] like Figure 1 As shown, the imaging system 10 may also include a variety of alternative embodiments generally configured to meet the specific needs of certain applications. For example, the imaging system 10 may be a fixed system, a mobile system, or a mobile C-arm system, wherein the x-ray detector 34 is either permanently mounted inside one end of the C-arm 14 or removable from the system. Furthermore, the imaging system 10 may be a table and / or wall-mounted system in a fixed x-ray room, wherein the x-ray detector 34 is either permanently mounted to the system or portable. Alternatively, the imaging system 10 may be a mobile x-ray system with a portable x-ray detector. Such a portable x-ray detector may be further configured to have a detachable tether or cable for connecting the detector readout electronics to the scanner's data acquisition system. When not in use, the portable x-ray detector can be detached from the scanning station for storage or transfer. In implementation, the imaging system 10 may be any suitable x-ray-based imaging system, including but not limited to conventional x-ray radiography systems, CT imaging systems, tomographic x-ray radiography systems, C-arm systems, fluoroscopy systems, mammography systems, dual-energy or multi-energy systems, navigation or interventional imaging systems, etc. Furthermore, while an example of a flat panel detector has been described above, a digital detector system that includes an image intensifier and a camera can be used to convert the incident X-rays 20 into a video signal.

[0032] As used herein, the phrase "reconstructed image" is not intended to exclude embodiments of the invention in which data representing an image is generated rather than a visual image. Therefore, as used herein, the term "image" broadly refers to both a visual image and the data representing a visual image. However, many embodiments generate (or are configured to generate) at least one visual image.

[0033] As will be described in more detail below, subject 18 can be imaged using an X-ray imaging chain comprising an X-ray source 56 and an X-ray detector 34. Although not explicitly shown, it is understood that the X-ray imaging chain may also include various lenses (e.g., collimating lenses and / or focusing lenses) and apertures. The X-ray imaging chain is positioned around subject 18 at different angles selected by the operator (e.g., a clinician). Subject 18 lies on a worktable 32, and the position of the worktable 32 may also change throughout the imaging process. The acquired X-ray images are 2D conic projections, and the changes in the position of the imaging chain and worktable 32 allow the operator to view the anatomy of subject 18 at different angles and magnifications. For example, the workflow of the imaging procedure may include acquiring several image sequences that can be used to diagnose subject 18 and, if necessary, intervene in subject 18. Anatomical locations of interest can be observed in real time within these different images, which can aid in guide needle insertion or other interventions.

[0034] See Figure 2 An exemplary medical image processing system 200 is illustrated. In some embodiments, the medical image processing system 200 is integrated into a medical imaging system such as a fluoroscopic imaging system (e.g., Figure 1 The medical image processing system 200 is used in imaging systems such as imaging systems 10), magnetic resonance imaging (MRI) systems, CT systems, and single-photon emission computed tomography (SPECT) systems. In some embodiments, at least a portion of the medical image processing system 200 is located at a device (e.g., an edge device or server) that is communicatively coupled to the medical imaging system via a wired and / or wireless connection. In some embodiments, the medical image processing system 200 is located at a separate device (e.g., a workstation) that can receive images from the medical imaging system or from a storage device storing images generated by the medical imaging system. The medical image processing system 200 may include an image processing unit 231, a user input device 232, and a display device 233. For example, the image processing unit 231 may be operatively and / or communicatively coupled to the user input device 232 and the display device 233.

[0035] The image processing unit 231 includes a processor 204 configured to execute machine-readable instructions stored in non-transitory memory 206. The processor 204 may be single-core or multi-core, and the program executed by the processor 204 may be configured for parallel or distributed processing. In some embodiments, the processor 204 may optionally include individual components distributed across two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of the processor 204 may be virtualized and executed by a remotely accessible networked computing device in a cloud computing configuration. In some embodiments, the processor 204 may include other electronic components capable of performing processing functions, such as a digital signal processor, a field-programmable gate array (FPGA), or a graphics board. In some embodiments, the processor 204 may include multiple electronic components capable of performing processing functions. For example, the processor 204 may include two or more electronic components selected from a plurality of possible electronic components, including a central processing unit, a digital signal processor, a field-programmable gate array, and a graphics board. In another embodiment, the processor 204 may be configured as a graphics processing unit (GPU), including a parallel computing architecture and parallel processing capabilities.

[0036] Non-transitory memory 206 may store segmentation module 210, trajectory prediction module 212, and medical image data 214. Segmentation module 210 may include one or more machine learning models (such as deep learning networks) including multiple weights and biases, activation functions, loss functions, gradient descent algorithms, and instructions for implementing the one or more deep neural networks to process input medical images. For example, segmentation module 210 may store instructions for implementing the neural network. Segmentation module 210 may include trained and / or untrained neural networks and may also include training routines or parameters (e.g., weights and biases) associated with the one or more neural network models stored therein. In one example, training routines may include instructions for receiving training datasets from medical image data 214, which include a set of medical images, associated ground truth labels / images, and associated model outputs used to train one or more machine learning models stored in segmentation module 210. In some implementations, the training routine may receive medical images, associated ground truth labels / images, and associated model outputs for training one or more machine learning models from sources other than the medical image data 214 (such as other image processing systems, the cloud, etc.). In some implementations, one or more aspects of the training routine may include remotely accessible networked storage devices in a cloud computing configuration. Additionally or alternatively, in some implementations, the training routine may be used offline and remotely from the image processing system 200 to generate the segmentation module 210. In such implementations, the training routine may not be included in the segmentation module 210, but may generate data stored in the segmentation module 210.

[0037] Similarly, trajectory prediction module 212 may include one or more algorithms, including a machine learning module, and instructions for implementing one or more algorithms for processing input medical images. In some embodiments, trajectory prediction module 212 may additionally use data received from segmentation module 210 and / or the user as input. As will be described in detail herein, segmentation module 210 may be used to identify and segment inserted medical devices (e.g., needles) within a subject during interventional medical imaging (e.g., real-time fluoroscopy), and trajectory prediction module 212 may precisely locate the distal portion of the inserted medical device and estimate (e.g., infer) its trajectory within the patient.

[0038] Non-transitory memory 206 also stores medical image data 214. Medical image data 214 includes, for example, images captured by an X-ray imaging modality, such as fluoroscopic X-ray images, anatomical images captured by an MRI system or CT system, etc. In some embodiments, non-transitory memory 206 may include components located at two or more devices that can be remotely located and / or configured for coordinated processing. In some embodiments, one or more aspects of non-transitory memory 206 may include a remotely accessible, networked storage device configured for cloud computing.

[0039] The image processing system 200 may also include a user input device 232. The user input device 232 may include one or more of a touchscreen, keyboard, mouse, touchpad, motion-sensing camera, or other devices configured to enable a user to interact with and manipulate data within the image processing unit 231. As an example, the user input device 232 may enable a user to annotate an image structure, such as indicating the target location for insertion of a medical device.

[0040] Display device 233 may include one or more display devices utilizing any type of display technology. In some embodiments, display device 233 may include a computer monitor and may display unprocessed images, processed images, parameter mapping diagrams, and / or examination reports. Display device 233 may be combined with processor 204, non-transitory memory 206, and / or user input device 232 in a shared housing, or may be a peripheral display device. Display device 233 may include a monitor, touch screen, projector, or another type of display device that enables a user to view medical images and / or interact with various data stored in non-transitory memory 206.

[0041] It should be understood that, Figure 2 The image processing system 200 shown is one non-limiting embodiment of an image processing system, and other imaging processing systems may include more, fewer, or different components without departing from the scope of this disclosure.

[0042] As described above, image processing systems (e.g., Figure 2 The image processing system 200) can receive signals from the imaging system (e.g., Figure 1The image acquired by the imaging system 10, or may be included in the imaging system, is processed by an image processing system that can analyze and annotate the acquired image in real time. For the purposes of this disclosure, the term "real time" is defined as actions performed without any intentional delay (e.g., substantially as they occur). For example, the imaging system may acquire images at a predetermined frame rate, reconstruct images for display, and display the reconstructed images in real time, and the image processing system may also process the reconstructed images to identify an inserted medical device, determine the trajectory of the medical device, and overlay the trajectory of the medical device onto the displayed image in real time.

[0043] In order to coordinate real-time imaging and image processing, Figure 3 Method 300 is illustrated, which is used to determine the insertion trajectory of a medical device and accordingly enhance / annotate real-time images. References will be made in this document. Figures 1 to 2 The system and components described herein are used to describe the method; however, it is understood that the method may be implemented using other systems and components without departing from the scope of this disclosure. Method 300 may be implemented as a non-transitory memory of a computing device (such as...). Figure 1 The memory 38 and / or Figure 2 Executable instructions in non-transitory memory 208, and operable by one or more processors (such as...) Figure 1 Processor 30 and / or Figure 2 The processor 204) executes the method in a coordinated manner. Method 300 will be described with reference to X-ray fluorescence imaging, but it will be understood that other imaging modalities may be used without departing from the scope of this disclosure.

[0044] At 302, method 300 includes receiving imaging scheme and procedure information. For example, an imaging system (e.g., Figure 1 The operator of the imaging system 10) can, for example, via an operator workstation (e.g., Figure 1 The operator workstation 40) inputs or selects imaging protocol and procedure information. Protocol information may include the type of interventional imaging procedure being performed and / or the imaging protocol used for that procedure. For example, the protocol type or imaging protocol may specify the anatomical structure being imaged, define the series of views to be acquired (and the corresponding location of the imaging system acquiring those views), and indicate whether a contrast agent was used and its type. For example, a contrast agent may be administered to the subject to aid in the visualization of the anatomical structure of interest. As an example, a contrast agent may be injected to visualize the vascular system. Exemplary examples of protocol types include endobronchial lung biopsy, transjugular intrahepatic portosystemic shunt (TIPS) procedures, percutaneous tumor ablation, cancer treatments (such as cryoablation, radiofrequency ablation, etc.), and bone consolidation procedures.

[0045] As another example, the protocol type may further specify the physical properties of the medical device to be inserted into the imaged subject, such as the type, thickness, gauge, length, and / or shape (e.g., curvature) of the medical device to be inserted. The medical device may be a needle, fiber optic cable, rod, screw, or other type of implantable device or medical endoscope. The properties may also include the stiffness or flexibility (e.g., deformability) of the medical device, which may be determined based on one or more materials forming the medical device. For example, a smaller gauge needle (e.g., gauge 14) may be thicker and stiffer than a larger gauge needle (e.g., gauge 20). As another example, fiber optic cables are more flexible than steel bars. In some examples, the medical device to be inserted and some or all of its physical properties may be automatically selected based on the protocol type and / or image analysis. Additionally or alternatively, the operator may select, adjust, or otherwise input the physical properties of the medical device. The physical properties of the medical device can be used to determine the trajectory of the medical device within the subject during insertion, as will be described in detail below (e.g., at 316).

[0046] As another example, imaging scheme information may include X-ray tube voltage, current, and pulse width settings for the X-ray source, and frame rate and magnification settings for acquiring images at the X-ray detector. The X-ray detector can then generate a synchronization signal when an equilibrium condition is reached and send this signal back to the system controller. The X-ray source can wait for the synchronization signal from the detector and begin generating X-rays at the synchronization signal once X-ray exposure is enabled. The workstation / image processing unit can stop its current activity, initialize the acquisition and image processing modules, and wait for incoming frames. Furthermore, in some examples, the X-ray tube filament can be preheated during the exposure preparation phase (e.g., by applying a certain amount of voltage to the filament before X-ray exposure) to reduce the amount of time it takes for the X-ray tube to reach the current setting. For example, the filament can be heated to a predetermined temperature based on the current setting so that the current setting is reached quickly once exposure begins.

[0047] At 304, method 300 includes acquiring images of the subject for entry point and path planning. For example, before the commencement of real-time imaging for performing device insertion, oblique CBCT cross-sections of the anatomy of interest may be obtained to plan the device entry point and path, as will be described in detail below. In some examples, 3D volumetric reconstruction may be performed based on the acquired CBCT projection images.

[0048] At 306, method 300 includes determining a device access point and path based on the acquired images. When more than one medical device is to be inserted, a separate access point and path can be determined for each device. For example, a 3D volume can be used to identify anatomical structures of interest, such as target structures or regions, and to plan procedures by determining access points, paths (e.g., passages), and structures to be avoided. In some examples, an operator (or another clinician) can select, mark, or otherwise annotate (e.g., comment) or indicate the target structure or region for insertion of the device on the 3D volume. In other examples, a processor can use computer vision to automatically determine the target structure or region based on received procedure information. For example, a processor can identify intrabronchial lung biopsy sites and present the identified sites on a 3D volume for the operator to view and optionally adjust.

[0049] Furthermore, protocol planning can be used to determine operational parameters for acquiring real-time (e.g., live) images during the protocol. Examples of such parameters may include, for instance, the desired frame rate, C-arm position and rotation angle, and stage position. For example, parameters can be selected such that the device channels and the anatomical structures of interest are approximately within the imaging plane during the protocol. As another example, the processor may consider the size and location of the subject, as well as radiation dose considerations, when determining the protocol plan.

[0050] At 308, method 300 includes acquiring images of the subject at a desired frame rate and displaying the images in real time. For example, images may be acquired based on an imaging protocol received at 302 and a procedural plan determined at 306. For example, the processor may access the images via a motor controller (e.g., Figure 1 The motor controller 36) adjusts the position and rotation angle of the C-arm and the position of the worktable to a predetermined position for imaging the target structure or area. As each image is acquired, the image data for display can be processed in real time and displayed on a display device (e.g., ...). Figure 1 On the display (42). In some examples, the 3D model can be overlaid on and displayed on a 2D real-time image.

[0051] At 310, the method includes displaying a determined entry point and path of the device on a real-time image. For example, the processor may display the entry point and path of the device as a visual annotation on the real-time image. As used herein, the term "annotation" may refer to a precise location defined by one (or more) points in a given image, a linear (e.g., line) segment having a precise location within the image, and / or a set of consecutive line segments each having a precise location within the image. Each annotation may or may not contain associated text (e.g., a label).

[0052] At 312, the method includes identifying the device during insertion via real-time image analysis. As an example, the processor may employ one or more object detection (e.g., image recognition) and segmentation algorithms to (e.g., via...) Figure 2 The segmentation module 210) distinguishes the medical device from tissues, bones, blood vessels, and other anatomical structures. For example, the segmentation algorithm may use shape or edge detection algorithms to define the boundaries between the medical device and tissues, organs, and structures.

[0053] At 314, method 300 includes locating the distal end of a device in a real-time image. The distal end of the device includes the portion of the medical device furthest from the attachment point or handle used by the clinician to insert the device. As another example, the distal end of the device includes the tip of the device that first enters the subject. For example, the distal end of a needle is the tip of the needle. As an example, the processor may locate the segmented end of the device based on the boundaries of the medical device and identify its position in 2D image space. Additionally or alternatively, the processor may locate the distal end of the device based on the determined entry point and path of the device, since the tip of the device is generally closest to the target. In some examples, the operator may input information, such as by selecting a general area of ​​the distal end of the device in the real-time image, to reduce uncertainty. In other examples, additionally or alternatively, the processor may consider knowledge of previous angles of the device. For example, a clinician may initially navigate the device by angles that reduce uncertainty regarding the position of the distal end of the device. When changing angles makes locating the distal end of the device more challenging, the processor may use knowledge from previous views to determine the current position of the distal end of the device.

[0054] When the processor initially fails to locate the distal end of the device, other methods besides those described above may be used. For example, an X-ray image may be acquired at a higher radiation dose level and / or by utilizing collimation around a desired area around the distal end to enhance the acquired image. As another example, the system may output a message to the operator suggesting minimum gantry rotation to reduce or eliminate overlap between the edges of the device and the imaging anatomy. As yet another example, when the device is at an angle seen along its axis in a top-down or bottom-up view (e.g., where a needle is represented as a point rather than a length in the image), the processor may temporarily suspend virtual trajectory inference and rendering until the angle changes.

[0055] At 316, method 300 includes inferring the trajectory of the device within the subject's body in real time. As an example, the processor can use data stored in memory (e.g., ...). Figure 2The processor uses a geometric model (in the trajectory prediction module 212) to infer or project the trajectory of a medical device within the subject's body. For example, the geometric model may employ curve fitting, such as a polynomial or power series curve fitting model, to identify the best-fit curve based on the currently observed geometric curvature of the medical device as observable in real-time images (e.g., captured via an X-ray detector). Furthermore, the processor may infer the best-fit curve based on the current position of the distal end of the device. For example, the distal end of the device may indicate the angle of the medical device's trajectory in the image plane. Therefore, the current position and orientation of the distal end of the medical device can be input into the geometric model. For example, the orientation may include the 2D and / or 3D position of the distal end and its angle relative to a reference (such as a determined path). In some examples, the best-fit curve may extend beyond a predetermined distance (e.g., length) of the distal end of the medical device. For example, the predetermined distance may be selected by the operator or pre-programmed according to a selected protocol to be proportionally consistent with the magnification of the real-time image and the acquired angle. For example, the angle of the imaging system relative to the device may make a given length appear shorter or longer in the real-time image. As another example, the best-fit curve can be extended to the target structure or region. Furthermore, it is understood that the inferred trajectory can be updated in real time as the medical device is further inserted or the angle of its trajectory changes.

[0056] In some examples, a relatively small portion of the medical device may be visible in real-time images when it is initially inserted, but the visible (e.g., captured) portion may increase as the device is further inserted. As an example, a processor may use only the visible portion of the medical device when inferring its real-time trajectory. In other examples, the processor may consider the known total length and known geometry of the device when determining its trajectory. For example, a 0.5 cm portion of a 14 cm linear (non-curved) needle may be visible, and the processor may infer the trajectory of the 14 cm linear needle. Inference may be less accurate when only the visible portion is considered, rather than the medical device as a whole. Therefore, inputting or programming the physical properties of the medical device for a given procedure (e.g., at 302) can help increase the accuracy of the inference.

[0057] Furthermore, in some examples, mechanical models may be used in addition to geometric models. Mechanical models may consider the physical properties of the medical device (e.g., length, thickness, and relative stiffness) and the physical properties of the tissue, bone, or other anatomical structures penetrated by the medical device (e.g., relative stiffness or flexibility). As an example, the processor may identify the relative stiffness or flexibility of tissue, bone, or other anatomical structures in part based on the protocol being executed. For example, a bone consolidation protocol may include inserting the medical device into one or more bones or bone fragments, while a TIPS protocol may not include inserting the medical device through bone. The processor may store a score of the relative stiffness (or flexibility) of each type of tissue in memory and may use these values ​​individually or in combination with the relative stiffness (or flexibility) of the medical device when determining the trajectory. As an illustrative example, stiffer tissue (e.g., bone) may constrain the movement of the medical device, while more flexible tissue (e.g., muscle) may allow for increased angular changes in the medical device during insertion. As another example, a thick needle may not deform easily and can therefore be predicted to travel within a linear channel. In contrast, thin, flexible needles may deform and deviate from the straight channel during insertion.

[0058] Therefore, in some examples, in addition to inferring the trajectory of the medical device, the processor can also determine the uncertainty in the inference. In some examples, the uncertainty may be based on geometric curvature. For example, there may be greater uncertainty in nonlinear curve fitting and corresponding inference compared to linear curve fitting and corresponding inference. Additionally or optionally, the processor can input one or more of the following into an uncertainty algorithm, which outputs a range of expected possible trajectories: geometric curvature, the stiffness of the medical device being inserted, the tissue being navigated / penetrated, and the type of procedure being performed. Generally, uncertainty can decrease with increasing stiffness of the medical device, increase with increasing distance to the distal end of the device, and increase with decreasing density of the puncture material. As another example, the processor can compare previously predicted trajectories relative to the current device position to adjust the uncertainty accordingly. For example, if the current position is outside the previously predicted uncertainty, the uncertainty can be further increased. Thus, uncertainty can be learned on a per-imaging-period, per-operator, and / or per-organ basis. Data can be collected prospectively to refine the uncertainty estimate for further insertions.

[0059] Uncertainty can typically follow an inferred shape, and the range of uncertainty, and therefore the range of possible trajectories, can increase with increasing distance from the distal end of the medical device within the projected trajectory. For example, uncertainty for a linear projected trajectory can be substantially conical, or it can be curved like the projected trajectory curve. Furthermore, uncertainty can include spatial regions mapped onto the area surrounding the inferred trajectory. Uncertainty can be visually represented on a real-time image as spatial regions (e.g., a conical region whose width increases with distance from the device), the opacity of variations in the inferred trajectory (e.g., becoming less visible with increasing uncertainty), variations in line width or line type (e.g., becoming dotted with increasing uncertainty), and / or color-mapped regions along the inferred trajectory (e.g., a first color for lower uncertainty, a second color for higher uncertainty).

[0060] At 318, method 300 includes displaying the inferred trajectory of the device on a real-time image. For example, the inferred trajectory may be displayed as an annotation on the image alongside or overlapping a determined path of the medical device (e.g., as determined at 306). As an example, the determined path may be displayed as a first-color annotation or graphic, such as a line or curve, and the inferred trajectory may be displayed as a second-color annotation or graphic, different from the first color. Other attributes, such as line thickness or style, may differ between the determined path and the inferred trajectory to distinguish them from each other. Additionally or alternatively, text-based labels may be provided to indicate a determined (e.g., planned) path relative to the inferred (e.g., estimated) trajectory. In some examples, uncertainty in the inference may also be visually displayed. For example, uncertainty may be visually represented as a cone or other shape around the inferred trajectory, which typically widens with increasing distance from the distal end of the medical device (and thus the uncertainty increases). Examples of displaying inferred trajectories with and without uncertainty are shown in [reference needed]. Figure 4 and Figure 5 As shown in the figure, and will be described below.

[0061] By displaying both the inferred trajectory and the desired trajectory in real-time on real-time (e.g., real-time) images, clinicians can more easily identify corrections to position the medical device within the planned path to reach the target structure or region. For example, if the inferred trajectory shows the medical device veering to the left of the planned path, the clinician can adjust the insertion angle of the medical device to the right to bring the inferred trajectory closer to the planned path. For example, the clinician continues to make incremental adjustments until the inferred trajectory substantially overlaps with the planned path. In contrast, without real-time feedback to show the effect of adjustments to the inferred trajectory relative to the planned path, clinicians may undercorrect or overcorrect the insertion angle, resulting in less accurate placement of the medical device.

[0062] In step 320, method 300 includes determining whether acquisition is complete. For example, acquisition is complete when all prescribed views of the current interventional imaging procedure and imaging scheme have been acquired. As another example, the operator can indicate that acquisition is complete via input.

[0063] If acquisition is not completed, method 300 returns to 308 to continue acquiring images of the subject at the desired frame rate and displaying the images in real time. Therefore, the device trajectory will continue to be tracked, and adjustments to the trajectory, such as those made by the clinician adjusting the insertion angle, will be determined and updated in real time to provide feedback to the clinician.

[0064] If acquisition is complete, method 300 proceeds to 322 and includes stopping the acquisition of images of the subject. For example, the X-ray source may be deactivated, such as by stopping power supply to the X-ray generator or placing the X-ray source in a low-power "standby" mode, where the X-ray source does not actively generate X-rays until the imaging system is shut down. Furthermore, in some examples, the X-ray detector may not be powered or may be placed in a "standby" mode, where power consumption is reduced and the X-ray detector does not actively generate electrical signals until the imaging system is powered off. Method 300 can then terminate. For example, the X-ray source may not be activated until the operator selects a new imaging protocol or provides another input to begin a new imaging sequence.

[0065] This allows for the display of the medical device's real-time trajectory during the interventional imaging procedure. Displaying the real-time trajectory improves the accuracy of device placement. Furthermore, the real-time guidance in augmented reality format makes device placement easier for clinicians. Overall, this reduces the length of the procedure, which in turn reduces the radiation dose delivered to the patient and increases patient comfort.

[0066] Turn now Figure 4 This demonstrates the first example of displaying a real-time estimated trajectory during the insertion of a medical device. Specifically, Figure 4 A real-time display image 400 is shown, which can be displayed to a clinician performing the insertion via a display device (e.g., display 42). The displayed image 400 includes a real-time x-ray image 401 and trajectory annotations, which will be described below. For example, during real-time image processing and analysis, trajectory annotations are superimposed on the real-time x-ray image 401, which can be generated from the raw x-ray image data. Furthermore, a fluoroscopic imaging system (such as...) can be used. Figure 1 An imaging system 10) is used to acquire real-time x-ray images 401. It is understood that although a real-time display image is shown, real-time display image 400 represents an annotated x-ray image from a sequence of x-ray images acquired, processed, and displayed in real-time at a programmed frame rate.

[0067] The live-time display image 400 shows a needle 402 with a partial distal end 404. In this example, the distal end 404 is visually represented by an "X," but in other examples, the distal end 404 may not be visually annotated on the live-time display image 400. The live-time display image 400 also shows the planned trajectory 406 (e.g., relative to...). Figure 3 The defined path described in 306) and the real-time estimated trajectory 408 (e.g., relative to...) Figure 3 (Described inferred trajectory). In the example shown, the real-time display image 400 uses different line types and text labels to distinguish the planned trajectory 406 from the real-time estimated trajectory 408, but other examples may visually distinguish the planned trajectory 406 from the real-time estimated trajectory 408 in different ways. Furthermore, in this example, the real-time estimated trajectory 408 is substantially parallel to but spaced apart from the planned trajectory 406. Because the real-time estimated trajectory 408 does not overlap with the planned trajectory 406, the real-time display image 400 can inform the clinician to adjust the angle of the needle 402 toward the planned trajectory 406.

[0068] Figure 5 A second example is shown, displaying a real-time estimated trajectory during the insertion of a medical device. Specifically, Figure 5 A real-time display image 500 is shown, which can be displayed to the clinician performing the insertion via a display device (e.g., display 42). The displayed image 500 includes a real-time X-ray image 501 and trajectory annotations, which will be described below. As described above relative to... Figure 4 As described, the trajectory annotation is superimposed on a real-time X-ray image 501, which can be visualized using a fluorescence microscope imaging system (e.g., Figure 1 The imaging system 10) acquires and displays in real time an image 500 representing an annotated X-ray image from a sequence of X-ray images acquired, processed, and displayed in real time at a programmed frame rate.

[0069] Real-time image 500 shows a medical device 502 with a local distal end 504. In the example shown, the medical device 502 is a fiber optic endoscope with a camera and a needle tip for puncturing a nodule. In this example, the distal end 504 (e.g., the needle tip) of the medical device 502 is not specifically annotated on the real-time image 500. The real-time image 500 also shows a target area 503 and a planned trajectory 506 for reaching the target area 503 (e.g., relative to...). Figure 3 The defined path described in 306) and the real-time estimated trajectory 508 (e.g., relative to...) Figure 3(Described inferred trajectory). In the example shown, the real-time display image 500 uses different lines to distinguish the planned trajectory 506 from the real-time estimated trajectory 508. Furthermore, in this example, the uncertainty in the real-time estimated trajectory 508 is indicated by the region 510 surrounding it. For example, because the medical device 502 is flexible, it may deviate from the real-time estimated trajectory 508 if the current trajectory is maintained but is expected to remain within region 510. Even if the real-time estimated trajectory 508 does not overlap with the planned trajectory 506, the real-time display image 500 shows the medical device 502 pointing towards the target region 503. Therefore, clinicians can make minor adjustments to the angle of the medical device 502 to reach the target region 503, rather than completely overlapping with the planned trajectory 506. If only the planned trajectory 506 is shown without the real-time estimated trajectory 508, clinicians might overcorrect the planned trajectory 506, as it might be difficult to predict whether the medical device 502 will reach the target region 503 when it deviates from the planned trajectory 506.

[0070] This allows for easier and more accurate placement of medical equipment. Consequently, it reduces the time spent performing procedures, enabling clinicians to treat more patients and free up operating rooms throughout the day. Furthermore, patients can reach the recovery room more quickly, increasing their comfort.

[0071] The advantage of inferring the real-time trajectory of a medical device during insertion into a patient from real-time images acquired during insertion is that the medical device can be placed more quickly and accurately.

[0072] This disclosure also provides support for a method for interventional imaging procedures, the method comprising: identifying a medical device based on real-time images of the insertion during insertion into a subject; inferring the trajectory of the medical device during insertion in real time based on the real-time images of the insertion; and displaying the inferred trajectory of the medical device on the real-time images. In a first example of the method, displaying the inferred trajectory of the medical device on the real-time images comprises extending the inferred trajectory from a distal end of the medical device as a linear or curvilinear inference. In a second example of the method, optionally including the first example, inferring the trajectory of the medical device during insertion in real time based on the real-time images of the insertion comprises: locating a distal end of the medical device; and inferring the trajectory of the medical device based on the current position of the distal end and the geometric curvature of the medical device. In a third example of the method, optionally including one or both of the first and second examples, the method further comprises: determining the geometric curvature by identifying a best-fit curve of the medical device based on a visible portion of the medical device in the real-time images, and wherein inferring the trajectory of the medical device further comprises extending the best-fit curve beyond the distal end of the medical device. In a fourth example of the method, optionally including one or more of the first to third examples, or each of them, the geometric curvature of the medical device is determined based on the known length and shape of the medical device. In a fifth example of the method, which optionally includes one or more of the first to fourth examples, or each of the methods, the inferred trajectory of the medical device is further based on an inserted mechanical model. In a sixth example of the method, which optionally includes one or more of the first to fifth examples, or each of the methods, the inserted mechanical model uses one or more physical properties of the medical device to estimate the deformation of the medical device during insertion. In a seventh example of the method, which optionally includes one or more of the first to sixth examples, or each of the methods, the one or more physical properties include at least one of the length, thickness, and stiffness of the medical device. In an eighth example of the method, which optionally includes one or more of the first to seventh examples, the inserted mechanical model uses one or more physical properties of the anatomical structures of the subject penetrated by the medical device during insertion. In a ninth example of the method, which optionally includes one or more of the first to eighth examples, the method further includes: determining uncertainty in the inferred trajectory; mapping the uncertainty to a spatial region surrounding the inferred trajectory; and displaying the uncertainty as a spatial region on a real-time image. In a tenth example of the method, which optionally includes one or more of the first to ninth examples, the uncertainty in the inferred trajectory is determined based on one or more of the following: the geometric curvature of the medical device, the stiffness of the medical device, the tissue the medical device is penetrating, and the type of interventional imaging procedure.

[0073] This disclosure also provides support for a method for interventional imaging, the method comprising: determining a desired trajectory for insertion based on images of the subject acquired prior to insertion of a medical device into the subject; acquiring real-time images of the subject during insertion; displaying the desired trajectory for insertion on the real-time images of the subject during insertion; estimating the real-time trajectory of the medical device during insertion; and displaying the real-time trajectory of the medical device on the real-time images of the subject. In a first example of the method, estimating the real-time trajectory of the medical device during insertion comprises: identifying the medical device in the real-time images via a segmentation algorithm; determining the position of the distal end of the medical device; determining the geometric curvature of the medical device; and estimating the real-time trajectory of the medical device based on the geometric curvature and the position of the distal end. In a second example of the method, optionally including the first example, displaying the real-time trajectory of the medical device on the real-time images of the subject comprises extending the real-time trajectory beyond the distal end of the medical device by a predetermined distance. In a third example of the method, optionally including one or both of the first and second examples, estimating the real-time trajectory of the medical device is further based on at least one of the length of the medical device, the thickness of the medical device, the stiffness of the medical device, the type of interventional imaging procedure, and the stiffness of the anatomical feature being penetrated by the medical device. In a fourth example of the method, which optionally includes one or more or each of the first to third examples, the method further includes: estimating uncertainty in the real-time trajectory based on at least one of the shape of the real-time trajectory, the length of the medical device, the thickness of the medical device, the stiffness of the medical device, the type of interventional imaging procedure, and the stiffness of the anatomical feature being penetrated by the medical device; and displaying a visual representation of the uncertainty on a real-time image.

[0074] This disclosure also provides support for an imaging system comprising: a radiation source configured to project a radiation beam toward a subject; a radiation detector configured to receive the radiation beam projected by the radiation source and impacted by the subject; a display; and a processor operatively coupled to a memory storing instructions which, when executed, cause the processor to: acquire a real-time image of a medical device penetrating the subject via the radiation detector; analyze the real-time image in real time to identify the medical device and estimate the real-time trajectory of the medical device; and display the real-time trajectory in real time on the real-time image via the display. In a first example of the system, for real-time analysis of the real-time image to identify the medical device and estimate the real-time trajectory of the medical device, the memory includes additional instructions which, when executed by the processor, cause the processor to: identify the medical device via a segmentation algorithm; locate the distal end of the medical device; and estimate the real-time trajectory of the medical device based on the current position of the distal end and a geometric model of the medical device. In a second example of the system that optionally includes the first example, in order to analyze real-time images in real time to estimate the real-time trajectory of the medical device, the memory includes additional instructions that, when executed by the processor, cause the processor to: further estimate the real-time trajectory of the medical device based on at least one of the stiffness, thickness, and length of the medical device. In a third example of the system that optionally includes one or both of the first and second examples, the geometric model of the medical device identifies a best-fit curve based on the curvature of the medical device, and the memory includes additional instructions that, when executed by the processor, cause the processor to: estimate the uncertainty in the real-time trajectory of the medical device based on at least one of the stiffness, thickness, and length of the medical device; and display the uncertainty in the real-time trajectory in real time on the real-time image via a display.

[0075] As used herein, elements or steps listed in the singular and beginning with the word "a" or "an" should be understood to not exclude a plurality of said elements or steps unless such exclusion is explicitly stated. Furthermore, references to "one embodiment" of the invention are not intended to be construed as excluding the existence of additional embodiments that also include the referenced features. Moreover, unless explicitly stated to the contrary, embodiments that "comprise," "include," or "have" elements or multiple elements having a particular characteristic may include additional such elements that do not have that characteristic. The terms "comprise" and "in..." are used as concise linguistic equivalents to the corresponding terms "comprising" and "wherein". Furthermore, the terms "first," "second," and "third," etc., are used merely as notations and are not intended to impose numerical requirements or a particular order of position on their objects.

[0076] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any included methods. The scope of patentability of the invention is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A medical image processing system, the medical image processing system comprising: User input devices; Display devices; and An image processing unit, operatively and / or communicatively coupled to the user input device and the display device, The image processing unit includes a processor configured to execute machine-readable instructions stored in a non-transitory memory, the machine-readable instructions causing the processor, when executed, to: The medical device is identified based on real-time images of the insertion during the insertion into the subject's body; The trajectory of the medical device during the insertion period is inferred in real time based on the inserted real-time image, wherein inferring the trajectory of the medical device includes extending the best-fit curve of the medical device beyond the distal end of the medical device by a predetermined distance, the predetermined distance being selected according to a selected procedure to be proportionally consistent with the magnification of the real-time image and the acquired angle. as well as The inferred trajectory of the medical device is displayed on the real-time image.

2. The medical image processing system of claim 1, wherein displaying the inferred trajectory of the medical device on the real-time image comprises extending the inferred trajectory from the distal end of the medical device as a linear or curvilinear inference.

3. The medical image processing system according to claim 1, wherein inferring the trajectory of the medical device during the insertion period in real time based on the inserted real-time image comprises: Position the distal end of the medical device; as well as The trajectory of the medical device is inferred based on the current position of the distal end and the geometric curvature of the medical device.

4. The medical image processing system of claim 3, wherein the machine-readable instructions, when executed, cause the processor to determine the geometric curvature by identifying the best-fit curve of the medical device based on the visible portion of the medical device in the real-time image.

5. The medical image processing system of claim 3, wherein the geometric curvature of the medical device is determined based on the known length and shape of the medical device.

6. The medical image processing system of claim 3, wherein inferring the trajectory of the medical device is further based on the inserted mechanical model.

7. The medical image processing system of claim 6, wherein the inserted mechanical model uses one or more physical properties of the medical device to estimate the deformation of the medical device during the insertion.

8. The medical image processing system of claim 7, wherein the one or more physical properties include at least one of the length, thickness and stiffness of the medical device.

9. The medical image processing system of claim 6, wherein the inserted mechanical model utilizes one or more physical properties of the subject's anatomical structures penetrated by the medical device during the insertion.

10. The medical image processing system of claim 1, wherein the machine-readable instructions, when executed, cause the processor to: Determine the uncertainty in the inferred trajectory; Mapping the uncertainty to a spatial region surrounding the inferred trajectory; and The uncertainty is displayed as the spatial region on the real-time image.

11. The medical image processing system of claim 10, wherein the uncertainty in the inferred trajectory is determined based on one or more of the geometric curvature of the medical device, the stiffness of the medical device, the tissue being penetrated by the medical device, and the type of interventional imaging procedure.

12. A medical image processing system, the medical image processing system comprising: User input devices; Display devices; and An image processing unit, operatively and / or communicatively coupled to the user input device and the display device, The image processing unit includes a processor configured to execute machine-readable instructions stored in a non-transitory memory, the machine-readable instructions causing the processor, when executed, to: The desired trajectory of the insertion is determined based on images of the subject acquired prior to the insertion of the medical device into the subject's body; Real-time images of the subject are acquired during insertion; The desired trajectory for the insertion is displayed on the subject's real-time image during the insertion; Real-time estimation of the real-time trajectory of the medical device during the insertion, wherein estimating the real-time trajectory of the medical device includes extending the real-time trajectory beyond the distal end of the medical device by a predetermined distance, the predetermined distance being selected according to a selected protocol to be proportionally consistent with the magnification of the real-time image and the acquired angle; as well as The real-time trajectory of the medical device is displayed on the real-time image of the subject.

13. The medical image processing system of claim 12, wherein real-time estimation of the real-time trajectory of the medical device during the insertion comprises: The medical device in the real-time image is identified using a segmentation algorithm; Determine the position of the distal end of the medical device; Determine the geometric curvature of the medical device; as well as The real-time trajectory of the medical device is estimated based on the geometric curvature and the position of the distal end.

14. The medical image processing system of claim 13, wherein estimating the real-time trajectory of the medical device is further based on at least one of the length of the medical device, the thickness of the medical device, the stiffness of the medical device, the type of interventional imaging procedure, and the stiffness of the anatomical feature being penetrated by the medical device.

15. The medical image processing system of claim 12, wherein the machine-readable instructions, when executed, cause the processor to: The uncertainty in the real-time trajectory is estimated based on at least one of the following: the shape of the real-time trajectory, the length of the medical device, the thickness of the medical device, the stiffness of the medical device, the type of interventional imaging procedure, and the stiffness of the anatomical feature being penetrated by the medical device; and A visual representation of the uncertainty is displayed on the real-time image.

16. An imaging system, the imaging system comprising: A radiation source configured to project a radiation beam toward the subject; A radiation detector configured to receive the radiation beam projected by the radiation source and struck by the subject; monitor; and A processor, operatively coupled to a memory storing instructions, which, when executed, cause the processor to: Real-time images of the medical device penetrating the subject are acquired via the radiation detector; The real-time image is analyzed in real time to identify the medical device and estimate the real-time trajectory of the medical device, wherein estimating the real-time trajectory of the medical device includes extending the real-time trajectory beyond the distal end of the medical device by a predetermined distance, the predetermined distance being selected according to a selected procedure to be proportionally consistent with the magnification of the real-time image and the acquired angle; as well as The real-time trajectory is displayed on the real-time image via the display.

17. The imaging system of claim 16, wherein, for real-time analysis of the real-time image to identify the medical device and estimate the real-time trajectory of the medical device, the memory includes additional instructions that, when executed by the processor, cause the processor to: The medical device is identified using a segmentation algorithm; Positioning the distal end of the medical device; and The real-time trajectory of the medical device is estimated based on the current position of the distal end and the geometric model of the medical device.

18. The imaging system of claim 17, wherein, in order to analyze the real-time image in real time to estimate the real-time trajectory of the medical device, the memory includes additional instructions that, when executed by the processor, cause the processor to: The real-time trajectory of the medical device is further estimated based on at least one of the stiffness, thickness, and length of the medical device.

19. The imaging system of claim 17, wherein the geometric model of the medical device identifies the best-fit curve based on the curvature of the medical device, and the memory includes additional instructions that, when executed by the processor, cause the processor to: The uncertainty in the real-time trajectory of the medical device is estimated based on at least one of the stiffness, thickness, and length of the medical device; and The uncertainty in the real-time trajectory is displayed in real time on the real-time image via the display.

Images