Method for operating a medical imaging device for the positionally correct representation of non-anatomical structures during an imaging examination, and device therefor
Patent Information
- Application Number
- ES2022186128T
- Authority / Receiving Office
- ES · ES
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-08-02
- Filing Date
- 2022-07-20
- Publication Date
- 2026-08-31
- Estimated Expiration
- 2042-07-20
Smart Images
Figure 00000014_0000 
Figure 00000015_0000 
Figure 00000016_0000
Abstract
Description
Method for operating a medical imaging device for the positionally correct representation of non-anatomical structures during an imaging examination, and device therefor The invention relates to a method for operating a medical imaging device for the positionally correct representation of non-anatomical structures during an imaging examination. The invention comprises a medical imaging device. In medical surgical procedures, such as vascular surgery, the goal is to perform these procedures inside a patient's body in a minimally invasive manner, as this approach places the least possible burden on the patient. Access to the body is thus achieved through small incisions made by the attending physician or, alternatively, through the patient's natural orifices. Since the target area of such a procedure is difficult for the attending physician to visualize, or in many cases cannot be visualized at all, non-anatomical structures—particularly interventional materials such as guidewires, catheters, stents, coils, and screws (i.e., structures introduced into the patient for the procedure)—and the patient's surrounding anatomy (anatomical structures) are visualized using imaging techniques.For this purpose, projective medical imaging techniques are often used, preferably 2D techniques. These are procedures that project the three-dimensional structure of a volume to be represented onto a two-dimensional surface, thus generating 2D images. 2D X-ray fluoroscopy is the preferred method here; that is, 2D images, particularly 2D X-ray images, are acquired sequentially and displayed on a screen. This method largely loses information about the spatial (3D) characteristics of the patient and, for example, the surgical material—its three-dimensional shape—since all structures within a projection beam, such as an X-ray beam, are projected onto a single point on a surface. This means that these structures appear superimposed on a 2D image.Two-dimensional images consist of pixels that can be assigned different values, such as integers like 1 or 0. Thus, a structure located in a projection beam, such as an X-ray beam, is projected onto a pixel in a 2D image. The pixel values depend on the type and nature of the structures being projected. If the attending physician had access to a rapid sequence of 3D reconstructions showing the introduced non-anatomical structures, such as surgical materials, and the patient's anatomy in a 3D view, they would obtain a 4D view—that is, three spatial dimensions plus one temporal dimension. This means the attending physician would have access to a 4D surgical guide for performing the procedure, allowing them to assess the position of the surgical material introduced into the patient at any given time. To provide a 3D reconstruction, it is necessary to acquire or generate a tomographic image, which can be done, for example, using a computed tomography (CT) scanner, a C-arm X-ray unit (which can be mobile or stationary), or a magnetic resonance imaging (MRI) scanner. A 3D reconstruction is a 3D volume calculated from a large number of individual measurements. In computed tomography, these individual measurements are 2D X-ray projections. The continuous and sequential acquisition of numerous 3D X-ray images required for continuous monitoring of the patient and the surgical instrument is associated with very high radiation exposure for the patient. This makes the current use of 4D surgical guidance impossible, particularly in vascular surgery.A computed tomography scanner can be any imaging device that captures 2D projection images and calculates a tomographic image from them. The current state of the art for guiding surgical materials, particularly within blood vessels, encompasses a variety of procedures. These methods face the challenge that blood vessels exhibit almost the same contrast as the surrounding tissue and are therefore initially invisible, especially on X-ray images or computed tomography scans. When acquiring X-ray images or CT scans, it is therefore necessary to administer a contrast agent into the vascular system, such as iodine or carbon dioxide, which absorbs X-rays more or less than the surrounding tissue, thus making the vascular system visible. When a contrast agent is administered to the patient, an additional benefit may be that even vessels with reduced or no blood flow can be detected by the surgeon. Contrast-enhanced images can be used in a variety of procedures. In 2D subtraction-based imaging procedures, they can represent blood flow dynamics or the shape of a vascular system after subtracting an image without contrast agent (mask), for example, in digital subtraction angiography (DSA). They can also be used to overlay a guide wire to guide an instrument within a vessel or vascular system. However, in this case, each change in the imaging geometry or each patient movement requires a new administration of contrast agent, accompanied by increased radiation exposure. Repeated administration of iodine-containing contrast agent can also have a toxic effect on the kidneys under certain circumstances and may therefore be contraindicated in cases of renal impairment.Furthermore, the depth information in the form of the third dimension is lost here, as described in the previous section. Therefore, it is recommended to use a 3D reconstruction of the vessels, which can be extracted, for example, from a previously mentioned contrast-enhanced computed tomography scan, and superimposed on a live 2D image as a contour in the correct orientation by determining and using the correct projection geometry. The procedures described in the preceding section have the disadvantage that contrast-enhanced vessels and, if present, non-anatomical structures are only visible to the treating physician in a live 2D image. Even when using multiple X-ray devices, particularly C-arm X-ray devices, especially mobile C-arm X-ray devices, which comprise different projection directions, such as orthogonal projection directions, complete 3D information is not available. Consequently, a significant amount of time, radiation dose, and contrast agent, for example, may be required to correctly guide a guide wire within a vascular system, placing a considerable burden on the patient. Document EP2656314B1 discloses a procedure and system that allow radiological guidance of an instrument during a medical examination based on a compressed scan. DE 102008 054 298 A1 discloses a procedure and a device for 3D visualization of the surgical path of a medical instrument, a medical instrument, and / or a specific tissue structure of a patient to assist in a medical procedure on patient tissue. In the procedure, and using the device, based on two 2D X-ray images of the body region comprising the patient tissue and / or the medical instrument and / or the specific tissue structure, generated by an X-ray device under mutually different projection directions, an access path is determined and / or the medical instrument and / or the specific tissue structure is identified.The surgical path and / or the medical instrument and / or the specific tissue structure are transferred to a 3D computational model of the patient, comprising the tissue as model tissue, and at least one section of the 3D computational model of the patient or of the model tissue, with the model intervention path and / or the model instrument and / or the model tissue structure superimposed, is displayed on a display device. The current state of the art has the disadvantage that the 2D X-ray projections required for 3D reconstructions of the surgical materials at a specific moment must be acquired simultaneously or almost simultaneously (pseudo-simultaneously) to accurately represent the surgical materials in the presence of movement, particularly caused by the surgeon guiding an instrument. If, on the other hand, there is a time lag between the acquisition of individual or multiple 2D X-ray projections, a coherent 3D reconstruction at a specific moment is generally not possible. Depending on the magnitude of the movement, motion artifacts may occur, or the 3D reconstructions may become completely unusable.This time lag occurs particularly when the X-ray equipment used for imaging has very few imaging chains, i.e., X-ray source-detector pairs comprising the X-ray system, since the X-ray system must change the imaging geometry between acquiring 2D update projections, for example by rotating a gantry. Thus, if the rotation does not occur quickly enough, the described disadvantage of an inconsistent and possibly unusable 3D reconstruction results. US 2021 / 0128011 A1 discloses an imaging procedure for radiological guidance of an instrument during medical procedures on an object, comprising: a) providing a first image of the object, followed by b) providing updated images to an operator during the procedure by measuring a subsampled set of projections of the object and reconstructing the updated image based on the changes between the first image or an update of the first image and the subsampled set of projections.Document D1 further refers to specific applications of the procedure and a system for radiologically guiding medical procedures on an object in accordance with the procedure, comprising means for providing an initial image of the object, an imaging device that acquires subsampled sets of projections, processing means connected to the imaging device to provide updated images during the procedure by reconstructing the updated image on the basis of the changes between the first image or an update of the first image and the subsampled set of projections. One solution to this problem that is obvious to the expert in the field is to allow very rapid changes in the imaging geometry, for example, by means of a rapidly rotating gantry. However, this places high demands on both the mechanical mount of the X-ray system, such as a gantry, and the components installed on it, as well as on the components required to acquire 2D X-ray projections. In order to minimize motion blur in the acquired 2D X-ray projections, which arises from the rapid change in imaging geometry during the exposure of the X-ray detectors, the X-ray source must emit radiation in very short but comparatively intense pulses, and / or the acquisition frequency of the X-ray detectors must be very high. This generally increases the cost of such a system or is technically impossible when using flat-panel X-ray detectors. Therefore, the object of the invention is to provide an improved procedure for the positionally correct representation of non-anatomical structures and anatomical structures during an imaging examination. The object of the invention is achieved according to the present invention by means of the subject matter of the independent claims. Advantageous embodiments are specified in the dependent claims. The procedure according to the present invention is based on making available a first 3D image (3D volume). This first 3D image is acquired during an interventional procedure using a 3D scan or, for example, is loaded from a patient file. In the latter case, when the first 3D image is loaded from a patient file, the first 3D scan was acquired preoperatively. The first 3D image, which contains at least one anatomical structure, can be acquired, for example, by preoperative computed tomography or magnetic resonance imaging. The at least one anatomical structure of the first 3D image can comprise, for example, partially bone structures and / or partially vascular structures, as well as surrounding anatomy, such as organs, muscles, or other soft tissues.Furthermore, the initial 3D image may include additional non-anatomical structures, such as implants or interventional materials from previous surgical procedures, and additional anatomical structures, such as skin margins. Alternatively, the initial 3D image may also be acquired intraoperatively, preferably using an intraoperative computed tomography scanner or a C-arm X-ray unit, particularly a mobile C-arm X-ray unit. Preferably, the initial 3D image is acquired using the device that is subsequently used for 4D interventional guidance.Furthermore, the first 3D image can be imported to an internal storage device, such as an internal image data storage device, or to an external storage device, such as a USB flash drive, an external hard drive, or online storage, accessible to the imaging device performing the procedure according to the present invention, preferably an X-ray machine. An anatomical model is extracted from at least one anatomical structure of the first 3D image, either before or after the import. According to the present invention, an extraction is the calculation of an anatomical model, where the model may correspond, for example, to a voxel-based segmentation or a parametric representation.If, for example, the at least one anatomical structure is a particular organ, preferably a hollow organ such as a contrast-enhanced blood vessel, a contrast-enhanced heart chamber, or an intestine, then, for example, a model of the surface of the hollow organ can be calculated. If the at least one anatomical structure is, for example, a bone such as a vertebral body, then, for example, a segmentation of all the associated voxels of the anatomical structure can be calculated. According to the present invention, at least two 2D update images are provided, wherein the 2D update images contain at least partially anatomical and / or at least partially non-anatomical structures of an examination area. The at least partially anatomical structures may be parts of bone structures, parts of vascular structures, or parts of both bone and vascular structures. The non-anatomical structures are, in particular, interventional materials introduced into the patient to be treated, such as guidewires, catheters, stents, coils, or screws. The at least two provided 2D update images may consist partly of newly acquired 2D update images and partly of 2D update images that were acquired during the surgical procedure and used to calculate a 3D reconstruction.The at least two 2D update images may have been acquired using an X-ray device, such as a C-arm X-ray machine or a computed tomography (CT) scanner (gantr-based system), where non-anatomical and / or anatomical structures may have changed position during the surgical procedure. Non-anatomical structures may, for example, have moved due to movement within the patient, such as the advancement of a guidewire. Anatomical structures may, for example, have changed position due to the patient's breathing or because the patient was repositioned by the surgeon.In the case of the provided 2D update images, particularly the most recently provided ones, i.e., the most recently acquired, they are preferably 2D update images acquired within a time interval, in particular at least two 2D update images acquired almost simultaneously ("pseudo-simultaneously"), and preferably 2D update images acquired simultaneously, i.e., at a single point in time. In embodiments of the procedure where the 2D update images are not acquired simultaneously, the 2D update images may, for example, be captured using a single imaging device, i.e., a single imaging chain.According to the present invention, at least two of the provided 2D update images are acquired at different times with different viewing directions, since calculating a 3D reconstruction of non-anatomical structures generally requires more 2D update images than available image strings. Following the provision of at least two 2D update images, non-anatomical structures are extracted from a first subset of these at least two 2D update images. This first subset may consist of, for example, zero, one, two, three, etc., 2D update images. The first subset according to the present invention differs in particular from any further subsets according to the present invention in that it differs, wholly or partially, i.e., in at least one 2D update image, from the others with respect to the 2D update images it contains. The number of 2D update images in each subset may differ. The extraction may thus correspond to a discrete pixel-by-pixel segmentation, for example.In a preferred embodiment of the invention, the extraction thus represents continuous values, in particular contributions of non-anatomical structures to the 2D update images, and here in particular physically correct or approximately correct line integrals along the projection rays. According to the present invention, anatomical structures are further extracted from a second subset of the 2D update images. This extraction of anatomical structures can also be performed in such a way that the extracted non-anatomical structures are thereby removed from the 2D update images, for example, by subtraction. In a preferred embodiment of the invention, the extraction of anatomical structures is provided for in such a way that no visible or measurable gaps, such as air, appear in the resulting images at the locations of the non-anatomical structures, but rather the extraction is carried out in a manner in which its result corresponds to the case that would exist if there were no non-anatomical structures present at all in the corresponding locations and the corresponding pixels were occupied by the tissue surrounding the corresponding locations. After the extraction of non-anatomical structures from the first subset of 2D update images, a non-anatomical 3D image is calculated from at least two partial reconstructions, where the at least two partial reconstructions are calculated from the extractions. Due to the limited number of possible imaging devices, the available set of 2D update images typically contains an insufficient number of 2D update images that were acquired simultaneously or pseudo-simultaneously. Therefore, to reconstruct a non-anatomical 3D image from the set of 2D update images acquired simultaneously or pseudo-simultaneously after extraction, the first subset preferably contains 2D update images that were acquired at different times.Consequently, the extractions also represent different points in time and may therefore, due to possible movement of the patient and / or non-anatomical structures, represent different states of the non-anatomical structures, in particular different positions and orientations. To account for the fact that the extractions may represent different states of the non-anatomical structures, initially only those extractions that were acquired simultaneously or pseudo-simultaneously are back-projected onto a common 3D volume (preferably using the projection geometry under which the 2D update images were acquired), referred to in the present invention as partial reconstruction.These images, acquired simultaneously or pseudo-simultaneously, thus originate from different imaging devices within the system, specifically imaging chains such as pairs of X-ray sources and detectors. Therefore, each partial reconstruction is consistent both internally and with respect to the represented state of the non-anatomical structures. However, in general, neither the individual partial reconstructions nor their combination can be considered a suitable non-anatomical 3D image, because the partial reconstructions contain artifacts resulting from the small number of extractions underlying each partial reconstruction; for example, a filtered or unfiltered backprojection can be used to calculate the partial reconstructions.The fusion of multiple partial reconstructions using conventional computed tomography procedures does not yet allow for an accurate representation of non-anatomical structures. According to the present invention, however, it is possible to fuse the partial reconstructions using image processing operations designed to eliminate these artifacts. To keep the computation time of such a procedure as low as possible, computational operations can preferably be used that take advantage of the fact that both the partial reconstructions and the non-anatomical 3D image are generally largely empty, since typically only a small portion of the examination area contains non-anatomical structures.To eliminate artifacts from partial reconstructions, machine learning procedures are the preferred approach. These procedures use a large number of uncorrected input images, particularly a set of partial reconstructions, to learn how to reconstruct corrected output images—specifically, correct non-anatomical 3D images—that are now free of artifacts caused by an insufficient number of 2D update images. Machine learning procedures can be particularly effective here because non-anatomical structures typically exhibit a high degree of symmetry, such as cylindrical symmetry in the case of a guide wire, allowing the machine to learn its actual shape. A reconstruction of an anatomical 3D image from the anatomical structures extracted from the second subset of 2D update images can be performed serially or in parallel with the calculation of the non-anatomical 3D image. The anatomical 3D image preferably shows only the patient's anatomy, but not the non-anatomical structures. This anatomical 3D image thus contains structures whose movement is generally significantly less in speed and / or amplitude than that of the structures contained in the non-anatomical 3D image, which are typically actively guided. However, since the patient anatomy contained in it can only be accurately converted into a 3D image from a relatively large number of 2D update images, it is possible to calculate the 3D images from a relatively large number of 2D update images acquired over a longer period of time. The inventors have thus recognized that a real-time 3D representation of interventional materials within a patient's anatomy requires only real-time 3D reconstructions of the interventional materials from a few projections, whereas the 3D reconstruction of the patient's anatomy requires many 2D update images, which, however, can be acquired over a longer period. The inventors further recognized that the described separation of the corresponding 3D reconstructions, particularly when the 2D update images are 2D X-ray projections, allows for reduced-dose imaging, since the 2D X-ray projections can be acquired at a comparatively low temporal frequency, for example, 5 or 10 per second. According to the present invention, it is envisaged that, after the reconstruction of the anatomical 3D image, the anatomical 3D image is registered with the first 3D image, determining a coordinate transformation; that is, the anatomical 3D image is transferred to the coordinate system of the first 3D image, or vice versa. An advantage of such a procedure may be that changes that have occurred between the anatomical 3D image and the first 3D image, which contains at least one anatomical structure, in particular a vessel made visible by a contrast agent, can be detected. These changes may be due, in particular, to the fact that the first 3D image may be acquired using an imaging device different from the one used to acquire the 2D update images.Since the differences between the anatomical 3D image and the non-anatomical 3D image are generally relatively small and essentially limited to the movements of non-anatomical structures, such as a guidewire or catheter, the specific coordinate transformation is now also applied between the first 3D image and the non-anatomical 3D image. Therefore, it is possible to show the non-anatomical 3D image—for example, a guidewire or catheter within a vessel—in the correct position relative to the first 3D image and, consequently, relative to at least one anatomical structure. This anatomical structure is thus visible only in the first 3D image, for example, through the single use of a contrast agent within a blood vessel, but not in the non-anatomical 3D image, which generally does not show the anatomical structure, since, for example, continuous administration of a contrast agent is not possible. A navigation volume is thus generated from at least one anatomical model, the non-anatomical 3D image, and, in alternative embodiments of the procedure according to the present invention, additionally the anatomical 3D image, using the determined coordinate transformation. In the navigation volume, both the non-anatomical structures of the non-anatomical 3D image and the extracted anatomical model, as well as, in alternative embodiments of the procedure according to the present invention, additionally the anatomical structures of the anatomical 3D image, are displayed in their correct relative positions. The coordinate transformation determined from the anatomical 3D image and the first 3D image is preferably deformable.When determining the coordinate transformation, the contrast-enhanced regions of the first 3D image are preferably disregarded, if present, as they can distort the determination of the coordinate transformation. The determination of the preferably deformable coordinate transformation between the first 3D image and the anatomical 3D image can be understood as image registration. In alternative embodiments of the procedure according to the present invention, the first subset of 2D update images from which the non-anatomical 3D image is calculated may consist of the most recently acquired 2D update images, for example, the last two or three 2D update images. The number of 2D update images in the first subset may be set by a user in an organ program of the medical imaging device. For example, this organ program may also be configured to use all acquired 2D update images for the reconstruction of the non-anatomical 3D image. In alternative embodiments, it is foreseen that, if there is no non-anatomical structure present in a 2D update image of the first subset, the non-anatomical 3D image can also be reconstructed according to the present invention, this non-anatomical 3D image being empty, i.e., containing only zeros. Furthermore, in alternative embodiments of the procedure according to the present invention, the non-anatomical 3D images and / or the anatomical 3D images can be reconstructed at a temporal frequency, wherein the temporal frequency of reconstruction of the anatomical 3D images corresponds in particular to the acquisition frequency of the 2D update images. Reconstruction at a temporal frequency allows the non-anatomical 3D image to be reconstructed, for example, in real time and displayed in the navigation volume.Real-time reconstruction of non-anatomical 3D images is feasible in particular because non-anatomical structures, especially intervention materials such as cylindrical structures, e.g., guides, particularly along their axial axis, generally comprise a high degree of symmetry, and because non-anatomical structures preferentially occupy only a small portion of the volume to be represented and, therefore, these non-anatomical structures can be reconstructed from only a few 2D update images. In alternative embodiments of the procedure according to the present invention, when a new 2D update image is acquired, it can be added to the second subset of the provided 2D update images, and the anatomical 3D image can be reconstructed, at least partially, with older 2D update images being reused for the new reconstruction. This can be called overlapping reconstruction. One way to keep the computational effort for the procedure according to the present invention low in this respect is, as part of the reconstruction, simply to add at least one newly acquired 2D update image to the anatomical 3D image, for example, by filtered backprojection, and to subtract the older 2D update image, which falls outside a temporal window, from the anatomical 3D image, for example, by subtracting its filtered backprojection.The number of 2D update images in the second subset can be set by the user in an organ program. Furthermore, it is possible to incorporate any available information regarding patient movement, such as respiratory movement, cardiac movement, or movement of the patient table, into the anatomical 3D image reconstruction. In alternative embodiments, the procedure according to the present invention, in particular the calculation of the non-anatomical 3D image from the partial reconstructions, can be performed using a machine learning procedure, specifically a neural network, such as a convolutional neural network. An advantage of these embodiments may be that only a few 2D update images are required to reconstruct a non-anatomical 3D image. According to the present invention, a few 2D update images can also be understood as a single 2D update image. The essence of a machine learning procedure lies in transforming input data into output data according to a specific transformation. To enable the machine learning procedure to perform the desired transformation, its free parameters can be appropriately set or learned in an iterative process (called training). Preferably, this occurs in supervised training; that is, the machine learning procedure is presented with a large number of pairs (called training pairs), each consisting of possible input data on one hand, and output data corresponding to the transformation to be learned on the other.In this process, the network's free parameters are iteratively adjusted to minimize the value of a function, preferably a cost function, that measures the deviation between the actual output data and the desired output data according to the transformation being learned. Training pairs are preferably generated using realistic simulation, as this allows for the creation of a large number of training pairs. One advantage of this approach is that the accuracy of the machine learning procedure increases with the number of training pairs.Preferably, a large number of training pairs are generated by realistic simulation of non-anatomical structures and realistic simulation of partial reconstructions obtained from them, which contain these non-anatomical structures, where each training pair consists of a set of partial reconstructions, on the one hand, and the associated non-anatomical 3D image, on the other. In alternative embodiments of the procedure according to the present invention, it is anticipated that patient movements, such as respiratory movement, cardiac movement, or movement of the patient table, will be taken into account. This consideration of movements can be performed implicitly, thereby allowing the partial reconstructions to represent separate input channels for the procedure according to the present invention. However, it can also be performed explicitly by preceding the procedure according to the present invention with a procedure that first performs a separate motion correction; for example, translation, rotation, and / or deformation can be corrected. The procedure can be, for example, a machine learning procedure, in particular a neural network, such as a convolutional neural network. In alternative embodiments, the procedure according to the present invention, in particular the extraction of anatomical and non-anatomical structures from the 2D update images, can be performed using at least one machine learning procedure, in particular a neural network, for example, a convolutional neural network. An advantage of these embodiments may be that the machine learning procedures can perform transformations similar to the segmentation extraction disclosed in the preceding section with high accuracy. In alternative embodiments of the procedure according to the present invention, it is provided that the calculation of the non-anatomical 3D image from the 2D update images is performed by a single machine learning procedure, for example by a neural network that has been trained to first convert 2D update images into other 2D images, such as extracted anatomical structures or non-anatomical structures, and then reconstruct these other 2D images into an artifact-free non-anatomical 3D image using the same machine learning procedure. In alternative embodiments of the procedure according to the present invention, the extraction of non-anatomical structures can be performed such that only certain non-anatomical structures are extracted, for example, those belonging to a specific object class or a specific combination of multiple object classes. Consequently, only those non-anatomical structures exhibiting these specific characteristics—that is, those belonging to the specific object class or the specific combination of multiple object classes—are reconstructed in non-anatomical 3D images. If a machine learning procedure is used for the extraction, it is trained accordingly not only for the extraction but also for the differentiation of such non-anatomical structures. An advantage of such alternative embodiments may be, for example, when guiding a guidewire within a patient, to extract and reconstruct only this guidewire.Other non-anatomical structures, such as those already present and, in particular, belonging to other object classes (e.g., stents or orthopedic implants from previous procedures), or non-anatomical structures not specific to the procedure (e.g., the patient table), which are not of interest, are not then extracted or reconstructed, so that the image shown to the treating physician is reduced to the essentials. In alternative embodiments of the procedure according to the present invention, the extraction of non-anatomical structures can be performed such that multiple separate extractions are calculated from each 2D update image, wherein each of the separate extractions shows only a subset of the non-anatomical structures, for example, only those belonging to a specific object class or a specific combination of multiple object classes. The extractions are then processed separately to obtain 3D non-anatomical images, wherein the separate 3D non-anatomical images calculated in this manner contain only the corresponding subset of the non-anatomical structures. The separate 3D non-anatomical images are then combined into a single 3D non-anatomical image, for example, by adding separate voxel-based 3D non-anatomical images.One advantage of such alternative realizations may be that an anatomical 3D image generated by separate 3D reconstructions of subsets of non-anatomical structures and subsequent combination may be more accurate than an anatomical 3D image reconstructed in a single pass, since the reconstruction problem is generally simpler the smaller the proportion of non-anatomical structures in the volume to be represented. An additional aspect of the invention relates to a medical imaging device, in particular a gantr-based system, for performing an imaging examination and displaying non-anatomical structures, comprising the following components: a provisioning unit to provide a first 3D image, wherein the first 3D image contains at least one anatomical structure; at least two image strings, wherein the image strings are adapted to acquire 2D images, in particular 2D refresh images and / or 3D images; a unit of calculation, where the computing unit is adapted to extract an anatomical model of at least one anatomical structure, where the computing unit is adapted to extract anatomical and non-anatomical structures from the 2D update images, where the calculation unit is adapted to calculate partial reconstructions from the extraction of non-anatomical structures, where the computing unit is adapted to calculate a non-anatomical 3D image from at least two partial reconstructions, where the computing unit is adapted to calculate an anatomical 3D image from the extraction of anatomical structures, and wherein the calculation unit is adapted to generate a navigation volume from the anatomical model and the non-anatomical 3D image by determining a coordinate transformation, wherein the coordinate transformation is adapted for the positionally correct positioning of the anatomical model within the navigation volume; a display unit where the navigation volume is shown. The device according to the present invention includes a provisioning unit, preferably a storage unit such as a USB flash drive, hard drive, or other portable or permanently installed data storage medium. The provisioning unit can also be understood as a connection to a storage unit that is connected to a network accessible to the medical imaging device, thereby providing a first 3D image, wherein the first 3D image contains at least one anatomical structure. The device according to the present invention comprises at least two imaging chains, and may also comprise, for example, three, four, or more imaging chains. An imaging chain comprises an X-ray generator for generating X-rays, for example, a rotating anode-based generator, and a receiver unit for detecting X-rays, such as a flat-panel detector, wherein the imaging chains are adapted to acquire 2D images, in particular 2D refresh images, and / or 3D images. One advantage of using at least two image chains is that multiple 2D update images can be acquired simultaneously, pseudo-simultaneously, or in direct succession. Preferably, the medical imaging device includes two image chains, since a greater number of image chains increases the complexity, cost, and probability of failure of the medical imaging device. 2D update image acquisition preferably occurs in tuples, where the gantr-based system has preferably not rotated further, or has only rotated approximately, between the acquisition of 2D update images of a tuple. In contrast, the gantr-based system can preferably rotate further by a minimum angle of, for example, at least 5° or 20° between the acquisition of tuples.When the gantr-based system rotates, the rotation speed is preferably low, since motion blur can occur at excessively high rotation speeds. Furthermore, the device according to the present invention includes a computing unit, wherein this computing unit may be a graphics processing unit (GPU) or another type of processing unit, such as a central processing unit (CPU), wherein, in the case of a GPU, it may be used to perform massively parallel or highly parallel computational operations. A computing unit may also consist of multiple computing units, which in their entirety may again be considered a computing unit according to the present invention. Moreover, this computing unit extracts an anatomical model of the anatomical structure from the first 3D image. The computing unit is also adapted to extract anatomical and non-anatomical structures from at least two 2D update images.Furthermore, the calculation unit reconstructs at least two partial reconstructions from the extractions of non-anatomical structures from the first subset of 2D update images, and also reconstructs an anatomical 3D image from the extraction of anatomical structures from the second subset of 2D update images. The calculation unit is also adapted to calculate a non-anatomical 3D image from at least two partial reconstructions. The calculation unit further serves to create a navigation volume from the anatomical model, the non-anatomical 3D image, and optionally the anatomical 3D image by determining a coordinate transformation, where the coordinate transformation is configured to correctly position the anatomical model, the non-anatomical 3D image, and optionally the anatomical 3D image, within the navigation volume. Furthermore, the device according to the present invention comprises a display unit in which the navigation volume is shown, as well as, optionally in each case, the non-anatomical 3D image, the anatomical 3D image, the first 3D image, and / or the anatomical model. The display unit can also be adapted to display 2D update images. In further advantageous embodiments of the present invention, the display unit can additionally, particularly when the gantry is stationary, display conventional X-ray images, such as fluoroscopy, as well as post-processing of such X-ray images performed by the computing unit, for example, using a frequency filter. In further embodiments of the present invention, it is provided that images calculated in the computing unit using a subtraction procedure, for example, a DSA, are displayed. The invention is explained in more detail below with reference to the figures. Fig. 1 schematically shows a sequence of the procedure according to the present invention. Fig. 2 shows, as an example, a flowchart of the procedure according to the present invention, in which new 2D update images are added. Fig. 3 shows an exemplary flowchart of the procedure according to the present invention, wherein new 2D update images are added and old 2D update images are replaced by them. Fig. 4 schematically shows an image acquisition device capable of performing the procedure according to the present invention. Figure 1 shows an exemplary flow of the procedure according to the present invention. First, the procedure according to the present invention is provided with a first 3D image 1 (3D volume). This first 3D image 1 can be acquired during or before an interventional procedure by means of a 3D scan. Furthermore, an anatomical model 5 is extracted from this first 3D image 1. Subsequently, according to the present invention, at least two 2D update images 2 are provided, wherein the 2D update images 2 contain at least partially anatomical structures 3 and / or at least partially non-anatomical structures 4 of an examination area. The at least partially anatomical structures 3 are thus parts of bone structures and / or parts of vascular structures and / or parts of soft tissue. In Fig. 1, a guide wire in an examination area is present as a non-anatomical structure 4. Furthermore, the at least two 2D update images 2 were acquired within a time interval. After the provision of at least two 2D update images 2, 6 non-anatomical structures 4 are extracted from a first subset 7 of the at least two 2D update images 2. Based on these extractions 6 of the non-anatomical structures 4 from the first subset 7 of the 2D update images 2, a calculation of a 3D non-anatomical image 10 is performed from several partial reconstructions 9. In Fig. 1, this is done using four partial reconstructions 9. The various partial reconstructions 9 are thus derived from 2D update images that were acquired at different times and may thus, due to possible movements of the patient and / or of the non-anatomical structures, represent different states, positions, and orientations of the non-anatomical structures 4. In addition to the extraction 6 of the non-anatomical structures 4, an extraction 13 of the anatomical structures 3 is also performed from a second subset 8 of the 2D update images 2, wherein the second subset 8 differs from the first subset 7 in at least one 2D update image 2, such that the two subsets (7, 8) are not identical. An anatomical 3D image 11 is calculated from the extraction 13 of anatomical structures 3. According to the present invention, it is envisaged that, after the reconstruction of the anatomical 3D image 11, the anatomical 3D image 11 is registered with the first 3D image 1, with the determination of a coordinate transformation 14. Subsequently, the coordinate transformation 14 thus obtained is applied 16 to the anatomical model 5. The coordinate transformation determined from the anatomical 3D image 11 and the first 3D image 1 is preferably deformable. When determining the coordinate transformation 14, the contrast-enhanced regions of the first 3D image 1, if present, are preferably disregarded, as they may distort the determination of the coordinate transformation. Then, a navigation volume 12 is created from at least one anatomical model 5 and the non-anatomical 3D image 10. Thus, in navigation volume 12, both the non-anatomical structures 4 from the non-anatomical 3D image 10 and the extracted anatomical model 5 are shown in the correct relative positions to each other. Figure 2 shows a sequence of the procedure according to the present invention, wherein, after acquiring two new 2D update images 21, these are added to the first subset 7 and the second subset 8 of the provided 2D update images 2, and the anatomical 3D image 11 is reconstructed at least partially, with all the older 2D update images 2 being able to be reused for the renewed reconstruction of the anatomical 3D image 11. In addition, the non-anatomical 3D image 10 is reconstructed using the new 2D update images 21. The sequence of the procedure according to the present invention described herein is particularly advantageous, for example, at the beginning of image acquisition, when there are not yet enough 2D update images 2 in the second subset 8 to reconstruct an anatomical 3D image 11 with few artifacts.Increasing the number of 2D refresh images in the second subset 8 preferentially results in higher image quality. Figure 3 shows an embodiment of the procedure according to the present invention, wherein two newly acquired 2D update images 21 are added to the second subset 8 and the two older 2D update images 31 are removed from the second subset. A new 3D anatomical image 11 is then reconstructed from the second subset modified in this manner. This may be termed overlapping reconstruction. An advantage of this embodiment is that the 3D anatomical image modified in this manner now contains more up-to-date information about the anatomical structures 3, which may have changed, for example, due to respiratory, cardiac, or table movement of the patient. Figure 4 schematically shows an imaging device adapted to perform the procedure according to the present invention. The imaging device comprises a rotatably mounted gantry 43, on which two X-ray generators 42 are mounted, as well as flat-panel X-ray absorbing detectors 41 mounted directly opposite the X-ray generators. The unit consisting of an X-ray generator and the flat-panel detector 41 arranged opposite it may be called an imaging chain. These imaging chains can be used to provide an initial 3D image and 2D update images for the procedure according to the present invention. The various imaging chains can thus acquire the 2D update images simultaneously, pseudo-simultaneously, or with a time lag. The region 44 to be examined using X-rays 45 may contain anatomical and non-anatomical structures.The computing units 47 required to process the 2D update images and the initial 3D image are integrated into an external processing unit 46 in Fig. 4. Alternatively, this processing unit 46 or the computing units 47 can be integrated directly into the imaging device. The navigation volume according to the present invention can be displayed on an external display unit 48. Alternatively, the display unit can also be mounted directly on the imaging device. List of reference signs 1 first 3D image 2D update image 3 anatomical structures 4 non-anatomical structure 5 anatomical model 6. Extraction of non-anatomical structures 7 first subset 8 second subset 9 partial reconstruction 10 non-anatomical 3D images 11. Anatomical 3D image 12 navigation volume 13. Extraction of anatomical structures 14 determination of coordinate transformation 15. Creation of navigation volume 16 coordinate transformation application 21 newly acquired 2D update image 31 2D update images removed 41 X-ray detector 42 X-ray generator 43 gantr and rotatably mounted 44 examination area 45 X-rays 46 processing unit 47 calculation unit 48 display unit
Claims
1. A method for operating a medical imaging device for the positionally correct representation of non-anatomical structures during an imaging examination, comprising the following steps: a.) making available a first 3D image (1) containing at least one anatomical structure (3); b.) extracting at least one anatomical model (5) from said at least one anatomical structure (3) from the first 3D image (1); c.) making available at least two 2D update images (2) by means of the medical imaging device,where at least two 2D update images (2) were acquired at different times; d.) extraction (6) of non-anatomical structures (4) from a first subset (7) of the 2D update images (2); e.) extraction (13) of anatomical structures (3) from a second subset (8) of the 2D update images (2); f.) calculation of a non-anatomical 3D image (10) from at least two partial reconstructions (9) of the first subset (7),wherein at least two partial reconstructions (9) are calculated from the extraction (6) of the non-anatomical structures (4) and artifacts from the partial reconstructions are removed by means of machine learning procedures; g.) reconstruction of an anatomical 3D image (11) from the extraction (13) of the anatomical structures (3) from the second subset (8); h.) registration of the anatomical 3D image (11) with the first 3D image (1) to determine a coordinate transformation (14); i.) creation of a navigation volume (12) from at least one anatomical model (5) and the non-anatomical 3D image (10) using the determined coordinate transformation (14).
2. A method according to claim 1, characterized in that the navigation volume (12) is further created from the anatomical 3D image (11) using the specified coordinate transformation (14).
3. A method according to claim 1,characterized in that the first 3D image (1) is acquired with an X-ray device, in particular an X-ray C-arm or a computed tomography scanner, or with a magnetic resonance imaging scanner.
4. A method according to any of the preceding claims, characterized in that the 2D update images (2) are acquired with an X-ray device, in particular an X-ray C-arm, or a computed tomography scanner.
5. A method according to any of the preceding claims, characterized in that, in the absence of non-anatomical structures (4) in a 2D update image (2) of the first subset (7), the non-anatomical 3D image (10) continues to be reconstructed.
6. A method according to any of the preceding claims, characterized in that newly acquired 2D update images (21) are added to the first subset (7) and / or the second subset (8),and the non-anatomical 3D images (10) and / or the anatomical 3D images (11) are reconstructed in particular at a temporal frequency, wherein the temporal frequency of reconstruction of the non-anatomical 3D images (10) and / or the anatomical 3D images (11) corresponds to a function of the acquisition frequency of the 2D update images (21).
7. A method according to any of the preceding claims, characterized in that, when a new 2D update image (21) is acquired, it is added to the second subset (8) of the provided 2D update images (2), and the anatomical 3D image (11) is reconstructed at least partially, with older 2D update images being reused or removed for the new reconstruction.
8. A method according to any of the preceding claims, characterized in that the method according to the invention,In particular, the calculation of the non-anatomical 3D image (10) from the partial reconstructions (9) is performed using a machine learning procedure, in particular a neural network.
9. A method according to any of the preceding claims, characterized in that patient movements are taken into account.
10. A method according to any of the preceding claims, characterized in that the method, in particular the extraction (6, 13) of the anatomical (3) and / or non-anatomical (4) structures from the 2D update images (2), is performed using at least one machine learning procedure, in particular a neural network.
11. A method according to any of the preceding claims, characterized in that the calculation of the non-anatomical 3D image (10) from the 2D update images (2) is performed by a single machine learning procedure.in particular a neural network.
12. A method according to any of the preceding claims, characterized in that only those non-anatomical structures (4) belonging to a specific object class or a combination of specific object classes are extracted.
13. A method according to any of the preceding claims, characterized in that non-anatomical structures (4) belonging to a specific object class or a combination of specific object classes are reconstructed separately.
14. A method according to any of the preceding claims, characterized in that the non-anatomical structures (4) are a guide wire and / or a catheter.
15. A medical imaging device for performing an imaging examination and rendering non-anatomical structures, comprising: A rendering unit for rendering a first 3D image (1),wherein the first 3D image contains at least one anatomical structure; at least two image strings (41, 42), wherein the image strings are configured to acquire 2D update images (2); a calculation unit (47), wherein the calculation unit (47) is configured to: extract an anatomical model (5) from said at least one anatomical structure (3), extract (6) non-anatomical structures (4) from a first subset (7) of the 2D update images (2); extract (13) anatomical structures (3) from a second subset (8) of the 2D update images (2); compute a non-anatomical 3D image (10) from at least two partial reconstructions (9) of the first subset (7),wherein at least two partial reconstructions (9) are calculated from the extraction (6) of the non-anatomical structures (4) and artifacts from the partial reconstructions are removed by means of machine learning procedures; reconstructing an anatomical 3D image (11) from the extraction (13) of the anatomical structures (3) from the second subset (8); registering the anatomical 3D image (11) with the first 3D image (1) to determine a coordinate transformation (14); and creating a navigation volume (12) from at least one anatomical model (5) and the non-anatomical 3D image (10) using the determined coordinate transformation (14); a display unit (48), in which the navigation volume (12) is displayed.