METHOD FOR OPERATING A MEDICAL IMAGE DEVICE FOR CORRECTLY DISTINGUISHING NON-ANATOMICAL STRUCTURES DURING AN IMAGE EXAMINATION AND DEVICE FOR THIS PURPOSE

Description

[0001] The invention relates to a method for operating a medical imaging device for the correct positioning of non-anatomical structures during an imaging examination. The invention further comprises a medical imaging device.

[0002] In medical surgical procedures, such as vascular surgery, the aim is to perform these interventions into the patient's body using minimally invasive techniques, as this approach places the least possible burden on the patient. Access to the body's interior is achieved through small incisions made by the surgeon or, alternatively, through the patient's body orifices. Because the target area of ​​such a procedure is difficult or, in many cases, impossible for the surgeon to visualize, non-anatomical structures, particularly interventional materials such as guide wires, catheters, stents, coils, and screws—that is, structures inserted into the patient for the procedure—and the surrounding patient anatomy (anatomical structures) are visualized using imaging techniques.Projective medical imaging techniques are often used for this purpose, preferably 2D techniques, i.e., techniques that project the three-dimensional structure of a volume to be imaged onto a two-dimensional surface, thus generating 2D images. 2D X-ray fluoroscopy is particularly common, meaning that 2D images, especially 2D X-ray images, are acquired sequentially and displayed on a screen. Information about the spatial (3D) characteristics of the patient and, for example, the intervention material—i.e., their three-dimensional shape—is largely lost in this process, since all structures lying on a projection beam, such as an X-ray beam, are projected onto a single point on a surface; these structures appear superimposed on a 2D image.2D images consist of pixels, each of which can be assigned different values, such as integer values ​​like 1 or 0. Thus, a structure lying on a projection beam, for example, an X-ray beam, is projected onto a pixel of a 2D image, with the pixel values ​​depending on the type and nature of the structures being projected. If the treating physician had access to a rapid sequence of 3D reconstructions showing the inserted non-anatomical structures, such as interventional materials, and the patient's anatomy in a 3D view, they would obtain a 4D view, i.e., three spatial dimensions plus one temporal dimension. This means the treating physician would have a 4D interventional guide available for performing the procedure, allowing them to assess the position of the interventional material inserted into the patient at any given moment.

[0003] Providing a 3D reconstruction requires a tomography scan, which can be performed using a computed tomography (CT) scanner, a C-arm X-ray unit (which can be mobile or stationary), or a magnetic resonance imaging (MRI) scanner. A 3D reconstruction is a 3D volume calculated from numerous individual measurements. In computed tomography, these individual measurements are 2D X-ray projections. The continuous, sequential acquisition of many 3D X-ray images necessary for the continuous recording of the patient and the surgical material results in a very high radiation exposure for the patient. This currently precludes the use of 4D interventional guidance in surgery, particularly in vascular surgery.A computed tomography scanner can be any imaging device that captures 2D projection images and calculates a tomography from them.

[0004] The state of the art for guiding interventional materials, particularly in blood vessels, encompasses a variety of methods. A key challenge with X-ray-based imaging techniques is that blood vessels exhibit nearly the same contrast as the surrounding tissue and are therefore initially invisible, especially on X-rays or computed tomography (CT) scans. When taking X-rays or CT scans, it is therefore necessary to administer a contrast agent into the vascular system, such as iodine or carbon dioxide, which exhibits higher or lower X-ray absorption than the surrounding tissue, thus making the vascular system visible. Administering contrast agents to the patient can also have the effect of allowing the surgeon to identify vessels with reduced or no blood flow within the vascular system.

[0005] Contrast-enhanced images can be used in a variety of procedures. In subtraction-based 2D imaging techniques, they can be used to depict the dynamics of blood flow or the shape of a vascular system after subtracting a non-contrast image (mask), for example, in digital subtraction angiography (DSA). They can also be used to create a guide for an instrument within a vessel or vascular system. However, this requires the administration of contrast agent again whenever the imaging geometry changes or the patient moves, resulting in increased radiation exposure. Repeated administration of iodine-based contrast agents can also be toxic to the kidneys and is therefore contraindicated in cases of renal insufficiency. Furthermore, the depth information in the form of the third dimension is lost in the manner described in the previous section.

[0006] It is therefore recommended to use a 3D reconstruction of the vessels, which can be extracted, for example, from a previously recorded contrast-enhanced computed tomography scan and superimposed, for example, as a contour in the correct position by determining and using the correct projection geometry into a live 2D image.

[0007] The procedures described in the previous section have the disadvantage that the contrast-filled vessels and, if present, the non-anatomical structures are only visible to the treating physician on a live 2D image. Even when using multiple X-ray units, especially C-arm X-ray units, particularly mobile C-arm X-ray units with different projection directions, such as orthogonal projection, complete 3D information is not available. Therefore, a considerable amount of time, radiation dose, and contrast agent may be required to, for example, correctly navigate a guidewire within a vascular system, which places an enormous burden on the patient.

[0008] Document EP2656314B1 discloses a method and a system which, based on a compressed scan, enable radiological guidance of an instrument during a medical examination.

[0009] German patent DE 10 2008 054 298 A1 discloses a method and a device for 3D visualization of a surgical path of a medical instrument, a medical instrument and / or a specific tissue structure of a patient to support a medical procedure on the patient's tissue. In the method and with the device, a surgical path is determined and / or the medical instrument and / or the specific tissue structure is identified based on two 2D X-ray images of the body area containing the patient's tissue and / or the medical instrument and / or the specific tissue structure, generated from different projection directions using an X-ray device.The intervention path and / or the medical instrument and / or the specific tissue structure are incorporated into a 3D patient computational model, which features the tissue as model tissue, and at least a section of the 3D patient computational model or the model tissue with the model intervention path and / or model instrument and / or model tissue structure superimposed is displayed on a viewing device.

[0010] The current state of the art has the disadvantage that the 2D X-ray projections required for 3D reconstructions of the interventional materials at a specific time must be acquired simultaneously or almost simultaneously (pseudo-simultaneously) in order to accurately depict the interventional materials in the presence of movement, particularly caused by the manipulation of an instrument by the treating physician. If, however, there is a time lag between the acquisition of one or more 2D X-ray projections, a consistent 3D reconstruction at a specific time is generally not possible. Depending on the degree of movement, motion artifacts may occur, or the 3D reconstructions may become completely unusable.Such a time lag occurs particularly when the X-ray machine used for the scan has too few image sequences, i.e., pairs of X-ray source and detector, because in this case the X-ray system has to change the scanning geometry between the acquisition of 2D update projections, for example by rotating a gantry. If this rotation does not occur quickly enough, the described disadvantage of inconsistent and potentially unusable 3D reconstruction results.

[0011] US Patent 2021 / 0128011 A1 discloses an imaging method for radiological guidance of an instrument during medical procedures on an object, comprising: a) providing an initial image of the object followed by b) providing updated images during the procedure to an operator by measuring an undersampled set of projections of the object and reconstructing the updated image based on changes between the initial image or an update of the initial image and the undersampled set of projections.Document D1 further relates to specific uses of the procedure and a system for radiologically guiding medical interventions on an object according to the procedure, comprising means for providing a first image of the object, an imaging device measuring undersampled sets of projections, processing means associated with the imaging device to provide updated images during the intervention by reconstructing the updated image on the basis of changes between the first image or an update of the first image and the undersampled set of projections.

[0012] One obvious solution to this problem, to those skilled in the art, is to enable very rapid changes in the imaging geometry, for example, by means of a rapidly rotating gantry. However, this places high demands on both the mechanical support of the X-ray system, such as the gantry, and its components, as well as on the components required for capturing 2D X-ray projections. To minimize motion blur in the captured 2D X-ray projections, which arises from the rapid changes in the imaging geometry during exposure of the X-ray detectors, the X-ray source must emit in very short but comparatively intense pulses, and / or the acquisition rate of the X-ray detectors must be very high. This generally increases the cost of such a system or is technically impossible when using flat-panel X-ray detectors.

[0013] The object of the present invention is therefore to provide an improved method for the correct positioning of non-anatomical structures and anatomical structures during an imaging examination.

[0014] The object of the invention is achieved according to the invention by the subject matter of the independent claims. Advantageous embodiments are specified in the dependent claims.

[0015] The method according to the invention is based on the provision of a first 3D image (3D volume). This first 3D image is either acquired during an interventional procedure using a 3D scan or, for example, loaded from a patient archive, wherein, in the case of loading the first 3D image from a patient archive, the first 3D scan was acquired preoperatively. The first 3D image, which contains at least one anatomical structure, can be produced, for example, by a preoperative computed tomography or magnetic resonance imaging scan. The at least one anatomical structure of the first 3D image can, for example, include partially bone and / or partially vascular structures as well as surrounding anatomy, such as organs, muscles, or other soft tissues.Furthermore, the first 3D image can include additional non-anatomical structures, such as implants or interventional materials from previous surgical procedures, and additional anatomical structures, such as skin edges. Alternatively, the first 3D image can also be acquired intraoperatively, preferably using an intraoperative computed tomography scanner or a C-arm X-ray unit, particularly a mobile C-arm X-ray unit. Preferably, the first 3D image is acquired using the same device that is subsequently used for 4D interventional guidance. Furthermore, the first 3D image can be imported into an internal storage unit, such as an internal image data storage device, or an external storage unit, such as a USB flash drive, an external hard drive, or online storage, which is accessible to the imaging device performing the method according to the invention, preferably an X-ray unit.An anatomical model of at least one anatomical structure from the first 3D image is extracted before or after the import process. According to the invention, extraction is the calculation of an anatomical model, which can, for example, correspond to a voxel-based segmentation or a parametric representation. If, for example, the at least one anatomical structure is an organ, preferably a hollow organ, such as a contrast-enhanced blood vessel, a contrast-enhanced heart chamber, or an intestine, then, for example, a model of the surface of the hollow organ can be calculated. If, for example, the at least one anatomical structure is a bone, such as a vertebral body, then, for example, a segmentation of all associated voxels of the anatomical structure can be calculated.

[0016] According to the invention, at least two 2D update images are subsequently provided, wherein the 2D update images contain at least partially anatomical structures and / or at least partially non-anatomical structures of an examination area. The at least partially anatomical structures can be parts of the bone structures, parts of the vascular structures, or parts of bone and vascular structures. The non-anatomical structures are, in particular, interventional materials inserted into the patient being treated, for example, guide wires, catheters, stents, coils, or screws. The at least two 2D update images provided can consist partly of newly acquired 2D update images and partly of 2D update images that were acquired during the surgical procedure and used to calculate a 3D reconstruction.The at least two 2D update images may have been acquired using an X-ray device, for example, a C-arm X-ray unit or a computed tomography scanner (gantry-based system), whereby the non-anatomical and / or anatomical structures may have undergone a change in their position during the surgical procedure. The non-anatomical structures may have been moved, for example, by movement within the patient, such as by advancing a guide wire. The anatomical structures may have been moved, for example, due to the patient's breathing or by the surgeon repositioning the patient. With regard to the provided images, especially the newly provided ones, i.e.,The most recently acquired 2D update images are preferably 2D update images acquired within a time interval, in particular at least two 2D update images acquired almost simultaneously ("pseudo-simultaneously"), preferably 2D update images acquired simultaneously, i.e., at a single point in time. In variants of the method in which 2D update images are not acquired simultaneously, the 2D update images can, for example, be acquired with a single recording device, i.e., a single image sequence. According to the invention, at least two of the provided 2D update images are acquired at different times with different viewing directions, since, as a rule, more 2D update images are required for the calculation of a 3D reconstruction of the non-anatomical structures than image sequences are available.

[0017] Following the provision of at least two 2D update images, non-anatomical structures are extracted from a first subset of these at least two 2D update images. This first subset can consist of, for example, zero, one, two, three, etc., 2D update images. The first subset according to the invention is, in particular, different from potential further subsets according to the invention in that they differ completely or partially, i.e., in at least one 2D update image, from each other with respect to the 2D update images they contain. The number of 2D update images in each subset can differ. The extraction can, for example, correspond to a discrete, pixel-wise segmentation.In an advantageous embodiment of the invention, the extraction can represent continuous values, in particular contributions of the non-anatomical structures to the 2D update images, and in particular physically correct or approximately correct line integrals along the projection rays.

[0018] According to the invention, anatomical structures are further extracted from a second subset of the 2D update images. This extraction of anatomical structures can also be carried out in such a way that the extracted non-anatomical structures are removed from the 2D update images according to their contribution to them, e.g., by subtraction. In an advantageous embodiment of the invention, the extraction of the anatomical structures is performed in such a way that no visible or measurable gaps appear in the resulting images at the locations of non-anatomical structures, such as air gaps, but rather that the extraction is carried out in such a way that the result corresponds to the situation that would exist if no non-anatomical structure were present at the corresponding locations and the corresponding pixels were occupied by the tissue surrounding the corresponding locations.

[0019] Following the extraction of non-anatomical structures from the first subset of 2D update images, a non-anatomical 3D image is calculated from at least two partial reconstructions, with the at least two partial reconstructions being calculated from the extractions. Due to the limited number of available acquisition devices, the set of 2D update images typically contains an insufficient number of 2D update images that were acquired simultaneously or pseudo-simultaneously. Therefore, in order to reconstruct a non-anatomical 3D image from the set of simultaneously or pseudo-simultaneously acquired 2D update images after extraction, the first subset preferably contains 2D update images that were acquired at different times.Consequently, the extractions also represent different points in time and can therefore, due to possible movement of the patient and / or the non-anatomical structures, depict different states of the non-anatomical structures, in particular different positions and orientations. To account for the fact that the extractions can depict different states of the non-anatomical structures, only those extractions that were acquired simultaneously or pseudo-simultaneously are initially reconstructed into a common 3D volume, referred to according to the invention as a partial reconstruction (preferably using a projection geometry under which the 2D update images were acquired). These simultaneously or pseudo-simultaneously acquired images originate in particular from different acquisition devices present in the system, especially image chains, for example, pairs of X-ray source and detector.Thus, each partial reconstruction is consistent both internally and with respect to the depicted state of the non-anatomical structures. However, neither individual partial reconstructions nor their combination can generally be considered a suitable non-anatomical 3D image, because the partial reconstructions contain artifacts. These artifacts result from the small number of extractions underlying each partial reconstruction; for example, a filtered or unfiltered backprojection can be used to calculate the partial reconstructions. Therefore, combining several partial reconstructions using conventional methods known in computed tomography does not yet allow for a correct representation of the non-anatomical structures. According to the invention, however, it is possible to combine the partial reconstructions using image processing operations that aim to remove these artifacts.To minimize the computation time for such a procedure, it is preferable to use computational operations that exploit the fact that both the partial reconstructions and the non-anatomical 3D image are generally largely empty, since typically only a small portion of the examination area contains non-anatomical structures. Machine learning methods can preferably be employed to remove the artifacts from the partial reconstructions. These methods learn to reconstruct corrected output images, particularly accurate non-anatomical 3D images, using a large number of uncorrected input images, especially a number of partial reconstructions. These output images then contain no artifacts resulting from an insufficient number of 2D update images.Machine learning methods can be particularly successful here because the non-anatomical structures usually have a high degree of symmetry, for example cylindrical in the case of a guide wire, which makes it possible to learn their actual shape.

[0020] An anatomical 3D image can be reconstructed serially or in parallel with the calculation of the non-anatomical 3D image from the anatomical structures extracted from the second subset of 2D update images. The anatomical 3D image preferably shows only the patient's anatomy, not the non-anatomical structures. This anatomical 3D image contains structures whose movement is generally significantly smaller in speed and / or amplitude than the structures in the non-anatomical 3D image, which are usually actively manipulated. However, since the patient anatomy contained in the non-anatomical 3D image can only be correctly converted into a 3D image from a comparatively large number of 2D update images, it is possible to calculate the 3D images from a relatively large number of 2D update images acquired over a longer period.

[0021] The inventors thus recognized that real-time 3D visualization of interventional materials in a patient's anatomy requires only real-time 3D reconstructions of the interventional materials from a few projections, whereas 3D reconstruction of the patient's anatomy requires many 2D update images, which, however, can be acquired over a longer period. The inventors further recognized that the described separation of the corresponding 3D reconstructions, especially when the 2D update images are 2D X-ray projections, enables dose-reduced imaging, since the acquisition of the 2D X-ray projections can be performed at a comparatively low rate, for example, 5 or 10 per second.

[0022] According to the invention, after the reconstruction of the anatomical 3D image, the anatomical 3D image is registered with the first 3D image by determining a coordinate transformation; that is, the anatomical 3D image is transformed into the coordinate system of the first 3D image, or the first 3D image is transformed into the coordinate system of the anatomical 3D image. An advantage of this procedure is that it allows for the detection of any changes that occur between the anatomical 3D image and the first 3D image, which contains at least one anatomical structure, in particular a vessel visualized by contrast medium. These changes may be caused, in particular, by the fact that the first 3D image may be acquired with a different imaging device than the one used for acquiring the 2D update images.Since the differences between the anatomical 3D image and the non-anatomical 3D image are generally relatively small and essentially limited to the movements of non-anatomical structures, such as a guide wire or a catheter, the specific coordinate transformation now also applies between the first 3D image and the non-anatomical 3D image. It is therefore possible to display the non-anatomical 3D image, for example, a guide wire or a catheter within a vessel, in the correct position relative to the first 3D image, and thus to at least one anatomical structure. This anatomical structure is only visible in the first 3D image, for example, through the one-time use of a contrast agent within a blood vessel, but not in the anatomical 3D image, which generally does not show the anatomical structure, as, for example, continuous contrast agent administration is not possible.

[0023] Finally, a navigation volume is created from the at least one anatomical model, the non-anatomical 3D image, and, in alternative embodiments of the method according to the invention, additionally the anatomical 3D image, using the determined coordinate transformation. Thus, in the navigation volume, both the non-anatomical structures from the non-anatomical 3D image and the extracted anatomical model, as well as, in alternative embodiments of the method according to the invention, the anatomical structures from the anatomical 3D image, are displayed in the correct relative positions. The coordinate transformation, which was determined from the anatomical 3D image and the first 3D image, is preferably deformable.When determining the coordinate transformation, the contrast-containing areas of the first 3D image are preferably disregarded, if present, as they can distort the determination of the coordinate transformation. The determination of the preferably deformable coordinate transformation between the first 3D image and the anatomical 3D image can be understood as image registration.

[0024] In alternative embodiments of the method according to the invention, the first subset of 2D update images from which the non-anatomical 3D image is calculated can be the most recently acquired 2D update images, for example, the last two or three 2D update images, wherein the number of 2D update images in the first subset can be set by a user in an organ program of the medical imaging device. For example, it can also be set in this organ program that all acquired 2D update images are used for the reconstruction of the non-anatomical 3D image.

[0025] In alternative embodiments, it is provided that if no non-anatomical structure is present in a 2D update image of the first subset, the non-anatomical 3D image can also be reconstructed according to the invention, wherein this non-anatomical 3D image is empty, i.e., contains only zeros, for example.

[0026] Furthermore, in alternative embodiments of the method according to the invention, the non-anatomical 3D images and / or the anatomical 3D images can be reconstructed, in particular at a specific time rate, wherein the time rate of the reconstruction of the anatomical 3D images corresponds, in particular, to the acquisition rate of the 2D update images. Reconstruction at a specific time rate makes it possible, for example, to reconstruct the non-anatomical 3D image in real time and display it in the navigation volume.The real-time reconstruction of non-anatomical 3D images is particularly feasible because non-anatomical structures, especially interventional materials such as cylindrical structures like guide wires, typically exhibit high symmetry, particularly along their axial axis, and because the non-anatomical structures preferably only fill a small portion of the volume to be imaged, and therefore these non-anatomical structures can be reconstructed from only a few 2D update images.

[0027] In alternative embodiments of the method according to the invention, when a new 2D update image is acquired, it can be added to the second subset of the available 2D update images, and the anatomical 3D image can be at least partially reconstructed, whereby older 2D update images can be reused for the reconstruction. This can be referred to as overlapping reconstruction. One way to keep the computational effort for the method according to the invention low is to simply add at least one newly acquired 2D update image to the anatomical 3D image during the reconstruction, for example by means of filtered backprojection, and to remove the oldest 2D update image, which falls outside a certain time window, from the anatomical 3D image by subtraction, for example by subtracting its filtered backprojection.The number of 2D update images in the second subset can be set by the user in an organ-specific program. Furthermore, it is possible to incorporate any available information about the patient's movement, such as breathing, heart rate, or table movement, into the reconstruction of the anatomical 3D image.

[0028] In alternative embodiments, the method according to the invention, in particular the calculation of the non-anatomical 3D image from the partial reconstructions, can be performed using a machine learning method, especially a neural network, for example a convolutional neural network. An advantage of these embodiments is that only a few 2D update images are required to reconstruct a non-anatomical 3D image. According to the invention, a few 2D update images can also be considered a single 2D update image.

[0029] The essence of a machine learning method lies in transforming input data into output data according to a specific transformation. To enable the machine learning method to perform the desired transformation, its free parameters can be appropriately adjusted or learned in an iterative process (called training). Preferably, this is done in supervised training, meaning the machine learning method is presented with a multitude of pairs (called training pairs), each consisting of a possible input data on the one hand and the corresponding output data, according to the transformation to be learned, on the other. Here, the free parameters of the network are iteratively adjusted so that the value of a function, preferably a cost function, which measures the deviation between the actual output data and the desired output data according to the transformation to be learned, is minimized.The training pairs are preferably created using realistic simulation, as this allows for the generation of a large number of training pairs. An advantage of this approach is that the accuracy of the machine learning method increases with the number of training pairs. Preferably, a large number of training pairs are generated by realistic simulation of non-anatomical structures and realistic simulation of the resulting partial reconstructions that include these non-anatomical structures. Each training pair consists of a set of partial reconstructions on the one hand and the corresponding non-anatomical 3D image on the other.

[0030] In alternative embodiments of the method according to the invention, it is provided that movements occurring during the patient's journey, such as breathing, heartbeat, or movement of the patient table, are taken into account. This consideration of movements can be implicit, by having the partial reconstructions represent separate input channels for the method according to the invention. However, it can also be explicit, by preceding the method according to the invention with a separate method that first performs a movement correction; for example, translation, rotation, and / or deformation can be corrected. This method can, for example, be a machine learning method, in particular a neural network, such as a convolutional neural network.

[0031] In alternative embodiments, the method according to the invention, in particular the extraction of the anatomical and non-anatomical structures from the 2D update images, can be performed using at least one machine learning method, in particular a neural network, for example a convolutional neural network. An advantage of these embodiments can be that machine learning methods can perform transformations, similar to the segmentation extraction disclosed in the preceding section, with high accuracy.

[0032] In alternative embodiments of the method according to the invention, it is provided that the calculation of the non-anatomical 3D image from the 2D update images is carried out by a single machine learning method, for example by a neural network that has been trained to first convert 2D update images into other 2D images, for example into extracted anatomical structures or non-anatomical structures, wherein these other 2D images are subsequently reconstructed in the same machine learning method to an artifact-free non-anatomical 3D image.

[0033] In alternative embodiments of the inventive method, the extraction of non-anatomical structures can be carried out in such a way that only specific non-anatomical structures are extracted, for example, those belonging to a particular object class or to a specific combination of several object classes. Subsequently, only those non-anatomical structures that depict these specific non-anatomical structures—that is, those belonging to the specific object class or the specific combination of several object classes—are reconstructed into non-anatomical 3D images. If a machine learning method is used for the extraction, it is trained not only on the extraction but also on the differentiation of such non-anatomical structures.An advantage of such alternative designs is that, for example, when navigating a guidewire in a patient, only the guidewire itself can be extracted and reconstructed. Other non-anatomical structures, such as those already present and belonging to other object classes (e.g., stents or orthopedic implants from previous procedures), or non-procedure-specific non-anatomical structures (e.g., the patient table), which are not of interest, are neither extracted nor reconstructed. This reduces the display shown to the treating physician to the essentials.

[0034] In alternative embodiments of the method according to the invention, the extraction of non-anatomical structures can be carried out such that several separate extractions are calculated from each 2D update image, wherein each of the separate extractions shows only a subset of the non-anatomical structures, for example, only those belonging to a specific object class or to a specific combination of several object classes. Subsequently, the extractions are processed separately into non-anatomical 3D images, wherein the separate non-anatomical 3D images calculated in this way contain only the corresponding subset of the non-anatomical structures. The separate non-anatomical 3D images are then combined into a single non-anatomical 3D image, for example, by adding voxel-based separate non-anatomical 3D images.An advantage of such alternative designs can be that an anatomical 3D image generated by separate 3D reconstructions of subsets of non-anatomical structures and subsequent combination can be more accurate than an anatomical 3D image reconstructed all at once, because the reconstruction problem is generally simpler the smaller the proportion of non-anatomical structures in the volume to be imaged.

[0035] Another aspect of the invention relates to a medical imaging device, in particular a gantry-based system, for performing an imaging examination and display of non-anatomical structures, comprising the following components: a provisioning unit for providing a first 3D image, wherein the first 3D image contains at least one anatomical structure; at least two image chains, wherein the image chains are configured for receiving 2D images, in particular 2D update images and / or 3D images;a computing unit, wherein the computing unit is configured for extracting an anatomical model from the at least one anatomical structure, wherein the computing unit is configured for extracting anatomical and non-anatomical structures from the 2D update images, wherein the computing unit is configured for calculating partial reconstructions from the extraction of non-anatomical structures, wherein the computing unit is configured for calculating a non-anatomical 3D image from at least two partial reconstructions, wherein the computing unit is configured for calculating an anatomical 3D image from the extraction of anatomical structures, and wherein the computing unit is configured for creating a navigation volume from the anatomical model and the non-anatomical 3D image by determining a coordinate transformation, wherein the coordinate transformation is configured for the positionally correct positioning of the anatomical model in the navigation volume;a display unit on which the navigation volume is displayed.

[0036] The device according to the invention includes a provisioning unit, preferably a storage unit, for example a memory stick, a hard drive, or another portable or permanently installed data carrier. The provisioning unit can also be understood as a connection to a storage unit which is connected to a network to which the medical imaging device has access, in order to provide a first 3D image, wherein the first 3D image contains at least one anatomical structure.

[0037] The device according to the invention comprises at least two image chains, and may also, for example, comprise three, four, or more image chains. An image chain comprises an X-ray generator for generating X-rays, for example, a rotating anode-based generator, and a receiving unit for receiving X-rays, for example, a flat-panel detector, wherein the image chains are configured for receiving 2D images, in particular 2D update images, and / or 3D images.

[0038] An advantage of using at least two image chains is that multiple 2D update images can be acquired simultaneously, pseudo-simultaneously, or directly sequentially. Preferably, the medical imaging device includes two image chains, since a higher number of image chains increases the complexity, cost, and failure probability of the medical imaging device. The acquisition of 2D update images preferably occurs in tuples, with the gantry-based system preferably not rotating further, or at least not nearly rotating further, between the acquisition of 2D update images for a tuple. However, the gantry-based system can preferably rotate further by a minimum angle of, for example, at least 5° or 20° between the acquisition of tuples.When rotating the gantry-based system, a low rotational speed is preferred, as motion blur can occur at excessively high rotational speeds.

[0039] Furthermore, the device according to the invention includes a computing unit, wherein this computing unit can be a graphics processing unit (GPU) or another processing unit, for example a central processing unit (CPU), wherein in the case of a GPU it can be used to perform massively parallel or highly parallel computational operations. A computing unit can also consist of several computing units, which together can again be considered a computing unit according to the invention. Furthermore, this computing unit extracts an anatomical model from the anatomical structure of the first 3D image. The computing unit is also configured to extract anatomical and non-anatomical structures from the at least two 2D update images.Furthermore, the processing unit reconstructs at least two partial reconstructions from the extractions of non-anatomical structures from the first subset of 2D update images and also reconstructs an anatomical 3D image from the extraction of anatomical structures from the second subset of 2D update images. The processing unit is also configured to calculate a non-anatomical 3D image from at least two partial reconstructions. The processing unit further serves to create a navigation volume from the anatomical model, the non-anatomical 3D image, and optionally the anatomical 3D image, by determining a coordinate transformation. This coordinate transformation is configured to position the anatomical model, the non-anatomical 3D image, and optionally the anatomical 3D image correctly within the navigation volume.

[0040] Furthermore, the device according to the invention includes a display unit on which the navigation volume is displayed, as well as, optionally, the non-anatomical 3D image, the anatomical 3D image, the first 3D image, and / or the anatomical model. The display unit can also be configured to display the 2D update images. In further advantageous embodiments of the method according to the invention, the display unit can also display conventional X-ray images, for example, fluoroscopies, as well as post-processing of such X-ray images performed by the processing unit, for example, by means of a frequency filter, particularly when the gantry is stationary. In further embodiments according to the invention, it is provided that images calculated by the processing unit using a subtraction method, for example, a DSA, can be displayed.

[0041] The invention will be explained in more detail below with reference to the illustrations. Fig. 1 schematically shows a sequence of steps of the method according to the invention. Fig. 2 Figure 1 shows an exemplary flowchart of the inventive method in which new 2D update images are added. Fig. 3 Figure 1 shows an exemplary flowchart of the inventive method in which new 2D update images are added and old 2D update images are replaced by them. Fig. 4 schematically shows an imaging device that can perform the method according to the invention.

[0042] Fig. 1Figure 1 shows an exemplary sequence of the inventive method. First, a first 3D image 1 (3D volume) is provided to the inventive method. This first 3D image 1 can be acquired during or before an interventional procedure using a 3D scan. An anatomical model 5 is then extracted from this first 3D image 1.

[0043] Subsequently, according to the inventive method, at least two 2D update images 2 are provided, wherein the 2D update images 2 contain at least partially anatomical structures 3 and / or at least partially non-anatomical structures 4 of an examination area. The at least partially anatomical structures 3 are parts of the bone structures and / or parts of the vascular structures and / or parts of the soft tissues. Fig. 1A guide wire is present in the examination area as a non-anatomical structure 4. Furthermore, at least two 2D update images 2 were acquired within a time interval.

[0044] Following the provision of at least two 2D update images 2, an extraction 6 of non-anatomical structures 4 is performed from a first subset 7 of the at least two 2D update images 2.

[0045] Based on these extractions 6 of the non-anatomical structures 4 from the first subset 7 of the 2D update images 2, a calculation of a non-anatomical 3D image 10 from several partial reconstructions 9 is performed. Fig. 1This is done using four partial reconstructions 9. The different partial reconstructions 9 are preferably derived from 2D update images which were taken at different times and can therefore, due to possible movements of the patient and / or the non-anatomical structures, represent different states, positions and orientations of the non-anatomical structures 4.

[0046] In addition to the extraction of 6 non-anatomical structures 4, an extraction 13 of anatomical structures 3 is also performed from a second subset 8 of the 2D update images 2, whereby the second subset 8 differs from the first subset 7 in at least one 2D update image 2, so that the two subsets (7, 8) are not identical. From the extraction 13 anatomical structures 3, an anatomical 3D image 11 is calculated.

[0047] According to the invention, after the reconstruction of the anatomical 3D image 11, the anatomical 3D image 11 is registered with the first 3D image 1, and a coordinate transformation 14 is determined. Subsequently, the coordinate transformation 14 thus obtained is applied 16 to the anatomical model 5. The coordinate transformation determined from the anatomical 3D image 11 and the first 3D image 1 is preferably deformable. When determining the coordinate transformation 14, the contrast-containing areas of the first 3D image 1 are preferably not taken into account, if present, as they can distort the determination of the coordinate transformation.

[0048] Finally, a navigation volume 12 is created 15 from the at least one anatomy model 5 and the non-anatomical 3D image 10. Thus, in the navigation volume 12, both the non-anatomical structures 4 from the non-anatomical 3D image 10 and the extracted anatomy model 5 are displayed in the correct position relative to each other.

[0049] Fig. 2Figure 1 shows a step of the inventive method in which, after acquiring two new 2D update images 21, these are added to the first subset 7 and the second subset 8 of the provided 2D update images 2, and the anatomical 3D image 11 is at least partially reconstructed, whereby all older 2D update images 2 can be reused for the reconstruction of the anatomical 3D image 11. Furthermore, the non-anatomical 3D image 10 is reconstructed using the new 2D update images 21. The step of the inventive method described here is particularly advantageous, for example at the beginning of the imaging process, when there are not yet enough 2D update images 2 in the second subset 8 to reconstruct an artifact-free anatomical 3D image 11.Increasing the number of 2D update images 2 in the second subset 8 preferably results in higher image quality.

[0050] Fig. 3 Figure 1 shows an embodiment of the method according to the invention, in which two newly acquired 2D update images 21 are added to the second subset 8, and the two oldest 2D update images 31 are removed from the second subset. An anatomical 3D image 11 is then reconstructed from the modified second subset. This can be described as an overlapping reconstruction. An advantage of this embodiment is that the modified anatomical 3D image now contains more up-to-date information about the anatomical structures 3, which may have changed, for example, due to breathing, cardiac activity, or movement of the patient table.

[0051] Fig. 4Figure 1 schematically shows an imaging device configured to perform the method according to the invention. The imaging device comprises a rotating gantry 43, on which two X-ray generators 42 are mounted, and X-ray-absorbing flat-panel detectors 41 are mounted directly opposite the X-ray generators. The unit consisting of an X-ray generator and the flat-panel detector 41 arranged opposite it can be referred to as an imaging chain. These imaging chains can be used to provide a first 3D image and 2D update images for the method according to the invention. The various imaging chains can acquire the 2D update images simultaneously or pseudo-simultaneously, or with a time offset.The area 44 to be examined using X-rays 45 can contain anatomical and non-anatomical structures. The computing units 47 required for processing the 2D update images and the first 3D image are located in . Fig. 4 The navigation volume according to the invention is integrated in a separate processing unit 46. Alternatively, this processing unit 46 or the computing units 47 can be integrated directly into the imaging device. The navigation volume according to the invention can be displayed on an external display unit 48. Alternatively, the display unit can also be attached directly to the imaging device. Reference symbol list

[0052] 1. First 3D image 2. 2. 3D update image 3. Anatomical structures 4. Non-anatomical structure 5. Anatomy model 6. Extraction of non-anatomical structures 7. First subset 8. Second subset 9. Partial reconstruction 10. Non-anatomical 3D image 11. Anatomical 3D image 12. Navigation volume 13. Extraction of anatomical structures 14. Determination of the coordinate transformation 15. Creation of the navigation volume 16. Application of the coordinate transformation 21 newly added 2D update images, 31 removed 2D update images 41 X-ray detector 42 X-ray generator 43 Rotating gantry 44 Examination area 45 X-rays 46 Processing unit 47 Computing unit 48 Display unit

Claims

1. A method for operating a medical imaging device for the positionally accurate display of non-anatomical structures during an imaging examination, comprising the following steps: a.) provisioning of a first 3D image (1) containing at least one anatomical structure (3); b.) extracting at least one anatomical model (5) from the at least one anatomical structure (3) of the first 3D image (1); c.) providing at least two 2D update images (2) using the medical imaging device, wherein at least two 2D update images (2) were captured at different times; d.) extracting (6) non-anatomical structures (4) from a first subset (7) of the 2D update images (2); e.) extracting (13) anatomical structures (3) from a second subset (8) of the 2D update images (2); f.) calculating a non-anatomical 3D image (10) from at least two partial reconstructions (9) from the first subset (7), wherein the at least two partial reconstructions (9) are calculated from the extraction (6) of the non-anatomical structures (4) and artifacts of the partial reconstructions are removed by means of machine learning methods; g.) reconstructing an anatomical 3D image (11) from the extraction (13) of the anatomical structures (3) from the second subset (8); h.) registering the anatomical 3D image (11) with the first 3D image (1) to determine a coordinate transformation (14); i.) creating of a navigation volume (12) from the at least one anatomical model (5) and the non-anatomical 3D image (10) using the determined coordinate transformation (14).

2. The method according to claim 1, characterized in that the navigation volume (12) is additionally created from the anatomical 3D image (11) using the determined coordinate transformation (14).

3. The method according to claim 1, characterized in that the first 3D image (1) is captured with an X-ray device, in particular an X-ray C-arm or a computer tomograph, or with a magnetic resonance tomograph.

4. The method according to one of the preceding claims, characterized in that the 2D update images (2) are recorded using an X-ray device, in particular an X-ray C-arm, or a computer tomograph.

5. The method according to one of the preceding claims, characterized in that, in the absence of non-anatomical structures (4) in a 2D update image (2) of the first subset (7), the non-anatomical 3D image (10) continues to be reconstructed.

6. The method according to one of the preceding claims, characterized in that newly captured 2D update images (21) are added to the first subset (7) and / or second subset (8) and the non-anatomical 3D images (10) and / or the anatomical 3D images (11) are reconstructed in particular at a temporal rate, wherein the temporal rate of reconstruction of the non-anatomical 3D images (10) and / or the anatomical 3D images (11) corresponds to a function of the acquisition rate of the 2D update images (21).

7. The method according to one of the preceding claims, characterized in that when a new 2D update image (21) is captured, it is added to the second subset (8) of the 2D update images provided (2) and the anatomical 3D image (11) is at least partially reconstructed anew, whereby temporally older 2D update images are reused or removed for the new reconstruction.

8. The method according to one of the preceding claims, characterized in that the method according to the invention, in particular the calculation of the non-anatomical 3D image (10) from the partial reconstructions (9), is performed using a machine learning method, in particular a neural network.

9. The method according to one of the preceding claims, characterized in that movements of the patient are taken into account.

10. The method according to one of the preceding claims, characterized in that the method, in particular the extraction (6, 13) of the anatomical (3) and / or non-anatomical (4) structures from the 2D update images (2), is performed using at least one machine learning method, in particular a neural network.

11. The method according to one of the preceding claims, characterized in that the calculation of the non-anatomical 3D image (10) from the 2D update images (2) is performed by a single machine learning method, in particular a neural network.

12. The method according to one of the preceding claims, characterized in that only those non-anatomical structures (4) are extracted which belong to a specific object class or a combination of specific object classes.

13. The method according to one of the preceding claims, characterized in that non-anatomical structures (4) belonging to a specific object class or a combination of specific object classes are reconstructed separately.

14. The method according to one of the preceding claims, characterized in that the non-anatomical structures (4) are a guide wire and / or catheter.

15. A medical imaging device for performing an imaging examination and displaying non-anatomical structures, comprising: ∘ a provision unit for providing a first 3D image (1), wherein the first 3D image contains at least one anatomical structure; ∘ at least two image chains (41, 42), wherein the image chains are designed to capture 2D update images (2); ∘ a computing unit (47), ∘ wherein the computing unit (47) is adapted to: extracting an anatomical model (5) from the at least one anatomical structure (3), extracting (6) non-anatomical structures (4) from a first subset (7) of the 2D update images (2); extracting (13) anatomical structures (3) from a second subset (8) of the 2D update images (2); calculating a non-anatomical 3D image (10) from at least two partial reconstructions (9) from the first subset (7), wherein the at least two partial reconstructions (9) are calculated from the extraction (6) of the non-anatomical structures (4) and artifacts of the partial reconstructions are removed by means of machine learning methods; reconstructing an anatomical 3D image (11) from the extraction (13) of the anatomical structures (3) from the second subset (8); registering the anatomical 3D image (11) with the first 3D image (1) to determine a coordinate transformation (14); and creation of a navigation volume (12) from the at least one anatomical model (5) and the non-anatomical 3D image (10) using the determined coordinate transformation (14); ∘ a display unit (48) on which the navigation volume (12) is displayed.