Method and system for replicating an insertion point for a medical device

Abstract

The present invention relates to a method and a medical system (200) for reconstructing insertion points (202a, 202b) for a medical instrument, the method comprising the steps of: providing at least one marker (210) on the surface of an object (216), the marker (210) having the property of being detectable both by tomography, in particular by fluoroscopy, and by optical means; generating tomography image data from which a fluoroscopy recording of the at least one marker (210) arranged on the surface of the object (216) together with the object (216) can be reconstructed; and, in the coordinate system of the tomography image data (218), the position of the medical instrument on the surface of the object (216) relative to the at least one marker (210). determining insertion points (202a, 202b) for the medical instrument; generating visual image data from which a visual image of the object (216) together with at least one marker (210) disposed on the surface of the object (216) can be reconstructed; converting coordinates of the insertion points in the coordinate system of the tomography image data (218) into the coordinate system of the visual image data (222) using relative positions of the insertion points (202a, 202b) with respect to the at least one marker (210); and reproducing the insertion points (202a, 202b) for the medical instrument in a view of the object (216) in real time.

Description

[Technical Field]

[0001] The present invention relates to a method for recreating an insertion point for a medical instrument, and further to a medical system for recreating an insertion point for a medical instrument.

[0002] Punctures are routinely performed in diagnostics and in medical treatments. Puncture refers to the deliberate insertion of a medical device, particularly a needle, such as a hollow needle or probe, into the human body. The medical device is inserted into the body and guided to a target location inside the body, where it can, for example, supply energy, remove fluid or tissue, or inject medicine.

[0003] Punctures are routinely performed under visual control, particularly when the target site includes sensitive body tissue, such as nerve or organ tissue, or when sensitive body tissue is near the target site. Visually controlled punctures typically involve detecting the location and orientation of the medical device within the human body using imaging methods such as computed tomography (CT), magnetic resonance imaging (MRT), or ultrasonography.

[0004] In particular, the puncture under visual control can be complemented by using a positioning device that can reproduce, and in particular mark, the insertion point and the insertion angle for the medical instrument.

[0005] For example, European Patent No. 1887960 describes a positioning device for positioning equipment inside an examination room, which uses directional electromagnetic radiation to mark the access area and relative orientation of the equipment in order to reach a target area located on the trajectory of the directional electromagnetic radiation.

[0006] The problem underlying the present invention is to provide an improved method for recreating a penetration point for a medical instrument.The problem underlying the present invention is also to provide an improved system for recreating a penetration point for a medical instrument.

[0007] Regarding the method, the above problem is a method for reproducing an insertion point for a medical device, the method comprising: providing at least one marker on a surface of the object, the marker having the property of being detectable both by fluoroscopy and optically; - generating fluoroscopy and / or tomography image data from which a fluoroscopy and / or tomography recording of the object together with at least one marker arranged on the surface of the object can be reconstructed; determining an insertion point for the medical device on the surface of the object relative to the at least one marker in a coordinate system of the fluoroscopy image data and / or the tomography image data; - generating visual image data from which a visual image of the object together with at least one marker arranged on the surface of the object can be reconstructed; - transforming coordinates of the insertion point in the coordinate system of the fluoroscopy image data and / or tomography image data into the coordinate system of the visual image data using the relative position of the insertion point with respect to the at least one marker; - reproducing in real time within a view of the object an insertion point for the medical device; The method is solved by the method having the following structure.

[0008] Within the framework of this specification, fluoroscopic and / or tomographic recordings and fluoroscopic and / or tomographic image data should be understood to mean recordings made using imaging modalities such as X-ray devices (e.g., C-arms), X-ray tomography devices (computed tomography devices), magnetic resonance imaging devices, or ultrasound devices, and image data obtained thereby. Images recorded by computed tomography are both fluoroscopic and tomographic recordings, while in the case of magnetic resonance imaging, the term fluoroscopy is typically not used.

[0009] For the purposes of the invention described herein, the term "tomographic image data" is also used in reference to fluoroscopic image data that is not strictly tomographic image data, but originates from an imaging modality such as, for example, a C-arm. Hence, in the following, tomographic recordings also refer to all recordings originating from an imaging modality, i.e., for example, C-arm fluoroscopic recordings.

[0010] "Reproduce" refers to displaying at least the penetration point and, if known, the penetration angle and / or penetration depth in the view of the object to be punctured. The location of the penetration point is marked in the view of the object by reproducing the penetration point on the surface of the object.

[0011] The view of the object may be a direct real-world view of the object, and the reproduction of the insertion point may be, for example, a marker projected onto a real surface. However, alternatively, the view of the object to be punctured may be a real-time image reproduction of the object on a monitor or virtual reality (VR) glasses, with the insertion point reproduced in real time. The view of the object may also be a real-time image reproduction of the object on a transparent optical display, with the insertion point reproduced in real time with correct perspective as augmented reality (AR).

[0012] The medical device is in particular a cannulated medical device, for example a hollow needle. Alternatively, the medical device may be a needle-like probe, for example as used for interstitial hyperthermia.

[0013] The penetration point is located on the actual surface of the object to be penetrated and refers, among other things, to the point at which the penetrating medical instrument penetrates the object. Furthermore, the penetration angle and / or penetration depth can also be reproduced in real time within the view of the object. The penetration angle refers to the angle relative to the surface at which the penetrating medical instrument penetrates the object. The penetration depth refers to the distance that the medical instrument inserted at the penetration angle at the penetration point must travel to reach the target location inside the human body.

[0014] To generate the tomography image data, an X-ray source and an X-ray detector of an X-ray device can be used. To be able to reconstruct a tomography record of at least one marker arranged on the surface of the object together with the object from the tomography image data, the object is arranged between the X-ray source and the X-ray detector so that the X-rays emitted from the X-ray source pass through the object and are attenuated with different intensities depending on the internal structure of the object, and are then detected by the X-ray detector. The tomography record reconstructed from the tomography image data can be a two-dimensional or three-dimensional tomography record.

[0015] The penetration point can be defined manually or mechanically, for example, software-based, in the tomographic recording. The penetration point is preferably defined so that the path that the medical device must travel inside the object to reach the target location is as short as possible. The penetration point is preferably defined so that sensitive tissue is not damaged during penetration.

[0016] The coordinates of the defined insertion points for the medical device on the surface of the object are determined in the coordinate system of the tomography image data, preferably by calculation by a calculation unit. The tomography recording reconstructed from the tomography image data shows the object and, in particular, the defined insertion points together with markers, so that the coordinates of the defined insertion points can be determined relative to at least one marker. This means that the spatial relationship between the insertion points and the markers, i.e., their relative positions, is known in the coordinate system of the tomography image data. Typically, the tomography recording does not show the complete object, but rather shows, in particular, a partial region of the object where the target location for the medical device is located. In particular, the markers are provided, i.e., arranged, so as to be visible together with the target location in the reconstructed tomography recording.

[0017] The visual image data can be generated using a camera, from which a static visual image of the surface together with the placed markers can be reconstructed, or, as preferred in the method according to the invention, a moving visual image of the surface together with the placed markers can be reconstructed.

[0018] The spatial relationship between the insertion point and the markers can be determined in the coordinate system of the tomographic image data, and then, with this spatial relationship known, the position and orientation of the markers can be determined in the coordinate system of the visual image data, so that, inter alia, the relative position of the insertion point with respect to the at least one marker can be used to transform the coordinates of the insertion point in the coordinate system of the tomographic image data to the coordinate system of the visual image data, and then the coordinates of the insertion point and its spatial relationship to the markers are also known in the coordinate system of the visual image data.

[0019] The coordinates of the insertion point in the coordinate system of the tomographic image data can be transformed into the coordinate system of the visual image data, among other reasons because the position of the marker is known in both the coordinate system of the tomographic image data and the coordinate system of the visual image data and can be used as a reference for transforming coordinates from one coordinate system to the other. Among other things, the position of the insertion point in the coordinate system of the visual image data can be used to reconstruct the insertion point in real time within the view of the object.

[0020] The real-time reproduction of the insertion point within the view of the object allows the user to reliably and accurately puncture the object. Real-time reproduction means, among other things, that possible delays in the reproduction are not resolvable by the human eye, i.e., remain unnoticed by the user. Preferably, the real-time reproduction of the insertion point is adapted to the changing view of the object, i.e., in the correct perspective. The view of the object may be a real view or a reconstructed view. The real view may be a direct view of the real surface without an intervening object, or an indirect view of the real surface via a transparent medium, e.g., a transparent optical display. The reconstructed view may be a static view reconstructed from visual image data. The reconstructed view of the object may also include a real-time image reproduction reconstructed from the visual image data, i.e., a moving visual image of the surface. The visual image data from which the moving visual image can be reconstructed can be generated using video technology, e.g., a video camera. The video camera can be configured to generate three-dimensional visual image data from which the three-dimensional moving visual image can be reconstructed.

[0021] The method according to the present invention allows the insertion point on the surface to be accurately reproduced within the view of the object, thereby enabling the user to puncture the object in a targeted and controlled manner. This method has the advantage that it is not necessary to record an X-ray image of the object during the puncture, or at least only a few X-ray images need to be recorded. In some situations, it may be sufficient to record an X-ray image only before the puncture for planning purposes to define the insertion point. In particular, if the insertion angle and penetration depth are also reproduced within the view of the object, it is generally not necessary to record an X-ray image after the puncture to control whether the medical device has actually reached the target location. Overall, the method according to the present invention allows for a significant reduction in radiation exposure to the object, and in particular to the patient, depending on the application.

[0022] The method according to the present invention has the further advantage that, in addition to the X-ray equipment already present, no additional bulky equipment is required that would take up additional space in the operating room. All that is needed to perform the method according to the present invention is a marker, a camera, and a computing unit with appropriate software. The surgeon, who is assisted by the method according to the present invention to reliably and accurately puncture an object, is not hindered or restricted in his or her movements by additional bulky equipment. To be able to perform the method according to the present invention, there is no need to renovate or modify the operating room, and there is no need to screw equipment to the wall or ceiling of the operating room.

[0023] The method according to the invention can also be carried out without the use of a laser, which is used to mark the insertion point with a laser beam. Reproducing the insertion point in the view of the object without using a laser has the advantage that the physician does not have to pay attention to the possibility that he may block the laser beam and thus the marking of the insertion point with the laser beam is no longer visible.

[0024] In the following, a preferred embodiment variant of the method according to the invention for reproducing an insertion point for a medical instrument is described.

[0025] Preferably, the visual image data is generated as three-dimensional visual image data. The three-dimensional visual image data can be generated, for example, using a light field camera, a stereo camera, a triangulation system, or a time-of-flight (TOF) camera. A three-dimensional visual image can be generated from the three-dimensional visual image data, and in this three-dimensional visual image, the insertion point and, in particular, the insertion angle for the medical device can be reproduced in the correct perspective.

[0026] In a preferred embodiment of the method according to the invention, the method comprises: determining a penetration angle and / or penetration depth for the medical device relative to the at least one marker in a coordinate system of the fluoroscopy image data and / or tomography image data; - transforming the puncture angle and / or the puncture depth determined in the coordinate system of the fluoroscopy image data and / or the tomography image data into the coordinate system of the visual image data using the relative orientation of the puncture angle with respect to the at least one marker and / or using the relative distance of the puncture depth with respect to the at least one marker; - reproducing in real time within a view of the object a puncture angle and / or a puncture depth for the medical device; It has.

[0027] Preferably, the penetration point, penetration angle and penetration depth are reproduced together in real time in the view of the object in true perspective, so that the physician can see at a glance where on the surface, at what angle and to what depth the object should be penetrated.

[0028] In the method according to the invention, visual image data are preferably generated continuously and at least the insertion points determined in the coordinate system of the fluoroscopy image data and / or tomography image data are transformed into the coordinate system of the respective last generated visual image data. Preferably, at least a reconstruction of the insertion points for the medical instrument is shown in real time in the view of the object.

[0029] From the continuously generated visual image data, a moving visual image can be reconstructed as a real-time image reproduction. By continuously generating visual image data, the relative motion of the object can be detected, and the penetration point, penetration angle, and / or penetration depth can be reproduced in real time with correct perspective within the view of the object.

[0030] In particular, the coordinates of the insertion point in the coordinate system of the tomography image data and the insertion angle and / or penetration depth can be transformed into the coordinate system of the finally generated visual image data and then reproduced in real time.

[0031] From the generated visual image data, a visual image of the surface can be reconstructed, and the insertion point for the medical instrument can be reproduced in this visual image. The visual image can be a two-dimensional or three-dimensional stationary visual image, or, as preferred, a two-dimensional or three-dimensional visual image of a moving visual image. The visual image can be displayed on a monitor. The insertion angle and / or the insertion depth for the medical instrument can also be reproduced in the visual image.

[0032] Insertion points for medical instruments can also be reproduced in an indirect view of the object on the transparent optical display, through which a view of the real surface is visible, and the reproduction of the insertion points on the transparent display is in correct perspective relative to the view of the real surface.

[0033] The optical display can be mounted in a user-wearable frame, similar to eyeglass lenses, so that the optical display is positioned in front of the user's eyes. In this case, the user can observe a real object through the transparent optical display, i.e., indirectly view a real image of the surface. Insertion points for the medical instrument can be reproduced on the transparent optical display in real time with the correct perspective relative to the view of the real surface, so that the user can see the insertion points on the object's surface in the indirect view of the object. Insertion angles and / or puncture depths for the medical instrument can also be reproduced on the transparent optical display in addition to the insertion angles. A camera that continuously generates visual image data is preferably mounted on the frame with the optical display. In this case, insertion points, insertion angles, and / or puncture depths can be reproduced in real time in the indirect view of the object, so that the insertion points, insertion angles, and / or puncture depths are shown with the correct perspective relative to the view of the real surface.

[0034] Additionally or alternatively, the insertion point for the medical device can be reproduced as an optical marking on the real surface of the object. For example, the insertion point can be reproduced directly on the real surface of the object using a laser beam as an optical marking or using a crosshair generated by a video projector, and in particular can be projected onto the surface. In this case, the laser and / or video projector are preferably automatically calibrated by the camera used to generate the visual image data. Furthermore, the insertion angle and / or the penetration depth can also be reproduced as an optical marking.

[0035] In some implementation variants, the penetration point and the penetration angle and / or penetration depth can be reproduced in real time within the view of the object in the form of a digital representation of a virtual tool, in particular the virtual tool can be reproduced in real time with the correct perspective within a real time image reproduction of the object on a transparent optical display or monitor.

[0036] In particular, in a variant implementation of the method according to the invention, in which the insertion point and the insertion angle and / or the insertion depth are reproduced in the form of a digital representation of a virtual tool, the method comprises: optically detecting the position and orientation of the medical device relative to at least one marker in a coordinate system of the generated visual image data; determining whether the position and orientation of the detected medical device coincides with the position and orientation of the reproduced virtual tool, and if so, notifying that the position and orientation of the detected medical device match the position and orientation of the reproduced virtual tool; can have:

[0037] The method of indicating that the detected position and orientation of the medical instrument matches the position and orientation of the reproduced virtual tool has the advantage that the user receives feedback as to whether the medical instrument is aligned with respect to the surface of the object so that it can be inserted into the object along the specified path. The indication that the detected position and orientation of the medical instrument matches the position and orientation of the reproduced virtual tool may be, for example, optically within the view of the object or acoustically.

[0038] The method may include calculating a trajectory between the position and orientation of the detected medical instrument and the position and orientation of the reproduced virtual tool when the position and orientation of the detected medical instrument do not match the position and orientation of the reproduced virtual tool.

[0039] For example, the calculated trajectory can be used to reproduce in real time within the view of the object a virtual directional indication, preferably indicating the direction in which the medical instrument must be moved in order to match the position and orientation of the medical instrument with the position and orientation of the virtual tool reproduced within the view of the object.

[0040] A virtual directional display can assist the user in matching the position and orientation of the medical instrument with the position and orientation of the reproduced virtual tool.

[0041] In a variant implementation of the method according to the invention, in which the insertion point and the insertion angle and / or the insertion depth can be reproduced in the form of a digital representation of a virtual tool, the method preferably comprises the steps of: - aligning the digital representation of the reconstructed virtual tool in real time with respect to the at least one marker with respect to a recording axis along which the visual image data was generated. It has.

[0042] By aligning the digital representation of the reconstructed virtual tool in real time with at least one marker relative to the recording axis along which the visual image data is generated, it is possible to reconstruct the virtual tool in real time in the correct perspective within the view of the object.

[0043] By aligning the reproduced digital representation of the virtual tool with the recording axis in real time, relative movement of the object or relative changes in perspective on the surface within the view of the object can be taken into account, so that the penetration point, penetration angle, and / or puncture depth are always reproduced in the view with the correct perspective. The user and the object can then be moved relative to each other, and the user can be confident that the penetration point, penetration angle, and / or puncture depth are correctly reproduced in the view at all times.

[0044] The problem set forth in the introduction is solved by a medical system for reconstructing an insertion point for a medical device, the medical system comprising a marker, an imaging modality, in particular an X-ray or computed tomography device, a camera, a calculation unit, and a reconstruction unit.

[0045] The marker is configured to be detectable both by tomography and / or fluoroscopy, and optically. The imaging modality, particularly the X-ray device, is configured to generate fluoroscopy image data and / or tomography image data and essentially includes an X-ray source and an X-ray detector. The X-ray device may be, for example, a computed tomography (CT) device or a C-arm device. However, the imaging modality may also be, for example, an ultrasound inspection device or a magnetic resonance imaging device. The camera is configured to generate visual image data and may be, for example, a light field camera, a stereo camera, a triangulation system, or a time-of-flight camera. The camera is preferably configured to continuously generate image data, particularly three-dimensional image data, using the camera. Instead of the camera, a scanner may be provided, which allows visual image data to be generated by light sectioning.

[0046] The computation unit is determining an insertion point for the medical device on the surface of the object relative to at least one marker in a coordinate system of the fluoroscopy image data and / or tomography image data; Transforming the coordinates of the insertion point in the coordinate system of the fluoroscopic image data and / or tomographic image data into the coordinate system of the visual image data using the relative position of the insertion point with respect to the at least one marker. It is structured as follows.

[0047] The reconstruction unit is configured to represent in real time in a real or reconstructed view of the object an insertion point for the medical instrument.

[0048] The medical system according to the invention is designed in particular in such a way that it can be used to carry out the method according to the invention for reproducing an insertion point for a medical instrument.

[0049] The camera and the X-ray device are each operatively connected to a computing unit, which can access and process the visual image data generated by the camera and the tomographic image data generated by the X-ray device, and the computing unit is further operatively connected to a reconstruction unit, among other things, for visualizing in real time within the view of the object the insertion points for the medical device.

[0050] In the following, a preferred embodiment of a medical system according to the invention for reproducing an insertion point for a medical instrument is described.

[0051] The computing unit can be configured as an electronic data processing system or as a component part of an electronic data processing system and has, among other things, a CPU (Central Processing Unit), a main memory, and a computer-readable storage medium with a permanently stored computer program.

[0052] The computing unit and / or the X-ray device can be configured to reconstruct a tomographic recording from the tomographic image data. The computing unit and / or a separate data processing system can be configured to reconstruct a visual image from the visual image data generated by the camera.

[0053] The reproduction unit may be an optical display operatively connected to the computing unit, on which the insertion point for the medical instrument can be visualized by the computing unit. The optical display may be part of an augmented reality (AR) system, among others. For example, the optical display may be a component of a pair of glasses to which a camera is also attached. The optical display may be transparent, so that the real surface can be indirectly observed through the optical display while the insertion point can be reproduced in the indirect view.

[0054] The reproduction unit may be a monitor operatively connected to the computing unit, e.g. a computer monitor, or a monitor of a virtual reality (VR) system, e.g. VR glasses, on which a reconstructed view of the object may be reproduced, e.g. as a real-time image reproduction of the object together with at least the insertion point in the correct perspective.

[0055] The reproduction unit may be a video projector, which is automatically calibrated by the camera and configured to reproduce an insertion point for the medical instrument on the real surface of the object as an optical marking. The video projector is preferably automatically calibrated by the camera and configured to project a crosshair onto the real surface of the object, the center of the crosshair representing the position of the insertion point. The insertion point can be reproduced simultaneously in an indirect view of the object and projected on the real surface as an optical marking. Due to the redundancy, the insertion point can be reproduced relatively reliably depending on the situation.

[0056] The at least one marker may be flexible and pliable, and may be formed, for example, by an adhesive tape that can be applied so as to adhere onto the real surface of the object. To carry out the method according to the invention, the adhesive tape can be used to adhere a regular or irregular pattern onto the surface, thus forming the marker.

[0057] Markers can also be formed from double-sided adhesive film or double-sided adhesive paper with a pre-punched pattern, which can be adhered to a real surface and then the support film removed, leaving only the pre-punched pattern on the surface to form the marker.

[0058] When the marker is provided on a carrier paper or a carrier film, the marker itself can be formed from multiple components that are not directly connected, and the relative positions of these components are still predetermined by the carrier film or the carrier paper. In use, after the marker is applied to the body surface, when the carrier film or the carrier paper is peeled off from the marker, these components of the marker will accordingly maintain their relative positions.

[0059] However, the marker may also be rigid, for example provided in the form of a solid block that can be adhered to a body surface in use.

[0060] Preferably, the adhesive tape or adhesive film comprises a metal such as titanium or stainless steel, or alternatively BaSO 4 , so as to be detectable by tomography, in particular by fluoroscopy. x , among others, with substances such as barium sulfate (BaSO4).

[0061] At least one marker may also have at least one tomographically, particularly fluoroscopically, detectable element and / or at least one optically detectable element. The tomographically detectable element may be made of metal and configured to be identifiable as a tomographically or fluoroscopically detectable element in tomographic or fluoroscopic recordings. For example, metal spheres may be distributed across the surface of the marker, and these metal spheres may be identified in fluoroscopic and / or tomographic recordings. The optically detectable element may be a light-emitting diode configured to emit electromagnetic radiation within a predetermined wavelength range. In particular, multiple light-emitting diodes may be distributed across the surface of the marker. The predetermined wavelength range preferably includes infrared radiation. In that case, the camera preferably has an infrared sensor for detecting infrared radiation emitted from the light-emitting diode. The tomographically detectable element and the optically detectable element preferably have a known spatial relationship to each other. It is also possible to use elements that are detectable both tomographically and optically, for example, metal spheres can be used as optically detectable elements.

[0062] The medical system may also include a robotic arm configured to hold a medical device and perform a puncture using the medical device. Preferably, the robotic arm is configured to perform the puncture under software control according to the determined puncture point, puncture angle, and puncture depth. Using a camera, the position and orientation of the robotic arm can be optically detected during the puncture and evaluated by a computing unit for management.

[0063] The present invention also relates to a computer program configured to determine an insertion point for a medical instrument on a surface of an object in the coordinate system of the generated tomographic image data relative to a marker, and to transform the coordinates of the insertion point in the coordinate system of the tomographic image data into the coordinate system of the generated visual image data using the relative position of the insertion point with respect to the marker. In particular, the steps "determining an insertion point for a medical instrument" and "transforming the coordinates of the insertion point" of the method according to the invention can be realized by execution of the computer program.

[0064] The invention further relates to a computer-readable storage medium having permanently stored thereon a computer program according to the invention, the computer-readable storage medium being preferably an element of a computing unit, the computer program stored thereon being preferably loadable into a main memory and processable and executable by a processor.

[0065] The invention will be explained in more detail below on the basis of a diagrammatically illustrated embodiment with reference to the drawings. [Brief explanation of the drawings]

[0066] [Figure 1] 1 is a flowchart of a method for recreating an entry point for a medical device. [Figure 2] 1 is a schematic diagram of a medical system for replicating an insertion point for a medical device.

[0067] FIG. 1 shows a flow chart of a method for recreating an entry point for a medical device.

[0068] The method proceeds as follows:

[0069] First (step S1), at least one marker is provided on the surface of the object. The marker has the property of being detectable both by tomography, particularly fluoroscopy, and optically. The marker can be formed, for example, by adhesive tape, which is adhesively applied to the surface in a regular or irregular pattern, thus forming the marker. The marker contains barium sulfate, which is visible in a fluoroscopic recording of the marker, preferably distributed across the surface of the adhesive tape or in selected areas, so as to be detectable by tomography. The marker can also be provided on the surface of the object by adhesively applying a double-sided adhesive film with a pre-punched pattern to the surface. To form the marker, the support film of the double-sided adhesive film can be peeled off so that only the adhesive sheet with the pre-punched pattern remains on the surface. In this case, the marker is formed by a predetermined adhesive film pattern. Based on the detected deformation of the pre-punched pattern, for example, the movement of the object can be identified. The marker can also be formed by a support material on which a fluoroscopically detectable element and an optically detectable element are arranged. For example, the fluoroscopically detectable element may be a metal sphere and the optically detectable element may be a light emitting diode. The fluoroscopically detectable element and the optically detectable element are preferably positioned in a known spatial relationship to each other.

[0070] Then (step S2), tomographic image data are generated from which a fluoroscopic recording of the object together with at least one marker arranged on the surface of the object can be reconstructed. The tomographic image data can be generated, for example, using an X-ray device having an X-ray source and an X-ray detector. To generate the tomographic image data, the object is arranged between the X-ray source and the X-ray detector so that X-rays emitted from the X-ray source pass through the marker and at least a partial region of the object in which the target site to be punctured is located and are then detected by the X-ray detector.

[0071] Thereafter (step S3), an insertion point for the medical instrument on the surface of the object is determined in the coordinate system of the tomography image data with reference to at least one marker provided on the surface. For example, the insertion point for the medical instrument can first be defined in a fluoroscopic recording reconstructed from the tomography image data, which can be performed, for example, by a doctor or software. The coordinates of the defined insertion point in the coordinate system of the tomography image data can then be determined by calculation by a calculation unit. Since the positions of the markers in the coordinate system of the tomography image data are known, it is possible to determine the spatial relationship between the markers and the insertion point in the coordinate system of the tomography image data. In particular, in this case, the relative position of the insertion point with respect to the marker in the coordinate system of the tomography image data is known.

[0072] In the coordinate system of the tomographic image data, the insertion point as well as the insertion angle and / or the puncture depth for the medical device can be determined relative to the at least one marker, in which case it is known where, at what angle and to what depth the puncture medical device should be inserted into the object in the coordinate system of the tomographic image data.

[0073] Next (step S4), visual image data is generated from which a visual image of the object together with at least one marker arranged on the surface of the object can be reconstructed, the visual image data being generated preferably using a camera configured to continuously generate the visual image data as three-dimensional visual image data.

[0074] Then (step S5), the coordinates of the penetration point in the coordinate system of the tomographic image data are transformed into the coordinate system of the visual image data using the relative position to the at least one marker. If the penetration angle and / or the puncture depth have also been determined in the coordinate system of the tomographic image data, the penetration angle is also transformed into the coordinate system of the visual image data using the relative orientation of the penetration angle to the at least one marker and / or the puncture depth is also transformed into the coordinate system of the visual image data using the relative distance of the puncture depth to the at least one marker.

[0075] Thereafter (step S6), the insertion point and (if determined) the insertion angle and / or the puncture depth for the medical instrument are also reproduced in the view of the object. If present, the insertion point, the insertion angle and the puncture depth for the medical instrument are preferably reproduced together in the view of the object.

[0076] The view of the object may be a real view or a reconstructed view. The real view may be a direct view of a real surface or an indirect view of the real surface, for example, via a transparent optical display. In the direct view of the object, a penetration point for the medical instrument may be reproduced, for example, by optical marking. In the indirect view of the real surface via a transparent optical display, a penetration point for the medical instrument may be reproduced in real time with the correct perspective relative to the surface. Furthermore, in the indirect view of the object, the penetration angle and penetration depth may also be reproduced in real time with the correct perspective relative to the surface. It is conceivable to reproduce the penetration point, penetration angle, and penetration depth in real time in the indirect view of the object in the form of a digital representation of a virtual tool. The reconstructed view of the object may be, in particular, a photograph of a real-time image recording of the object, reconstructed from the generated visual image data. The reconstructed view of the object may be reproduced, for example, on a monitor, for example, on a computer monitor or the monitor of VR glasses. The penetration point, penetration angle and penetration depth can be reproduced in the reconstructed view, for example in the form of a digital representation of a virtual tool.

[0077] It is possible to reproduce only the insertion point in a single view. It is also possible to reproduce the insertion point, the insertion angle, and the puncture depth together in one view. It is also possible to reproduce the insertion point, the insertion angle, and the puncture depth in one view and reproduce only the insertion point in an additional view. In this case, the insertion point is reproduced redundantly in two different views. For example, the insertion point, the insertion angle, and the puncture depth can be reproduced in the indirect view of the object in the form of a digital representation of a virtual tool, and the insertion point can also be reproduced in the direct view by optical marking. In that case, the user can, for example, select between the two views. Optical marking can also be incorporated into the indirect view of the object.

[0078] FIG. 2 shows a schematic diagram of a medical system 200 for replicating an insertion point for a medical instrument (not shown).

[0079] The medical system 200 includes an X-ray device 204, a camera 206, a calculation unit 208, a marker 210, and two reconstruction units 214a, 214b, and is particularly suitable for implementing the method described with reference to FIG.

[0080] The marker 210 can be placed on the object 216 (not part of the medical system 200) to be punctured, for example, on a patient. In this case, the marker 210 is preferably placed on the object 216 so as to follow the movement of the object 216 and prevent relative movement between the object 216 and the marker 210. Preferably, the marker 210 is adhesively applied to the object 216. The marker 210 can be formed, for example, by adhesive tape, which is adhered to the surface of the object 216 in a regular or irregular pattern. The marker 210 is configured to be detectable both by tomography, particularly fluoroscopy, and optically. To be optically detectable, the marker 210 is preferably configured with a color and / or shape that ensures a visible contrast with the surface of the object 210 in visual image recordings. The marker 210 can, for example, contain barium sulfate as a contrast agent in predetermined areas so that it is also visible in tomography recordings.

[0081] The X-ray device 204 may be, for example, a computed tomography (CT) device and includes an X-ray source and an X-ray detector (not shown). The object 216 is positioned between the X-ray source and the X-ray detector of the X-ray device 204 so that a tomographic record of the marker 210 together with the object 216 can be reconstructed from the generated tomographic image data to generate the tomographic image data. The reconstructed tomographic record can be used to initially plan the puncture of the object 216, for example, to define a penetration point on the surface of the object 210.

[0082] The calculation unit 208 can determine the coordinates of the insertion point in the coordinate system of the tomography image data 218 relative to the position of the marker 210. The calculation unit 208 is also configured to determine an insertion angle and a penetration depth for the medical instrument in the coordinate system of the tomography image data 218.

[0083] The computation unit accesses and processes the tomography image data generated by the X-ray device 204 to determine the penetration point, penetration angle, and penetration depth in the coordinate system of the tomography image data 218. The computation unit 208 is also operatively connected to the camera 206 to access and further process the visual image data generated by the camera.

[0084] To enable the insertion point 202 a, the insertion angle, and the insertion depth to be reproduced in the view 220 of the surface of the object 216, the calculation unit 208 is configured to transform the coordinates of the insertion point, the insertion angle, and the insertion depth determined in the coordinate system of the tomography image data to the coordinate system 222 of the visual image data generated by the camera 206. The calculation unit 208 is configured to use the relative position of the insertion point with respect to the at least one marker to transform the coordinates of the insertion point. Because the position of the marker 210 is known in both the coordinate system of the tomography image data 218 and the coordinate system of the visual image data 222, the relative position of the insertion point with respect to the at least one marker 210 can be used to transform the coordinates of the insertion point. Therefore, the position of the marker 210 can be used as a reference for transforming the coordinates of the insertion point from the coordinate system of the tomography image data 218 to the coordinate system of the visual image data 222.

[0085] The calculation unit 208 is further operatively connected to a reproduction unit 214a and configured to reproduce in real time, in the correct perspective, the insertion point 202a for the medical instrument in a view 220 of the surface.

[0086] The medical system 200 includes two reproduction units 214 a and 214 b. The medical system 200 may include only one of the two reproduction units 214 a and 214 b, or may include alternative reproduction units. The reproduction unit 214 b is a video projector that is automatically calibrated by the camera 206 and configured to reproduce the insertion point 202 b as an optical marking.

[0087] The reproduction unit 214a is a transparent optical display that can be mounted together with the camera 206 in a frame, for example, in an eyeglass frame.

[0088] The view 220 on the transparent optical display 214a is an indirect view of the real surface of the object 216, in which the insertion point 202a is reproduced. The insertion point 202a can be reproduced in real time with correct perspective in the view 220. Alternatively or in addition to the optical display 214a, the medical system 200 can also have a monitor on which the insertion point is reproduced in a reconstructed view of the object.

Claims

1. A medical system for reproducing a puncture point, an puncture angle, and a puncture depth for a medical device, comprising: The system is At least one marker configured to be detectable both tomographically and optically; an imaging modality for generating tomographic image data; a camera for generating visual image data; A computation unit, - determining the insertion point for the medical device on the surface of the object to be punctured with reference to the at least one marker in the coordinate system of the tomographic image data; Transforming the coordinates of the insertion point in the coordinate system of the tomography image data into the coordinate system of the visual image data using the relative position of the insertion point with respect to the at least one marker. a computation unit configured to: a representation unit for representing in real time the insertion point, the insertion angle and the insertion depth for the medical instrument in a real or reconstructed view of the object in the form of a digital representation of a virtual tool; and the representation unit is a transparent optical display through which a view of the real surface is visible; A medical system, wherein the reproduction of the insertion point, the insertion angle and the insertion depth on the transparent optical display is performed in correct perspective with respect to the view of the real surface.

2. The camera is a light field camera, a stereo camera, a triangulation system, or a time-of-flight camera. The medical system of claim 1 .

3. the at least one marker is formed by an adhesive tape that can be applied so as to be adhesive on the surface of the object; The medical system according to claim 1 or 2.

4. The adhesive tape is fluoroscopically detectable by BaSO ｘ having The medical system of claim 3.

5. the at least one marker having at least one fluoroscopically detectable element and / or at least one optically detectable element; A medical system according to at least one of claims 1 to 4.

6. the fluoroscopically detectable element is formed from a metal and configured to be identifiable as a tomographically detectable element in a tomographic recording. The medical system of claim 5.

7. the optically detectable element is a light emitting diode configured to emit electromagnetic radiation within a predetermined wavelength range; The medical system according to claim 5 or 6.

8. the predetermined wavelength range includes infrared radiation; the camera having an infrared sensor for detecting infrared radiation emitted from the light emitting diode; The medical system of claim 7.

Images