Method and system for outputting augmented reality information

By using augmented reality technology in medical devices and medical examinations, combining images and in-depth information, the problems of physician vision and body load are solved, the efficiency and accuracy of medical operations are improved, and the health and communication efficiency of patients are improved.

CN110148453BActive Publication Date: 2024-11-01SIEMENS HEALTHINEERS AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201910496334.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2015-12-23
Filing Date
2016-12-23
Publication Date
2024-11-01
Estimated Expiration
2038-05-17

AI Technical Summary

Technical Problem

In the use of medical devices and medical examinations, physicians face the problems of vision limitations and ergonomic loads during the intervention process, and it is difficult for patients to obtain clear and timely instructions in the clinical workflow, which affects health and communication efficiency.

Method used

By obtaining a combination of image information, depth information and coordinate information, creating augmented reality information and outputting it to the doctor and patient's field of view. Augmented reality information can be automatically or fully automatic, providing accurate location based on user input and medical data sets and motor control, supporting interventional and clinical workflow execution.

Benefits of technology

It improves the efficiency of medical devices and the accuracy of medical examinations, reduces the doctor's vision and body load, enhances the patient's health and communication efficiency, and provides clearer instructions and operational support.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN110148453B_ABST
    Figure CN110148453B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and system for outputting augmented reality information. A method for outputting augmented reality information (AR) to a first user (U1) is described, wherein the augmented reality information (AR) is related to a medical device (2, IT, MD) and / or a medical examination of a patient (13), wherein the method comprises the following steps: - Acquiring (R1) first information, wherein the first information is selected from the group consisting of image information, depth information, coordinate information and a combination thereof, wherein the first information is related to the medical examination of the medical device and / or the patient (13); - Based on the first information, creating (GA) augmented reality information (AR) related to the medical examination of the medical device (2, IT, MD) and / or the patient (13); - Outputting (D1) the augmented reality information (AR) so that the augmented reality information (AR) can be perceived in the field of vision of the first user (U1).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of a patent application with an application date of December 23, 2016, application number 201611209580.0, and invention name “Method and system for outputting augmented reality information”. Technical Field

[0002] The invention relates to a method for outputting augmented reality information related to a medical device and / or a medical examination of a patient to a first user. The invention also relates to a system for outputting augmented reality information, a computer program product and a computer-readable medium. Background Art

[0003] Many users with different expertise and perspectives are usually involved in the use of medical devices (e.g. medical imaging devices) and also in the medical examination of patients. These users include in particular patients, medical technicians, physicians with different degrees of experience, technicians and medical device manufacturers. Communication between these users is important for the efficient use of medical devices or the success of medical examinations. In these cases, verbal communication has previously been largely resorted to without any technical support. In addition, known communication aids such as brochures, information sheets, information films, operating instructions, etc. are used.

[0004] The medical examination of a patient may include, for example, interventions, treatments (particularly radiation therapy), medical imaging, clinical workflows, instructions for patients, and the like.

[0005] In intervention, it is important to accurately hit the target area and protect the surrounding healthy tissue. Therefore, in many cases, intervention will be supported by medical imaging. Previously, the transmission of imaging information provided by medical imaging for intervention depended largely on the ability, experience and spatial perception of the physician performing the intervention. Typically, the imaging information provided by medical imaging is displayed on one or more monitors. The monitor or monitors are each arranged on a flexible support structure in the examination room where the intervention is performed. The relatively large distance between such monitors and the intervention area may mean a huge additional load for the physician, especially when their ability to observe the monitor is limited. In addition, the physician must often move their line of sight, especially their focus, back and forth between the intervention area and one or more monitors during the intervention process. In many cases, this is also related to the ergonomic load of the physician.

[0006] In clinical workflow, for example in radiology, it is important that the patient receives clear instructions at the right time in order to efficiently pass through the various stations and steps. Another important aspect of clinical workflow is the patient's well-being, especially with regard to anxiety, uncertainty and limitations of the privacy sphere and protection of personal data. With medical imaging, timely and clear instructions, for example those affecting breathing and the patient's well-being, are important. Summary of the invention

[0007] It is an object of the present invention to enable an improved use of a medical device and / or an improved performance of a medical examination of a patient.

[0008] This object is achieved by a method as claimed in claim 1 , a system as claimed in claim 19 , a computer program product as claimed in claim 31 , and a computer readable medium as claimed in claim 32 .

[0009] In the method of the present invention for outputting augmented reality information related to a medical device and / or a medical examination of a patient to a first user, first information is obtained, the first information being selected from the group consisting of image information, depth information, coordinate information, and a combination thereof, wherein the first information is related to the medical device and / or the medical examination. Based on the first information, augmented reality information related to the medical device and / or the medical examination of the patient is created. The augmented reality information is output so that the augmented reality information can be perceived in the field of view of the first user.

[0010] Augmented reality information is understood in particular to be information with which reality, in particular the field of view of the first user and / or the field of view of the second user, can be augmented. In particular, augmented reality information can be used to create an augmented reality based on the field of view of the first user and / or based on the field of view of the second user. The augmented reality information is particularly perceptible in the field of view of the first user when the augmented reality information is visible to the first user in the field of view of the first user. In particular, the augmented reality information can be output such that the field of view of the first user is augmented by the augmented reality information.

[0011] The augmented reality information can be created and / or output, for example, by an augmented reality device. The terms "device for creating augmented reality" and the term "augmented reality device" are used synonymously. The augmentation of reality is particularly referred to as "augmentation" for those skilled in the art. Augmented reality is particularly referred to as "augmented reality" for those skilled in the art. Augmented reality devices are particularly referred to as "augmented reality devices" for those skilled in the art.

[0012] Known devices for creating augmented reality are, for example, the Microsoft The structure and functionality of the device for creating augmented reality are known to a person skilled in the art, in particular from US2014098226A1, US2015248793A, US2015294507A1, US2015234455A1, WO2015134958A1, WO2015161307A1 and [CR06].

[0013] One embodiment form of the present invention provides each step of the described method and / or one or more steps of the described method that are automatically or fully automatically performed. Specifically, augmented reality information can be created automatically and / or fully automatically, especially by augmented reality device. In the context of the present application, "automatically" means that the corresponding steps are automatically performed by software and / or by hardware and / or substantially do not require user interaction to the corresponding steps. In particular, when only one or more results need to be confirmed and / or one or more intermediate steps are performed only by the user, substantially no user interaction is required. In the context of the present application, "fully automatically" means that for the execution of the corresponding steps, no user interaction is required at all. Whether "automatically" or "fully automatically" one or more steps are performed, the method of the present invention can be an element of the workflow that requires additional user interaction. For example, the user's interaction can include the user manually creating or selecting an inspection scheme and / or inspection plan and / or clinical problem from a menu presented, for example, by a screen.

[0014] As an alternative or in addition, the augmented reality information can be created based on the input of the first user and / or based on the input of the second user. For example, based on the input of the first user and / or based on the input of the second user, a mark can be created and / or adapted with respect to the type, position, size and / or shape, wherein the augmented reality information has the mark.

[0015] The image information may be, for example, a camera image and / or a 2D image (two-dimensional image). One embodiment of the invention provides that the first information is a 3D image (three-dimensional image). The 3D image may in particular be a combination of a 2D image and depth information. In this case, an image value of the 2D image and a depth value of the depth information may be assigned to each pixel of the 3D image.

[0016] The image information can be acquired, for example, by a first camera unit. The first camera unit can be, for example, a 3D camera. The 3D image and / or depth information can be acquired, for example, by a 3D camera. The first augmented reality device can, for example, have a first camera unit. The 3D camera can be particularly designed for the detection of electromagnetic radiation, in particular for detecting electromagnetic radiation in a lower frequency range of the spectrum compared to x-ray radiation, for example in the visible or infrared range of the spectrum. The 3D camera can, for example, be implemented as a stereo camera or as a time-of-flight measurement system. The time-of-flight measurement system is particularly referred to as a "time-of-flight camera" for those skilled in the art. The 3D camera can, for example, be implemented for acquiring a 3D image and / or for acquiring depth information by structured lighting.

[0017] The coordinate information may, for example, relate to an image data set. The image data set may, for example, have an image of a spatial region in the field of vision of the first user. In particular, at least one part of the medical device may be located in the spatial region and / or at least one sub-step of the medical examination may be performed in the spatial region. The coordinate information may, for example, specify a position and / or orientation, in particular in relation to the image data set and / or the image of the spatial region.

[0018] According to one aspect of the invention, the coordinate information comprises at least one patient coordinate relating to at least one part of the patient and / or at least one device coordinate relating to at least one part of the medical device. For example, the at least one patient coordinate may, for example, specify a position and / or orientation of at least one part of the patient, in particular a position and / or orientation relative to a map data set and / or a map of the spatial region. The at least one device coordinate may, for example, specify a position and / or orientation of at least one part of the medical device, in particular a position and / or orientation relative to a map data set and / or a map of the spatial region. At least one part of the medical device may, for example, be a movable component of the medical device.

[0019] One embodiment form of the invention provides that the position and / or the gaze direction of a first user can be determined, in particular in relation to a map data set and / or a map of a spatial region, and / or the position and / or the alignment of a first augmented reality device can be determined, in particular in relation to a map data set and / or a map of a spatial region. Augmented reality information can be created, for example, based on the position and / or the gaze direction of the first user and / or based on the position and / or the alignment of the first augmented reality device.

[0020] This can be done, for example, with the aid of image information, depth information and / or motion information. The motion information can be acquired, for example, by one or more acceleration sensors. In particular, the first augmented reality device can have one or more acceleration sensors.

[0021] In a medical device having a patient support device with a transfer plate, the coordinate information can, for example, specify the position and / or orientation of the transfer plate. In a medical device having a kinematic chain and / or a robot arm, in particular a C-arm, the coordinate information can, for example, specify the position and / or orientation for one or more elements of the kinematic chain and / or the robot arm in each case.

[0022] A number of different representations are conceivable for the coordinate information. A selection and / or changeover between these representations can be made without departing from the scope of the invention, provided that it is specified by the claims.

[0023] For example, coordinate information relating to a transfer plate may, in a first representation, have the coordinates of four corner points of the transfer plate, in a second representation have the coordinates of a center point of the transfer plate and coordinates specifying an orientation that roughly corresponds to a plane of the transfer plate, and in a third representation have a coordinate value, wherein the coordinate value is assigned a position and / or orientation of the transfer plate, for example, by means of a list, in particular a lookup table.

[0024] The coordinate information can be established, for example, by establishing (especially measuring) a position and / or an orientation. The coordinate information can be established, for example, by establishing (especially measuring) a change in position and / or a change in orientation.

[0025] According to one aspect of the invention, the coordinate information is provided by the medical device. The coordinate information may, for example, relate to a movable component of the medical device.

[0026] According to one form of embodiment of the present invention, a device coordinate system for a medical device is provided. In this case, the movable component can be controlled based on the device coordinate system, for example by a control device of the medical device, in particular for positioning and / or orientation. The coordinate information may, for example, include device coordinates with respect to the device coordinate system.

[0027] An embodiment of one form of the invention provides for performing a transformation of coordinate information and / or one of the coordinates included in the coordinate information (e.g., device coordinates) from a first reference system (e.g., device coordinate system) to a second reference system (e.g., map data set) during creation of augmented reality information. For example, the transformation may be performed based on transformation information related to the first reference system and the second reference system and / or transformation information by which coordinates of the first reference system may be assigned to coordinates of the second reference system. For example, the coordinate information may include the transformation information. Optionally, the transformation information may be established and / or obtained.

[0028] With the aid of the coordinate information, augmented reality information can be created, for example, independently of the image information and / or independently of the depth information. In this case, in particular, a device coordinate system that has been provided for the control of the medical device can advantageously be used additionally for creating augmented reality information. In particular, with the aid of the coordinate information in combination with a map data set and / or a 3D model of the movable component and / or the medical device, the position and / or orientation of the movable component can be established very accurately and independently of the visual environment. Optionally, the coordinate information can be combined with the image information and / or with the depth information.

[0029] According to one aspect of the invention, a control command relating to the movement of a movable part of a medical device is output. Preferably, coordinate information is established based on the control command. The movement of the movable part can be controlled, for example, by the control command. In particular, the control command can specify a target position, a change in position, a target orientation, a change in orientation, etc. of the movable part. In particular, the coordinate information can be assigned to the control command and / or the movement parameters.

[0030] According to one aspect of the invention, motion parameters relating to the motion of a movable part of a medical device may be acquired. Preferably, the coordinate information is established based on the motion parameters. The motion parameters may in particular depend on the motion of the movable part. In particular, the motion parameters may be a measurement of the motion of the movable part. The motion parameters may for example specify geometric variables, in particular position, length and / or angle. The motion parameters may for example specify electrical variables, in particular current, voltage, charge and / or energy. The electrical variables may for example relate to a drive unit, with which the motion of the movable part is driven.

[0031] The movable part may be, for example, a C-arm, a robot arm, a kinematic chain, a transfer plate, an interventional tool or the like.

[0032] According to one form of embodiment of the invention, the first information relates to a spatial region in the field of vision of the first user. At least a part of the medical device can be located in the spatial region. At least one sub-step of the medical examination can be performed in the spatial region. One form of embodiment of the invention provides that at least a part of the medical device is located in the field of vision of the first user and / or at least one sub-step of the medical examination is performed in the field of vision of the first user.

[0033] The part steps of the medical examination can be, for example, the part steps of intervention and / or the part steps of the clinical workflow. The part steps of intervention can be, for example, the introduction of intervention tools and / or implants into the patient, the separation and / or connection of the patient's structures, etc. The part steps of the clinical workflow can be, for example, calling the patient waiting in the waiting room, identifying the patient, clarifying other steps of the patient's clinical workflow, preparation of the patient in the examination room, indication of the patient, preparation of the medical device, registration of the patient at the medical device, medical imaging, intervention, treatment, diagnosis, etc.

[0034] According to one aspect of the present invention, a medical data set is provided by a medical device, wherein augmented reality information is created based on the medical data set. According to one form of embodiment of the present invention, the medical data set has a medical imaging data set. In particular, the medical imaging data set may relate to the structure of the patient. According to another form of embodiment of the present invention, the medical data set has a medical signal related to the patient. The medical signal may be, for example, a physiological signal and / or a bioelectric signal. In particular, the medical signal may relate to the patient's heartbeat, the patient's breathing, the patient's body temperature, the concentration of a substance in the patient's blood, and the like. The medical signal may be, for example, an electrocardiogram signal (EKG signal).

[0035] An embodiment of one form of the invention provides augmented reality information to have an output domain, wherein a medical data set is represented in the output domain. The medical data set may be represented in the output domain, for example, as a medical image, a timing characteristic, a signal curve and / or a number.

[0036] An embodiment of one form of the invention provides for establishing a position of a medical device relative to a field of view of a first user based on first information, wherein augmented reality information is created based on the position of the medical device. In this case, based on the position of the medical device, a position of an output field relative to the field of view of the first user can be determined. The output field can, for example, be displayed near the medical device and / or overlaid on the medical device. For example, the output field can follow the medical device in response to a change in the position of the medical device relative to the field of view of the first user.

[0037] According to one aspect of the present invention, examination information related to a medical examination of a patient is established based on first information, wherein augmented reality information is created based on the examination information.

[0038] The examination information can be, for example, information about which sub-steps of the medical examination have been performed and / or which sub-step is to be performed as the next step. In particular, the examination information can be established based on an examination plan including the planned sub-steps of the medical examination and / or based on an examination protocol. The examination plan can be, for example, an intervention plan and / or a process plan of a clinical workflow. The examination information can, for example, relate to the position of the patient relative to the medical device and / or the position of the intervention tool relative to the patient's structure. For example, the examination information can have the information that the intervention tool is in the correct position ready for use. The augmented reality information can have corresponding markings, which indicate the next movement of the intervention tool, which is performed, for example, in the form of a simulation.

[0039] According to one aspect of the present invention, patient information related to the patient is established based on first information, wherein augmented reality information is created based on the patient information.

[0040] According to one aspect of the present invention, the patient information is selected from the group consisting of the following information and combinations thereof:

[0041] - Patient identification information related to the identification of the patient,

[0042] - patient status information relating to the patient's physiological and / or emotional state,

[0043] - Patient location information relating to the location of the patient.

[0044] The patient identification information is designed to identify the patient. The identification can be performed in particular automatically and / or fully automatically. For example, based on the patient identification information, the patient data of the patient can be selected from a patient database and / or the patient data can be output by augmented reality information. The patient data can, for example, include personal data and / or data related to the patient's medical history. In particular, it can be checked in this way whether a given sub-step of a medical examination is planned for the patient and / or which sub-step of the medical examination is planned. In particular, patient data and / or examination results can be output by augmented reality information. For example, based on the patient identification information, the data acquired during the medical examination of the patient and / or by the medical device are automatically assigned to the patient and / or stored in a patient database. Therefore, this can avoid incorrect examinations and incorrect assignments.

[0045] The patient identification information can be established, for example, based on image information, in that a biometric feature of the patient is established and / or in that a code, in particular a QR code, assigned to the patient and / or to a patient document assigned to the patient is captured and evaluated.

[0046] The patient status information may, for example, relate to the patient's breathing, the patient's body temperature, the patient's pulse, the patient's pupil dilation, the patient's facial expression, etc. Based on the patient status information, it may be determined, for example, whether the patient is showing stress and / or to what extent. For example, if the patient's stress may adversely affect the success of the medical examination, a warning signal may be output via augmented reality information. The augmented reality information may, for example, have a calming image and / or a series of calming images for calming the patient.

[0047] The patient's position can be selected from a patient position group, for example, which is composed of the patient's position in the clinic, the patient's position in the examination room, the patient's position relative to the medical device, the patient's position relative to the first user, the patient's position relative to the second user, and a combination of the above positions. Augmented reality information can be created based on the patient position information, so that the position of the augmented reality information relative to the field of view of the first user is determined based on the patient's position. The augmented reality information can be shown near the patient and / or overlaid on the patient, for example. The augmented reality information can follow the patient, for example, to target changes in the patient's position relative to the field of view of the first user.

[0048] According to one aspect of the invention, a medical imaging dataset related to a structure of a patient is provided, wherein augmented reality information is created based on the medical imaging dataset. According to one aspect of the invention, the augmented reality information has a structure labeling that labels the structure of the patient. The medical imaging dataset can be provided, for example, by means of a medical device, in particular by means of a medical imaging device and / or by means of a medical image database. In particular, the medical image database can have a storage area, wherein the medical image dataset is maintained in the storage area for retrieval.

[0049] The structure of the patient can be, for example, the anatomical structure of the patient. The structure of the patient can have, for example, an organ of the patient and / or an implant of the patient. The implant can be, for example, a screw, a prosthesis, a vascular prosthesis, a heart valve prosthesis or the like. The structural marker can, for example, have a medical image of the structure of the patient and / or be shown as being covered on the structure of the patient in the field of view of the first user. The medical image of the structure can, for example, be segmented into partial structures. In this case, the structure of the patient does not necessarily have to be visible. For example, based on the first information and / or based on other information (for example, a topogram and / or an anatomical landmark), the position of the structure relative to the patient and / or the position of the structural marker relative to the field of view of the first user (which corresponds to the position of the structure relative to the patient) is established. The structural marker can, for example, have a 3D image of the structure and / or a 3D model of the structure. With the aid of the structural marker, for example, a medical examination, particularly an intervention, can be planned and / or supported. The structural marker can, for example, have a three-dimensional rendered image of the structure and / or an unfolded representation of the structure.

[0050] In order to improve the visualization of the medical problem or the environment, a computer-generated image can be overlaid on the patient in the correct position. In this way, for example, a three-dimensionally rendered image of the patient's structures, in particular the internal anatomical structures, can be overlaid on the patient. With such an image, for example, the path of an interventional procedure and / or for one or more parts of a vessel, the diagnostic value in each case (e.g. the fractional flow reserve (FFR)) can be displayed. For example, an expanded representation of the patient's chest can be shown overlaid on the patient's chest with the aid of augmented reality information.

[0051] An embodiment of one form of the invention provides anatomical landmarks based on first information and / or based on a medical imaging dataset to be detected and / or used as orientation points. Based on the anatomical landmarks, a rendered image can be recorded relative to the patient and / or relative to a structure of the patient. Thus, the rendered image can be displayed as an overlay on the structure of the patient.

[0052] According to one aspect of the present invention, calibration position information related to the calibration position of a patient and / or the calibration position of a medical device is provided, wherein augmented reality information is created based on the calibration position information, wherein the augmented reality information has a calibration position marker, which marks the calibration position of the patient and / or the calibration position of the medical device.

[0053] If the calibration position information relates to the calibration position of a movable part of the medical device, the calibration position information particularly relates to the calibration position of the medical device. The calibration position information can, for example, be established based on a pre-examination, based on an examination plan and / or based on an examination plan for a medical examination. The examination plan can particularly be an imaging plan. The calibration position information can, for example, specify how to position a patient for a medical examination and / or relative to the medical device. The calibration position information can, for example, specify how the medical device and / or the movable part are positioned relative to the patient and / or relative to the examination room. The calibration position marking can, for example, have a 3D avatar of the patient, wherein the 3D avatar shows the patient in the calibration position of the patient. The calibration position marking can, for example, have a 3D model of the medical device and / or the movable part, wherein the 3D model shows the medical device in the calibration position of the medical device and / or the movable part in the calibration position of the movable part.

[0054] According to one aspect of the present invention, region information relating to a region of a patient and / or a region of a medical device is provided, wherein augmented reality information is created based on the region information, wherein the augmented reality information has a region mark marking the region of the patient and / or the region of the medical device.

[0055] The area of ​​the patient can, for example, have the structure of the patient. The area of ​​the patient can, for example, be an area of ​​the patient to be examined and / or selected for the medical examination in the medical examination. The area of ​​the patient can, for example, be an area of ​​the patient located in the effective area of ​​the medical device and / or selected for transfer to the area of ​​action of the medical device. The area of ​​action of the medical device can, for example, be the area of ​​action of the intervention tool, the original image data acquisition area of ​​the medical imaging device, etc. The area of ​​the patient can, for example, be an area of ​​the patient to be intervened and / or from which the medical imaging data set is recorded. The area of ​​the medical device can, for example, be an area to be used and / or actuated by the medical device. The area marking can, for example, have a 3D image of the area of ​​the medical device and / or a 3D model of the area of ​​the medical device. This enables planning and / or supporting the operation of the medical device, for example, in the medical examination and / or in the preparation of the medical device.

[0056] According to one form of embodiment of the invention, based on the input of the first user and / or based on the input of the second user, the region information can be created and / or the region marking can be adapted with respect to type, position, size and / or shape. One form of embodiment of the invention provides for establishing the region information based on an examination plan, based on an examination protocol of a medical examination, based on patient position information and / or based on calibration position information.

[0057] According to one aspect of the invention, a marking selected from the group of markings consisting of structure markings, calibrated position markings, area markings and combinations thereof has a representation selected from the group of representations consisting of a hologram, a holographic type representation, a stereoscopic representation, a three-dimensional representation and combinations thereof. This makes it possible in particular to represent 3D avatars, 3D images and / or 3D models in a particularly advantageous manner. A holographic type representation may in particular be understood as a visual representation which gives an observer, for example the first user and / or the second user, the impression that they are observing a hologram.

[0058] According to one aspect of the present invention, an input of a first user is detected, wherein augmented reality information is created based on the input of the first user. In particular, a mark selected from a mark group consisting of a structure mark, a calibrated position mark, an area mark, and a combination thereof can be created based on the input of the first user, and / or these marks can be adapted in terms of type, position, size, and / or shape.

[0059] One aspect of the present invention provides outputting augmented reality information such that the augmented reality information can be perceived in a field of view of a second user, and such that an input of the second user is detected, and the augmented reality information is created based on the input of the second user.

[0060] The augmented reality information is particularly detectable in the field of view of the second user when the augmented reality information is visible to the second user in the field of view of the second user. In particular, the augmented reality information may be output such that the field of view of the second user is enhanced by the augmented reality information. According to one form of embodiment of the invention, at least a portion of the field of view of the first user is shown in the field of view of the second user. According to one form of embodiment of the invention, at least a portion of the field of view of the first user is shown in the field of view of the second user such that the second user sees the patient and / or the medical device from the perspective of the first user. According to one form of embodiment of the invention, the augmented reality information is shown in the field of view of the second user such that the second user sees the augmented reality information from the perspective of the first user.

[0061] In particular, features described with respect to the first user without reference to the second user also relate accordingly to the second user, i.e. in these features, the term "first user" is replaced by the term "second user" in each case. In particular, a mark selected from the group of marks consisting of structure marks, calibrated position marks, area marks and combinations thereof can be created based on the input of the second user and / or the mark can be adapted with respect to type, position, size and / or shape. According to one aspect of the invention, the calibrated position information and / or the area information can be established based on the input of the first user and / or based on the input of the second user.

[0062] Preferably, the input of the first user is a gesture of the first user and / or the input of the second user is a gesture of the second user.

[0063] According to one aspect of the invention, a first user profile associated with a first user and / or a second user profile associated with a second user is selected, wherein augmented reality information is created based on the first user profile and / or based on the second user profile.

[0064] The user profile may, for example, include a user category, a function in a medical examination, an authorization to access patient data of a patient, an authorization to use a medical device, an authorization related to the creation, change, overwriting and / or removal of augmented reality information, etc. A user category may, for example, relate to a group of people. Optionally, a group of people may be assigned to the first user and / or the second user, respectively. For example, multiple groups of people may be patients, medical technicians, physicians with different levels of experience, technicians, and medical device manufacturers. Functions in an intervention may, for example, be selected from a group of functions consisting of: a first physician who performs the intervention himself; a second physician who performs the intervention under guidance; a third physician who observes the intervention and learns while it is in progress; a fourth physician who observes the intervention and gives instructions for performing the intervention while it is in progress.

[0065] An embodiment of one form of the present invention provides for creating augmented reality information based on first information and based on basic information. The basic information may be selected from a basic information group, the basic information group consisting of a medical data set, examination information, patient information, a medical imaging data set, calibration position information, region information, an input of a first user, an input of a second user, a first user profile, and a second user profile.

[0066] One form of embodiment of the invention provides for creating augmented reality information based on first information and based on a plurality of basic information items. Each of the plurality of basic information items can, for example, be selected in each case from a basic information group consisting of a medical data set, examination information, patient information, a medical imaging data set, calibration position information, area information, an input of a first user, an input of a second user, a first user profile and a second user profile.

[0067] In the context of the present application, the expression "based on" can be understood in particular in the sense of the expression "when used". In particular, the statement that a first feature is created (alternatively established, determined, etc.) based on a second feature does not exclude that the first feature is created (alternatively established, determined, etc.) based on a third feature.

[0068] The system of the present invention for outputting augmented reality information related to a medical device and / or a medical examination of a patient to a first user has a first acquisition module, a creation module, and a first output module. The first acquisition module is implemented to acquire first information selected from a group consisting of image information, depth information, coordinate information, and a combination thereof, wherein the first information is related to the medical device and / or the medical examination. The creation module is implemented to create augmented reality information related to the medical device and / or the medical examination of the patient based on the first information. The first output module is implemented to output the augmented reality information so that the augmented reality information can be perceived in the field of view of the first user.

[0069] According to one aspect of the invention, the system comprises a medical device. Preferably, the medical device is embodied to provide coordinate information.

[0070] According to one aspect of the invention, a medical device has a movable component.

[0071] According to one form of embodiment of the present invention, the system and / or the medical device has a control device, which is implemented to output a control command related to the movement of the movable part. According to one form of embodiment of the present invention, the system has a motion parameter acquisition unit implemented to acquire motion parameters related to the movement of the movable part. The motion parameter acquisition unit can, for example, have a sensor. The motion parameters can be acquired, for example, by the motion parameter acquisition unit.

[0072] One form of embodiment of the invention provides for the movable component to be controllable, in particular to be positioned and / or oriented, by means of the first augmented reality device and / or by means of the second augmented reality device.

[0073] According to one form of embodiment of the invention, the medical device has a drive unit, which is implemented to drive the movement of the movable part. Preferably, the drive unit is implemented for receiving control commands and / or for driving the movement of the movable part based on the control commands. The drive unit can be implemented as a motor, for example. The movement parameters can, for example, specify the charge consumed by the drive unit when driving the movement and / or the electrical energy consumed by the drive unit when driving the movement.

[0074] The motion parameter acquisition unit can be integrated, for example, in the control device and / or in the drive unit. The motion parameter acquisition unit can be implemented, for example, for measuring motion parameters. The motion parameters can, for example, specify the rotational speed angle, in particular the number of revolutions. The motion parameters can, for example, be measurements of the motion of the movable part and / or relate to the movable part, the drive unit and / or the motion parameter acquisition unit. In particular, a calibration and / or simulation of the medical device and / or the control device and / or the movable part and / or the drive unit and / or the motion of the movable part can be performed.

[0075] Preferably, the system has a creation unit which is designed to create the coordinate information based on the control command and / or based on the motion parameters. The coordinate information can be created by the creation unit based on the control command and / or based on the motion parameters, for example.

[0076] According to one form of embodiment of the present invention, the first acquisition module has a first camera unit, wherein the first camera unit is implemented to acquire image information and / or depth information. According to another form of embodiment of the present invention, the first acquisition module can access a storage area of ​​the information providing module, wherein the first information is acquired by data transmission from the information providing module to the first acquisition module. According to one form of embodiment of the present invention, the system has an information providing module. The information providing module is implemented to provide the first information.

[0077] According to one form of embodiment of the present invention, the first output module is implemented to output augmented reality information so that the field of vision of the first user is enhanced by the augmented reality information. According to one form of embodiment of the present invention, the first output module has a first observation device, in particular in the form of data glasses, data contact lenses and / or retinal projectors. In particular, augmented reality information can be output by the first observation device. According to another form of embodiment of the present invention, the first observation device can access a first storage area of ​​the first output module, wherein the augmented reality information will be output to the first observation device via data transmission of the first output module. According to one form of embodiment of the present invention, the system has a first observation device.

[0078] According to one aspect of the invention, the system has a medical device, wherein the medical device is implemented to provide a medical data set. Preferably, the creation module is implemented to create augmented reality information based on the medical data set.

[0079] According to one aspect of the invention, the system has a medical imaging device, wherein the medical imaging device is implemented to provide a medical imaging data set related to a structure of a patient. Preferably, the creation module is implemented to create augmented reality information based on the medical imaging data set. Preferably, the augmented reality information has a structure label marking the structure of the patient.

[0080] According to one aspect of the present invention, the system has a first input module implemented to obtain input from a first user and / or a second input module implemented to obtain input from a second user. Preferably, the creation module is implemented to create augmented reality information based on the input from the first user and / or based on the input from the second user.

[0081] An embodiment of one form of the present invention provides that: the first acquisition module has a first input module and / or the first acquisition module is implemented to acquire the input of the first user and / or the first camera unit is implemented to acquire the input of the first user. According to one aspect of the present invention, the system has a second output module and a second acquisition module. The second output module is implemented to output augmented reality information so that the augmented reality information can be perceived in the field of view of the second user. An embodiment of one form of the present invention provides that: the second acquisition module has a second input module and / or the second acquisition module is implemented to acquire the input of the second user and / or the second camera unit is implemented to acquire the input of the second user.

[0082] According to one form of embodiment of the present invention, the second output module is implemented to show at least a portion of the first user's field of view in the second user's field of view. According to one form of embodiment of the present invention, the second output module is implemented to output augmented reality information so that the field of view of the second user is enhanced by the augmented reality information.

[0083] According to one form of embodiment of the invention, the second output module has a second viewing device, in particular in the form of data glasses, data contact lenses and / or a retinal projector. In particular, the augmented reality information can be output via the second viewing device. According to another form of embodiment of the invention, the second viewing device has access to a storage area of ​​the second output module, wherein the augmented reality information is to be output to the second viewing device via data transmission of the second output module. According to one form of embodiment of the invention, the system has a second viewing device.

[0084] According to one form of embodiment of the present invention, the second acquisition module has a second camera unit, wherein the second camera unit is implemented to acquire the input of the second user. According to another form of embodiment of the present invention, the second acquisition module can access the storage area of ​​the input providing module, wherein the input of the second user is acquired by data transmission from the input providing module to the second acquisition module. According to one form of embodiment of the present invention, the system has an input providing module.

[0085] According to one aspect of the present invention, the system has a first augmented reality device, which has a first acquisition module and / or a first output module. The first augmented reality device can be implemented as data glasses and / or data headphones, for example. The first augmented reality device is worn and / or wearable by a first user, for example. According to an embodiment of one form of the present invention, the first augmented reality device has a creation module.

[0086] According to one aspect of the invention, the system has a second augmented reality device, which has a second acquisition module and / or a second output module. The second augmented reality device can be implemented as data glasses and / or data headphones, for example. The second augmented reality device is worn and / or wearable by the second user. According to an embodiment of one form of the invention, the second augmented reality device has a creation module.

[0087] One form of embodiment of the present invention provides that: the creation module has a first creation submodule and a second creation submodule, the first augmented reality device has the first creation submodule, and the second augmented reality device has the second creation submodule.

[0088] One form of embodiment of the present invention provides for creating augmented reality information by a machine learning algorithm. The machine learning algorithm may, for example, evaluate the first information, the examination information, the patient information, the calibration position information, the region information, the input of the first user and / or the input of the second user.

[0089] According to one aspect of the present invention, the system has a user profile selection module. The user profile selection module is implemented as a first user profile associated with the first user and / or a second user profile associated with the second user. Preferably, the creation module is implemented as creating augmented reality information based on the first user profile and / or based on the second user profile. The user profile selection module can, for example, provide a graphical user interface, wherein the first user profile and / or the second user profile can be set and / or selected by the graphical user interface. According to an embodiment of one form of the present invention, the user profile selection module has a security interface for obtaining a security certificate. The security certificate can be a password and / or a hardware key, such as a smart card. In particular, this enables protection of access to the first user profile and / or access to the second user profile.

[0090] According to one aspect of the invention, the system is implemented to perform the method of the invention for outputting augmented reality information related to a medical examination of a medical device and / or a patient.

[0091] In particular, the invention enables communication between multiple users, in particular improves communication between multiple users from different groups of people involved in the use of medical devices and / or involved in the medical examination of patients. For a user of a device for creating augmented reality, additional (audio) visual elements can be added to the real world he or she perceives in their field of vision, existing elements can be edited to delete and / or have their appearance changed. In addition, images and sounds can be picked up from the user's perspective, and / or the user's viewing direction can be determined.

[0092] This information can be provided to one or more other users of another device in each case for creating an augmented reality, in particular as augmented reality information. In this case, the augmented reality information can be provided at the same location, or at another remote location by remote data transmission. The augmented reality information and / or the created augmented reality can be used to support communication between users involving the use of a medical device (e.g., a medical imaging device) and / or involving a medical examination. The user can, for example, be selected from a user group consisting of a first user and a second user. The number of users or further users can be selected from a user group consisting of a first user and a second user in each case.

[0093] One application of medical imaging is in the field of intervention, where images provided by a medical imaging device are used during a medical intervention in order to plan the intervention and to check the progress of the intervention. This type of use of medical imaging devices places high demands on the physician in many cases and requires a high level of training; in these trainings, usually an experienced physician transfers his or her knowledge to a less experienced physician. In this case, the communication between the two physicians can be supported by a device for creating an augmented reality, so that the experienced physician observes the images recorded from the perspective of the inexperienced physician and can insert hints about how to proceed further directly in the inexperienced physician's field of vision. In this case, the inexperienced physician wears a device for creating an augmented reality, while the experienced physician likewise wears such a device or uses a conventional device for information processing, such as a laptop or tablet. One form of embodiment of the invention provides that in each case the first input module and / or the second input module has a mouse, a joystick, a touchpad, a touch-sensitive input device and / or the like.

[0094] An experienced physician can, for example, display the calibrated position of the ablation needle on the patient during an intervention for ablation, and thus instruct an inexperienced physician. Likewise, the roles can be reversed, and an experienced physician can perform the intervention. The inexperienced physician can then observe the intervention from the perspective of the experienced physician, thereby learning by example. In this case, it does not matter whether the experienced and inexperienced physicians are located in the same room, so that the instruction can also be completed "remotely" (i.e., by remote transmission). For this reason, it is useful to also have a two-way transmission of audio signals and a transmission of augmented reality information so that the two physicians can talk to each other. An inexperienced physician does not absolutely need to place him or herself in the perspective of an experienced physician while performing the intervention. For example, the first information and / or augmented reality information can be recorded and / or reinforced with additional information inserted or retroactively inserted during the medical examination, and is available to the inexperienced physician later.

[0095] One embodiment of the invention provides that the support of an inexperienced physician by an experienced physician does not take place in the sense of training, but rather during normal operation. Thus, for example in rural areas, when complications occur or in particularly complex cases, an experienced physician is available on demand as an expert in order to support the physician. In particular, the expert can support the physician in performing imaging-supported biopsies. This support can also be provided prior to an intervention in order to determine the best possible access for the intervention. For example, the expert can assist the physician in planning an interventional path on the basis of a medical imaging dataset.

[0096] The invention in particular makes it possible to support a dialogue between a patient and a physician. This can take place, for example, before an intervention, wherein the physician explains the execution or the results of the intervention to the patient or visualizes the risks in augmented reality and / or via augmented reality information. Likewise, after the execution of the intervention, the physician can display the results of the examination to the patient via augmented reality information and / or in augmented reality.

[0097] In this case, these can advantageously be displayed to the patient in a "virtual mirror". It is in fact also conceivable to display the results of the examination to the patient by inserting the augmented reality information directly into the examination points on the patient's body. However, on the one hand, this may be complicated for the patient or not be visible at all (for example for the spine). On the other hand, if such a representation of the results is too realistic, it may be very disturbing for the patient. Therefore, it may be advantageous not to insert such a representation of the results into the patient him / herself, but on a "virtual mirror", i.e. on a surface within his augmented reality; this "virtual mirror" shows the patient similar to a mirror image, and the examination results can be inserted into this "virtual mirror". Alternatively or additionally, a monitor together with a camera can be used for such a representation of the results.

[0098] The invention makes it possible, for example, to improve the guidance of patients before and / or during medical examinations. For patients, medical examinations, for example, which can be performed with medical devices, often indicate abnormal conditions. Generally speaking, patients have little understanding of the procedures during the clinic and examination.

[0099] The communication between clinical staff and patients can be supported by using a device for creating augmented reality. For this purpose, for example, the patient can be guided through the clinic to the examination position by augmented reality information inserted into his or her field of vision, such as an arrow on the floor. At the examination position, he or she can be shown whether he or she still needs to wait, how many patients are in front of him or her, how long the waiting time may be, and the examination room where he or she should go to the examination by augmented reality information, especially augmented reality information in the form of audio-visual information. Within the framework of examination preparation, he or she can be instructed to take off his or her clothes and / or put on other clothes according to the examination by the augmented reality information inserted, especially augmented reality information in the form of audio-visual information. The patient can be further instructed to take a specific posture in the examination room, such as lying on the examination bed in a specific orientation and / or raising his or her arms. This can also be supported by a "virtual mirror", in which the patient sees himself / herself and / or his or her own avatar in a mirror generated in augmented reality. After the sub-step of the examination, it can be indicated that he or she can leave the way of the examination room and / or be guided to another sub-step of the examination (for example in another examination room).

[0100] A particular advantage of using a device for creating augmented reality is the fact that the augmented reality information can be tailored to the individual patient and that multiple patients in the same room can each perceive their own augmented reality information, e.g. instructions about which examination room they should go to. Another advantage is the fact that confidentiality is better maintained when each patient can perceive only the instructions for him or her and that, for example, calling patients by name in the waiting room can be dispensed with.

[0101] The invention also enables improved support for personal communication between the patient and the operator of the medical device. During a CT or MRT examination, it is usually not possible for the operator to always be with the patient. On the contrary, for CT, for example for radiation protection reasons, they exit from the examination room into a control room. By means of augmented reality information, the presence of the operator in the examination room can be shown to the patient, for example with simultaneous two-way sound transmission. By means of augmented reality information, the patient can be displayed as an element of augmented reality in the control room for the operator.

[0102] The two-way voice transmission can be carried out, for example, via an intercom system. Alternatively or additionally, the two-way voice transmission can be modified in real time, for example, in particular by simultaneous machine translation. In this way, the patient and the operator can understand each other even if they do not speak a common language.

[0103] By these applications of augmented reality, the patient does not feel alone during the examination and his or her anxiety can be reduced. In particular, an operator represented by an avatar can be displayed for the patient in the examination room with the aid of augmented reality information. The avatar can, for example, be always present. The operator can, for example, only associate with the avatar when, for example, a patient condition requiring attention (e.g. noise and / or movement) is detected by sensors. In this way, the operator can do their work, while on the other hand a feeling of always being present is created for the patient, which helps to reduce his or her tension and anxiety.

[0104] The invention in particular makes it possible to support an operator, for example, in the placement of a patient, in the selection of a support aid or in the placement of EKG electrodes or a device for measuring respiration. In this case, this support can be provided on the one hand by another person (e.g., an application expert of the manufacturer, who inserts information for the placement and / or selection of the operator via augmented reality information). On the other hand, augmented reality information can be automatically created, for example, based on the manufacturer's specifications, wherein these specifications can additionally be adapted to the specific requirements of the individual clinics, for example, based on the measurement scheme of the selected medical device or based on other data. The other data can be, for example, anatomical landmarks of the patient. Anatomical landmarks can be established in particular based on image information and / or based on depth information. The optimal position of the EKG electrodes can be calculated, for example, via anatomical landmarks.

[0105] Thus, communication from the manufacturer to the user can be supported by augmented reality information, which would otherwise first take place via training and via operating instructions.

[0106] The invention makes it possible in particular to improve the training of operating personnel. For this purpose, for example, a virtual patient can be introduced into the augmented reality of the operating personnel to be trained by means of augmented reality information, wherein an examination can be carried out as an exercise with the virtual patient. In this case, the virtual patient can be generated completely by computer, i.e. also by input from another person, for example likewise by means of a device for creating augmented reality. This procedure offers the advantage, in particular for X-ray devices, that in this way the triggering of X-ray radiation can also be simulated during training. Training with real patients is associated with severe restrictions, in particular due to the radiation load.

[0107] According to one form of embodiment of the invention, the first augmented reality device has a first camera unit, wherein the first camera unit is embodied for acquiring image information and / or depth information.

[0108] One form of embodiment of the invention provides that the first augmented reality device is implemented as data glasses. Such data glasses can be worn by a user, for example. The user can be, for example, a radiologist or an auxiliary medical technician. In one form of embodiment of the invention, an image of the patient is acquired by a first camera unit. The image is compared with an entry in a patient database. This makes it possible to determine which patient is involved and / or whether a planned examination matches. This identification of the examination makes it possible to avoid incorrect examinations and incorrect assignment of examination results.

[0109] In another variant of the invention, information, annotations and / or images from the pre-examination of the patient are communicated to the user via augmented reality information and / or are made available in augmented reality.

[0110] Through smarter presentation and interactive interaction with data presented through augmented reality information (images, histology, blood values, etc.), the user can generate a comprehensive holistic picture of the patient very quickly.

[0111] Based on the patient information, in particular based on the patient identification information, patient history information related to the patient's medical history can be established. Based on the patient information, for example, characteristic features of the patient can be established. For example, with the aid of a database storing reference data sets, based on the patient information, a suitable reference data set can be established, which can be used, for example, for dose adjustment in medical examinations using ionizing radiation.

[0112] A medical image, for example a 3D model of a patient's body region corresponding to the medical image, which was created within the framework of a preliminary examination, can be shown projected on and / or overlaid by the augmented reality information. In this way, the user can quickly and accurately recognize whether diagnosis-related information is already present at a specific point, for example a pain point.

[0113] An embodiment of one form of the invention provides for identifying a patient based on image information, establishing patient information and / or patient data based on the first information, and making the patient information and / or patient data available via augmented reality information. This may be advantageous, particularly when an emergency doctor is deployed to an accident scene.

[0114] According to another form of embodiment of the invention, the augmented reality information is presented in a location-specific manner in the user's field of view. For example, a diagnostic display arranged in an examination room, in particular an OP room, can be enhanced by augmented reality information. The augmented reality information can, for example, have an EKG signal, a respiratory curve, a patient's medical history, a patient's biological data. The augmented reality information can be output in particular during observation of the medical device and / or the diagnostic display.

[0115] In another form of embodiment of the invention, a user, such as a radiologist, a paramedical technician or a nurse, will be equipped with a device for creating an augmented reality, such as a device in the form of data glasses. Preferably, a unique feature that can be automatically evaluated, such as a QR code, is arranged on the patient and / or the patient document. When viewing this feature, for example, a virtual ticket can be shown as augmented reality information. Thus, the user can see the current information related to the patient.

[0116] Another form of embodiment of the invention provides that the user can log into an account generated for the user. With this account, for example, a user profile of the user can be managed, which user profile includes in particular the rights of the user. In this way, in particular by means of augmented reality information, only that information and / or those images can be output for which the user is authorized and / or which relate to the patient assigned to the user.

[0117] The first information can, for example, have a topogram, in particular a camera-based topogram. In particular, the topogram can be a combination of image information and depth information. According to one form of embodiment of the present invention, a camera-based topogram is acquired by, for example, a first camera unit integrated in a first augmented reality device. The camera-based topogram can, for example, be shown as being overlaid on the patient by augmented reality information. In addition, characteristic features of the patient can be established based on the camera-based topogram. By means of a database that maintains a reference data set, suitable reference data for a medical examination can therefore be established based on the camera-based topogram, which reference data can, for example, be used for dose adjustment.

[0118] According to one form of embodiment of the invention, the patient status information is transmitted to the user via augmented reality information. The patient status information may be, for example, the patient's facial expression, the patient's body temperature, the patient's pupil dilation, etc. The patient status information may, for example, relate to an emotion indicator, which may in particular point to the patient's stress. By means of augmented reality information, for example, one or more physiological parameters of the patient may be displayed, in particular pulse, respiratory rhythm, etc.

[0119] One form of embodiment of the invention provides, for example, acquiring patient information by means of data glasses immediately before an intervention, comparing the acquired patient information with entries in a database, establishing matching entries from the database, for example based on similar physical features of the patient and / or based on similar interventions, and displaying the intervention result assigned to the matching entry as a projection and / or overlay on the surface of the patient. In this way, a better impression of the desired intervention result can be exerted on the user, in particular the radiologist and / or surgeon.

[0120] According to another form of embodiment of the invention, the patient is equipped with data glasses. Physiological parameters of the patient, such as the heartbeat, can be acquired, which allow to get an impression of the patient's emotional state. If these physiological parameters point to the patient being under stress, a calming environment for relieving stress can be displayed via the data glasses. For example, a patient with claustrophobia who is examined using a medical imaging device can be shown a countryside suggesting a large open space. As an alternative or in addition, examination-related instructions, such as breathing commands, can be displayed via the data glasses. In addition, audio signals can also be used.

[0121] According to one form of embodiment of the present invention, virtual, for example computer-generated data related to the medical examination can be displayed as an overlay on the patient by means of augmented reality information. Thus, in particular, better capabilities for conceiving and performing medical examinations can be achieved.

[0122] With the device for creating augmented reality, in particular medical images (e.g. DICOM images), additional information can be displayed as an overlay on the patient via the augmented reality information. Using the device for creating augmented reality for medical examinations (e.g. interventions), a great improvement can be achieved, in particular in terms of visualization, display of information and provision of virtual processing tools (in particular image processing tools).

[0123] The medical imaging data set may, for example, have medical images, in particular DICOM images, CT images, MRT images, ultrasound images, angiographic image series, endoscopic videos, etc. The augmented reality information may have a medical imaging data set and / or a marker established based on the medical imaging data set. Therefore, the medical imaging data set and / or the marker may be displayed as an overlay on the patient in the user's field of view. In this case, the structural markers shown and / or established by the medical imaging data set may be aligned with the corresponding structures of the patient.

[0124] In this way, a user undergoing a medical examination may access information from a medical imaging dataset more quickly, ergonomically, more conveniently, and with reduced error susceptibility.

[0125] The device for creating augmented reality can be implemented to create a virtual screen in the user's field of vision. In particular, the virtual screen can be available in any position and direction of the user's line of sight. This is particularly advantageous in terms of ergonomics. In addition, the restrictions on the user's freedom of movement by the monitor and / or the support structure are reduced.

[0126] The augmented reality information may have a virtual screen, for example. For example, the virtual screen may be displayed or edited and deleted as needed. Medical data sets (particularly medical images), examination information and / or patient information may be displayed in the virtual screen, for example.

[0127] An embodiment of one form of the invention provides that a structure of the patient is selected based on a medical imaging dataset and the augmented reality information has a structure label marking the selected structure of the patient. In this way, it becomes easier to identify small and hard-to-find structures of the patient.

[0128] The structural markings may, for example, mark the interventional area. The structural markings may also have a representation of an interventional planning model, wherein the model relates to a planned intermediate step or a planned outcome of the intervention. The interventional planning model may, for example, be established by simulation. With the aid of augmented reality information, the interventional planning model may be displayed as an overlay on the patient. The interventional planning model may be used to improve navigation during the intervention. For example, the interventional planning model may have a planned instrument path. In this way, users with less experience in a given intervention may also be able to successfully perform a given intervention.

[0129] The intervention planning model may have a model of the implant. The intervention planning model may be adapted to the patient's structure and / or the patient's anatomical features, for example, with the aid of gesture information from the user. In this way, the optimal position, size and shape of the implant may be established.

[0130] The structural marking can, for example, have a segmented representation of a structure (particularly an organ) of the patient. With the aid of augmented reality information, for example, diagnostic information related to the diagnosis result can be displayed as an overlay on the patient. Thus, the user is enabled to identify critical areas of the structure and avoid damage during the intervention.

[0131] According to one form of embodiment of the present invention, patient status information is outputted via augmented reality information. The patient status information may, for example, relate to information such as the patient's heartbeat, the patient's breathing, the patient's medical history, the patient's laboratory values, the patient's demographic description, etc. In this way, the patient status information may be outputted in the user's field of view. Thus, an improved decision basis may be provided for the user, in particular a physician who is examining the patient.

[0132] The augmented reality information may have a virtual processing tool, in particular a virtual image processing tool. For example, an area of ​​the patient may be marked and / or measured by the virtual processing tool. In the measurement of an area of ​​the patient, for example, the position, orientation, shape, length, surface, volume, etc. of the area may be established.

[0133] The augmented reality information may include diagnostic information. The diagnostic information may be provided, for example, by a PACS dataset and / or by a CAD dataset. The diagnostic information may be established, for example, based on a medical imaging dataset. The medical imaging dataset may include, for example, a rendered image (rendered image) and / or an expanded image (expanded image) of a patient structure (e.g., a skull, a thorax, in particular one or more ribs). The rendered image may in particular be based on a user-specific rendering.

[0134] The calibration position information can be established, for example, based on the position of the patient during a pre-examination and / or a treatment (particularly a radiotherapy treatment) performed at a time prior to the medical examination. Thus, the reproducibility of the patient's position can be improved.

[0135] According to one form of embodiment of the invention, the medical device is selected from the group consisting of a medical imaging device, an interventional tool, an examination device implemented for acquiring and / or providing a medical data set, and combinations thereof.

[0136] The features described with respect to the medical device also apply accordingly to the virtual medical device, i.e. in these features, the term "medical device" is replaced by the term "virtual medical device" in each case. The features described with respect to the medical examination apply accordingly to the virtual medical examination, i.e. in these features, the term "medical examination" is replaced by the term "virtual medical examination" in each case. The features described with respect to the patient apply accordingly to the virtual patient, i.e. in each case, the term "patient" is replaced by the term "virtual patient". In this way, the training of medical personnel and / or the planning of medical examinations can be improved. In particular, realistic training and / or realistic planning can be achieved even without using real patients and / or real medical devices.

[0137] One form of embodiment of the invention provides that a first user is able to perceive their environment through a first augmented reality device. In particular, an image of a patient and / or an avatar of a patient may be displayed to the first user instead of a real patient, and / or an image of a medical device and / or a model of a medical device may be displayed to a user in the correct position instead of a real medical device. In this way, a device for creating virtual reality may also be used as an augmented reality device. A known device for creating virtual reality is, for example, the Rift from Oculus VR Ltd. in the United States.

[0138] An embodiment of one form of the present invention provides that the system of the present invention and / or multiple components of the system of the present invention are implemented at least in part in the form of software on a processor system. Specifically, the first acquisition module, the creation module, the first output module, the information providing module, the second output module, and the second acquisition module can each form a component of the system of the present invention and / or each be implemented at least in part in the form of software on a processor system. An embodiment of one form of the present invention provides that the system of the present invention and / or multiple components of the system of the present invention are implemented at least in part in the form of software-supported hardware (e.g., FPGA, processor system, etc.).

[0139] One form of embodiment of the invention provides that the system of the invention and / or multiple components of the system of the invention are at least partially formed by the cloud by means of cloud computing. One form of embodiment of the invention provides that the creation module and / or at least submodules of the creation module are formed by the cloud by cloud computing. The cloud can in particular have a network of storage areas that are spatially remote from each other and processor systems that are spatially remote from each other. For data transmission from and / or to the cloud, the first augmented reality device can have a first cloud interface and / or the second augmented reality device can have a second cloud interface.

[0140] Data can be transferred between components of the system, for example, via a suitable interface. An embodiment of one form of the invention provides that the interface for data transfer to and / or from components of the system of the invention is implemented at least partially in the form of software. In particular, the interface can have access facilities to a suitable storage area, wherein in this storage area, data can be appropriately buffered, retrieved and updated. The interface can also be implemented as an interface consisting of hardware, which can be activated by appropriate software.

[0141] A predominantly software-based implementation of the system of the invention has the advantage that even augmented reality devices that have already been used previously can be upgraded by a software update so as to operate in the manner of the invention. To this end, the object is also achieved by a corresponding computer program product with a computer program that can be loaded into a storage device of a computer, wherein the steps of the method of the invention can be performed with the computer program when the computer program is executed on the computer. Such a computer program product as well as the computer program can include additional software components (e.g. documentation) and / or hardware components (e.g. a hardware key (dongle etc.) for the software).

[0142] In order to transfer the computer program to and / or to store the computer program in the augmented reality device, a computer-readable medium (e.g. a memory stick, a hard disk or another transferable or permanently installed data medium) can be used, wherein the computer program is stored on the medium, the computer program can be loaded into the storage device of the computer, wherein when the computer program is executed on the computer, the steps of the method of the present invention will be performed using the computer program. One form of embodiment of the present invention provides that the first augmented reality device has a first computer and / or the second augmented reality device has a second computer. The first computer and / or the second computer can each have a processor system, for example a microprocessor or a plurality of microprocessors working together.

[0143] According to one aspect of the present invention, the medical imaging device is selected from the group consisting of a C-arm X-ray device, a computed tomography device (CT device), a single photon emission computed tomography device (SPECT device), a positron emission tomography device (PET device), a magnetic resonance tomography device (MRT device) and a combination thereof. In particular, the medical imaging device may have an X-ray device, an ultrasound device, etc. The medical imaging device may also have a combination of multiple imaging and / or radiation modalities. In this case, the radiation modality may, for example, have a radiation device for therapeutic radiation.

[0144] Within the framework of the present invention, features described in different forms of embodiments and / or different claim types (methods, systems, etc.) can be combined into other forms of embodiments. In particular, the features, advantages and forms of the embodiments described in the method of the present invention are also transferred to the system of the present invention, the computer program product of the present invention and the computer-readable medium of the present invention, and vice versa. In other words, it is also possible to further form a device claim with features described or claimed in conjunction with the method. In this case, the functional features of the method of the present invention can also be realized by components or modules of the corresponding implementation of the system of the present invention. The use of the indefinite article "a" or "an" does not exclude that the features involved also exist multiple times. A module can have multiple submodules that are physically separated from each other, and a unit can have multiple subunits that are physically separated from each other.

[0145] The described method and the described system relate only to the form(s) of embodiment of the invention. The invention may be varied by a person skilled in the art without departing from the scope of the invention, provided it is defined by the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0146] The invention will be explained again in more detail below with reference to the drawings based on exemplary embodiments. The representations in the drawings are schematic and greatly simplified and are also not to absolute true scale.

[0147] In the attached picture:

[0148] Figure 1 A flow chart showing a method according to a first form of embodiment of the invention;

[0149] Figure 2 A diagram showing a system according to a second form of embodiment of the invention;

[0150] Figure 3 shows a system according to a third form of embodiment of the present invention;

[0151] Figure 4 A flow chart showing a method according to a fourth form of embodiment of the invention;

[0152] Figure 5 A diagram showing a system according to a fifth form of embodiment of the present invention;

[0153] Figure 6 shows a system according to a sixth form of embodiment of the present invention;

[0154] Figure 7 A cross-sectional view from a second user's field of view is shown, wherein the field of view is augmented by augmented reality information. DETAILED DESCRIPTION

[0155] Figure 1 A flow chart of a method for outputting augmented reality information AR related to a medical examination of a medical device 2, IT, MD and / or a patient 13 to a first user U1 according to a first form of embodiment is shown. In step R1, first information selected from the group consisting of image information, depth information, coordinate information and a combination thereof is acquired, wherein the first information is related to a medical examination of the medical device 2, IT, MD and / or the patient 13. In step GA, augmented reality information AR related to the medical examination of the medical device 2, IT, MD and / or the patient 13 is created based on the first information. In step D1, the augmented reality information AR is output so that the augmented reality information AR can be perceived in the field of vision of the first user U1.

[0156] Figure 2 A diagram of a system 1 according to a second form of embodiment of the invention is shown. The system 1 is implemented for outputting augmented reality information AR related to a medical device 2, IT, MD and / or a medical examination of a patient 13 to a first user U1, and has a first acquisition module R1-M, a creation module GA-M and a first output module D1-M. The system 1 is particularly implemented for performing a method according to a first form of embodiment of the invention. In particular, the detection R1 of image information can be performed by the first acquisition module R1-M, the creation GA of the augmented reality information AR can be performed by the creation module GA-M, and the output D1 of the augmented reality information AR can be performed by the first output module D1-M.

[0157] Figure 3A system 1 according to a third form of embodiment of the present invention is shown. The system 1 has a first augmented reality device H1. The first augmented reality device H1 is implemented as data glasses. The first augmented reality device H1 has a first camera unit C1, a first eye tracking unit E1, a first viewing device V1 and a first control unit N1. The first control unit N1 is a first computer, in particular a first digital computer, and is implemented for controlling the first augmented reality device H1. The first control unit N1 has a first acquisition module R1-M, a creation module GA-M and a first output module D1-M. The first control unit N1 has a first storage unit M1 and a processor system. The first storage unit M1 is implemented for loading a computer program, wherein when the computer program is executed on the first control unit N1, the steps of the method of the present invention are performed using the computer program. The first acquisition module R1-M, the creation module GA-M and the first output module D1-M are each implemented in the form of software on the processor system of the first control unit N1. Image information and / or depth information can be recorded by the first camera unit C1. Image information and / or depth information can be acquired by data transmission from the first camera unit C1 to the first acquisition module R1-M. The first observation device V1 can access a storage area of ​​the first output module D1-M, wherein augmented reality information can be output by data transmission from the first output module D1-M to the first observation device V1. The first eye tracking unit E1 is implemented to obtain eye tracking of the first user U1. This makes it possible in particular to determine the position and / or orientation of the field of view of the first user U1 relative to the image data set. In this way, the position and / or orientation of a real object (e.g. a patient 13) and / or a virtual object (e.g. a structure marker MI) relative to the field of view of the first user U1 can be determined based on the position and / or orientation relative to the image data set.

[0158] Figure 4 A flow chart of a method for outputting augmented reality information AR related to a medical device 2, IT, MD and / or a medical examination of a patient 13 to a first user U1 according to a fourth form of embodiment is shown. In step D2, the augmented reality information AR is output so that the augmented reality information AR can be perceived in the field of vision of a second user U2. In step R2, the input of the second user U2 is obtained. In this case, the augmented reality information AR is created based on the input of the second user U2.

[0159] Figure 5A diagram of a system 1 according to a fifth form of embodiment of the present invention is shown. The system 1 has a first acquisition module R1-M, a creation module GA-M, a first output module D1-M, a second output module D2-M, and a second acquisition module R2-M. The system 1 is particularly implemented to perform a method according to a fourth form of embodiment of the present invention. In particular, the output D2 of augmented reality information AR can be performed by the second output module D2-M, and the acquisition R2 of the input of the second user U2 can be performed by the second acquisition module R2-M.

[0160] Figure 6 A system 1 according to a sixth form of embodiment of the invention is shown. The system 1 has a medical imaging device 2 having an acquisition area 4 formed by a tunnel-shaped opening. Without limiting the overall inventive idea, a computed tomography device 1 for a medical imaging device 2 is shown by way of example.

[0161] The medical imaging device 2 has a frame 20, an acquisition area 4, a patient support device 10, raw image data acquisition devices 26, 28 and a control device 30. The frame 20 has a fixed carrying frame 21 and a rotating body 24. The rotating body 24 is supported to allow rotation around a rotation axis via a rotating support device. The acquisition area 4 is formed by a tunnel-shaped opening in the frame 20. The area of ​​the object to be imaged (in particular, the patient 13) can be arranged in the acquisition area 4. The patient support device 10 has a support table 11 for supporting the patient 13, wherein a transfer plate 12 is movably arranged on the support table 11 relative to the support table 11, so that the transfer plate 12 can be introduced into the acquisition area 4 in the longitudinal direction of the transfer plate 12.

[0162] The original image data acquisition device 26, 28 is a projection data acquisition device 26, 28 having a radiation source 26 (e.g., an X-ray source) and a detector 28 (e.g., an X-ray detector). The radiation source 26 is arranged on the rotating body 24 and is implemented for emitting radiation, such as X-ray radiation, having X-ray quanta 27. The detector 28 is arranged on the rotating body 24 and is implemented for detecting the X-ray quanta 27. The X-ray quanta 27 can reach the area of ​​the patient 13 to be imaged from the radiation source 26 and can hit the detector 28 after interacting with the area to be imaged. In this way, projection data of the area to be imaged can be acquired. The projection data acquired by the projection data acquisition device 26, 28 is transmitted to the control device 30. The control device 30 is implemented for controlling the medical imaging device 2. The control device 30 has an image reconstruction device 34. The medical imaging data set can be a reconstruction means of the image reconstruction device 34 based on the projection data. The medical imaging device 2 is implemented for providing a medical imaging data set related to the structure SP of the patient 13.

[0163] The medical imaging device 2 is implemented to provide coordinate information. The medical imaging device has a movable part 12. The control device 30 is implemented to output a control command related to the movement of the movable part 12. Without limiting the overall inventive concept, a transfer plate 12 for the movable part 12 is shown. The support table 11 has a drive unit, i.e., a motor, which is implemented to drive the movement of the transfer plate 12. The drive unit is implemented to receive a control command and / or drive the movement of the transfer plate 12 based on the control command. The control device 30 has a coordinate information providing unit 50, an establishment unit 51, and a motion parameter acquisition unit 52. The coordinate information providing unit 50 is implemented to provide coordinate information. The motion parameter acquisition unit 52 is implemented to acquire motion parameters related to the movement of the movable part 12. The establishment unit 51 is implemented to establish coordinate information based on the control command and / or based on the motion parameter.

[0164] The medical imaging device 2 has an input device 38 and an output device 39. The input device 38 is designed to input control information, such as image reconstruction parameters and / or examination parameters. The output device 39 is used to output control information and / or images.

[0165] The system 1 has a first augmented reality device H1 with a first control unit N1 and a second augmented reality device H2 with a second control unit. The second augmented reality device H2 is implemented as data glasses. The structure and function of the second augmented reality device H2 basically correspond to the structure and function of the first augmented reality device. Instead of the first acquisition module R1-M of the first augmented reality device H1, the second augmented reality device H2 has a second acquisition module R2-M. The second control unit has a second output module D2-M, a second camera unit, a second storage unit and a second observation device V2 to replace the corresponding components of the first augmented reality device H1. The creation module GA-M provided by the sixth form of embodiment of the present invention has a first creation submodule and a second creation submodule, the first control unit N1 provided has a first creation submodule and the second control unit provided has a second creation submodule. The first control unit N1, the second control unit and the control device 30 are connected to each other by a data transmission device, in particular wirelessly connected to each other. In particular, data transmission can be realized between the modules of the first control unit N1, the modules of the second control unit and the control device 30 by the data transmission device. The control of the medical imaging device 2 by the first augmented reality device H1 and / or by the second augmented reality device H2 can therefore be optionally realized. In particular, control commands related to the movement of the movable part 12 can be output by the first augmented reality device H1 and / or by the second augmented reality device H2. In particular, the first augmented reality device H1 and / or the second augmented reality device H2 can have a creation unit 51. Optionally, the control device 30 can form a third creation submodule of the creation module GA-M.

[0166] The first augmented reality device H1 is worn by the first user U1. The second augmented reality device H2 is worn by the second user U2. Optionally, the third augmented reality device H3 is worn by the patient 13. With the third augmented reality device H3, the planned sub-steps and the expected results can be explained by showing them to the patient 13, for example, in preparation for the intervention, and / or during imaging in preparation for the intervention, breathing instructions and calming sequences of images can be output.

[0167] First information can be acquired by the first acquisition module R1-M, the first information being selected from the group consisting of image information, depth information, coordinate information and combinations thereof, wherein the first information relates to a medical examination of the medical device 2, IT, MD and / or the patient 13. The first information relates to a spatial region in the field of view of the first user U1. The spatial region particularly comprises an intervention area IA. Without limiting the general idea of ​​the invention, the medical imaging device 2, the examination device MD and the intervention tool IT are shown in each case as examples for medical devices. Without limiting the general idea of ​​the invention, intervention is shown by way of examples for medical examinations. An intervention tool IT, such as a scalpel, is used for intervention. In this case, it is a question of the correct position of the intervention tool IT relative to the structure SP of the patient 13. The intervention is performed by the first user U1. The second user U2 is located outside the examination room, the medical imaging device 2 is arranged in the examination room and the intervention is being performed in the examination room. The system 1 particularly makes it possible for the second user U2 to guide the first user U1 in the correct position of the intervention tool IT relative to the structure SP of the patient 13.

[0168] Figure 7 A cross-sectional view from the field of view of the second user U2 is shown, wherein the field of view of the second user U2 is enhanced with augmented reality information AR. Part of the field of view of the first user U1 is shown in the field of view of the second user U2, so that the second user U2 sees the patient 13, the interventional tool IT and the transfer plate 12 from the perspective of the first user U1. The augmented reality information AR is shown in the field of view of the second user U2, so that the second user U2 sees the augmented reality information AR from the perspective of the first user U1.

[0169] From Figure 7 The shown sectional view of the field of view of the second user U2 therefore differs from the corresponding sectional view from the field of view of the first user U1 in that the patient 13, the patient support device 10, the interventional tool IT are visible to the second user U2 as images and / or as a model, and in that the arm of the second user U2 is visible to the first user U1 as an image and / or a model.

[0170] Based on the first information, examination information is established about the position of the interventional tool IT relative to the structure SP of the patient 13. Augmented reality information AR is created based on the examination information such that the augmented reality information AR has an indicia EI representing a movement of the interventional tool IT to be performed as the next movement, for example in the form of a moving stereoscopic representation and / or a holographic type representation of the interventional tool IT.

[0171] According to a sixth form of embodiment of the invention, the system 1 has an examination device MD. The augmented reality information AR is created based on a medical signal, such as an EKG signal provided by the examination device MD, so that the augmented reality information AR has a signal output field DF, wherein the medical signal is displayed as a timing characteristic in the signal output field DF.

[0172] In preparation for the intervention, the patient 13 is automatically identified by the first augmented reality device H1, in that patient identification information is established based on an image of the patient 13, for example by means of biometrics. Based on the patient identification information, augmented reality information AR is created so that personal patient data and data related to the medical history of the patient 13 will be displayed in the patient data output field PI.

[0173] Patient position information will be established based on the first information. Augmented reality information AR is created based on the patient position information so that the position of the augmented reality information AR relative to the field of view of the first user U1 is determined based on the position of the patient 13. In particular, the augmented reality information AR is shown anatomically correctly overlaid on the patient 13.

[0174] The creation module GA-M is implemented to create augmented reality information AR based on a medical imaging dataset. The medical imaging dataset relates to a structure SP of the patient 13. The structure SP may be, for example, an organ of the patient 13. The augmented reality information AR has a structure marker MI, which marks the structure SP of the patient 13. The structure marker MI is shown as anatomically correctly overlaid on the structure SP. The structure marker MI is designed so that a first partial area of ​​the structure SP can be distinguished from a second partial area of ​​the structure SP by the structure marker MI. Figure 7 , a first partial area of ​​the structure SP is shown as a cross-hatched surface and a second partial area of ​​the structure SP is shown as a white surface. For example, the first partial area may relate to a part of an organ to be removed and / or the second partial area relates to the remaining part of the organ. In this way, the first user U1 and the second user U2 can quickly and accurately identify the first partial area of ​​the structure SP and / or the second partial area of ​​the structure SP.

[0175] The augmented reality information AR has a nominal position marking TP, which relates to a nominal position of the patient 13 on the transfer plate 12. The reproducibility between a plurality of medical imaging data sets recorded after different sub-steps of an intervention is improved.

[0176] The augmented reality information AR has a calibrated position mark TT, which is related to the calibrated position of the intervention tool IT. The calibrated position mark TT is provided so that the second user U2 creates a model of the intervention tool IT through the second augmented reality device H2 and positions the model of the intervention tool IT at the calibrated position of the intervention tool IT. The second augmented reality device H2 is implemented to recognize the gesture of the second user U2 and create the augmented reality information AR based on the gesture.

[0177] The augmented reality information AR has a region marking SR which marks a region of the transfer plate 12 selected according to the selected scheme for imaging by the medical imaging device 2. Therefore, the region marking SR also marks a region of the patient 13 selected for an unchanged position of the patient 13 relative to the transfer plate 12 for imaging by the medical imaging device 2. The structure marking SP, the marked position marking TP, the marked position marking TT and the region marking SR each have a stereoscopic representation and / or a holographic type representation.

[0178] References:

[0179] [CR06] Ozan Cakmakci and Jannick Rolland, “Head-Worn Displays: A Review,” Journal of Display Technology, Volume 2, Issue 3, September 2006.

Claims

1. A method for outputting augmented reality information to a user, the method comprising: - Acquire medical image datasets related to the patient's anatomical structure; - acquiring first information, the first information being selected from the group consisting of image information, depth information, coordinate information, and a combination thereof, wherein the first information relates to a medical device and / or a medical examination; - based on the first information and the medical image dataset, creating the augmented reality information, the augmented reality information relating to the medical device and / or the medical examination and comprising structure markings, the structure markings comprising marking a 3-dimensional (3D) image of the anatomical structure of the patient; - In response to the user placing the patient's anatomical structure in the user's field of view, outputting the augmented reality information, the augmented reality information being overlaid on the patient's anatomical structure in an anatomically correct manner, and the augmented reality information being perceptible in the user's field of view.

2. The method according to claim 1, - wherein the coordinate information is provided by a medical device (2, IT, MD).

3. The method according to claim 1, - wherein control commands related to the movement of a movable part of the medical device (2, IT, MD) are output; - wherein the coordinate information is established based on the control command.

4. The method according to claim 1, - wherein movement parameters related to the movement of a movable part of the medical device (2, IT, MD) are acquired; - wherein said coordinate information is established based on said motion parameters.

5. The method according to claim 1, - wherein examination information (EI) related to a medical examination of the patient (13) is established based on the first information; - wherein the augmented reality information (AR) is created based on the inspection information.

6. The method according to claim 1, - wherein patient information related to the patient (13) is established based on the first information; - wherein the augmented reality information (AR) is created based on the patient information.

7. The method according to claim 6, - wherein the patient information is selected from the group of patient information consisting of the following information and combinations thereof: - patient identification information related to the identification of the patient (13), - patient state information related to the physiological state and / or emotional state of the patient (13), - Patient position information relating to the position of the patient (13).

8. The method according to claim 1, - wherein reference position information relating to a reference position of the patient (13) and / or a reference position of the medical device (2, IT, MD) is provided; - wherein the augmented reality information (AR) is created based on the calibration position information; - wherein the augmented reality information (AR) has a reference position mark marking the reference position of the patient (13) and / or the reference position of the medical device (2, IT, MD).

9. The method according to claim 8, - wherein area information is provided relating to the area of ​​the patient (13) and / or the area of ​​the medical device (2, IT, MD), - wherein the augmented reality information (AR) is created based on the area information; - wherein the augmented reality information (AR) has region markings (SR) marking regions of the patient (13) and / or regions of the medical device (2, IT, MD).

10. The method according to claim 9, - wherein the marking selected from the group of markings consisting of said structure marking (SP), said nominal position marking, said region marking (SR) and a combination thereof has a stereoscopic representation and / or a holographic type representation.

11. The method according to claim 9, wherein the user is a first user, an input of the first user (U1) is obtained; and wherein the augmented reality information (AR) is created based on the input of the first user (U1).

12. The method according to claim 11, wherein: The method comprises the following steps: - outputting the augmented reality information (AR) so that the augmented reality information (AR) can be perceived in the field of vision of the second user (U2); - obtaining input from the second user (U2); - wherein the augmented reality information (AR) is created based on input of the second user (U2).

13. The method according to claim 12, - wherein the input of the first user (U1) is a gesture of the first user (U1), and / or wherein the input of the second user (U2) is a gesture of the second user (U2).

14. The method according to claim 12, - wherein the marked position information and / or the area information is established based on an input of the first user (U1) and / or based on an input of the second user (U2).

15. The method according to claim 12, - wherein a first user profile related to said first user (U1) and / or a second user profile related to said second user (U2) is selected; - wherein said augmented reality information (AR) is created based on said first user profile (U1) and / or based on said second user profile (U2).

16. A system (1) for outputting augmented reality information (AR) to a first user (U1), the system (1) comprising: - at least one memory and at least one processor, the at least one memory comprising computer readable code, which when executed by the at least one processor configures the at least one processor to: - Acquire medical image datasets related to the patient's anatomical structure; - acquiring first information, the first information being selected from the group consisting of image information, depth information, coordinate information, and a combination thereof, wherein the first information relates to a medical device and / or a medical examination; - based on the first information and the medical image dataset, creating the augmented reality information (AR), the augmented reality information relating to the medical device and / or the medical examination and comprising structure markings, the structure markings comprising markings of a 3-dimensional (3D) image of the anatomical structure of the patient; - In response to the first user placing the anatomical structure of the patient in the field of view of the first user, the augmented reality information (AR) is output, the augmented reality information (AR) is overlaid on the anatomical structure of the patient in an anatomically correct manner, and the augmented reality information can be perceived in the field of view of the first user (U1).

17. The system (1) according to claim 16, further comprising a medical device (2, IT, MD).

18. System (1) according to claim 17, - wherein the medical device (2, IT, MD) is implemented to provide the coordinate information.

19. The system (1) according to claim 18, - wherein the medical device (2, IT, MD) has a movable part, and the system (1) also has a control device (30), the control device (30) being configured to: - outputting control commands related to the movement of the movable component; - establishing the coordinate information based on the control command.

20. The system (1) according to claim 16, - wherein the medical device (2, IT, MD) has a movable part, and the system further comprises a control device (30), the control device (30) being configured to: - obtaining movement parameters related to the movement of the movable part; and - establishing said coordinate information based on said motion parameters.

21. The system (1) of claim 16, wherein the at least one processor is further configured to: - outputting the augmented reality information (AR) so that the augmented reality information (AR) can be perceived in the field of vision of the second user (U2); - obtaining input from the second user (U2); - Creating said augmented reality information (AR) based on input of said second user (U2).

22. The system (1) according to claim 21, further comprising a first augmented reality device (H1), wherein the first augmented reality device (H1) comprises a first processor of the at least one processor.

23. The system (1) according to claim 22, further comprising a second augmented reality device (H2), wherein the second augmented reality device (H2) comprises a second processor of the at least one processor.

24. The system (1) of claim 21, wherein the at least one processor is configured to: - selecting at least one of a first user profile associated with a first user (U1) and a second user profile associated with a second user (U2), and - creating said augmented reality information (AR) based on said first user profile (U1) and / or based on said second user profile (U2). 25 . The system ( 1 ) according to claim 16 , wherein the system ( 1 ) is designed to carry out the method according to claim 1 .

26. A computer program product, comprising a computer program, wherein the computer program can be loaded into a storage device of a computer, wherein when the computer program is executed on the computer, the steps of the method according to one of claims 1 to 15 are performed by means of the computer program.

27. A computer-readable medium having a computer program stored thereon, wherein the computer program can be loaded into a storage device of a computer, wherein when the computer program is executed on the computer, the steps of the method according to any one of claims 1 to 15 are performed using the computer program.