Medical device control

Abstract

Method (Z) for controlling a medical imaging device (1) and / or an information display device (7) that displays data (BD) generated by the medical imaging device (1) to a user (13) by means of a user input (EI, AI), wherein the user input (EI, AI) is performed at least on the basis of an eye position and / or movement detection (Y) in combination with a further non-contact user input recognition logic (X), - wherein the user input (EI, AI) includes a selection input, wherein the user (13) views an object and / or region to be selected with the eye (15) and initiates a selection of the viewed object by means of an expression of intent signal, and the selection input is continued by the user (13) moving the selected object to a location he views after initiating the selection by means of a displacement signal, and / or - the user input (EI, AI) based on eye position and / or movement detection (Y) in combination with further non-contact user input recognition logic (X) includes a graphical input of an object.

Description

[0001] The present invention relates to a method for controlling a medical imaging device and / or an information display device that displays data generated by the medical imaging device to a user, by means of user input. It further relates to a control system for controlling a medical imaging device and / or an information display device that displays data generated by the medical imaging device to a user.

[0002] The control of medical imaging devices such as computed tomography (CT) scanners, ultrasound devices, magnetic resonance imaging (MR) scanners, X-ray machines, angiographs, single proton emission computed tomography (SPECT) scanners and positron emission tomography (PET) scanners, etc., is usually carried out by means of touch control, i.e. via input on a keyboard, touch surface, mouse or joystick.

[0003] To perform such a control, a user, i.e. a radiologist or radiology professional, usually has to leave the room in which the relevant medical imaging device is located, or at least turn away from the subject of the examination (usually a human patient) and then make their input while turned away.

[0004] US 2006 / 0 109 237 A1 already shows a system in which additional information about the image can be displayed without contact.

[0005] The object of the present invention is to provide an alternative control option for medical imaging devices or for the aforementioned information display devices, which are preferably easier, simpler, or more convenient for the user (and / or the subject of the examination) to operate.

[0006] This problem is solved by a method according to claim 1 and by a control system according to claim 10.

[0007] According to the invention, in a method of the type mentioned above, the user input is carried out at least on the basis of eye position and / or movement detection in combination with a further non-contact user input recognition logic.

[0008] The invention utilizes, firstly, so-called "eye tracking," a technology that detects the eye position (i.e., the direction of gaze and / or the focus of a human eye) and / or the movement of the eye. This technology is currently used in attention research in advertising as well as in communication with people with severe disabilities. Fixating on points in space is a consciously controllable process, whereas eye movements (saccades) are ballistic and therefore linear, and generally not entirely controllable by intention (cf. Khazaeli, CD: Systemic Design. Hamburg 2005. p. 68).Both the fixation of points and eye movements can now be determined using eye tracking, and both pieces of information can be used for user input recognition—the former as a representation of volitional processes, the latter for verifying such expressions of will by examining subconscious reactions. Eye-tracking devices for computers are offered, for example, by the company Tobii from Danderyd, Sweden. However, other eye-tracking algorithms can also be used within the scope of this invention.

[0009] In addition to eye tracking, user input is now combined with another contactless user input recognition logic. Examples of further contactless user input systems are described below. All contactless user input technologies have in common that a user does not need to come into haptic contact with or be in close proximity to input hardware to input data. Instead, their input is remotely detected by sensor technology. The user can be in various positions (usually in virtually any conceivable spatial position), meaning they are not limited to a single, specific position for input.

[0010] This combination of two non-contact user inputs into a single, combined user input has at least two effects: First, it offers the advantage of redundant systems. It is therefore preferable that user commands are only recognized as such if both user inputs produce a meaningfully consistent overall result. Second, different information from the two user inputs can relate to different facts, movements, and expressions of intent, which, when combined, define the overall picture of the user input. Eye-tracking, for example, offers the advantage of enabling the precise location of a point targeted by the user, such as on a display. Other non-contact user input recognition logics can then query additional information about the targeted location, such as what should be done with an object at that location.

[0011] The method provided is therefore easy to use, can be designed to be very precise and reliable, and also offers the advantage of high security (i.e., low susceptibility to errors) in user input.

[0012] According to the invention, the aforementioned control system comprises a control command generation unit for generating control commands from a user input, which control command generation unit is implemented in such a way that, during operation, it performs the user input at least on the basis of eye position and / or movement detection in combination with a further non-contact user input recognition logic.

[0013] The control system according to the invention is thus configured to carry out the method according to the invention. It can be implemented as a standalone unit or as part of the medical imaging device. Therefore, the invention also relates to a medical imaging device with a recording unit and a control system according to the invention.

[0014] Overall, a large proportion of the components for implementing the control system according to the invention, in particular the control command generation unit, can be implemented wholly or partially in the form of software modules on a processor. Several of the units can also be combined in a common functional unit.

[0015] Interfaces of the control system do not necessarily have to be implemented as hardware components, but can also be implemented as software modules. This is the case, for example, when data can be received from another component already implemented on the same device, such as an image reconstruction device or similar, or when data only needs to be transferred to another component via software. Similarly, interfaces can consist of both hardware and software components, such as a standard hardware interface that is specifically configured by software for the intended application. Furthermore, several interfaces can be combined into a single interface, such as an input-output interface.

[0016] The invention therefore also includes a computer program product that can be directly loaded into a processor of a programmable control system, with program code means to execute all steps of a method according to the invention when the program product is executed on the control system.

[0017] Further particularly advantageous embodiments and developments of the invention also arise from the dependent claims and the following description. The control system can also be further developed into the method according to the respective dependent claims, or vice versa.

[0018] According to a first embodiment of the invention, the further contactless user input recognition logic comprises motion detection of the user's extremities. The extremities include, in particular, the limbs (or parts thereof, especially the fingers) and the user's head. Such motion detection is also known as "motion tracking." Devices for this purpose are marketed, for example, under the name "Leap Motion Controller" by Leap Motion, a company based in San Francisco, USA. However, other motion detection algorithms can also be used within the scope of the invention.

[0019] The combination of the eye-tracking described above with motion-tracking into a kind of "combined gesture" is particularly advantageous because signals of intent can be recognized especially well with motion tracking. A simple, intuitive signal of intent is nodding or shaking the head, but finger movements are not only easily recognized by a motion-tracking system, but also intuitively learnable by a user. This combination therefore offers a particularly high level of control security.

[0020] A second option, which can be used as an alternative or supplement to the first (and also in combination with other, unspecified, contactless user input recognition logics), involves the further integration of contactless user input recognition logic that includes the recognition of acoustic signals, particularly speech signals, from the user. Such acoustic signals can therefore include sounds or noises like those we use in everyday spoken language—for example, sounds of affirmation ("mhh") or negation ("uh-uh")—but they primarily include speech signals that can be recognized as user input using speech recognition algorithms. For example, the company Nuance from Burlington, USA, offers speech recognition software called Dragon Naturally Speaking, which can be used within the framework of this second option.However, other speech recognition algorithms can also be used in principle within the scope of the invention.

[0021] Each variant has its specific advantages. Speech recognition offers the advantage that a user does not need to learn a specific "vocabulary" of movements to make user inputs, but can control the system completely intuitively based on their speech or sounds: The speech recognition algorithm learns the user's vocabulary instead. In contrast, motion recognition has the advantage that patients are not disturbed by the user's (i.e., the practitioner's) speech information during the control (or image playback) of the imaging process, nor do they feel personally addressed.

[0022] It is particularly preferred that user input takes place in the same room as the medical imaging device and / or the information display device. This means that the user enters their inputs in the room where the corresponding device is operating. This allows for direct user interaction with the device (including a quick overview of the effects of the user controls). Therefore, moving from this room to another for user control purposes is neither necessary nor desirable, except for safety reasons, particularly radiation protection.

[0023] Furthermore, a particularly preferred application of the method according to the invention is to perform it during an interventional procedure on a patient. In this context, the interventional procedure is supported by images acquired by the medical imaging device. The advantages of the invention are particularly evident in this specific application, since the immediate proximity of the practitioner to the patient is especially desirable in such an image-assisted procedure: A surgeon or practitioner plans their procedure, such as a needle path for an infusion or interventional needle, based on the images acquired by the imaging device. They can, for example, mark the desired position of the needle (or other interventional instrument) and its desired path through the tissue on the previously acquired images.He wears sterile gloves, which makes controlling the imaging device or the information display connected to it particularly complicated and time-consuming under these circumstances. Previously, the surgeon had to leave the operating room to check the needle position in the tissue, or operate a monitor located in the operating room using a joystick or other touchscreen. This, in turn, meant that the joystick or touchscreen also had to be kept strictly sterile, for example, using sterile films. This, however, limited the ease of use of the controls because they naturally became less responsive.

[0024] However, when using the inventive method in the context of such an interventional procedure, both the intervention planning and the acquisition of further images by the imaging device are significantly easier, as they are contactless yet reliable.

[0025] Below, some particularly preferred applications of the control method according to the invention are explained in more detail. This is not to be understood as limiting, but rather shows particularly advantageous applications of the invention and illustrates by way of example the interplay of purpose, type and form of user input.

[0026] A particularly preferred embodiment relates to a method according to the invention in which the user input comprises a trigger input for initiating an imaging procedure by the medical imaging device. In the context of an image-guided interventional procedure, this means that both the initial image acquisition for generating a first image (for the purpose of procedure planning) and (in particular) further image acquisitions during the procedure can be carried out using the method according to the invention. Thus, during such further image acquisitions during the procedure, the practitioner can, for example, check the position of the needle by triggering a further image acquisition while a needle is being inserted into the body of the patient.Looking at a specific location, such as a monitor or imaging device, can be detected using eye position and / or movement detection, and a gesture—for example, using the free hand while the other hand continues to hold the needle—can be detected using motion tracking. The detection signals based on eye tracking and motion tracking are used to control, i.e., to initiate, image acquisition.

[0027] In one embodiment of the invention, the user input comprises a selection input, wherein the user views an object and / or region to be selected and initiates a selection of the viewed object by means of an expression of intent signal. This embodiment therefore relates primarily to the image display on the relevant information display device. A selection can be generated similarly to a mouse click. This mouse click can be represented by a corresponding finger gesture (such as bending a finger and immediately extending it, or by a tapping motion of the finger – in particular, an index finger), while the position or focus of the eyes indicates where the "click" is to be performed. Examples of applications for such simulated mouse clicks include, for example, selecting an interaction button on a monitor or marking or selecting (image) elements on a monitor display.

[0028] The selection process is further enhanced by the user moving the selected object to a location they are looking at after initiating the selection, using a displacement signal. This enhancement thus involves a kind of "drag" gesture: By looking at an element (for example, a slider) on an information display device, such as a monitor, and performing a selection gesture like the one described above, the drag action can be executed. A subsequent movement of the hand and / or eye up or down, or in any lateral direction—that is, relative to the element's initial position—can be used as a signal to move the element, i.e., the selected object. Such drag gestures can generally be used both for selecting and moving control elements and for performing (area) selections.

[0029] Furthermore, the resulting movement can be terminated, i.e., confirmed, by a movement confirmation signal from the user. This means that the movement process includes a kind of "drag and drop" functionality.

[0030] The shift confirmation signal completes the shift, i.e., it fulfills the "drop" function in the drag & drop process.

[0031] Another embodiment of the invention relates to a method in which the user input comprises a graphical input of an object. In particular, the graphical input can include drawing objects such as straight and / or curved lines, closed and / or open shapes, and much more within an image, which is displayed on the information display device. The example of drawing a needle path for intervention planning has already been mentioned above. Such a "drawing" function in the user input can be understood and implemented similarly to the drag-and-drop function mentioned above. A gesture recognized by motion tracking—for example, of a finger—can be detected as the initiation of a drawing process, and a subsequent movement of the finger, hand, or eyes defines the spatial extent of the entered object. Another gesture—again, for example, of the same finger (and / or a different finger)—can then be used to initiate the drawing process.(of another extremity) - can bring the drawing process to completion.

[0032] Another embodiment relates to a method according to the invention in which the user input comprises forward and / or backward movement and / or upward and / or downward movement and / or scrolling within the displayed data. This type of user input thus includes a kind of navigation within the data, for example, images, displayed by the information display device. For example, scrolling in image acquisition layers can be achieved by moving a (for example, flat) hand up or down, or this scrolling can be performed in DICOM layers using eye tracking.

[0033] Another embodiment relates to a method according to the invention in which the user input includes a confirmation signal that releases previously made user inputs and / or a delete signal that reverses previously made user inputs, particularly those made before a confirmation signal. This type of user input is analogous to pressing an "Enter" or "Delete" key on a computer. It generally serves to trigger a safety signal, i.e., either the final confirmation of a user input or the deletion of a user input. This ensures that no erroneous input is implemented by the user without the user's explicit consent. Furthermore, this makes it possible to correctly time the implementation of the control commands generated by the user input.Within the scope of the invention, which is (at least potentially) based exclusively on contactless user input, such a confirmatory final user input or the provision of a delete function before the control commands are executed offers the advantage of increased process reliability and, above all, increased user confidence in the system. This can therefore increase user acceptance of the novel contactless control system.

[0034] The invention is explained in more detail below with reference to the accompanying figures and exemplary embodiments. The different figures show identical components with identical reference numerals. They show: Fig. 1 a perspective view of an embodiment of an imaging device according to the invention, Fig. 2 a detailed view from Fig. 1, Fig. 3 a schematic block representation of the same imaging device with an embodiment of a control system according to the invention, Fig. 4 a schematic block diagram of an embodiment of the method according to the invention.

[0035] Fig. Figure 1 shows an imaging device 1 according to the invention, here a magnetic resonance imaging (MRI) scanner 1 with a scanning unit 5 into which an examination object (not shown) can be moved on a patient table 3. The imaging device 1 includes an information display device 7 in the form of a monitor 7, on which a user 13, here a treating physician 13, receives image data from an image acquisition by the scanning unit 5. In addition, two contactless input systems 9, 11 are integrated in the area of ​​the monitor 7, namely an eye-tracking system 9 and a motion-tracking system 11. For contactless user input, the physician 13 uses these two input systems 9, 11 and is located in the same room R as the imaging device 1.For example, this allows him to retain direct access to all images and the control of both the recording unit 5 and the display of the monitor 7 during an interventional procedure supported by the image data from the imaging device 1 displayed on the monitor 7.

[0036] In detail in Fig. Section 2 provides a more detailed explanation of user input. The imaging device 1 includes a control system 21 for controlling the imaging device 1 and / or the monitor 7. The monitor is currently displaying cross-sectional images from an image acquisition by the recording unit 5. To navigate the image data and modify or enrich it if necessary – for example, by drawing a desired needle path for an interventional procedure – a combined user input is provided via the eye-tracking system 9 and the motion-tracking system 11. The eye-tracking system 9 detects the position and / or movement of the physician's eye 15 13. The motion-tracking system 11 detects the movement of the physician's finger 19 or hand 17 13. Control commands are derived from the combination of the two motion detections (eye 15 and finger 19 or eye 15 and hand 17), which control the image display on the monitor 7.In the same way, for example, another image acquisition can be initiated by the recording unit 5.

[0037] Fig. Figure 3 shows the imaging device 1 schematically in block representation. It in turn comprises the recording unit 5 and the monitor 7 (where an analog information display device can also be implemented as a separate unit from the imaging device 1) and the control system 21.

[0038] The control system 21 comprises an input interface 25 and an output interface 27. It also includes an eye-tracking system 9 and a second contactless input system 11, which, as mentioned above, is implemented here as a motion-tracking system 11, but could also include, for example, acoustic signal detection instead of motion detection. Furthermore, the control system includes a control command generation unit 33.

[0039] The eye-tracking system 9 comprises a number of input sensors 29 and a first evaluation unit 31; analogously, the second contactless input system 11 comprises a number of input sensors 37 and a second evaluation unit 35. The input sensors 37 of the second contactless input system 11, implemented here as a motion-tracking system 11, are designed as optical sensors 37; in an acoustic signal detection system, they would, for example, comprise acoustic sensors (e.g., a number of microphones).

[0040] The acquisition unit, 5, generates data BD during image acquisition, in particular image data BD of an object under investigation. This data is transferred to the control system 21 via the input interface 25 and there forwarded to the control command generation unit 33.

[0041] Initial user inputs EI in the form of eye movements and / or eye positions EI are recorded by the input sensors 29 and recognized in the first evaluation unit 31. This results in eye recognition data EID, which is fed into the control command generation unit 33. Similarly, second user inputs AI – in this case, movements AI of one or more extremities, namely the finger 19 or the hand 17 – are recorded via the input sensors 37 and recognized in the second evaluation unit 35, resulting in second recognition data AID, in this case, motion recognition data AID, which is also fed into the control command generation unit 33.The control command generation unit 33 derives a combined user input from this and generates a number of control commands SB based on it, which are forwarded via the output interface 27 to the recording unit 5 and / or to the monitor 7 (depending on the type of control commands SB) and control the recording unit 5 and / or the monitor 7.

[0042] Fig. Figure 4 shows the steps of an embodiment of the inventive method Z for controlling a medical imaging device 1 and / or an information display device 7 in block diagram with reference to Fig.3. In a first step, eye position and / or movement detection (Y) is performed, from which the first user inputs (EI) are detected and the corresponding eye recognition data (EID) are generated. In a second step (X), which may occur concurrently or before / after the first step, the second user inputs (AI) are detected and the corresponding motion recognition data (AID) are generated. In a third step (W), which may follow or occur simultaneously with steps Y and X, the control commands (SB) are generated based on a combination of the first and second user inputs (EI and AI).

[0043] Finally, it should be noted once again that the method described in detail above, as well as the devices shown, are merely exemplary embodiments which can be modified in various ways by a person skilled in the art without departing from the scope of the invention. Furthermore, the use of the indefinite articles "a" or "an" does not preclude the possibility that the features in question may be present multiple times.

Claims

[1] Method (Z) for controlling a medical imaging device (1) and / or an information display device (7) that displays data (BD) generated by the medical imaging device (1) to a user (13) by means of a user input (EI, AI), wherein the user input (EI, AI) is performed at least on the basis of an eye position and / or movement detection (Y) in combination with a further non-contact user input recognition logic (X), - wherein the user input (EI, AI) includes a selection input, wherein the user (13) views an object and / or region to be selected with the eye (15) and initiates a selection of the viewed object by means of an expression of intent signal, and the selection input is continued by the user (13) moving the selected object to a location he views after initiating the selection by means of a displacement signal, and / or - the user input (EI, AI) based on eye position and / or movement detection (Y) in combination with further non-contact user input recognition logic (X) includes a graphical input of an object. [2] Method according to claim 1, characterized by , that the further non-contact user input detection logic includes motion detection of the user's extremities (13). [3] Method according to any one of the preceding claims, characterized by , that the further non-contact user input detection logic includes detection of acoustic signals, in particular speech signals, of the user (13). [4] Method according to any one of the preceding claims, characterized by , that the user input (EI, AI) takes place in the same room (R) in which the medical imaging device (1) and / or the information display device (7) is located. [5] Method according to any one of the preceding claims, characterized by that it is performed during an interventional procedure on an object of investigation, which is supported by images (BD) acquired by the medical imaging device (1). [6] Method according to any one of the preceding claims, characterized by , that the user input (EI, AI) includes a trigger input to initiate an imaging procedure by the medical imaging device (1). [7] Method according to any one of the preceding claims, characterized by , that the displacement is terminated by a displacement confirmation signal from the user (13) via the displacement signal. [8] Method according to any one of the preceding claims, characterized by , that the user input (EI, AI) includes forward and / or backward movement and / or upward and / or downward movement and / or scrolling within the displayed data (BD). [9] Method according to any one of the preceding claims, characterized by , that the user input (EI, AI) includes an acknowledgment signal that releases previously made user inputs (EI, AI) and / or a delete signal that undoes previously made user inputs (EI, AI), especially those made before an acknowledgment signal. [10] Control system (21) for controlling a medical imaging device (1) and / or an information display device (7) that displays data (BD) generated by the medical imaging device (1) to a user (13), comprising a control command generation unit (33) for generating control commands (SB) from a user input (EI, AI), wherein the control command generation unit (33) is implemented such that, in operation, it performs the user input (EI, AI) at least on the basis of an eye position and / or movement detection (Y) in combination with a further non-contact user input recognition logic (X), - wherein the user input (EI, AI) includes a selection input, wherein the user (13) views an object to be selected and / or a region to be selected with the eye (15) and initiates a selection of the viewed object by means of an expression of intent signal, and the selection input is continued by the user (13) moving the selected object to a location he views after initiating the selection by means of a displacement signal, and / or - the user input (EI, AI) based on eye position and / or movement detection (Y) in combination with further non-contact user input recognition logic (X) includes a graphical input of an object. [11] Medical imaging device (1) comprising a recording unit (5) and a control system (21) according to claim 10. [12] Computer program product that can be loaded directly into a processor of a programmable control system (21), comprising program code means to execute all steps of a method according to any one of claims 1 to 9 when the program product is executed on the control system (21).

Images