Systems and methods for multimodal image acquisition and visualization - Patents.com

Abstract

To provide a system and a method for multimodal image acquisition and image visualization.SOLUTION: A system 100 for multimodal image acquisition and image visualization comprises an optical unit 11 and an image sensor 12, and includes a microscope system 10 for surgical operation that is designed to acquire a time resolution image signal 13 of a selected imaging visual field 14 of a sample 15. The system 100 further includes an OCT system 20 designed to acquire a time resolution OCT signal 27 of the selected imaging visual field 14, and includes display means 30 designed for time resolution display of image data 31, 32, and a control unit 40. The control unit confirms video image data corresponding to the acquired image signal and presents the video image data in the display means, and confirms the time resolution OCT image corresponding at least to a portion of the presented video image data, on the basis of the acquired OCT signal and presents the OCT image at a position of the portion in the display means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system and method for multimodal image acquisition, in particular for time-resolved image acquisition of a surgical site by acquiring image signals by video and optical coherence tomography (OCT), and for optimized presentation of multimodal image data, in particular volumetric image data. [Background technology]

[0002] The use of technological assistance is an important part of modern medicine. To date, imaging methods and robotic systems are used equally in both surgery and diagnosis, as a matter of course. In this regard, the use of imaging methods allows the presentation and differentiation of various structures within the patient's body, and image data acquired from the patient can be advantageously used in diagnostics as well as in therapeutic and surgical methods.

[0003] For example, image data from a patient not only allows a surgeon to better plan a surgical intervention, but also assists the surgeon in performing the intervention. A robotic visualization system can be used to assist a surgeon in performing a surgical intervention. The system generally includes at least one camera for recording images of the surgical target area, the camera being carried by a stand having an articulated structure. The stand allows the camera to be positioned with respect to the object by translational and / or rotational movements to obtain images of a desired field of view (FOV) of the surgical target area. In this regard, the use of an optical stereo camera allows the acquisition of 3D image data.

[0004] In addition to obtaining surface information from a desired imaging field, for example based on reflected or backscattered visible light, methods have also been developed to obtain depth information from the imaging field. These methods include optical coherence tomography (OCT), which allows three-dimensional microscopic image capture of optically transparent and / or reflective objects and thus recording of volumetric images of biological tissue within the imaging field observed. Optical coherence tomography (OCT) is essentially an interferometric method using broadband light with a short coherence length. Thus, a system for obtaining OCT data includes an interferometer and a broadband light source with a spectral width of more than 1% of the central wavelength.

[0005] The OCT data can be acquired sequentially or in parallel. For example, sequential acquisition of OCT data is performed by splitting a light beam of a low-coherence light source into a sample beam and a reference beam with a beam splitter, sending them through two arms of an interferometer, placing a movable reference mirror in the reference beam path, and placing an object to be examined in the object beam path. The difference in the paths of the object beam and the reference beam, and therefore the measured depth of the sample, can be set by moving the reference mirror. The object beam is scanned in two dimensions over the sample by a mirror in the object beam path, which allows three-dimensional scanning of the sample.

[0006] For the acquisition of such OCT data in the time domain (time domain OCT - TD OCT), the spectral width Δλ of the light source is L C = λ * / Δλ coherence length L C The axial resolution of an OCT system is determined by the coherence length L Cand indicates the ability to resolve objects with a spacing of at least the coherence length along the optical axis. For example, a light source in the near infrared range with a central wavelength of 800 nm and a spectral width of 80 nm has a coherence length of 7 μm, and an OCT system with such a light source therefore has an axial resolution of about 1-10 μm. The lateral resolution of an OCT system is determined by the optical units used in the object beam path, in particular the objective lens that focuses the light onto the object to be examined.

[0007] Sequential acquisition of OCT data is also possible via the frequency domain (frequency domain OCT - FD OCT), where a distinction is generally made between the use of a tunable light source (frequency-scanned light source OCT) and the use of distributed detectors (spectral domain OCT - SD OCT). In frequency-scanned light source OCT, the frequency of the excitation light source, often a laser, is tuned, so that the path difference between the sample beam and the reference beam, and thus the depth of the scanned sample, can be varied without the need for a movable reference mirror. In the case of SD OCT, a broadband light source is likewise used, but the frequency components of the interference signal are separated before detection, for example by an optical grating.

[0008] Slice and volume data of biological tissues can be acquired by OCT, which can significantly increase the information content for the surgeon. As a result, it is desirable to integrate OCT into the surgical microscope in order to acquire both video data from the surface of the desired imaging field and depth and / or slice images of the imaging field. However, the combined presentation of intraoperative OCT images and conventional (3D) images within the surgical microscope is complex and presents new challenges to the surgeon. While surgical systems have had to date been able to provide volumetric OCT images, their recording times are relatively long and, moreover, rendering has been limited to post-processing only for time and resource reasons. Thus, volumetric imaging in real time during surgery is currently not possible.

[0009] However, as OCT technology and the power and speed of graphics processors (GPUs) evolve, faster OCT modalities for intraoperative imaging are becoming available. A drawback, however, is that the real-time presentation of OCT volumetric images thus made possible can overwhelm or distract the surgeon, especially when the OCT and conventional video data are presented on different screens or different screen regions, or when switching between modalities is not easily possible. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] “A practical comparison between Zhang's and Tsai's calibration approaches”,Li et al.,Proceedings of the 29th International Conference on Image and Vision Computing New Zealand,November 2014 Pages 166-171,DOI:10.1145 / 2683405.2683443 Summary of the Invention [Problem to be solved by the invention]

[0011] It is an object of the present invention to overcome or at least mitigate the disadvantages of the prior art and to provide an improved system and an improved method for multimodal image acquisition and image visualization. [Means for solving the problem]

[0012] The object according to the invention is achieved by the subject matter of the independent claims. Preferred developments are the subject matter of the dependent claims.

[0013] A first aspect of the present disclosure relates to multimodal image acquisition and image visualization, in particular to a system for acquiring and visualizing images by means of a medical instrument, such as a surgical microscope.

[0014] The system according to the present disclosure includes in particular a surgical microscope system with an optical unit for detecting light reflected or backscattered from a sample, for example visible light. For example, the optical unit includes an objective lens and an eyepiece, which may further include other components, in particular other lens elements, mirrors, beam splitters and / or others. The surgical microscope system further includes an image sensor designed to acquire a time-resolved image signal of a selected imaging field of the sample (region of interest - ROI). For example, the sample is a surgical site of a patient, in particular the eye in the case of ophthalmic surgery. However, the sample can be any other surgical site, for example brain tissue in the case of brain surgery, tissue in the ENT area in the case of ENT surgery or gums, dental calculus or dental nerve in the case of dental surgery. The sample can also be any other tissue or preparation (in vivo, in vitro or in situ). The imaging field is preferably selectable by the user.

[0015] The system according to the present disclosure further comprises an OCT system. The OCT system preferably comprises a broadband light source, preferably a tunable (frequency-scanned light source) laser, such as a broadband laser, a supercontinuum laser and / or an ultrashort pulsed laser. In this case, at any point in time, the tunable laser may be a narrow-bandwidth light source, but whose center frequency can be selectively varied in time, or may be formed from a number of narrow-bandwidth light sources. However, any other broadband light source may be used, for example a superluminescent diode, for example in FD-OCT. The OCT system preferably further comprises an interferometer, such as a Michelson, Mach-Zehnder or Koster interferometer. The interferometer preferably comprises a beam splitter, a reference beam path and a sample beam path for generating (and superimposing) the sample and reference beams from the light of the broadband light source. Further preferably, the interferometer comprises a means for setting the sample depth to be examined. Depending on the measurement method, this may be a means for generating a path difference (e.g. a movable mirror in the reference beam in the case of SD-OCT), a means for isolating light of a specific path difference (e.g. an optical grating in the case of FD-OCT) or a means for generating light with a specific path difference (e.g. a tunable light source in the case of frequency-scanned light source OCT). The OCT system preferably further comprises a scanning mechanism for scanning the sample beam, in particular the sample beam, two-dimensionally on the sample. The scanning mechanism is preferably a scanning mirror, but other scanning mechanisms, such as optical fiber scanners, prism scanners, Palmer scanners or others, may also be used. The scanning mechanism may be absent in the case of an OCT system configured for full-field OCT. The OCT system is designed to acquire a time-resolved OCT signal of a selected imaging field, and preferably has a detector for this purpose. For example, the detector is a line detector, a two-dimensional detector array, a photodetector, a distributed detector, a CCD detector and / or a CMOS detector.

[0016] The system according to the present disclosure further comprises a display means designed for time-resolved display of the image data. The display means is preferably one or more screens, such as at least one screen of a surgical microscope, a screen fixedly installed in the operating room or a head-mounted display (HMD), such as video glasses. The screen is preferably a 4K and / or 8K compatible screen and / or a 3D screen designed for stereoscopic presentation.

[0017] The system according to the present disclosure further comprises a control unit connected to the surgical microscope system, the OCT system and a display system, in particular for one-way or two-way data transmission. The control unit is designed and configured to ascertain video image data corresponding to the acquired time-resolved image signal. In particular, the time-resolved image signal is a plurality of signals assigned to surface elements of the sample, which are acquired sequentially or simultaneously for a specific scan of the sample surface, which scan is determined by a scanning mechanism and / or an image sensor. Furthermore, the time-resolved image signal has a clock frequency (image refresh rate), which is determined by the scanning mechanism and / or the image sensor. From this image signal, the control unit generates video image data with a raster (resolution) and image refresh rate suitable for presentation on a display means. The control unit is further configured to display the video image data by the display means.

[0018] The control unit is further configured to ascertain a time-resolved OCT image based on the OCT signal. The OCT signal is an interference signal, and the modulation of the envelope of the interference signal encodes the reflective properties of the sample. The scanning mechanism allows the sample to be scanned in two dimensions at a sample depth set by the path difference. The clock frequency (image refresh rate) of the time-resolved OCT signal also results from the scanning mechanism used, the means used to select or generate the path difference, such as an adjustable mirror in the reference beam, an optical grating or a tunable broadband light source upstream of the detector, and the refresh rate of the detector. The control unit determines the time-resolved OCT image based on the OCT signal by calculation, for example by volume rendering, ray tracing and / or ray marching. A person skilled in the art is familiar with various methods of generating a time-resolved OCT image from a time-resolved OCT signal.

[0019] In the system according to the present disclosure, the control unit generates the OCT images such that the generated time-resolved OCT images correspond at least to a portion of the video image data to be presented. Preferably, OCT signals are acquired from the entire imaging field of view and an OCT image of at least a portion of the imaging field of view is created. Also preferably, OCT signals are acquired from a portion of the imaging field of view and an OCT image of at least a portion of that portion of the imaging field of view is created. The local correspondence between the video image data and the OCT image is ensured according to the present invention by either the measurement method or the image processing method used. The control unit is further designed to display the time-resolved OCT image on the display means at the position of that portion of the video image data. Preferably, the video image data and the OCT image of the entire imaging field of view are created and each is presented on the entire display means. Also preferably, an OCT image of a portion of the imaging field of view is generated and displayed on the display means at the position of the video data corresponding to this portion of the imaging field of view. In other words, the video image data and the OCT image corresponding to the same portion of the sample are presented at the same position on the display means. As a result, the system according to the present disclosure allows for a seamless integration of video image data and OCT images on the display means, allowing the user to easily view the multimodal image data, which can be viewed without moving the head or eyes, which has a beneficial effect on the surgeon's concentration, especially in the case of imaging during a surgical intervention.

[0020] In a particularly preferred embodiment of the system according to the present disclosure, the surgical microscope system comprises a stereo camera, which is designed, so to speak, to acquire stereo time-resolved image signals. A stereo camera for this purpose comprises, in particular, a first image sensor for acquiring a time-resolved first image signal of the imaging field and a second image sensor for acquiring a time-resolved second image signal of the imaging field. The first image sensor acquires the first image signal, preferably along a first optical axis, by a first optical unit, in particular a first objective lens. The second image sensor acquires the second image signal, preferably along a second optical axis, by a second optical unit, in particular a second objective lens. The first optical axis and the second optical axis form an angle between them. This angle between the optical axes and the working distance between the sample and the objective lens determine the stereo angle of the stereo image acquisition. According to this embodiment, the display means is designed to present stereo image data, the stereo angle determining the impression of depth during image visualization on the display means.

[0021] Further preferably, the control unit of the system according to the invention is configured to generate a first video image signal corresponding to the first image signal and a second video image data corresponding to the second image signal, each as described above, and to present it stereoscopically on the display means. The display means is thus designed to present the image data stereoscopically. The display means is preferably a display means designed to display the image data in different (e.g. orthogonal) polarizations together with polarized glasses. However, the display means can also be a 3D screen, e.g. a bright field monitor or other. The control unit is further configured to ascertain a time-resolved first OCT image and a time-resolved second OCT image from the acquired time-resolved OCT signal. The OCT system thus offers the possibility of generating stereoscopic image data without the need for acquisition of a stereoscopic signal. According to this preferred embodiment, the first OCT image corresponds at least to a portion of the presented first video image data and the second OCT image corresponds at least to the same portion of the presented second video image data, each as described above. The control unit is further designed to present the first OCT image and the second OCT image stereoscopically on the display means at the position of the portions. This also results in the inventive advantage of an optimized multimodal image visualization as part of the stereoscopic presentation of the image data. The control unit preferably takes into account the stereo angle of the surgical microscope system during the generation of the stereoscopic OCT images. The same depth impression of the stereoscopic video image data and the stereoscopic OCT images is thus advantageously ensured.

[0022] In the system according to the present disclosure, the image signal preferably includes a plurality of first tuples, each of which includes (or represents) a surface element of the sample and at least one grayscale value. In this case, the surface element of the sample is preferably represented by two lateral spatial coordinates (e.g. x and y) and can be interpreted as a sample pixel, for example. In addition to the grayscale value resulting from the final detected intensity, each of the first tuples may further include a color value during the detection of the intensity for different colors, for example by a color filter connected upstream of the image sensor. Furthermore, the OCT signal includes a plurality of second tuples, each of which includes (or represents) a volume element of the sample and a scattering intensity. In this case, the volume element of the sample is preferably represented by three spatial coordinates (e.g. x, y and z) and can be interpreted as a sample voxel, for example. The second tuple may include another value in addition to the scattering intensity. According to this embodiment, the display means comprises a plurality of pixels, and the control unit is configured to ascertain the video image data based on the first tuple and based on the resolution of the display means in such a way that a particular pixel displays a particular surface element of the sample. In other words, the control unit ascertains a first assignment of pixels of the display means to surface elements of the sample. In this case, this assignment is preferably constant over time for a certain setting, although it may also depend on another setting, for example the zoom level of the surgical microscope system. The control unit of this embodiment is further configured to ascertain the OCT image from the second tuple and the resolution of the display means and / or the video image data in such a way that a volume element corresponding to a particular surface element is presented on a particular pixel. In other words, the control unit ascertains a second assignment of pixels of the display means to volume elements of the sample, which corresponds to the ascertained first assignment. Thus, the control unit realizes a spatial matching of pixels of the display means to the image signal (video image data) and the OCT signal (OCT image).

[0023] Further preferably, the control unit is configured to locally match the image signals acquired by the surgical microscope system with the OCT signals acquired by the OCT system based on acquisition parameters of the OCT system and of the surgical microscope system. In this regard, local matching of these signals refers to a correct association of these signals with a reference coordinate system, for example the coordinate system of the patient during the intervention, which allows a unique mapping of coordinates in the patient space to corresponding coordinates in the signal space. Matching of the signals preferably requires a calibration of the surgical microscope system and of the OCT system. The acquisition parameters of the surgical microscope system preferably comprise the calibration parameters of the surgical microscope system and / or the optical settings, for example the focal length and / or the zoom level of the optical system (camera) used. Furthermore, the acquisition parameters preferably also comprise a set of intrinsic parameters of the surgical microscope system. In this case, the intrinsic parameters determine the relationship between the coordinate system of the image signals and the coordinate system of the associated image sensor. In this case, the type of intrinsic parameters depends in particular on the type of imaging sensor used, which in this case refers to both the actual sensor and the utilized optical unit. The intrinsic parameters include, for example, in the case of Tsai's camera calibration known to a person skilled in the art, the effective focal length, the coordinates of the image principal point (center of distortion) of the image signal, a first scale factor and / or a first radial lens error coefficient (distortion coefficient). Instead of the above-mentioned intrinsic parameters of the Tsai camera calibration, other intrinsic parameters may also be used, for example Zhang's camera calibration (see, for example, (Non-Patent Document 1)). The OCT system likewise has acquisition parameters, which preferably include calibration parameters. The acquisition parameters of the OCT system preferably also take into account the scanning mechanism and / or the detector of the OCT system. Using local matching based on the acquisition parameters, it is advantageously possible to correctly represent a patient's structure at a given coordinate in the patient space in the corresponding coordinates in the image space of the video image data and in the image space of the OCT image.

[0024] In a likewise preferred embodiment of the system according to the present disclosure, the control unit is alternatively or additionally designed to match the video image data and the OCT image locally by image analysis. In this case, local matching of these image data refers to a correct association of these images in a common image coordinate system. Matching based on image data allows a relative association of the image data for an image representation of the same structures as congruent as possible as a result. For example, boundaries of structures or tissues can be recognized in the video image data and in the OCT image by image analysis, for example edge detection or otherwise, and compared with each other. These structures can be superimposed on each other on the display by translation, rotation and / or scaling. The local matching of the image data is preferably performed in addition to the local matching of the acquired signals.

[0025] The control unit of the system is preferably configured to present the video image data and the time-resolved OCT image simultaneously on the display means. This allows, particularly advantageously, the user to consider both image modalities simultaneously. Nevertheless, in order to be able to distinguish between the different image data, the control unit is preferably further configured to present the video image data at a first transparency level and the time-resolved OCT image at a second transparency level. In this case, the first and second transparency levels are preferably different from each other. The first and second transparency levels are likewise time-varying. For example, the image signal is initially presented with 0% transparency, whereas the OCT image is presented with 100% transparency. Over time, the transparency of the image signal is continuously adjusted from 0% to 100%, whereas the transparency of the OCT image is simultaneously adjusted from 100% to 0%. This consequently ensures a continuous transition between the presentation of the video image and the OCT image.

[0026] In a similarly preferred embodiment of the system according to the present disclosure, the control unit is configured to sequentially present the video image data and the time-resolved OCT image on the display means, in other words only one of the video image data and the OCT image at a time is presented at least at a specific location on the display means, which advantageously allows a clear distinction between the video image data and the OCT data, as well as advantageously allowing different image data to be presented simultaneously at different locations on the display means.

[0027] Particularly preferably, the control unit is configured to present the video image data and the time-resolved OCT image at the same magnification, the same viewpoint and / or the same stereo angle. If both video image data are presented simultaneously at the same position on the display means, this may advantageously allow a perfect superimposition of both image data. In the case of sequential presentation, preferably the presentation at the same magnification, the same viewpoint and / or the same stereo angle is performed when transitioning between the locally matched image data. As a result, a fluid transition occurs between the presentation of the image data. In the case of video image data, only a surface presentation (top view) is possible. This corresponds, for example, to an "en face" OCT image. As soon as the transition from the video image data to the OCT image has been made, the user can further preferably adjust the magnification, the viewpoint and / or the stereo angle to advantageously obtain an improved view with optimal depth perception. In particular, the user can switch from an initial "en face" view to a perspective representation or a slice view (OCT B-scan), for example by successively transitioning the viewing direction from the perspective presentation. As a result, advantageously, a perfect view in the various phases is ensured.

[0028] In the system according to the present disclosure, the time-resolved OCT image preferably corresponds to a part of the presented video image data. In other words, the OCT image presents only a part of the imaging field observed by the surgical microscope system. For example, a time-resolved OCT of only a part of the imaging field observed of the sample can be acquired. According to this preferred embodiment, the control unit is configured to present the OCT image on that part and to present the video image data on the remaining part of the display means. For example, the video image data and the OCT image are displayed simultaneously in their entirety on the display means, but sequentially at certain positions of the display means. This advantageously allows the display of the entire sample or the entire surgical field by the video image data, while at the same time parts or details, for example the intervention area of ​​the surgical instrument, are presented by the OCT image. Since the OCT image also allows a fluoroscopic presentation, this embodiment advantageously allows the combination of a top view in the video image data with a fluoroscopic OCT view.

[0029] In a particularly preferred embodiment, the system according to the present disclosure further comprises an interface for obtaining a user input. The interface is preferably a hand switch or a foot switch. Also preferably, the interface is a means for recognizing head movements and / or eye movements, for example integrated into video glasses or a head mounted display HMD. The interface may further be designed to capture voice commands and may therefore comprise at least one microphone. Also preferably, the interface is a keyboard, a joystick, a mouse, a touch screen or a combination thereof. According to this embodiment, the control unit is configured to initiate a switch of presentation from video image data to OCT images or from OCT images to video image data based on the obtained user input.

[0030] In an equally particularly preferred embodiment of the system according to the present disclosure, it may further comprise a medical instrument, for example a probe, a pointer, a tweezers, a puncture needle, a phaco tip, an endoscope, an endoscopic LED or the like.

[0031] According to this embodiment, the control unit is further configured to determine the position, type and / or status of the medical instrument. In this case, a type of medical instrument may preferably be implemented based on an interface for connecting the medical instrument and / or based on an input through a user interface. Also preferably, the type of medical instrument introduced into the imaging field of the surgical microscope system is identified by image analysis of the video image data, for example by segmentation and object recognition. The position of the medical instrument is preferably ascertained based on detection of a marker, which may be a label or a structure on the medical instrument. The marker is preferably detected by the surgical microscope system, optionally using additional light sources (e.g. infrared LEDs) and / or after matching / calibration of the medical instrument (e.g. by positioning the tip of the medical instrument at a predetermined position).

[0032] The state of the medical instrument introduced into the imaging field of the surgical microscope system is likewise preferably determined based on image analysis of the video image data. For example, whether the forceps are open or closed can be identified based on the image data. Furthermore, a user input for changing the state can be read by the control unit, e.g. a user input for activating a phaco tip indicating the change of state. Furthermore, a sensor attached to the medical instrument can detect the change of state, e.g. the forceps being closed, and can send a corresponding sensor signal to the control unit.

[0033] According to this preferred embodiment, the control unit is further configured to initiate a switch of presentation from video image data to time resolved OCT images or from time resolved OCT images to video image data based on the position, type and / or state of the medical instrument. A change of state of a medical instrument of a particular type and / or in a particular position preferably indicates a particular phase of operation. Recognizing the position, type and / or state of the medical instrument may then be used to select an optimal form of presentation for this phase. This includes not only switching image data, but also optionally changing presentation parameters within the image data, such as zoom level, stereo angle, viewing direction, presented depth and / or cutting direction.

[0034] Also preferably, the control unit is configured to check the zoom level of the surgical microscope system and to initiate a switch between the presentation of the video image data and the time-resolved OCT image based on the zoom level. As a rule, a small zoom value corresponds to a large imaging field of view, and the presentation of details is generally not desired. Therefore, preferably, based on the small zoom level, it is concluded that the presentation of the OCT image is not desired. On the other hand, a large zoom value corresponds to a small observation field of view, and the presentation of details is often desired. Therefore, preferably, based on the large zoom level, it is concluded that the presentation of the OCT image is desired.

[0035] Further preferably, the control unit is configured to ascertain the phase of the operation being performed. For this purpose, the control unit is preferably connected to a memory in which a trained learning algorithm, for example a neural network (CNN) or other, is stored. This algorithm is preferably trained with a plurality of video image data and / or OCT images or image signals and / or OCT signals, which have been assigned as classifications to the corresponding operation phases during training. Thus, the trained algorithm can independently recognize the phase of the operation as a classification based on the video image data and / or OCT images or image signals and / or OCT signals. The control unit according to this embodiment is preferably designed to initiate a switch of presentation from the video image data to the OCT images or from the OCT images to the video image data based on the ascertained operation phase. Preferably, image data suitable for the different operation phases are stored in the memory.

[0036] For cataract surgery, the surgical phases may include, for example, resting state, incision, insertion of ophthalmic viscosurgical device (OVD), capsulotomy, hydrodissection, phacoemulsification, rinsing / aspiration, implantation of intraocular lens, incision closure / wetting, and abortion. For refractive surgery, the surgical phases may include, for example, idling, docking, applanation, application of eye / CG rotation, phacotomy, phacoside cut, flap cut, flapside cut, exposure of eye, transition to surgical microscope, positioning of surgical microscope, incision opening, plane definition, flap bed cutting, lens bed resection, lens extraction and / or inspection, wiping, rinsing, slit lamp, speculum removal. For dental interventions, the surgical phases may include, for example, access, pulp removal, debridement, drying, root canal filling, and prosthetics. It should be noted that all or only some of these phases may be part of the corresponding surgery, other surgery phases may also be present, and / or some phases may be omitted.

[0037] The functions of the control unit according to the invention may be implemented by electric or electronic devices or components (hardware), by firmware (ASIC) and / or by executing suitable programs (software). Preferably, the functions of the control unit according to the invention are realized or implemented by a combination of hardware, firmware and / or software. For example, the individual components of the control unit according to the invention are separate integrated circuits or are located on a common integrated circuit.

[0038] The individual functions of the control unit according to the invention are more preferably in the form of one or more processes executed on one or more processors in one or more electronic computers and generated when executing one or more computer programs. In this case, the control unit is designed to cooperate with other components, in particular the surgical microscope system, the OCT system and the display means, to realize the functions of the system according to the invention described herein. Furthermore, it is clear to the skilled person that the functions of several computers (data processing devices, control units, controllers) can be combined or combined in one device or the functions of a certain data processing device can be distributed among several devices to realize the functions of the control unit according to the invention.

[0039] In a particularly preferred embodiment of the system according to the invention, it is integrated into a surgical microscope. In this case, the surgical microscope preferably comprises a surgical microscope system and an OCT system, each as described above. Further preferably, the surgical microscope comprises or is connected to a display means. The image sensor and the optical unit are preferably part of a camera, in particular a main observer camera or a surround camera of the surgical microscope. A control unit of the surgical microscope is preferably designed as a control unit of the system according to the invention, in particular designed to execute the method according to the invention on the basis of commands stored in a memory unit of the surgical microscope, as described below.

[0040] Within the scope of this disclosure, a surgical microscope is understood to mean in the broadest sense a microscope suitable for use during surgery. The surgical microscope preferably comprises a mount that allows imaging of the surgical site independent of the surgeon's head movements. More preferably, the surgical microscope comprises at least one beam splitter and at least two eyepieces. Alternatively, the surgical microscope is a pure "digiscopy" without eyepieces. Also preferably, the surgical microscope comprises at least one imaging sensor. More preferably, the surgical microscope comprises a main observer camera and a surround camera. The surgical microscope may comprise kinematic or robotic assistance means for performing the surgical intervention. Alternatively, the surgical microscope may be called a medical engineering microscope, a medically approved microscope or a medical microscope.

[0041] Another aspect of the present disclosure relates to a method of multimodal image acquisition and image visualization, which includes the method steps of acquiring by a surgical microscope system a time-resolved image signal of a selected imaging field of a sample, where the image signal comprises a plurality of first tuples each representing a surface element of the sample and at least one grayscale value corresponding to the surface element. The method further includes acquiring by an OCT system a time-resolved OCT signal of the selected imaging field of view, where the OCT signal comprises a plurality of second tuples each representing a volume element of the sample and a scattering intensity corresponding to the volume element. The method further includes a step of ascertaining video image data, where the video image data is ascertained in such a way that pixels identified in the video image data are assigned to specific surface elements of the sample based on the first tuple and based on the resolution of the display means having a plurality of pixels. The method further includes a step of ascertaining the time-resolved OCT image based on the second tuple and based on the resolution of the display means and / or the video image data. In the OCT image, a volume element corresponding to a specific surface element is now assigned to a specific pixel. As a result, the OCT image corresponds to at least a portion of the video image data. The method further comprises the steps of presenting the video image data on the display means and presenting the OCT image on the display means, specifically at the location of the portion of the display means. The method according to the present disclosure achieves the same advantages as the system according to the present disclosure, and in this regard reference is made to the above description.

[0042] In a preferred implementation of the method according to the present disclosure, this further comprises a step of acquiring a time-resolved first image signal of the imaging field and a time-resolved second image signal of the imaging field. This acquisition is preferably performed by a stereo camera or other camera designed to acquire stereoscopic image signals. A preferred implementation of the method further comprises a step of ascertaining a first video image data corresponding to the first image signal and a step of ascertaining a second video image data corresponding to the second image signal, i.e. these are stereoscopic video image data. The step of ascertaining a time-resolved first OCT image corresponding at least to a portion of the first video image data is performed based on the OCT signal, in particular based on the second tuple and based on the resolution of the display means and / or the video image data. Furthermore, the step of ascertaining a time-resolved second OCT image corresponding at least to that portion of the second video image data is performed based on the OCT signal, in particular based on the second tuple and based on the resolution of the display means and / or the video image data. According to this implementation, the stereoscopic presentation of the first video image data, the second video image data, the first time-resolved OCT image and the second time-resolved OCT image on the display means is further performed with the same magnification, the same viewpoint and / or the same stereo angle. The method according to this implementation advantageously allows a stereoscopic simultaneous or sequential presentation of the video image data and the OCT image at the same position on the display means.

[0043] Further preferred implementations of the method according to the present disclosure correspond to the further preferred embodiments of the system according to the present disclosure and achieve the same advantages as these embodiments.

[0044] Another aspect of the present disclosure relates to a computer program comprising commands, preferably when executed by a control unit of a surgical microscope as described above, causing a system as described above or a surgical microscope to execute a method according to the invention as described above. The computer program preferably comprises commands, preferably when executed by a control unit of a surgical microscope as described above, causing a system as described above or a surgical microscope to execute a method according to the invention according to one of the preferred implementations as described above. In this case, the computer program according to the invention is preferably stored in a volatile memory, for example a RAM element, or a non-volatile storage medium, for example a CD-ROM, a flash memory or otherwise.

[0045] Further preferred embodiments of the invention will become apparent from the other features set forth in the dependent claims and from the drawings as follows: The various embodiments of the invention described in this application can be advantageously combined with one another, unless otherwise specified.

[0046] The present invention will now be described in exemplary embodiments with reference to the associated drawings, in which: [Brief description of the drawings]

[0047] [Figure 1] 1 shows a schematic diagram of a system according to a first embodiment; [Diagram 2] FIG. 2 shows a schematic diagram of a system according to a second embodiment. [Diagram 3] 4 shows a sample and a schematic diagram of a display means according to a third embodiment; [Figure 4] 4 shows a schematic diagram of a display means according to a fourth embodiment; [Diagram 5] FIG. 13 shows a schematic diagram of a display means according to a fifth embodiment. [Figure 6] 13 shows a schematic diagram of a display means and a medical instrument according to a fifth embodiment; [Figure 7] 1 shows a schematic flow chart of a method according to an implementation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0048] FIG. 1 shows a schematic diagram of a system 100 for multimodal image acquisition and image visualization according to a first embodiment.

[0049] The system 100 comprises a surgical microscope system 10 with an optical unit 11 and a camera 19 having an image sensor 12. The camera is designed to acquire a time-resolved image signal 13 of a selected imaging field 14 of a sample 15. The camera 19 is for example a main observer camera of the surgical microscope which may additionally comprise a surround camera. Likewise, the surgical microscope may comprise an additional beam path generated by a beam splitter 18, allowing observation through an eyepiece 17 by an observer's eye 16. The sample 15 is in particular a surgical site of a patient.

[0050] The system 100 further includes an OCT system 20 with a broadband light source 21, e.g. a superluminescent diode. Light from the light source 21 is fed to an interferometer including a movable mirror 22 and a beam splitter 23. The light is split into a sample beam 25 and a reference beam 28 in the interferometer. The sample beam 25 is scanned on the sample 15 by a scanning mirror 24, so that at least a part of the imaging field 14 of the sample 15 is scanned. The reference beam 28 is directed to the movable mirror 22, which reflects it again to the beam splitter 23. The sample beam 25 interacts with the sample 15, in particular with the volume of the sample 15, and thereby backscatters to the scanning mirror 24, which directs the beam to the beam splitter 23. The backscattered sample beam 25 and the reflected reference beam 28 are superimposed there, and the path difference between the superimposed beams 25, 28 is set by the movable mirror 22. The interference pattern 29 thus produced is captured by a detector 26, for example a CCD detector or a CMOS detector.

[0051] The thus obtained time-resolved OCT signal 27 is transmitted from the detector 26 to a control unit 40, which also receives the time-resolved image signal 13 from the camera 19. The control unit 40 identifies video image data 31 corresponding to the acquired image signal 13 and an OCT image 32 corresponding to the acquired time-resolved OCT signal 27 and corresponding at least to a portion of the video image data 31. The control unit 40 is configured to present the video image data 31 on a display means 30 designed for a time-resolved display of the image data 31, 32. The control unit is further designed to display the OCT image 32 on the display means 30 at the position of at least a corresponding portion 60 of the video image data 31. For illustration, please refer to the image of FIG. 3.

[0052] 3 shows a schematic diagram of a sample 15 and a display means 30 according to a third embodiment. In this case, the sample 15 comprises a number of surface elements 151 and a number of volume elements 152, with a particular volume element 152 corresponding to a particular surface element 151. An exemplary selection of the surface element 151 is depicted with hatching, the number of volume elements 152 is depicted with dashed lines, and four volume elements 152 corresponding to the hatched surface element 151 are depicted with solid lines. Furthermore, double-headed arrows connect these volume elements 152 with the associated surface element 151.

[0053] The surface of the sample 15 can be captured by means of the surgical microscope system 10 of the system 100, in particular by long-wavelength (visible) light reflected or backscattered from said surface being projected again via an optical unit 11 onto an image sensor 12 of a camera. The image signal 13 acquired by the image sensor 12 comprises a number of first tuples 133, the number of which depends for example on the resolution of the image sensor 12, where each of the first tuples 133 corresponds to one of the presented surface elements 151 and represents a greyscale value g corresponding to the intensity of the light projected again onto the image sensor 12. i Furthermore, each of the first tuples 133 has two lateral spatial coordinates x i , yi In the illustrated example, the first tuple 133 is assigned the horizontal spatial coordinate x 1 , y 1 and the grayscale value g 1 has.

[0054] The OCT system 20 of the system 100 can be used to obtain a particular volume of the sample 15, by means of which the short wavelength light of the sample beam 25 scattered thereby is superimposed by an interferometer on the reference beam 28 via the scanning mirror. The interference pattern thus produced, which is captured by the detector 26 as a time-resolved OCT signal 27, comprises a number of second tuples 271, the number of which depends, for example, on the number of points of the sample 15 scanned with the scanning mirror 24. In this case, each of the second tuples 271 corresponds to one of the presented volume elements 152 and produces a scattering intensity value s i Furthermore, each of the second tuples 271 has a value of three spatial coordinates x i , y i , z i In the illustrated example, the second tuple 271 is assigned the spatial coordinate x 1 , y 1 , z 1 and the scattering intensity value s 1 has.

[0055] The display means 30, also shown in Fig. 3, comprises a number of pixels 33, in particular 42 pixels, with 7 pixels in the horizontal direction and 6 pixels in the vertical direction. In the illustrated example, the resolution of the image sensor 12 has resulted in the acquisition of the sample surface with image signals 13 in 21 surface elements 151, with 7 surface elements 151 in the horizontal direction and 3 surface elements 151 in the vertical direction. Thus, in the video image data 31, one pixel 33 corresponds to one surface element 151 in the horizontal direction and two pixels 33 correspond to one surface element 151 in the vertical direction. The pixels 33 corresponding to the surface elements 151 depicted with hatching are likewise depicted with hatching, the allocation being further indicated by arrows. As also depicted by the arrows, the control unit 40 of the system 100 also generates an associated OCT image 32 of the volume element 152 corresponding to the surface element 151 and renders the OCT image of each pixel 33 such that the video image data 31 of a particular surface element 151 is rendered on the particular pixel 33 and the OCT image 32 of the volume element 152 corresponding to the surface element 151 is likewise rendered on the particular pixel 33. As a result, the corresponding video image data 31 and OCT image 32 are rendered in the same position on the display means 30.

[0056] 2 shows a schematic diagram of a system 100 for multimodal image acquisition and visualization according to a second embodiment. The same components are indicated with the same reference numerals as in FIG. 1, and repeated descriptions of these components are omitted for the sake of brevity.

[0057] The difference between the system 100 of FIG. 2 and that of FIG. 1 is that the surgical microscope system 10 includes a first camera 191 with a first optical unit 111 and a first image sensor 121, and a second camera 192 with a second optical unit 112 and a second image sensor 122. Each of these cameras 191, 192 can be used to acquire an imaging field 14 of the sample 15 along an optical axis, so that a stereoscopic image signal is acquired, which comprises a first image signal 131 and a second image signal 132. The stereo angle α between the optical axes OA1, OA2 of the two cameras defines the depth impression or depth perception of the stereoscopic video image data 31 created on the basis of the stereoscopic image signals 131, 132. In this case, the stereoscopic video image data 311, 312 includes a first video image data 311 corresponding to the first image signal 131 and a second video image data 312 corresponding to the second image signal 132. In addition to the spacing of the cameras, the stereo angle α depends on the working distance between the cameras and the sample 15.

[0058] In the system 100 of Fig. 2, the time-resolved OCT signal 27 is acquired in the same manner as described with respect to Fig. 1. However, the control unit 40 uses the OCT signal 27 to identify a first OCT image 321 corresponding to the first video image data 311 and a second OCT image 322 corresponding to the second video image data 312. In this case, the first OCT image 321 and the second OCT image 322 are generated, for example, as if they were acquired at the same stereo angle α as the first and second image signals 131, 132. However, the stereo angle of the OCT images 321, 322 can also be set differently based on a user input acquired by the interface 50 as well. The first and second video image data 311, 312 are presented simultaneously or sequentially with the first and second OCT images 321, 322 on the display means 30, and the change between the OCT images 321, 322 and the video image data 311, 312 is made, for example, based on an input by the interface 50.

[0059] A schematic diagram of the display means 30 according to a fourth embodiment is shown in Fig. 4. Fig. 4(A) shows the presentation of video image data corresponding to the sample 15 on the pixels 33 of the display means 30 as described with reference to Fig. 3. In particular, from a user input made by the interface 50, for example a foot switch, the control unit 40 infers that it is desired to switch from the video image data 31 of the sample 15 to an OCT image 32. In this process, the control unit 40 successively increases the transparency of the video image data 31, while at the same time superimposing an OCT image 32 of decreasing transparency at the position of the video image data 31 on the display means 30. Intermediate stages of the simultaneous superimposed presentation of the semi-transparent video image data 31 and the semi-transparent OCT image 32 are shown in Fig. 4(B). Finally, Fig. 4(C) therefore shows only the OCT image 32 with 0% transparency, i.e. with full coverage.

[0060] A schematic diagram of the display means 30 according to the fifth embodiment is shown in Fig. 5. Fig. 5(A) shows the presentation of video image data corresponding to the samples on the pixels 33 of the display means 30 described with reference to Fig. 3. In particular, from a user input made by the interface 50, for example a sensor detecting head and eye movements, the control unit 40 infers that it is desired to switch from the video image data 31 of the sample 15 to an OCT image 32 at a certain position of the presentation. The control unit 40 consequently identifies the OCT image 32 corresponding to the portion 60 of the video image data 31 and displays the OCT image 32 at the position of the portion 60 of the display means 30. In this process, the OCT image 32 is first presented in an "en face" presentation to ensure a seamless transition to the video image data 31 captured in a top view. From another user input made by the interface 50, the control unit 40 then infers that a perspective presentation of the OCT image 32 of the portion 60 is desired, and therefore changes the viewing direction of the OCT image 32 so that it is presented as a perspective volumetric image, as shown in Fig. 4(C). No further video image data 31 is therefore displayed at the location of the OCT image 32, in this respect the display of the OCT image 32 and the video image data 31 is sequential. However, the video image data 31 continues to be displayed in the area of ​​the display means 30 surrounding the OCT image 32, in this respect the display of the image data 31, 32 is simultaneous.

[0061] A schematic diagram of the display means 30 according to a sixth embodiment is shown in Fig. 6. In this case, the system 100 further comprises a medical instrument 70, in particular tweezers 70, as shown in Fig. 2. In the phase in which the tweezers 70 are brought closer to the sample 15, as shown in Fig. 6(A), the open tweezers 70 and the sample 15 are presented on the display means 30 in the form of video image data 31 as described with reference to Fig. 3. The control unit 40 determines, based on the sensor values ​​and / or based on the video image data 31, whether the tweezers 70 are closed and whether the vertical distance between the tips of the tweezers 70 and the surface of the sample 15 exceeds a predefined limit value Δz 1Upon detecting that the tweezers 70 is not yet fully assembled, the control unit 40 changes the presentation of the sample 15 and optionally the tweezers 70 on the display means 30 from the video image data 31 to the OCT image 32 as shown in FIG. 6(B). In the illustrated example, the tweezers 70 are used to lift or remove a portion of the surface of the sample 15. The control unit 40 determines, based on the sensor readings and / or based on the video image data 31, that the tweezers 70 are closed and the vertical distance between the tip of the tweezers 70 and the surface of the sample 15 remote from the lifted or removed surface of the sample 15 (e.g., spaced laterally by a minimum value) is less than a predetermined limit value Δz 2 As soon as this is recognised, the control unit 40 additionally changes the observation direction or point of view of the presentation of the OCT image 32, so that it is therefore displayed in a lateral perspective as a volumetric image on the display means 30, as shown in Fig. 6(C). In this way, the interaction between the surgeon and the surgical microscope is improved continuously and objectively by advantageous presentation of the acquired signals, in particular the image signal 13 and the OCT signal 27, through the free selection of the locally collated presented image data, i.e. the video image data 31 or the OCT image 32, and / or through the selection of the presentation method, e.g. sequential, simultaneous and / or fluoroscopic.

[0062] 7 shows a schematic flow chart of a method according to an implementation. The method comprises a first method step S100 of acquiring a time-resolved image signal 13 of a selected imaging field 14 of a sample 15 by a surgical microscope system 10. In this case, the image signal 13 comprises a plurality of first tuples 133 each representing a surface element 151 of the sample 15 and at least one grayscale value corresponding to said surface element 151. In a second method step S200, a time-resolved OCT signal 27 of the selected imaging field 14 is acquired by an OCT system 20. In this case, the OCT signal 27 comprises a plurality of second tuples 271 each representing a volume element 152 of the sample 15 and a scattering intensity corresponding to said volume element 152. In a step S300, the video image data 31 is ascertained based on the first tuples 133 and based on the resolution of a display means 30 having a plurality of pixels 33, and a particular pixel 33 in the video image data 31 is assigned to a particular surface element 151. In step S400, the time resolved OCT image 32 is ascertained based on the second tuple 271 and based on the resolution of the display means 30 and / or the video image data 31. This is illustrated in Fig. 7 by two arrows pointing to step S400, but only one of the arrows needs to be realized. In this case, a volume element 152 corresponding to a particular surface element 151 is assigned to a particular pixel 33, so that the OCT image 32 corresponds at least to a portion 60 of the video image data 31. Finally, in step S500, the video image data 31 is presented on the display means 30 and the OCT image 32 is presented at the position of the portion 60 of the display means 30. [Explanation of symbols]

[0063] 10. Surgical Microscope System 11 Optical unit 111 First Optical Unit 112 Second Optical Unit 12 Image Sensor 121 First Image Sensor 122 Second Image Sensor 13 Time-resolved image signal 131 First Image Signal 132 Second Image Signal 133 First Tuple 14 Imaging field 15 Samples 151 Surface Elements 152 Volume Elements 16 The Eye of the Observer 17 Eyepiece 18 Beam splitter 19 Camera 191 First Camera 192 Second Camera OA1 First optical axis OA2 Second Optical Axis 20 OCT System 21 Broadband Light Source 22 Movable mirror (interferometer) 23 Beam splitter (interferometer) 24 Scanning mechanism (scanning mirror) 25 Sample beam 26 Detector 27 Time-resolved OCT signal 271 Second Tuple 28 Reference Beam 29 Interference Signals 30 Display means 31 Video image data 311 First video image data 312 Second video image data 32 OCT images 321 First OCT Image 322 Second OCT image 33 pixels 40 Control Unit 50 User Interface 60 portions 70 Medical Equipment

Claims

1. 1. A method for multimodal image acquisition and visualization, comprising: A method step (S100) of acquiring a time-resolved image signal (13) of a selected imaging field (14) of a sample (15) by means of a surgical microscope system (10), said image signal (13) comprising a plurality of first tuples (133) each representing a surface element (151) of said sample (15) and at least one greyscale value corresponding to said surface element (151); a method step (S200) of acquiring a time-resolved OCT signal (27) of the selected field of view (14) by an OCT system (20), the OCT signal (27) comprising a plurality of second tuples (271) each representing a volume element (152) of the sample (15) and a scattering intensity corresponding to the volume element (152); A method step (S300) of identifying video image data (31) based on said first tuple (133) and on a resolution of a display means (30) having a plurality of pixels (33), whereby a particular pixel (33) in said video image data (31) is assigned to a particular surface element (151), a method step (S400) of identifying a time-resolved OCT image (32) based on the second tuple (271) and on the resolution of the display means (30) and / or the video image data (31), in which a volume element (152) corresponding to the particular surface element (151) is assigned to the particular pixel (33) such that the OCT image (32) corresponds at least to a portion (60) of the video image data (31); a method step (S500) of presenting the video image data (31) on the display means (30) and presenting the OCT image (32) on the display means (30) at the position of the portion (60), wherein the video image data (31) and the time resolved OCT image (32) are presented sequentially on the display means (30); determining a zoom level of the surgical microscope system (10); - initiating a modification of the presentation of the video image data (31) and the time resolved OCT images (32) based on the zoom level; The method includes:

2. - acquiring a time-resolved first image signal (131) of said imaging field of view (14) and a time-resolved second image signal (132) of said imaging field of view (14), - the method steps of identifying first video image data (311) corresponding to said first image signal (131) and second video image data (312) corresponding to said second image signal (132), - identifying a first time-resolved OCT image (321) corresponding at least to a portion (60) of the first video image data (311) and a second time-resolved OCT image (322) corresponding at least to the portion (60) of the second video image data (312), - stereoscopically presenting said first video image data (311), said second video image data (312), said time-resolved first OCT image (321) and said time-resolved second OCT image (322) on said display means (30) with the same magnification, the same viewpoint and / or the same stereo angle α. The method of claim 1 further comprising:

3. 3. The method according to claim 2, wherein the surgical microscope system (10) comprises a stereo camera (11, 12) with a first image sensor (121) for acquiring the time-resolved first image signal (131) of the imaging field (14) and a second image sensor (122) for acquiring the time-resolved second image signal (132) of the imaging field (14), and the display means (30) is designed to present stereo image data (31, 32).

4. The method according to claim 2 , wherein the first OCT image and the second OCT image (321, 322) are stereoscopically presented on the display means (30) at the location of the portion (60).

5. 2. The method according to claim 1, wherein the image signal (13) and the OCT signal (27) are matched locally based on acquisition parameters of the OCT system (20) and the surgical microscope system (10) and / or the video image data (31) and the OCT image (32) are matched locally by image analysis.

6. 2. The method of claim 1, wherein the time-resolved OCT image (32) corresponds to a portion (60) of the presented video image data (31), and the OCT image (32) is presented in the portion (60) and the video image data (31) is presented in a remaining portion of the display means (30).

7. The method of claim 1 , wherein the switching of the presentation of the video image data (31) and the time-resolved OCT images (32) is initiated based on a user input acquired by an interface (50) for acquiring a user input.

8. A system (100) for multimodal image acquisition and image visualization, comprising: A surgical microscope system (10) comprising an optical unit (11) and an image sensor (12) designed to acquire a time-resolved image signal (13) of a selected imaging field (14) of a sample (15), said image signal (13) comprising a plurality of first tuples (133) each representing a surface element (151) of said sample (15) and at least one grayscale value; an OCT system (20) configured to acquire a time-resolved OCT signal (27) of the selected field of view (14), the OCT signal (27) including a plurality of second tuples (271) each representing a volume element (152) and a scattering intensity of the sample (15); a display means (30) designed for time-resolved display of image data (31, 32) having a plurality of pixels (33); A control unit (40), determining video image data (31) corresponding to the acquired image signal (13) and presenting it on the display means (30), the video image data (31) being determined in such a way that a particular pixel (33) represents a particular surface element (151) based on the first tuple (133) and a resolution of the display means (30); determining, based on the acquired OCT signal (27), a time-resolved OCT image (32) corresponding at least to a portion (60) of the presented video image data (31) and presenting the OCT image (32) on the display means (30) at the position of the portion (60), wherein the OCT image (32) is determined in such a way that a volume element (152) corresponding to the particular surface element (151) is presented on the particular pixel (33) based on the second tuple (271) and the resolution of the display means (30) and / or the video image data (31), and the video image data (31) and the time-resolved OCT image (32) are presented sequentially on the display means (30); Ascertaining the zoom level of the surgical microscope system (10) and initiating a change in the presentation of the video image data (31) and the time-resolved OCT images (32) based on the zoom level. A control unit (40) configured to A system (100) comprising:

9. A computer program comprising commands which, when executed by a control unit of a system according to claim 8, cause the system according to claim 8 to carry out the method according to any one of claims 1 to 7.

Images