System for Acquisition and Visualization of OCT Signals
The system addresses the challenge of real-time volumetric OCT image display by using a tunable laser and control unit to generate time-resolved OCT images with adjustable viewing directions, improving surgical precision and integration with surgical microscope data.
Patent Information
- Application Number
- JP2023111570
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-08-10
- Filing Date
- 2023-07-06
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-07-06
AI Technical Summary
Existing surgical imaging systems lack methods for real-time display of volumetric OCT images, which are essential for optimal surgical guidance, often leading to user distraction and potential complications.
A system and method for acquiring and visualizing time-resolved OCT signals using a wavelength tunable laser, interferometer, scanning mechanism, detector, and control unit to generate time-resolved OCT images, allowing for dynamic virtual viewing directions and seamless integration with surgical microscope images.
Enables real-time, optimized visualization of OCT signals with adjustable virtual viewing directions, enhancing surgical precision and reducing user distraction by integrating volumetric OCT images with surgical microscope data.
Smart Images

Figure 0007594050000001 
Figure 0007594050000002 
Figure 0007594050000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a system for the acquisition and visualization of OCT signals, in particular for the acquisition of a time-resolved OCT signal and for the visualization of the OCT signal in the form of a time-resolved OCT image. The present invention further relates to a method for the acquisition and visualization of an OCT signal. [Background technology]
[0002] The use of technical aids is an integral part of modern medicine. Imaging procedures and robotic systems are nowadays used as a matter of course not only in diagnostics but also in the surgical department. The use of imaging procedures allows the display and identification of various structures of a patient, and the image data obtained from the patient can be advantageously used not only in diagnostics but also in therapeutic and surgical procedures.
[0003] For example, the surgeon can better plan and perform a surgical procedure based on the image data of the patient. A robotic visualization system is used to assist the surgeon in performing the surgical procedure. The system typically includes at least one camera for taking images of the surgical site, and the camera is supported by a jointed tripod. The tripod allows the camera to be positioned with translational and / or rotational motion relative to the subject to obtain an image of a desired field of view (FOV) of the surgical site. In this case, the acquisition of 3D image data is possible by using an optical stereo camera.
[0004] In addition to obtaining surface information of a desired field of view, for example based on reflected or backscattered visible light, there are currently also methods for obtaining depth information of the field of view. These methods include optical coherence tomography (OCT), which allows three-dimensional microscopic depiction of optically transparent and / or reflective objects, and thus allows the acquisition of volumetric images of biological tissue in the observation field. Optical coherence tomography (OCT) is essentially an interferometric method using broadband light with a short coherence length. Therefore, a system for obtaining OCT data comprises an interferometer and a broadband light source with a spectral width larger than 1% of the central wavelength.
[0005] The acquisition of OCT data can be sequential or parallel. Sequential acquisition of OCT data is performed, for example, by splitting the light beam of a low-coherence light source into a sample beam and a reference beam in a beam splitter and passing them through two arms of an interferometer. A movable reference mirror is then arranged in the reference beam path and the test object in the object beam path. By displacing the reference mirror, the path difference between the object beam and the reference beam and thus the measurement depth can be adjusted. By means of a mirror in the path of the object beam, the object beam is scanned two-dimensionally over the sample, which allows a three-dimensional scanning of the sample.
[0006] When acquiring data in the time domain, the spectral width Δλ of the light source is L C = λ * / Δλ coherence length L C The axial resolution of an OCT system is determined by the coherence length L of the light used. Cand indicates the resolving power of an object having a distance of at least the coherence length along the optical axis. For example, a light source in the near infrared range with a central wavelength of 800 nm and a spectral width of 80 nm has a coherence length of 7 μm, whereby an OCT system equipped with such a light source has an axial resolution of about 1-10 μm. The lateral resolution of an OCT system is determined by the optics used in the object beam path, in particular by the objective lens that focuses the light on the examined object.
[0007] Sequential acquisition of OCT data is also possible in the frequency domain (frequency domain OCT: FD OCT), where a distinction is usually made between the use of a tunable light source (swept source OCT) and the use of a distributed detector (spectral domain OCT: SD OCT). In swept source OCT, the frequency of the excitation light source (often a laser) is changed, so that the optical path difference between the sample and reference beams and thus the scanned sample depth can be varied without the need for a movable reference mirror. In SD OCT, a broadband light source is likewise used, but the frequency components of the interference signal are separated before detection, for example by an optical grating.
[0008] OCT allows the acquisition of incision and volume data of biological tissues and can significantly increase the amount of information for the surgeon. It is therefore desirable to integrate OCT into a surgical microscope so that it can display surface image data of the desired field of view as well as depth and / or cross-sectional images of that field. Even if a surgical system is able to provide volumetric OCT images, their capture time is relatively long and rendering is limited to post-processing only due to time and resource constraints. Therefore, real-time volumetric imaging during surgery has not been possible until now.
[0009] However, with further advances in OCT technology and the increasing power and speed of GPUs (graphics processing units), faster OCT methods are becoming available for intraoperative imaging. The resulting real-time display of OCT volumetric images allows for a wide variety of display possibilities for the acquired OCT signals, but methods for this are not yet available. In particular, methods that allow for display formats of OCT signals optimized for medical or surgical applications are lacking. The use of suboptimal methods carries the risk of over-demanding or distracting the user, which may have serious consequences, especially for the surgeon. Summary of the Invention [Problem to be solved by the invention]
[0010] It is an object of the present invention to overcome or at least mitigate the disadvantages of the prior art and to provide an improved system and method for acquiring and visualizing OCT signals. [Means for solving the problem]
[0011] The problem according to the invention is solved by the subject matter of the independent patent claims. Preferred further embodiments are the subject matter of the dependent claims.
[0012] A first aspect of the present disclosure relates to a system for acquiring and visualizing OCT signals, and in particular for acquiring and visualizing such signals by a medical instrument, such as a surgical microscope.
[0013] The system according to the present disclosure comprises an OCT system. The OCT system preferably has a broadband light source configured to illuminate the sample. The light source is preferably a tunable laser (swept source), such as a broadband laser, a supercontinuum laser and / or an ultrashort pulsed laser. The tunable laser is then a narrowband light source at a given time, the central frequency of which can be precisely varied in time, or can be formed from a number of narrowband light sources. However, other broadband light sources, such as superluminescent diodes, can also be used, for example in FD-OCT.
[0014] The OCT system preferably further comprises an interferometer, for example a Michelson interferometer, a Mach-Zehnder interferometer or a Koester interferometer, formed for the generation and superposition of the sample beam and the reference beam. This interferometer preferably comprises a beam splitter for the generation and superposition of the sample beam and the reference beam from the light of a broadband light source, a reference beam path and a sample beam path. Furthermore, the interferometer preferably comprises means for the adjustment of the sample depth to be examined. Depending on the measurement method, this may then be means for the generation of a path difference (such as a displaceable mirror in the reference beam in the case of SD-OCT), means for the separation of light of a specific path difference (such as an optical grating in the case of FD-OCT) or means for the generation of light of a specific path difference (such as a tunable light source in the case of swept-source-OCT).
[0015] The OCT system further comprises a scanning mechanism configured for scanning the sample with the sample beam, the scanning mechanism being in particular configured to scan the sample beam two-dimensionally over the sample. Preferably, the scanning mechanism is a scanning mirror, but other scanning mechanisms can be used, such as, for example, a glass fiber scanner, a prism scanner, a Palmer scanner, etc. In an OCT system configured for full-field OCT, a scanning mechanism is not required.
[0016] The OCT system further comprises a detector configured for acquisition of an interference pattern generated by superimposing the sample beam and the reference beam. The detector is for example a line detector, a two-dimensional detector array, a photodetector or a distributed detector. The detector is for example configured as a CCD detector or a CMOS detector.
[0017] The system according to the present disclosure further comprises a display means configured for time-resolved display of the image data. The display means is preferably one or more displays, such as at least one display of a surgical microscope, a display permanently installed in the operating room, or a head-mounted display (HMD), such as video glasses. The display is preferably a 4K and / or 8K capable display and / or a 3D display configured for stereoscopic display.
[0018] The system according to the present disclosure further comprises a control unit connected to the OCT system and to the display means, in particular for one-way or two-way data transmission. The control unit is configured to control the OCT system to acquire a time-resolved OCT signal of a selected region of interest (ROI) of the sample. The sample is, for example, a surgical site of a patient, in particular an eyeball in the case of ophthalmic surgery. However, the sample can be other surgical areas, such as brain tissue in neurosurgery, tissue located in the ear, nose and throat area in ENT surgery, gums, dental calculus or dental nerve in dental surgery. It can also be any other tissue or specimen (in vivo, in vitro or in situ). The field of view is preferably selected by the user. The time-resolved OCT signal is preferably acquired by generating an optical signal, directing the optical signal partially as a sample beam by means of an interferometer to the sample and bringing it into the interferometer for superposition with a reference beam generated from the optical signal in order to generate an interference pattern.
[0019] The control unit of the system according to the present disclosure is further formed to determine a time-resolved OCT image based on the acquired time-resolved OCT signal and a pre-settable virtual line of sight direction. The OCT signal is an interference signal, whereby the modulation of the envelope of the interference signal encodes the reflection properties of the sample. By means of the scanning mechanism, the sample can be scanned two-dimensionally at a sample depth set by the path difference. Based on the scanning mechanism used, the means used to select or generate the path difference, e.g. an adjustable mirror in the reference beam, an optical grating or a tunable broadband light source in front of the detector, and the repetition rate of the detector, a clock frequency (image repetition rate) of the time-resolved OCT signal is obtained. The control unit determines the time-resolved OCT image computationally, e.g. by volume rendering, ray tracing and / or ray marching, based on the OCT signal. For this purpose, the control unit first determines a virtual line of sight direction and takes this virtual line of sight direction into account when determining the OCT image by means of volume rendering, ray tracing and / or ray marching. Thereby, the OCT image is determined by the system according to the present disclosure on the basis of the OCT signal and a predefinable virtual gaze direction, whereby the degrees of freedom offered by the OCT system can be advantageously used for the perspective determination of the OCT image. The virtual gaze direction is then in principle different from the real gaze direction of the user. Rather, the virtual gaze direction represents a spatial direction that can be predefinable by the user as described below and / or can be determined on the basis of device parameters as described below, which spatial direction is used as the basis for the calculation of the OCT image.
[0020] The control unit of the system according to the present disclosure is further configured to display the determined time-resolved OCT image on the display means, whereby the display is advantageously performed in a virtual gaze direction. The virtual gaze direction is then preferably set for a defined time. Also preferably, the virtual gaze direction is set in the form of a sequence and is changed automatically, for example after a predefined time, upon detection of a defined user input (e.g. operation of a foot pedal), upon determination of a change in the state or position of the medical instrument, upon determination of a change in the device parameters and / or upon recognition of a change in the stage of the performed operation. Also preferably, the virtual gaze direction follows the user input continuously or discontinuously, for example based on the determined actual gaze direction of the user. Thereby, for example, the tilt of the user's head can be obtained and an OCT image with a corresponding tilt can be created and displayed on the display means. The system according to the present disclosure thereby advantageously enables an optimal visualization of the OCT signal in response to the user's requirements, whereby the potential of the OCT signal is optimally utilized and the maximum information content is available to the user.
[0021] In a preferred embodiment of the system according to the present disclosure, the virtual line of sight direction is defined by an azimuth angle and an elevation angle relative to the optical axis of the OCT system. The optical axis of the OCT system then preferably corresponds to the optical axis of a surgical microscope system (described below) of the system according to the present disclosure and / or is preferably perpendicular to the surface of the sample. The azimuth angle preferably denotes an angle between 0° and 360° in a plane perpendicular to the optical axis, preferably in the plane of the sample surface. The azimuth angle is a planar angle, measured between the zero point and the projection of the virtual line of sight direction onto this plane. The azimuth angle therefore defines the virtual line of sight direction in the narrow sense. The elevation angle preferably denotes an angle between 0° and 90° in a plane containing the optical axis, preferably a plane perpendicular to the sample surface. The elevation angle is a planar angle, measured for example between the virtual line of sight direction and the sample surface. The elevation angle therefore defines the vertical viewing angle.
[0022] In a further preferred embodiment of the system according to the present disclosure, the control unit is adapted to generate time-resolved OCT images in three dimensions and from virtual viewing directions. Saw In addition to displaying the OCT image in an "en-face" view, i.e. in a plan view, the control unit is configured to determine the OCT signals as a volumetric image. Viewpoint Image Also preferably, the control unit is further adapted to display the OCT signals in the form of cross-sectional images in the height direction of the sample (so-called B-scans). Saw Tertiary Original body Displaying time-resolved OCT images as product images advantageously allows obtaining maximum depth information about the object. In particular, direct viewing from a bird's-eye view (plan view, "en face"), direct viewing from different virtual viewing directions (azimuth and elevation), and cross-sectional images (B-scans) along different cross-sectional lines are useful, for example, to obtain relevant information about the specimen at different stages of surgery.
[0023] In other words, especially in medical (e.g. ophthalmic) applications, the most useful virtual viewing direction may vary depending on the application and the surgical situation. Therefore, in order to provide the user with the best overview, it may be useful to first display the time-resolved OCT images (4D-iOCT) from the technical viewing direction of the video of the surgical microscope (wherein the technical viewing direction is determined, for example, on the basis of the device parameters). In further applications or in the course of surgery, however, other, for example lateral, virtual viewing directions can ensure much better depth information. The resulting perspective corresponds more closely to the natural perspective of the human gaze with the naked eye, thereby also offering advantages for microsurgical tasks.
[0024] The system according to the present disclosure preferably comprises an interface for obtaining user input. The interface is preferably a hand switch or a foot switch. Also preferably, the interface is a means for head movement and / or eye movement or actual gaze direction recognition, for example integrated in video glasses or a head mounted display (HMD). The interface is further configured for recognition of voice commands and may include at least one microphone for this purpose. Also preferably, the interface is a keyboard, a joystick, a mouse, a touch screen or a combination thereof.
[0025] According to this preferred embodiment, the control unit is furthermore formed to control the interface for obtaining a user input. Further preferably, the user input is preceded by an input request, which is made known to the user, for example by the output means. Also preferably, the user input is a selection from a plurality of predefined input options, which input options specify, for example, a virtual gaze direction, an operation to be performed and / or an operation to be performed. Also preferably, the user input is a user input that is freely definable by the user within a predefined range. Also preferably, the control unit is formed to assist the user in the user input, for example by the display of a plurality of dialogs for purposeful interrogation of a predefined user input.
[0026] According to this embodiment, the control unit is further configured to determine the virtual line of sight based on the user input. The user can thereby advantageously select the virtual line of sight that is considered optimal. Particularly preferably, the user input specifying the azimuth angle and the elevation angle is obtained by the interface, for example after a corresponding input request to the user. Also preferably, the azimuth angle and the elevation angle are derived from the user input, for example from the line of sight that is specified directly by the user using head and / or eye control.
[0027] Alternatively or additionally, the system further comprises a device interface configured for the acquisition of device parameters. The device interface is preferably an interface for the connection of a surgical microscope system, as will be explained in detail below. Also preferably, the device interface is, however, also an interface for the connection of any other imaging system. The device interface may further be an interface for the connection of a medical instrument. Also preferably, via the device interface, the system is connectable for the tracking of a medical or medical-technical instrument, whereby the tracking system may be a surgical microscope system or another imaging system. The medical-technical instrument may be, for example, a pointer, a probe, tweezers, a piercing needle, a phaco-tip, an endoscope, an Endo-LED of a gripper, etc. According to this embodiment, the control unit is preferably configured for communication with the device interface, in particular for bidirectional communication, and further configured for controlling the device interface for the acquisition of device parameters of the device connected by said interface.
[0028] According to this embodiment, the control unit is further formed to determine a virtual line of sight based on the device parameters acquired by the device interface. Preferably, based on the device parameters of the connected surgical microscope system or other imaging system, the technical line of sight of the surgical microscope system or other imaging system can be determined and the virtual line of sight can be set equal to the technical line of sight. This allows for a multimodal image acquisition and display, where the virtual line of sight of the OCT image advantageously coincides with the (technical) line of sight of the other image data.
[0029] Also preferably, the device parameters represent the pose, position and / or state of the connected instrument or of the medico-technical (or tracked medical) instrument tracked by the tracking system, and the virtual line of sight is determined based on the pose, position and / or state of the medico-technical (or medical) instrument. The virtual line of sight can thus be determined, for example, depending on the position of the pointer (if necessary along the longitudinal axis of the pointer) or depending on the state of the gripper. The spatial position of the medico-technical instrument can then be determined by the instrument itself and / or the spatial position of the medical instrument or medico-technical instrument can be determined via the tracking system. The medical instrument does not then have to make use of its own electronics, but its characteristics can for example be determined solely by the tracking system, possibly with the aid of markers arranged on the medical instrument. Also preferably, the (image) recognition of the medico-technical instrument is initiated by a user input, which can take place on the instrument itself or elsewhere.
[0030] The device parameters received via the device interface, relating to the state of the connected medical-technical instrument, preferably also include the operation of the input means of the medical-technical instrument. Thus, in this case, the device parameters also represent the user input at the same time. Also preferably, various device parameters are taken into account in common, such as the position of the medical-technical instrument (for example determined by the instrument itself and / or by the tracking system) and the operation of the input means of the medical-technical instrument. Thus, for example, a series of virtual viewing directions corresponding to a series of positions of the instrument can advantageously be obtained. The device parameters describing the medical instrument or medical-technical instrument can be received via the device interface from the instrument itself or from the tracking system. Furthermore, preferably, the azimuth and elevation angles of such virtual viewing directions are derived from the received device parameters.
[0031] In a preferred embodiment of the system according to the present disclosure, the OCT signal comprises a number of (first) tuples each including (or representing) a volume element of the sample and a scattering intensity. The volume element of the sample is then preferably represented by three spatial coordinates (e.g. x, y and z) and can be interpreted as a sample voxel for example. In addition to the scattering intensity, the tuples can have other values. According to this embodiment, the display means comprises a number of pixels, and the control unit is configured to determine the time-resolved OCT image based on the (first) tuples, based on the resolution of the display means and based on the virtual line of sight, such that a particular pixel corresponds to a particular volume element, i.e. such that a particular pixel shows a particular volume element of the sample. In other words, the control unit determines an assignment of pixels of the display means and volume elements of the sample. This assignment may then depend on further settings, such as the raster resolution of the scanning mechanism used, but is preferably constant in time for a given selection of settings, such as the virtual line of sight, zoom level and stereo angle. Thereby, the control unit achieves local registration between the pixels of the display means and the OCT signal or the determined OCT image.
[0032] Furthermore, the control unit is preferably configured to locally register the OCT signals acquired by the OCT system based on the acquisition parameters of the OCT system. The local registration of these signals then represents a correct link of these signals with a reference coordinate system, for example with a coordinate system of a patient during surgery, and allows an unambiguous depiction (mapping) of coordinates of the patient space to corresponding coordinates of the signal space. The signal registration preferably requires a calibration of the OCT system. The acquisition parameters of the OCT system preferably comprise calibration parameters. The acquisition parameters of the OCT system preferably take into account the scanning mechanism and / or the detector of the OCT system. Based on the local registration based on the acquisition parameters, advantageously structures of the patient located at defined coordinates of the patient space can be correctly displayed in corresponding coordinates in the image space of the OCT image, in particular in a correct relative position to each other.
[0033] In a particularly preferred embodiment of the system according to the present disclosure, the system further comprises a surgical microscope system configured for acquisition of a time-resolved image signal of a selected field of view of the sample. The surgical microscope system preferably comprises an optical system for acquisition of light, for example visible light, reflected or backscattered from the sample. The optical system comprises, for example, an objective lens and an eyepiece, and may further comprise other components, in particular other lenses, mirrors, beam splitters and / or the like. The surgical microscope system further comprises an image sensor configured for acquisition of a time-resolved image signal of a selected field of view (region of interest: ROI) of the sample. The optical system and the image sensor are preferably formed integrally, for example as part of a camera of the surgical microscope. According to this embodiment, the control unit is further formed to determine video image data corresponding to the acquired time-resolved image signal. The time-resolved image signal is in particular a plurality of signals assigned to surface elements of the sample, which signals are acquired sequentially or simultaneously for a certain raster of the sample surface, whereby the raster is determined by a scanning mechanism and / or an image sensor. The time resolved image signal further has a clock frequency (frame rate) determined by the scanning mechanism and / or the image sensor. The control unit generates from the image signal video image data having a raster (resolution) and frame rate suitable for display on the display means. The control unit is further configured to display the video image data by the display means.
[0034] In the system according to the present disclosure, the image signal preferably comprises a plurality of (second) tuples, each (second) tuple then comprising (or representing) a surface element of the sample and at least one greyscale value. The surface element of the sample is then preferably represented by two lateral spatial coordinates (e.g. x and y) and can be interpreted, for example, as a sample pixel. In addition to the greyscale value finally obtained from the acquired intensity, each (second) tuple can also detect a colour value, for example during the acquisition of different colour intensities by means of a colour filter connected upstream of the image sensor. According to this embodiment, the control unit is further adapted to determine the video image data based on the (second) tuple and on the resolution of the display means, whereby a particular pixel represents a particular surface element of the sample. In other words, the control unit determines a (second) association of the pixels of the display means with the surface element of the sample in addition to the (first) association of the pixels of the display means with the volume element of the sample. This (second) association may then depend on further settings, such as the zoom level of the surgical microscope system, but is preferably constant in time for a given setting. The (second) association preferably corresponds to the (first) association. The control unit thereby also realizes a local registration between the pixels of the display means and the image signals (video image data) of the surgical microscope system.
[0035] Furthermore, the control unit is preferably further configured for locally registering the image signals acquired by the surgical microscope system based on acquisition parameters of the surgical microscope system. The registration of the signals preferably requires a calibration of the surgical microscope system. The acquisition parameters of the surgical microscope system therefore preferably comprise calibration parameters and / or optical settings of the surgical microscope system, such as, for example, the focal length and / or the zoom level of the optical system (camera) used. Furthermore, the acquisition parameters preferably also comprise a set of intrinsic parameters of the surgical microscope system. The intrinsic parameters then determine the relationship between the coordinate system of the image signals and the coordinate system of the associated image sensor. The type of intrinsic parameters depends in particular on the type of image sensor used, whereby with respect to the image sensor we refer here to the original sensor and the optical system used. The intrinsic parameters comprise, in the case of a camera calibration according to Tsai, for example, the effective focal length, the coordinates of the image principal point (center of distortion) of the image signal, a first scaling factor and / or a first radial lens error factor (distortion factor). As an alternative to the above intrinsic parameters of the camera calibration by Tsai, other intrinsic parameters can be used, for example, for the camera calibration by Zhang (see, for example, "A practical comparison between Zhang's and Tsai's calibration approaches", Li et al, Proceedings of the 29th International Conference on Image and Vision Computing New Zealand, November 2014 Pages 166-171, DOI:10.1145 / 2683405.2683443).
[0036] The local registration of the OCT signal together with the local registration of the image signal advantageously allows the generation and display of OCT images and video image data, whereby the generated time-resolved OCT image corresponds to at least a portion of the displayed video image data. Preferably, the OCT signal is acquired from the entire field of view and the OCT image is generated from at least a portion of the field of view. Also preferably, the OCT signal is acquired from a portion of the field of view and the OCT image is generated from at least a portion of the portion of the field of view. The control unit is further configured to display the time-resolved OCT image at the position of a section of the video image data on the display means. Preferably, the video image data and the OCT image are generated from the entire field of view and are respectively displayed on the entire display means. Also preferably, the OCT image is generated from a portion of the field of view and is displayed on the display means at the position of the video image data corresponding to the portion of the field of view. In other words, the video image data and the OCT image corresponding to the same portion of the sample are displayed at the same position on the display means. Thus, the system according to the present disclosure allows seamless integration of the video image data and the OCT image on the display means, thereby allowing the user to more easily observe the multimodal image data. Thus, multimodal image data can be viewed without head or eye movement, which benefits the surgeon's attention, especially during intraoperative imaging.
[0037] The control unit of the system is preferably configured to simultaneously display the video image data and the time-resolved OCT image on the display means. This particularly advantageously allows simultaneous consideration of both image modalities by the user. Nevertheless, in order to allow a distinction between the different image data, the control unit is further preferably configured to display the video image data in a first transparency and the time-resolved OCT image in a second transparency. In this case, the first transparency and the second transparency are preferably different. Also preferably, the first transparency and the second transparency vary in time. For example, initially the image signal is displayed with 0% transparency, while the OCT image is displayed with 100% transparency. Then, over time, the transparency of the image signal is continuously adjusted from 0% to 100%, while at the same time the transparency of the OCT image is adjusted from 100% to 0%. This ensures a continuous transition between the display of the video image and the OCT image.
[0038] In a likewise preferred embodiment of the system according to the present disclosure, the control unit is further formed to locally register the video image data and the OCT image by image analysis, whereby the local registration of these image data represents a correct combination of these images in a common image coordinate system. The registration based on the image data thereby allows a relative combination of the image data for a representation of identical structures as closely as possible. For example, the boundaries of structures or tissues in the video image data and the OCT image can be recognized by image analysis, such as edge recognition, and compared with each other. By translation, rotation and / or scaling, these structures can be superimposed on each other on the display means. Preferably, the local registration of the image data is performed in addition to the local registration of the acquired signals described above.
[0039] In a similarly preferred embodiment of the system according to the present disclosure, the control unit is configured to display the video image data and the time-resolved OCT image sequentially on the display means. In other words, at least at certain positions of the display means, only one of the video image data and the OCT image is displayed each time. This advantageously allows a clear distinction between the video image data and the OCT data, and also advantageously allows the simultaneous display of different image data at different positions of the display means.
[0040] Particularly preferably, the control unit is adapted to convert the video image data and the time-resolved OCT image into a single image at the same magnification and at the same time. line of sight direction In the case of simultaneous display of both video image data at the same position of the display means, this preferably allows a perfect overlay of both image data and advantageously allows a display with optimized contrast. In the case of sequential display, it is preferably possible to display the locally registered image data with the same magnification, the same image quality, and / or the same stereo angle. line of sight direction , and / or with the same stereo angle, thereby ensuring a smooth transition between the display of the image data. In the case of video image data, only a surface (top) display is possible, which corresponds for example to an "en-face" OCT image. As soon as the transition from the video image data to the OCT image takes place, it is furthermore preferably possible to select a magnification, in order to advantageously enable an improved view with optimal depth perception. line of sight direction , and / or the stereo angle is adjustable, in particular from an initial "en-face" view, e.g. by continuous shifting of the virtual line of sight, Gaze image It is possible to switch to a 3-D view or to a cross-sectional view (OCT-B scan), thereby advantageously ensuring a perfect field of view at the various stages.
[0041] According to a further preferred embodiment of the system according to the present disclosure, the display means is preferably formed to display the image data stereoscopically. The display means is preferably a display means formed for displaying image data with different (e.g. orthogonal) polarizations in combination with polarized glasses. Also preferred is encoding of the stereoscopic image data by color filtering and the combination with color filtered 3D glasses. However, the display means may also be a 3D display, for example a light field monitor. According to this preferred embodiment, the control unit is further configured to determine a time-resolved first OCT image and a time-resolved second OCT image based on the time-resolved OCT signal, the virtual line of sight and the stereo angle, and to display the first OCT image and the second OCT image stereoscopically on the display means. The stereo angle has a decisive influence on the sense of depth during image visualization on the display means. In the case of a stereo camera, the stereo angle corresponds to the angle between the optical axes of the individual cameras of the stereo camera, which in turn depends on the distance between the cameras from each other and on the working distance between the cameras and the object (sample).
[0042] During the generation of the stereo OCT image, such stereo angles are used as a basis for calculations, which allow for variable adjustment of the stereo angles. A large stereo angle typically corresponds to a strong depth perception (such as the human eye's perception of a close object), and a small stereo angle typically corresponds to a weak depth perception (such as the human eye's perception of a distant object). Thereby, the system according to the present disclosure advantageously allows for the display of OCT images with a variable sense of depth.
[0043] In a preferred embodiment of the system according to the present disclosure, the control unit is configured to determine the stereo angles based on user input obtained by the interface, thereby advantageously allowing a user, e.g. an operator, to individually adjust the stereo angles, so that by taking the user input into account, subjective preferences can also be taken into account when determining the stereo angles.
[0044] In a further preferred embodiment of the system according to the present disclosure, the control unit is further configured to determine the stereo angle on the basis of optical parameters of the surgical microscope system, which are then acquired or read in the form of metadata by the control unit of the system according to the present disclosure. The control unit is thereby advantageously always informed of the variable and fixed optical parameters of the surgical microscope system. The fixed optical parameters then preferably characterize components of the surgical microscope system, such as, for example, lenses, eyepieces, etc. used, and the variable optical parameters then preferably characterize adjustable quantities, such as, for example, field of view, resolution, tilt of the optical axis, etc. Taking into account the optical parameters of the surgical microscope system advantageously enables simultaneous display of OCT images and video image data, with optimal adaptation of the image data to one another, and also enables sequential display of OCT images and video image data, with optimal transition.
[0045] In a particularly preferred embodiment of the system according to the present disclosure, the surgical microscope system has a stereo camera including a first stereo objective (in combination with a first image sensor) arranged along a first optical axis and configured for the acquisition of a first image signal of the field of view, and a second stereo objective (in combination with a second image sensor) arranged along a second optical axis and configured for the acquisition of a second image signal of the field of view. The surgical microscope system is thus configured for the acquisition of a stereo time-resolved image signal. The first and second optical axes of the surgical microscope system then enclose an angle with each other, whereby this angle between the optical axes together with the working distance between the specimen and the objective determines the stereo angle of the stereo image acquisition in the surgical microscope system. According to this embodiment, the control unit is preferably formed to determine a stereo angle on which the determination of the first OCT image and the second OCT image is based, corresponding to a stereo angle of the surgical microscope system. Furthermore, the control unit is preferably formed to determine the first video image data based on the first image signal and to determine the second video image data based on the second image signal. This embodiment thereby advantageously allows stereoscopic display of video image data and OCT images with corresponding stereo angles. This advantageously allows stereoscopic simultaneous display with minimal shifts as well as stereoscopic sequential display with optimal transition between different image modes (video image data and OCT image data). According to this embodiment, the optical parameters of the surgical microscope system are or include at least those parameters of the surgical microscope system that influence the stereo angle.
[0046] In an embodiment which is also preferred, the control unit is configured to determine the stereo angle based on the zoom level and / or working distance of the stereo camera. In this case, the distance between the cameras of the stereo camera is preferably fixed relative to one another, whereby the stereo angle depends only on the working distance of the cameras to the sample. This working distance can preferably be determined based on the zoom level, for example in combination with the size of the imaged field of view or a marker. If the distance between the cameras of the stereo camera is configured to be variable relative to one another, for example in the case of cameras which can be positioned by camera movement, the distance between the cameras is preferably also taken into account when determining the stereo angle relative to one another. According to this embodiment, the control unit is preferably configured to determine the stereo angle of the surgical microscope system based at least on the zoom level and / or working distance, and to use this stereo angle also for the OCT image.
[0047] In a similarly preferred embodiment, the zoom level is also taken into account independently of the working distance. For example, in the display of the field of view by the surgical microscope system, a small zoom value is preferred to use a small stereo angle. A small zoom value usually corresponds to a wide field of view, where a strong sense of depth is usually not desired. In the display of the field of view by the surgical microscope system, a large zoom value is preferred to use a large stereo angle. A large zoom value corresponds to a small field of view, where a detailed display with a strong sense of depth is often desired. Therefore, at high magnifications, the sense of depth is advantageously also increased in the display of the determined OCT image.
[0048] In a further preferred embodiment of the system according to the present disclosure, the control unit is configured to determine a stage of the performed operation and to determine the stereo angle based on the stage of the performed operation. To that end, the control unit is preferably connected to a memory in which a trained machine learning algorithm, such as a neural network (CNN), is stored. This algorithm is preferably trained by a plurality of video image data and / or OCT images or image signals and / or OCT signals to which the corresponding stages of the OP have been assigned as classifications during training. The trained algorithm is thus able to independently recognize the stages of the operation as classifications based on the video image data and / or OCT images or image signals and / or OCT signals. Preferably, the control unit according to this embodiment is formed to select the stereo angle corresponding to the determined stage of the operation. The stereo data suitable for the different stages of the operation are preferably stored in a look-up table (LUT) in the memory or are similarly determined using a machine learning algorithm. This can be trained, for example, by assigning a number of video image data and / or OCT images, or image signals and / or OCT signals, as one or more classifications, with corresponding OP stages and / or corresponding stereo angles during training.
[0049] In cataract surgery, the surgical steps may include, for example: rest, incision, injection of intraocular viscosurgical device (OVD), capsulotomy, hydrodissection, phacoemulsification, irrigation / aspiration, intraocular lens implantation, wound closure / wetting, non-surgical. In refractive surgery, the surgical steps may include, for example: idling, docking, applanation, application of globe / CG rotation, phaco incision, phaco incision, phaco incision, phaco incision, phaco incision, release of globe, transfer to OPMI, positioning of OPMI, opening of incision, setting of plane, separation of Kappenbett, separation of Linsenbett, removal and / or inspection of lens, wiping, irrigation, slit lamp, removal of speculum. In dental surgery, the surgical steps may include, for example: access, excision, debridement, drying, fixation, repair. It should be noted that all or only some of these steps may be part of the corresponding surgery, and that there may be other surgical steps as well and / or some steps may be omitted.
[0050] In a further preferred embodiment of the system according to the present disclosure, the system further comprises a medical instrument, such as a probe, a pointer, a tweezers, a puncture needle, a phaco tip, an endoscope, an Endo-LED, etc.
[0051] According to this embodiment, the control unit is further configured to determine the position, type and / or state of the medical instrument. The type of the medical instrument can then preferably be determined based on a device interface for connecting the medical instrument, by the tracking system (e.g. by recognition of a target ID) and / or based on an input via a user interface. Also preferably, the recognition of the type of the medical instrument brought into the field of view of the surgical microscope system is performed by image analysis of the video image data, for example segmentation and object recognition. The position of the medical instrument is preferably determined based on the detection of a marker and / or a target comprising a plurality of markers, where the marker can be a landmark on the medical instrument or a structure of the medical instrument. The acquisition of the marker and / or the target is preferably performed using the surgical microscope system and, if necessary, using an additional light source (e.g. infrared LED) and / or after registration / calibration of the medical instrument (e.g. by positioning the tip of the medical instrument at a predetermined position).
[0052] The state of the medical instrument brought into the field of view of the surgical microscope system is preferably determined based on an image analysis of the video image data. For example, identification of whether the forceps are open or closed can be made based on the image data. Furthermore, a user input for changing the state can be read by the control unit, for example a user input for the activation of a phaco tip signaling the change of state. Furthermore, a sensor attached to the medical instrument can detect the change of state, for example the closure of the forceps, and send a corresponding sensor signal to the control unit.
[0053] According to this preferred embodiment, the control unit is further adapted to determine a stereo angle for the determination of the stereoscopic OCT images based on the position, type and / or state of the medical instrument. A change in state of a medical instrument of a given type and / or in a given position is preferably indicative of a given stage of the procedure. Thus, based on the knowledge of the position, type and / or state of the medical instrument, a display format optimal for this stage can be selected. This includes, in addition to the selection of the stereo angle, if necessary, also changing further display parameters of the OCT image data, such as the zoom level, the virtual viewing direction, the display depth and / or the slice direction.
[0054] Based on said information, the control unit can determine, automatically or based on user input, for example, that the performed operation is a membrane peeling with tweezers, in which case it can further determine the distance or position of the tweezers relative to the eyeball and can adapt or adjust the virtual gaze direction and / or the stereo angle of the display of the OCT image depending on said determination. In a further example, the control unit can preferably determine, automatically or based on user input, that the performed operation is a subretinal injection of a bleb "Bleb" or the placement of a retinal stent, and based thereon, for example, adjust the virtual gaze direction and / or the stereo angle of the display of the OCT image with a large sense of depth (stereo angle). Gaze image By selecting the display, the user can adapt the virtual viewing direction and / or stereo angle to optimally assist in positioning the needle or stent.
[0055] The functionality of the control unit according to the invention can be implemented by electrical or electronic parts or components (hardware), by firmware (ASIC) and / or by execution of suitable programs (software). Preferably, the functionality of the control unit according to the invention is realized or implemented by a combination of hardware, firmware and / or software. For example, the individual components of the control unit according to the invention for performing the individual functionalities are formed as separate integrated circuits or are arranged on a common integrated circuit.
[0056] The individual functionalities of the control unit according to the invention are further preferably formed as one or more processes, which are executed on one or more processors in one or more electronic computing devices and are generated during the execution of one or more computer programs, whereby the control unit is formed to cooperate with other components, in particular a user interface, an OCT system and a display means, in order to realize the functionality of the system according to the invention described herein. It is further obvious to a person skilled in the art that the functionality of several computers (data processing devices, control units, control devices) can be combined or combined in a single device to realize the functionality of the control unit according to the invention, or the functionality of a particular data processing device can be present distributed in several devices.
[0057] In a particularly preferred embodiment of the system according to the invention, the system is integrated into a surgical microscope, which preferably comprises an OCT system as described above. Further preferably, the surgical microscope comprises an interface for user input and a display means or the interface is connected to the display means. Preferably, the surgical microscope further comprises a surgical microscope system as described above, whereby the image sensor and the optical system are part of a camera, in particular a main observer camera or a surrounding camera of the surgical microscope. A control unit of the surgical microscope is preferably formed as a control unit of the system according to the invention and is in particular formed to execute the method according to the invention described below on the basis of instructions stored in a memory unit of the surgical microscope.
[0058] Within the scope of this disclosure, a surgical microscope is broadly understood as a microscope suitable for use during surgery. The surgical microscope preferably has a mount that allows the depiction of the surgical field independent of the surgeon's head movements. Furthermore, preferably, the surgical microscope has at least one beam splitter and at least two eyepieces. Alternatively, the surgical microscope is a pure "digiscopy" without eyepieces. Also preferably, the surgical microscope has at least one image sensor. Even more preferably, the surgical microscope has a main observer camera and a peripheral camera. The surgical microscope may have kinematic or robotic aids for the execution of the surgery. Alternatively, the surgical microscope may be called a medical technical microscope, a medically approved microscope or a medical microscope.
[0059] Another aspect of the present disclosure relates to a method for acquiring and visualizing OCT signals, which includes acquiring, by an OCT system (as described above), a time-resolved OCT signal of a selected field of view of a sample, where the OCT signal has a number of tuples, each tuple representing a volume element of the sample and a scattering intensity corresponding to the volume element, the method further includes determining a predetermined virtual line of sight and visualizing the time-resolved OCT image in three dimensions and from the virtual line of sight based on the tuples, the resolution of the display means and the virtual line of sight. Saw Finally, in the method according to the present disclosure, the time-resolved OCT image is imaged from a virtual line of sight. Saw It is displayed on the display means as a volumetric image.The method according to the present disclosure achieves similar advantages as the system according to the present disclosure, and in this respect reference is made to the above description.
[0060] In a preferred embodiment of the method according to the present disclosure, the method further comprises determining a time-resolved first OCT image and a time-resolved second OCT image, where the first OCT image and the second OCT image are respectively based on the tuple, based on the resolution of the display means, based on the viewing direction, and based on the stereo angle, and each in three dimensions and from a virtual viewing direction. SawAccording to this embodiment, the time-resolved first OCT image and the time-resolved second OCT image are furthermore displayed stereoscopically on a display means, which advantageously allows a stereoscopic display with a variable virtual viewing direction and a variable stereo angle.
[0061] In a further preferred embodiment of the method according to the present disclosure, the method further comprises a step of determining a predetermined virtual line of sight direction based on user input acquired by the interface and / or based on device parameters acquired by the device interface.
[0062] Further preferred embodiments of the method according to the present disclosure correspond to the further preferred embodiments of the system according to the present disclosure and achieve the same advantages as said embodiments.
[0063] A further aspect of the disclosure relates to a computer program comprising instructions which, when executed by a control unit as described above, preferably a control unit of a surgical microscope as described above, cause a system as described above or a surgical microscope to execute the method according to the invention as described above. The computer program preferably comprises instructions which, when executed by a control unit as described above, preferably a control unit of a surgical microscope, cause a system as described above or a surgical microscope to execute the method according to the invention according to one of the preferred embodiments as described above. The computer program according to the invention is then preferably stored in a volatile memory, for example a RAM element, or in a non-volatile storage medium, for example a CD-ROM, a flash memory, etc.
[0064] Further preferred embodiments of the invention are evident from the further features mentioned in the dependent claims and from the figures described below. The various embodiments of the invention mentioned in this application can be advantageously combined with one another, unless specified otherwise in individual cases.
[0065] In the following, the invention will be described in examples with reference to the accompanying drawings, in which: [Brief description of the drawings]
[0066] [Figure 1] FIG. 1 is a schematic diagram of a system according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram of a system according to a second embodiment. [Diagram 3] FIG. 2 is a schematic diagram of a virtual line of sight direction. [Figure 4] FIG. 2 is a schematic diagram of the visualization of the OCT signal as an OCT image depending on the virtual gaze direction. [Diagram 5] FIG. 2 is a schematic diagram of a sample and an indicator according to one embodiment. [Figure 6] 4 is a schematic diagram of an indication means and a medical instrument according to a further embodiment; [Figure 7] 1 is a schematic flow chart of a method according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0067] FIG. 1 is a schematic diagram of a system 100 for acquiring and visualizing an OCT signal 19 according to a first embodiment.
[0068] The system 100 comprises an OCT system 10 having a broadband light source 11, for example a superluminescent diode. The light of the light source 11 is directed to an interferometer having a movable mirror 15 and a beam splitter 14. At the beam splitter 14, the light is split into a sample beam 12 and a reference beam 13. The sample beam 12 is scanned by a scanning mirror 16 over the sample 65, so that at least a part of a selected field of view 66 of the sample 65 is scanned. The reference beam 13 is directed to the movable mirror 15 and reflected from the movable mirror 15 to the beam splitter 14. The sample beam 12 interacts with the sample 65, in particular with the volume of the sample 65, and is thereby backscattered to the scanning mirror 16 and is then directed by the scanning mirror to the beam splitter 14. At the beam splitter, the backscattered sample beam 12 and the reflected reference beam 13 are superimposed, with the path difference between the superimposed beams 12, 13 being adjusted by the movable mirror 15. The interference pattern 17 thus produced is acquired by a detector 18, for example a CCD detector or a CMOS detector.
[0069] The time-resolved OCT signal 19 thus acquired is sent from the detector 18 to the control unit 40. The control unit 40 further receives a user input regarding a desired virtual gaze direction by means of a user interface 50, preferably an HMD having means for detecting the user's head movements and / or eye movements. Alternatively or additionally, the control unit 40 receives device parameters of a connected device, e.g. the surgical microscope system 20, another imaging system or a medical instrument 70, via a device interface 55. Furthermore, the device parameters, which may come from a tracking system (not shown) connected to the device interface 55, characterize the medical instrument, e.g. its spatial position.
[0070] The control unit 40 determines the time-resolved OCT image 31 based on a predefined virtual line of sight direction 60 for the acquired time-resolved OCT signal 19 and sends the time-resolved OCT image 31 to the display means 30 for display, where the virtual line of sight direction 60 is determined by the control unit 40 based on a user input at the user interface 50 and / or based on device parameters determined by the device interface 55. For an explanation of the virtual line of sight direction 60, please refer to the description of Figures 3 and 4 below.
[0071] Figure 2 is a schematic diagram of a system 100 for acquiring and visualizing an OCT signal 19 according to a second embodiment. Components identical to those in Figure 1 are given the same reference numerals and a repeated description of these components is omitted for reasons of brevity.
[0072] The system 100 of Fig. 2 differs from the system of Fig. 1 in that it further comprises a surgical microscope system 20 having a first camera 21 having a first optical system 211 and a first image sensor 212, and a second camera 22 having a second optical system 221 and a second image sensor 222. Each of these cameras 21, 22 allows a field of view 66 of a sample 65 to be acquired along an optical axis, so that a stereoscopic image signal 23 having a first image signal 231 and a second image signal 232 is acquired. The cameras 21, 22 are, for example, two cameras of a main observer camera of a surgical microscope formed as a stereo camera.
[0073] According to the embodiment of Fig. 2, the control unit 40 determines the stereoscopic video image data 32 corresponding to the acquired stereoscopic image signal 23. A stereo angle α between the optical axes 213, 223 of the two cameras 21, 22 then determines the depth sensation or depth perception of the stereoscopic video image data 32 generated on the basis of the stereoscopic image signal 23. The stereoscopic video image data 32 then comprises a first video image data 321 corresponding to the first image signal 231 and a second video image data 322 corresponding to the second image signal 232. The stereo angle α depends on the working distance between the cameras 21, 22 and the sample 65 in addition to the distance between the cameras 21, 22.
[0074] In the system 100 of Fig. 2, the time-resolved OCT signal 19 is acquired in a similar manner as already described with reference to Fig. 1. However, the control unit 40 determines a first OCT image 311 corresponding to a first video image data 321 and a second OCT image 312 corresponding to a second video image data 321 based on the OCT signal 19. In this case, for example, the first OCT image 311 and the second OCT image 312 are generated as if they were acquired under the same stereo angle α as the first and second image signals 231, 232. However, the stereo angles of the OCT images 311, 312 can be set differently based on a user input acquired by the interface 50. The first and second video image data 321, 322 are displayed simultaneously or sequentially on the display means 30 together with the first and second OCT images 311, 312, and switching between the OCT images 311, 312 and the video image data 321, 322 is performed based on an input via the interface 50, for example.
[0075] 3 is a schematic diagram of a virtual line of sight 60, in particular a first virtual line of sight 601 and a second virtual line of sight 602. Each of these virtual line of sight 601, 602 is then defined by an azimuth angle 61 and an elevation angle 62 with respect to an optical axis 63 of the OCT system 10, the optical axis of which is preferably perpendicular to the surface of the sample 65. The azimuth angle 61 is a planar angle between 0° and 360° in a plane perpendicular to the optical axis 63. The elevation angle 62 is a planar angle between 0° and 90° in a plane containing the optical axis 63. The elevation angle is a planar angle and is measured between the virtual line of sight 601, 602 and a horizontal plane (sample surface). In the representation of FIG. 3, a first virtual line of sight 601 is defined by a first azimuth angle α1 of approximately 45° and a first elevation angle β1 of approximately 30°, and a second virtual line of sight 602 is defined by a second azimuth angle α2 of approximately 315° and a second elevation angle β2 of approximately 60°.
[0076] Fig. 4 shows a schematic visualization of the OCT signal 19 as an OCT image 31 depending on a virtual viewing direction 60. In this regard, Fig. 4(A) shows an OCT image 31 of a sample 65 (image of sample "65") from a first virtual viewing direction 601 with a first azimuth angle α1, and Fig. 4(B) shows a different OCT image 31 of the sample 65 (image of sample "65") from a second virtual viewing direction 602 with a second azimuth angle α2. For reasons of clarity, the influence of different elevation angles β1 and β2 is not shown in the figure.
[0077] 5 is a schematic diagram of a sample 65 and a display means 30 according to an embodiment, where the sample 65 comprises a number of volume elements 651 and a number of surface elements 652, where a particular volume element 651 corresponds to a particular surface element 652. An exemplary selection of the surface elements 652 is shown hatched. Also, most of the volume elements 651 are shown with dashed lines, while four volume elements 152 corresponding to the hatched surface elements 652 are shown with solid lines. Furthermore, double-sided arrows connect these volume elements 651 with the associated surface elements 652.
[0078] 1, the volume of the sample 65 in particular can be detected by the OCT system 10 of the system 100 by superimposing the short wavelength light of the sample beam 12 scattered thereby by an interferometer via the scanning mirror 16 onto the reference beam 13. The interference pattern 17 thus generated and acquired by the detector 18 as a time-resolved OCT signal 19 comprises a plurality of first tuples 191, the number of which results for example from the number of points on the sample 65 scanned by the scanning mirror 16. Each of the first tuples 191 then corresponds to one of the represented volume elements 651 and produces a scattered intensity value s i Based on the calibration or registration of the OCT system 10 to the coordinate system of the sample 65 (patient), three spatial coordinates x i ,y i ,z i is further assigned to each of the first tuples 191. In the illustrated example, the first tuple 191 has spatial coordinates x1, y1, z1 and a scattering intensity value s1.
[0079] The surgical microscope system 20 of the system 100 of Fig. 2 can be used to capture, in particular, the surface of the sample 65 by reflecting long-wavelength (visible) light reflected or backscattered from the sample 65 via the optical system 211, 221 to the image sensors 212, 222 of the cameras 21, 22. The image signals 231, 232 captured by the image sensors 212, 222 each comprise a number of second tuples 233, the number of which is due to the resolution of the associated image sensors 212, 222, respectively. Each of the second tuples 233 then corresponds to one of the represented surface elements 652 and has a grey level value g corresponding to the intensity of the light reflected back to the respective image sensor 212, 222. i Based on the calibration or registration of the image sensors 212, 222 to the coordinate system of the sample 65 (patient), two lateral spatial coordinates x i , y i is further associated with each of the second tuples 233. In the illustrated example, the second tuples 233 have horizontal spatial coordinates x1, y1 and a greyscale value g1.
[0080] The display means 30 further shown in Fig. 5 comprises a number of pixels 33, in particular 42 pixels, 7 pixels in the horizontal direction and 6 pixels in the vertical direction. In the illustrated example, the resolution of the image sensor 212 results in the acquisition of the sample surface by the image signal 23 in 21 surface elements 652, with 7 surface elements 652 in the horizontal direction and 3 surface elements 652 in the vertical direction. In the video image data 32, therefore, in the horizontal direction, one pixel 33 corresponds to one surface element 652, and in the vertical direction, two pixels 33 correspond to one surface element 652. The pixels 33 corresponding to the hatched surface elements 652 are likewise hatched and their allocation is further indicated by arrows. As further indicated by the arrows, by the control unit 40 of the system 100, an OCT image 31 of a volume element 651 corresponding to a surface element 652 is generated and displayed at a respective pixel 33 such that the video image data 32 of a particular surface element 652 is displayed at a particular pixel 33, and the OCT image 31 of the volume element 651 corresponding to the surface element 652 is likewise displayed at the particular pixel 33. In this manner, the corresponding OCT image 31 and video image data 32 are displayed at the same location on the display means 30.
[0081] A schematic diagram of the display means 30 according to a further embodiment is shown in Fig. 6. In this case, the system 100 further comprises a medical instrument 70, in particular tweezers 70, as shown in Fig. 6. During the approach phase of the tweezers 70 to the sample 65 shown in Fig. 6(A), the open tweezers 70 and the sample 65 are displayed on the display means 30 in the form of an OCT image 31 in an "en face" view, i.e. at an elevation angle of 90° in Fig. 3. As soon as the control unit 40 recognizes, based on the sensor values and / or based on the evaluation of the OCT image 31 or possibly the video image data 32, that the tweezers 70 are closed and that the perpendicular distance between the tip of the tweezers 70 and the surface of the sample 65 is less than or equal to a predefined limit value Δz1, the control unit 40 changes the display of the sample 65 and possibly the tweezers 70 on the display means 30 with the OCT image 31 from an "en face" view to another virtual viewing direction 60. In particular, the control unit 40 captures an OCT image 31 as shown in FIG. 6B, i.e., having an elevation angle of about 45°. line of sight direction 6C, the tweezers 70 are used to lift or remove a portion of the surface of the sample 65. As soon as the control unit 40 recognizes, based on the sensor values, the OCT image 31 and / or the video image data 32, that the tweezers 70 are closed and that the perpendicular distance between the tips of the tweezers 70 and the surface of the sample 65 outside the lifted or removed surface of the sample 65 (e.g. spaced a minimum distance laterally) is equal to or greater than a predefined limit value Δz2, the control unit 40 again changes the virtual viewing direction 60 of the display of the OCT image 31. As shown in FIG. 6C, this is in this case the case of a vertical distance from the lateral direction. line of sight direction 5, i.e. at an elevation angle close to 0° on the display means 30. Thereby, the interaction of the surgeon with the surgical microscope is continuously and objectively improved by an advantageous display of the acquired OCT signal 19 by improving the virtual line of sight 60 of the display based on user input acquired by the interface 50.
[0082] 7 shows a schematic flow chart of a method according to an embodiment. The method comprises a first step S100 of acquiring a time-resolved OCT signal 19 of a selected field of view 66 of a sample 65 by the OCT system 10, where the OCT signal 19 comprises a number of tuples 191 each representing a volume element 651 of the sample 65 and a corresponding scattering intensity of the volume element 651. The method further comprises a second step S200 of determining a predefined virtual line of sight 60. In a third step S300, a three-dimensional image of the sample 65 is displayed based on the tuples 191, the resolution of the display means 30 and the virtual line of sight 60. Saw The time-resolved OCT image 31 is determined as a volumetric image. Finally, in a fourth step S400, the time-resolved OCT image 31 is displayed on the display means 30. [Explanation of symbols]
[0083] 10. OCT System 11 Broadband light source 12 Sample beam 13 Reference beam 14 Beam splitter (interferometer) 15 Movable mirror (interferometer) 16 Scanning mechanism (scanning mirror) 17 Interference Patterns 18 Detector 19 Time-resolved OCT signal 191 Tuples (OCT) 20 Surgical Microscope System 21 First Camera 211 First Optical System 212 first image sensor 213 First Optical Axis 22 Second Camera 221 Second Optical System 222 Second Image Sensor 223 Second Optical Axis 23 Time-resolved image signal 231 first image signal 232 Second Image Signal 233 tuple (image signal) 30 Display means 31 OCT images 311 First OCT image 312 Second OCT Image 32 Video image data 321 first video image data 322 Second video image data 33 pixels 40 Control Unit 50 User Interface 55 Equipment Interface 60 Virtual line of sight 601 First virtual line of sight 602 Second virtual line of sight 61 Azimuth 62 elevation angle 63 Optical axis of the OCT system 65 samples 66 field of view 651 Surface Elements 652 Volume Elements 70 Medical Equipment
Claims
1. A system (100) for acquisition and visualization of OCT signals, comprising: An OCT system (10); a display means (30) configured to display a time resolved representation of the image data (32) as a stereoscopic image; a control unit (40) configured to control the OCT system (10) for acquisition of a time-resolved OCT signal (19) of a selected field of view (66) of a sample (65), to determine a time-resolved first OCT image (311) and a time-resolved second OCT image (312) based on the acquired time-resolved OCT signal (19), a predefinable virtual line of sight direction (60) and a stereo angle α, and to stereoscopically display the first OCT image (311) and the second OCT image (312) on the display means (30), The control unit (40) is configured to determine a position, type and / or status of a medical instrument (70) and / or a stage of a procedure being performed, and to determine the stereo angle α and / or the virtual line of sight (60) based on the position, type and / or status of the medical instrument (70), and / or to determine the stereo angle α and / or the virtual line of sight (60) based on the stage of the procedure being performed, in the system (100).
2. The system (100) of claim 1 , wherein the virtual line of sight (60) is defined by an azimuth angle (61) and an elevation angle (62) relative to an optical axis (63) of the OCT system (10).
3. The system (100) of claim 1, wherein the control unit (40) is configured to determine the first OCT image (311) and the second OCT image (312) as three-dimensional volumetric images viewed from the virtual line of sight (60).
4. The system (100) further comprises an interface (50) configured for obtaining a user input, and the virtual gaze direction is determined based on the user input; and / or The system (100) of claim 1, further comprising a device interface (55) configured for acquisition of device parameters, and the virtual line of sight direction is determined based on the device parameters acquired by the device interface (55).
5. the time-resolved OCT signal (19) comprises a plurality of tuples (191) each representing a volume element (651) and a scattering intensity of the sample (65); The display means (30) has a plurality of pixels (33), 2. The system (100) of claim 1, wherein the control unit (40) is configured to determine the first OCT image (311) and the second OCT image (312) based on the tuple (191), the resolution of the display means (30), and the virtual line of sight (60) such that a particular pixel (33) corresponds to a particular volume element (651).
6. a surgical microscope system (20) configured for acquisition of a time-resolved image signal (23) of the selected field of view (66) of the sample (65), The system (100) of claim 1, wherein the control unit (40) is further configured to determine corresponding video image data (32) based on the acquired time-resolved image signal (23) and to display the video image data (32) on the display means (30) simultaneously or sequentially with the first OCT image (311) and the second OCT image (312).
7. The system (100) of claim 6, wherein the control unit (40) is configured to sequentially display the video image data (32) and the first OCT image (311) and the second OCT image (312) on the display means (30) with the same magnification, the same line of sight, and / or the same stereo angle α.
8. The system (100) of claim 6, wherein the control unit (40) is configured to simultaneously display the video image data (32) at a first transparency and the first OCT image (311) and the second OCT image (312) at a second transparency on the display means (30).
9. The system (100) of claim 1, wherein the control unit (40) is configured to determine the stereo angle α based on a user input obtained by an interface (50).
10. The system (100) of claim 1, wherein the control unit (40) is configured to determine the stereo angle α based on optical parameters of a surgical microscope system (20).
11. The surgical microscope system (20) comprises a stereo camera (21, 22) having a first stereo objective lens (211) arranged along a first optical axis (213) and a second stereo objective lens (221) arranged along a second optical axis (223); the stereo angle α is the angle between the first optical axis (213) and the second optical axis (223); 11. The system (100) of claim 10, wherein the control unit (40) is further configured to determine the stereo angle α based on a zoom level and / or a working distance of the stereo cameras (21, 22).
12. A method for OCT signal acquisition and visualization, the system (100) further comprising a medical instrument (70), acquiring (S100) a time-resolved OCT signal (19) of a selected field of view (66) of a sample (65) by an OCT system (10), wherein the time-resolved OCT signal (19) comprises a number of tuples (191) each representing a volume element (651) of the sample (65) and a scattering intensity corresponding to the volume element (651); determining (S200) a position, type and / or state of the medical instrument (70) and / or a stage of the procedure being performed and determining a stereo angle α and / or a virtual line of sight direction (60) based on said position, type and / or state of the medical instrument (70) and / or determining a stereo angle α and / or a virtual line of sight direction (60) based on said stage of the procedure being performed; determining (S300) a time-resolved first OCT image (311) and a time-resolved second OCT image (312) as volumetric images in three dimensions and viewed from the virtual line of sight (60) based on the tuple (191), the resolution of the display means (30), the virtual line of sight (60), and the stereo angle α, respectively; and a step (S400) of stereoscopically displaying the first OCT image (311) and the second OCT image (312) on the display means (30).
13. The method according to claim 12, wherein the step (S200) of determining the predetermined virtual line of sight (60) is performed based on user input acquired by an interface (50) and / or based on device parameters acquired by a device interface (55).
14. A computer program comprising instructions, A computer program product comprising instructions which, when executed by a control unit (40) of a system (100) according to any one of claims 1 to 11, cause the system (100) according to any one of claims 1 to 11 to perform the method according to claim 12.
Citation Information
Patent Citations
Optical coherence tomography position indicator for ocular visualization
JP2019502434A
Surgical visualization systems
JP2020000921A
Optical coherence tomography augmented reality based surgical microscope imaging system and method
JP2022539784A
Surgical applications using integrated visualization camera and optical coherence tomography
JP2023505173A
Systems and Methods for Surgical Microscope and Optical Coherence Tomography (OCT) Imaging
US20120092615A1
