Apparatus for an optical imaging system, optical imaging system, method and computer program

Abstract

Examples relate to an apparatus for an optical imaging system comprising one or more processors and one or more storage devices. The apparatus is configured to obtain first sensor data of a first channel of the optical imaging system, obtain second sensor data of a second channel of the optical imaging system and obtain third sensor data of a third channel of the optical imaging system. The first sensor data indicates a first view a sample. The second sensor data indicates a second view of the sample. The third sensor data indicates a third view of the sample. Further, the apparatus is configured to generate first output data indicating a first stereoscopic representation of the sample and generate second output data indicating a second stereoscopic representation of the sample. The first output data is generated based on the first sensor data and the second sensor data. The second output data is generated based on the third sensor data and the second sensor data. Further, the apparatus is configured to transmit the first output data for displaying on a first display device and the second output data for displaying on a second display device.

Description

[0001] Apparatus for an Optical Imaging System, Optical Imaging System, Method and Computer Program

[0002] Technical field

[0003] Examples relate to an apparatus for an optical imaging system, an optical imaging system, a method, and a computer program.

[0004] Background

[0005] In microsurgery, display devices are often used to provide a magnified view of a sample. For example, multiple display devices may be used for multiple users. However, the user’s hand eye coordination can vary depending on their position relative to the sample. Therefore, displaying information on multiple display devices for multiple users may be challenging. Thus, there may be a desire for an improved concept for displaying information on multiple display devices.

[0006] Summary

[0007] This desire is addressed by the subject-matter of the independent claims.

[0008] The concept proposed in the present disclosure is based on the insight, that an image to be displayed to a user may depend on a position of the user relative to the sample and optionally the position of the display device. For example, multiple users can have different viewing angles on a sample, and each use an assigned display device. Thus, providing a substantially same field of view for the multiple users may require cropping or rotating images to be displayed on different display devices, which may decrease the performance and / or user experience. Therefore, different stereoscopic representations of the sample may be generated to account for different display devices. In this way, multiple images to be displayed on multiple display devices can be adjusted individually to the display device which may prevent cropping, rotation and / or underutilization of a display device area. Examples provide an apparatus for an optical imaging system comprising one or more processors and one or more storage devices. The apparatus is configured to obtain first sensor data of a first channel of the optical imaging system, to obtain second sensor data of a second channel of the optical imaging system and to obtain third sensor data of a third channel of the optical imaging system. The first sensor data indicates a first view a sample. The second sensor data indicates a second view of the sample. The third sensor data indicates a third view of the sample. Further, the apparatus is configured to generate first output data indicating a first stereoscopic representation of the sample and to generate second output data indicating a second stereoscopic representation of the sample. The first output data is generated based on the first sensor data and the second sensor data. The second output data is generated based on the third sensor data and the second sensor data. Further, the apparatus is configured to transmit the first output data for displaying on a first display device and the second output data for displaying on a second display device. Generating two different stereoscopic representations may allow to generate a stereoscopic representation for an individual display device. The difference between the channels, e.g., the first channel and the third channel, may allow to generate different stereoscopic representations for different display devices while considering different viewing angles of multiple users. In this way, the first display device and the second display device may receive specific information for displaying, such that image post-pro- cessing, such like cropping, or underutilization of a display device area can be avoided. This may enable multiple users to view substantially the same field of view, which can be adapted to the respective display device.

[0009] In an example, the first stereoscopic representation is for displaying on the first display device in a different display mode than the second stereoscopic representation on the second display device. That is, the first stereoscopic representation and the second stereoscopic representation may be different such that they are adjusted to different display modes of a display device. For example, a first user and a second user may be arranged substantially rectangular to a sample under test of a microscope. Thus, the different modes of the first display device and the second display device may account for the different viewing angles of the first user and the second user.

[0010] In an example, the apparatus may be configured to obtain position data indicating a position of the second display device and to transmit, based on the position data, the first output data for displaying on the second display device. The position data may allow to determine, for example, an orientation of the second display device. The second display device can be aligned in the same way as the first display device, for example, so that the information to be displayed can be the same. In this case, the first output data may be transmitted to the second display device for displaying the same stereoscopic representation on the first display device and the second device.

[0011] In an example, the apparatus may be configured to obtain mode data indicating a display mode of the second display device and to transmit, based on the mode data, the first sensor data for displaying on the second display device. Optionally or as an alternatively to the position data, the mode data may allow to determine, for example, an orientation of the second display device. Thus, the information to be displayed on the second display device can be adjusted accordingly based on the orientation of the second display device.

[0012] In an example, the apparatus may be configured to trigger the second display device to adjust the display mode such that the display mode is a stereoscopic portrait mode and to transmit the second output data after triggering the second display device. By triggering the adjustment of the display device, it can be ensured that the second stereoscopic representation matches the display mode of the second display device.

[0013] In an example, the apparatus may be configured to obtain user data indicating a position of a user relative to the second display device and to trigger, based on the user data, the second display device to adjust the display mode. In this way, the display mode can be adjusted based on a position of the user, e.g., when the user is at a predefined position relative to the second display device and / or the sample.

[0014] In an example, the second stereoscopic representation of the sample is a stereoscopic portrait image of the sample, and the first stereoscopic representation of the sample is a stereoscopic landscape image of the sample. That is, the first stereoscopic representation and the second stereoscopic representation may be particularly advantageous for using multiple display devices, especially when a first user and a second user have viewing angles that differ by substantially 90°.

[0015] In an example, a field of view of the first stereoscopic representation of the sample is identical to a field of view of the second stereoscopic representation of the sample. In this way, it can be ensured that multiple users using multiple display devices can see the same information. This may improve the cooperation between multiple users which use different display devices.

[0016] Examples provide an optical imaging system comprising an apparatus as described above.

[0017] In an example, the optical imaging system may comprise at most three optical imaging sensors providing information for the first channel, the second channel and the third channel. That is, the difference stereoscopic representations can be generated based on data of at exactly three different optical imaging sensors.

[0018] In an example, the at most three optical imaging sensors may be aligned in an L-shape. The L-shape can make it possible to improve the different stereoscopic representations for several users whose viewing angles differ by approximately 90°.

[0019] Examples provide a method comprising obtaining first sensor data of a first channel of the optical imaging system, the first sensor data indicating a first of view a sample and obtaining second sensor data of a second channel of the optical imaging system, the second sensor data indicating a second view of the sample. Further, the method comprises obtaining third sensor data of a third channel of the optical imaging system, the third sensor data indicating a third view of the sample. The method further comprises generating, based on the first sensor data and the second sensor data, first output data indicating a first stereoscopic representation of the sample and generating, based on the third sensor data and the second sensor data, second output data indicating a second stereoscopic representation of the sample. The method further comprises transmitting the first output data for displaying on a first display device and at least one of the first output data or the second output data for displaying on a second display device.

[0020] Various examples of the present disclosure relate to a corresponding computer program with a program code for performing the above method when the computer program is executed on a processor.

[0021] Short description of the Figures Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0022] Figs, la and lb show schematic diagrams of examples of an apparatus for an optical imaging system and of a corresponding optical imaging system;

[0023] Fig. 2 shows a schematic top view of an application scenario of an apparatus as described in Fig. 1;

[0024] Fig. 3 shows a flow chart of an example of a method for an optical imaging system; and

[0025] Fig. 4 shows a schematic diagram of a system comprising a microscope and a computer system.

[0026] Detailed Description

[0027] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and / or regions may be exaggerated for clarity.

[0028] Figs, la and lb show schematic diagrams of an example of an apparatus 130 for an optical imaging system 100 and of a corresponding optical imaging system 100 comprising the apparatus 130. The apparatus 130 is tasked with controlling various aspects of a microscope 120 of the optical imaging system 100, which may be a surgical optical imaging system, and of the entire optical imaging system 100 and / or with processing various types of sensor data of the optical imaging system 100. Consequently, the apparatus 130 may be implemented as a computer system, which interfaces with the various components of the optical imaging system 100, e.g., the optical imaging sensor 122, 124, 126. The apparatus 130 may be part of the optical imaging system 100. Alternatively, the apparatus 130 may be communicatively coupled to the optical imaging system 100. For example, the apparatus 130 may be a ready-to- use-module, that could be connected to the optical imaging system 100.

[0029] The apparatus 130 comprises, as shown in Fig. la, one or more processors 134 and one or more storage devices 136. Optionally, the apparatus 130 further comprises one or more interfaces 132. The one or more processors 134 are coupled to the one or more storage devices 136 and to the optional one or more interfaces 132. In general, the functionality of the apparatus 130 may be provided by the one or more processors 134 (e.g., for generating the first output data and the second output data), in conjunction with the one or more interfaces 132 (for exchanging information, e.g., for transmitting the first output data and the second output data) and / or with the one or more storage devices 136 (for storing and / or retrieving information).

[0030] The apparatus 130 is configured to obtain first sensor data of a first channel of the optical imaging system 100. The first channel may correspond to a first optical imaging sensor 122 of the optical imaging system 100. The first sensor data indicates a first of view a sample 110. The first sensor data may be received from the optical imaging sensor 122. Alternatively, the first sensor data may be determined by measuring, e.g., the optical imaging sensor may be part of the apparatus 130 and the apparatus may control the optical imaging sensor 122 to measure the first sensor data.

[0031] In principle, each channel of the optical imaging system 100 may correspond to a separate physical sensor. That is, each channel may correspond to a distinct optical imaging sensor or detector dedicated to capturing a specific type of signal, wavelength and / or detection angle, for example. Optionally, multiple optical imaging sensors 122, 124, 126 may be arranged in a sensor array. In this case, the sensor area may comprise multiple separated physical sensors.

[0032] The apparatus 130 is configured to obtain second sensor data of a second channel of the optical imaging system 100. The second channel may correspond to a second optical imaging sensor 124 of the optical imaging system 100. The second sensor data indicates a second view the sample 110. The second sensor data may be received from the optical imaging sensor 124. Alternatively, the second sensor data may be determined by measuring, e.g., the optical imaging sensor may be part of the apparatus 130 and the apparatus may control the optical imaging sensor 124 to measure the second sensor data.

[0033] The apparatus 130 is configured to obtain third sensor data of a third channel of the optical imaging system 100. The third channel may correspond to a third optical imaging sensor 126 of the optical imaging system 100. The third sensor data indicates a third view the sample 110. The third sensor data may be received from the optical imaging sensor 126. Alternatively, the third sensor data may be determined by measuring, e.g., the optical imaging sensor may be part of the apparatus 130 and the apparatus may control the optical imaging sensor 126 to measure the third sensor data.

[0034] The first sensor data, the second sensor data and the third sensor data can be different from each other. That is, the apparatus 130 may obtain different sensor data indicating three different views of the sample 110, i.e., the first view, the second view and the third view can be different from each other.

[0035] Further, the apparatus 130 is configured to generate first output data indicating a first stereoscopic representation of the sample 110. The first output data is generated based on the first sensor data and the second sensor data.

[0036] A stereoscopic representation may be a pair of two images, e.g., of the sample 110, which allows to perceive a three-dimensional image. For example, the pair of two images may be designed to be viewed together, each from a slightly different detection angle corresponding to the perspective of the left and right eyes of a user. The pair of two images may be perceived by a user as a single three-dimensional image with a sense of depth. The slightly different detection angles may be provided using different optical imaging sensors 122, 124, 126, for example.

[0037] Further, the apparatus 130 is configured to generate second output data indicating a second stereoscopic representation of the sample 110. The second output data is generated based on the third sensor data and the second sensor data. That is, the first stereoscopic representation is different from the second stereoscopic representation. Generating two different stereoscopic representation may allow providing different images of the sample 110 to different users. For example, the stereoscopic representation may be assigned to a specific display device 150, 152 and optionally a specific display mode, e.g., a portrait mode. In this way, different information to be displayed can be transmitted to different display devices 150, 152. Thus, the information displayed to a user using a display device 150, 152 can be adjusted, e.g., to a viewing angle of the user, such that a hand eye coordination can be improved or preserved and / or a field of view can be synchronized with a field of view of another user. The apparatus 130 is further configured to transmit the first output data for displaying on a first display device 150 and the second output data for displaying on a second display device 152. The first display device 150 may be different from the second display device 152. A display device cannot accommodate or display different orientations effectively, leading to confusing image rotations for a user, e.g., an assistant surgeon, who has a different viewing angle, e.g. a 90° offset viewing angle, than the main user, e.g., a main surgeon. Therefore, an image that is to be displayed to different users with different display devices is often cropped (for at least one display device) so that it fits multiple display devices resulting in significant loss of field of view and / or an underutilization of the display device area. It is a finding of the inventors, that a loss of information due to the use of different displays can be reduced by providing different stereoscopic representations for different display devices. The different stereoscopic representations may account for different viewing angles of different users. In this way, utilization of the display device area can be improved and / or information to be displayed can be adapted to a display mode. For example, cropping and / or rotating images for a display device to adapt to different orientations that result in a loss of field of view or underutilization of display device area can be reduced or even avoided.

[0038] As can be seen in Fig. lb, for example, two display devices 150, 152 may be coupled to or part of the optical imaging system 100. The (first) display device 150 may be assigned to a first user, e.g., a main surgeon. The (second) display device 152 may be assigned to a second user, e.g., an assistant surgeon. The (second) display device 152 can be rotated into portrait mode (not shown in Fig lb, see for example Fig. 2). In this way, the field of view of the image (i.e., the second stereoscopic representation) displayed on the (second) display device 152 can be aligned with the field of view of the image (i.e., the first stereoscopic representation) displayed on the (first) display device 150. As described in more detail below, the information to be displayed on the (second) display device 152 may depend on a condition such like a position of the (second) display device 152, a position of the second user and / or a mode of the (second) display device 152. Further, it can be ensured that the first user and the second user can view the same image of the sample 110 without losing field of view. That is, the apparatus 130 may allow to address the need for an assistant surgeon to view the same field of view as the main surgeon without confusion (e.g., reduced hand eye coordination) and / or loss of visual information (e.g., cropped field of view). This may enhance collaboration and / or efficiency during a usage of the optical imaging system 100, e.g., during a surgery. For example, the apparatus 130 can improve communication and / or collaboration between a main surgeon and an assistant surgeon by providing a consistent visual experience. Thus, the apparatus 130 may allow reduction of confusion and discomfort associated with misaligned images on separate display devices. The reduction of underutilization may allow to improve the use of display device area, maintaining high resolution and clarity.

[0039] The proposed concept may be built around two main components - the microscope 120, which comprises the optical components, and the apparatus 130, which may be used to control the optical imaging system 100, process sensor data of the microscope 120, e.g., the optical imaging sensor 122, and / or to generate the first output data and second output data.

[0040] In general, a microscope, such as the microscope 120, is an optical instrument that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope 120 may provide an optical magnification of a sample, such as a sample 110 shown in Fig. la. In modem microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensor 122 of the microscope 120. The microscope 120 may further comprise one or more optical magnification components that are used to magnify a view of the sample 110, such as an objective.

[0041] There are a variety of different types of optical imaging systems. If the optical imaging system 100 is used in the medical or biological fields, the sample 110 may be a sample of organic tissue, e.g., arranged within a petri dish or present in a part of a body of a patient. In some examples of the present disclosure, the optical imaging system 100 may be a surgical optical imaging system, e.g., an optical imaging system that is to be used during a surgical procedure, such as an oncological surgical procedure or during tumor surgery. However, the proposed concept may also be applied to other types of microscopes, e.g., a microscope in a laboratory or a microscope for the purpose of material inspection.

[0042] Fig. lb shows a schematic diagram of an example of a surgical optical imaging system 100 comprising the apparatus 130 (not shown). In general, a (surgical) optical imaging system is a system that comprises a microscope 120 and additional components, which are operated together with the microscope 120. In other words, a (surgical) optical imaging system is a system that comprises the microscope 120 (e.g., comprising the optical imaging sensors 122, 124, 126) and one or more additional components, such as the apparatus 130 (which may be a computer system being adapted to control the microscope 120 and, for example, generate the first output data), an illumination system (which is used to illuminate a sample being imaged by the microscope 120 and can be controlled by the apparatus 130), additional sensors, display devices 150, 152 etc.

[0043] The optical imaging system 100 shown in Fig. lb is coupled to or comprises two display devices 150, 152, a first display device 150 and a second display device 152. The first display device 150 may be assigned to a first user of the optical imaging system 100 and the second display device 152 may be assigned to second user of the optical imaging system 100. At least the second display device 152 may be rotatable. That is, the position of the second display device 152 can be adjusted, e.g., by rotating the second display device 152. In this way, the second display device 152 can be rotated to account for a viewing angle of the second user, such that the field of view display on both display devices 150, 152 can be preserved unchanged. This may be especially useful if the first user and the second user have viewing angles differing substantially 90° from each other.

[0044] In an example, the first stereoscopic representation is for displaying on the first display device in a different display mode than the second stereoscopic representation (for displaying) on the second display device. For example, the first display device 150 may be used in a landscape mode and the second display device 152 may be used in a portrait mode. Thus, the first stereoscopic representation may be displayed on the first display device 150 in a landscape mode and the second stereoscopic representation may be displayed on the second display device 152 in a portrait mode. That is, the apparatus 130 may generate the first stereoscopic representation according to a landscape mode and the second stereoscopic representation according to a portrait mode. Thus, the first stereoscopic representation may have an orientation where the width of the image is greater than its height. The second stereoscopic representation may have an orientation where the width of the image is smaller than its height. In this way, the field of view of the two stereoscopic representations can be adapted by the apparatus 130 to the specific display devices 150, 152 on which the information is to be displayed.

[0045] In an example, the first stereoscopic representation of the sample 110 is a stereoscopic portrait image of the sample 110, and the second stereoscopic representation of the sample 110 is a stereoscopic landscape image of the sample 110. Thus, the first stereoscopic representation and the second stereoscopic representation may be advantageous in particular when the first user and the second user are substantially perpendicular to each other (see Fig. 2). In this case, an underutilization of the display device area of both displays 150, 152 can be reduced. Further, when the first stereoscopic representation of the sample 110 is a stereoscopic portrait image, and the second stereoscopic representation of the sample 110 is a stereoscopic landscape image the first display device 150 in landscape mode and the second display device 152 in portrait mode can be utilized in an improved way.

[0046] In an example, the apparatus 130 may be configured to obtain position data indicating a position of the second display device 152 and transmit, based on the position data, the first output data for displaying on the second display device 152. A position may be an orientation, such like landscape mode or portrait mode and / or a location of the second display device 152 relative to the (assigned) user. For example, the position data may indicate that the second display device 152 is in a landscape mode. In this case, the first stereoscopic representation may fit better on the display device area than the second stereoscopic representation. Thus, the first stereoscopic representation may be transmitted for displaying on both display devices 150, 152, for example. Optionally or alternatively, the position of the user relative to the second display device 152 may be used to determine the stereoscopic representation to be displayed. For example, a first user and a second user may be close together, leading to a small difference in the viewing angles of the users on the sample. Thus, the second display device 152 could be used to mirror the first stereoscopic representation, since the viewing angles of the first user and the second user may not require a change in the display mode of the second display device 152. Optionally, the second display device 152 can still be used to display specific information to the second user even if the same stereoscopic representation is to be displayed.

[0047] Therefore, in an example the second display device 152 may receive both the first stereoscopic representation and the second stereoscopic representation and may be configured to determine which one is to be displayed. For example, a processing circuitry of the second display device 152 may determine the display mode of the second display device 152 and may select based on the display mode the first stereoscopic representation or the second stereoscopic representation for displaying.

[0048] In an example, the apparatus 130 may be configured to obtain mode data indicating a display mode of the second display device and transmit, based on the mode data, the first sensor data for displaying on the second display device. Optionally or as an alternatively to the position data, the mode data may allow to determine, for example, an orientation of the second display device. The mode data may indicate a current setting of the second display device 152. Thus, receiving information about a stereoscopic representation to be displayed (and thus to be transmitted) can be facilitated.

[0049] In an example, the apparatus 130 may be configured to trigger the second display device 152 to adjust the display mode such that the display mode is a stereoscopic portrait mode and transmit the second output data after triggering the second display device 152. The apparatus 130 may trigger the second display device by transmitting a control signal (e.g., generated by the apparatus 130) and or by controlling the second display device (e.g., when the second display device is part of the apparatus 130). Triggering the second display device 152 to set (stereoscopic) portrait mode may allow to ensure that the second display device 152 is in the correct mode for displaying the second stereoscopic representation. Therefore, transmitting the second output data after triggering the display mode may improve a user experience, since the information to be displayed is adapted to the second display device 152. Optionally, the apparatus 130 may receive acknowledgment data indicating that the second display device 152 has changed the display mode accordingly to the trigger of the apparatus 130. In this case, the apparatus 130 may transmit the second output data after receiving the acknowledgment data. That is, the apparatus 130 may control the stereoscopic representation to be displayed.

[0050] In an example, the apparatus 130 may be configured to obtain user data indicating a position of a user relative to the second display device 152 and trigger, based on the user data, the second display device 152 to adjust the display mode. Using the position of the user to trigger the adjustment of the second display device 152 may allow to improve a reliability of the adjustment of the second display device. For example, the adjustment of the display mode may only be triggered when the (second) user is at a predefined position, has a predefined viewing angle difference from another (first) user, and / or has a predefined viewing angle on the second display device 152. In this way, a user experience can be improved.

[0051] In an example, a field of view of the first stereoscopic representation of the sample 110 is identical to a field of view of the second stereoscopic representation of the sample 110. In this way, it can be ensured that multiple users using multiple display devices 150, 152 can see the same information. Thus, cooperation between multiple users can be improved. In an example, the apparatus 130 may be configured to synchronize the display output between different display devices 150, 152. For example, the apparatus 130 may trigger the content to be displayed by different display devices 150, 152. That is, the apparatus 130 may be an orchestrator of the first display device 150 and the second display device 152, such that the apparatus 130 can synchronize the information displayed. In this way, it can be ensured that different users can view a consistent field of view regardless of positions, e.g. orientations, of the display devices 150, 152. Thus, the apparatus 130 can ensure that the same image is displayed on both display devices 150, 152, preserving the field of view and / or resolution of the image of the sample 110.

[0052] In an example, the optical imaging system 100 may comprise at most three optical imaging sensors 122, 124, 126 providing information for the first channel, the second channel and the third channel. That is, the optical imaging system 100 may comprise only three optical imaging sensors 122, 124, 126, to acquire images of the sample 110 from three different viewing angles. Based on the images of the sample 110 from three different viewing angles the first stereoscopic representation of the second stereoscopic representation can be generated. Thus, the usage of at most three optical imaging sensors 122, 124, 126 may reduce costs and / or a dimension of the optical imaging system 100.

[0053] In an example, the at most three optical imaging sensors 122, 124, 126 may be aligned in an L-shape. The L-shape may allow to improve the different stereoscopic representations with respect to multiple users which have viewing angles different by substantially 90°.

[0054] As shown in Fig. la the optional one or more interfaces 132 is coupled to the respective one or more processors 134 at the apparatus 130. In examples the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the one or more processors 134 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The one or more processors 134 is capable of controlling the one or more interfaces 132, so that any data transfer that occurs over the one or more interfaces 132 and / or any interaction in which the one or more interfaces 132 may be involved may be controlled by the one or more processors 134.

[0055] In an embodiment the apparatus 130 may comprise a memory, e.g., the one or more storage devices 136 and at least one or more processors 134 operably coupled to the memory and configured to perform the method described below.

[0056] In examples the one or more interfaces 132 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g. any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The one or more interfaces 132 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.

[0057] The apparatus 130 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system.

[0058] More details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described below (e.g., Fig. 2 - 4).

[0059] Fig. 2 shows a schematic top view of an application scenario of an apparatus as described in Fig. 1. The apparatus may be part of the microscope 120 or the optical imaging system, for example. The microscope 120 may comprise three different optical imaging sensors 122, 124, 126. The three different optical imaging sensors 122, 124, 126 can be used to acquire data for generating two different stereoscopic representations of the sample 110. The two different stereoscopic representations (i.e., the first stereoscopic representation of the second stereoscopic representation) may be displayed on different display devices 250, 252. The microscope 120 is part of an optical imaging system. The first display device 250 and the second display device 252 may be part of or coupled to the optical imaging system. For example, the optical imaging system may comprise an additional imaging channel for stereoscopic visualization of different stereoscopic representations as opposed to an existing digital stereomicroscope. Using an additional imaging channel to acquire sensor data may ensure true stereoscopic viewing for both users 270, 272. For example, the first stereoscopic representation to be displayed on the first display device 250 may be generated based on the sensor data of the first optical imaging sensor 222 and the second optical imaging sensor 224. The second stereoscopic representation to be displayed on the second display device 252 may be generated based on the sensor data of the second optical imaging sensor 224 and the third optical imaging sensor 226.

[0060] At least the second display device 252 may be a rotatable display device 252. The rotatable display device 252 may be assigned to a second user 272 such like an assistant surgeon. The first display device 250 may be assigned to the first user 270 such like the main surgeon. The rotatable display device 252 may be capable of switching between different display modes, e.g., landscape mode, portrait mode, autorotation mode. Thus, the rotatable display device 252 may allow to adjust an orientation of the rotatable display device 252 such that the same field of view can be displayed correctly on both display devices 250, 252. As can be seen in Fig. 2, the rotatable display device 252 can be physically rotated, e.g., in portrait mode. In this way, both users 270, 272 can view the same image of the sample 210 with the same field of view. Physically rotating the second display device 252, e.g., into portrait mode, may allow maintaining the correct orientation and / or aspect ratio. Rotating the rotatable display device 252 into portrait mode as shown in Fig. 2 is only for illustrative purposes. The rotatable display device 252 can be rotated in every desired orientation. For example, when the second user 272 is positioned at an angle of 80° relative to the first user 270, the rotatable display device 252 may be rotated accordingly. In this case, there may be a need to crop or shrink the second stereoscopic representation. However, by rotating the rotatable display device 252 to better reflect the surgeon's natural direction of movement, a hand eye coordination of the second user 272 can be improved. For example, the second user 272 may need to make fewer mental adjustments to correctly interpret the movements of his hands. Thus, there may be a trade-off between utilization of the display device area of the second display device 252 and hand eye coordination of the second user 272, when the second user 272 is not positioned perpendicular to the first user 270 and the second display device 252 is not arranged vertically (i.e., in portrait mode). As can be seen in Fig. 2, the first user 270 may use the first display device 250 and the second user 272 may use the rotatable display device 252. Both display devices 250, 252 may display the same field of view. For example, the different optical imaging sensors 222, 224, 226 may be arranged in an L-shape configuration. The L-shape configuration may be advantageous when the rotatable display device 252 is rotated into the portrait mode. In this case, alignment of the field of view of the two different stereoscopic representations may be facilitated, particularly when the second user 272 is positioned perpendicular to the first user 270. The L- shape configuration is only for illustrative purposes. For example, the second optical imaging sensor 224 may be movable relative to the other optical imaging sensors 222, 226. In this way, an arrangement of the optical imaging sensors 222, 224, 226 can be adjusted, e.g., to a current position of the second user 272 relative to the first user 270. Thus, the stereoscopic representation can be adapted to current positions of the users 270, 272.

[0061] The rotatable display device 252 may allow the second user 272 to switch between landscape and portrait modes without interrupting the display output. In this way, the display mode of the rotatable display device 252 can be adjusted to a current position of the second user 272. For example, when the viewing angle of the second user 272 is close to the viewing angle of the first user 270, the rotatable display device 252 may be arranged horizontally. In this case, both display devices 250, 252 may display same stereoscopic representation. Optionally, the rotatable display device 252 can display specific information for the second user 272. When the viewing angle of the second user 272 deviates more from the viewing angle of the first user 270, e.g., exceeds a certain threshold, the rotatable display device 252 may be rotated to a certain orientation. The rotation may be triggered by the apparatus as described with reference to Fig. 1.

[0062] The rotatable display device 252 part of or coupled to the optical imaging system may ensure that the field of view of the second user 272 matches the field of view of the first user 270. Allowing the rotatable display device 252 to rotate into different orientation, e.g., into a vertical orientation suitable for portrait mode, may provide consistent field of view and full utilization of display device areas. In this way, collaboration and / efficiency in an operating room can be enhanced. For stereoscopic three-dimensional microscopes, the optical imaging system may include an additional imaging channel (e.g., an additional optical imaging sensor) to maintain true three-dimensional visualization. More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 2 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1) and / or below (e.g., Fig. 3 - 4).

[0063] Fig. 3 shows a flow chart of an example of a method 300 for an optical imaging system. The method 300 may be performed by an apparatus as described with reference to Fig. 1. The method 300 comprises obtaining 310 first sensor data of a first channel of the optical imaging system, the first sensor data indicating a first of view a sample and obtaining 320 second sensor data of a second channel of the optical imaging system, the second sensor data indicating a second view of the sample. Further, the method 300 comprises obtaining 330 third sensor data of a third channel of the optical imaging system, the third sensor data indicating a third view of the sample. The method 300 further comprises generating 340, based on the first sensor data and the second sensor data, first output data indicating a first stereoscopic representation of the sample and generating 350, based on the third sensor data and the second sensor data, second output data indicating a second stereoscopic representation of the sample. The method 300 further comprises transmitting 360 the first output data for displaying on a first display device and at least one of the first output data or the second output data for displaying on a second display device.

[0064] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 3 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 2) and / or below (e.g., Fig. 4).

[0065] Some embodiments relate to a microscope comprising an apparatus as described in connection with Fig. 1. Alternatively, a microscope or an optical imaging system can be communicatively connected to an apparatus as described in connection with Fig. 1. Fig. 4 shows a schematic illustration of a system 400, e.g., an optical imaging system, configured to perform a method described herein, e.g., with reference to Fig. 3. The system 400 comprises a microscope 410 and a computer system 420. The microscope may comprise the apparatus as described above, e.g., with reference to Fig. 1. The microscope 410 is configured to take images and is connected to the computer system 420. The computer system 420 is configured to execute at least a part of a method described herein. The computer system 420 may be configured to execute a machine learning algorithm. The computer system 420 and microscope 410 may be separate entities but can also be integrated together in one common housing. The computer system 420 may be part of a central processing system of the microscope 410 and / or the computer system 420 may be part of a subcomponent of the microscope 410, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 410.

[0066] The computer system 420 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 420 may comprise any circuit or combination of circuits. In one embodiment, the computer system 420 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g., camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 420 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 420 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 420 may also include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 420. More details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 4 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 3).

[0067] Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

[0068] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0069] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0070] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0071] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0072] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer. A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

[0073] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

[0074] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

[0075] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0076] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0077] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0078] If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system. The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

[0079] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.

[0080] List of reference Signs optical imaging system sample microscope , 124, 126 optical imaging sensor apparatus interface processor storage device , 152 display device sample microscope , 224, 226 optical imaging sensor first display device second display device first user second user system microscope computer system

Claims

Claims1. An apparatus (130) for an optical imaging system (100), comprising one or more processors (134) and one or more storage devices (136), wherein the apparatus (130) is configured to: obtain first sensor data of a first channel of the optical imaging system (100), the first sensor data indicating a first of view a sample (110); obtain second sensor data of a second channel of the optical imaging system (100), the second sensor data indicating a second view of the sample (110); obtain third sensor data of a third channel of the optical imaging system (100), the third sensor data indicating a third view of the sample (110); generate, based on the first sensor data and the second sensor data, first output data indicating a first stereoscopic representation of the sample (110); generate, based on the third sensor data and the second sensor data, second output data indicating a second stereoscopic representation of the sample (110); transmit the first output data for displaying on a first display device (150) and the second output data for displaying on a second display device (152).

2. The apparatus (130) according to claim 1, wherein the first stereoscopic representation is for displaying on the first display device (150) in a different display mode than the second stereoscopic representation on the second display device (152).

3. The apparatus (130) according to any one of the preceding claims, wherein the apparatus (130) is configured to: obtain position data indicating a position of the second display device (152); and transmit, based on the position data, the first output data for displaying on the second display device (152).

4. The apparatus (130) according to any one of the preceding claims, wherein the apparatus (130) is configured to obtain mode data indicating a display mode of the second display device (152); and transmit, based on the mode data, the first sensor data for displaying on the second display device (152).

5. The apparatus (130) according to any one of the preceding claims, wherein the apparatus (130) is configured to trigger the second display device (152) to adjust the display mode such that the display mode is a stereoscopic portrait mode; and transmit the second output data after triggering the second display device (152).

6. The apparatus (130) according to claim 5, wherein the apparatus (130) is configured to obtain user data indicating a position of a user (272) relative to the second display device (152); and trigger, based on the user data, the second display device (152) to adjust the display mode.

7. The apparatus (130) according to any one of the preceding claims, wherein the second stereoscopic representation of the sample (110) is a stereoscopic portrait image of the sample (110), and the first stereoscopic representation of the sample (110) is a stereoscopic landscape image of the sample (110).

8. The apparatus (130) according to any one of the preceding claims, wherein a field of view of the first stereoscopic representation of the sample (110) is identical to a field of view of the second stereoscopic representation of the sample (110).

9. An optical imaging system (100), comprising an apparatus (130) according to any one of the preceding claims 1-8.

10. The optical imaging system (100) according to claim 9, further comprising at most three optical imaging sensors (122; 124; 126) providing information for the first channel, the second channel and the third channel.

11. The optical imaging system (100) according to claim 10, wherein the at most three optical imaging sensors (122; 124; 126) are aligned in an L-shape.

12. A method (300) for an optical imaging system, comprising:obtaining (310) first sensor data of a first channel of the optical imaging system, the first sensor data indicating a first of view a sample; obtaining (320) second sensor data of a second channel of the optical imaging system, the second sensor data indicating a second view of the sample; obtaining (330) third sensor data of a third channel of the optical imaging system, the third sensor data indicating a third view of the sample; generating (340), based on the first sensor data and the second sensor data, first output data indicating a first stereoscopic representation of the sample; generating (350), based on the third sensor data and the second sensor data, second output data indicating a second stereoscopic representation of the sample; transmitting (360) the first output data for displaying on a first display device and at least one of the first output data or the second output data for displaying on a second display device.

13. A computer program with a program code for performing the method (300) according to claim 12 when the computer program is executed on a processor.

Images