Apparatus for a surgery system, surgical imaging system, navigation system, method and computer program

Abstract

Examples relate to an apparatus for a surgery system comprising one or more processors and one or more storage devices. The apparatus is configured to obtain channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system. Further, the apparatus is configured to obtain guidance data indicating a structure for guidance of a user. Further, the apparatus is configured to obtain navigation data indicating a position of a microscope of the surgical imaging system relative to a situs of the surgical imaging system and to obtain setting data indicating a setting of the surgical imaging system. The apparatus is configured to determine, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel, the first guidance image different from the second guidance image and to transmit the first guidance image and the second guidance image for displaying to the user.

Description

[0001] Apparatus for a Surgery System, Surgical Imaging System, Navigation System, Method and Computer Program

[0002] Technical field

[0003] Examples relate to an apparatus for a surgery system, a surgical imaging system, a navigation system, a method, and a computer program.

[0004] Background

[0005] Neurosurgery is characteristic by the complexity of its procedures and the need for surgeons to treat the pathology without incurring new health issue in the patient. The latter suggests the resection and removal of the least amount of healthy tissue while performing the surgical procedure (contrary to other surgeries where a significant margin of healthy tissue may be removed to prevent tumor relapse). It also requires a surgeon to have a strong ability to discriminate anatomical structures as well as healthy tissue versus the pathology. Three typical techniques can be used to support the surgeon, namely surgical planning, intra-operative navigation and microsurgery.

[0006] For example, planning a surgical procedure, such like a cranial neurosurgery, is a key step of the workflow to maximize the chance of treating the pathology successfully while minimizing risks. A biological structure can have a complex geometry, e.g., the biological structure may be a sphere-like organ such like a brain, which makes it spatially difficult to operate. The planning of the surgical procedure may depend on guidance data for guidance of a surgeon during a surgery. This means that the planning of the surgical procedure can use a technology to create the guidance data that, for example, combines different image data sources of a biological structure, co-registers them in a single reference frame and creates a 2D or 3D volume of the biological structure with a segmentation of the relevant anatomical structures (e.g., parts of the brain, pathology, skull). A surgical planning tool may allow the surgeon to plan the surgical procedure on this 2D or 3D volume of the biological structure, choosing the approach, the trajectory of the dissection and / or highlighting the most important structures to treat or preserve. In the operating room, i.e., during a surgical procedure, the guidance data may be imported in a navigation device for surgical navigation. Surgical navigation is a technology that correlates the guidance data with the actual position of the biological structure in the operating room. For example, a tripartite system with one camera and two trackers can be used to triangulate the volume of the biological structure in the operating room, allowing to match the position of the biological structure with the guidance data. Using a pointer or trackable instrument, the surgeon can identify their location in the biological structure, e.g., a patient’ s brain, on the 2D or 3D guidance data live during a surgical procedure. This helps the surgeon to keep the trajectory to the pathology as planned, while avoiding key brain structures to preserve. It also helps the surgeon to determine when the pathology is completely removed like in a tumor resection case. However, combining the guidance data with sensor data of an optical imaging sensor of a microscope can be a challenge. Thus, there may be a desire for an improved concept for image guided surgery.

[0007] Summary

[0008] This desire is addressed by the subject-matter of the independent claims.

[0009] The concept proposed in the present disclosure is based on the insight, that two different images can be generated for guidance of the user. That is, guidance data may be combined with microscope data using two different guidance images which are determined based on information about a first channel of the surgical imaging system and a second channel of the surgical imaging system. In this way, both the first channel and the second channel can correspond to or be assigned a separate guidance image for guiding a user.

[0010] Examples provide an apparatus for a surgery system comprising one or more processors and one or more storage devices. The surgery system may be a surgical imaging system or a navigation system. The apparatus is configured to obtain channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system. The first channel of the surgical imaging system may be different from the second channel of the surgical imaging system. Further, the apparatus is configured to obtain guidance data indicating a structure for guidance of a user. The guidance data may provide the user of the surgical imaging system information for guidance of the user during a surgical procedure. Further, the apparatus is configured to obtain navigation data indicating a position of a microscope of the surgical imaging system relative to a situs of the surgical imaging system and to obtain setting data indicating a setting of the surgical imaging system. The apparatus is configured to determine, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel. The first guidance image is different from the second guidance image. Further, the apparatus is configured to transmit the first guidance image and the second guidance image for displaying to the user. Determining two different guidance images may allow to provide the user information about two different views corresponding to the first channel and the second channel. That is, the user may receive stereoscopic information for guidance, which may allow to improve a guidance of the user and / or an integration with microscope data. For example, the stereoscopic information for guidance may correspond to a stereoscopic view of a microscope of the surgical imaging system. That is, the microscope data may be a stereoscopic view of a biological structure acquired using the surgical imaging system. Thus, an integration of the guidance data with a (real-time) image of the biological structure can be improved. For example, a composite image comprising the first guidance image, the second guidance image and a stereoscopic view image of a biological structure can enable an improved stereoscopic perception by the user.

[0011] In an example, the first channel may be different from the second channel in an angular disparity and / or a depth of field. Specifically, the first channel may vary from the second channel by a fixed value, such as a geometric configuration like angular disparity, or by an adjustable parameter, such as the depth of field for a particular channel. Therefore, the difference between the channels may depend on the overall design and / or the current setting of the surgical imaging system.

[0012] In an example, the guidance data may indicate a three-dimensional model of the structure for guidance. Further, the apparatus may be configured to determine the first guidance image and / or the second guidance image by rendering the three-dimensional model. For example, the apparatus may be part of a navigation system external to the surgical imaging system. The navigation system may receive the channel data and the setting data and may render the three- dimensional model of a pre-operative scan to determine the first guidance image and / or the second guidance image. In this way, the first guidance image and / or the second guidance can be matched to the first channel and / or the second channel. In an example, the channel data may indicate a geometrical configuration of the surgical imaging system. In this case, the first guidance image and the second guidance image can be determined to represent the geometric configuration of the surgical imaging system. This can be used to account for the hardware configuration of the surgical imaging system in image guided surgery.

[0013] In an example, the channel data may indicate an interpupillary distance of a user of the surgical imaging system. That is, the first guidance image and the second guidance image can be determined based on a specific characteristic of a current user. For example, different channels of the surgical imaging system may be adjusted to the specific characteristic of the user. Thus, the specific characteristic of the user may indicate a difference between the first channel and the second channel. Thus, the first guidance image and the second guidance image can be determined based on the specific characteristic of the user.

[0014] In an example, the setting data may indicate a position of a focal plane of the surgical imaging system relative to the situs. Thus, the first guidance image and / or the second guidance image may correspond to a current focal plane of the surgical imaging system. This may improve an accuracy of the integration of the guidance data and the microscope data.

[0015] In an example, the apparatus may be configured to obtain first sample data from a first optical imaging sensor of a microscope of the surgical imaging system and to obtain second sample data from a second optical imaging sensor of the microscope of the surgical imaging system. The first sample data indicates a first image of a biological structure, and the second sample data indicates a second image of the biological structure. The first image of the sample may be different from the second image of the biological structure. Further, the apparatus may be configured to generate a first composite image by combining the first image of the biological structure with the first guidance image and to generate a second composite image by combining the second image of the biological structure with the second guidance image. The apparatus may further be configured to transmit the first composite image and the second composite image for displaying to the user. In this way, the user can receive information about the guidance data and the microscope image in a stereoscopic way. The first sample data and the second sample are also referred to as microscope data. In an example, the first guidance image may be separated from the second guidance image by an angular disparity corresponding to an angular disparity between the first channel and the second channel. That is, a difference between the first image and the second image may correspond to a difference between the first channel and the second channel.

[0016] In an example, the guidance data may indicate a pre-operation image of a biological structure for surgical navigation. That is, the image guided surgery may be surgical navigation. Thus, the apparatus can be used to improve surgical navigation based on pre-operation image such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI), derivative techniques.

[0017] Embodiments relate to a surgical imaging system comprising an apparatus as described above. In an example, the surgical imaging system may comprise a microscope comprising two different channels.

[0018] Embodiments relate to a navigation system comprising an apparatus as described above.

[0019] Embodiments relate to a method for a surgical imaging system. The method comprises obtaining channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system and obtaining guidance data indicating a structure for guidance of a user. Further, the method comprises obtaining navigation data indicating a position of the surgical imaging system relative to a situs of the surgical imaging system and obtaining setting data indicating a setting of the surgical imaging system. The method further comprises determining, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel, the first guidance image different from the second guidance image and transmitting the first guidance image and the second guidance image for displaying to the user.

[0020] Various examples of the present disclosure relate to a corresponding computer program with a program code for performing the above method when the computer program is executed on a processor.

[0021] Short description of the Figures Some examples of apparatuses and / or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0022] Fig. la shows a schematic diagram of an example of an apparatus for a surgery system;

[0023] Fig. lb shows a schematic diagram of an example of a surgical imaging system;

[0024] Fig. 1c shows a schematic diagram of an example of a navigation system;

[0025] Fig. Id shows a schematic diagram of an example of a complete surgery system.

[0026] Fig. 2 shows a flow chart of an example of a method for an optical imaging system; and

[0027] Fig. 3 shows a schematic diagram of a system comprising a microscope and a computer system.

[0028] Detailed Description

[0029] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and / or regions may be exaggerated for clarity.

[0030] Fig. la shows a schematic diagram of an example of an apparatus 130 for a surgery system. The surgery system may be complete surgery system 198 for image guided surgery (see Fig. Id). A complete surgery system 198 may comprise a surgical imaging system 100 (see Fig. lb) and a navigation system 192 (see Fig. 1c). Alternatively, the surgery system may be part of a complete surgery system 198. For example, the surgery system may be a surgical imaging system 100 (see Fig. lb) or a navigation system 192 (see Fig. 1c).

[0031] The apparatus 130 may be tasked with controlling various aspects of a microscope 120 of the surgical imaging system 100 and of the complete surgery system 198 and / or with processing various types of sensor data of the surgical imaging system 100, the navigation system 192. Consequently, the apparatus 130 may be implemented as a computer system, which interfaces with the various components of the surgical imaging system 100, e.g., optical imaging sensors 122, 124, and / or the navigation system 192, e.g., the camera system 190. The apparatus 130 may be part of the surgical imaging system 100. Alternatively, the apparatus 130 may be communicatively coupled to the surgical imaging system 100. For example, the apparatus 130 may be a ready -to-use-module, that could be connected to the surgical imaging system 100 or the apparatus 100 may be part of the navigation system 192. A navigation system 192 may be a technology that provides real-time tracking and / or visualization of surgical instruments relative to a pre-operative 2D or 3D data of the patient's anatomy, enhancing precision and safety during a surgical procedure.

[0032] The apparatus 130 comprises, as shown in Fig. la, one or more processors 134 and one or more storage devices 136. Optionally, the apparatus 130 further comprises one or more interfaces 132. The one or more processors 134 are coupled to the one or more storage devices 136 and to the optional one or more interfaces 132. In general, the functionality of the apparatus 130 may be provided by the one or more processors 134 (e.g., for determining the first guidance image), in conjunction with the one or more interfaces 132 (for exchanging information, e.g., with the optical imaging sensor 122, a navigation system, a surgical imaging system) and / or with the one or more storage devices 136 (for storing and / or retrieving information).

[0033] The apparatus 130 is configured to obtain channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system. If the apparatus 130 is part of the surgical imaging system 100, the channel data may be retrieved from the storage device 136, for example. Alternatively, if the apparatus 130 is external to the surgical imaging system 100, e.g., part of a navigation system, the channel data may be received from the surgical imaging system, for example.

[0034] A channel in a surgical imaging system 100 may be a specific path that processes certain aspects of light, like different wavelengths, polarizations, or intensities. A channel may allow the surgical imaging system 100 to capture and optionally analyze distinct types of optical information separately. For example, the first channel and the second channel may be different channels which may allow to capture images of a biological structure 110 at different wavelengths, from different viewing angles and / or with different field of views. For example, a channel in a surgical imaging system can include an optical imaging sensor 122, 124. The optical imaging sensor 122, 124 can be a key part of a channel, as it is configured to detect and convert light signals into electronic data. In a surgical imaging system 100 with multiple channels, each channel might have its own optical imaging sensor 122, 124 (as shown in Fig. la) or share an optical imaging sensor while capturing different types of information (e.g., using techniques like beam splitting or multi -aperture setups to create multiple views on the same optical imaging sensor).

[0035] In an example, the first channel of the surgical imaging system 100 may be different from the second channel of the surgical imaging system 100. For example, the first channel may comprise an optical imaging sensor 122 and the second channel may comprise another (different) optical imaging sensor 124. In this case, the information about the first channel and the second channel may comprise information about an angular disparity of the optical imaging sensor 122 and the other optical imaging sensor 124, for example.

[0036] Further, the apparatus 130 is configured to obtain guidance data indicating a structure for guidance of a user. If the apparatus 130 is part of the surgical imaging system 100, the guidance data may be received from a navigation system or retrieved from a storage device 136, for example. Alternatively, if the apparatus 130 is external to the surgical imaging system 100, e.g., part of a navigation system, the guidance data may be retrieved from a storage device of the navigation system, for example.

[0037] The guidance data may provide the user of the surgical imaging system information for guidance of the user during a surgical procedure. For example, the guidance data may indicate pre-operative data. Pre-operative may refer to the imaging and / or information obtained before a surgical procedure that aids in accurately planning and guiding the surgical procedure. Preoperative data may be a stack of 2D images and / or a 3D volume, for example. Pre-operative data may include detailed anatomical images from modalities like CT or MRI, which may help create a precise map of the surgical site. By using this information, a navigation system 192 can track instruments in real time, allowing for better accuracy and safety during the surgical procedure.

[0038] Optionally or alternatively, the guidance data may comprise ultrasound data. Ultrasound may provide real-time imaging, which may allow for immediate visualization of the anatomy and any changes that may occur during the surgical procedure. This can be particularly useful for guiding surgical decisions and interventions based on the current state of the tissue, rather than relying on pre-operative imaging data. Thus, the image guided surgery may be a surgical navigation and / or an intraoperative ultrasound.

[0039] The structure for guidance indicated by the guidance data may pertain to the removal of a part of a biological structure, such as a tumor. The structure could include the boundaries of the part to be removed to define its extent, critical surrounding structures to avoid, and / or a 3D coordinate system to map both the part to be removed and surrounding anatomy. The structure may also comprise segmentation information to differentiate between the part to be removed and healthy tissue that should not be removed, along with optional orientation data regarding the position and / or angle of a surgical instrument relative to the part to be removed. In this way, precise navigation during the surgical procedure can be facilitated.

[0040] However, the precise navigation during the surgical procedure may rely on an actual position of the surgical imaging system 100. For example, navigation data may allow to accurately combine or correlate the guidance data with other data during the surgical procedure, e.g., microscope data from the microscope 120 comprising a (stereoscopic) view of the biological structure. This integration may enhance the ability of the surgeon to use the guidance data.

[0041] Thus, the apparatus 130 is configured to obtain navigation data indicating a position of a microscope 120 of the surgical imaging system 100 relative to a situs of the surgical imaging system 100. If the apparatus 130 is part of the surgical imaging system 100, the navigation data may be received from a navigation system. Alternatively, if the apparatus 130 is external to the surgical imaging system 100, e.g., part of a navigation system, the navigation data may be measured, e.g., using a camera system 190 (see Fig. 1c and Fig. Id).

[0042] The situs may refer to a general anatomical location or region of the body of a patient where a surgical procedure is taking place. It may designate the broader area of interest, such as the abdomen, thorax, or cranial cavity, without focusing on the specific tissue or biological structure being directly operated on. The situs may comprise an operative field. Thus, the navigation data may indicate a position of a microscope 120 of the surgical imaging system 100 relative to an operative field of the surgical imaging system 100, for example. The operative field of the surgical imaging system 100 may refer to a specific area or volume within a body of a patient comprising the biological structure, that is being visualized or acquired during a surgical procedure. The operative field may encompass the biological structure. The biological structure may be a human tissue, an organ, a cell, a structure (e.g., a bone, an artery) that is directly involved in the surgical procedure, for example. The structure part of the guidance data may be the biological structure encompassed by the operative field. In this way, the guidance data can be correlated to an actual position of the surgical imaging system 100.

[0043] Further, the precise navigation during the surgical procedure may rely on an actual setting of the surgical imaging system 100. For example, information about the setting can be used in conjunction with information about a relative position to determine the field of view of the microscope 120 relative to a body of a patient, e.g., a head 110. That is, the navigation data can be combined with setting data to improve image guided surgery.

[0044] Therefore, the apparatus 130 is configured to obtain setting data indicating a setting of the surgical imaging system 100. If the apparatus 130 is part of the surgical imaging system 100, the setting data may be retrieved from a storage device 136, for example. Alternatively, if the apparatus 130 is external to the surgical imaging system 100, e.g., part of a navigation system, the setting data may be received from the surgical imaging system 100, for example.

[0045] The setting of the surgical imaging system 100 may involve specific condition and / or parameter configured for a surgical procedure. The setting may include the type of imaging modality being used, an adjustment to the imaging equipment to accommodate the patient’s position, calibration of the surgical imaging system 100, for example.

[0046] The channel data in conjunction with the navigation data and the setting data may allow to determine different images for guidance to improve an image guided surgery. For example, different images can be determined to take into account a geometric configuration required for a stereoscopic view of the biological structure. Therefore, the apparatus 130 is configured to determine, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel. The first guidance image is different from the second guidance image. That is, the first guidance image and the second guidance image may provide two different views of the structure for guidance of the user. Thus, the first guidance image and the second guidance image may allow to provide the user a stereoscopic view of the structure for guidance of the user. In this way, a stereoscopic view acquired using the surgical imaging system 100 can be combined with a stereoscopic view of the structure for guidance of the user. Thus, an integration of guidance data, e.g., pre-operative data, with microscope data, e.g., real-time image data of the surgical imaging system 100, can be facilitated.

[0047] Further, the apparatus 130 is configured to transmit the first guidance image and the second guidance image for displaying to the user. For example, the first guidance image and the second guidance image may be transmitted from the navigation system 192 to the surgical imaging system 100 for generating a composite image by combining with microscope data, as described below.

[0048] The apparatus 130 may be used, for example, for cranial neurosurgery. Cranial neurosurgery planning may mainly depend on pre-operative data, such as medical imaging data (e.g., CT, MRI, and derivative techniques). The pre-operative data may comprise a stack of 2D images, which can be interpolated into 3D volumes. For example, surgical planning as described above may be used for the cranial neurosurgery. The apparatus 130 can be used in neurosurgery, e.g., cranial neurosurgery, but is not restricted to this. The apparatus 130 can also be used in spinal surgery or orthopedic surgery, for example.

[0049] The apparatus 130 may enable a user, e.g., a surgeon, to integrate microscope data (e.g., a view of the biological structure) and guidance data together during a surgical procedure in an improved way. This may enhance a workflow and / or cognitive advantage, while supporting the decision making. A surgeon operates while maintaining a high focus on the microsurgical image, which guides visualization of the surgical situs, e.g., comprising the operative field, and surgical gestures. If the guidance data is not integrated with the microscope data a surgeon may sporadically look at guidance data, while using a trackable instrument, to correlate their position onto the guidance data, e.g., digital pre-operative data of the patient. This leads to a complex 2D-visualization modality situation where the surgeon must constantly shift attention from one device to the other. In response to this issue, navigation system and surgical imaging system have been configured to provide integration possibilities, where the surgeon is able to input guidance data onto the microscope visualization. By tracking the position of the microscope, the navigation system is able to locate its plane of focus and to correlate it with the pre-operative images of the anatomy corresponding to the situs. The surgeon is in turn able to operate on the situs while visualizing which important structures are directly in front of, or below the surface the surgeon is operating on. However, the volumes sent from the navigation system to the microscope do at best correlate to the plane of focus. It is a finding of the inventors, that an integration for image guided surgery can be improved by determining two different images for guidance of the user (the first guidance image and the second guidance image), which correspond to two channels of the surgical imaging system 100. The two different images for guidance of the user may allow to provide a stereoscopic view of the structure for guidance of the user. In this way, a stereoscopic view of the structure for guidance of the user can be combined with a stereoscopic image of the microscope 120 of the surgical imaging system 100. Thus, the apparatus 130 may allow the surgeon to see the situs or the biological structure in 3D through the oculars or on a display device, combined with guidance data in 3D shapes. Awkward floating effect of the guidance data, e.g., caused by an overlay of 3D microscope data with 2D guidance data can be avoided. The overlay of the 3D microscope data with 2D guidance data provides an unsatisfactory feeling and confuses the surgeon on how to correlate the guidance data and the microscope data, taking away the benefit this technology was set to bring. The apparatus 130 may allow for a seamless merging of the (real-time 3D) microscope data, e.g., a surgical image, with the guidance data, e.g., a pre-operative model. In this way, image guided surgery can be improved.

[0050] The apparatus 130 may allow to combine guidance data in 3D with an stereoscopic view of the biological structure. For example, the guidance data may allow a volume overlay in 3D onto a surgical image (acquired using the surgical imaging system 100), while maintaining the accuracy of the overlay. That is, the apparatus 130 may allow to determine two different images for guidance, which could be displayed accordingly onto two image channels to correlate the stereoscopic view of the surgical situs or biological structure with the topography of the guidance data, e.g., a pre-operative volume.

[0051] The proposed concept may be built around two main components - the microscope 120, which comprises the optical components, and the apparatus 130, which may be used to control the surgical imaging system 100 and / or the navigation system, process sensor data of the microscope 120, e.g., the optical imaging sensors 122, 124, the guidance data of the navigation system and / or to generate the first guidance image and the second guidance image.

[0052] In general, a microscope, such as the microscope 120, is an optical instrument that is suitable for examining objects that are too small to be examined by the human eye (alone). For example, a microscope 120 may provide an optical magnification of a biological structure. In modern microscopes, the optical magnification is often provided for a camera or an imaging sensor, such as the optical imaging sensors 122, 124 of the microscope 120. The microscope 120 may further comprise one or more optical magnification components that are used to magnify a view of the biological structure 110, such as an objective.

[0053] Fig. lb shows a schematic diagram of an example of a surgical imaging system 100. The surgical imaging system 100 comprises the microscope 120 and may comprise the apparatus 130. In general, a surgical imaging system is a system that comprises a microscope 120 and additional components, which are operated together with the microscope 120. In other words, a surgical imaging system is a system that comprises the microscope 120 and one or more additional components, such as the apparatus 130 (which may be a computer system being adapted to control the microscope 120), an illumination system (which is used to illuminate a biological structure being imaged by the microscope 120 and can be controlled by the apparatus 130), additional sensors, displays etc.

[0054] A neurosurgeon often uses a surgical microscope 120 during a part of a surgical procedure. Such segment of a surgical procedure is also referred to as microsurgery. Surgical microscopes 120 primarily support surgeons by providing a high amount of light into and a magnified image of the surgical situs. Operating on a magnified, crisp and contrasted image helps the neurosurgeon better discriminate the biological structures they are operating on as well as perform very fine surgical gestures. A neurosurgeon can sometimes operate with a surgical microscope 120 for hours uninterrupted. The microscope 120 may be a microscope, which provides an optical image to the user through oculars 140, 145 (and optional display device such as the 3D display device 180) or an exoscope, which provides a 3D digital image to the user, visualized on a 3D monitor or on a VR headset.

[0055] The surgical imaging system 100 shown in Fig. lb comprises a number of optional components, such as a base unit 105 (which may comprise the apparatus 130) with a stand, ocular displays 140, 145 that are arranged at the microscope 120, a 3D display device 180, and a (robotic or manual) arm 160 which holds the microscope 120 in place, and which is coupled to the base unit 105 and to the microscope 120. In general, these optional and non-optional components may be coupled to the apparatus 130, which may be configured to control and / or interact with the respective components. Fig. 1c shows a schematic diagram of an example of a navigation system 192. The navigation system 192 may comprise the apparatus 130. The navigation system 192 comprises a camera system 190 configured to measure the navigation data.

[0056] Fig. Id shows a schematic diagram of an example of a complete surgery system 198 comprising a surgical imaging system 100, a navigation system 192 and an apparatus (not shown). The apparatus may be part of the imaging system 100, the navigation system 100 or external, e.g., a ready-to-use-module. The navigation system 192 is communicatively coupled with the surgical imaging system 100. That is information such as navigation data, setting data can be shared between the navigation system 192 and the surgical imaging system 100. The apparatus can be part of either of the navigation system 192 or the surgical imaging system 100 and may receive information to determine the first image of guidance and the second image of guidance from the other component, as described above. That is, the first image of guidance and the second image of guidance can be determined by the navigation system 192, the surgical imaging system 100 or an external module (comprising the apparatus 130).

[0057] For example, the navigation system 192 may be configured to obtain (e.g., retrieve from a storage device or generate based on a stack of 2D images) the guidance data, e.g., a 3D volume model. The 3D volume model may comprise voxels representing the biological structure, e.g., a brain of a patient. The 3D volume model may be created from pre-operative imaging data, such as CT, MRI, for example.

[0058] Further, the navigation system 192 may be configured to obtain the navigation data (measure using the camera system 190). The navigation data may indicate the physical location in real time of the patient (i.e., the biological structure), the microscope of the surgical imaging system, e.g., a focal plane (i.e., the physical plane where the microscope is focusing on, representing the surgical situs that the surgeon is visualizing with the microscope) and optional a trackable instrument.

[0059] Further, the navigation system 192 may be configured to obtain the channel data (e.g., receive from the surgical imaging system 100 or determine a type of the surgical imaging system 100 and retrieve from a storage device based on the determined type). Further the navigation system 192 may receive setting data, e.g., from the surgical imaging system 100. The channel data may indicate a difference between the two channels of the surgical imaging system 100.

[0060] The navigation system 192 may be configured to determine or create two guidance images (i.e., the first guidance image and the second guidance image) based on the navigation data, the channel data, the setting data and the guidance data. The two guidance images may be separated by a distance corresponding to the distance between the two images of the microscope of the surgical imaging system 100. In this way, a stereoscopic impression of the guidance data can be created. The navigation system 192 may correlate two guidance images with the image of the microscope 120 in such way that the anatomical volume would be accurately overlayed onto the surgical image (with both stereoscopic channels correctly aligned). The navigation system 192 may be configured to transmit the two guidance images for displaying to the user. For example, the two guidance images may be transmitted to the surgical imaging system 100 or to a display device external to the surgical imaging system. Optionally or alternatively, the navigation system 192 may transmit the two guidance images to a display device 194 part of the navigation system 122.

[0061] If the surgical imaging system 100 receives the two guidance images from the navigation system 192, the surgical imaging system 100 may combine each guidance image with one of the channels of the stereoscopic image of the microscope 120 of the surgical imaging system 100. The stereoscopic image may be acquired by optical imaging sensors 122, 124 part of the surgical imaging system 100. The surgical imaging system 100 may transmit the combined image for displaying on the display device, e.g., the 3D display device 180 part of the surgical imaging system 100.

[0062] The described example of the generation of the combined images (also referred to as composite image) is for illustrative purposes only. For example, the process can be fully processed by an apparatus part of surgical imaging system 100. In this case, the surgical imaging system 100 may receive the guidance data from the navigation system 192, for example. Or likewise the navigation system 192 can generate the combined images. In this case, the navigation system 192 may receive the microscope data (i.e., the first sample data and the second sample data) indicating a first image of the biological structure and the second image of the biological structure from the surgical imaging system 100. That is, the process described above may be performed in whole or in part by an apparatus part of the surgical imaging system 100 or the navigation system 192 or in cooperation.

[0063] In an example, the first channel may be different from the second channel in an angular disparity and / or a depth of field. That is, the first channel and the second channel may provide a user of the surgical imaging system 100 a different view of the biological structure, e.g., to provide a stereoscopic view. The stereoscopic view, i.e., a difference between a first microscope image of the biological structure and a second microscope image of the biological structure, may depend on the difference between the first channel and the second channel. Thus, the apparatus 130 may determine a first guidance image and a second guidance image corresponding to the difference between the first channel and the second channel. In this way, information part of the guidance data to be displayed to the user can be adjusted to an actual difference of the first channel and the second channel. That is, a stereoscopic view of the microscope 120 can be matched with a stereoscopic view of the guidance data. This may improve an integration of different data for image guided surgery.

[0064] In an example, the guidance data may indicate a three-dimensional model of the structure for guidance. Further, the apparatus 130 may be configured to determine the first guidance image and / or the second guidance image by rendering the three-dimensional model. For example, the apparatus 130 may be part of the navigation system 192. Therefore, the navigation system may retrieve the 3D model of the structure for guidance from a storage device. Further, the apparatus 130 may receive the setting data from the surgical imaging system 100 and may determine the first guidance image and / or the second guidance image by rendering. This may improve a flexibility, since images from various angles, perspectives, and lighting conditions can be rendered. Thus, rendering the first guidance and / or the second guidance image may allow for better understanding and analysis of complex structures and / or a better match between microscope data and guidance data. This may be particularly valuable in fields like surgery, where 3D models of anatomy help in pre-operative planning and surgical navigation.

[0065] In an example, the channel data may indicate a geometrical configuration of the surgical imaging system. The geometrical configuration of a surgical imaging system 100 may refer to the spatial arrangement and positioning of its components, including optical imaging sensors, light sources, and the patient relative to the system. It may define how these elements are aligned, the angles between them, and their distances from each other. For example, the geometrical configuration may affect or define factors like a field of view, a depth of field, an angular disparity, affecting how images are captured and visualized. Thus, the channel data may allow to reflect the true spatial relationships in the operative field.

[0066] In an example, the channel data may indicate an interpupillary distance of a user of the surgical imaging system 100. For instance, different channels of the surgical imaging system 100 may be adjusted to accommodate the user's interpupillary distance. Consequently, the user's specific characteristic may define the difference between the first and second channels, allowing the first guidance image and the second guidance image to be tailored accordingly. For example, the navigation system 192 may determine the interpupillary distance of the user using the camera system 190. Thus, the channel data can be obtained by measuring with the camera system 190.

[0067] In an example, the setting data may indicate a position of a focal plane of the surgical imaging system relative to the situs. Knowing the focal plane my provide the possibility of aligning the surgical imaging system with the exact anatomical structures that are critical to the surgical procedure. This may allow for more precise integration of guidance data with the microscope data, ensuring that guidance is accurate at the depth where the surgery is focused. Additionally, it may enhance the clarity of the visualized structure, helping the surgeon maintain precision and / or avoid errors when working on targeted areas or structures.

[0068] In an example, the apparatus 130 may be configured to obtain first sample data from a first optical imaging sensor of a microscope 120 of the surgical imaging system 100 and to obtain second sample data from a second optical imaging sensor of the microscope 120 of the surgical imaging system 100. The first sample data indicates a first image of a biological structure and the second sample data indicates a second image of the biological structure. The first image of the biological structure may be different from the second image of the biological structure. Further, the apparatus 130 may be configured to generate a first composite image by combining the first image of the biological structure with the first guidance image and to generate a second composite image by combining the second image of the biological structure with the second guidance image. The apparatus 130 may further be configured to transmit the first composite image and the second composite image for displaying to the user. Thus, microscope data, i.e., the first image of the biological structure and the second image of the biological structure, can be combined with the guidance data to be displayed on a display device. For example, the microscope data can be overlaid with the guidance data. This may enhance a situational awareness and may allow for more intuitive decision-making during the surgical procedure without needing to switch between different display devices. An alternative to overlaying the guidance data with the microscope data could be using augmented reality systems. In this case, the guidance data could be projected onto the operative field in real time, allowing the surgeon to visualize anatomical structures and navigation data directly within their line of sight.

[0069] In an example, the first guidance image may be separated from the second guidance image by an angular disparity corresponding to an angular disparity between the first channel and the second channel. That is, a difference between the first image and the second image may correspond to a difference between the first channel and the second channel. The difference between the first channel and the second channel could be fixed by a geometrical configuration of the surgical imaging system 100.

[0070] In an example, the guidance data may indicate a pre-operation image of a biological structure for surgical navigation. That is, the image guided surgery may be surgical navigation. Thus, the apparatus 130 can be used to improve surgical navigation, e.g., for cranial neurosurgery.

[0071] As shown in Fig. la the optional one or more interfaces 132 is coupled to the respective one or more processors 134 at the apparatus 130. In examples the one or more processors 134 may be implemented using one or more processing units, one or more processing devices, any means for processing, such as a processor, a computer or a programmable hardware component being operable with accordingly adapted software. Similar, the described functions of the one or more processors 134 may as well be implemented in software, which is then executed on one or more programmable hardware components. Such hardware components may comprise a general-purpose processor, a Digital Signal Processor (DSP), a micro-controller, etc. The one or more processors 134 is capable of controlling the one or more interfaces 132, so that any data transfer that occurs over the one or more interfaces 132 and / or any interaction in which the one or more interfaces 132 may be involved may be controlled by the one or more processors 134. In an embodiment the apparatus 130 may comprise a memory, e.g., the one or more storage devices 136 and at least one or more processors 134 operably coupled to the memory and configured to perform the method described below.

[0072] In examples the one or more interfaces 132 may correspond to any means for obtaining, receiving, transmitting or providing analog or digital signals or information, e.g., any connector, contact, pin, register, input port, output port, conductor, lane, etc. which allows providing or obtaining a signal or information. The one or more interfaces 132 may be wireless or wireline and it may be configured to communicate, e.g., transmit or receive signals, information with further internal or external components.

[0073] The apparatus 130 may be a computer, processor, control unit, (field) programmable logic array ((F)PLA), (field) programmable gate array ((F)PGA), graphics processor unit (GPU), application-specific integrated circuit (ASICs), integrated circuits (IC) or system-on-a-chip (SoCs) system. The apparatus 130 may be part of the surgical imaging device 100. Alternatively, the apparatus 130 may be part of the navigation system 192.

[0074] More details and aspects are mentioned in connection with the examples described below. The example shown in Fig. 1 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described below (e.g., Fig. 2 - 3).

[0075] Fig. 2 shows a flow chart of an example of a method 200 for an optical imaging system. The method 200 may be performed by an apparatus as described above, e.g., with reference to Fig. 1. The method 200 comprises obtaining 210 channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system and obtaining 220 guidance data indicating a structure for guidance of a user. Further, the method 200 comprises obtaining 230 navigation data indicating a position of the surgical imaging system relative to a situs of the surgical imaging system and obtaining 240 setting data indicating a setting of the surgical imaging system. The method 200 further comprises determining 250, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel, the first guidance image different from the second guidance image and transmitting 260 the first guidance image and the second guidance image for displaying to the user.

[0076] More details and aspects are mentioned in connection with the examples described above and / or below. The example shown in Fig. 2 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1) and / or below (e.g., Fig. 3).

[0077] Some embodiments relate to a microscope comprising an apparatus as described in connection with Fig. 1. Alternatively, a microscope or a surgical imaging system can be communicatively connected to an apparatus as described in connection with Fig. 1. Fig. 3 shows a schematic illustration of a system 300, e.g., a surgical imaging system, configured to perform a method described herein, e.g., with reference to Fig. 2. The system 300 comprises a microscope 310 and a computer system 320. The microscope may comprise the apparatus as described above, e.g., with reference to Fig. 1. The microscope 310 is configured to take images and is connected to the computer system 320. The computer system 320 is configured to execute at least a part of a method described herein. The computer system 320 may be configured to execute a machine learning algorithm. The computer system 320 and microscope 310 may be separate entities but can also be integrated together in one common housing. The computer system 320 may be part of a central processing system of the microscope 310 and / or the computer system 320 may be part of a subcomponent of the microscope 310, such as a sensor, an actor, a camera or an illumination unit, etc. of the microscope 310.

[0078] The computer system 320 may be a local computer device (e.g., personal computer, laptop, tablet computer or mobile phone) with one or more processors and one or more storage devices or may be a distributed computer system (e.g., a cloud computing system with one or more processors and one or more storage devices distributed at various locations, for example, at a local client and / or one or more remote server farms and / or data centers). The computer system 320 may comprise any circuit or combination of circuits. In one embodiment, the computer system 320 may include one or more processors which can be of any type. As used herein, processor may mean any type of computational circuit, such as but not limited to a microprocessor, a microcontroller, a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a graphics processor, a digital signal processor (DSP), multiple core processor, a field programmable gate array (FPGA), for example, of a microscope or a microscope component (e.g., camera) or any other type of processor or processing circuit. Other types of circuits that may be included in the computer system 320 may be a custom circuit, an application-specific integrated circuit (ASIC), or the like, such as, for example, one or more circuits (such as a communication circuit) for use in wireless devices like mobile telephones, tablet computers, laptop computers, two-way radios, and similar electronic systems. The computer system 320 may include one or more storage devices, which may include one or more memory elements suitable to the particular application, such as a main memory in the form of random access memory (RAM), one or more hard drives, and / or one or more drives that handle removable media such as compact disks (CD), flash memory cards, digital video disk (DVD), and the like. The computer system 320 may also include a display device, one or more speakers, and a keyboard and / or controller, which can include a mouse, trackball, touch screen, voice-recognition device, or any other device that permits a system user to input information into and receive information from the computer system 320.

[0079] More details and aspects are mentioned in connection with the examples described above. The example shown in Fig. 3 may comprise one or more optional or additional features corresponding to one or more aspects mentioned in connection with the proposed concept or one or more examples described above (e.g., Fig. 1 - 2).

[0080] Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a processor, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.

[0081] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a non- transitory storage medium such as a digital storage medium, for example a floppy disc, a DVD, a Blu-Ray, a CD, a ROM, a PROM, and EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable. Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0082] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0083] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0084] In other words, an embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0085] A further embodiment of the present invention is, therefore, a storage medium (or a data carrier, or a computer-readable medium) comprising, stored thereon, the computer program for performing one of the methods described herein when it is performed by a processor. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary. A further embodiment of the present invention is an apparatus as described herein comprising a processor and the storage medium.

[0086] A further embodiment of the invention is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example, via the internet.

[0087] A further embodiment comprises a processing means, for example, a computer or a programmable logic device, configured to, or adapted to, perform one of the methods described herein.

[0088] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein. A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0089] In some embodiments, a programmable logic device (for example, a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0090] If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method and vice versa. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.

[0091] The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.

[0092] The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example. List of reference Signs

[0093] 100 surgical imaging system

[0094] 105 base

[0095] 110 biological structure

[0096] 120 microscope

[0097] 122 sensor

[0098] 130 apparatus

[0099] 132 interface

[0100] 134 processor

[0101] 136 storage device

[0102] 140, 145 ocular display

[0103] 160 arm

[0104] 180 display device

[0105] 190 camera system

[0106] 192 navigation system

[0107] 194 display device

[0108] 198 complete surgery system

[0109] 200 method for a surgical imaging system

[0110] 210 obtaining sensor data

[0111] 220 obtaining guidance data

[0112] 230 obtain irrigation data

[0113] 240 obtaining setting data

[0114] 250 determining guidance images

[0115] 260 transmitting guidance images

[0116] 300 system

[0117] 310 microscope

[0118] 320 computer system

Claims

Claims1. An apparatus (130) for a surgery system (100), comprising one or more processors (134) and one or more storage devices (136), wherein the apparatus (130) is configured to: obtain channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system; obtain guidance data indicating a structure for guidance of a user; obtain navigation data indicating a position of a microscope of the surgical imaging system relative to a situs of the surgical imaging system; obtain setting data indicating a setting of the surgical imaging system; determine, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel, the first guidance image different from the second guidance image; and transmit the first guidance image and the second guidance image for displaying to the user.

2. The apparatus according to claim 1, wherein the first channel is different from the second channel in at least one of an angular disparity, and a depth of field.

3. The apparatus according to any one of the preceding claims, wherein the guidance data indicates a three-dimensional model of the structure for guidance, and the apparatus is configured to determine at least one of the first guidance image and the second guidance image by rendering the three-dimensional model.

4. The apparatus according to any one of the preceding claims, wherein the channel data indicates a geometrical configuration of the surgical imaging system.

5. The apparatus according to any one of the preceding claims, wherein the channel data indicates an interpupillary distance of a user of the surgical imaging system.

6. The apparatus according to any one of the preceding claims, wherein the setting data indicates a position of a focal plane of the surgical imaging system relative to the situs.

7. The apparatus according to any one of the preceding claims, wherein the apparatus is configured to: obtain first sample data from a first optical imaging sensor (122) of a microscope of the surgical imaging system, the first sample data indicating a first image of a biological structure (110); obtain second sample data from a second optical imaging sensor (124) of the microscope of the surgical imaging system, the second sample data indicating a second image of the biological structure (110), the first image of the sample (110) different from the second image of the sample (110); generate a first composite image by combining the first image of the biological structure (110) with the first guidance image; generate a second composite image by combining the second image of the biological structure (110) with the second guidance image; and transmit the first composite image and the second composite image for displaying to the user.

8. The apparatus according to any one of the preceding claims, wherein the first guidance image is separated from the second guidance image by an angular disparity corresponding to an angular disparity between the first channel and the second channel.

9. The apparatus according to any one of the preceding claims, whereinthe guidance data indicates a pre-operation image of a biological structure for surgical navigation.

10. A surgical imaging system (100), comprising an apparatus (130) according to any one of the preceding claims.

11. The surgical imaging system (100), further comprising a microscope, wherein the microscope comprises two different channels.

12. A navigation system (100), comprising an apparatus (130) according to any one of claims 1-9.

13. A method for a surgical imaging system, comprising: obtaining channel data indicating information about a first channel of the surgical imaging system and a second channel of the surgical imaging system; obtaining guidance data indicating a structure for guidance of a user; obtaining navigation data indicating a position of the surgical imaging system relative to a situs of the surgical imaging system; obtaining setting data indicating a setting of the surgical imaging system; determining, based on the guidance data, the channel data, the navigation data and the setting data, a first guidance image corresponding to the first channel and a second guidance image corresponding to the second channel, the first guidance image different from the second guidance image; and transmitting the first guidance image and the second guidance image for displaying to the user.

14. A computer program with a program code for performing the method according to claim 13 when the computer program is executed on a processor.

Images