AUTOMATED REGISTRATION OF PREOPERATIVE VOLUME IMAGE DATA USING SEARCH IMAGE

Description

TECHNICAL AREA

[0001] Several examples concern techniques for acquiring depth-resolution measurement data for a target region, whereby a transformation rule can be determined between image data acquired by means of a robotic visualization system and preoperative volume image data using the depth-resolution measurement data. BACKGROUND

[0002] For example, in order to be able to overlay preoperative volume image data (e.g., computed tomography, CT, or magnetic resonance tomography, MRI) onto the patient's anatomy with pinpoint accuracy during the operation for navigation applications in microsurgery, it is necessary to register the preoperative image data to the patient's position in the operating room; that is, a transformation rule is determined that describes how preoperative image data must be transformed in order to be overlaid onto the patient with pinpoint accuracy.

[0003] To determine the transformation formula, corresponding points in the preoperative volumetric imaging data and of the patient in the operating room are required. The transformation formula maps images acquired using a robotic visualization system in the operating room onto preoperative volumetric imaging data, and / or vice versa.

[0004] In some examples, physical markers are attached to the patient, which are visible in both the preoperative volumetric imaging data and the image data of a visualization system in the operating room. A disadvantage of this approach is that the physical markers must be attached to the patient, which is error-prone and time-consuming. Furthermore, it is necessary to attach the physical markers both for the acquisition of the preoperative volumetric imaging data and before the actual operation, either to the patient or in a fixed position relative to the patient.

[0005] Other examples employ a markerless approach. Here, the surface geometry (anatomical topography) of a target region of the patient is visible both in measurement data acquired in the operating room, which has depth resolution, and in preoperative image data. For example, significant points of the skull geometry (ear base, cheeks, nose, etc.) can be touched with a handheld pointer instrument, and the corresponding measurement points (defined in three-dimensional space) from the acquired data can be transferred to the navigation system. This results in a sparse scan of the skull surface, which is registered against the surface acquired from a preoperative dataset. Such techniques require additional hardware and involve a certain amount of time to acquire the measurement data by operating the pointer instrument.

[0006] To capture the surface more comprehensively, quickly, and without additional hardware, methods have been proposed that utilize stereoscopic image data from a robotic visualization system (e.g., a surgical microscope) to determine the surface of the patient's head. See, for example, US7561733B2 or DE102014210051A1. However, these methods require that the robotic visualization system is already aligned with the patient so that the stereoscopic image data depicts the target region, enabling registration. This is not typically the case, as this information would only be available after patient registration. Therefore, manual alignment may be necessary, which is time-consuming and prone to errors.

[0007] US Patent 5,795,294 A concerns a method for correlating different coordinate systems in computer-assisted stereotactic surgery. In this method, topographical partial information is acquired without additional reference markers using an operating microscope mounted on a carrier system and spatially correlated with the diagnostic data generated before the operation using a suitable correlation algorithm.

[0008] US 2007 / 270690 A1 concerns a medical registration device. The registration device includes a localization device capable of recording the spatial position of treatment devices, treatment aids, patients, or patient parts. The localization device includes all means that can determine the spatial position of the patient, treatment devices, or treatment aids (e.g., a position in three-dimensional space of an instrument or its tip). The registration device may further include a data processing unit that maps the detected positions of patients or patient parts to the corresponding points in a captured patient image set. The registration device may include a distance measuring device whose spatial position can be determined by the localization device. The distance measuring device can transmit the distance data for measured points to the data processing unit. BRIEF SUMMARY

[0009] There is a need for improved techniques to enable the determination of a transformation rule between images acquired using a robotic visualization system and preoperative volumetric image data of the patient. In particular, there is a need for techniques that overcome or reduce the aforementioned disadvantages and limitations of existing techniques.

[0010] This task is solved by the features of the independent patent claims. The features of the dependent patent claims define embodiments.

[0011] The techniques described herein enable rapid and efficient patient registration. This means that the transformation formula can be determined reliably and automatically. The target region, for which depth-resolution measurement data is acquired to determine the transformation formula, can be automatically located and identified.

[0012] Using the techniques described herein, it is therefore possible to register preoperative volume image data in a reference coordinate system associated with images acquired using a robotic visualization system.

[0013] A computer-implemented method comprises controlling a robotic visualization system to acquire at least one target image. This target image depicts at least a portion of the patient. The method also includes determining the arrangement of a target region within this target image. Furthermore, the method includes controlling the robotic visualization system to acquire depth-resolution measurement data based on the target region's arrangement within the target image. This measurement data is indicative of the patient's anatomical topography. The measurement data enables the determination of a transformation formula between images acquired by the robotic visualization system and preoperative volumetric imaging data of the patient.

[0014] The target region can refer to an area of ​​the patient for which preoperative volumetric imaging data is available. The preoperative volumetric imaging data can therefore also be indicative of the patient's anatomical topography within the target region. For example, the target region could refer to a surgical site for a planned operation. The target region could, for instance, encompass a patient's head. The target region could also encompass a portion of a patient's head.

[0015] The arrangement of the target region can refer to a position and / or orientation and / or shape and / or size of the target region.

[0016] By capturing at least one search image using the robotic visualization system, the target region can be searched for and identified.

[0017] Manually locating the target region, for example by manually positioning the robotic visualization system, is no longer necessary. Automatic alignment of the robotic visualization system, enabling the measurement data to be captured as observables of the patient's anatomical topography in the target region, is achieved based on the specific arrangement of the target region in the at least one search image.

[0018] The preoperative volumetric imaging data could be, for example, CT or MRI images. It could also be positron emission tomography (PET) images.

[0019] For example, it would be conceivable to capture a single search image using the robotic visualization system. Specifically, an overview image could be captured using a suitable camera from an overview imaging unit, which employs a lens that provides no or comparatively low magnification. In such a case, the search image could depict the target region and other surrounding areas. For example, during a typical operation on the patient's head, the image could cover the patient's head down to the chest, torso, or hip area.

[0020] In other examples, it would also be conceivable to capture multiple search images, each depicting a different area. A grid of search images could be recorded.

[0021] The robotic visualization system is controlled to record at least one search image sequence. Each of these at least one search image sequence comprises several corresponding search images.

[0022] This means that the search images can be captured, for example, in one or two stages, in one or two (or more) sequences.

[0023] For example, the search region associated with each image sequence could be adapted or, in particular, refined from one image sequence to the next. For example, the images in different image sequences could have different resolutions. For example, the images in different image sequences could have increasing magnification.

[0024] The search images in a search image sequence scan a corresponding search region. For this purpose, at least one motor of the robotic visualization system is controlled during the recording of the at least one search image sequence in order to reposition the robotic visualization system multiple times.

[0025] For example, a suitable optics and camera of an imaging unit, which is used to capture the search images, could be attached to a microscope head of an operating microscope, wherein the microscope head can be positioned in space via a stand with one or more movable axes, for example translationally and / or rotationally.

[0026] Scanning allows for the coverage of a comparatively large search area. At the same time, a relatively high magnification can still be used, ensuring reliable identification of the target region.

[0027] A first search image sequence of at least one search image sequence is assigned a first search region, and a second search image sequence of at least one search image sequence is assigned a second search region.

[0028] The second search region could be included within the first search region. The second search region could also be partially different from the first search region.

[0029] The robotic visualization system is then controlled to first record the first search image sequence and subsequently record the second search image sequence.

[0030] The procedure involves evaluating the one or more search images of the first search image sequence. Based on this evaluation, values ​​are then determined for at least one imaging parameter of the robotic visualization system used to capture the second search image sequence.

[0031] As a general rule, imaging parameters can refer to the spatial arrangement of the imaging unit of the robotic visualization system used to capture the corresponding images and / or the parameterization of the imaging chain (zoom, focus, light, exposure parameters, etc.). Imaging parameters can therefore be those parameters that influence the appearance of an image and / or the field of view of an image.

[0032] Using such techniques, it is therefore possible to adjust the values ​​for at least one imaging parameter to the specific situation. Depending on the concrete situation related to the visualization of the respective patient, the values ​​for at least one imaging parameter can be appropriately adjusted so that the target region can be reliably located using the search images of the second search image sequence.

[0033] For example, evaluating the one or more search images of the first search image sequence could involve determining the arrangement of a marker fixed to the patient within those images. Then, the values ​​for at least one imaging parameter could be determined based on the arrangement of the marker.

[0034] For example, the arrangement of the second search region relative to the first search region could be determined by adjusting the fields of view of the search images of the second search region. For instance, the relative arrangement of the marker relative to the target region could be known in advance. The marker could, for example, be attached to the patient using a clamp or other fixation device, offset from the target region. This offset can be known. Then the second search region could be arranged relative to the first search region or to the marker found in the search images of the first search region.

[0035] Another exemplary implementation of the evaluation of the one or more search images from the first search image sequence could involve determining values ​​for one or more lighting parameters for capturing the second search image sequence. For example, brightness histograms for the one or more search images of the first search image sequence could be evaluated, and then the exposure time and / or the brightness of the lighting could be adjusted for capturing the one or more search images of the second search image sequence, so that the brightness histograms for the one or more search images of the second search image sequence match a target specification. In this way, it can be ensured that the contrast with which certain anatomical features or, for example, skin tissue are depicted in the one or more search images of the second search image sequence accurately reflects the details needed to locate the target region.

[0036] In other examples, it would also be conceivable that, based on the evaluation of one or more search images of the first search image sequence, it is determined whether one or more search images of the first search image sequence can be reused for the second search image sequence.

[0037] For example, if the second search region for the second search image sequence is defined in such a way that it overlaps with the first search region of the first search image sequence, then search images already recorded for the first search image sequence could be reused for the second search image sequence within the corresponding overlap area. This reduces the time required to capture the search images.

[0038] In general terms, it is conceivable that at least one search image is both part of the first search image sequence and part of the second search image sequence.

[0039] For example, it would be conceivable that the second search image sequence could be evaluated to detect the patient's skin tissue in the one or more search images of the second search image sequence. The arrangement of the target region could then be determined based on the detection of the skin tissue.

[0040] Typically, the patient is mostly covered, for example with a surgical drape. Only the target region and immediately surrounding areas may be exposed. Therefore, the target region can be identified by detecting skin tissue.

[0041] The procedure may involve receiving tracking data from a tracking system. The tracking system may be collocated to the robotic visualization system. This means that the robotic visualization system and the tracking system can operate in a common reference coordinate system, or the tracking system can be used to determine the position of the robotic visualization system (and thus also the pose or field of view of measurement data or images acquired by the robotic visualization system) within the reference coordinate system. The tracking system may also be configured to determine the arrangement (i.e., the position and / or orientation) of other markers within the reference coordinate system. The reference coordinate system may be defined relative to the operating room.

[0042] The tracking data could, for example, describe the arrangement of a marker fixed to the patient in relation to the robotic visualization system. The marker could be implemented, for example, as passive machine-readable symbols or active machine-readable light sources.

[0043] Accordingly, the procedure could further include determining values ​​for at least one imaging parameter of the robotic visualization system used to capture the at least one search image sequence, based on the tracking data.

[0044] For example, the arrangement of a corresponding search region, covered by at least one search image sequence, could be determined based on the tracking data. If, for instance, the marker is detected by the tracking system, its relative position in relation to the target region can be known in advance.

[0045] In general, it may therefore be possible to determine values ​​for at least one imaging parameter of the robotic visualization system used to capture the at least one search image sequence based on prior knowledge. For example, the brightness of a light source could be set as an example imaging parameter. An exposure time could be set as an example imaging parameter. An arrangement of corresponding search regions could be determined. The reuse of search images between different search image sequences could be determined.

[0046] Such prior knowledge could, for example, relate to the (rough) position of the patient in relation to the robotic visualization system. Alternatively or additionally, such prior knowledge could also relate to the position of the target region in relation to a marker fixed to the patient. For example, the patient might be roughly pre-positioned in relation to the robotic visualization system. For instance, the patient might be positioned on a treatment table, and the position of the table in relation to the robotic visualization system is known. In this way, a first step could involve roughly aligning the robotic visualization system with the (presumed) position of the patient, and then, based on this prior knowledge, generating the search images of at least one

[0047] The search image sequence is captured. The same applies to a marker fixed to the patient.

[0048] The method could further include evaluating already captured search images from the at least one search image sequence during the recording of the at least one search image sequence. Depending on the evaluation, the recording of the at least one search image sequence could then be selectively aborted, meaning that further search images could be omitted. Alternatively or additionally, it would also be conceivable to adjust values ​​for at least one imaging parameter of the robotic visualization system used to record the at least one search image sequence, depending on the evaluation.

[0049] If multiple search images are captured, it can be continuously checked whether it is necessary to capture further search images (or whether, for example, the target region has already been identified, making further capture unnecessary). Furthermore, it could be checked whether the values ​​for at least one imaging parameter are set appropriately, or whether an adjustment should be made, such as adjusting the brightness or exposure time, etc.

[0050] In general, the arrangement of the target region can be determined based on the structural recognition of one or more predefined structures in the at least one search image. For example, the structure of skin tissue could be recognized in the at least one search image. Alternatively or additionally, the structure of specific anatomical features (e.g., nose, ear, mouth, eye, etc.) could be recognized in the at least one search image. Suitable algorithms can be used. However, the specific implementation of such an algorithm for determining the arrangement of the target region is not essential for the implementation of the techniques described herein, and previously known implementations can be used.

[0051] To capture the at least one search image, for example, an overview image imaging unit of the robotic visualization system and / or a microscopy imaging unit of the robotic visualization system can be activated. In other words, this means that the at least one search image can, for example, be an overview image with relatively low magnification and / or a microscopy image with relatively high magnification.

[0052] The measurement data can be acquired, for example, using a measurement modality selected from the following group: stereoscopic imaging; time-of-flight measurement; structured illumination; and defocus depth estimation.

[0053] Stereoscopic imaging uses two cameras positioned in different poses to capture the same area and obtain information about the topography.

[0054] In time-of-flight measurement, information about the topography can be obtained by means of short light pulses and the corresponding monitoring of the travel time.

[0055] In structured lighting, a predefined pattern, such as a line pattern, is projected onto a surface to be measured. The distortion of the predefined pattern can then be used to obtain information about the topography.

[0056] Defocus depth estimation allows information about the topography to be obtained through the local use of an autofocusing method.

[0057] As a general rule, it would be conceivable for the measurement data to include at least one search image. This means that it is not absolutely necessary for the measurement data to be recorded completely separately from the search images.

[0058] The procedure could further include filtering the measurement data. For example, data elements that do not depict the patient's skin tissue could be removed. Areas depicting surgical drapes, markers, or surgical instruments could be removed. This prevents the measurement data from being incorrectly registered with the preoperative volumetric imaging data.

[0059] Before capturing the first or at least one search image, the motor of the robotic visualization system can be activated to position the system in a predefined reference arrangement. For example, the predefined reference arrangement could be defined in relation to an operating table. The predefined reference arrangement could be defined such that an area of ​​the operating table, on which the patient's head is to be positioned, lies within the field of view of the imaging unit of the robotic visualization system, which is used to capture the first or at least one search image.

[0060] It would be conceivable for the method to further include setting at least one imaging parameter of the robotic visualization system to a predetermined value before capturing the at least one search image. This could, for example, involve setting a target magnification for a zoom factor of a corresponding imaging unit of the robotic visualization system. A specific brightness of the lighting could be selected. It would also be conceivable for the exposure time to be set to a specific value.

[0061] The procedure described herein can be automated based on a predefined control script. For example, the control script can be triggered by a single user command (e.g., "one-click"). This allows for particularly simple and fast acquisition of measurement data, which then enables the determination of the transformation formula; manual repositioning of the robotic visualization system can be eliminated.

[0062] For example, such a control script could include an analysis module for evaluating image or measurement data. Furthermore, the control script could include at least one positioning module for positioning the robotic visualization system. Such positioning could, for example, be used to capture search images from a sequence using a grid, covering a search region larger than the field of view of a single search image. The analysis module could, for example, evaluate image data to locate skin tissue. The analysis module could, for example, evaluate image data to detect a marker that is fixed in position relative to the patient.

[0063] The analysis module can, for example, include one or more machine-learned algorithms. Artificial neural networks could be used, for instance, to recognize predefined structures or objects in images. A convolutional neural network could be used for this purpose. Appropriate training can be achieved, for example, through manual annotation of relevant training image data or training measurement data.

[0064] On the other hand, it is conceivable that the positioning module does not include machine-learned algorithms. A manually parameterized algorithm could be used. This can be particularly desirable because it allows the new position of the positioning module to be validated for each input parameter during a validation phase, thus preventing uncontrolled or dangerous movements of the robotic visualization system.

[0065] The process can further include generating a topography dataset based on the measurement data. This means that the measurement data can be evaluated, for example using an analysis module as described above.

[0066] The procedure can further include registering the topography dataset with the preoperative volumetric imaging data. This means that corresponding points can be found in the topography dataset and the preoperative volumetric imaging data. Based on the relative arrangement of such corresponding points to each other, the transformation rule can then be determined. In general, the transformation rule can be determined based on the registration.

[0067] A data processing unit is configured to control a robotic visualization system to acquire at least one target image. This target image depicts at least a portion of the patient. Furthermore, the data processing unit is configured to determine the position of a target region within the target image and, based on this position, to control the robotic visualization system to acquire depth-resolution measurement data. This measurement data is indicative of the patient's anatomical topography. This enables the determination of a transformation formula between images acquired by the robotic visualization system and preoperative volumetric image data of the patient. Specifically, the data processing unit is configured to execute the procedure described above.

[0068] A system includes the data processing unit and the robotic visualization system.

[0069] A computer program, a computer program product, or a computer-readable storage medium comprises program code. The program code can be loaded and executed by a processor. This causes the processor to execute a procedure. The procedure includes controlling a robotic visualization system to acquire at least one search image. The at least one search image depicts at least a portion of a patient. Furthermore, the procedure includes determining the arrangement of a target region within the at least one search image. The procedure also includes controlling the robotic visualization system to acquire depth-resolution measurement data based on the arrangement of the target region within the at least one search image. The measurement data is indicative of the patient's anatomical topography.The measurement data enables the determination of a transformation rule between images captured using the robotic visualization system and preoperative volume image data of the patient.

[0070] The features set out above and those described below can be used not only in the corresponding explicitly set out combinations, but also in further combinations or in isolation, without leaving the scope of protection of the present invention. BRIEF DESCRIPTION OF THE FIGURES

[0071] FIG. 1 schematically illustrates an optical visualization system according to various examples. FIG. 2 schematically illustrates a data processing unit according to various examples. FIG. 3 This is a flowchart of an exemplary procedure. FIG. 4 illustrates a single search image and a target region according to various examples. FIG. 5illustrates a search image sequence and a target region according to various examples. DETAILED DESCRIPTION OF EXECUTION FORMS

[0072] The properties, features and advantages of this invention described above, as well as the manner in which they are achieved, will become clearer and more easily understood in connection with the following description of the exemplary embodiments, which are explained in more detail in conjunction with the drawings.

[0073] The present invention is explained in more detail below with reference to preferred embodiments and the drawings. In the figures, identical reference numerals denote identical or similar elements. The figures are schematic representations of various embodiments of the invention. Elements depicted in the figures are not necessarily shown to scale. Rather, the various elements depicted in the figures are represented in such a way that their function and general purpose are understandable to a person skilled in the art. Connections and couplings between functional units and elements shown in the figures can also be implemented as indirect connections or couplings. A connection or coupling can be implemented as a wired or wireless connection. Functional units can be implemented as hardware, software, or a combination of hardware and software.

[0074] The following describes techniques that make it possible to determine a transformation rule to map or convert image data acquired using a robotic visualization system (hereinafter simply called an operating microscope) onto preoperative volume image data; the reverse mapping rule can also be captured by the transformation rule.

[0075] For this purpose, a target region is automatically detected or its arrangement is automatically determined based on one or more search images. It is then possible to determine depth-resolution measurement data for this target region, which can be registered onto the preoperative volume image data in order to derive the transformation rule from this registration.

[0076] Based on this transformation rule, various applications can then be enabled. For example, an assistance function could be provided to a surgeon based on the preoperative volumetric image data and the transformation rule. For instance, the preoperative volumetric image data, or parts thereof, could be precisely superimposed onto the field of view of the operating microscope or onto image data acquired with the operating microscope. Specific areas marked in the preoperative volumetric image data could be highlighted in the image data acquired with the operating microscope. A navigation assistance function that guides the surgical procedure can also be implemented.

[0077] FIG. 1Figure 801 schematically shows an operating microscope 801 for surgery. In the example shown, the operating microscope 801 has an eyepiece 803. The surgeon can view magnified images of an object located within a field of view 804 of the operating microscope 801 through the eyepiece 803. In the example shown, this is a patient 805 lying on a treatment table.

[0078] Alternatively or in addition to an optical eyepiece, a camera 809 could also be provided, which transmits images to a screen (digital surgical microscope).

[0079] In general terms, the 801 operating microscope can include one or more imaging units configured to acquire digital images. Examples of imaging units would be a microscopy imaging unit and a panoramic imaging unit. The microscopy imaging unit could include one, two, or more cameras; for example, stereoscopic imaging with multiple cameras would be possible.

[0080] An operating device 808 is also provided as a human-machine interface, which can be designed, for example, as a handle or foot switch. In the illustrated embodiment of the FIG. 1It is a handle. The operating device 808 allows the eyepiece 803, which is attached to the traverse 850, to be moved. Motors can be provided to perform the movement automatically based on control data, according to a corresponding setting of the operating microscope. The motors could also assist the movement initiated by the handle 808.

[0081] Furthermore, a control unit 880 is provided for the operating microscope 801, which controls the operation of the operating microscope 801 and the display of images as well as additional information and data in the eyepiece 803. The control unit 880 can interact with the surgeon.

[0082] The operating microscope 801 can also be equipped with one or more additional sensors 860, such as a peripheral camera, a time-of-flight sensor (TOF camera), an image acquisition device for capturing images under structured illumination, etc. Measurement data, such as depth resolution, can be acquired using these sensors 860. This means that the depth resolution can describe the distance between the corresponding sensor 860 and the patient 805.

[0083] Another implementation of such a sensor 860 would involve an internal tracking system. Using such an internal tracking system, markers that are fixed relative to the patient and are machine-readable can be detected. Markers are sometimes also referred to as targets. In particular, the relative arrangement of the markers with respect to the operating microscope 801 can be determined.

[0084] FIG. 1Figure 890 also illustrates an external tracking system. This system is collocated with the operating microscope 801. It is possible for tracking data 890 to be transmitted from the tracking system 890 to the control unit 880, where the tracking data is, for example, indicative of the arrangement of the camera 809 or the eyepiece 803, or more generally the field of view 804 of an imaging unit in space, for example in relation to markers positioned at fixed positions relative to the patient 805, or a reference coordinate system fixed in space.

[0085] FIG. 2 illustrates aspects related to a data processing unit 910. The data processing unit 910 could, for example, be implemented by a computer. The data processing unit 910 could, for example, implement the control unit 880 of the operating microscope 801 (compare FIG. 1 ) or could also be trained separately from the operating microscope 801 (and then communicate with the control unit 880, for example).

[0086] The data processing unit 910 comprises a computing unit 911, a memory 912, and a communication interface 913. The computing unit 911 can, for example, load and execute program code from the memory 912. The computing unit 911 can communicate with other devices, nodes, or fixtures via the communication interface 913. For example, tracking data could be received from a tracking system. It would be possible to receive pre-operative volumetric image data, for example, from an image archive system or another database. User input could be received from a human-machine interface.

[0087] When the processing unit 911 executes program code, which is loaded, for example, from the memory 912, this can cause the processing unit 911 to perform techniques as described herein, such as: acquiring search images or measurement data; evaluating search images or measurement data; determining the arrangement of a target region in search images; controlling the operating microscope 801 to acquire images, search images, or measurement data and / or to effect a specific position of the field of view 804; providing control data to enable an assistance function for a surgeon; determining a transformation rule between image data acquired by means of the operating microscope 801 and preoperative volume image data, for example, based on a corresponding registration between depth-resolution measurement data and the preoperative volume image data.Various functionalities that can be provided and implemented by the computing unit 911 based on speaking program code are described below in connection with the procedure in . FIG. 3 described.

[0088] FIG. 3 This is a flowchart of an example procedure. For example, the procedure of FIG. 3 from a data processing unit such as data processing unit 910 FIG. 2 be carried out. In particular, the procedure could consist of FIG. 3 executed by a computing unit, such as a processor, when it loads and executes program code.

[0089] The procedure of FIG. 3This serves to enable the determination of a transformation formula between images acquired using an operating microscope and preoperative volume imaging data – for example, CT or MRI images. In other words, the method serves to FIG. 3 This method is used to calibrate the patient's positioning in relation to the operating microscope and to the preoperative volumetric imaging data. In particular, it supports the following: FIG. 3 Automated acquisition of depth-resolution measurement data by the surgical microscope allows this data to be registered against the preoperative volume image data. Based on this registration, the transformation formula can then be determined. Alternatively, the transformation formula could be determined in a subsequent process, for example, by another data processing system, such as a control unit of a tracking system.

[0090] For example, it would be conceivable that the procedure according to FIG. 3 The process is executed based on an automated control script. This means that the various process steps can be processed automatically, one after the other, by a computer program. This has the advantage that the user, for example, can trigger the corresponding control script with a single human-machine interaction, and no manual interactions are necessary during the execution of the respective process steps.

[0091] Optional boxes are included. FIG. 3 Represented with dashed lines.

[0092] First, a motor of the operating microscope can be controlled in box 3005 to position the operating microscope in a predefined reference arrangement. For example, this reference arrangement could ensure that the operating microscope's field of view is focused on the head area of ​​a patient when the patient is positioned on a suitable table.

[0093] Box 3010 then offers the option to initialize one or more imaging parameters. This means that at least one imaging parameter of the surgical microscope can be set to a predefined value before acquiring search images. For example, the brightness of an illumination system could be initialized, or a magnification factor of a zoom lens could be set. A predefined focus could be selected, or a specific field of view could be set.

[0094] Sometimes, Box 3010 and Box 3005 can be implemented together. For example, if the imaging parameter is set to position a corresponding imaging unit of the operating microscope in Box 3010.

[0095] Subsequently, at least one search image, depicting at least part of the patient, is acquired in box 3011. This can be achieved by activating the operating microscope, or more precisely, a corresponding imaging unit or control device. A command to acquire the corresponding search image can then be sent.

[0096] For example, an overview imaging unit of the surgical microscope and / or a microscopy imaging unit of the surgical microscope could be used to acquire the at least one search image. This means that, for example, one or more overview images at low magnification and / or one or more microscopy images at high magnification can be used to implement the at least one search image.

[0097] There are different implementation variants for Box 3011. Before these implementation variants are discussed in detail below, the following procedure will first be explained in more detail.

[0098] In Box 3030, the arrangement of a target region within the at least one search image is determined. This means that the position, orientation, and / or extent of a target region within the at least one search image is determined. An analysis module can be used. For example, a machine-learned algorithm could be employed. Such a machine-learned algorithm could be trained based on training search images and manual annotations of target regions. The machine-learned algorithm could, for example, be implemented using a convolutional neural network.

[0099] The target region can, for example, refer to the surgical area of ​​a planned operation. The target region can also refer to an area of ​​the patient's body for which preoperative volumetric imaging data was acquired or which is depicted by the preoperative volumetric imaging data.

[0100] For example, the target region could be determined based on the recognition of one or more predefined structures in at least one search image. Skin tissue or anatomical features could be recognized, for instance. Alternatively, a machine-readable marker could be detected.

[0101] It is then possible to acquire depth-resolution measurement data in Box 3035 based on this arrangement of the target region. For this purpose, the operating microscope can be controlled, for example, by a corresponding sensor, an imaging unit, or an associated control device. The measurement data are therefore indicative of the topography of the patient's anatomy in the area of ​​the target region.

[0102] For example, the measurement data could be acquired using a measurement modality selected from the group that includes: stereoscopic imaging; time-of-flight measurement; structured illumination; and defocus depth estimation.

[0103] In some examples, it would also be possible for the measurement data to include at least one of the search images. This means that it would be conceivable for certain images to not only be used as search images in Box 3011, but also to be considered within the measurement data in Box 3035, thus fulfilling a dual function.

[0104] In box 3036, it would be optionally possible to filter the measurement data. For example, noise reduction could be performed. Background could be removed. It would be possible to remove data elements that do not depict the patient's skin tissue.

[0105] In this way, it may be possible to determine the transformation formula between images acquired with the operating microscope and preoperative volumetric image data of the patient. This transformation formula can be determined in optional box 3040. It would also be possible to save the measurement data for later determination or to transmit it to another data processing unit so that this unit can determine the transformation formula.

[0106] The transformation formula can be determined as follows: based on the measurement data, a topography dataset can first be generated. This means that, for example, a height profile of the patient's head or, more generally, of the patient's skin surface in the target region can be determined. This topography dataset can then be registered with the preoperative volumetric imaging data. The preoperative volumetric imaging data can, in particular, depict the topography of the patient's skin surface. Based on this registration, the transformation formula can then be determined, for example, taking into account translational and / or rotational degrees of freedom. Distortions can also be considered.

[0107] Optionally, an application based on this transformation rule can be enabled in Box 3045. For example, computer-assisted surgery could be enabled, perhaps using augmented images acquired with the operating microscope, containing information determined from the preoperative volume image data. Navigation assistance functionality is also enabled.

[0108] The following details relate to Box 3011, specifically the recording of at least one search image. In a simpler version, it would be possible to record a single search image and determine the arrangement of the target region within that single image. Such a scenario is described in FIG. 4 depicted. In FIG. 4 A search image 111 is shown. The search image 111 depicts the target region 131. For example, the search image 111 could be used with an overview imaging unit of the operating microscope 801 (compare FIG. 1 ) be recorded).

[0109] In another variation, one or more search image sequences could also be captured. For example, in FIG. 5 A scenario is shown in which a total of 25 search images 111-113 (for clarity, only the first three search images are labeled) are captured. For example, each of these search images in the sequence could have a particularly high resolution. This would allow for high magnifications. This corresponds to scanning a specific search region. Scanning can be achieved by controlling a motor on the operating microscope, so that the microscope is repositioned multiple times, for example, between capturing the different search images in the sequence.

[0110] In some examples, several search image sequences could be captured one after the other. Such a scenario is in FIG. 3described in connection with Box 3011. First, a search image sequence, to which a first search region is assigned, is recorded in Box 3015; then, a second search image sequence, to which a second search region is assigned, is recorded in Box 3025. In between, it is possible in Box 3020 to evaluate one or more search images from the first search image sequence in Box 3015 and, based on this evaluation, to determine values ​​for at least one imaging parameter of the operating microscope used to record the second search image sequence in Box 3025.

[0111] This evaluation in Box 3020 could, for example, include determining the arrangement of a marker fixed to the patient in the one or more search images of the first search image sequence from Box 3015.

[0112] For example, the arrangement of the second search region, for which the second search image sequence in box 3025 is captured, could be determined in relation to the first search region of the first search image sequence from box 3015. It could also be determined whether a search image from the first search image sequence is reused for the second search image sequence. For example, a lighting parameter could be set for capturing the second search image sequence. A magnification factor could be set for the search images of the second search image sequence.

[0113] If multiple search image sequences are used - as for example in FIG. 3 As shown in connection with Box 3015 and Box 3025, it is fundamentally possible that at least one search image is part of several search image sequences.

[0114] Table 1 below describes a workflow that outlines the procedure of FIG. 3 can implement. Table 1: Exemplary implementation of a workflow that enables the determination of a transformation rule between images acquired using an operating microscope and preoperative volume image data. The steps do not necessarily have to be executed rigidly in this order, but can also be combined or run in parallel to potentially reuse previously acquired data. Step Brief description Exemplary details 1 Start control script First, the user starts the patient registration via a command, e.g., pressing a button, using a foot switch, speaking, etc. This means that a corresponding control script, which automatically performs the following steps, is started. 2 Rough alignment The surgical microscope is positioned in a reference arrangement, for example pointing straight down towards the head area of ​​a bed. A motor can be controlled accordingly. The robotic surgical microscope is automatically aligned so that the surrounding camera is oriented in the negative z-direction (pointing downwards), maximizing the possible working distance. Alternatively or additionally, another camera, such as one from a microscopy imaging unit, could also be oriented in the negative z-direction. Then the operating microscope is "roughly" positioned over the patient, i.e., it can be assumed that the cameras of the operating microscope are located spatially in the z-direction (orthogonal from the floor upwards) above the patient. Cf. FIG. 3 : Box 3005. 3 initialization Then, if adjustable, the zoom factor of the surrounding camera is automatically set to the minimum beforehand. Optionally, the operating microscope's light source is switched to a sensitive mode to protect the patient's eye (assuming the eyelid is closed). This means the light intensity is set to a comparatively low value. The zoom factor and the brightness of the lighting are just two examples of imaging parameters that can be set to a predefined value to achieve initialization. Cf. FIG. 3 : Box 3010. 4 Detection Marker Then the approximate position of the patient's head is determined. For this purpose, search images from a surrounding camera (i.e., an overview imaging unit) are evaluated to see whether a marker attached to the patient's head or fixed to it (for example, on a Mayfield clamp or a ring rigidly connected to the clamp) is visible. Instead of the ambient camera, a microscopy imaging unit could also be used. For example, stereoscopic search images could be captured. So, a first search image sequence is recorded. The operating microscope then performs a scanning movement (e.g., spiraling outwards) while simultaneously evaluating the image. The scan was performed along the xy plane until the marker in the image was located and its position relative to the tracking camera, or more generally in relation to the operating microscope, could be determined. Other scan patterns for scanning a search region are conceivable. Generally, 2D or 3D scan patterns are possible. Further examples would be spherical patterns or patterns that include a pivot movement. Optionally, it would be conceivable to continuously detect whether the patient's eye is open while capturing the search images. If so, the scanning process would be automatically interrupted / aborted or the light for the eye area would be switched off. Cf. FIG. 3 : Box 3015, 3020. Sometimes, the relative position of the surgical microscope with respect to a marker is already known, for example, based on tracking data received from a tracking system. In this case, it is not necessary to capture search images or a corresponding sequence of search images to determine the marker's position; instead, the marker's position can be determined based on the tracking data. 5 Recognition of patient head as target region Knowing the marker's location in space, the system then searches for the patient's head position within a defined volume around the marker. The corresponding search region is thus determined based on the marker's spatial arrangement and prior knowledge of typical head size in all spatial directions around the marker. Generally, values ​​for at least one imaging parameter can be used to acquire... Search images are determined based on prior knowledge regarding the patient's arrangement in relation to the operating microscope and / or the arrangement of the target region in relation to a marker. Optionally, the search region for this second search image sequence can be narrowed down by knowing the patient's position and the placement of the marker (whether right, left, top, bottom, etc.). The operating microscope scans the search region (or at least uses part of the search images from step 4). An analysis module identifies the patient's head as the target region. For this purpose, the analysis module uses image processing (e.g., machine learning approaches) to automatically recognize skin tissue or specific anatomical features (such as the mouth, nose, or eye sockets) in the image. Additionally or alternatively, topographic data (i.e., measurement data with depth resolution) are also created, so that the three-dimensional data can be matched against a typical shape of the human skull, thus providing an additional indication for the identification of the patient's head. Cf. FIG. 3 : Box 3025, 3030.. 6 Measurement data Once the position or general arrangement of the patient's head as the target region is known, the topography of the patient's head is recorded. This involves acquiring measurement data with depth resolution. Optionally, rasterization can also be performed here, i.e., automatic movement of the surgical microscope can be used to stitch the field of view. It would be conceivable to reuse search images from step 4 or 5 to reduce scan time. Cf. FIG. 3 : Box 3035. 7 Filter The measurement data is optionally (automatically) cleaned up, i.e., areas with non-skin tissue (e.g., cloths) are removed and / or metallic supports / instruments are removed. This is done either by evaluating image contrast (automatic detection of skin tissue) and / or via topographic information (e.g. structures that are not directly connected to the skull and are a distance >5cm are removed). Cf. FIG. 3 : Box 3036. 8 Transformation regulation The determined topography is used for patient registration (for example, sent to a navigation system). Cf. FIG. 3 : Box 3040.

[0115] In summary, the preceding sections described techniques that enable the automatic detection of a target region for patient registration. An automatic alignment of a robotic visualization system, allowing the acquisition of measurement data with depth resolution that maps the target region, was disclosed.

[0116] Naturally, the features of the embodiments and aspects of the invention described above can be combined with each other within the scope of the claims.

Claims

1. Computer-implemented method, comprising: - actuating a robotic visualization system (801) for recording at least one search image (111-113) which at least partly images a patient (805), wherein the robotic visualization system is actuated in order to record at least one search image sequence, wherein each of the at least one search image sequence comprises a plurality of corresponding search images, - during the recording of the at least one search image sequence, actuating at least one motor of the robotic visualization system in order to reposition the robotic visualization system a number of times, such that the plurality of search images of the respective search image sequence scan a search region assigned to the respective search image sequence, wherein a first search region is assigned to a first search image sequence of the at least one search image sequence, wherein a second search region is assigned to a second search image sequence of the at least one search image sequence, wherein the robotic visualization system firstly is actuated for recording the first search image sequence and subsequently is actuated for recording the second search image sequence, - evaluating one or more search images of the first search image sequence and, based on evaluating the one or more search images of the first search image sequence, determining values for at least one imaging parameter of the robotic visualization system which is used for recording the second search image sequence, - determining an arrangement of a target region (131) in the at least one search image (111-113), and - based on the arrangement of the target region (131) in the at least one search image (111-113), actuating the robotic visualization system (801) for recording measurement data with depth resolution, wherein the measurement data are indicative of a topography of an anatomy of the patient (805) in order in this way to enable the determination of a transformation specification between images recorded by means of the robotic visualization system and preoperative volume image data of the patient.

2. Computer-implemented method according to Claim 1, wherein evaluating the one or more search images of the first search image sequence comprises determining an arrangement of a marker stationarily fixed with respect to the patient in the one or more search images of the first search image sequence, and wherein the values for the at least one imaging parameter are determined based on the arrangement of the marker stationarily fixed with respect to the patient.

3. Computer-implemented method according to Claim 1 or 2, wherein the at least one imaging parameter comprises an arrangement of the second search region in relation to the first search region.

4. Computer-implemented method according to any of the preceding claims, wherein the at least one imaging parameter comprises a reuse of a search image of the first search image sequence for the second search image sequence.

5. Computer-implemented method according to any of the preceding claims, wherein the at least one imaging parameter comprises an illumination parameter for recording the second search image sequence.

6. Computer-implemented method according to any of the preceding claims, wherein at least one search image is both part of the first search image sequence and part of the second search image sequence.

7. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - evaluating one or more search images of the second search image sequence for recognizing skin tissue of the patient in the one or more search images of the second search image sequence, wherein the arrangement of the target region (131) is determined based on recognizing the skin tissue.

8. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - receiving tracking data from a tracking system collocated with respect to the robotic visualization system, wherein the tracking data describe an arrangement of a marker stationarily fixed with respect to the patient in relation to the robotic visualization system, and - determining values for at least one imaging parameter of the robotic visualization system which is used for recording the at least one search image sequence, based on the tracking data.

9. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - determining values for at least one imaging parameter of the robotic visualization system which is used for recording the at least one search image sequence, based on prior knowledge concerning at least one out of an arrangement of the patient in relation to the robotic visualization system and an arrangement of the target region (131) in relation to a marker stationarily fixed with respect to the patient.

10. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - during the recording of the at least one search image sequence, evaluating already captured search images of the at least one search image sequence, and - depending on the evaluating, terminating the recording of the at least one search image sequence and / or adapting values for at least one imaging parameter of the robotic visualization system which is used for recording the at least one search image sequence.

11. Computer-implemented method according to any of the preceding claims, wherein the measurement data are captured with a measurement modality selected from the following group: stereoscopic imaging; time-of-flight measurement; structured illumination; and defocus depth estimation, and / or wherein the measurement data comprise at least one of the at least one search image.

12. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - filtering (3036) the measurement data in order to remove data elements which do not image skin tissue of the patient.

13. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - actuating (3005) a motor of the robotic visualization system (801) in order to arrange the robotic visualization system (801) in a predefined reference arrangement before the recording of the at least one search image (111-113), and / or - setting (3010) at least one imaging parameter of the robotic visualization system (801) to a predefined value before the recording of the at least one search image (111-113).

14. Computer-implemented method according to any of the preceding claims, wherein the method is carried out in an automated manner based on a predefined control script, wherein the control script comprises an analysis module for evaluating image data or measurement data, wherein the control script comprises a positioning module for positioning the robotic visualization system.

15. Computer-implemented method according to any of the preceding claims, wherein the method furthermore comprises: - generating a topography data set based on the measurement data, - carrying out a registration of the topography data set with the preoperative volume image data, and - based on the registration, determining the transformation specification.

16. Data processing unit (910), configured to carry out the following steps: - actuating a robotic visualization system for recording at least one search image which at least partly images a patient, wherein the robotic visualization system is actuated in order to record at least one search image sequence, wherein each of the at least one search image sequence comprises a plurality of corresponding search images, - during the recording of the at least one search image sequence, actuating at least one motor of the robotic visualization system in order to reposition the robotic visualization system a number of times, such that the plurality of search images of the respective search image sequence scan a search region assigned to the respective search image sequence, wherein a first search region is assigned to a first search image sequence of the at least one search image sequence, wherein a second search region is assigned to a second search image sequence of the at least one search image sequence, wherein the robotic visualization system firstly is actuated for recording the first search image sequence and subsequently is actuated for recording the second search image sequence, - evaluating one or more search images of the first search image sequence and, based on evaluating the one or more search images of the first search image sequence, determining values for at least one imaging parameter of the robotic visualization system which is used for recording the second search image sequence, - determining an arrangement of a target region (131) in the at least one search image, and - based on the arrangement of the target region (131) in the at least one search image, actuating the robotic visualization system for recording measurement data with depth resolution, wherein the measurement data are indicative of a topography of an anatomy of the patient in order in this way to enable the determination of a transformation specification between images recorded by means of the robotic visualization system and preoperative volume image data of the patient.