Method for operating a medical-optical display system

Abstract

The invention relates to a method for operating a medical optical display system. A method for operating a medical optical display system for displaying an object image (OB) of an observed object (O) is available, the object image having been obtained by means of a medical optical observation device (2, 48), wherein the medical optical display system comprises a data superimposition unit (74) for superimposing data of at least one image data record (BD) into the object image (OB). The method comprises the following steps: (S100) determining at least one region (B) with little activity within the object image (OB), and (S200) superimposing the at least one image data record (BD) into the at least one region (B) with little activity.

Description

Technical Field

[0001] The invention relates to a method for operating a medical-optical display system having a data superposition device. In addition, the invention relates to a computer program product for executing such a method, and a medical-optical display system having a data superposition device. Background Art

[0002] A medical optical observation device is known from DE 10 2014 210 150 A1, in which an image data record having additional information items is mirrored into an image provided by the medical optical observation device by means of a device for mirroring input data.

[0003] Generally speaking, the position in the image into which the image data record is mirrored is set by the device for superimposing the data without reference to the processing of the image content of the image.

[0004] In unsuitable cases, the overlay is implemented exactly at the location where the user (eg surgeon or operating room nurse) wants to have an "unobstructed view" of the real scene (site, display, instruments, personnel, ...).

[0005] In some information items to be enhanced, a spatially accurate superposition is desired (e.g. tumor contour data), but in the case of many information items (e.g. instrument status recording), positioning is not restricted to a fixed position, i.e., ideally, the information item recorded by the image data is superimposed somewhere out of the way because there is no action / activity in that area at the current time.

[0006] There are approaches to superimposing image data records relative to surgical instruments (eg tool tips), but these approaches have the disadvantage that the tool identification must be done explicitly (in the extreme case, the algorithm must be reparameterized for each new tool setting).

[0007] Furthermore, in the case of data superposition, it is known to perform fixed encoding of the superposition position in pixel coordinates regardless of the image content, for example to take into account the geometry of the space so that data superposition is always achieved in front of the object and enables data superposition to be docked with, for example, a surgical instrument. Summary of the invention

[0008] With reference to the above-mentioned prior art, the object of the present invention is to obtain an advantageous method for operating a medical-optical observation device, in particular a surgical microscope, and a medical-optical observation device in which there is no obstruction of relevant image content in the case of data superposition of image data records.

[0009] The above object is achieved by the aspect according to claim 1 and by the medical-optical viewing device according to claim 9. The dependent claims contain advantageous embodiments of the invention.

[0010] According to the invention, a method for operating a medical-optical display system for displaying an object image of an observed object, which object image has been obtained by means of a medical-optical observation device, is obtained, wherein the medical-optical display system comprises a data superimposition unit for superimposing data of at least one image data record into the object image. The method comprises the following steps:

[0011] - determining at least one region having little activity within the object image, and

[0012] - superimposing the at least one image data record onto the at least one region having little activity.

[0013] Therefore, the present invention chooses a novel approach for avoiding undesired occlusion of image content with the attendant loss of information.

[0014] Here, the object image can also be a stereoscopic image consisting of left and right partial images. Thus, the image data record can be superimposed on the left partial image or the right partial image. However, it is also possible to superimpose the image data record on both the left partial image and the right partial image. Specifically, the image data record can then include a right image data record and a left image data record that together produce a stereoscopic image data record. Thus, the image information item superimposed by means of the image data record can be a stereoscopic image information item. In addition, the object image can also be a surrounding environment image showing the surrounding environment of the object area to which the image data record in question is superimposed.

[0015] In principle, the medical-optical viewing device can be any medical device which generates images of an object, such as, for example, a surgical microscope, an endoscope, an endoscope or the like.

[0016] Accordingly, the medical-optical display system may be any system suitable for displaying an image of an object recorded by the medical-optical viewing device. Examples include mirrors and head mounted displays that receive an image of an object from a medical-optical viewing instrument, or a surgical microscope, an endoscope, or a portion of an endoscopic microscope that includes at least one eyepiece.

[0017] The at least one image data record contains items of status information about, for example, the instrument or patient data, and it is presented in electronic form. For example, the at least one image data record can be converted into optical data by means of a display or any other suitable converter unit, which optical data is then superimposed on the beam path of the medical-optical display system (for example mirrored therein) to be superimposed on the beam path of the object image. However, it can also be digitally superimposed on the digital object image in digital form. The at least one region with little activity can be determined by means of an algorithm, or using an adaptive system, for example based on a neural network, in particular a multilayer neural network.

[0018] According to one embodiment, the following steps are performed to determine areas with little activity:

[0019] - capture raw image data records representing a plurality of individual object images in a temporal sequence,

[0020] - determining the temporal variability of image points in the raw image data records, wherein the image points may be pixels or groups of pixels of the raw image data records,

[0021] - compares this time variability with a set variability limit, and

[0022] - clustering those image points whose determined temporal variability is less than the variability limit value to produce at least one continuous region with small temporal variability, and

[0023] - designating the at least one continuous region with small time variability as the at least one region with little activity.

[0024] Thus, for example, a plurality of individual object images are captured in a certain time sequence in the form of a video sequence from a video stream in which the object is recorded. In this case, these raw image data records are available in electronic form. Subsequently, at least one area with little activity (i.e. very little pixel variability) is identified (which is suitable for superimposing image data records) based on a comparison of the temporal variability of the image points of temporally consecutive object images with a predetermined variability limit value. Subsequently, the image data record is superimposed in one of these areas. If there are a plurality of image data records, these can be superimposed in different parts of the same area with little activity or, if there are at least two areas with little activity, in different areas with little activity.

[0025] In order to obtain a sufficiently large continuous region with low time variability and a size suitable for superposition of image data recording, it may be advantageous to combine a plurality of continuous regions with low time variability into a larger continuous region with low time variability by image processing, wherein the larger continuous region with low time variability is then designated as the at least one region with little activity.

[0026] The position of the superimposition of the image data record in the at least one region with little activity can be determined taking into account the size and / or form of the image content displayed in the image data record. This makes it possible, for example, to indicate whether the at least one region with little activity is sufficient for superimposing the image data records or whether, if necessary, a plurality of individual regions with little activity and low temporal variability must be combined into at least one larger region with little activity.

[0027] In a development of the method according to the invention, the depth map of the observed object can be used to exclude certain areas of the raw image data records where a large amount of activity is expected from becoming the at least one area with little activity. Additionally or alternatively, areas of the raw image data records where an instrument is present or facing can be identified based on pattern recognition and excluded in advance from becoming the at least one area with little activity.

[0028] A medical-optical display system for displaying an image of an object obtained by a medical-optical observation device according to the present invention, the system comprising:

[0029] - a data superimposition unit which is designed to superimpose at least one image data record into the object image, and

[0030] - a region determination unit which is designed to determine at least one region with little activity within the object image.

[0031] The data superimposition unit is designed to superimpose the at least one image data record on the at least one region of the object image having little movement.

[0032] The medical-optical viewing device according to the invention is adapted to carry out the method according to the invention.

[0033] Furthermore, the medical-optical display system may include a medical-optical observation device for observing the object and for recording the observed object as an object image.

[0034] The region determination unit of the medical-optical display system may specifically include:

[0035] - a reading unit for reading raw image data records representing a plurality of independent object images in a certain time sequence,

[0036] an evaluation unit for determining the temporal variability of image points in the raw image data records, wherein the image points may be pixels or groups of pixels of the raw image data records,

[0037] - a comparison unit for comparing the time variability with a set variability limit value,

[0038] - a clustering unit for clustering the image points whose determined temporal variability is smaller than the variability limit value to generate at least one continuous region with small temporal variability, and

[0039] - a designating unit for designating the at least one continuous region having small time variability as the at least one region having little activity.

[0040] In order to obtain a continuous region with small temporal variability and a size suitable for superposition of image data recording, there may be an image processing unit that synthesizes a plurality of independent continuous regions with small temporal variability into a larger continuous region with small temporal variability by morphological image processing. The designation unit is then configured to designate the larger continuous region with small temporal variability as the at least one region with little activity.

[0041] In the medical-optical display system according to the invention, the device for superimposing the data of the image data records can be designed to take into account the size and / or form of the image content displayed in the image data record in order to determine the position at which the image data record is superimposed in the at least one region with little activity. This makes it possible, for example, to indicate whether the at least one region with little activity is sufficient for superimposing the image content of the image data records or whether, if necessary, a plurality of individual regions with little activity must be combined to form a larger continuous region with little activity.

[0042] Optionally, in the medical-optical display system according to the present invention, the area determination unit may include a pre-selection device, which excludes in advance certain areas in the original image data records where a large amount of activity is expected based on the depth map of the observed object to prevent them from becoming the at least one area with little activity, and / or identifies areas in the original image data records where imaging instruments appear or are facing based on pattern recognition, and excludes these areas in advance to prevent them from becoming the at least one area with little activity.

[0043] Furthermore, the present invention comprises a computer program product containing a program code having computer-readable instructions for executing the method according to the present invention when the program code is loaded into a computer and / or executed on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Other features, characteristics and advantages of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings.

[0045] Figure 1 The typical structure of a surgical microscope is shown schematically.

[0046] Figure 2 An objective lens for a surgical microscope is shown, which can be used instead of Figure 1 Objective lens shown in .

[0047] Figure 3 An alternative configuration of a surgical microscope is shown.

[0048] Figure 4 A scenario using a medical-optical display system is shown.

[0049] Figure 5 An exemplary embodiment of an apparatus for carrying out the method according to the invention is shown.

[0050] Figure 6 A flow chart of an exemplary embodiment of a method according to the invention is shown. DETAILED DESCRIPTION

[0051] See also Figure 1 , which schematically shows the structure of a surgical microscope. In the current exemplary embodiment, the medical-optical display system is a part of a surgical microscope 2.

[0052] The surgical microscope 2 is understood to mean a microscope used in minimally invasive surgery and microsurgery. It has a relatively low magnification (approximately 6 to 40 times) and usually provides a three-dimensional image. The magnification is beyond the scope of a magnifying glass. In the medical field, it is used in almost all surgical operations.

[0053] Figure 1 The surgical microscope 2 shown comprises an objective lens 5 which should face the object field 3 and which can be embodied as an achromatic or apochromatic objective lens in particular. In the present exemplary embodiment, the objective lens 5 consists of two partial lenses which are connected to each other and form an achromatic objective lens. The object field 3 is arranged in the focal plane of the objective lens 5 so that it can be imaged at infinity by the objective lens 5. In other words, the divergent light beams 7A, 7B originating from the object field 3 are converted into parallel light beams 9 during their transmission through the objective lens 5.

[0054] The magnification changer 11 is arranged on the observer side of the objective lens 5, and the magnification changer can be embodied as a zoom system (as in the exemplary embodiment shown) for changing the magnification factor in a continuously variable manner, or a so-called Galilean changer for changing the magnification factor in a step-by-step manner. In a zoom system (for example, composed of a lens combination with three lenses), two object-side lenses can be shifted to change the magnification factor. However, in practice, the zoom system can also have more than three lenses, such as four or more lenses, in which case the outer lenses can then be arranged in a fixed manner. In contrast, in the Galilean changer, there are multiple fixed lens combinations, which represent different magnification factors and can be alternately introduced into the beam path. Both the zoom system and the Galilean changer convert the object-side parallel beam into an observer-side parallel beam with different beam diameters. In the current exemplary embodiment, the magnification changer 11 is already part of the binocular beam path of the surgical microscope 2, that is, it has a dedicated lens combination for each stereoscopic partial beam path 9A, 9B of the surgical microscope 2. In the present exemplary embodiment, the magnification factor is adjusted by means of a magnification changer 11 through a motor-driven actuator, which together with the magnification changer 11 is part of a magnification changing unit for adjusting the magnification factor.

[0055] In the present example, the magnification changer 11 is adjacent to the observer side via an interface arrangement 13A, 13B, by means of which external devices can be connected to the surgical microscope 2, and in the present exemplary embodiment, the interface arrangement comprises beam splitters 15A, 15B. In principle, however, other types of beam splitters, such as partially transparent mirrors, can also be used. In the present exemplary embodiment, the interface arrangements 13A, 13B are used to couple out a beam from the beam path of the surgical microscope 2 (beam splitter 15B) and couple in a beam to the beam path of the surgical microscope 2 (beam splitter 15A).

[0056] In the present exemplary embodiment, the beam splitter 15A is used in the partial beam 9A to mirror information or data for the surgeon into the partial beam 9A of the surgical microscope 2 by means of a display 37 (e.g. a digital mirror device (DMD) or an LCD display) and an associated optical unit 39 (by means of the beam splitter 15A). The camera adapter 19 has a camera 21 fastened thereto, which camera, equipped with an electronic image sensor 23, for example a CCD sensor CMOS sensor, is arranged in the interface arrangement 13B out of the other beam path 9B. By means of the camera 21, an electronic image, in particular a digital image, of the tissue region 3 can be recorded. In particular, a hyperspectral sensor can also be used as an image sensor, which hyperspectral sensor has not only three spectral channels (e.g. red, green and blue), but also a variety of spectral channels. In order to provide an image with depth information, it can be provided that a camera with components assigned thereto is provided accordingly in each partial beam path 9A, 9B. Furthermore, provision can be made such that in each partial beam path 9A, 9B a display with components assigned thereto is provided accordingly in order to be able to provide an image data record BD with depth information.

[0057] In the present example, the binocular tube 27 is adjacent to the interface arrangement 13A, 13B on the observer side. The binocular tube has two tube objective lenses 29A, 29B, which focus the corresponding parallel beams 9A, 9B onto the corresponding intermediate image planes 31A, 31B, that is, the observed object 3 is imaged onto the corresponding intermediate image planes 31A, 31B. The intermediate image located in the intermediate image planes 31A, 31B is then imaged by the eyepiece lenses 35A, 35B at infinity, so that the observer can observe the intermediate image with relaxed eyes. In addition, the distance between the two partial beams 9A, 9B is increased in the binocular tube by means of a reflector system or by means of prisms 33A, 33B, so that the distance is adapted to the intraocular distance of the observer. In addition, image erection is performed by means of a reflector system or prisms 33A, 33B.

[0058] In this surgical microscope, binocular tube 27 and interface arrangements 13A, 13B form a medical-optical display system, wherein a data superposition unit is formed by beam splitter prism 15A, as well as display 37 and optical unit 39 .

[0059] Furthermore, the surgical microscope 2 is equipped with an illumination device, by means of which the object field 3 can be illuminated with broadband illumination light. For this purpose, in the present example, the illumination device has a white light source 41, such as a halogen lamp or a gas discharge or the like. The light originating from the white light source 41 is guided via a deflection mirror 43 or a deflection prism in the direction of the object field 3 to illuminate the object field. Furthermore, an illumination optical unit 45 is present in the illumination device, which ensures that the entire object field 3 observed is illuminated uniformly.

[0060] See the following facts: Figure 1 The illumination beam paths shown in FIG. 1 are very schematic and do not necessarily replicate the actual reality of illumination beam paths. In principle, the illumination beam paths can be implemented as so-called oblique illumination, which is closest to Figure 1 In this type of oblique illumination, the beam path extends at a relatively large angle (6° or more) relative to the optical axis of the objective lens 5 and Figure 1 As shown, it can extend completely outside the objective lens. However, alternatively, it is also possible to allow the illumination beam path of the oblique illumination to extend over the marginal area of ​​the objective lens 5. Another option for the arrangement of the illumination beam path is the so-called 0° illumination, in which the illumination beam path extends over the objective lens 5 and is coupled into the objective lens along the optical axis of the objective lens 5 between the two partial beam paths 9A, 9B in the direction of the object field 3. Finally, the illumination beam path can also be implemented as a so-called coaxial illumination, in which there is a first illumination partial beam path and a second illumination partial beam path. These illumination partial beam paths are coupled into the surgical microscope 2 by means of one or more beam splitters in a manner parallel to the optical axes of the observation partial beam paths 9A, 9B, so that the illumination extends coaxially relative to the two observation partial beam paths.

[0061] exist Figure 1 In the embodiment variant of the surgical microscope 2 shown, the objective 5 consists only of an achromatic lens with a fixed focal length. However, it is also possible to use an objective lens system made of a plurality of lenses, in particular a so-called varioscope objective, by means of which the working distance of the surgical microscope 2, i.e. the distance between the object-side focal plane and the vertex of the first object-side lens surface of the objective 5 (also called the front focal length), can be varied. The object field 3 arranged in the focal plane is also imaged at infinity by the varioscope objective 50, and therefore there are parallel light beams on the observer side.

[0062] Figure 2 An example of a varioscope objective is schematically shown. The varioscope objective 50 comprises a positive component 51 (i.e. an optical element having a positive refractive power), Figure 2In addition, the varioscope objective 50 includes a negative component 52 (i.e. an optical element with negative refractive power) in Figure 2 The negative element 52 is located between the positive element 51 and the object field 3. In the illustrated varioscope objective 50, the negative element 52 has a fixed or immovable arrangement, while the positive element 51 is arranged to be displaceable along the optical axis OA, as indicated by the double arrow 53. When the positive element 51 is displaced to Figure 2 In the position shown by the dashed line in , the back focal length increases and thus the working distance of the surgical microscope 2 from the object field 3 changes.

[0063] Even if the positive member 51 has Figure 2 Although the negative element 52 is a displaceable configuration in FIG. 1 , it is in principle also possible to arrange the negative element 52 to be displaceable along the optical axis OA instead of the positive element 51. However, the negative element 52 usually forms the last lens of the varioscope objective 50. Therefore, the stationary negative element 52 offers the advantage that it is easier to seal the interior of the surgical microscope 2 against external influences. Furthermore, it should be noted that even if Figure 2 The positive component 51 and the negative component 52 in the figure are shown only as individual lenses, but each of these components can also be realized in the form of a lens group or a fixed lens instead of a single lens, for example in order to implement the varioscope objective as achromatic or apochromatic.

[0064] Figure 3 FIG. 4 shows a schematic representation of an example of a digital surgical microscope 48. In this surgical microscope 48, the main objective 5, the magnification changer 11, and the illumination systems 41, 43, 45 are connected with a Figure 1 The surgical microscope 2 with optical eyepiece shown in FIG. 1 is no different. The difference is that Figure 3 The surgical microscope 48 shown does not include an optical binocular tube. Figure 1 The tube objective lenses 29A and 29B, Figure 3The surgical microscope 48 comprises focusing lenses 49A, 49B, by means of which the binocular observation beam paths 9A, 9B are imaged onto digital image sensors 61A, 61B. Here, the digital image sensors 61A, 61B can be, for example, CCD sensors or CMOS sensors. The images recorded by the image sensors 61A, 61B are digitally transmitted to digital displays 63A, 63B, which can be implemented as LED displays, LCD displays, or displays based on organic light-emitting diodes (OLED). Also in the current example, eyepiece lenses 65A, 65B can be assigned to the displays 63A, 63B, by means of which the images displayed on the displays 63A, 63B are imaged at infinity so that the observer can observe the images with relaxed eyes. In this surgical microscope, the displays 63A, 63B and the eyepiece lenses 65A, 65B form the medical-optical display system. Here, the data superposition unit is a digital unit for superimposing image data records into at least one image displayed on the displays 63A, 63B. The image data records an image electronically superimposed on a corresponding image. The display 63A, 63B and the eyepiece lenses 65A, 65B may be part of a digital binocular tube, but they may also be part of a head mounted display (HMD), such as a pair of smart glasses. The head mounted display may be implemented as a virtual reality display or an augmented reality display.

[0065] even though Figure 3 Similar to Figure 1 Only the achromatic lens 5 with fixed focal length is shown. Figure 3 The surgical microscope 48 shown can be used as Figure 1 The surgical microscope 2 shown in FIG. 1 also includes a varioscope objective lens instead of the objective lens 5. In addition, Figure 3 The transfer of images recorded by the image sensors 61A, 61B to the displays 63A, 63B by means of cables 67A, 67B is shown. However, instead of in a wired manner, these images can also be transferred wirelessly to the displays 63A, 63B, in particular if the displays 63A, 63B are part of a head mounted display.

[0066] Now see additionally Figure 4 .

[0067] The situation in which the surgeon observes the object image OB obtained by the surgical microscope 2 from the object O with the aid of the medical-optical display system is shown. The object O can be an independent object or a group of objects. In addition, the object O can also be a region, such as an operating region, and thus this can also be the region in question (ROI). Then, the remaining area of ​​the object image OB shows the surroundings of the operating region. It should be noted that the object image OB passes through Figure 1The surgical microscope shown in 2 and Figure 3 48. In other words, these object images are optical data displayed optically or electronically.

[0068] In order to provide the surgeon with additional information items, such as a status record of an instrument, the medical-optical display system has a device for superimposing the data of the image data record BD into the object image OB, wherein the visualization of the image data record BD provides the additional information items. For this purpose, the device comprises, for example, an interface arrangement 13B with a beam splitter prism 15B and a display 37 with an assigned optical unit 39. Thus, during operation, the image data record BD presented in electronic form is converted by the display 37 into optical data and input coupled or mirrored into the beam path of the surgical microscope 2. Deviating from this, the superposition can also include an electronic superposition of the image content, i.e. the image data record BD is presented in electronic form and is inserted into the electronically presented image by replacing a part of the image or covering the part in an electronically transparent or opaque manner (e.g., as in Figure 3 48 shown in the surgical microscope).

[0069] Now see also Figure 5 , an explanation is provided regarding the structure of a device 70 which, during operation, ensures that relevant image content of an object image OB is not obscured during data superimposition of an image data record BD.

[0070] In the present exemplary embodiment, the device 70 has an area determination unit 72 and a data superposition unit 74. By using the area determination unit 72, an area B with little activity can be determined within the object image OB, while the data superposition unit 74 can be used to superimpose the image data record BD in this area B. Here, the area B with little activity can be an area in the object image OB, for example, in which the surgeon is not active and therefore has only a slight change in the image content. For example, if the surgeon moves with his fingers in the corresponding area or the surgical instrument is in a moving state, the change in the image content will not be very small or very large. Therefore, there is a significant change in the intensity value of the corresponding pixel.

[0071] In the present exemplary embodiment, the region determination unit 72 has a read-in unit 76 , an evaluation unit 80 , a comparison unit 82 , a clustering unit 84 , and a specification unit 86 .

[0072] The read-in unit 76 reads in a raw image data record RD. The raw image data record RD forms a time series of a plurality of independent object images OB, which reproduce the temporal outline of the events shown in the object image OB. The raw image data record RD can be a video sequence of a video stream in which the object O is recorded. The raw image data record can be obtained, for example, by means of Figure 1 Camera 21 or with the aid of Figure 3 The original image data record RD is preferably generated by the image sensors 61A, 61B. If a plurality of cameras 21 are used, these corresponding raw image data records are initially fused (registered or stitched). Preferably, the original image data record RD does not contain a superimposed image data record BD, at least when the superimposed image data record BD contains time-varying image content.

[0073] The evaluation unit 80 determines the temporal variability V of the image points of the raw image data records. It represents the temporal change of the image portion represented by the image point. In this case, the image points can be pixels of the raw image data records RD or groups of pixels made up of 2×2, 2×3, 2×4, 3×3, 3×4, 4×4, etc. pixels of the raw image data records RD. For example, the temporal variability can be determined by summing the absolute values ​​of the differences in pixel values ​​between the same pixel points in temporally consecutive raw images. The more frequently pixel value differences with large absolute values ​​occur, the greater the sum becomes. In contrast, in the case of pixel value differences with only small absolute values, the sum remains small. In the case of pixel groups, the sum of the contributions of the differences in the average pixel values ​​of the pixel groups can be summed up accordingly. When forming the sum, preferably a plurality of object images are used, which go back a certain time length starting from the current object image. This time length can be adjustable in order to be able to adapt it to different temporal dynamics of the image content. Rapidly changing image content requires in this case a shorter duration than slowly changing image content.

[0074] The comparison unit 82 compares the temporal variability V with a set variability limit value GV. The set variability limit value may be adjustable to be able to take into account an expected temporal variability V of image points in a region B with little activity. If a relatively low temporal variability V is desired in the region B, the variability limit value may be kept lower than a relatively high temporal variability V desired in the region B (but still lower than in regions with a lot of activity).

[0075] The clustering unit 84 clusters the image points whose confirmed time variability is less than the variability limit value GV. Due to the clustering, one or more continuous regions with small time variability are formed. Optionally, there may also be an image processing unit, which synthesizes multiple independent continuous regions in these continuous regions with small time variability into a larger continuous region with small time variability through morphological image processing. Then the designation unit 86 designates the at least one continuous region with small time variability as the region B with little activity.

[0076] In the current exemplary embodiment, the data superimposition unit 74 is implemented to take into account the size and / or form of the image data record BD to determine the position in the area B with little activity at which the image data record is superimposed, so as to perform the superimposition at a portion in the area B with little activity that matches the size and form of the image data record BD to be superimposed.

[0077] Now see also Figure 6 , the following method is explained: the method ensures that the image content of the object image OB is not obscured during the data superposition of the image data record BD. To this end, in a first step S100, a region B with little activity is identified within the object image OB. Then, in a further step S200, the image data record BD is superimposed on the identified region B.

[0078] In order to confirm the region B with little activity, the original image data record RD of the object image OB is acquired in the intermediate step S110, and the temporal variability V of the image points of the original image data record is confirmed (intermediate step S120). The confirmed temporal variability V is compared with the set variability limit value GV (intermediate step S130), and the pixels whose confirmed temporal variability is less than the variability limit value GV are gathered (intermediate step S140) to generate at least one continuous region with small temporal variability. Finally, the at least one continuous region with small temporal variability is designated as the region B with little activity (intermediate step S150).

[0079] In a development of the method, the depth map of the observed object can be used to exclude in advance certain areas where a lot of activity is expected. For example, in the case of a deep operating channel, a lot of activity is expected, especially in the area of ​​the operating channel.

[0080] The step of confirming the area B with little activity within the object image OB can be performed by using a suitable algorithm. For example, a possible algorithmic approach is an analysis by means of optical flow or machine learning, for example via a neural network. The method can be performed on a PC or a computer, which has a data transfer link with the medical-optical observation device to read data from the camera 21 and output the image data record BD to actuate the display 37. To this end, the PC or computer can have appropriate hardware components and / or software components. Alternatively, the method can also be performed by a computer component with appropriate hardware components and / or software components, which belongs to the medical-optical observation device.

[0081] The present invention has been described in detail for the purpose of explanation based on exemplary embodiments. However, those skilled in the art recognize that there may be deviations from the exemplary embodiments within the scope of the present invention. For example, a medical-optical display system that deviates from this exemplary embodiment can be implemented as a part of an endoscope or an endoscopic microscope. Those skilled in the art will determine other options for developing exemplary embodiments within the scope of the present invention. Therefore, the present invention is not intended to be limited to the described exemplary embodiments, but is only limited by the appended claims.

[0082] Reference Number List

[0083] 2. Surgical Microscope

[0084] 3 Operation field / observation object

[0085] 5 Objective lens

[0086] 7 Beam

[0087] 9 Beam

[0088] 9A Stereoscopic Partial Beam Path

[0089] 9B Stereoscopic Partial Beam Path

[0090] 11 Magnification Changer

[0091] 13A interface arrangement

[0092] 13B Interface Arrangement

[0093] 15A Beam Splitter Prism

[0094] 15B Beam Splitter Prism

[0095] 19 Camera Adapter

[0096] 21 Camera

[0097] 23 Image Sensor

[0098] 27 Binocular tube

[0099] 29A Tube Objective

[0100] 29B Tube Objective

[0101] 31A Intermediate Image Plane

[0102] 31B Intermediate Image Plane

[0103] 33A Prism

[0104] 33B Prism

[0105] 35A Eyepiece Lens

[0106] 35B Eyepiece Lens

[0107] 37 Display

[0108] 39 Optical unit

[0109] 41 White light source

[0110] 43 Deflecting mirror

[0111] 45 Illumination optical unit

[0112] 48. Surgical Microscope

[0113] 49A Focusing lens

[0114] 49B Focusing lens

[0115] 50 Varifocal Objectives

[0116] 51 Positive component

[0117] 52 Negative component

[0118] 53 Double-headed arrow

[0119] 61A Image Sensor

[0120] 61B Image Sensor

[0121] 63A Display

[0122] 63B Display

[0123] 65A Eyepiece Lens

[0124] 65B Eyepiece Lens

[0125] 67A Cable

[0126] 67B Cable

[0127] 70 Devices

[0128] 72 Area Determination Unit

[0129] 74 Data Overlay Unit

[0130] 76 Reading unit

[0131] 80 evaluation units

[0132] 82 Comparison unit

[0133] 84 Aggregation Unit

[0134] 86 Designated Units

[0135] Area B

[0136] BD image data recording

[0137] O Object

[0138] OA optical axis

[0139] OB Object Image

[0140] RD Raw Image Data Record

[0141] V Variability

[0142] GV Variability Limit

[0143] S100 Identify areas with little activity

[0144] S110 captures raw image data records

[0145] S120 Confirm the time variability of image points

[0146] S130 compares the time variability with the set variability limit

[0147] S140 Gathering

[0148] S150 Designates an area with little activity

[0149] S200 Overlay image data recording.

Claims

1. A method for operating a medical optical display system for displaying an object image (OB) of an observed object (O), the object image having been acquired by means of a medical optical viewing device (2, 48), wherein the medical optical display system comprises a data superimposition unit (74) for superimposing data of at least one image data record (BD) into the object image (OB), the method comprising the following steps: - (S100) determining at least one region (B) with little activity in the object image (OB), wherein the region with little activity is a continuous region with little temporal variability, and - (S200) superimposing the at least one image data record (BD) into the at least one region (B) with little activity, The following intermediate steps are performed to determine the region (B): - (S110) capturing a raw image data record (RD) representing a time series of a plurality of independent object images (OB), - (S120) determining the temporal variability of the image points in these raw image data records (RD), - (S130) comparing the time variability (V) with a set variability limit value (GV), and - (S140) aggregating those image points whose confirmed temporal variability is less than the variability limit value (GV) to generate at least one continuous region with small temporal variability, and -(S150) designating the at least one continuous region with small time variability as the at least one region with little activity (B).

2. The method according to claim 1, wherein: A plurality of independent continuous regions with small time variability are synthesized into a larger continuous region with small time variability through image processing, and the larger continuous region with small time variability is designated as the at least one region with little activity (B).

3. The method according to claim 1 or 2, wherein: The image points are the pixels of the raw image data records (RD).

4. The method according to claim 1 or 2, wherein: The image points are groups of pixels of the raw image data records (RD).

5. The method according to claim 1 or 2, wherein: The position of superimposing the image data record (BD) in the at least one region (B) with little activity is determined taking into account the size and / or the form of the image content displayed in the image data record (BD).

6. The method according to claim 1 or 2, wherein: The depth map of the observed object (O) is used to exclude in advance certain areas in the raw image data records (RD) where a lot of activity is expected as the at least one area with little activity (B), and / or areas in the raw image data records (RD) where an instrument or a face is imaged are identified and excluded in advance as the at least one area with little activity (B) based on pattern recognition. 7 . A computer program product containing a program code, the program code having computer-readable instructions for implementing the method according to claim 1 when the program code is loaded into a computer and / or executed in a computer.

8. A medical-optical display system for displaying an object image (OB) obtained by using a medical-optical observation device (2, 48) of an observed object (O), comprising: a data superimposition unit (74) which is designed to superimpose at least one image data record (BD) into the object image (OB), and a region determination unit (72) which is designed to determine at least one region (B) with little activity within the object image (OB), wherein the data superimposition unit (74) is designed to superimpose the at least one image data record (BD) on the at least one region (B) of the object image (OB) having little movement, in, The area determination unit (72) comprises: - a read-in unit (76) for reading in a raw image data record (RD) representing a temporal sequence of a plurality of individual object images (OB), an evaluation unit (80) for determining the temporal variability of the image points in the raw image data records (RD), - a comparison unit (82) for comparing the time variability (V) with a set variability limit value (GV), - a clustering unit (84) for clustering those image points whose determined temporal variability is smaller than the variability limit value (GV) to generate at least one continuous region with small temporal variability, and - a designation unit (86) for designating the at least one continuous region with small time variability as the at least one region (B) with little activity.

9. The medical-optical display system according to claim 8, further comprising: A medical-optical observation device (2, 48) is used for observing an object (O) and for recording an image of the observed object (O) as the object image (OB).

10. The medical-optical display system according to claim 8, wherein: - there is an image processing unit (85) which combines a plurality of independent continuous regions with small time variability into a larger continuous region with small time variability by image processing, and - the designation unit (86) is configured to designate the larger continuous area with small time variability as the at least one area with little activity (B).

11. The medical-optical display system according to claim 8, wherein: The image points are the pixels of the raw image data records (RD).

12. The medical-optical display system according to claim 10, wherein: The image points are the pixels of the raw image data records (RD).

13. The medical-optical display system according to claim 8, wherein: The image points are groups of pixels of the raw image data records (RD).

14. The medical-optical display system according to claim 10, wherein: The image points are groups of pixels of the raw image data records (RD).

15. The medical-optical display system according to claim 8 or 9, wherein: The data superimposition unit (74) is implemented to take into account the size and / or form of the image content displayed in the image data record (BD) in order to determine a position for superimposing the at least one image data record (BD) in the at least one area (B) with little activity.

16. The medical-optical display system according to claim 8 or 9, wherein: The area determination unit (72) includes a pre-selection device, which excludes in advance certain areas in the original image data records (RD) where a large amount of activity is expected as the at least one area with little activity (B) based on the depth map of the observed object (O), and / or excludes in advance certain areas in the original image data records (RD) where an imaged instrument or a face is visible as the at least one area with little activity (B) based on pattern recognition.

Images