Projection apparatus and method for the shadow-free projection of image data

Abstract

The invention relates to a projection apparatus (10) for projecting image data onto an uneven surface (20), comprising: a depth-information acquisition device (26) which comprises at least one acquisition unit (28, 30) for acquiring depth information; a plurality of projection units (12, 14); a shadow detection unit (44) for detecting an object (42) in the light beam (40); and a control unit (46) which is designed to determine, on the basis of the position data, the direction data and the shadow information, which of the projection units (12, 14) is suitable for projecting onto the surface (20), without shadows, a predetermined part of the image data which are to be projected, to process the original image data (34) into changed image data (36) using the depth information relating to the surface (20), the position data and the direction data relating to a particular projection unit (12, 14), and to output the predetermined part of the changed image data (36) to a projection unit (12, 14) for projection onto the surface (20) by said projection unit, so that a reflection (38) of the changed image data (36) which are projected onto the uneven surface (20) substantially corresponds to the original image data (34).

Description

[0001] Projection device and method for shadow-free projection of image data

[0002] Description

[0003] The present invention relates to a projection device suitable for projecting image data onto an uneven surface, in particular a surgical drape.

[0004] Projection devices suitable for projecting image data onto an uneven surface are known in the prior art. Such projection devices are used, for example, for projection onto uneven walls or fabric projection screens. For this purpose, the surface of the projection area can be determined before the projection begins, and corresponding image data can be output to a stationary projection unit in such a way that the unevenness of the projection surface is taken into account in the image data to be projected, so that the unevenness of the projection surface is virtually imperceptible to a viewer of the projected image.

[0005] However, a disadvantage of such projection devices is twofold: firstly, they are often only designed to project onto unchanging uneven surfaces, meaning that changes in the unevenness of the projection surface cannot be taken into account during the projection process; and secondly, they are not designed to display a complete projection image even in cases where an object obstructs the beam path between the projection unit and the projection surface.

[0006] It is therefore the object of the present invention to provide a

[0007] To provide a projection device with improved projection characteristics. In particular, it can be considered an object of the present invention that the projection device is configured to adapt image data to be projected to a changing unevenness of the projection surface during projection operation and / or to ensure the projection of a complete projection image even in the case in which an object enters the beam path of a projection unit.

[0008] This problem is solved according to the invention by a projection device suitable for projecting image data onto an uneven surface, in particular a surgical drape, wherein the device comprises: a depth information acquisition device comprising at least one acquisition unit and configured to acquire at least spatial depth information of the uneven surface onto which the image data is to be projected; a plurality of projection units configured to emit light such that image data provided to a respective projection unit is projected onto the surface; a shadow detection unit configured to detect an object located in the light beam from at least one of the projection units to the surface to be irradiated, which obscures the surface to be irradiated in such a way thatthat the light emitted by the respective projection unit incompletely reaches the surface to be illuminated, and in this case to output corresponding shading information, and a control unit which is configured to receive and / or contain depth information from the depth information acquisition device, position data of a spatial position and direction data of a light emission direction of a respective projection unit and original image data, wherein the control unit is further configured to determine, on the basis of the position data, the direction data and the shading information, which of the projection units is suitable to project a predetermined part of the image data to be projected onto the surface without shading, to modify the original image data with the depth information of the surface, the position data and the direction data of a respective projection unit,In particular, distorted image data is processed, and the predetermined part of the modified image data is output to a correspondingly defined projection unit to be projected onto the surface, so that a reflection of the modified image data projected onto the uneven surface essentially corresponds to the original image data.

[0009] Thus, the projection device according to the invention is designed to take into account a changing surface of the projection surface by repeatedly capturing the spatial depth information of the uneven surface and outputting correspondingly modified image data to a respective projection unit. In this way, the impression of an image perceived by at least one viewer can be maintained essentially unchanged even with a changing unevenness of the surface.

[0010] Furthermore, the projection device according to the invention, by comprising a plurality of projection units and by providing a shadow detection unit which is configured to detect that the light beam emitted by a particular projection unit is at least partially obscured by an object, such as the head or hand of an operator, is to distribute the output of predetermined sub-areas of the image to be projected to the different projection units, which project light onto the surface from different angles to each other, in such a way that the part of the light beam shadowed by the object in the beam path of one projection unit can be projected onto the surface through the (at least in this sub-area) unshaded beam path of another projection unit.

[0011] In a conceivable application, relevant patient information and / or system information from devices used in an operating room can be projected onto suitable surfaces in such a way that a surgeon or another person in the room is provided with relevant information directly in the respective work area, for example, directly next to a surgical wound. If a hand, head, or other object moves into the beam path of one of the projection units, this would result in the relevant information being displayed incompletely. To compensate for this, the projection device according to the invention can immediately display the area shadowed in the beam path of one projection unit using at least one other projection unit.

[0012] In this context, it should be noted that the term "surface" is not limited to the surface of a single element, such as a single surgical drape. The surface onto which the image data is to be projected can be composed of a wide variety of elements, materials, or colors.

[0013] Furthermore, the surface to be irradiated, or the image data to be projected, need not be a single, contiguous area. The image data can comprise several separate areas, which are to be projected onto different parts of the surface. These separate areas could, for example, include at least one surgical drape on the one hand and an operating table or instrument table on the other. It is also conceivable to project one area of ​​the image data onto a surgical drape near the surgical field and a separate area onto a section of the surgical drape farther from the surgical field or onto a different surgical drape altogether.

[0014] Thus, the majority of projection units can be configured to irradiate a significantly larger projection field than is actually used for projecting the image data. For example, the maximum possible irradiation field of the majority of projection units can encompass at least an area of ​​2 m x 1 m, and in particular at least an area of ​​2 m x 3 m, such as the entire sterile area of ​​an operating room.

[0015] In contrast, the area actually irradiated at any given time can be approximately 10% of the maximum possible irradiation field, for example, 20 cm x 30 cm or 30 cm x 40 cm. Accordingly, the area actually irradiated can be adjusted to the ambient conditions as needed, meaning its size and / or position can be changed without requiring any adjustment of the projection units.

[0016] Furthermore, it should be noted that the functional units described within the scope of this invention do not necessarily have to be designed as separate components. For example, it is conceivable that the functions of the depth information acquisition device and the shadow detection unit can be implemented by one and the same component. For example, the depth information acquisition device can not only acquire spatial information about the surface to be irradiated, but also spatial information, in particular depth information, about areas surrounding the surface, especially an entire operating field.Based on this and with knowledge of the positions and orientations of the projection units in space, it can then be determined whether there is an object in the beam path of at least one projection unit which can lead to a shadowing of the image to be projected by the respective projection unit (function of the shadow detection unit).

[0017] Alternatively or additionally, it is conceivable that at least one of the projection units also forms part of the depth information acquisition system or is used for this purpose. For example, a predetermined pattern, such as a grid, can be projected from one of the projection units onto the surface to be irradiated, which is then captured by at least one camera, and in particular by several cameras. Based on the distortion of the predetermined pattern by the uneven surface, the unevenness of the surface can then be determined without the depth information acquisition system itself needing to be configured to determine spatial depth information.

[0018] Furthermore, it should be noted that the "light beam" emitted by a particular projection unit should be directed in a primary direction. Typically, the light beam emitted by a given projection unit will spread out conically from the projection unit towards the surface; that is, the light rays are not parallel to each other. The "primary direction" therefore refers to the fundamental direction of the light emitted from the projection unit to the surface. For example, the primary direction of emission may be located at the center of a light cone emanating from the projection unit. The light-emitting unit of a given projection unit can be, for example, an LED light source, a laser light source, or another suitable light source.

[0019] Advantageously, the control unit can be configured to determine the suitable projection units for shadow-free projection, to process the depth information of the surface, the position data, and the direction data of each projection unit into modified image data, and to output the predetermined portion of the modified image data to each projection unit at least once per second, in particular at least 30 times per second, and advantageously at least 90 times per second. In this way, the projection device can adapt to changing conditions, such as a change in surface unevenness, a change in the operating table, or a change in shading caused by an object (shading begins, increases, decreases, or ends), or similar events, almost in real time.

[0020] Especially in cases where the projection device is designed to adapt to changing conditions at least once per second, but not limited to this, it may be intended that an object moving through the beam path of a projection unit at a certain minimum speed, such as at least 0.5 m / s, and particularly at least 1 m / s, does not cause any change in the projection properties. In such a case, a brief dimming of the projection may be acceptable or even more intuitive for a viewer of the projected image data.

[0021] Furthermore, at least one recording unit of the

[0022] The depth information acquisition device shall be configured to acquire depth information based on stereoscopy and / or time-of-flight and / or frequency modulation, wherein the acquisition unit is configured in particular as an RGB camera or depth camera, or as an IR camera, or as a LiDAR sensor, or as a pattern transmitter / receiver unit. Of course, the foregoing mention of possible technologies for depth acquisition is not to be understood as an exhaustive list. Any suitable technology that makes it possible to obtain information about the unevenness of the surface and / or information that can be used to detect an object that shadows the projection can be used in conjunction with the projection device according to the invention.

[0023] The depth information acquisition device can, in particular, comprise a plurality of acquisition units configured to acquire depth information of the uneven surface from different perspectives. This prevents the loss of depth information when a light beam from a projection unit and a acquisition unit of the depth information acquisition device are both obscured and the view axis of the acquisition unit of the depth information acquisition device is simultaneously obscured, as the necessary information can be gathered by at least one additional acquisition unit of the depth information acquisition device.

[0024] In one possible embodiment of the present invention, the at least one detection unit of the depth information acquisition device can be arranged in a fixed position relative to at least one of the projection units, and in particular to all projection units. This can mean that the at least two projection units arranged in a fixed position relative to each other are fixed with respect to their relative position and / or their relative orientation. This fixed relative position and / or orientation can be determined, for example, by an initial measurement or calibration, and stored as data in the control unit. Based on this known relative position and / or orientation, the position and / or orientation of the respective projection unit in space and relative to the surface can then be deduced.This means that by knowing the position and / or orientation of a given projection unit in space and the position of the surface onto which image data is to be projected, a main emission direction of this projection unit can be determined. Thus, based on a known position and size of an object (for example, measured by the depth information acquisition device), it can be determined for which projection unit and for which part of its emitted light beam the object will cast a shadow.

[0025] The at least one detection unit of the depth information acquisition device and at least one of the projection units, in particular all projection units, can be arranged on or in a room wall, especially a room ceiling. A room wall here is understood to be, in particular, a wall that delimits a room, especially an operating room. Thus, suspended or projecting walls (climate-controlled ceilings, wall panels, etc.) are also to be understood as room walls. For example, the at least one detection unit of the depth information acquisition device and the at least one projection unit can be arranged in a climate-controlled ceiling of an operating room. It is conceivable that the arrangement is such that at least one of the units is flush with the surrounding ceiling, or that at least one of the units projects from the ceiling into the room.In the event that multiple lighting units are to be provided in the respective room wall, at least one detection unit of the depth information detection device and at least one projection unit can be arranged in the room wall instead of or in addition to a respective lighting unit.

[0026] Alternatively or additionally, at least one of the depth information acquisition units can be arranged to be movable relative to at least one of the projection units. It should be noted that it is entirely conceivable that the depth information acquisition unit is arranged to be movable relative to at least one of the projection units, but fixed relative to at least one other of the projection units. A relocation, in particular a change of orientation, of the respective projection unit and / or the depth information acquisition unit can be effected, for example, by associated actuators, such as at least one electric motor.

[0027] The projection device can further include a calibration unit configured to detect changes in the position and / or orientation of at least one detection unit of the depth information acquisition device relative to at least one of the projection units and to output correspondingly changed spatial coordinates of the detection unit and / or the projection unit, or a correspondingly changed relative position and / or relative orientation, to the control unit. This recalibration of the projection device can be performed in various ways. For example, it is conceivable that an extrinsic camera-based calibration is performed, in which a known pattern is projected onto the surface, which is then detected by at least one camera.Due to a deviation between the target image data and the actual image data, a change in the relative position and / or orientation of the projection device units can be inferred, and the relative positions and / or orientations stored in the control unit can be updated accordingly. It is also conceivable to equip the projection device units with a tracking function, so that the positions and / or orientations of the projection device units in space can be detected by a tracking device. Components of the projection device, such as at least one detection unit of the depth information acquisition device, can also be part of the tracking device.Furthermore, it is described that a changed relative position and / or relative orientation of the detection unit and / or the projection unit can also be determined on the basis of a control signal, which is issued to an assigned actuator for repositioning the unit(s), by means of a target change of the position and / or orientation of the unit(s) by the actuator based on the control signal, which also simultaneously serves to determine the changed relative position and / or relative orientation of the unit(s).

[0028] For example, at least one detection unit of the depth information acquisition device and / or at least one of the projection units can be arranged in an operating light, in particular in the lamp housing of the operating light. The term "lamp housing" is understood to mean the basic body in which the light sources of the operating light are arranged. Since it is in the interest of the operating personnel, especially the surgeon, that the surgical field is always adequately illuminated, an operating light is usually positioned so that it can freely illuminate the surgical field.If at least one detection unit of the depth information detection device and / or at least one of the projection units is also arranged in the operating light, then adjusting the operating light also ensures that the projection device has an unobstructed line of sight / irradiation to the operating field or to the projection surface.

[0029] In this context, the operating light can include an articulated arm to which the light body is attached. At least one, and in particular all, of the joints of the articulated arm is equipped with a sensor unit designed to detect movement of the respective joint and output a corresponding signal to the control unit. Thus, the recalibration of the projection device units described above—that is, the detection of changes in the relative positions and / or orientations of the projection device units and their output to the control unit—can also be performed based on corresponding sensor data. The articulated arm can, in turn, be mounted on a wall or on a tripod, which is particularly movable within the room.

[0030] Furthermore, at least one of the depth information acquisition units and / or a gesture acquisition unit can be configured to capture user gestures and output corresponding signals to the control unit, and / or a speech acquisition unit can be configured to capture user speech and output corresponding signals to the control unit, and / or a remote control unit can be configured to capture a user's operation of a remote control device and output corresponding signals to the control unit. In the (or another) control unit, these signals can then be compared with stored gestures and / or stored voice commands and / or stored remote control commands, and if there is a sufficiently high degree of similarity, a control command can be assigned or executed.Such a remote control unit can be, in particular, a (tracked) pointer and / or a joystick and / or a touchscreen. The control unit can be configured to spatially relate the user's gestures, detected by the at least one detection unit of the depth information detection device and / or the gesture detection unit, to the image data projected onto the surface. In this way, identical gestures in different areas of the projected image can trigger different control actions. For example, a linear movement of the user's finger in a first area of ​​the projected image can cause a change in the brightness of the operating light, and a linear movement of the user's finger in a second area of ​​the projected image can cause a change in the focus of the operating light.In particular, it may be provided that the user does not have to touch the surface onto which the image is projected, but that it is sufficient for control if the user performs the gesture at a certain distance from the surface in order not to violate sterility requirements in surgical procedures.

[0031] Advantageously, the predetermined portion of the modified image data, which is projected onto the surface by a respective projection unit, can exhibit a gradually decreasing brightness at its outer edge, from the inside out. The predetermined portion can be an arbitrarily shaped, particularly non-rectangular, sub-area of ​​the image data to be projected. The gradually decreasing brightness can decrease essentially uniformly from the predetermined portion outwards, or it can decrease in at least one step, and in particular in several steps, from the predetermined portion outwards, with this at least one step being located, in particular, at the edge. Of course, it is also conceivable that only the edge section exhibits a gradually decreasing brightness towards the outside.The edge section of the image data portion can, for example, be formed by an outer edge of the image data, which extends inwards from an outer edge by at most 30%, particularly at most 20%, and advantageously at most 10%, of the largest diameter of the image data portion. The edge section of the image data portion, viewed in its circumferential direction, can be formed only partially or completely. In this way, when the predetermined parts of the projected image data are assembled on the surface, it can be achieved that adjacent parts overlap slightly without a disturbingly perceptible, for example, abrupt, increase in brightness in the overlap area.This also prevents the case where the parts do not exactly border each other and a gap would arise between the parts, since the boundary areas of two mutually adjacent predetermined parts of the projecting image data are present in each of the two parts.

[0032] According to a further aspect of the present invention, the problem described above is solved by a projection method to project image data onto an uneven surface, in particular a surgical drape, wherein the method comprises the following steps:

[0033] Acquisition of at least spatial depth information of the uneven surface onto which the image data are to be projected, by a depth information acquisition device comprising at least one acquisition unit,

[0034] Providing a plurality of projection units which emit light in such a way that image data provided to a respective projection unit is projected onto the surface; detecting, by means of a shadow detection unit, an object located in the light beam from at least one of the projection units to the surface to be irradiated, which obscures the surface to be irradiated in such a way that the light emitted by the respective projection unit incompletely reaches the surface to be irradiated; and, in this case, outputting corresponding shadow information.

[0035] Providing a control unit which receives depth information from the depth information acquisition device, position data of a spatial position and direction data of a light emission direction of a respective projection unit and original image data, wherein the method further comprises the steps of:

[0036] The control unit determines which of the projection units is suitable for projecting a predetermined portion of the image data onto the surface without shadowing, based on the position data, the direction data, and the shadowing information.

[0037] Processing the original image data with the depth information of the surface, the position data and the direction data of a respective projection unit to create modified, especially distorted, image data, and

[0038] Outputting the predetermined portion of the modified image data to a suitably defined projection unit to be projected onto the surface, so that a reflection of the modified image data projected onto the uneven surface essentially corresponds to the original image data.

[0039] It should be noted at this point that at least some, in particular all, of the features, advantages and effects mentioned with reference to the projection device according to the invention may also be applicable to the projection method according to the invention, and vice versa.

[0040] A “reflection” in the sense of the projection method according to the invention can be at least one specific reflection, that is, light reflected from the surface in at least one specific direction.

[0041] In particular, the steps of determining the suitable projection units for shadow-free projection, processing the depth information of the surface, the position data, and the direction data of a respective projection unit to modify image data, and outputting the predetermined portion of the modified image data to a correspondingly determined projection unit can be performed at least once per second, in particular at least 30 times per second, advantageously at least 90 times per second. As already described with reference to the projection device according to the invention, continuous adaptation to the actual situation can thus take place. This can result in a user, in particular an operator, remaining unaware that the beam path of a projection unit has been obscured and that the obscured area is projected onto the surface by an unobstructed projection unit.

[0042] It should be added at this point that the selection of the projection unit intended to project a predetermined part of the image data to be projected, or the assignment of the respective predetermined part of the image data to be projected to a respective projection unit, can be carried out in at least one of the following ways, for example, and it should be explicitly mentioned here that these two ways can be used individually or in combination.

[0043] Firstly, at least one pixel of the image data to be projected, in particular a predetermined portion (i.e., a pixel range) of the image data to be projected, which is advantageously designed as a contiguous area, can be assigned to the projection unit whose positional and / or directional data of the associated light emission direction, compared to the other projection units, exhibit the greatest distance to a shadow cast by an object in the beam path of at least one projection unit from the object to the surface onto which the image data is to be projected, and / or to the source object itself. In this way, a fast response time of the system can be ensured, effectively compensating for any latency of a sensor unit used, in particular the depth information acquisition device, when dealing with a moving source object.

[0044] Secondly, a section of the image data to be projected can be defined, which is specifically surrounded by an area in which no image data to be projected is present (particularly in the case of the image data to be projected being displayed as separately represented tiles) and / or which is adjoined by a section of the image data to be projected that differs, for example, in terms of content and / or graphics. The section of the image data to be projected in this way can then be assigned to the projection unit that can display this section without shadowing.In particular, the assigned projection unit suitable for projecting the entire specified section (the entire tile) may also be the projection unit that has the greatest distance to a shadow cast by an object in the beam path of at least one projection unit from the object to the surface onto which the image data are to be projected, and / or to the providing object itself.

[0045] If the area shadowed by the object changes and a new allocation of the projected image data to the respective projection units is required, a temporary blending between the projection units can occur. This means that a given image data area can be projected by more than one projection unit for a period of time before being projected onto the surface by only a single projection unit again. This can be particularly advantageous in the case of tile-like area allocation, as soon as part of the tile is shadowed.

[0046] If none of the projection units is able to project such a tile completely onto the surface, then the first (pixel-based) display mode can be used for that one tile, i.e., for that specific section of the image data to be projected, or for the entire image data to be projected.

[0047] The above-mentioned assignments can be carried out in advance, that is, before or at the beginning of the use of the projection device or the projection method, or during the use, for example automatically by segmentation algorithms.

[0048] In both methods, only a subset, and in particular only a single projection unit, can be used for the primary projection of a given portion of the image data to be projected. "Primary projection" here means that the respective projection is largely, and especially completely, performed by a single projection unit, but that there can be exceptions, such as the edge sections described above or the temporary multiple projection of a portion of the image data by several projection units when switching a projection assignment from one projection unit to at least one other.

[0049] To determine the distance of a given projection unit to a shadow, a three-dimensional model of the surface onto which the image data is to be projected, and / or of at least one object located in a beam path of at least one projection unit to the surface, can first be generated, for example, using the at least one depth information acquisition device. This three-dimensional model is then calculated ("rendered") from the perspective of at least one projection unit, in particular from all respective perspectives of the projection units. In this way, it can be calculated which areas of the surface are not irradiated for which of the projection units. These non-irradiated areas of the surface are thus defined as "shadows" with respect to the respective projection unit.

[0050] The shadow can then also be calculated as a three-dimensional model (either as a standalone model or as additional information within the model described above). Alternatively or additionally, the shadow points can be stored as spatial coordinates in a shadow map. Based on known relative spatial coordinates and orientations of the respective projection unit(s), a distance to the respective projection unit can be determined for each spatial coordinate on the shadow map.

[0051] The following is an example of a process described in individual steps:

[0052] (1) To determine the shadow cast for each point in the separate point clouds of each depth information acquisition device (for example, an RGB-D camera), a given situation, in particular the surface onto which the image data is to be projected, and / or at least one object located in a beam path of at least one projection unit to the surface, is first rendered from the perspective of each projection unit. Distances are then rendered and combined from this. The result is a distance image of the given situation for each perspective of each projection unit.

[0053] (2) Next, each point from the structured point clouds is examined and its distance to each projection unit is determined. Each point is projected onto the projection unit's distance image and compared: if the distances match, the point is not included; if the stored distance is smaller, the point is shadowed by an object from the projection unit's perspective. This information is stored as values, specifically Boolean values, in texture channels, with each texture channel representing a specific projection unit. This results in a shadow-per-point map for each depth information acquisition device, corresponding to the original depth resolution of that device.

[0054] (3) From the previously calculated shadow-per-point map, shadow distance maps are efficiently generated, for example, using the jump-flooding algorithm. The shadow distance maps can be generated, for example, with a complexity of O(log n), where n denotes the maximum distance considered in a truncated distance map. By considering each texture channel of each projection unit independently, channel-specific shadow distance maps can be generated, which specify the respective distances to shadows from the depth perspective.

[0055] (4) Due to occlusions, it is possible that different depth information acquisition devices may determine the distance to a given shadow point and / or shadow area differently, and in particular, underestimate it. To resolve such discrepancies between the depth information acquisition devices, the respective shadow distance maps can be standardized. For this purpose, the distance values ​​from each depth information acquisition device can be projected onto another depth information acquisition device and blended. This can be done, for example, using interpolated texture lookups to ensure uniform, gapless sampling. To avoid temporal flickering due to occasional holes in depth images, temporal smoothing can be applied. (5) A projection unit assignment map can be created for the per-pixel assignment.Each point can be assigned to the projection unit with the greatest distance in the unified and time-smoothed shadow distance maps. The channel of the corresponding projection unit can then be set to 1, while all other channels can be set to 0. During distortion correction, this assignment map is sampled using a 3x3 kernel and interpolated texture lookups. This results in smooth transitions between the projection units in the resulting geometrically corrected image data ready for projection.

[0056] (6) For tile-based allocation, the minimum shadow distance for each tile can be determined to identify the optimal projection unit with the largest shadow distance within the tile section. To achieve this efficiently, a tile ID can be rendered onto each depth image at corresponding tile positions in the image data to be projected. The resulting texture (whose resolution may be slightly reduced for performance reasons, if necessary) can then be copied into the memory of a compute unit. The tile ID of each pixel and the corresponding uniform, time-smoothed value of the shadow distance map can then be used to calculate a minimum distance per tile.When rendering a corrected display, the projection unit assignments for pixels within a tile segment can be overridden, and the projection unit determined by the tile-based method can be assigned, provided the minimum distance is greater than zero. This ensures that the per-pixel assignment is used when no projection unit can render the entire tile without shadows.

[0057] Optionally, the maximum shadow distance can be limited to exclude movements and / or shadows that are too far away from the assignment decision, so that both assignment methods can be executed efficiently on the computing unit, especially a GPU.

[0058] The present invention will below be described in greater detail with reference to an exemplary embodiment and the accompanying drawings. It illustrates:

[0059] Figure 1 shows a schematic view of a projection device according to the invention;

[0060] Figure 2 shows a schematic view of an operating field in which projected image data is displayed;

[0061] Figure 3 shows a top view of the surgical field from Figure 2; and

[0062] Figure 4 shows the projection device according to the invention from Figure 1, in which an unevenness of the surface, original and projected image data are additionally shown.

[0063] In Figure 1, a projection device according to the invention is generally designated by reference numeral 10. In the embodiment shown in Figure 1, the projection device 10 comprises a first projection unit 12 and a second projection unit 14, each of which is configured to project light onto a surface at a light outlet opening 16 and 18, respectively.

[0064] As can be seen in Figure 1, the surface onto which the light or image data is to be projected by the projection units 12 and 14 is formed here by a surgical drape 20. The image data 22 to be projected onto the surgical drape 20 are arranged adjacent to a surgical wound 24. With further reference to Figures 2 and 3, it can be seen that the surface of the surgical drape 20 is uneven. Accordingly, a projection of the original image data would lead to a distorted reflection due to the uneven surface 20 and thus to a distorted image impression for a viewer.

[0065] To counteract this, the projection device 10 according to the invention further comprises a depth information acquisition device 26, which, in the embodiment shown in Figure 1, is arranged in a common housing with the projection unit 12 or with the projection unit 14. In the embodiment shown here, the depth information acquisition device 26 comprises two depth cameras 28 and 30, which are configured to capture at least that part of the uneven surface 20 onto which the image data 22 are to be projected, but in particular an entire surgical field, in three dimensions. Thus, an unevenness of the surface 20 can be captured three-dimensionally.The unevenness of the surface 20 (see box 32 in Figure 4) can then be calculated with the original image data (see box 34 in Figure 4) in such a way that projectable image data 36 are generated which, when projected onto the surface 20, generate reflected image data 38 which in at least one reflection direction essentially correspond to the original image data 34.

[0066] Furthermore, in the illustrated embodiment, the spatial depth information can be used to determine whether an object is located in the beam path (exemplarily designated by reference numeral 40 in Figure 1) that at least partially interrupts the beam path 40. Such an object is exemplified in Figure 1 by an operator 42. It is also conceivable that an object obscuring the beam path 40 could be a medical instrument, a medical device such as an operating light, or another person. In the embodiment shown here, the depth cameras 28 and 30 thus also fulfill the function of a shadow detection unit 44.

[0067] Knowing the position of the object obscuring at least one beam path 40 and knowing the positions and directions of the projection units 12, 14, it is possible to determine which part of the image data to be projected cannot be projected onto the surface 20 by a respective projection unit 12, 14. In response, the projection units 12, 14 are controlled by an associated control unit 46 such that the predetermined part of the image data to be projected, which cannot be represented by one projection unit (projection unit 12 in Figure 1), is projected onto the surface 20 by the corresponding other projection unit (projection unit 14 in Figure 1).

[0068] As a result, even in the case where an object, in particular the viewer himself, moves into the beam path 40 of at least one of the projection units, a viewer of the image data projected onto the surface 20 receives a complete image 22 projected onto the surface 20.

[0069] In addition to simply displaying image data projected onto the surface 20 as information, it is also conceivable that the user is given the opportunity to interact with the projection device 10. For example, within a specific area of ​​the projected image data, and particularly while maintaining a minimum distance from the surface 20 to meet sterility requirements, a user can perform a predetermined gesture to control medical equipment without interacting with the equipment itself. For instance, the user can perform a linear movement with their hand or at least one finger within the predetermined area of ​​the projected image data and thereby change the brightness of an operating light and / or room lighting.Of course, it is also conceivable that a predetermined user gesture influences the projected or to-be-projected image data. For example, a gesture such as a swipe can trigger a change in the displayed information, or bringing together or moving apart at least two fingertips of the operator's hand can cause a reduction or enlargement of projected information or even the entire projection field 22.

[0070] Such gestures can be captured, for example, via a gesture recognition unit (not shown). However, it is also conceivable that the gesture recognition, or the function of the gesture recognition unit, is implemented by the depth information acquisition device, with reference to the projection device 10 according to Figure 1, the depth cameras 28, 30.

[0071] Alternatively or additionally, it is also conceivable that, instead of or in addition to a gesture control, a voice control is included in the projection device 10, which is designed to capture voice commands and output corresponding signals to the control unit 46.

[0072] As a result, the operating personnel, and in particular the operator 42, are given the opportunity by the projection device 10 according to the invention to focus their concentration completely on the operating field and to receive all information and / or control options relevant to the operation in this field.

Claims

Claims 1. Projection device (10) suitable for projecting image data onto an uneven surface (20), in particular a surgical drape (20), the device comprising: - a depth information acquisition device (26) comprising at least one acquisition unit (28, 30) and configured to acquire at least spatial depth information of the uneven surface (20) onto which the image data (22) are to be projected, - a plurality of projection units (12, 14) which are configured to emit light in such a way that image data (36) provided to a respective projection unit (12, 14) are projected onto the surface (20), - a shadow detection unit (44) which is configured to detect an object (42) located in the light beam (40) from at least one of the projection units (12, 14) to the surface (20) to be irradiated, which obscures the surface (20) to be irradiated in such a way that the light emitted by the respective projection unit (12, 14) incompletely reaches the surface (20) to be irradiated, and in this case to output corresponding shadow information, and - a control unit (46) which is configured to receive and / or contain depth information from the depth information acquisition device (26), position data of a spatial position and direction data of a light emission direction of a respective projection unit (12, 14) and original image data (34), wherein the control unit (46) is further configured to - to determine, based on the position data, the direction data and the shading information, which of the projection units (12, 14) is suitable to project a predetermined To project part of the image data (22) to be projected onto the surface (20) without shadowing, - to process the original image data (34) with the depth information of the surface (20), the position data and the direction data of a respective projection unit (12, 14) to produce modified, in particular distorted, image data (36), and - the predetermined part of the modified image data (36) is output to a correspondingly defined projection unit (12, 14) in order to be projected by it onto the surface (20), so that a reflection (38) of the modified image data (36) projected onto the uneven surface (20) is essentially the same as the original image data (34).

2. Projection device (10) according to claim 1, characterized in that the control unit (46) is configured to perform the determination of the suitable projection units (12, 14) for shadow-free projection, the calculation of the depth information of the surface (20), the position data and the direction data of a respective projection unit (12, 14) to modified image data (36) and the output of the predetermined part of the modified image data (36) to each projection unit (12, 14) at least 1 time per second, in particular at least 30 times per second, advantageously at least 90 times per second.

3. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) is configured to detect depth information based on stereoscopy and / or time-of-flight and / or frequency modulation, wherein the detection unit (28, 30) is configured in particular as an RGB camera or depth camera (28, 30) or as an IR camera or as a LiDAR sensor or as a pattern transmit / receive unit.

4. Projection device (10) according to one of the preceding claims, characterized in that the depth information acquisition device (26) comprises a plurality of acquisition units (28, 30) which are configured to acquire depth information of the uneven surface (20) from different perspectives on the uneven surface (20).

5. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) is arranged in a non-movable manner in relation to at least one of the projection units (12, 14), in particular to all projection units (12, 14).

6. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) and at least one of the projection units (12, 14), in particular all Projection units (12, 14) are arranged on or in a room wall, in particular a room ceiling.

7. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) is arranged to be displaceable in relation to at least one of the projection units (12, 14).

8. Projection device (10) according to the preceding claim, characterized in that the projection device (10) further comprises a calibration device which is configured to detect a change in the position and / or orientation of the at least one to acquire the acquisition unit (28, 30) of the depth information acquisition device (26) in relation to at least one of the projection units (12, 14) and to output correspondingly changed spatial coordinates of the acquisition unit (28, 30) and / or the projection unit (12, 14) or a correspondingly changed relative position and / or relative orientation to the control unit (46).

9. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) and / or at least one of the projection units (12, 14) are in a Operating light, in particular a light body of the operating light, are / is arranged.

10. Projection device (10) according to the preceding claim, characterized in that the operating light comprises an articulated arm to which the light body is attached, wherein at least one of the joints, in particular all joints, of the articulated arm is assigned a sensor unit which is configured to detect a movement of the respective joint and to output a corresponding signal to the control unit (46).

11. Projection device (10) according to one of the preceding claims, characterized in that the at least one detection unit (28, 30) of the depth information detection device (26) and / or a gesture detection unit is configured to detect gestures of a user and output corresponding signals to the control unit (46), and / or that a speech detection unit is configured to detect speech of a user and output corresponding signals to the control unit (46), and / or that a remote control unit is configured to control operation to detect a remote control device by a user and output corresponding signals to the control unit (46).

12. Projection device (10) according to the preceding claim, characterized in that the control unit (46) is configured to place the gestures of the user detected by the at least one detection unit (28, 30) of the depth information detection device (26) and / or the gesture detection unit in spatial relation to the image data projected onto the surface (20).

13. Projection device (10) according to one of the preceding claims, characterized in that the predetermined part of the modified image data, which is projected onto the surface (20) by a respective projection unit (12, 14), has a gradually decreasing brightness at an outer edge section thereof, in the course from the inside out.

14. Projection method for projecting image data onto an uneven surface (20), in particular a surgical drape, wherein the method comprises the following steps: - Acquisition of at least spatial depth information of the uneven surface (20) onto which the image data are to be projected, by a depth information acquisition device (26) which comprises at least one acquisition unit (28, 30), - Providing a plurality of projection units (12, 14) which emit light in such a way that image data provided to a respective projection unit (12, 14) is projected onto the surface (20), - Detection by a shadow detection unit (44) of an object located in the light beam (40) of at least one of the projection units (12, 14) to the surface (20) to be irradiated, which covers the surface (20) to be irradiated in such a way that the light emitted by the respective projection unit (12, 14) does not illuminate the surface (20). irradiated surface (20) incompletely reached, and, in this case, output of corresponding shading information, and - Providing a control unit (46) which receives depth information from the depth information acquisition device (26), position data of a spatial position and direction data of a light emission direction of a respective projection unit (12, 14) and original image data, the method further comprising the steps of: - Determining by the control unit (46) which of the projection units (12, 14) is suitable to project a predetermined part of the image data to be projected onto the surface (20) without shadowing, based on the position data, the direction data and the shadowing information, - Processing the original image data with the depth information of the surface (20), the position data and the direction data of a respective projection unit (12, 14) to create modified, in particular distorted, image data, and - Outputting the predetermined part of the modified image data to a correspondingly defined projection unit (12, 14) to be projected by it onto the surface (20), so that a reflection of the modified image data projected onto the uneven surface (20) essentially corresponds to the original image data.

15. Projection method according to the preceding claim, characterized in that the steps of determining the suitable projection units (12, 14) for shadow-free projection, calculating the depth information of the surface (20), the position data and the direction data of a respective projection unit (12, 14) to modified image data and outputting the predetermined part of the modified image data to a correspondingly determined projection unit (12, 14) at least 1 time per second, in particular at least 30 times per second, are advantageously at least 90 times per second.

Images