System for representing objects in a virtual three-dimensional image space

The system enhances three-dimensional image display by using multiple projectors and cameras to track viewer movements, ensuring accurate stereoscopic representation and interaction, providing a fully immersive experience.

DE102016102868B4Active Publication Date: 2024-12-05DREWES ADRIAN
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
DE102016102868
Authority / Receiving Office
DE · DE
Patent Type
Patents
Current Assignee / Owner
Filing Date
2016-02-18
Publication Date
2024-12-05
Estimated Expiration
2036-02-18

AI Technical Summary

Technical Problem

Existing three-dimensional image display systems fail to provide a fully immersive experience for viewers, as they restrict viewing angles and positions, lack interaction capabilities, and often suffer from inaccuracies in tracking viewer movements and shadows.

Method used

A system incorporating a projection device, synchronization device, tracking device, and data processing unit that uses multiple projectors, shutter glasses, and multiple cameras to track viewer positions and movements, ensuring stereoscopic representation from any angle, allowing interaction and minimizing shadows and double images.

Benefits of technology

Enables a fully immersive, interactive three-dimensional experience by accurately tracking viewer movements and perspectives, reducing shadows and double images, and allowing viewers with visual impairments to perceive spatial relationships correctly.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

System (17) for representing objects (18) in a virtual three-dimensional image space, comprising - a projection device (1) with at least one projection surface (16) and at least one projector (2) to form a projection space; - a synchronization device (4) for the synchronized provision of the objects to be displayed (18) to at least one viewer (15) using the synchronization device (4); - an input device (6) for the input of control commands by the at least one viewer (15); - a tracking device (3) for determining the position of objects arranged in the projection space by means of at least three cameras for capturing position tracking means (5, 7) arranged on the synchronization device (4), the input device (6) and / or the at least one viewer (15), and further comprising at least two cameras arranged on the synchronization device for determining the position of the eyes, such as the direction of gaze, the interpupillary distance and / or the adaptation of the eyes (10) of the at least one viewer (15); and - a data processing facility (19) - for the exchange of information and / or control commands between the projection device (1), the synchronization device (4), the tracking device (3) and / or the input device (6), and - for processing data relating to the objects (18) to be displayed; wherein the data processing device (19) includes means for calculating the objects (18) to be displayed, which, in order to ensure the stereoscopic display of objects (18) from each viewing angle, take into account the viewing angle according to a position of the eyes (10) of the at least one viewer (15) determined by the tracking device (3).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present invention relates to a system for displaying objects in a virtual three-dimensional image space, comprising: - a projection device with at least one projection surface and at least one projector to create a projection room; - a synchronization device, preferably shutter glasses, for the synchronized provision of the objects to be displayed to at least one viewer using the synchronization device; - an input device for the input of control commands by at least one viewer; - a tracking device for determining the position of objects arranged in the projection space, in particular the position of the head of the at least one viewer in the projection space, by means of at least three cameras for capturing position tracking means arranged on the synchronization device, the input device and / or the at least one viewer, and further comprising at least two cameras arranged on the synchronization device for determining the position of the eyes, such as the direction of gaze, the interpupillary distance and / or the adaptation of the eyes of the at least one viewer; and - a data processing facility - for the exchange of information and / or control commands between the projection device, the synchronization device, the tracking device and / or the input device, and - for processing data belonging to the objects to be displayed.

[0002] It is known from the prior art to use various stereo 3D methods for the static visual representation of three-dimensional data. However, with such conventional methods, the spatial impression is merely an effect, since the viewing angle, standpoint, and movement are predetermined by the imaging technology. A viewer always remains outside the scene of the visual representation and does not perceive any real spatial dimensions. Furthermore, there is no way for the viewer to interact directly with the three-dimensional data or its visual representation in a perspectively correct manner.

[0003] For example, German patent DE 10 2007 057 208 A1 discloses a method for displaying image objects in a virtual three-dimensional image space, in which the viewer's gaze direction is detected and taken into account for the display of the image objects as well as for interactions with them. However, this method only uses an autostereoscopic display device, which, on the one hand, provides only a limited spatial sense for the viewer and, on the other hand, introduces further problems such as shadows cast by the viewer. Furthermore, only the viewer's eye movements are detected, not their head movements.

[0004] Furthermore, DE 101 34 430 A1 discloses a projection system for displaying virtual worlds and objects, which combines ground-level and omnidirectional projection with additional viewer tracking. However, the use of an optical infrared positioning system is considered a disadvantage, as it results in drawbacks in terms of accuracy and speed.

[0005] GB 2 451 461 A further discloses a camera-based tracking system in which a position marker is attached to the viewer's head to locate the viewer, or alternatively, the viewer's eyes are tracked using two cameras to determine their gaze direction. Furthermore, the system proposes the use of one or more input devices held by the viewer in their hand(s) for interaction with a virtual environment. A disadvantage here, too, is that only a stereoscopic display device is used, which does not allow for complete immersion of the viewer in a virtual world.

[0006] German patent DE 10 2007 028 654 A1 deals with solutions for the ergonomic representation of the spatial depth of objects in virtual environments, in which the view of at least one object is presented to at least one viewer by means of a stereoscopic projection system in the form of superimposed images for each eye of the viewer. DE 10 2007 028 654 A1 considers only the position of the viewer's eyes, which may be further influenced by the viewer's body movements.

[0007] US 2005 / 0096889A1 deals with solutions for moving a virtual link object in a virtual environment while avoiding collisions between the link object and the environment.

[0008] The article SUKTHANKAR, R. [et al.]: Dynamic shadow elimination for multi-projector displays. Computer Vision and Pattern Recognition, Vol. 2, 2001, pp. II-151 - II-157. IEEE Computer Society Conference on. IEEE, 2001. deals with solutions for the dynamic elimination of shadows for multi-projector displays.

[0009] The article GERBER, Dominique; BECHMANN, Dominique. Design and evaluation of the ring menu in virtual environments. Immersive projection technologies, 2004. deals with solutions for creating a so-called ring menu in virtual environments.

[0010] US patent 2004 / 0027394A1 discloses solutions for navigating a user in a projected virtual reality setup with a multitude of waypoint elements, each defined by its own waypoint position. The user can move automatically to corresponding waypoint positions via a waypoint element.

[0011] Against this background, the present invention aims to provide a system for displaying objects in a virtual three-dimensional image space, which further improves the feeling of immersion for a viewer and avoids the disadvantages known from the prior art.

[0012] The present invention proposes a system for representing objects in a virtual three-dimensional image space as a technical solution, comprising - a projection device with at least one projection surface and at least one projector to create a projection room; - a synchronization device, preferably shutter glasses, for the synchronized provision of the objects to be displayed to at least one viewer using the synchronization device; - an input device for the input of control commands by at least one viewer; - a tracking device for determining the position of objects arranged in the projection space, in particular the position of the head of the at least one viewer in the projection space, by means of at least three cameras for capturing position tracking means arranged on the synchronization device, the input device and / or the at least one viewer, and further comprising at least two cameras arranged on the synchronization device for determining the position of the eyes, such as the direction of gaze, the interpupillary distance and / or the adaptation of the eyes of the at least one viewer; and - a data processing facility - for the exchange of information and / or control commands between the projection device, the synchronization device, the tracking device and / or the input device, and - for processing data belonging to the objects to be displayed; where - the data processing device includes means for calculating the objects to be displayed, which, in order to ensure the stereoscopic display of objects from every viewing angle, take into account the viewing angle according to a position of the eyes of at least one viewer determined by the tracking device.

[0013] The invention makes particular use of the insight that by supplementing a conventional representation of three-dimensional data with a fourth dimension of time, a viewer gains the ability to perceive data presented as part of a virtual scene in the correct relation to themselves, since the virtual objects can be shown from different sides within the viewer's range of motion. Adding time as a fourth dimension is necessary because every person is constantly in motion, and the perception of distances and spaces is always influenced by the perception of different perspectives as well as by the layering of objects during a person's movement.A moving observer therefore has a different perception at one point in time than at a second point in time, and only through the combination of these different perceptions at different times is a spatial image or an understanding of it generated for the observer. In this way, even people with spatial visual impairment are able to gain a spatial impression.

[0014] The system according to the invention comprises a projection device with at least one projection surface and at least one projector to create a projection space. In principle, any imaging device can be used for this purpose, provided it is dimensioned such that the peripheral field of vision of a viewer is filled, as this allows for the overall impression necessary for immersion. Due to insufficient keystone correction in short-throw projectors known from the prior art, the invention provides for the use of custom-made prisms to compensate for horizontal distortion of the objects being displayed without the loss of pixels. When using projectors, at least one projector is used per projection surface, requiring separate, alternating output of individual images to provide a so-called shutter system.Alternatively, two projectors per projection surface can be used, with separate individual images being output alternately to each eye of a viewer by the projector designated for that eye. The use of a shutter system has a particular advantage over a polarization-filtered system in that, even when the head is turned, the stereo shift for the left and right eye of a viewer is always reproduced correctly, and there is no rotation of the eye as occurs when using polarization-filtered glasses with polarization-filtered systems.

[0015] To display an object to each viewer's left and right eyes, a synchronization device, preferably in the form of shutter glasses, is provided. This device ensures the synchronized presentation of the objects to be displayed to the viewer. This synchronization device allows the viewer to alternately see the object with their left and right eyes.

[0016] The invention further provides that, to create a projection space, several projection devices can be arranged side by side in any desired configuration, with the image content of the video projections seamlessly blending together at the image edges of the individual projection devices. The projection resolution for a projection device can be achieved to virtually unlimited levels by appropriately increasing the number of projectors.

[0017] According to the present invention, a tracking device is provided for capturing all objects located in the projection space, such as a viewer. This device determines the position of objects arranged in the projection space, in particular the position of the head of at least one viewer, by means of at least three cameras for capturing a position tracking device arranged on the viewer, and / or for determining the viewing direction, interpupillary distance, and / or eye adaptation of the viewer by means of at least two cameras. The tracking device preferably uses infrared cameras with integrated infrared LEDs arranged in a ring around a lens, enabling the detection of the position tracking device arranged on the viewer.The position tracking device is accordingly equipped with reflective markers that can be detected by the infrared cameras. By limiting the image information to the infrared light component, the infrared cameras only capture the infrared light emitted by the infrared camera illumination, which is reflected or emitted by the reflective markers.

[0018] According to the invention, an input device is further provided for the input of control commands by the at least one viewer, by means of which interaction with the objects to be displayed can be achieved.

[0019] Furthermore, a data processing unit is provided for the exchange of information and / or control commands between the projection unit, the synchronization unit, the tracking unit and / or the input unit, as well as for the processing of data relating to the objects to be displayed.

[0020] According to a preferred embodiment of the invention, the at least one projector of the projection device includes means for increasing the display frequency of the objects being projected, thereby increasing the number of viewers simultaneously present in the system. In this way, an image projection can be achieved for external viewers or viewers positioned next to the main viewer, enabling a geometrically correct stereoscopic representation. A first viewer sees the images that are displayed synchronously with their open shutter glasses or synchronization device and which are calculated based on their position. Correspondingly, another viewer sees the images calculated for their position, with a time delay, through their correspondingly synchronized open shutter glasses or synchronization device.

[0021] In another advantageous embodiment of the system according to the invention, the position tracking means of the tracking device for determining the position of the at least one viewer is arranged at the level of the viewer's head, in particular on the synchronization device. In this way, an adjustment of the position- and rotation-dependent stereoscopic image offset in the form of a variable deviation is possible, so that the stereo offset of the displayed image content required for spatial perception is represented in the same position of the viewer's eyes. This ensures a stereoscopic representation from every viewing angle and prevents any false image impression from arising due to changes in the position of the viewer's head.

[0022] In a further preferred embodiment of the system according to the invention, the at least three cameras of the tracking device for determining the position of the objects arranged in the projection space are designed as 3D object scanners. This makes it possible to capture all objects arranged in the projection space in real time and to use the data obtained in this way for perspective correction of the image content for the projection device, wherein an evaluation of the objects captured by means of the 3D object scanner and the calculation of the perspective correction take place in the data processing device.

[0023] Preferably, the data processing device includes means for avoiding diplopia of the objects being displayed, in particular by detecting the interpupillary distance and / or convergence of the eyes of the at least one viewer using the tracking device and adjusting the displayed objects accordingly. For this purpose, the gaze direction, interpupillary distance, and / or eye adaptation of the at least one viewer are detected by means of the at least two cameras of the tracking device, which are preferably arranged on the synchronization device designed as shutter glasses. This allows the convergence angle data of the eyes to be detected and the focal point of the eyes of the at least one viewer to be determined.In this way, the data processing device can be used to adjust the objects being viewed, so that the resulting display prevents double images of objects, which occur particularly in the viewer's immediate vicinity and cause discomfort-inducing irritations.

[0024] In a further preferred embodiment of the system according to the invention, the data processing device includes means for reducing so-called "visually indicated motion sickness," wherein, in the case of moving objects or image content, a primary image content in the peripheral field of vision of at least one viewer is replaced by a neutral, stationary structure, thereby restricting moving objects or image content to a small area of ​​the perceptual center of the at least one viewer. For this purpose, the at least two cameras of the tracking device detect whether the pupils of the at least one viewer are dilated. If this is the case, the image content in the peripheral field of vision of the at least one viewer is replaced during a phase of movement of the object or image content being displayed.Accordingly, at least one viewer sees a neutral, stationary structure in their peripheral field of vision, appearing on the projection plane. This establishes a fixed reference point for at least one viewer to real space and significantly reduces the perceptual discrepancy. If the viewer's movement ceases, the image content reappears in their peripheral field of vision. Manual activation of this function is particularly desirable.

[0025] In a preferred embodiment of the system according to the invention, the input device further provides that it includes a preferably ring-shaped virtual menu for transmitting control commands via the hand of at least one user. This avoids a classic, two-dimensional interaction – as known from computers. In the system according to the invention, the user is the only spatial constant in relation to themselves and always acts within a spatial environment. Therefore, it is advantageous to locate the interface with which the user calls up commands and functions on the user themselves. The interface is displayed on the input device, which the at least one user holds in their hand, and includes all the functions necessary for operating the system according to the invention, which are arranged in a ring around the hand of the at least one user by means of symbols in virtual space.The individual functions can be called up via one or more function keys on the input device, whereby it is preferably provided that this interface is structured in different levels and that sub-functions can also be accessed by extension, which are not visible on the first, ring-shaped level.

[0026] Furthermore, according to the invention, the input device includes means for inputting and evaluating voice commands. For this purpose, the input device includes a microphone for receiving voice commands from at least one viewer, which are evaluated by the input device and transmitted to the data processing device for execution. To provide acoustic feedback to the at least one viewer, the input device may include means for outputting audio signals to confirm received voice commands.

[0027] In another advantageous embodiment of the system according to the invention, the data processing device includes means for object-based file representation and management, wherein objects are arranged in spatially distinct structures in front of the at least one viewer in virtual three-dimensional space. The at least one viewer can move around these structures and initiate interaction by touching one or more objects in virtual three-dimensional space using the input device or by means of gesture recognition. Access to subordinate structures, which can be reached from superior structures, is indicated by specially marked objects, so that these are visible to the at least one viewer in their external environment and thus also directly accessible.Advantageously, it is provided that objects can be marked as always visible, so that an object remains visible even when the at least one viewer is located many levels deeper within the subordinate structures of an object. Preferably, it is further provided that, to improve orientation and clarity, chronological sorting as well as sorting of a large number of objects in a two-dimensionally arranged plane is performed in front of the at least one viewer, who can freely position the objects arranged in this plane in space.

[0028] In a further preferred embodiment of the system according to the invention, the data processing device includes means for collision detection between the at least one viewer, who is located in the virtual three-dimensional image space, and objects also located in the virtual three-dimensional image space. In the system according to the invention, the at least one viewer has the ability to move freely towards virtual objects, which makes it possible for them to pass through objects freely arranged in space, such as the walls of a building represented in the virtual three-dimensional space, since there is no physical barrier. However, such passage may not be desirable.The collision detection system according to the invention causes the entire scene displayed in the virtual three-dimensional space to move away from the at least one viewer as soon as a collision occurs between the at least one viewer and an object designated as impenetrable in the virtual three-dimensional space. The actual position of the at least one viewer is determined by means of the tracking device, and the position data is subsequently transmitted to the data processing device. The collision detection system according to the invention operates in all spatial directions, including downwards, so that it is possible for the at least one viewer to ascend stairs or ramps in the virtual three-dimensional space.Particularly preferred is the provision that a warning structure is displayed on the real projection surfaces by the projection device, since the system according to the invention knows both the exact location of the real projection surfaces and the position of the at least one viewer.

[0029] In a particularly preferred embodiment of the system according to the invention, the data processing device includes a means for identifying the position of the at least one viewer within the virtual three-dimensional image space, preferably in the form of a structure surrounding the viewer. In addition to the possibility for the at least one viewer to move within the virtual scene through real physical movements, there is also the possibility of navigating freely through the virtual three-dimensional image space using the input device as a control instrument. In scenes with a small spatial extent or near ground level, the viewer therefore has no general difficulty understanding their spatial position, since – as in the real world – they have a direct connection to their environment.When at least one viewer moves, their perspective on the surrounding objects changes significantly. However, if the viewer moves within a large structure, such as a stadium bowl, they lose this sense of reference, as the changes in perspective are barely perceptible within the viewer's physical range of motion. This leads to confusion, particularly regarding the size, distance, proportions, and speed of objects in the virtual three-dimensional space. Such confusion is eliminated by providing, according to the invention, a means of identifying the viewer's position within the virtual three-dimensional image space, especially by means of a virtual vehicle that represents a structure surrounding the viewer with a base and side boundaries.The virtual vehicle can be controlled by input via the input device and moved through the virtual three-dimensional space, with the at least one viewer also being able to move physically within this vehicle up to its respective spatial boundary. The collision detection according to the invention is used again at the side boundaries of the virtual vehicle and prevents the at least one viewer from leaving the vehicle in the virtual three-dimensional space.

[0030] In a particularly advantageous embodiment of the system according to the invention, the data processing device further provides that it includes means for navigation from a definable starting point to a definable target point within the virtual three-dimensional image space. For this purpose, a target point located in the virtual three-dimensional space is marked using the input device, for example, by pointing the input device at a distant object. At the intersection of this pointing vector and the marked object, a target marker is then generated, which can be defined as the target point by input via the input device. Subsequently, the system calculates the ideal path to this target point and moves the at least one viewer accordingly within the virtual three-dimensional space.With reference to the collision detection according to the invention, it can be provided that either objects that cannot be penetrated are avoided, or that objects arranged in the virtual three-dimensional space may be penetrated in order to enable the fastest possible navigation.

[0031] In a particularly preferred embodiment of the system according to the invention, navigation to the target point in the virtual three-dimensional image space is provided such that the target point can be defined on a virtual site plan or overview map via the input device. The at least one viewer has the option of activating a virtual screen within the scene, which displays overview representations such as site plans, floor plans, sections, elevations, isometrics, and the like, which can be accessed as needed. Using the input device, the at least one viewer of the system according to the invention can, for example, directly mark a desired target point position in a site plan.After appropriate input via the input device, this marked target point is fixed, so that the calculation according to the invention of the ideal path from the current user position to the target point in the scene and the movement to the target point can then be carried out.

[0032] In another preferred embodiment of the system according to the invention, navigation to the target point in the virtual three-dimensional image space is provided, wherein the target point can be defined by means of an external input device and the external input device can be connected to the data processing unit for data transmission. This type of extended navigation within the virtual three-dimensional space is implemented using, for example, touchscreen input devices such as a tablet PC or a smartphone. An overview of the scene is displayed on the screen of the touchscreen input device, so that by touching the touchscreen input device, at least one user can mark a desired target point position, for example in the overview, and then the ideal path from the current user position to this position can be calculated.

[0033] In a further advantageous embodiment of the system according to the invention, the data processing device includes means for detecting a shadow cast by the at least one viewer by the at least one projector of the projection device, wherein the position of the at least one viewer in relation to the projection device can be determined by means of the tracking device. For this purpose, the position of the at least one viewer is first determined by means of the tracking device by detecting the position tracking means arranged on the at least one viewer.Based on this positional information about the at least one viewer and the knowledge of the system according to the invention about the position of the at least one projector of the projection device, the data processing device is able to adjust the representation of objects in the virtual three-dimensional image space in such a way that a shadow cast by the at least one viewer can be avoided. This is made possible in particular by the fact that, according to the invention, at least two projectors of the projection device are arranged in the projection space such that a viewer located in the projection space can only cause a maximum of one shadow in front of one of the at least two projectors, so that this shadow can be eliminated by appropriately adjusting the image content to be output by the other projector.

[0034] Advantageously, it is further provided that any movement and / or rotation triggered by the input device occurs relative to the viewer's actual position. For this purpose, the tracking system continuously determines the position of at least one viewer in real time, ensuring that the rotation axis of the virtual projection space is always centered on the viewer. If the virtual projection space is rotated within the scene by activating the input device, this rotation is correctly displayed to the viewer as a rotation around their own axis. The position of the input device, which might be held in the viewer's hand, is irrelevant in relation to the viewer; only the viewer's own orientation matters.In this way, unwanted lateral movements caused by operating the input device can be avoided, since the rotation device of the virtual projection space is always located with the at least one viewer and not at a position in the projection space or at the input device.

[0035] Furthermore, a preferred embodiment of the present invention provides for movement, rotation, and / or interaction with respect to the real-world position of a second viewer. The aforementioned representation of the objects or image content to be displayed in the virtual three-dimensional space relates to a primary viewer's position in the projection space, ensuring correct perspective distortion of the image information for this primary viewer. However, this becomes problematic if another viewer is to handle navigation and interaction with scene content. Particularly when two viewers are widely separated, interaction with scene content is difficult for the second viewer because the projection space's pivot point is located at the primary viewer's location, potentially leading to distortions for the second viewer.According to the invention, a second virtual representative of the projection space can be activated by positioning the dynamic rotation axis of the virtual projection space at the second viewer's location in real time. This rotation axis can, if necessary, replace the primary rotation axis at the main viewer's location, so that selected image content is displayed with correct perspective for the second viewer and allows them to interact with it directly.

[0036] Further details, features and advantages of the invention are explained in more detail below with reference to the exemplary embodiments shown in the figures of the drawings.

[0037] This shows: Fig. 1 in a perspective view an embodiment of a projection space according to the invention; Fig. 2 in a top view an embodiment of a synchronization device according to the invention with position tracking means arranged thereon; Fig. 3 in a top view an embodiment of an input device according to the invention with position tracking means arranged thereon; Fig. 4 in a schematic representation an embodiment for determining the direction of view by the tracking device according to the invention; Fig. 5 in a schematic representation an embodiment of the field of vision restriction according to the invention for reducing “Visually Indicated Motion Sickness”; Fig. 6 in a schematic representation an embodiment of a menu provided according to the invention of the input device according to the invention; Fig. 7 in a schematic side view a means according to the invention for identifying the position of the at least one viewer within the virtual three-dimensional image space; and Fig. Figure 8 shows a schematic representation of an embodiment for the detection of a shadow and its corresponding elimination according to the invention.

[0038] In Fig. Figure 1 shows a projection device 1 according to the invention, which comprises three projectors 2 and three cameras of a tracking device 3 for detecting a position tracking means 5, 7. The projection device 1 has two projection surfaces onto which the objects 18 to be displayed or image content are projected. Fig. One of the projectors 2 is configured to perform a floor projection, so that the floor of the projection room serves as a further, third projection surface. Each of the three projectors 2 is a projector 2 suitable for displaying 3D content to a viewer in conjunction with a synchronization device according to the invention. The at least one viewer of the system 17 according to the invention can also be used as a further, additional projection surface.

[0039] Fig. Figure 2 shows an embodiment of a synchronization device 4 according to the invention, which is designed here as shutter glasses. This synchronization device 4 comprises two controllable lenses made of liquid crystal surfaces, which can be switched electronically to be transparent and opaque to enable stereoscopic vision. Furthermore, the synchronization device 4 has a total of six position tracking means 5, which are used by the tracking device 3 according to the invention to determine the position of objects arranged in the projection space.The position tracking means 5 attached to the synchronization device 4 enable the tracking device 3 according to the invention, comprising at least three cameras, to detect the position tracking means 5 arranged on the shutter glasses of the at least one viewer and thereby track the position of the head of the at least one viewer 15. The synchronization device 4 further comprises at least two cameras, which are preferably integrated into the frame of the synchronization device 4, for determining the direction of gaze, the interpupillary distance and / or the adaptation of the eyes of the at least one viewer, which are in . Fig. 2, however, are not shown.

[0040] In Fig. Figure 3 shows an embodiment of an input device 6 according to the invention. This input device 6 also comprises several position tracking means 7, which can be used by the tracking device 3 to determine the position of the input device 6 arranged in the projection space. Furthermore, the input device 6 comprises a control pad 8a and an analog stick 8b for inputting control commands by the at least one viewer 15 of the system 17 according to the invention. Furthermore, means 8c for inputting voice commands are also provided on the input device 6 according to the invention, in particular in the form of a microphone, thereby enabling voice control of the system 17 according to the invention for the at least one viewer 15. For acoustic confirmation of input control commands and / or voice commands, a means 9 for outputting acoustic confirmations is also provided on the input device 6 according to the invention.According to the invention, the position tracking means 7 arranged on the input device 6 make it possible for a viewer 15 using the input device 6 to control the system 17 according to the invention by performing movements with the input device 6.

[0041] In Fig. Figure 4 illustrates the functioning of the inventive detection of a focus point 11 by the eyes 10 of the viewer 15. The gaze direction of the eyes of at least one viewer 15 is detected by means of the at least two cameras provided on the synchronization device 4. A focus point 11 can be determined based on the pupils of the eyes 10 of the viewer 15, since, as can be seen here, when a focus point 11 is located further away from the viewer 15, the gaze direction of each pupil is essentially parallel to the gaze direction of the other pupil. However, if a focus point 11 moves closer to a viewer 15, the gaze direction of the viewer's eyes 10 is significantly altered.By means of the tracking of the focus point 11 achieved in this way, it is possible according to the invention to detect the viewing direction of the viewer 15 and to prevent the occurrence of diplopia or double images by means of a corresponding adjustment of the objects 18 to be displayed or image content by the data processing device 19.

[0042] In Fig. Figure 5 provides an example of field-of-view restriction for reducing the "Visual Indicated Motion Sickness" of a viewer 15. For this purpose, the system detects whether the objects 18 or image content being displayed involve rapid movements, so that the displayed object 18 or image content can then be adjusted accordingly. Such an adjustment is described in Fig. 5 recognizable where an area 12 outside the perceptual center of the viewer 15 is replaced by a neutral, non-moving structure during rapid movements, so that irritations in the peripheral visual field of a viewer 15 can be avoided.

[0043] In Fig. Figure 6 shows an embodiment of a toolwheel according to the invention. This includes a ring-shaped, virtual menu 13, which comprises various menu items that can be selected by the viewer 15 using input means of the input device 6. The ring-shaped, virtual menu 13 is positioned on the viewer 15 itself, or rather on its hand, since the position of the input device 6 in the projection space can always be determined by the position tracking means 7 of the input device 6. Thus, according to the invention, a representation of the ring-shaped, virtual menu 13 is calculated by means of the data processing device 19 in spatial proximity to the input device 6 of the viewer 15, and a corresponding representation is carried out by the projectors 2 in the projection space.

[0044] Fig. Figure 7 shows a schematic representation of a means 14 according to the invention for identifying the position of the viewer 15, which consists of a virtual platform arranged below the viewer 15 and a virtual railing structure arranged around the viewer 15. The means 14 for identifying the position of the viewer 15 itself is not visible, but merely serves to facilitate the movement of the viewer 15 in the virtual space. The possible directions of movement of the means 14 for identifying the position of the viewer 15 are shown according to Fig. 7 is given by means of the arrows shown, whereby movement in any direction of a plane as well as vertically is possible. The transmission of commands for controlling the means 14 for identifying the position of the observer 15, which can also be referred to as a virtual vehicle, is carried out by the observer 15 by means of the input device 6 or the corresponding devices arranged on it in the form of a control pad 8a, an analog stick 8b and means 8c for inputting voice commands.

[0045] In Fig. Figure 8 illustrates the function of shadow elimination in an exemplary embodiment. Here, a viewer 15 is shown in front of a projection surface 16, which is filled with image content by only one projector 2. Due to the use of only one projector 2, it is unavoidable that a shadow cast by the viewer 15, who is located in the projection space, will be visible on the projection surface 16. In contrast, with the use of two projectors 2 according to the invention, the shadow cast by the viewer 15 can be eliminated. This is due to the cameras arranged in the projection space for capturing position tracking means, which are located in Fig.The position of the viewer 15 can be precisely determined using the data processing unit 19, which may be configured as 3D object scanners (not shown in Figure 8). With this information about the viewer's exact position, the data processing unit 19 can calculate the image content or objects 18 to be displayed, so that the image content can be adjusted for each of the two projectors 2. This prevents the viewer 15 from casting a shadow on the projection surface 16.

[0046] The embodiments shown in the figures of the drawings and the embodiments explained in connection with them serve only to illustrate the invention and are not limiting to it. Reference symbol list: 1 Projection device 2 projectors 3. Tracking device with at least three cameras 4 Synchronization device 5 Position tracking devices on the synchronization device 6 Input device 7 Position tracking devices at the input device 8a Control pad for entering control commands 8b Analog stick for inputting control commands 8c Means of inputting voice commands 9 Means of providing acoustic confirmation of entered commands 10. Eye of the beholder 11 Focus point 12 Area outside the viewer's perception center 13 ring-shaped, virtual menus in the form of a toolwheel 14 means of identifying the viewer's position 15 viewers 16 projection surfaces 17 System for representing objects in a virtual three-dimensional image space 18. Object or image content to be depicted 19 Data processing equipment

Claims

[1] System (17) for displaying objects (18) in a virtual three-dimensional image space, comprising - a projection device (1) with at least one projection surface (16) and at least one projector (2) for forming a projection space; - a synchronization device (4) for the synchronized provision of the objects (18) to be displayed for at least one viewer (15) using the synchronization device (4); - an input device (6) for inputting control commands by the at least one viewer (15); - a tracking device (3) for determining the position of objects arranged in the projection space by means of at least three cameras for detecting position tracking means (5, 7) arranged on the synchronization device (4), the input device (6) and / or the at least one viewer (15), and further comprising at least two cameras arranged on the synchronization device for determining the position of the eyes, such as the viewing direction, the interocular distance and / or the adaptation of the eyes (10) of the at least one viewer (15); and - a data processing device (19) - for exchanging information and / or control commands between the projection device (1), the synchronization device (4), the tracking device (3) and / or the input device (6), and - for processing data associated with the objects (18) to be displayed; wherein the data processing device (19) has means for calculating the objects (18) to be displayed, which means, in order to ensure the stereoscopic display of objects (18) from any viewing angle, take into account the viewing angle corresponding to a position of the eyes (10) of the at least one viewer (15) determined by the tracking device (3). [2] System (17) according to claim 1, characterized by that the at least one projector (2) of the projection device (1) has means for increasing the reproduction frequency of the objects (18) to be displayed in order to increase the number of viewers (15) provided simultaneously in the system (17). [3] System (17) according to claim 1 or claim 2, characterized bythat the position tracking means (5, 7) of the tracking device (3) for determining the position of the at least one observer (15) by the tracking device (3) is arranged at the level of the head of the at least one observer (15), in particular on the synchronization device (4). [4] System (17) according to one of claims 1 to 3, characterized by that the at least three cameras of the tracking device (3) for determining the position of objects (18) arranged in the projection space are designed as 3D object scanners. [5] System (17) according to one of claims 1 to 4, characterized bythat the data processing device (19) has means for avoiding diplopia of objects (18) to be displayed, in particular by detecting the eye distance and / or convergence of the eyes (10) of the at least one observer (15) by means of the tracking device (3) and a corresponding adaptation of the objects (18) to be displayed by the data processing device (19). [6] System (17) according to one of claims 1 to 5, characterized by in that the data processing device (19) has means for detecting the spatial position of the focal point (11) of the eyes (10) of the at least one observer (15), whereby a blurred representation of objects and / or image contents (18) is enabled depending on the position of the at least one observer (15). [7] System (17) according to one of claims 1 to 6, characterized bythat the data processing device (19) has means for reducing the so-called "Visual Indicated Motion Sickness", wherein in the case of moving objects and / or image contents (18) a primary image content in the peripheral field of view of the at least one observer (15) is replaced by a neutral, non-moving structure, whereby moving objects and / or image contents (18) are limited only to a small area of ​​the perception center of the at least one observer (12). [8] System (17) according to one of claims 1 to 7, characterized by that the input device (6) has a preferably ring-shaped virtual menu (13) for transmitting control commands (8a, 8b, 8c) by a hand of the at least one viewer (15). [9] System (17) according to one of claims 1 to 8, characterized bythat the input device (6) has means for inputting voice commands (8c) and for evaluating input voice commands (8a, 8b, 8c). [10] System (17) according to one of claims 1 to 9, characterized by that the data processing device (19) has means for the object-like representation and management of files, wherein a representation is provided in the virtual three-dimensional image space (17). [11] System (17) according to claim 10, characterized by that the object-like representation and management of files provides access to subordinate menu structures via specially marked objects (18) provided in the virtual three-dimensional image space (17). [12] System (17) according to one of claims 1 to 11, characterized bythat the data processing device (19) has means for collision detection of the at least one observer (15), who is located in the virtual three-dimensional image space (17), with objects (18) also located in the virtual three-dimensional image space (17). [13] System (17) according to one of claims 1 to 12, characterized by that the data processing device (19) has a means arranged in the virtual three-dimensional image space (17) for identifying the position of the at least one viewer (14) within the virtual three-dimensional image space (17), preferably in the form of a structure surrounding the at least one viewer (15). [14] System (17) according to one of claims 1 to 13, characterized by that the data processing device (19) has means for detecting collisions between the at least one viewer (14) and objects (18) located in the virtual three-dimensional image space (17). [15] System (17) according to one of claims 1 to 14, characterized by that the data processing device (19) has means for navigation from a definable starting point to a definable destination point within the virtual three-dimensional image space (17). [16] System (17) according to claim 15, characterized by that the navigation to the target point is provided in the virtual three-dimensional image space (17), which is defined by the intersection point of a pointing vector generated by means of the input device and of an image object arranged in the virtual three-dimensional image space (17). [17] System (17) according to claim 15, characterized by that the navigation to the destination point is provided in the virtual three-dimensional image space (17), which can be defined via the input device on a virtual site and / or overview plan. [18] System (17) according to claim 15, characterized bythat the navigation to the target point is provided in the virtual three-dimensional image space (17), which can be defined on an external input device, wherein the external input device can be connected to the data processing device (19) for the purpose of data transmission. [19] System (17) according to one of claims 1 to 18, characterized by that the data processing device (19) has means for detecting a shadow of the at least one observer (15) generated by the at least one projector (2) of the projection device (1), wherein the position of the at least one observer (15) in relation to the projection device (1) can be determined by means of the tracking device (3).

Citation Information

Patent Citations

  • Immersive stereoscopic projection system for use in virtual reality with software corrected projection onto a concave projection screen and floor and masking of non-visible regions

    DE10134430A1

  • Method for ergonomic representation of spatial depth of objects in virtual environments, involves presenting view of object to observer in form of overlapping images for each eye of observer by stereoscopic projection system

    DE102007028654A1

  • Method for displaying image objects in a virtual three-dimensional image space

    DE102007057208A1

  • Camera based 3D user and wand tracking human-computer interaction system

    GB2451461A

  • Virtual reality method and apparatus with improved navigation

    US20040027394A1