SYSTEM AND METHOD FOR ILLUMINATING IMAGES FROM A LIGHT FIELD DISPLAY BASED ON POSITION TRACKING DATA - Patent application
The light field display system addresses the lack of immersive 3D effects in amusement parks by tracking and calculating light field vector functions for multiple 3D images, enhancing visitor engagement and thematic immersion.
Patent Information
- Application Number
- JP2024523644
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-10-19
- Filing Date
- 2022-10-17
- Publication Date
- 2024-11-19
Smart Images

Figure 2024542957000001_ABST
Abstract
Description
[Background technology]
[0001] This disclosure relates generally to amusement park special effects, and more specifically to the use of light field displays to create amusement park special effects.
[0002] In an amusement park environment, it may be desirable to provide special effects (e.g., visual effects) that create feelings of excitement, fear, and curiosity in visitors. Specifically, for example, it may be desirable to create an illusion that is the opposite of what people traditionally experience in everyday life. Such illusions can be used to create excitement and can be used to provide an immersive experience in a themed environment. Techniques can be used to provide such special effects. Thus, it is now recognized that improved special effects techniques and technologies, as described herein, are desirable with respect to improving the entertainment experience.
[0003] This section is intended to introduce the reader to various technical aspects that may be related to various aspects of the present technology, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. As such, it should be understood that these discussions are to be read in this light, and not as admissions of prior art. Summary of the Invention
[0004] Certain embodiments consistent with the scope of the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of the claimed subject matter, but rather these embodiments are merely intended to provide a brief summary of possible forms of the subject matter. Indeed, the subject matter may encompass a variety of forms that may be similar to or different from the embodiments set forth below.
[0005] In one embodiment, a light field display system includes a light field display configured to display a first glasses-free 3D (autostereoscopic) image and to project light rays towards one or more display devices for displaying a second glasses-free 3D image via the one or more display devices. The light field display system also includes a control system configured to track positions, orientations, and / or movements of the one or more display devices and / or one or more human parts associated with the one or more display devices in a physical environment, to calculate respective light field vector functions for light rays projected by the light field display towards the one or more display devices based at least in part on the tracked positions, orientations, and / or movements of the one or more display devices and / or one or more human parts associated with the one or more display devices, and to cause the light field displays to project light rays towards the one or more display devices according to the respective calculated light field display vector functions for displaying the second glasses-free 3D image via the one or more display devices.
[0006] In one embodiment, a method includes displaying a first glasses-free 3D image via a light field display. The method also includes tracking, via a control system, a position, orientation, and / or movement of one or more display devices and / or one or more human parts associated with the one or more display devices in a physical environment. The method further includes calculating, via the control system, respective light field vector functions for light rays cast by the light field display towards the one or more display devices based at least in part on the tracked position, orientation, and / or movement of the one or more display devices and / or one or more human parts associated with the one or more display devices. Additionally, the method includes causing the light field display to cast light rays towards the one or more display devices via the control system according to the respective calculated light field vector functions to display a second glasses-free 3D image via the one or more display devices.
[0007] In one embodiment, a light field display system includes one or more light field displays configured to project light rays toward one or more display devices to display a first glasses-free 3D image and to display a second glasses-free 3D image via the one or more display devices, and also includes a control system configured to track a position, orientation, and / or movement of the one or more display devices and / or one or more human parts associated with the one or more display devices in a physical environment and to cause the one or more light field displays to project light rays toward the one or more display devices according to a light field vector function to display a second glasses-free 3D image via the one or more display devices, where the light field vector function is calculated based at least in part on the tracked position, orientation, and / or movement of the one or more display devices and / or one or more human parts associated with the one or more display devices.
[0008] Various refinements of the features set forth above may be made in relation to the various aspects of the present disclosure. Additional features may also be incorporated in these various aspects as well. These refinements and additional features may exist individually or in any combination.
[0009] These and other features, aspects and advantages of the present disclosure will be better understood when the following detailed description is read with reference to the drawings, in which like characters represent parts throughout. [Brief description of the drawings]
[0010] [Figure 1] FIG. 1 is a schematic diagram of a light field display configured to generate glasses-free 3D images showing display objects that appear in three dimensions, according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 shows a reflective light field display system in FIG. 1 in which the light field display of FIG. 1 not only generates a glasses-less 3D image on the light field display, but also simultaneously emits light rays that, when reflected by one or more reflecting devices, generate a glasses-less 3D image on the reflecting device(s), in accordance with an embodiment of the present disclosure. [Diagram 3] FIG. 3 illustrates a reflective light field display system in which the light field display of FIG. 1 projects light onto a plurality of respective reflective devices held by a plurality of respective amusement park attendees, according to an embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates a light field display system in which the light field display of FIG. 1 not only generates a glasses-free 3D image on the light field display, but also substantially simultaneously emits light rays that, when viewed through one or more cameras, display the glasses-free 3D image through the cameras, in accordance with an embodiment of the present disclosure. [Diagram 5]FIG. 5 is a flow chart of a method for operating the light field display system of FIGS. 2-4 according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram of the effect produced by a light field display system according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram of another effect produced by a light field display system according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a schematic diagram of another effect produced by a light field display system according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a schematic diagram of another effect produced by a light field display system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] One or more specific embodiments of the present disclosure will be described below. To provide a concise description of these embodiments, not all features of an actual implementation may be described in the specification. It should be understood that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's particular objectives, such as adhering to system-related and business-related constraints that may vary from implementation to implementation. Moreover, it should be understood that such a development effort may be complex and time-consuming, but would nevertheless be a routine undertaking for those skilled in the art having the benefit of this disclosure to design, fabricate, and manufacture. Furthermore, to the extent that certain terms, such as parallel, perpendicular, etc., are used herein, it should be understood that these terms permit some deviation from the strict mathematical definition, for example, to permit deviations associated with manufacturing imperfections and tolerances.
[0012] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean that there are one or more elements. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. In addition, it should be understood that references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features.
[0013] The present disclosure generally relates to systems and methods for projecting images from a light field display based at least in part on position tracking data (e.g., position, direction, and / or movement) of a display device toward which light projected from the light field display is directed and / or of a person associated with the display device (e.g., holding the display device). This technique can be used to create special effects where an image seen in a mirror is not seen in the physical environment. For example, this effect can make amusement park visitors feel as if they are using a magical device. If amusement park visitors could hold a mirror in their hands rather than holding the mirror fixed in a fixed position, the magical nature of this effect would be greatly increased. For this effect to be maximally successful, it must appear correct to many amusement park visitors at the same time.
[0014] As used herein, the terms "automatic" and "automatically" refer to actions being performed by a computing device or computer system (e.g., one or more computing devices) without human intervention. For example, an automatically performed function may be performed by a computing device or computer system based solely on data stored and / or received by the computing device or computer system, despite the fact that no human user has prompted the computing device or computer system to perform such function. However, as a non-limiting example, a computing device or computer system may make decisions and / or initiate other functions based solely on a decision made by the computing device or computer system, regardless of any other inputs related to that decision.
[0015] As used herein, the terms "real time" and "substantially real time" refer to operations that are performed substantially simultaneously with one another without any humanly perceptible delay during execution. For example, two functions performed in "substantially real time" occur within a few seconds (or even within milliseconds) of each other. However, as one non-limiting example, two functions performed in substantially real time may occur within 1 second, within 0.1 seconds, within 0.01 seconds, etc. of each other.
[0016] Embodiments of the present disclosure enable systems and methods for projecting an image from a light field display based at least in part on tracked data (e.g., position, orientation, and / or movement) of a display device toward which light is projected from the light field display and / or a human associated with (e.g., holding) the display device. Specifically, a light field display described herein is configured to project light toward a display device (e.g., either a reflector or a camera) to display a first glasses-free 3D image and simultaneously (e.g., while displaying its first glasses-free 3D image in real time) display a second glasses-free 3D image via the display device. As described in more detail herein, the light projected from the light field display is controlled by a control system based at least in part on tracking data (e.g., position, orientation, and / or movement) of the display device and / or a part of the human (e.g., a part of the human itself or a part of a particular human, such as a human eye for eye tracking, a human head for head tracking, etc.), which position tracking data can be detected via a sensor of the display device and / or a camera positioned around the physical environment in which the display device and the human are located. Specifically, the control system calculates a light field vector function for light rays to be projected toward each individual display device based at least in part on that particular display device and / or its associated human position tracking data.
[0017] Additionally, although this specification primarily describes projecting light rays toward a display device to simultaneously display a first glasses-less 3D image and a second glasses-less 3D image via the display device, in other embodiments, the light field display may instead project light rays toward a display device to display a first non-glasses-less 3D image and / or a second non-glasses-less 3D image via the display device. For example, in such embodiments, the light field display may still be capable of displaying different images in different directions than the display device, but the angular density of those images would be lower than that required for glasses-less 3D images, as described in more detail herein.
[0018] 1 is a schematic diagram of a light field display 10 configured to generate glasses-free 3D images that display objects viewed in three dimensions. However, while primarily described herein as applicable to amusement park attractions, it will be appreciated that the light field display system described herein can be extended to other applications that would benefit from the generation of glasses-free 3D images. Additionally, while primarily described herein as a single light field display 10, in other embodiments, multiple light field displays 10 can be used in combination with one another to perform the illumination described herein.
[0019] As shown in FIG. 1, the light field display 10 can include a large number of pixels 12, where light emanating from each individual pixel 12 has an individual three-dimensional directionality. In particular, in some embodiments, each individual pixel 12 can be configured to display any number of images depending on the position of the eyes of an amusement park visitor viewing the glasses-less 3D image displayed by the light field display 10. For example, in some embodiments, each individual pixel 12 can be configured to display hundreds, thousands, or even more different combinations of light and color depending on the position of the eyes 14 of the amusement park visitor viewing the particular pixel 12. It will be understood that the number of pixels 12 displayed as part of the light field display 10 is merely exemplary, and that in actual implementations, the light field display 10 can include millions of pixels 12 or even more pixels 12.
[0020] Thus, in one embodiment, control system 16 can be configured to calculate a light field (e.g., a vector function) that determines how pixels 12 generate light rays to generate the glasses-less 3D image displayed by light field display 10. Because each individual pixel 12 generates light rays with individual three-dimensional directionality, the vector function calculated by control system 16 can include a five-dimensional space (e.g., the three-dimensional space of individual pixels 12 combined with the three-dimensional directionality) as well as the color and radiance of the light rays themselves.
[0021] In one embodiment, the control system 16 may include processing circuitry such as a processor 18 (e.g., a general-purpose processor or other processor) and a memory 20, where the processor 18 may execute computer-executable instructions stored in the memory 20, which, when executed by the processor 18, cause the processor 18 to calculate a light field vector function and then send control signals to the individual pixels 12 of the light field display 10 to create the light rays that form the light field displayed by the light field display 10. The computer-executable instructions for performing these functions may be encoded in a program or code stored in a tangible, non-transitory computer-readable medium, such as the memory 20 and / or other storage. In one embodiment, the processor 18 may be a general-purpose processor, a system-on-chip (SoC) device, an application-specific integrated circuit (ASIC), or some similar processor configuration. In one embodiment, the processor 18 may include multiple processors.
[0022] As described above, the images generated by the light field display 10 are glasses-free 3D. Thus, no head-mounted display is required to view the images generated by the light field display 10. However, the embodiment shown in FIG. 1 is only one way in which glasses-free 3D images can be generated. For example, FIG. 2 shows a reflective light field display system 22, in which the light field display 10 of FIG. 1 not only generates a first glasses-free 3D image on the light field display 10, but also simultaneously (e.g., in real time while displaying the glasses-free 3D image) emits light rays 24, which, when reflected by one or more reflectors 26 (i.e., display devices such as mirrors, glossy surfaces, screens for mobile devices, etc.), generate a second glasses-free 3D image on the reflectors 26, which can be quite different from the first glasses-free 3D image generated by the light field display 10. Thus, in addition to the control system 16 directly controlling the pixels 12 of the light field display 10, the control system 16 shown in FIG. 2 is also configured to simultaneously (e.g., in real time while controlling the light field display 10 to display its first glasses-less 3D image) control light beams 24 emitted from the light field display 10, which can be reflected to the reflector 26 to generate a second glasses-less 3D image thereon. In some embodiments, each reflector 26 can essentially consist of only a mirror with a protective cover disposed on the non-reflective side of the mirror. Indeed, in some embodiments, the reflector 26 does not include any active display element, but rather uses only a mirror to reflect the light beams 24 generated by the light field display 10.
[0023] As explained above, while primarily described herein as displaying a first glasses-free 3D image and simultaneously illuminating light beams 24 via reflector 26 to display a second glasses-free 3D image, in other embodiments, light field display 10 can instead display a first non-glasses-free 3D image and / or illuminate light beams via reflector 26 to display a second non-glasses-free 3D image. For example, in such embodiments, light field display 10 can still display different images in different directions from reflector 26. However, the angular density of those images will be lower than required for glasses-free 3D images, as described in more detail herein.
[0024] It will be appreciated that in order to calculate a light field vector function that determines the light beam 24 that should be emitted from the light field display 10 for any particular reflecting device 26, the control system 16 will need to know at least the position, orientation and / or movement of the particular reflecting device 26. To that end, in one embodiment, each reflecting device 26 may include one or more position / orientation sensors 28 (e.g., accelerometers, magnetometers, gyroscopes, GPS receivers, motion tracking sensors such as electromagnetic and solid-state motion tracking sensors, etc.) configured to sense the position, orientation and / or movement of the reflecting device 26 relative to the physical environment 30 within which the reflecting device 26 is disposed and for which the light field display 10 generates the light beam 24. Additionally, in one embodiment, each reflecting device 26 may include communication lines 32 configured to facilitate communication of data regarding the position, orientation and / or movement of the reflecting device 26 relative to the physical environment 30. In particular, such data may be communicated to the control system 16 via communication circuitry 34 of the control system 16. As such, the control system 16 can utilize data regarding the position, orientation and / or movement of a particular reflecting device 26 relative to the physical environment 30 to automatically calculate a light field vector function that determines the light rays 24 to be cast from the light field display 10 for that particular reflecting device 26. This calculation can be done substantially in real-time with respect to the detection of the position, orientation and / or movement of the reflecting device 26 relative to the physical environment 30.
[0025] Additionally, in some embodiments, the control system 16 may itself be configured to detect the position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26 relative to the physical environment 30 in which the reflector 26 and the amusement park visitor holding the reflector 26 are located. The control system 16 may utilize data regarding the detected position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26 relative to the physical environment to automatically calculate a light field vector function that determines the light rays 24 to be cast from the light field display 10 for a particular reflector 26. The automatic calculation by the control system 16 may be made substantially in real-time with respect to the detection of the position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26 relative to the physical environment 30. For example, in one embodiment, one or more cameras 36 associated with the control system 16 can be configured to capture images and / or videos of the physical environment 30, and the control system 16 can be configured to analyze the captured images and / or videos of the physical environment 30 to uniquely recognize the reflector 26 and / or the amusement park visitor holding the reflector 26, for purposes of tracking the position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26, relative to the physical environment 30. In particular, the position, orientation and / or movement of the reflector 26 relative to the eyes 14 of the amusement park visitor holding the reflector 26 (e.g., via head tracking and / or eye tracking algorithms implemented by the control system 16 in relation to the images and / or videos captured by the one or more cameras 36) are parameters for calculating an appropriate light field vector function for casting light rays 24 associated with the particular reflector 26.
[0026] In one embodiment, the processor 18 of the control system 16 may execute computer-executable instructions stored in the memory 20 of the control system 16 which, when executed by the processor 18, cause the processor 18 to receive data from one or more position / or orientation sensors 28 of the reflector 26 and / or one or more cameras 36 relating to the position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26 relative to the physical environment 30, and to detect the position, orientation and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26 relative to the physical environment 30. having the processor 18 analyze the data to automatically calculate a light field vector function that determines which light rays 24 should be cast from the light field display 10 with respect to a particular reflector 26 in substantially real time with respect to the detection of motion, and having the processor 18 automatically send a control signal to the light field display 10 to cast the light rays 24 in accordance with the calculated light field vector function so that a glasses-free 3D image is seen on the particular reflector from the perspective of an amusement park visitor holding the reflector 26 in substantially real time with respect to the calculation of the light field vector function.
[0027] Thus, in some embodiments, the control system 16 may utilize a combination of data related to the position, orientation and / or movement of a particular reflecting device 26 based on feedback from one or more position / orientation sensors 28 of the particular reflecting device 26 and / or data related to the position, orientation and / or movement of the particular reflecting device 26 and / or the amusement park visitor holding the particular reflecting device 26 based on feedback from one or more cameras 36 of the control system 16 to determine how to calculate a light field vector function that determines the light rays 24 that should be cast from the light field display 10 relative to the particular reflecting device 26. Thus, the position / orientation sensors 28 of one or more reflecting devices 26 and / or the one or more cameras 36 of the control system 16 described herein may collectively function as a three-dimensional tracking system for the tracked position, orientation and / or movement of the reflecting device 26 and / or the amusement park visitor holding the reflecting device 26 for purposes of determining the light rays 24 that should be cast from the light field display 10 relative to the reflecting device 26.
[0028] In one embodiment, the light field vector function calculated by the control system 16 causes the light field display 10 to output only light rays 24 that are intended to be viewed by the eyes 14 of a particular amusement park visitor holding the reflector 26, but that may appear black (e.g., no visible light) from other positions from where the reflector 26 is viewed. Thus, the glasses-free 3D image reflected by the particular reflector 26 can be directed towards the eyes 14 of a particular amusement park visitor holding the particular reflector 26, whereas the reflector 26 appears, for example, as a surface void (e.g., a black surface, in some cases) of the additional image provided by the light rays 24 when viewed by other amusement park visitors from other viewpoints. Instead, the glasses-free 3D image reflected by a particular reflector 26 can be directed directly to the eyes 14 of the amusement park visitor holding the particular reflector 26, whereas the reflector 26 can project different images when viewed by other amusement park visitor from other viewpoints in a particular state. In other words, different viewpoints from different positions relative to the particular reflector 26 can project entirely different images (or there is no additional image provided by the light beam 24). It will be appreciated that the ability of the light field display 10 to only project the light beam 24 to the eyes 14 of a particular amusement park visitor holding a particular reflector 26 allows the light field display 10 to project the displayed glasses-free 3D image through the reflector 26 (and indeed also through the camera 44 and / or through the light field display 10 itself) without being recorded in an unauthorized manner by the amusement park visitor.
[0029] The embodiment shown in Figure 2 includes only one reflector 26 in use. However, in other embodiments, the light field display 10 can be used to project light beams 24 onto multiple reflectors 26 in a particular physical environment 30. Figure 3 illustrates a light field display system 22 in which the light field display 10 of Figure 1 projects light beams 24A, 24B onto a plurality of respective reflectors 26A, 26B held by a plurality of respective amusement park attendees 38A, 38B in a particular physical environment 30. Although shown in Figure 3 as including only two reflectors 26A, 26B held by two respective amusement park attendees 38A, 38B, it will be understood that any number of reflectors 26 held by amusement park attendees 38 can be used in a particular physical environment 30.
[0030] In embodiments such as those shown in FIG. 3, the control system 16 is configured to enable the light field display 10 to emit light beams 24A, 24B to assign reflective devices 26A, 26B in an appropriate manner, as described above with respect to FIG. 2, to distinguish particular amusement park visitors 38A from other amusement park visitors 38B, and to distinguish particular reflective devices 26A from other reflective devices 26B. In one embodiment, the control system 16 can be configured to individually recognize a particular amusement park visitor 38A, 38B based at least in part on facial recognition techniques, clothing recognition techniques, motion recognition techniques (e.g., detecting gait consistent with a particular amusement park visitor 38A, 38B) and / or other visitor recognition techniques (e.g., via visitor recognition algorithms executed by the control system 16 with respect to images and / or videos captured by one or more cameras 36 of the control system 16), and / or to individually recognize a particular reflecting device 26A, 26B based at least in part on geometric aspects or markings present on the reflecting devices 26A, 26B (e.g., via reflecting device recognition algorithms executed by the control system 16 with respect to images and / or videos captured by one or more cameras 36 of the control system 16).
[0031] In some embodiments in which the light field display 10 projects light beams 24 to a plurality of respective reflectors 26 held by a plurality of respective amusement park guests 38, the light field display 10 can be configured to project the same glasses-free 3D image to all amusement park guests 38 in a particular state via all reflectors 26. Alternatively, in other states the light field display 10 can project the glasses-free 3D image to only one or a selected subset of the amusement park guests 38 via their respective reflectors 26. In some embodiments, only a particular amusement park guest 38 can be able to view a particular glasses-free 3D image via their respective reflectors 26 based on data about the particular amusement park guest 38. For example, in one embodiment, the control system 16 can keep track of which amusement park guests 38 are members of particular teams in a game and can instruct the light field display 10 to only illuminate the reflectors 26 of the amusement park guests 38 on that particular team with glasses-less 3D images.
[0032] In some embodiments, instead of using a reflecting device 26 (e.g., having a mirror) to generate a reflected image, a camera can be used. FIG. 4 shows a light field display system 40 in which the light field display 10 of FIG. 1 not only generates a glasses-free 3D image on the light field display 10, but also simultaneously projects light rays 24 through one or more cameras 42 (e.g., display devices) to display a glasses-free 3D image when viewed through the cameras 42. The embodiment shown in FIG. 4 is substantially similar to the embodiment shown in FIG. 2 and FIG. 3. However, instead of the light rays 24 projected from the light field display 10 being reflected by a reflective surface (e.g., a mirror) of the reflecting device 26, the light rays 24 are received by one or more cameras 42 through which the glasses-free 3D image is generated. Each of the features described above with respect to the embodiment shown in FIG. 2 and FIG. 3 can be extended to the embodiment shown in FIG. 4 as well. For example, in one embodiment, an amusement park visitor 38 using the camera 42 shown in FIG. 4 may be able to see a particular image through the camera 42, while other amusement park visitor 38 may be unable to see the glasses-free 3D image directly from the light field display 10. Alternatively, in other embodiments, the glasses-free 3D image may be made viewable directly from the light field display 10, without going through the camera 42. Rather, in such a situation, a different glasses-free 3D image (or no image at all) may be viewable through the camera 42. Thus, for example, a projection of a person may be viewable when the amusement park visitor 38 is able to view an area with the naked eye, but the person may be invisible to the amusement park visitor 38 when observing the same area through the camera 42 (or via a reflecting device 26, such as a mirror).
[0033] 3 , it will be appreciated that occlusion between the light field display 10 and the display devices 26, 42 described herein can occur when amusement park guests 38 move their respective reflectors 26, 42 around the physical environment 30, as well as when other amusement park guests 38 move around their own physical environment 30. When the light beams 24 projected onto a display device 26, 42 are blocked (e.g., physically blocked) from reaching their intended display device 26, 42 by an amusement park guest 38 and / or other display devices 26, 42, the immersion of the experience may be interrupted and / or the amusement park guest 38 may become frustrated because they can no longer see the glasses-free 3D image through their respective reflectors 26, 42.
[0034] In some embodiments, the control system 16 can provide a warning when an occlusion is about to occur based on the detected position, orientation, and / or movement of the reflector 26 and / or the amusement park visitor holding the reflector 26, 42. In some embodiments, after a warning of an upcoming occlusion occurs for a particular display device 26, 42, and before an occlusion actually occurs at the particular display device 26, 42, the control system 16 can cause the light field display 10 to project a light beam 24 that displays an image to the amusement park visitor 38 holding the particular display device 26, 42 to justify and explain the upcoming occlusion. For example, the displayed image can include decorative static (e.g., magic particles, static, etc.) that matches the visual aesthetic of the amusement park attraction that includes the physical environment 30. In other words, the control system 16 can create a rendition that makes the occlusion contextually interpretable in a manner that is consistent with the immersiveness of the themed environment. As specific examples, the control system 16 may provide a warning of an impending occlusion, or some indication that a character being displayed on the display is about to magically disappear (e.g., in a magically themed scenario), or that technical difficulties are occurring (e.g., in a science fiction themed scenario).
[0035] In one embodiment, after a warning of an upcoming occlusion occurs for a particular display device 26, 42, and before occlusion actually occurs at the particular display device 26, 42, the control system 16 can cause the light field display 10 to emit light beams 24 that display instructive images (e.g., directional arrows, instruction messages, etc.) to amusement park visitors 38 holding the particular display device 26, 42 as to what instructions they need to move the particular display device 26, 42 to avoid occlusion. In one embodiment, the messages can be displayed on some other display device that is associated with the particular display device 26, 42 after occlusion occurs. For example, in one embodiment, a liquid crystal display (LCD) can be positioned behind the particular (e.g., partially transparent) display device 26, 42 and can be switched on (e.g., based on a control signal received from the control system 16) when occlusion occurs, for example, to show an arrow pointing in the direction in which the particular display device 26, 42 can be moved to undo the occlusion.
[0036] Due to the relatively limited update rate of the tracking system (e.g., the position / orientation sensor 28 and / or the one or more cameras 36) and the light field display 10 described herein, if the display device 26,42 or the amusement park visitor 38 holding the display device 26,42 moves relatively quickly, the image may stop appearing on the display device 26,42 because the display device 26,42 has moved out of the area in which the image is being illuminated. In one embodiment, to avoid or at least mitigate this problem, the light field display 10 may illuminate the light beam 24 over a larger area than the area that the display device 26,42 actually takes up. This solution may give the display device 26,42 some additional space to move before an update needs to occur.
[0037] 5 is a flow chart of a method 44 for operating the light field display system 22, 40 of FIGS. 2-4. As shown in FIG. 5, in one embodiment, the method 44 includes displaying a first glasses-free 3D image via the light field display 10 (block 46). Additionally, in one embodiment, the method 44 includes tracking, via the control system 16, a position, orientation, or movement of one or more display devices 26, 42 in the physical environment 30 and / or a portion of one or more humans (e.g., amusement park visitors 38) associated with the one or more display devices (e.g., a portion of the humans themselves or a portion of a particular one of the humans, such as a human's eye for eye tracking, a human's head for head tracking, etc.) (block 48). Additionally, in some embodiments, the method 44 includes calculating, via the control system 16, respective light field vector functions for light rays 24 to be cast by the light field display 10 toward the one or more displays 26, 42 based at least in part on the tracked position, orientation, and / or movement of the one or more displays 26, 42 and / or a portion of one or more people (e.g., amusement park attendees 38) associated with the one or more displays 26, 42 (block 50). Additionally, in some embodiments, the method 44 includes causing the light field display 10, via the control system 16, to cast the light rays 24 toward the one or more displays 26, 42 according to the respective calculated light field vector functions to display the second glasses-free 3D image via the one or more displays 26, 42 (block 52).
[0038] As described in more detail herein, in some embodiments, a first glasses-free 3D image can be displayed via the light field display 10 while the light field display 10 also generates and emits light beams 24 that are directed toward the one or more displays 26, 42 to display a second glasses-free 3D image via the one or more displays 26, 42. Thus, the embodiments described herein can enable a variety of different optical effects that would not be possible otherwise. For example, in some embodiments, a glasses-free 3D image can be visible when looking directly at the light field display 10, but not visible when viewed through the reflector 26. As a non-limiting example, there can be a character such as a Bambier that is visible when looking directly at the light field display 10, but cannot be seen in the reflector 26, as shown in FIG. 6. In such circumstances, the light beams 24 emitted by the light field display 10 can be disabled.
[0039] Additionally, in some embodiments, glasses-free 3D images can be generated in such a way that their reflections in the reflector 26 blend in with the physical objects in the physical environment 30. In this way, as shown in FIG. 7, the reflector 26 can act as a magic filter that shows objects that are not physically located in the physical environment when held in close proximity to the objects in the physical environment 30 (e.g., an x-ray projection 40 passing through a physical wall 42). In such an embodiment, a semi-transparent mirror (e.g., partially reflective and partially transparent) can be used as the reflector 26. In other words, in some embodiments, only a portion of the reflector 26 can be reflected, whereas some light also passes through the reflector 26.
[0040] Additionally, in some embodiments, glasses-free 3D images can be generated in such a way that their reflections at the reflector 26 are seen in the physical space associated with the reflector 26. For example, as shown in Fig. 8, in some embodiments, an amusement park visitor 38 holding a particular reflector 26 can perceive the reflection of light rays 24 emitted from the light field display 10 as the three-dimensional space inside a box 58 behind the reflector 26 (which can be of a much larger size than the reflector 26 itself, for example). An amusement park visitor 38 holding a particular reflector 26 can potentially interact with these images using an input device 60 (e.g., a touch panel) at the reflector 26 (see, for example, Fig. 2). Additionally, in one embodiment, a small digital object 62 appearing in a box behind the reflector 26 can move around (e.g., as indicated by arrow 64) based on a physics simulation performed by the control system 16 based on the orientation of the particular reflector 26, which can be detected by one or more position / orientation sensors 28 of the reflector 26, as described in more detail herein.
[0041] Additionally, although primarily described herein as displaying a first glasses-less 3D image and simultaneously illuminating light beams 24 toward reflector 26 to display a second glasses-less 3D image via reflector 26, in other embodiments light field display 10 can instead display a first non-glasses-less 3D image and / or illuminate light beams toward reflector 26 to display a second non-glasses-less 3D image via reflector 26. For example, in such embodiments light field display 10 can still be capable of displaying different images in different directions than reflector 26, but the angular density of these images will be lower than that required for glasses-less 3D images, as described in more detail herein.
[0042] Additionally, in some embodiments, the control system 16 can calculate the light field vector function described herein such that the light rays 24A, 24B are reflected off of multiple reflectors 26A, 26B in a particular order. In this way, the effect can still be seen when amusement park visitors 38 see the reflections of the reflections. Figure 9 shows how each reflection can be varied on the various reflectors 26A, 26B.
[0043] While only certain features have been illustrated and described herein, numerous modifications and changes will occur to those skilled in the art, and it is therefore understood that the appended claims are intended to cover all such modifications and changes which fall within the true spirit of the disclosure.
[0044] The technology described and claimed herein clearly advances the state of the art, and thus reference is made to and applies tangible objects and specific examples of a practical nature that are not abstract, vague, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements specified as a "method of [performing] [function]" or a "step of [performing] [function]," it is intended that such elements be construed under 35 U.S.C. 112(f). However, with respect to any claim containing an element specified in any other manner, such elements are not to be construed under 35 U.S.C. 112(f).
Claims
1. one or more light field displays configured to display a first glasses-free 3D image and to project light towards the one or more displays to display a second glasses-free 3D image via the one or more displays; 1. A control system comprising: tracking the position, orientation and / or movement of the one or more displays and / or one or more human parts associated with the one or more displays in a physical environment; calculating respective light field vector functions for the light rays projected by the one or more light field displays towards the one or more display devices based at least in part on tracked positions, orientations and / or movements of the one or more display devices and / or the one or more human parts associated with the one or more display devices; a control system configured to cause the one or more light field displays to project the light rays towards the one or more display devices according to the light field vector functions respectively calculated for displaying the second glasses-less 3D image via the one or more display devices. Light field display system.
2. The one or more display devices include:
2. The light field display system of claim 1, further comprising one or more reflecting devices configured to reflect the light beams emitted towards the one or more reflecting devices to display the second glasses-free 3D image via the one or more reflecting devices.
3. The one or more display devices include: and one or more cameras configured to pass the light beam projected toward the one or more cameras to display the second glasses-free 3D image via the one or more cameras.
10. The light field display system of claim 1.
4. One display device of the one or more display devices is one or more position / orientation sensors configured to detect a position, orientation and / or movement of the display device; and a communication circuit configured to send data related to the detected position, orientation and / or movement of the display device to the control system.
10. The light field display system of claim 1.
5. the light field display system comprising one or more cameras configured to capture images and / or video of the one or more display devices and / or the one or more people associated with the one or more display devices; The control system includes: analyzing the captured images and / or videos for uniquely recognizing the one or more displays and / or the one or more people associated with the one or more displays; configured to track a position, orientation and / or movement of a part of the one or more display devices and / or the one or more humans associated with the one or more display devices based on an identification of the one or more display devices and / or the one or more humans associated with the one or more display devices.
10. The light field display system of claim 1.
6. the second glasses-free 3D image displayed on one of the one or more display devices is configured to be visible to both eyes of a person associated with the display device and configured to appear black from other positions relative to the display device; 10. The light field display system of claim 1.
7. The control system includes: and causing the one or more light field displays to emit the light beams toward the one or more display devices only if the one or more display devices or the one or more people associated with the one or more display devices satisfy a certain requirement.
10. The light field display system of claim 1.
8. The control system includes: the second glasses-free 3D image is configured to be displayed on the one or more display devices in a manner consistent with physical objects in the physical environment.
10. The light field display system of claim 1.
9. the second glasses-free 3D image displayed on each of the one or more displays appears to be located in a physical space associated with the display.
10. The light field display system of claim 1.
10. the control system is configured to cause the one or more light field displays to sequentially project the light beams towards the one or more display devices.
10. The light field display system of claim 1.
11. The control system includes: the one or more light field displays; predicting an upcoming occlusion between one of the one or more display devices and / or a portion of the one or more humans associated with the one or more display devices based at least in part on a position, orientation, and / or movement of the one or more display devices in the physical environment; configured to adjust the light rays emitted from the one or more light field displays to the display device prior to an occlusion to justify and account for an upcoming occlusion.
10. The light field display system of claim 1.
12. Displaying a first glasses-free 3D image via one or more light field displays; tracking, via a control system, the position, orientation and / or movement of one or more display devices and / or one or more human parts associated with said one or more display devices in a physical environment; calculating, via the control system, respective light field vector functions for light rays projected by the one or more light field displays towards the one or more display devices based at least in part on the tracked position, orientation and / or movement of the one or more display devices and / or the one or more human parts associated with the one or more display devices; causing, via the control system, the one or more light field displays to project the light rays towards the one or more display devices according to the light field vector functions respectively calculated to display a second glasses-less 3D image via the one or more display devices; The method includes:
13. reflecting the light beam onto the one or more display devices to display the second glasses-free 3D image via the one or more display devices; 13. The method of claim 12, comprising:
14. passing the light beam through the one or more display devices to display the second glasses-free 3D image via the one or more display devices; 13. The method of claim 12, comprising:
15. detecting a position, orientation and / or movement of one of the one or more display devices via one or more position / orientation sensors of the display device; sending data relating to the detected position, orientation and / or movement of the display device to the control system via communication circuitry of the display device; 13. The method of claim 12, comprising:
16. capturing images and / or video of the one or more displays and / or the one or more people associated with the one or more displays via one or more cameras; analyzing, via the control system, the captured images and / or videos to uniquely recognize the one or more displays and / or the one or more people associated with the one or more displays; tracking, via the control system, a position, orientation and / or movement of a portion of the one or more displays and / or the one or more humans associated with the one or more displays based on an identification of the one or more displays and / or the one or more humans associated with the one or more displays; 13. The method of claim 12, comprising:
17. causing, via the control system, the one or more light field displays to project the light beams towards the one or more display devices only if the one or more people associated with the one or more display devices satisfy certain requirements; 13. The method of claim 12, comprising:
18. causing the one or more light field displays to sequentially project the light beams towards the one or more display devices via the control system; 13. The method of claim 12, comprising:
19. via the control system, predicting an upcoming occlusion between the one or more light field displays and a display device of the one or more display devices based at least in part on a position, orientation and / or movement of the one or more display devices in the physical environment and / or a portion of the one or more humans associated with the one or more display devices; and adjusting, via the control system, the light rays emitted from the one or more light field displays to the display device prior to an occlusion to justify and account for the upcoming occlusion. The method of claim 12.
20. one or more light field displays configured to display a first glasses-free 3D image and to project light towards the one or more displays to display a second glasses-free 3D image via the one or more displays; 1. A control system comprising: tracking the position, orientation and / or movement of the one or more displays and / or one or more human parts associated with the one or more displays in a physical environment; a control system configured to cause the one or more light field displays to project the light rays towards the one or more display devices according to a light field vector function to display the second glasses-less 3D image via the one or more display devices, the light field vector function being calculated based at least in part on tracked positions, orientations and / or movements of the one or more display devices and / or the one or more human parts associated with the one or more display devices. Light field display system.
Citation Information
Patent Citations
Image processing device, image processing method, and program
JP2014078095A
Method and apparatus for image processing
JP2015091136A
Image processing device and method for rendering subpixels
JP2015504641A
Multi-view (MV) display systems and methods for quest experiences, challenges, scavenger hunts, treasure hunts and alternate reality games
US20190015747A1
Blended reality systems and methods
US9858719B2
