Virtual Window
The system projects VR environments onto a distant surface using sensors and projectors, addressing the discomfort and sickness issues of wearable devices by providing a comfortable, immersive, and interactive VR experience.
Patent Information
- Application Number
- JP2024522464
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-10-14
- Filing Date
- 2022-09-29
- Publication Date
- 2024-11-13
AI Technical Summary
Existing virtual reality (VR) technologies require users to wear interactive devices such as goggles or headsets, which can be uncomfortable and cause VR sickness due to conflicting sensory inputs.
A system that projects a simulated VR environment onto a surface remote from the user using a projector and sensors to detect the user's location and movement, eliminating the need for wearable devices by rendering the environment in real-time based on the user's position and perspective.
Provides a fully immersive VR experience without the discomfort of wearable devices, reducing VR sickness and allowing users to move freely in a large area while maintaining spatial presence and interaction.
Smart Images

Figure 2024541846000001_ABST
Abstract
Description
[Technical field]
[0001] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure generally relate to methods and systems for visual projector technology, for example, to generate virtual reality (VR) environments that may be immersive and interactive for a user. [Background technology]
[0002] Virtual reality (VR) uses computer modeling and simulation to enable humans to interact with an artificial three-dimensional (3D) visual or other sensory environment. Currently, many VR applications immerse users in computer-generated environments that simulate reality using interactive devices that send and receive information. Typically, these interactive devices are often placed and / or worn directly on the user's body, for example as goggles around the user's eyes, a headset that covers the user's head (and eyes), gloves, or a body suit. In a typical VR format, a user wears a VR headset with a stereoscopic screen and can view animated images of the simulated environment. Summary of the Invention
[0003] According to one embodiment of the disclosed technology, a system and method are described for projecting a simulated environment, such as virtual reality (VR), onto a surface remote from a user in a manner that may obviate the need for the use of a wearable interactive device.
[0004] According to one embodiment of the disclosed technology, the system may include a projector that projects an image onto a projection surface. The projection surface may be located at a distance away from a user. The system may further include a plurality of sensors that form a detection field with respect to a spatial region encompassing a separation distance between the projection surface and the user. The plurality of sensors may capture user position data representative of the user's current location within the detection field. The system may also include at least one processor that renders the computer simulated environment as a projected image, the at least one processor processing the captured position data for the user such that the computer simulated environment is generated with a perspective relative to the user's current location within the detection field.
[0005] According to another embodiment of the disclosed technology, a method can include enabling two or more projector devices. Each of the two or more projector devices can be disposed in a separate physical environment, each separate physical environment can include a respective user within a field of view of the respective projector device. Further, the two or more projector devices can be used to create a shared computer simulated environment, the shared computer simulated environment including a three-dimensional (3D) image rendering including a visual representation of each user in the separate physical environment. The shared computer simulated environment can then be projected onto two or more projection surfaces, each of the two or more projection surfaces being disposed in a separate physical environment within a field of view of the respective user.
[0006] Other features and aspects of the disclosed technology will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, features in accordance with embodiments of the disclosed technology. This Summary is not intended to limit the scope of the invention(s) described herein, which is defined only by the claims appended hereto. [Brief description of the drawings]
[0007] The present disclosure, in accordance with one or more various embodiments, will now be described in detail with reference to the following figures, which are provided for purposes of illustration only and merely depict typical or exemplary embodiments.
[0008] [Figure 1] 1 illustrates an exemplary environment in which a virtual reality (VR) projector may be implemented to, for example, immerse a single user in a VR environment, according to one embodiment of the technology disclosed herein.
[0009] [Diagram 2] 1 illustrates an exemplary environment in which two VR projectors can be implemented, for example, to immerse multiple remote users in a VR environment, according to one embodiment of the technology disclosed herein.
[0010] [Diagram 3] FIG. 1 is an operational flow diagram illustrating an example of a process for performing projection of a VR environment in accordance with some embodiments.
[0011] [Figure 4] FIG. 1 is an operational flow diagram illustrating an example process for implementing the projection of a shared simulation, such as a VR environment, that can be presented to multiple remote users simultaneously, in accordance with some embodiments.
[0012] [Diagram 5] 1 illustrates an exemplary computing system that can be used to implement various features of the embodiments described in this disclosure.
[0013] The figures are not exhaustive and are not intended to limit the disclosure to the precise forms disclosed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] As referred to herein, the term virtual reality (VR) technology can encompass multiple forms of immersive technologies, such as VR, augmented reality (AR), mixed reality, extended reality (XR), and three-dimensional visualization, which can generate realistic and immersive computer simulations of three-dimensional (3D) environments that allow for user interaction.
[0015] By way of background, VR environments are specifically designed to create a phenomenon known as presence. Presence refers to the human physical and mental sensations that simulate the user being physically "there" while immersed in a simulated (or VR) environment. Presence can be provided by motion sensors that detect the user's movements and adjust the on-screen view accordingly, nominally in real time (e.g., to match the user's movements). Presence can be further supported in VR environments because simulated objects in the environment have a sense of "spatial presence." Spatial presence can be described as objects in the environment effectively having a location in three-dimensional space that is relative to and independent of the user's location. One goal of "presence" and "spatial presence" can be to generate a simulation that presents the correct cues to the human perceptual and cognitive system, so that the brain interprets those cues as objects in the three-dimensional world from a perspective that is consistent with the user's visual perspective (e.g., the user's eye position) as they move through the virtual world.
[0016] In some embodiments, the VR environment generated by the disclosed methods and systems is a computer-generated 3D environment, including both computer graphics and 360-degree video, that surrounds the user and responds to the user's interactions. Hand and body tracking is provided by gesture recognition and handheld controllers, and haptic (or touch-sensitive) feedback may also be incorporated. Room-based systems may provide 3D experiences while moving through a large area, or may be used with multiple participants. Thus, VR environments can simulate many real-world experiences, such as moving through a series of different simulated scenes and experiencing changing perspectives and views that relate convincingly to the user's own head rotation and gait. Another common VR feature involves wearing data gloves with a force feedback device that provides a sense of touch, and the user can even pick up and manipulate objects seen in the virtual environment. Due to the technology's many powerful interactive capabilities and realistic immersive experiences, VR is emerging as a tool used in a wide range of applications, such as entertainment (e.g., video games), education (e.g., medical training, military training, armed forces training), and business (e.g., virtual meetings). However, such VR-like experiences typically require the user to hold a device in their hands or wear a device (e.g., goggles) on their head, and being restricted to such interactive devices can be uncomfortable or cumbersome for many users.
[0017] Furthermore, as VR designers attempt to improve the "presence" of the simulated environment, users may experience a phenomenon known as "VR sickness." As an example, while a person is using a VR headset, their eyes perceive simulated motion around them, and their inner ear also senses that virtual motion is occurring around them. However, the person's muscles and eyes continue to sense that they are still and not moving, despite the images being generated by the VR headset. As a result, users of many interactive devices, such as VR goggles and headsets, experience their inner ears (e.g., sensing motion) and their bodies (e.g., sensing stillness) sending contradictory messages to the brain simultaneously. As a result, it is not uncommon for users who are immersed in a VR experience to experience "VR sickness," which mimics the sickness caused by real motion.
[0018] The system disclosed herein implements a device, hereafter referred to as a View Rendering Device (VRD), that integrates VR (or 3D) projection and real-time VR (or 3D) computer rendering capabilities. Thus, a VRD can be used to provide a fully immersive VR experience that does not require various interactive devices, such as portable computing devices (e.g., tablets, smartphones), VR goggles, VR headsets, or other wearable interactive devices, and does not restrict the user's movements. While the description refers to a "VR environment" for illustrative purposes, this is not intended to be limiting, and it should be understood that the disclosed embodiments can be implemented to present other forms of 3D and / or immersive environments known in the art. According to an embodiment, the VRD is explicitly designed to project 3D images onto a projection surface separated at a distance from the user. Furthermore, the VRD supports real-time VR rendering capabilities, such as detecting the user's position and / or movement, determining spatial presence within the simulation, which are essential to building a simulated VR environment for the user. Thus, the disclosed embodiments realize the benefits of creating an immersive VR environment for a user by utilizing projection in a manner that does not require interactive devices of a common type, ultimately mitigating the undesirable effects on the user that can result from using such devices, such as disorientation, confusion, and VR sickness.
[0019] FIG. 1 illustrates an exemplary environment for implementing a system 100 for projecting three-dimensional (3D) simulated images that ultimately creates a complete VR environment that can be viewed and interacted with by a single user or by multiple users located in close proximity to one another (e.g., in the same vicinity or physical area). In the example of FIG. 1, the system 100 includes a view rendering device (VRD) 120, which may include at least a projector 122 configured to project 3D images onto a projection surface 130 according to a 3D projection technique, a central processing unit (CPU) 124 configured to process data and algorithms according to the VR technique and render 3D images used in creating the 3D computer simulated environment, and a number of sensors 126 configured to capture position data of a user 150. In various embodiments, the VRD 120 is configured to, among other things, integrate VR (or 3D) projection functionality and real-time VR (or 3D) computer rendering functionality. In the example of FIG. 1, the VRD 120 is shown projecting a projected image 132 (e.g., a 3D image) onto a projection surface 130, thereby creating a VR environment that can fully immerse the user 150 in a computer-generated simulation. The projection surface 130 can be a wall, a floor, a ceiling, a panel, or any suitable stable fixed surface (or combination of surfaces) having suitable dimensions to display a desired 3D image. The projected image 132 can be a computer-rendered multi-dimensional image or graphic, such as a 3D image, that is visible to the user 150, thus providing an image that creates a simulated environment in VR, for example. The projected image 132 can be rendered and projected by the VRD 120 such that from the perspective of the user 150 immersed in the simulated environment according to VR (or other immersive simulation technology), the image has depth, changing space, and other visual cues.1 illustrates that by projecting the projected image 132 onto a projection surface 130 that is at a distance from the user 150, the system 100 creates a perception in the user 150 that he is looking at the "virtual world" through a "virtual window" to experience the VR environment, as opposed to looking at a simulation through an interactive device that the user wears (in close proximity to the eyes), such as VR goggles or a VR headset. In other words, by looking at the 3D image projected, for example, by the VRD 120 onto a wall, the user 150 can view a fully immersive 3D "virtual world" in a manner similar to looking out a window to see a physical scene in the "real world."
[0020] In general, the system 100 may be placed in a variety of physical environments with a sufficient amount of space for the components of the system 100 to be located at an appropriate distance from the user 150 to adequately support room-scale VR. As an example, an environment suitable for room-scale VR and useful for implementing the disclosed system 100 may have approximate dimensions of free space of at least 2 meters long (L) by 2 meters wide (W) (6.5 feet by 6.5 feet). Physical environments such as large indoor rooms and outdoor areas may be ideal places to set up the system 100 and project 3D images according to room-scale VR. Furthermore, the physical environment must provide the necessary clearance for the user 150 to move freely (e.g., rotate 360 degrees) to enable immersion in the VR environment and user interaction with the simulation in room-scale VR. As alluded to above, an important aspect of system 100 includes projecting the VR environment substantially away from the eyes (e.g., at a foot) so that user 150 views the VR environment, rather than projecting it very close to the eyes (e.g., millimeters) as is common with VR headsets, VR goggles, etc. Thus, as seen in FIG. 1, projection surface 130 is positioned at a distance away from the user.
[0021] Additionally, the multiple sensors 126 of the VRD 120 may be positioned at various locations on the device such that the sensors 126 form a detection field 128 (shown in FIG. 1 by a dashed line surrounding the user 150) for a spatial region encompassing the separation distance between the projection surface 130 and the user 150. Thus, as the user 150 moves within the detection field 128 during the simulation, the multiple sensors 126 may detect the user 150 and capture corresponding position data representative of the current location of the user 150 within the detection field 128.
[0022] As shown in FIG. 1, the VRD 120 may include a CPU 124. The CPU 124 may include various hardware components, such as a processor and memory, and software elements, such as algorithms, modules, and applications, for providing real-time 3D image rendering capabilities and various other real-time VR capabilities, such as depth perception, spatial presence, and tracking of user movement in conjunction with other components of the system 100 (e.g., detecting the user's position and / or orientation). For example, position data of the user 150 captured by the multiple sensors 126 may then be processed and / or analyzed by the CPU 124, such that a computer simulation environment (e.g., a VR environment) may be dynamically rendered in real time with a perspective relative to the user 150's current position within the detection field 128. The CPU 124 may be considered a high-performance computer having the processing power required to execute calculations and / or algorithms for performing 3D image rendering and other functions related to VR and / or simulation immersion technology, as well as algorithms for performing complex data analysis functions, such as predictive calculations based on the user's 150 current and recent past movements. As an example, CPU 124 may be configured to calculate predictive data associated with user 150 based on currently captured position data representative of the user's current location within detection field 128 and previously captured position data representative of the user's past locations within detection field 128. As a result, CPU 124 can then generate a 3D image rendering using the predictive data representative of the user's current and / or predicted future movements and the historical data representative of the user's past movements. By having the ability to predict the user's position and / or movements, CPU 124 can re-render the 3D simulation image with improved efficiency and reduced latency (e.g., reducing network latency).
[0023] In some embodiments, the CPU 124 may have other hardware and / or software modules (not described herein) present thereon to perform functions related to providing a simulated environment in real time using VR and 3D projection technologies. Although the CPU 124 is shown as being physically integrated into a single device that includes the projector 124, the plurality of sensors 126, and other components of the VRD 120, this configuration is not intended to be limiting. In some embodiments, the CPU 124 may be a physically standalone component that is physically separated from other elements of the VRD 120 while continuing to communicate (e.g., communicatively coupled by a communications network or physical connector) and function in concert with the projector 122 and the plurality of sensors 126 to perform the features and functions described herein.
[0024] FIG. 1 further illustrates that the VRD 120 can include multiple sensors 126. In one embodiment, the sensors 126 can be implemented as rear-facing cameras. That is, the multiple sensors 126 can be cameras pointed toward a user 150 standing behind the VRD 120. Each individual sensor in the multiple sensors 126 can be positioned to have an independent position and field of view. Collectively, the collection of independent fields of view of each of the multiple sensors 126 can form a larger detection field for a volume of space encompassing the separation distance between the projection surface 130 and the user 150. In some embodiments, the multiple sensors 126 are aligned to a single 3D coordinate system, which allows the sensors to detect position, movement, and depth values corresponding to the user 150 in the real world (within the detection field 128), ultimately allowing the rendering of the VR environment to be calibrated for spatial presence at the perspective of the user 150. For example, multiple sensors 126 may detect the body position and orientation of user 150, particularly the head position and orientation, and relay this information to other components of VRD 120, such as CPU 124, for further processing and / or analysis.
[0025] In some embodiments, multiple sensors 126 can be used to capture the movements of the user 150 with respect to interacting and controlling virtual objects in the VR environment. For example, the sensors 126 can be utilized in a manner that allows the user to sense and control various virtual objects in real time. This further enables the VR environment to include interactive responses (e.g., scene changes) and / or feedback to the user's input detected by the sensors 126. The system 100 can use multiple sensors 126 to support gesture interactive techniques, gesture interactive input devices, and gesture interactive recognition techniques.
[0026] In some embodiments, the sensors 126 may be included to detect not only the position and / or movement of the user, but also other objects and users that may be present within the detection field 126 during the simulation. This may enable various additional features of the system 100, such as depicting other users in the simulation, supporting augmented reality (AR) to model surrounding real objects in the simulation, projecting aspects of the simulation onto surrounding objects, etc. The sensors 126 may be implemented as various types of sensors capable of detecting position and / or movement within the surrounding environment, such as, for example, sonar, ultrasonic distance sensors, radar, light detection and ranging (LIDAR), and cameras or other image sensors. In some embodiments, the camera may be a high dynamic range (HDR) camera or an infrared (IR) camera. Thus, the sensors 126 may be configured to obtain and / or sense position data and / or movement data of the user 150. For example, the sensors 126 may be configured to detect, quantify, and / or sense objects in at least a portion of the detection field 128, information / data regarding such users and / or objects. As described above, the sensors 126 can detect users and objects, which can be stationary and / or dynamic objects. Additionally, the sensors 126 can be configured to detect, measure, quantify, and / or sense other spatial or environmental characteristics that can be within the detection field 128.
[0027] FIG. 1 also shows that the VRD 120 can include a projector 122. The projector 122 can be described as a hardware device with light emitting capabilities for imposing a multi-dimensional (3D) image on a surface dedicated to displaying a projected image, such as the projection surface 130, in a manner that supports room-scale VR. In some embodiments, the projector 122 is implemented as a stereoscopic projector that supports high resolution, high image quality, and high 3D stereo conformity suitable for VR technology. The projector 122 can be implemented using other types of known projection technologies, such as laser projection. In operation, the projector 122 receives rendered 3D image content from the CPU 124. The VRD 120 can then project these rendered 3D images and / or graphics with multi-dimensional effects onto the projection surface 120 to create a VR environment that includes an immersive and interactive experience for the user 150. The projector 122 can include at least one projection port and an optical system for optically projecting a 3D image. According to an embodiment, projector 122 is configured to enable 3D and VR projection capabilities, among other things, so that a multi-dimensional image can be imposed on projection surface 130. As previously mentioned, it is an important aspect of system 100 to leverage the 3D and VR projection capabilities of VRD 120 to enable user 150 to experience a VR environment (or simulation) within the volume of a space (e.g., a room). In some embodiments, projector 122 can project according to other types of simulation and immersion techniques, such as, for example, to provide mixed reality and / or AR effects to the simulation.
[0028] For illustrative purposes, the exemplary environment shown in FIG. 1 may be an indoor room, and the system 100 may be implemented to project 3D images to ultimately provide room-scale VR. FIG. 1 illustrates an example of a VR environment that may be used for target practice, with the projected image 132 being a shooting range (including multiple targets). In some cases, the virtual environment may be a landscape conceived by a designer, or may be a recreation of a real scene. The VRD 120 is generally positioned between the user 150 and the projection surface 130 (which may be a wall in this example). As previously mentioned, the projection surface 130 is positioned at a distance away from the user 150. The VRD 120 may be set at any position within this separation distance between the user 150 and the VRD 120, generally speaking. In exemplary operation, as the user 150 moves within the detection field 120, the position and orientation of the user's body, and in particular the position and orientation of the user's 150's head, may be captured by the multiple sensors 126 and relayed to the CPU 124 for processing.
[0029] VRD 120 can use this position data obtained by multiple sensors 126 to render a 3D image, shown as projected image 132, onto projection surface 130 that takes into account the position of user 150 in relation to how the VR environment is presented. In this manner, as user 150 moves within detection field 128, VRD 120 can dynamically re-render projected image 132 in a manner that mimics a simulated 3D representation with perceived depth, for example, so that the user can move within detection field 128 and see behind or around objects in the VR simulation that are being projected onto projection surface 130.
[0030] As an example, if the user 150's head turns to the left, the graphics of the exemplary VR environment can detect this movement and react accordingly, e.g., adjusting the rendering of the projected image 132 to move the central target more to the right of the screen. Thus, the 3D projection and 3D rendering capabilities of the disclosed system 100 may enable the rendering of multi-dimensional (e.g., 3D) images and graphics to be projected in a controlled manner within a room in a manner that provides an immersive VR environment within the open area of the room.
[0031] According to various disclosed embodiments, the system 100 including the VRD 120 can provide an immersive experience to the user by generating various types of interactive activities related to visual, auditory, and tactile perception. Specialized equipment such as VR headsets and data gloves allow the user to sense and control various virtual objects in real time, creating experiences not available in the real world and producing realistic reactions and feedback.
[0032] 2 illustrates an exemplary environment 200 that can implement two VRDs, shown as a first VRD 120a and a second VRD 120b, to simultaneously immerse users at multiple remote locations in a shared VR environment. In other words, the exemplary environment 200 is configured such that a shared VR environment can be presented to multiple users at different locations via multiple "virtual windows" located at each of the respective locations. FIG. 2 mainly includes the same structure and functionality as described above with reference to FIG. 1. Therefore, for the sake of brevity, the details of the system and the respective VRDs 120a, 120b will not be described again with respect to FIG. 2.
[0033] The configuration of FIG. 2 differs from that of FIG. 1 because the environment 200 includes a first VRD 120a and a second VRD 120b that can be utilized to present a shared VR environment to multiple users, and the VRDs 120a, 120b and users 150a, 150b can be located in separate physical environments. This configuration of FIG. 2 can be advantageous in various real-world applications where a simulation that can be experienced simultaneously by a group of users (possibly geographically distributed) is useful. For example, the configuration shown in FIG. 2 can be used to realize simulated force-on-force combat training, where other users participating in the combat simulation can be included in the 3D graphics projected by the remote VRDs.
[0034] 2, a first VRD 120a can be set up in a first physical location 260a, such as a room in a building where a first user 150a is physically present, and a second VRD 120b can be set up in a second physical location 260b, such as another room on a different floor of a building where a second user 150b is physically present. In this configuration, the multiple users 150a and 150b can be located remotely with respect to each other, while the 3D images and / or graphics presented on the respective projection surfaces 120a, 120b create a shared simulation.
[0035] In one embodiment, each of the VRDs 120a, 120b can be programmed to enable shared simulation. For example, a "shared simulation" feature implemented by the first VRD 120a can be enabled, which allows the first VRD 120a to recognize other remotely located VRDs that are enabled to participate in the shared simulation, e.g., the second VRD 120b. The enabled VRDs 120a, 120b can then be communicatively coupled, and these devices can transmit and / or receive data. In particular, location data corresponding to users 150a, 150b, which may be captured separately by the VRDs 120a, 120b located at their respective locations, can be communicated to the remote VRDs. For example, the first VRD 120a can capture location data of the user 150a at its particular physical location and then transmit the captured location data to a remotely located VRD, in this example the second VRD 120b. As a result, both VRDs 120a, 120b can obtain and analyze the positional data of each user 150a, 150a, and furthermore, these devices have the necessary information to create a shared VR environment that users 120a and 120b can experience simultaneously. FIG. 2 helps to show that the shared VR environment presented to each user 150a, 150b includes renderings of substantially the same 3D images. As shown, the 3D rendering projected on the projection surface 130a in the field of view of user 120a depicts the same virtual shooting range as projected on the projection surface 130b in the field of view of user 120b. However, VRDs 120a, 120b adjust their respective 3D renderings to include visual representations of other users participating in the shared simulation in separate physical environments. To illustrate this function, the 3D image projected on the projection surface 130a in the first position is specially rendered to simulate the presence of user 120b. As shown, user 120b is spatially modeled in the shared simulation with respect to his or her physical location in the field while interacting with the VR environment, for example, user 120b is standing to the right of the detection field in a second position 260b.In response, VRD 120a can render a 3D image imposed on projection surface 130a to show a representation of user 150b generally on the right side of the VR environment. Alternatively, VRD 120b can render a 3D image imposed on projection surface 130b to show a representation of user 120a generally on the left side of the VR environment (user 120a standing on the right side of the detection field at first position 260a). It should be understood that Figure 2 is one exemplary configuration of the disclosed system, and other configurations not shown in the figures can be utilized to support the shared simulation functionality in a manner that can be expanded and / or scaled to include various numbers of VRDs and users as disclosed herein.
[0036] Further details of the disclosed projected VR environment features are now described with reference to Figure 3, which illustrates an example process 300 in accordance with one embodiment of the systems and methods described herein. As seen in Figure 3, process 300 is illustrated as a series of executable operations in a machine-readable storage medium 306 that are executed by a hardware processor 304. Computing component 302 may be a VRD (shown in Figure 1) that implements VR projection and VR rendering capabilities, as previously described.
[0037] In operation 305, processing of position data related to the user may be performed. In some embodiments, the processing may be performed by one or more processors, such as the CPU of the VRD (shown in FIG. 1). The position data may represent the user's current position within a detection field. The detection field may be formed by multiple sensors with respect to a spatial region encompassing the separation distance between the projection surface and the user. The position data processed in operation 305 may be captured by multiple sensors implemented as cameras, for example.
[0038] Next, in operation 310, a computer-simulated environment using the processed position data may be rendered. In some embodiments, the rendering may be performed by one or more processors, such as the CPU of the VRD (shown in FIG. 1). The computer-simulated environment may be comprised of multi-dimensional (3D) images and / or graphics, such that the simulated environment is implemented as an immersive and interactive VR environment for the user. Furthermore, the rendering performed in operation 310 may include generating the computer-simulated environment in a manner that has depth perception relative to the user's current location. In this manner, as the user moves within the detection field, the computer-simulated environment may be rendered to mimic a simulated 3D representation with perceived depth. Furthermore, in operation 310, the computer-simulated environment may be rendered such that the viewpoint of the environment is relative to the user's current location within the detection field. That is, as the user moves within the detection field, the computer-simulated environment may be re-rendered to react according to the user's movements.
[0039] After the computer simulated environment has been rendered in previous operation 310, process 300 can proceed to operation 315, where the rendering can be projected onto a projection surface. According to an embodiment, the projection surface can be positioned at a distance from the user, for example, in a manner that supports room-scale VR, where the user is immersed in the VR environment within the volume of a room space.
[0040] Further details of the shared simulation feature are now described with reference to Figure 4, which illustrates an example process 400 according to one embodiment of the systems and methods described herein. As seen in Figure 4, the process 400 is illustrated as a series of executable operations in a machine-readable storage medium 406 that are executed by a hardware processor 404. The computing component 402 may be a VRD (shown in Figure 1) that implements VR projection and VR rendering capabilities, as previously described. In general, according to some embodiments, the process 400 effects the projection of a shared simulation, such as a VR environment that can be presented simultaneously to multiple remotely located users.
[0041] Process 400 may begin at operation 405. Operation 405 may include enabling two or more projector apparatuses. According to an embodiment, the projector apparatuses may be implemented as VRDs (shown in FIG. 2). Enabling the projector apparatuses for shared simulation may communicatively couple each apparatus to other apparatuses, and information such as location data may be communicatively coupled between the projector apparatuses. Thus, operation 405 may include receiving location data for each user in a separate physical environment representing each user's current location. The location data for each user in the separate physical environment may be captured by a respective projector apparatus using, for example, multiple sensors implemented in a VRD (shown in FIG. 1). Each of the two or more projector apparatuses may be located in a separate physical environment, each separate physical environment including a user within the field of view of the respective projector apparatus.
[0042] Step 410 can then include using two or more projector devices to create a collaborative computer simulated environment. The collaborative computer simulated environment can be generated to include 3D image renderings that include visual representations of each user in separate physical environments.
[0043] Thereafter, process 400 may proceed to operation 415. At operation 415, the shared computer simulated environment may be projected onto two or more projection surfaces, each of which may be located in a separate physical environment within the field of view of a respective user. Thus, other users remotely participating in the shared computer simulated environment may be modeled in the particular 3D rendering presented to the user.
[0044] 5 illustrates a block diagram of an exemplary computer system 500 that may implement the VR projection and rendering functionality described herein. For example, computer system 500 may be used to implement the CPU of a VRD, as shown in FIG. 1. Computer system 500 includes a bus 502 or other communication mechanism for communicating information, and one or more hardware processors 504 coupled to bus 502 for processing information. Hardware processor 504 may be, for example, one or more general-purpose microprocessors.
[0045] The computer system 500 also includes a main memory 508, such as a random access memory (RAM), cache, and / or other dynamic storage device coupled to the bus 502 for storing information and instructions executed by the processor 504. The main memory 508 may also be used to store temporary variables or other intermediate information during execution of instructions executed by the processor 504. Such instructions, when stored in a storage medium accessible to the processor 504, place the computer system 500 in a dedicated machine state customized to perform the operations specified in the instructions. The description of the functionality provided by the various instructions described herein is for illustrative purposes and is not intended to be limiting, as any instruction may provide more or less functionality than is described. For example, one or more instructions may be eliminated and some or all of its functionality may be provided by other instructions. As another example, the processor 504 may be programmed with one or more additional instructions that may perform some or all of the functionality attributed to one of the instructions herein.
[0046] The computer system 500 further includes a storage device 510. Various instructions described herein, including the advanced analytics demand forecasting techniques, may be stored in the storage device 510, which may include a read-only memory (ROM) and / or other static storage device coupled to the bus 502 for storing static information and instructions for the processor 504. A storage device 510, such as a magnetic disk, optical disk, or USB thumb drive (flash drive), is provided and coupled to the bus 502 for storing information and instructions. The storage device 510 may store computer program instructions (e.g., the instructions discussed above) that are executed by the processor 504, as well as data that may be manipulated by the processor 504. The storage device may comprise one or more non-transitory machine-readable storage media, such as a floppy disk, hard disk, optical disk, tape, or other physical storage medium for storing computer-executable instructions and / or data.
[0047] Computer system 500 may be coupled via bus 502 to a display 512, such as a liquid crystal display (LCD) (or touch screen), for displaying information to a computer user. An input device 514, including alphanumeric and other keys, is coupled to bus 502 for communicating information and command selections to processor 504. Another type of user input device is a cursor control device 516, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to processor 504 and for controlling cursor movement on display 512. In some embodiments, the same directional information and command selection as cursor control may be implemented by receiving touches on a touch screen without a cursor.
[0048] The computing system 500 may include a user interface module for implementing a GUI that may be stored in mass storage as executable software code executed by a computing device. This and other modules include, by way of example, software components, object-oriented software components, components such as class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.
[0049] In general, words such as "component," "engine," "system," "database," "data store," and the like, as used herein, may refer to logic embodied in hardware or firmware, or a collection of software instructions, possibly having entry and exit points, written in a programming language, such as, for example, Java, C, or C++. Software components may be compiled and linked into an executable program, installed in a dynamic link library, or written in an interpreted programming language, such as, for example, BASIC, Perl, or Python. It will be understood that software components may be callable from other components or from themselves, and / or in response to detected events or interrupts. Software components configured to run on a computing device may be provided on a computer-readable medium, such as a compact disc, digital video disc, flash drive, magnetic disk, or any other tangible medium, or as a digital download (which may have originally been stored in a compressed or installable format that requires installation, decompression, or decryption before execution). Such software code may be stored partially or completely on a memory device of the running computing device for execution by the computing device. The software instructions may be embedded in firmware, such as an EPROM. It will be further appreciated that the hardware components may be comprised of connected logic units, such as gates and flip-flops, and / or may be comprised of programmable units, such as programmable gate arrays or processors.
[0050] The computer system 500 may implement the techniques described herein using customized hardwired logic, one or more ASICs or FPGAs, firmware and / or program logic that in combination with the computer system renders or programs the computer system 500 as a special purpose machine. According to one embodiment, the techniques described herein are performed by the computer system 500 in response to the processor 504 executing one or more sequences of one or more instructions contained in the main memory 508. Such instructions may be read into the main memory 508 from another storage medium, such as the storage device 510. Execution of the sequences of instructions contained in the main memory 508 causes the processor 504 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.
[0051] As used herein, the term "non-transitory media" and similar terms refer to any medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such non-transitory media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 510. Volatile media include dynamic memory, such as main memory 508. Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid state drives, magnetic tape or other magnetic data storage media, CD-ROMs, other optical data storage media, any physical media with a pattern of holes, RAM, PROMs, and EPROMs, FLASH-EPROMs, NVRAMs, other memory chips or cartridges, and network versions thereof.
[0052] Non-transitory media are distinct from, but may be used in conjunction with, transmission media. Transmission media involves transferring information between non-transitory media. For example, transmission media include coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 502. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
[0053] Computer system 500 also includes a communication interface 518 coupled to bus 502. Network interface 518 provides a two-way data communication coupling to one or more network links connected to one or more local networks. For example, communication interface 518 may be an Integrated Services Digital Network (ISDN) card, a cable modem, a satellite modem, or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, network interface 518 may be a local area network (LAN) card to provide a data communication connection to a network, which may include, for example, any one or more of the Internet, an intranet, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a storage area network (SAN), a metropolitan area network (MAN), a wireless network, a cellular communication network, a public switched telephone network, and / or other networks. Wireless links may also be implemented. In any such implementation, network interface 518 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information. The nodes of the distributed model building system disclosed herein may be coupled to other participant nodes via the network described above.
[0054] A network link typically provides data communication through one or more networks to other data devices. For example, a network link may provide a connection through a local network to a host computer or to data equipment operated by an Internet Service Provider (ISP). The ISP provides data communication services through the worldwide packet data communication network now commonly referred to as the "Internet." Both the local network and the Internet use electrical, electromagnetic or optical signals that carry digital data streams. The signals through the various networks and the signals through the network link and communication interface 518, which carry the digital data to and from computer system 500, are exemplary forms of transmission media.
[0055] Computer system 500 can send messages and receive data, including program code, through the network(s), network link and communication interface 518. In the Internet example, a server might transmit a requested code for an application program through the Internet, an ISP, a local network and communication interface 518.
[0056] The received code may be executed by processor 504 as it is received, and / or stored in storage device 510, or other non-volatile storage for later execution.
[0057] The various processes, methods, operations and / or data flows illustrated in Figures 3 and 4 (and other diagrams) described in the previous section may be embodied in code components executed by one or more computer systems or computer processors configured in computer hardware and may be fully or partially automated. The processes and algorithms may be partially or wholly implemented in application specific circuits. The various features and processes described above may be used independently of each other or may be combined in various ways. Different combinations and subcombinations are intended to fall within the scope of the disclosure, and in some implementations, certain method or process blocks may be omitted. The methods and processes described herein are also not limited to any particular order, and the blocks or states associated therewith may be performed in other suitable orders, or may be performed in parallel, or may be performed in some other manner. Blocks or states may be added or removed from the disclosed exemplary embodiments. The execution of certain operations or processes may be distributed between computer systems or computer processors, and may be spread across multiple machines as well as being present in a single machine.
[0058] As used herein, a circuit may be implemented using any form of hardware, software, or a combination thereof. For example, one or more processors, controllers, ASICs, PLAs, PALs, CPLDs, FPGAs, logic components, software routines, or other mechanisms may be implemented to constitute a circuit. In implementation, the various circuits described herein may be implemented as separate circuits, or the described functions and features may be shared in part or in whole between one or more circuits. Although various features or elements of functionality may be described individually or claimed as separate circuits, these features or functionality may be shared between one or more common circuits, and such description should not require or imply that separate circuits are required to implement such features or functionality. When a circuit is implemented in whole or in part using software, such software may be implemented to operate in a computing or processing system capable of performing the functionality described with respect thereto, such as computer system 500.
[0059] Although various embodiments of the disclosed technology have been described above, it should be understood that they are presented by way of example only and not limitation. Similarly, various figures may depict exemplary architectures or other configurations of the disclosed technology, which is done to aid in understanding the features and functionality that may be included in the disclosed technology. The disclosed technology is not limited to the exemplary architectures or configurations depicted, but various alternative architectures and configurations may be used to implement the desired features. Indeed, it will be apparent to one skilled in the art how alternative functional, logical, or physical divisions and configurations may be implemented to implement the desired features of the technology disclosed herein. Also, many different component module names other than those depicted herein may be applied to the various divisions. Furthermore, with respect to flow diagrams, operational descriptions, and method claims, the order of steps presented herein does not mandate that various embodiments implement the recited functionality in the same order unless the context dictates otherwise.
[0060] While the disclosed technology has been described above with reference to various exemplary embodiments and implementations, it should be understood that the various features, aspects and functionality described in one or more of the respective embodiments are not limited in their applicability to the particular embodiment for which they are described, but instead can be applied alone or in various combinations to one or more of the other embodiments of the disclosed technology, regardless of whether such an embodiment is described and whether such features are presented as part of a described embodiment. Thus, the breadth and scope of the technology disclosed herein should not be limited by any of the above-described exemplary embodiments.
[0061] As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. Moreover, descriptions of resources, acts, or structures in the singular should not be interpreted as excluding the plural. In particular, conditional language such as "can," "could," "could," or "may" is generally intended to convey that certain embodiments include certain features, elements, and / or steps, and other embodiments do not, unless expressly stated or understood otherwise within the context in which it is used.
[0062] Terms and phrases used herein, and variations thereof, unless expressly stated otherwise, should be construed as open ended rather than limiting. Adjectives such as "conventional," "traditional," "normal," "standard," "known," and terms of similar meaning should not be construed to limit the described items to those available during a given period or at a given time, but should instead be construed to encompass conventional, traditional, ordinary, or standard technologies that may be available or known at any time now or in the future. In some cases, the presence of scope-expanding words and phrases, such as "one or more," "at least," "but not limited to," or other similar phrases, should not be construed to imply that a narrower case is intended or required in the absence of such a scope-expanding phrase.
[0063] In some cases, the presence of scope-expanding words and phrases such as "one or more," "at least," "but not limited to," or other similar phrases should not be construed to imply that a narrower case is intended or required in the absence of such scope-expanding phrase. The use of the term "module" does not imply that the components or functionality described or claimed as part of a module are all configured within a common package. Indeed, any or all of the various components of a module, whether control logic or other components, may be combined into a single package or maintained separately, and may even be distributed into multiple groups or packages, or at multiple locations.
[0064] Moreover, various embodiments described herein are described with reference to exemplary block diagrams, flow charts, and other illustrations. As will be apparent to one of ordinary skill in the art upon reading this specification, the illustrated embodiments and their various alternatives can be implemented without being limited to the illustrated examples. For example, the block diagrams and their accompanying description should not be construed as mandating a particular architecture or configuration.
Claims
1. a projector for projecting an image onto a projection surface, the projection surface being positioned at a distance away from a user; a plurality of sensors forming a detection field with respect to a spatial region encompassing the separation distance between the projection surface and the user, the plurality of sensors capturing position data of the user representative of a current position of the user within the detection field; at least one processor for rendering a computer simulated environment as the projected image, the at least one processor processing the captured position data for the user such that the computer simulated environment is generated with a perspective relative to the current position of the user within the detection field; A system comprising:
2. 10. The system of claim 1, wherein the plurality of sensors comprises at least one of a camera, an ultrasonic distance sensor, a radar sensor, and a light detection and ranging (LIDAR) sensor.
3. The system of claim 2 , wherein the camera comprises at least one of a front-facing camera and a rear-facing camera.
4. The system of claim 3 , wherein the captured position data of the user includes at least one of a position of the user's head, an orientation of the user, and a movement of the user.
5. 5. The system of claim 4, wherein the at least one processor processes the captured position data of the user such that the computer simulated environment includes a three-dimensional (3D) image rendering of an environment having depth perception relative to the position of the head of the user.
6. 6. The system of claim 5, wherein the at least one processor processes the captured position data of the user such that the 3D image rendering is generated using predictive data calculated based on currently captured position data representing the user's current movement within the detection field and historically captured position data representing the user's past movement within the detection field.
7. The system of claim 1 , wherein the at least one processor processes additional data from a plurality of remote sensors disposed at a remote location and capturing images representative of the remote location.
8. The system of claim 7 , wherein the computer simulation environment includes a three-dimensional (3D) image rendering that simulates the remote environment.
9. The system of claim 1 , wherein the projection surface comprises at least one of a screen, a wall, or multiple screens and multiple walls.
10. The system of claim 1 , wherein the projector comprises at least one projection port and optics for optically projecting an image.
11. 1. A method comprising: activating two or more projector apparatuses, each of the two or more projector apparatuses being located in a distinct physical environment, each of the distinct physical environments including a user within view of the respective projector apparatus; creating a shared computer simulated environment using the two or more projector devices, the shared computer simulated environment including three-dimensional (3D) image renderings including a visual representation of each of the users in the separate physical environments; projecting the shared computer-simulated environment onto two or more projection surfaces, each of the two or more projection surfaces being respectively positioned within the separate physical environment within a field of view of the user; A method comprising:
12. 12. The method of claim 11, further comprising the step of receiving position data for each of the users in the separate physical environments representing a current location of each of the users, the position data for each of the users in the separate physical environments being captured by the respective projector apparatus.
13. 13. The method of claim 12, wherein creating the shared computer simulation environment includes positioning the visual representation of each of the users within the 3D image relative to the current location corresponding to each of the users based on the received location data.
14. The method of claim 13 , wherein projecting the shared computer-simulated environment includes projecting a perspective of the shared computer-simulated environment for a particular user from the users in the separate physical environment.
15. 15. The method of claim 14, wherein projecting a viewpoint of the shared computer simulated environment includes using the position data corresponding to the particular user to render the shared computer simulated environment from a viewpoint relative to the current location of the particular user.
16. A non-transitory machine-readable storage medium comprising instructions executable by at least one processor, the instructions causing the at least one processor to processing location data associated with a user and representative of a current location of said user within a detection field; rendering a computer simulated environment using the processed position data, the computer simulated environment being projected onto a projection surface positioned at a distance from the user and generated with a perspective relative to the current position of the user within the detection field; A non-transitory machine-readable storage medium programmed to
17. The non-transitory machine-readable storage medium of claim 16 , wherein the detection field is formed by a plurality of sensors for a spatial region encompassing the separation distance between the projection surface and the user.
18. 20. The non-transitory machine-readable storage medium of claim 17, wherein the location data of the user is captured by the plurality of sensors.
19. The at least one processor rendering the computer simulated environment to include a three-dimensional (3D) image rendering of the environment having depth perception relative to the current position of the user; 20. The non-transitory machine-readable storage medium of claim 17, further programmed to:
20. The at least one processor calculating predictive data associated with the user based on currently captured location data representative of a current location of the user within the field of detection and historically captured location data representative of past locations of the user within the field of detection; rendering the computer simulated environment to include the 3D image rendering of the environment at a viewpoint relative to a predicted position of the user within the field of detection based on the predicted data associated with the user.
20. The non-transitory machine-readable storage medium of claim 19, further programmed to:
Citation Information
Patent Citations
Video viewing device
JP1996336166A
Interactive display device
JP1998208075A
Image processing unit, depth image measuring device, composite reality presenting system, image processing method, depth image measuring method, composite reality presenting method and storage medium for program
JP1999331874A
High efficiency laser modulation
JP2016526177A
Information processing apparatus, information processing method, and program
JP2018085571A
