Dynamic Browser Stage

Abstract

The present disclosure relates to systems and methods for displaying 3D content in a three-dimensional (3D) spatial environment. The systems and methods may include: receiving a request to display 3D content of a specific size at a location within a 3D spatial environment from a network domain, identifying whether the placement is within an authorized portion of the 3D spatial environment, expanding the authorized portion of the 3D spatial environment to display the 3D content based on the user's authorization to resize the authorized portion, and displaying the 3D content.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for facilitating an interactive virtual or augmented reality environment for one or more users. Background Art

[0002] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality", "augmented reality", or "mixed reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that they appear to be real or they can be perceived as real. A virtual reality or "VR" scenario typically involves the presentation of digital or virtual image information and is opaque to other actual real-world visual inputs; an augmented reality or "AR" scenario typically involves digital or virtual image information being presented as an enhancement to the visualization of the actual world around the user; and mixed reality or "MR" involves merging the real world and the virtual world to create a new environment in which physical and virtual objects coexist and interact in real time. In fact, the human visual perception system is very complex, and VR, AR, or MR technologies that produce a rich presentation that promotes a comfortable natural sense of virtual image elements among other virtual or real-world visual elements are challenging. The systems and methods disclosed herein address various challenges associated with VR, AR, and MR technologies. Summary of the Invention

[0003] Embodiments of the present disclosure relate to devices, systems, and methods for facilitating virtual or augmented reality interactions for one or more users.

[0004] Further details of the features, objects, and advantages of the present disclosure are described below in the detailed description, the drawings, and the claims. The foregoing general description and the following detailed description are exemplary and explanatory and are not intended to limit the scope of the present disclosure.

[0005] In some configurations, a display system for displaying virtual content in a three-dimensional (3D) spatial environment may include: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and circuitry communicatively coupled to the head-mounted display. The circuitry may be configured to: receive a request to access network-based 3D content; identify parameters associated with the network-based 3D content, including at least one of the following: a location in the user's 3D spatial environment at which to display the network-based 3D content, an orientation of the 3D content, or a size of the 3D content; based on the parameters, determine whether the 3D content can be displayed in an authorized portion of the user's 3D spatial environment; and in response to determining that the 3D content cannot be displayed in the authorized portion of the 3D spatial environment, adjust the size of the authorized portion to allow the 3D content to be displayed in the resized authorized portion.

[0006] In some configurations, a display system for displaying virtual content in a three-dimensional (3D) spatial environment may include: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and circuitry communicatively coupled to the head-mounted display. The circuitry may be configured to: receive a request to access content; display the content in an authorized portion of the user's 3D spatial environment in a first orientation; receive a request to display the content in a second orientation in the user's 3D spatial environment; determine whether the content can be displayed in the authorized portion of the user's 3D spatial environment in the second orientation; and in response to determining that the content cannot be displayed in the authorized portion of the 3D spatial environment in the second orientation, adjust the size of the authorized portion to allow the content to be displayed in the resized authorized portion in the second orientation.

[0007] In some configurations, a display system for displaying virtual content in a three-dimensional (3D) spatial environment may include: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and circuitry communicatively coupled to the head-mounted display. The circuitry may be configured to: receive a request to access 3D content; identify parameters associated with the 3D content, including at least one of the following: a location in the user's 3D spatial environment at which to display the 3D content, an orientation of the 3D content, and a size of the 3D content; based on the parameters, determine whether the 3D content can be displayed in an authorized portion of the user's 3D spatial environment; and in response to determining that the 3D content cannot be displayed in the authorized portion of the 3D spatial environment, display a representation of the 3D content at a location in the 3D spatial environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In all of the figures, reference numerals are reused to indicate corresponding relationships between the elements being referenced. The following figures and associated descriptions are provided to illustrate embodiments of the present disclosure and do not limit the scope of the claims.

[0009] The figures illustrate the design and utility of various embodiments of the present disclosure. It should be noted that the figures are not drawn to scale and elements of similar structure or function are denoted by the same reference numeral throughout the figures. To better understand how to obtain the described and other advantages and objects of the various embodiments of the present disclosure, a more detailed description of the present disclosure briefly described above will be provided with reference to specific embodiments of the present disclosure shown in the figures. Understanding that these figures only depict typical embodiments of the present disclosure and should not be considered as limiting its scope, the present disclosure will be described and explained with additional specificity and detail by using the figures, in which:

[0010] Figure 1 An illustration depicting a mixed reality scene having certain virtual reality objects and certain physical objects viewed by a person.

[0011] Figure 2An example of a wearable system that can implement an example waypoint system is schematically shown.

[0012] Figure 3 Aspects of a method for simulating a three-dimensional image using multiple depth planes are schematically shown.

[0013] Figure 4 An example of a waveguide stack for outputting image information to a user is schematically shown.

[0014] Figure 5 An example outgoing light beam that can be output by a waveguide is shown.

[0015] Figure 6 A schematic diagram showing an optical system for generating a multi-focus volumetric display, image, or light field, which includes a waveguide device, an optical coupler subsystem that optically couples light to or from the waveguide device, and a control subsystem.

[0016] Figure 7 An augmented reality environment for deconstructing 2D content to be displayed in a user's 3D environment according to some embodiments is shown.

[0017] Figure 8 An example mapping of elements of 2D content to a user's 3D environment according to some embodiments is shown.

[0018] Figure 9A An example browser tile displayed in a bounded volume of 3D space is shown.

[0019] Figure 9B An example browser tile with 3D web content in a bounded volume of 3D space is shown.

[0020] Figure 10A An example situation where the 3D web content is larger than the bounded volume of 3D space is shown.

[0021] Figure 10B An example situation where the center of the 3D web content does not intersect with a fixed volume of 3D space is shown.

[0022] Figure 11 An example content stage or volume is shown.

[0023] Figure 12A1 and Figure 12A2 An example of placing 3D content in a 3D volume is shown.

[0024] Figure 12B1 and Figure 12B2 An example of resizing a 3D volume using a fixed range is shown.

[0025] Figure 13AIt is a flowchart of an exemplary volume resizing process.

[0026] Figure 13B An exemplary volume resizing is shown.

[0027] Figure 14A It is a flowchart of an exemplary page rotation process.

[0028] Figure 14B1 and Figure 14B2 An exemplary page rotation is shown.

[0029] Figure 15A It is a flowchart of an exemplary content processing process.

[0030] Figure 15B Two examples of the content processing process are shown. Detailed Description

[0031] A. Introduction

[0032] Virtual and augmented reality environments are generated by a computer part using data that describes the environment. This data can describe, for example, various objects that a user can perceive and interact with. Examples of such objects include objects that are rendered and displayed for the user to see, audio that is played for the user to hear, and tactile feedback for the user to feel. The user can perceive and interact with virtual and augmented reality environments through various visual, auditory, and tactile means.

[0033] Virtual or augmented reality (AR) systems can be useful for many applications, covering fields such as scientific visualization, medical and military training, engineering design and prototyping, teleoperation and tele-presence, and personal entertainment. In contrast to virtual reality, augmented reality includes one or more virtual objects related to real objects in the physical world. Such experiences greatly enhance the user experience and usability of adopting augmented reality systems, and also open the door to various applications that allow users to experience real objects and virtual objects simultaneously.

[0034] However, there are significant challenges in providing such systems. To provide users with a realistic augmented reality experience, an AR system should always know the user's physical environment in order to correctly correlate the positions of virtual objects with real objects. Further, the AR system should correctly know how to position virtual objects relative to the user's head, body, etc. This requires always having a broad knowledge of the user's position relative to the world. In addition, these functions should be advantageously performed so that costs (such as energy consumption, etc.) remain low while maintaining speed and performance.

[0035] Therefore, there is a need to improve systems to provide users with a realistic augmented reality experience.

[0036] B. Example of 3D display of a wearable system

[0037] A wearable system (also referred to herein as an augmented reality (AR) system) can be configured to present 2D or 3D virtual images to a user. The images can be still images, frames of a video, or a video, or a combination, etc. At least a portion of the wearable system can be implemented on a wearable device, which can present a VR, AR, or MR environment, alone or in combination, for user interaction. The wearable device can be a head-mounted device (HMD), which can be interchangeably used as an AR device (ARD). Further, for the purposes of the present disclosure, the terms “AR” and “MR” are used interchangeably.

[0038] Figure 1 An illustration depicting a mixed reality scene with certain virtual reality objects and certain physical objects viewed by a person is shown. In Figure 1 , an MR scene 100 is depicted, where a user of MR technology sees a real-world park-like setting 110 featuring people, trees, and buildings in the background, as well as a concrete platform 120. In addition to these items, the user of MR technology also perceives that he “sees” a robotic statue 130 standing on the real-world platform 120 and a flying cartoon-like avatar character 140, which appears to be an anthropomorphized bumblebee, even though these elements do not exist in the real world.

[0039] To enable a 3D display to produce a realistic sense of depth, and more specifically, a simulated sense of surface depth, it may be desirable for each point in the field of view of the display to generate an accommodation response corresponding to its virtual depth. If the accommodation response to a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues of convergence and stereopsis, the human eye can experience an accommodation conflict, resulting in unstable imaging, harmful eye fatigue, headaches, and, in the absence of accommodation information, an almost complete lack of surface depth.

[0040] VR, AR, and MR experiences can be provided by a display system having a display that provides images corresponding to multiple depth planes to a viewer. The images can be different for each depth plane (e.g., providing slightly different renditions of a scene or object), and can be separately focused by the viewer's eyes, thus contributing to providing depth cues to the user based on the accommodation of the eyes required to focus on different image features located on different depth planes or based on observing different image features on different depth planes being out of focus. As discussed elsewhere herein, such depth cues provide a credible perception of depth.

[0041] Figure 2An example of a wearable system 200 that can be configured to provide AR / VR / MR scenarios and can include the example waypoint system described herein is shown. The wearable system 200 may also be referred to as an AR system 200. The wearable system 200 includes a display 220 and various mechanical and electronic modules and systems that support the functions of the display 220. The display 220 may be coupled to a frame 230 that can be worn by a user, wearer, or viewer 210. The display 220 may be positioned in front of the eyes of the user 210. The display 220 may present AR / VR / MR content to the user. The display 220 may include a head-mounted display worn on the user's head. In some embodiments, a speaker 240 is coupled to the frame 230 and is located near the user's ear canal (in some embodiments, another speaker, not shown, may be located near the user's other ear canal to provide stereo / plastic sound control). The display 220 may include an audio sensor (e.g., a microphone) 232 for detecting an audio stream from the environment and capturing ambient sound. One or more other audio sensors, not shown, may be positioned to provide stereo sound reception. Stereo sound reception may be used to determine the location of the sound source. The wearable system 200 may perform sound or speech recognition on the audio stream.

[0042] The wearable system 200 may include an outward-facing imaging system 464 (shown in Figure 4 ), which observes the world in the environment around the user. The wearable system 200 may also include an inward-facing imaging system 462 (shown in Figure 4 ), which can be used to track the eye movements of the user. The inward-facing imaging system may track the movement of one eye or both eyes. The inward-facing imaging system 462 may be attached to the frame 230 and may be in electrical communication with a processing module 260 or 270, which may process the image information acquired by the inward-facing imaging system to determine, for example, the pupil diameter or orientation of the eyes of the user 210, eye movement, or eye pose.

[0043] As an example, the wearable system 200 may use the outward-facing imaging system 464 or the inward-facing imaging system 462 to acquire an image of the user's pose. The image may be a still image, a frame of a video, or a video.

[0044] The display 220 may be operably coupled to a local data processing module 260, for example, by a lead or a wireless connection 250. The local data processing module 260 may be installed in various configurations, for example, fixedly attached to the frame 230, fixedly attached to a helmet or hat worn by the user, embedded in headphones, or otherwise detachably attached to the user 210 (e.g., in a backpack configuration, in a belt-connected configuration).

[0045] The local processing and data module 260 may include a hardware processor and a digital memory (such as non-volatile memory (e.g., flash memory)), both of which may be used to assist in processing, caching, and storing data. The data may include: a) data captured from sensors (which may be operatively coupled to the frame 230 or otherwise attached to the user 210, for example), such as image capture devices (e.g., cameras in an inward-facing imaging system or an outward-facing imaging system), audio sensors (e.g., microphones), inertial measurement units (IMUs), accelerometers, compasses, global positioning system (GPS) units, radio devices, or gyroscopes; or b) data obtained or processed using the remote processing module 270 or the remote data repository 280, which may be used to be passed to the display 220 after such processing or retrieval. The local processing and data module 260 may be operatively coupled to the remote processing module 270 or the remote data repository 280 via a wired or wireless communication link, such as communication links 262 or 264, so that these remote modules can be used as resources for the local processing and data module 260. Additionally, the remote processing module 270 and the remote data repository 280 may be operatively coupled to each other.

[0046] The remote processing module 270 may include one or more processors configured to analyze and process data or image information. The remote data repository 280 may include a digital data storage facility, which may be available via the Internet or other network configurations in a “cloud” resource configuration. Data and computations may be stored and performed in the local processing and data module, which allows for the full autonomous use of the remote modules.

[0047] The human visual system is complex and poses challenges to providing a realistic sense of depth. Without being limited by theory, it is believed that a viewer of an object may perceive the object as three-dimensional due to the vergence-accommodation combination. The vergence movement of the two eyes relative to each other (e.g., the rolling movement of the pupils towards or away from each other to converge the lines of sight of the eyes on an object) is closely associated with the focusing (or “accommodation”) of the lenses of the eyes. Under normal circumstances, changing the focus of the lenses of the eyes or accommodating the eyes to change the focus from one object to another object at a different distance will automatically result in a matching change in vergence to the same distance according to a relationship known as the “accommodation-vergence reflex”. Similarly, under normal circumstances, a change in vergence will trigger a matching change in accommodation. A display system that provides a better match between accommodation and vergence can form a more realistic and comfortable three-dimensional image simulation.

[0048] Figure 3 Aspects of a method for simulating a three-dimensional image using multiple depth planes are shown. Refer to Figure 3, objects at different distances 302 and 304 from the eyes along the z-axis are accommodated by the eyes 302 and 304 such that these objects are in focus. The eyes 302 and 304 assume a specific accommodation state to align the focus with objects at different distances along the z-axis. Thus, a specific accommodation state can be considered to be associated with a specific depth plane in the depth plane 306, which has an associated focal length such that when the eyes are in the accommodation state for that depth plane, objects or portions of objects in the specific depth plane are in focus. Three-dimensional images can be simulated by providing different presentations of images to each eye 302, 304 and also by providing different presentations of images corresponding to each depth plane. Although shown as separate for clarity of illustration, it should be understood that the fields of view of the eyes 302 and 304 can overlap, for example, as the distance along the z-axis increases. Additionally, although shown as flat for ease of illustration, it will be understood that the profile of the depth plane can be curved in physical space such that when the eyes are in a specific accommodation state, all features in the depth plane are in focus. Without being limited by theory, it is believed that the human eye can typically interpret a limited number of depth planes to provide depth perception. Thus, by providing different presentations of images corresponding to each of these limited number of depth planes to the eyes, a highly believable simulated depth perception can be achieved.

[0049] C. Waveguide stack assembly

[0050] Figure 4 An example of a waveguide stack for outputting image information to a user is shown. The wearable system 400 includes a waveguide stack or stacked waveguide assembly 480, which can be used to provide three-dimensional perception to the eyes / brain using a plurality of waveguides 432b, 434b, 436b, 438b, 440b. The wearable system 400 can correspond to Figure 2 the wearable system 200, where Figure 4 some parts of the wearable system 200 are shown schematically in more detail. For example, the waveguide assembly 480 can be integrated into Figure 2 the display 220.

[0051] Continuing to refer to Figure 4 , the waveguide assembly 480 can also include a plurality of features 458, 456, 454, 452 between the waveguides. The features 458, 456, 454, 452 can be lenses. The features 458, 456, 454, 452 can not be lenses. Instead, they can be merely spacers (e.g., cladding layers or structures for forming air gaps).

[0052] Waveguides 432b, 434b, 436b, 438b, 440b or multiple lenses 458, 456, 454, 452 may be configured to send image information to the eye with various levels of wavefront curvature or light ray divergence. Each waveguide stage may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. Image injection devices 420, 422, 424, 426, 428 may be used to inject image information into waveguides 440b, 438b, 436b, 434b, 432b, and each waveguide may be configured to distribute incident light over each respective waveguide for output toward the eye 410. Light leaves the output surface of image injection devices 420, 422, 424, 426, 428 and is injected into the corresponding input edges of waveguides 440b, 438b, 436b, 434b, 432b. A single light beam (e.g., a collimated beam) may be injected into each waveguide to output the entire field of replicated collimated beams that point toward the eye 410 at a particular angle (and amount of divergence) corresponding to the depth plane associated with a particular waveguide.

[0053] Image injection devices 420, 422, 424, 426, 428 may be discrete displays, each of which generates image information for injection into a corresponding one of waveguides 440b, 438b, 436b, 434b, 432b. Additionally or alternatively, image injection devices 420, 422, 424, 426, 428 may be output ports of a single multiplexed display that may convey image information to each of image injection devices 420, 422, 424, 426, 428 via one or more light pipes (such as fiber optic cables), for example.

[0054] Controller 460 controls the operation of the stacked waveguide assembly 480 and image injection devices 420, 422, 424, 426, 428. Controller 460 includes programming (e.g., instructions in a non-transitory computer-readable medium) that regulates the timing and provision of image information to waveguides 440b, 438b, 436b, 434b, 432b. Controller 460 may be a single integrated device or a distributed system connected via wired or wireless communication channels. In some embodiments, controller 460 may be part of processing module 260 or 270 (shown in Figure 2 ).

[0055] Waveguides 440b, 438b, 436b, 434b, 432b can be configured to propagate light within each respective waveguide by total internal reflection (TIR). Each of waveguides 440b, 438b, 436b, 434b, 432b can be planar or have another shape (e.g., curved), having a major top surface and a major bottom surface and an edge extending between these major top and bottom surfaces. In the illustrated configuration, each of waveguides 440b, 438b, 436b, 434b, 432b can include light extraction optical elements 440a, 438a, 436a, 434a, 432a that are configured to extract light from the waveguide by redirecting light propagating within each respective waveguide out of the waveguide to output image information to the eye 410. The extracted light can also be referred to as coupled-out light, and the light extraction optical elements can also be referred to as coupled-out optical elements. The extracted light beam can be output from the waveguide at a location where the light propagating in the waveguide impinges on the light redirecting element. The light extraction optical elements (440a, 438a, 436a, 434a, 432a) can be, for example, reflective or diffractive optical features. Although shown for ease of description and clarity of drawing as being disposed on the major bottom surface of waveguides 440b, 438b, 436b, 434b, 432b, the light extraction optical elements 440a, 438a, 436a, 434a, 432a can be disposed on the major top surface or the major bottom surface, or can be disposed directly within the volume of waveguides 440b, 438b, 436b, 434b, 432b. The light extraction optical elements 440a, 438a, 436a, 434a, 432a can be formed in a material layer attached to a transparent substrate to form waveguides 440b, 438b, 436b, 434b, 432b. Waveguides 440b, 438b, 436b, 434b, 432b can be a monolithic material, and the light extraction optical elements 440a, 438a, 436a, 434a, 432a can be formed on or within the surface of that material.

[0056] Continue to refer to Figure 4, as discussed herein, each waveguide 440b, 438b, 436b, 434b, 432b can be configured to output light to form an image corresponding to a particular depth plane. For example, the waveguide 432b closest to the eye can be configured to deliver collimated light injected into such waveguide 432b to the eye 410. This collimated light can represent an optically infinite focal plane. The next up waveguide 434b can be configured to emit collimated light that passes through a first lens 452 (e.g., a negative lens) before reaching the eye 410. The first lens 452 can be configured to produce a slightly convex wavefront curvature such that the eye / brain interprets the light from this next up waveguide 434b as coming from a first focal plane that is closer in towards the eye 410 from the optically infinite. Similarly, the third up waveguide 436b passes its output light through the first lens 452 and a second lens 454 before reaching the eye 410. The combined optical power of the first lens 452 and the second lens 454 can be configured to produce another increment of wavefront curvature such that the eye / brain interprets the light from the third waveguide 436b as coming from a second focal plane that is closer in towards the person from the optically infinite, rather than the light from the next up waveguide 434b.

[0057] Other waveguide layers (e.g., waveguides 438b, 440b) and lenses (e.g., lenses 456, 458) are similarly configured, with the topmost waveguide 440b in the stack sending its output through all of the lenses between it and the eye for the aggregate optical power representing the focal plane closest to the person. To compensate for the stack of lenses 458, 456, 454, 452 when viewing / interpreting light from the world 470 on the other side of the stacked waveguide assembly 480, a compensating lens layer 430 can be provided at the top of the stack to compensate for the aggregate refractive power of the underlying lens stack 458, 456, 454, 452. Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairs. The light extraction optics of the waveguides and the focusing aspects of the lenses can both be static (e.g., non-dynamic, or electroactive). Additionally or alternatively, either or both of them can be dynamic using electroactive features.

[0058] Continuing to refer to Figure 4, the light extraction optical elements 440a, 438a, 436a, 434a, 432a can be configured to redirect light out of their respective waveguides for a particular depth plane associated with the waveguide and output the light with an appropriate amount of divergence or collimation. As a result, waveguides with different associated depth planes can have different light extraction optical element configurations, and the light extraction optical elements output light with different amounts of divergence according to the associated depth plane. As discussed herein, the light extraction optical elements 440a, 438a, 436a, 434a, 432a can be volume features or surface features that can be configured to output light at a particular angle. For example, the light extraction optical elements 440a, 438a, 436a, 434a, 432a can be volume holograms, surface holograms, or diffraction gratings. The light extraction optical elements, such as diffraction gratings, are described in U.S. Patent Publication No. 2015 / 0178939, published on June 25, 2015, the entire content of which is incorporated herein by reference.

[0059] In some embodiments, the light extraction optical elements 440a, 438a, 436a, 434a, 432a are diffraction features that form a diffraction pattern or are “diffractive optical elements” (also referred to herein as “DOEs”). Preferably, the DOE has a relatively low diffraction efficiency such that only a portion of the light of the light beam is deflected towards the eye 410 with each intersection with the DOE, while the remainder continues to travel through the waveguide via total internal reflection. Thus, the light carrying the image information can be split into multiple related outgoing light beams that exit the waveguide at multiple locations, and for this particular collimated light beam bouncing within the waveguide, the result is a fairly uniform outgoing emission pattern towards the eye 304.

[0060] One or more DOEs can be switched between an “on” state in which they actively diffract and an “off” state in which they do not significantly diffract. For example, a switchable DOE can include a polymer dispersed liquid crystal layer, where microdroplets include a diffraction pattern in a host medium, and the refractive index of the microdroplets can be switched to be substantially matched to the refractive index of the host material (in which case the pattern does not significantly diffract the incident light), or the microdroplets can be switched to a refractive index that does not match the refractive index of the host medium (in which case the pattern actively diffracts the incident light).

[0061] The number and distribution of depth planes or depth of field can vary dynamically based on the pupil size or orientation of the viewer's eye. The depth of field can vary inversely with the pupil size of the viewer's eye. As a result, as the size of the pupil of the viewer's eye decreases, the depth of field increases such that a plane that was not distinguishable due to its position beyond the depth of focus of the eye can become distinguishable and appears more focused and commensurate with the increase in depth of field as the pupil size decreases. Similarly, as the pupil size decreases, the number of spaced-apart depth planes used to present different images to the viewer can be reduced. For example, without adjusting the accommodation of the eye to move from one depth plane to another, the viewer may not be able to clearly perceive the details of both a first depth plane and a second depth plane at one pupil size. However, both of these depth planes can be sufficiently in focus for the user simultaneously at another pupil size without changing the accommodation.

[0062] The display system can change the number of waveguides receiving image information based on a determination of the pupil size or orientation, or upon receiving an electrical signal indicating a particular pupil size or orientation. For example, if the user's eye cannot distinguish between two depth planes associated with two waveguides, the controller 460 (which can be an embodiment of the local processing and data module 260) can be configured or programmed to stop providing image information to one of these waveguides. Advantageously, this can reduce the processing burden on the system, thereby increasing the responsiveness of the system. In embodiments where the DOE for the waveguide can be switched between an on and off state, the DOE can be switched to the off state when the waveguide does receive image information.

[0063] It may be desirable for the exit beam to satisfy the condition that its diameter is less than the diameter of the viewer's eye. However, given the variability in the size of the viewer's pupil, satisfying this condition can be challenging. By changing the size of the exit beam in response to determining the size of the viewer's pupil, this condition can be satisfied over a wide range of pupil sizes. For example, as the pupil size decreases, the size of the exit beam can also decrease. A variable aperture can be used to change the size of the exit beam.

[0064] The wearable system 400 may include an outward-facing imaging system 464 (e.g., a digital camera) that images a portion of the world 470. This portion of the world 470 may be referred to as the field of view (FOV) of the world camera, and the imaging system 464 is sometimes referred to as the FOV camera. The FOV of the world camera may be the same as or different from the FOV of the viewer 210, which includes the portion of the world 470 that the viewer 210 perceives at a given time. For example, in some cases, the FOV of the world camera may be larger than the FOV of the viewer 210 of the wearable system 400. The entire area available for viewing or imaging by the viewer may be referred to as the field of regard (FOR). The FOR may include 4π steradians of solid angle around the wearable system 400, since the wearer may move his body, head, or eyes to perceive substantially any direction in space. In other contexts, the movement of the wearer may be more restricted, and thus, the FOR of the wearer may subtend a smaller solid angle. Images obtained from the outward-facing imaging system 464 may be used to track gestures made by the user (e.g., hand or finger gestures), detect objects in the world 470 in front of the user, etc.

[0065] The wearable system 400 may include an audio sensor 232 (e.g., a microphone) to capture ambient sound. As described above, one or more other audio sensors may be positioned to provide stereo sound reception useful for determining the location of a voice source. As another example, the audio sensor 232 may include a directional microphone, which may also provide useful directional information about where the audio source is located. The wearable system 400 may use information from both the outward-facing imaging system 464 and the audio sensor 230 to locate a voice source or determine the active speaker, etc., at a particular moment. For example, the wearable system 400 may use voice recognition alone or in combination with a reflected image of the speaker (e.g., as seen in a mirror) to determine the identity of the speaker. As another example, the wearable system 400 may determine the location of the speaker in the environment based on the sound obtained from the directional microphone. The wearable system 400 may parse the sound from the location of the speaker with a voice recognition algorithm to determine the content of the speech and use sound recognition techniques to determine the identity of the speaker (e.g., name or other demographic information).

[0066] The wearable system 400 may also include an inward-facing imaging system 466 (e.g., a digital camera) that observes the user's movements, such as eye movements and facial movements. The inward-facing imaging system 466 may be used to capture an image of the eye 410 to determine the size or orientation of the pupil of the eye 304. The inward-facing imaging system 466 may be used to obtain an image for determining the direction in which the user is looking (e.g., eye pose) or for biometric identification of the user (e.g., via iris recognition). For each eye, at least one camera may be utilized to independently and separately determine the pupil size or eye pose of each eye, thereby allowing the presentation of image information to each eye to be dynamically customized for that eye. The pupil diameter or orientation of only one eye 410 may be determined (e.g., using a single camera per pair of eyes), and this pupil diameter or orientation may be considered similar for both eyes of the user. The images obtained by the inward-facing imaging system 466 may be analyzed to determine the user's eye pose or mood, and the wearable system 400 may use this eye pose or mood to decide which audio or visual content should be presented to the user. Additionally or alternatively, the wearable system 400 may use sensors (such as an IMU, accelerometer, gyroscope, etc.) to determine the head pose (e.g., head position or head orientation).

[0067] The wearable system 400 may include a user input device 466 through which the user may input commands to the controller 460 to interact with the wearable system 400. For example, the user input device 466 may include a touchpad, a touchscreen, a joystick, a multi-degree-of-freedom (DOF) controller, a capacitive sensing device, a game controller, a keyboard, a mouse, a D-pad, a stick, a haptic device, a totem (e.g., acting as a virtual user input device), and so on. A multi-DOF controller may sense some or all of the possible translations (e.g., left / right, forward / backward, or up / down) or rotations (e.g., yaw, pitch, or roll) of the controller. A multi-DOF controller that supports translational motion may be referred to as 3DOF, while a multi-DOF controller that supports both translational and rotational motion may be referred to as 6DOF. The user may use a finger (e.g., the thumb) to press or slide on the touch-sensitive input device to provide input to the wearable system 400 (e.g., provide user input to the user interface provided by the wearable system 400). The user input device 466 may be held by the user's hand during the use of the wearable system 400. The user input device 466 may communicate with the wearable system 400 in a wired or wireless manner.

[0068] Figure 5Shows an example of an outgoing light beam output from a waveguide. One waveguide is shown, but it should be understood that other waveguides in the waveguide assembly 480 can have a similar function, where the waveguide assembly 480 includes a plurality of waveguides. Light 520 can be injected into the waveguide 432b at the input edge 432c of the waveguide 432b and propagate within the waveguide 432b by TIR. At the point where the light 520 impinges on the DOE 282, a portion of the light exits the waveguide as the outgoing light beam 510. The outgoing light beams 510 are shown as being substantially parallel, but they can also be redirected to propagate at an angle to the eye 410 (e.g., to form a diverging outgoing light beam), depending on the depth plane associated with the waveguide 432b. It should be understood that the substantially parallel outgoing light beams can indicate a waveguide having a light extraction optical element that couples out light to form an image on a depth plane that appears to be set at a relatively large distance from the eye 410 (e.g., optical infinity). Other waveguides or other groups of light extraction optical elements can output a more diverging pattern of outgoing light beams, which would require the eye 410 to accommodate to a closer distance to focus on the retina and would be interpreted by the brain as light from a distance closer to the eye 410 than optical infinity.

[0069] Figure 6 Is a schematic diagram showing an optical system for generating a multi-focal volumetric display, image, or light field, the optical system including a waveguide device, an optical coupler subsystem for optically coupling light to or from the waveguide device, and a control subsystem. The optical system can include a waveguide device, an optical coupler subsystem for optically coupling light to or from the waveguide device, and a control subsystem. The optical system can be used to generate a multi-focal volume, image, or light field. The optical system can include one or more main planar waveguides 632a (only one is shown in Figure 6 ), and one or more DOEs 632b associated with each of at least some of the main waveguides 632a. The planar waveguide 632a can be similar to the waveguides 432b, 434b, 436b, 438b, 440b discussed with reference to Figure 4 . The optical system can employ a distributed waveguide device to along a first axis ( Figure 6relays light along a vertical axis or Y-axis (in the view of ) and expands the effective exit pupil of the light along a first axis (e.g., the Y-axis). The distributed waveguide device may include, for example, a distributed planar waveguide 622b and at least one DOE 622a (shown by the dashed-dotted line) associated with the distributed planar waveguide 622b. The distributed planar waveguide 622b may be similar or identical to the main planar waveguide 632b in at least some aspects, with a different orientation from it. Similarly, at least one DOE 622a may be similar or identical to DOE 632a in at least some aspects. For example, the distributed planar waveguide 622b and / or DOE 622a may include the same materials as the main planar waveguide 632b or DOE 632a, respectively. Figure 6 The embodiment of the optical display system 600 shown in Figure 2 may be integrated into the wearable system 200 shown in

[0070] The relayed and exit pupil-expanded light may be optically coupled from the distributed waveguide device into one or more main planar waveguides 632b. The main planar waveguide 632b may relay light along a second axis (preferably orthogonal to the first axis (e.g., Figure 6 the horizontal axis or X-axis in the view of )). In particular, the second axis may be a non-orthogonal axis to the first axis. The main planar waveguide 632b expands the effective exit pupil of the light along this second axis (e.g., the X-axis). For example, the distributed planar waveguide 622b may relay and expand light along the vertical axis or Y-axis and transfer the light to the main planar waveguide 632b, and the main planar waveguide 632b may relay and expand light along the horizontal axis or X-axis.

[0071] The optical system may include one or more color light sources (e.g., red, green, and blue lasers) 610, which may be optically coupled to the proximal end of a single-mode optical fiber 640. The distal end of the optical fiber 640 may pass through or be received through the hollow tube 642 of the piezoelectric material. The distal end extends out of the tube 642 as a fixed-free flexible cantilever 644. The piezoelectric tube 642 may be associated with four quadrant electrodes (not shown). The electrodes may be, for example, electroplated on the outside, outer surface, outer periphery, or diameter of the tube 642. A core electrode (not shown) may also be located in the core, center, inner periphery, or inner diameter of the tube 642.

[0072] (e.g., electrically coupled via wiring 660) The driving electronic device 650 drives pairs of opposite electrodes to bend the piezoelectric tube 642 independently on two axes. The protruding distal end of the optical fiber 644 has mechanical resonance modes. The resonance frequency may depend on the diameter, length, and material properties of the optical fiber 644. By vibrating the piezoelectric tube 642 close to the first mechanical resonance mode of the optical fiber cantilever 644, the optical fiber cantilever 644 is vibrated, and the optical fiber cantilever 644 may sweep a large deflection.

[0073] By stimulating resonances on two axes, the tip of the fiber optic cantilever 644 is scanned bidirectionally in the area filled with a two-dimensional (2D) scan. By modulating the intensity of the light source(s) 610 synchronously with the scan of the fiber optic cantilever 644, light emitted from the fiber optic cantilever 644 can form an image. A description of such an arrangement is provided in U.S. Patent Publication No. 2014 / 0003762, the entire content of which is incorporated herein by reference.

[0074] Components of the optical coupler subsystem can collimate the light emitted from the scanning fiber optic cantilever 644. The collimated light can be reflected by the mirror 648 to the narrow distribution planar waveguide 622b, which includes at least one diffractive optical element (DOE) 622a. The collimated light can propagate vertically (relative to Figure 6 the view) along the distribution planar waveguide 622b by TIR and, in so doing, intersect the DOE 622a repeatedly. The DOE 622a preferably has a low diffraction efficiency. This can cause a small portion of the light (e.g., 10%) to be diffracted toward the edge of the larger main planar waveguide 632b at each intersection with the DOE 622a and cause a small portion of the light to continue its original trajectory downward along the length of the distribution planar waveguide 622b via TIR.

[0075] At each intersection with the DOE 622a, additional light can be diffracted toward the entrance of the main waveguide 632b. By splitting the incident light into multiple coupled-out sets, the exit pupil of the light can be vertically expanded by the DOE 622a in the distribution planar waveguide 622b. The vertically expanded light coupled out of the distribution planar waveguide 622b can enter the edge of the main planar waveguide 632b.

[0076] The light entering the main waveguide 632b can propagate horizontally (relative to Figure 6 the view) along the main waveguide 632b via TIR. Since the light intersects the DOE 632a at multiple points when the light propagates horizontally along at least a portion of the length of the main waveguide 632b via TIR. The DOE 632a can advantageously be designed or configured to have a phase profile (which is the sum of a linear diffraction pattern and a radially symmetric diffraction pattern) to produce deflection and focusing of the light. The DOE 632a can advantageously have a low diffraction efficiency (e.g., 10%) so that only a portion of the light beam is deflected toward the eye 210 at each intersection with the DOE 632a while the rest of the light continues to propagate within the waveguide via TIR.

[0077] At each intersection between the propagating light and the DOE 632a, a small portion of the light is diffracted towards the adjacent surface of the main waveguide 632b to allow the light to escape from TIR and emit from the surface of the main waveguide 632b. Additionally, the radially symmetric diffraction pattern of the DOE 632a can impart a focusing level to the diffracted light, shape the wavefront of the individual light beams (e.g., impart curvature), and steer the light beams at an angle matching the designed focusing level.

[0078] Thus, these different paths can enable light to be coupled out of the main planar waveguide 632b through multiple DOEs 632a at different angles and focusing levels, or generate different filling patterns at the exit pupil. The different filling patterns at the exit pupil can be beneficially used to generate a light field display with multiple depth planes. Each layer in the waveguide assembly or a set of layers in the stack (e.g., 3 layers) can be used to generate a corresponding color (e.g., red, blue, green). Thus, for example, a first set of three adjacent layers can be used to generate red, blue, and green light at a first focal depth respectively. A second set of three adjacent layers can be used to generate red, blue, and green light at a second focal depth respectively. Multiple sets can be used to generate a full 3D or 4D color image light field with various focal depths.

[0079] D. Other components of the wearable system

[0080] In many embodiments, in addition to or as an alternative to the components of the wearable system described above, the wearable system can include other components. The wearable system can include, for example, one or more haptic devices or components. The haptic device or component is operable to provide a tactile sensation to the user. For example, when touching virtual content (e.g., virtual objects, virtual tools, other virtual constructs), the haptic device or component can provide a tactile sensation of pressure or texture. The tactile sensation can replicate the feeling of the physical object represented by the virtual object, or can replicate the feeling of an imaginary object or character (e.g., a dragon) represented by the virtual content. In some embodiments, the user can wear the haptic device or component (e.g., a user-wearable glove). In some embodiments, the haptic device or component can be held by the user.

[0081] A wearable system may include, for example, one or more physical objects manipulable by a user to allow input or interaction with the wearable system. These physical objects may be referred to herein as totems. Some totems may take the form of inanimate objects, for example, a piece of metal or plastic, a wall, a table surface. In some embodiments, a totem may not actually have any physical input structure (e.g., buttons, triggers, joysticks, trackballs, rocker switches). Instead, a totem may provide only a physical surface, and the wearable system may render a user interface so that it appears to the user to be on one or more surfaces of the totem. For example, the wearable system may render an image of a computer keyboard and touchpad to appear to reside on one or more surfaces of the totem. For example, the wearable system may render a virtual computer keyboard and virtual touchpad to appear on the surface of a thin rectangular aluminum plate serving as a totem. The rectangular plate itself does not have any physical buttons or touchpad or sensors. However, the wearable system may detect a user's manipulation or interaction or touch with the rectangular plate as a selection or input via the virtual keyboard or virtual touchpad. The user input device 466 ( Figure 4 shown in) may be an example of a totem, which may include a touchpad, a touch panel, a trigger, a joystick, a trackball, a rocker or virtual switch, a mouse, a keyboard, a multi-degree-of-freedom controller, or another physical input device. A user may use a totem alone or in combination with a gesture to interact with the wearable system or other users.

[0082] Examples of haptic devices and totems that may be used with the wearable devices, HMDS, and display systems of the present disclosure are described in U.S. Patent Publication No. 2015 / 0016777, the entire contents of which are incorporated herein by reference.

[0083] E. Web page deconstruction

[0084] Using virtual reality, augmented reality, and / or mixed reality systems (collectively referred to hereinafter as "mixed reality" systems) provides a three-dimensional environment for presenting content to a user. Traditional methods of displaying 2D content within a browser do not work well when used in a 3D environment. One reason is that, with a traditional 2D web browser, the display area of the display device is limited to the screen area of the monitor's positive display content. Thus, traditional browsers are configured to only know how to organize and display content within that monitor display area. In contrast, a 3D environment is not limited by the strict confines of the monitor display area. Thus, traditional 2D browsers do not perform well when used in a 3D environment because traditional browsing techniques do not have the functionality or ability to utilize the 3D environment to display content.

[0085] For example, consider a scenario where a user is using a mixed reality device and has placed multiple browser windows associated with different physical locations. For example, the user may have opened a first browser window in a first room and a second browser window while in a second room. Since traditional 2D-based browsers are limited to the display of a given monitor area, this means that traditional browsers do not even have the technology to understand the concept of physically remote windows, let alone the ability to handle a situation where multiple windows are opened at multiple physical locations, making it impossible for the user to effectively view, navigate to, and use these multiple windows.

[0086] Accordingly, there is a need for an improved approach to implementing browsing technology in a 3D environment.

[0087] Embodiments of the present disclosure deconstruct 2D web pages for display in a spatially organized 3D environment. The 2D web pages can originate from a web browser of a head-mounted system, a mobile device (e.g., a cellular phone), a tablet, a television, an application, etc. In some embodiments, the 2D web pages can be received from another application or device (such as a laptop computer, a desktop computer, an email application having a link to the 2D web page, an electronic message that references or includes a link to the 2D web page, etc.).

[0088] Reference Figure 7, the environment 700 represents a physical environment and a system for implementing the processes described below (e.g., deconstructing 2D content from a web page for display on a 3D surface in the user's physical environment 705 or providing authentication for an application or providing a modal browser window). Representative physical environments and systems of the environment 100 include the user's physical environment 705 as viewed by the user 708 through the head-mounted system 760. Representative systems of the environment 100 also include accessing 2D content (e.g., web pages) via the web browser 710 operatively coupled to the network 720. The network 720 can be the Internet, an intranet, a private cloud network, a public cloud network, etc. The web browser 710 is also operatively coupled to the processor 770 via the network 720. Although the processor 770 is shown as an isolated component separate from the head-mounted system 760, in alternative embodiments, the processor 770 can be integrated with one or more components of the head-mounted system 760 and / or can be integrated into other system components within the environment 100, such as, for example, the network 720 to access the computing network 725 and the storage device 730. The processor 770 can be configured with software 750 for receiving and processing information received from the head-mounted system 760, the local storage device 740, the web browser 710, the computing network 725, and the storage device 730, such as video, audio, and content. The software 750 can communicate with the computing network 725 and the storage device 730 via the network 720. The software 750 can be installed on the processor 770, or in another embodiment, the features and functions of the software can be integrated into the processor 770. The processor 770 can also be configured with a local storage device 740 for storing information used by the processor 770 for quick access without relying on information remotely stored on an external storage device near the user 708. In other embodiments, the processor 770 can be integrated within the head-mounted system 760.

[0089] The user's physical environment 705 is the physical environment around the user 708 when the user moves around and views the user's physical environment 705 through the head-mounted system 760. For example, referring to Figure 7, the user's physical environment 705 shows a room with two walls (e.g., a main wall 780 and a side wall 784, where the main wall and the side wall are relative to the user's view) and a table 788. On the main wall 780, there is a rectangular surface 782 depicted by a solid black line to show a physical surface with physical boundaries (e.g., a painting or a window hanging or attached to the wall), and this physical surface can be a candidate surface onto which a certain 2D content is projected. On the side wall 784, there is a second rectangular surface 786 depicted by a solid black line to show a physical surface with physical boundaries (e.g., a painting or a window hanging or attached to the wall). On the table 788, there can be different objects. 1) A virtual business card holder 790 in which a certain 2D content can be stored and displayed; 2) A horizontal surface 792 depicted by a solid black line to represent a physical surface with physical boundaries onto which a certain 2D content is projected; and 3) Multiple stacks of virtual square surfaces 794 depicted by a dashed black line to represent, for example, stacked virtual newspapers in which a certain 2D content can be stored and displayed.

[0090] The web browser 710 can also display blog pages from the Internet or within an intranet or a private network. Additionally, the web browser 710 can also be any technology that displays digital 2D content. The 2D content can include, for example, web pages, blogs, digital pictures, videos, news articles, newsletters, or music. The 2D content can be stored in a storage device 730 accessible to the user 708 via the network 720. In some embodiments, the 2D content can also be streaming content, e.g., a live video feed or a live audio feed. The storage device 730 can include, for example, a database, a file system, a persistent memory device, a flash drive, a cache, etc. In some embodiments, the web browser 710 that displays 2D content (e.g., a web page) via the computing network 725.

[0091] The computing network 725 accesses the storage device 730 to retrieve and store 2D content for display in the web page on the web browser 710. In some embodiments, the local storage device 740 can provide the user 708 with 2D content of interest. The local storage device 740 can include, for example, a flash drive, a cache, a hard drive, a database, a file system, etc. The information stored in the local storage device 740 can include recently accessed 2D content or content recently displayed in the 3D space. The local storage device 740 allows for performance improvement of the system of the environment 100 by locally providing certain content to the software 750 to help deconstruct the 2D content for display on a 3D surface (e.g., in the user's physical environment 705) in the 3D space environment.

[0092] Software 750 includes a software program stored in a non-transitory computer-readable medium that performs the function of deconstructing 2D content for display within the user's physical environment 705. The software 750 can run on a processor 770, where the processor 770 can be locally attached to the user 708, or in some other embodiments, the software 750 and the processor 770 can be included within a head-mounted system 760. In some embodiments, portions of the features and functions of the software 750 can be stored and executed on a computing network 725 remote from the user 708. For example, in some embodiments, the deconstruction of the 2D content can occur on the computing network 725, and the result of the deconstruction can be stored in a storage device 730, where an inventory of the surfaces of the user's local environment for presenting the deconstructed 2D content thereon can occur within the processor 770, and the inventory and mapping of the surfaces are stored in a local storage device 740. In one embodiment, the processes of deconstructing the 2D content, inventorying the local surfaces, mapping the elements of the 2D content to the local surfaces, and displaying the elements of the 2D content can all occur locally within the processor 770 and the software 750.

[0093] The head-mounted system 760 can be a virtual reality (VR) or augmented reality (AR) head-mounted system that includes a user interface, a user sensing system, an environmental sensing system, and a processor (all not shown). The head-mounted system 760 presents an interface for the user 708 to interact with and experience the digital world. Such interaction can involve the user and the digital world, one or more other users interfacing with the environment 100, and objects within the digital and physical worlds.

[0094] The user interface can include receiving 2D content and selecting elements within the 2D content via user input through the user interface. The user interface can be at least one or a combination of a haptic interface device, a keyboard, a mouse, a joystick, a motion capture controller, an optical tracking device, and an audio input device. A haptic interface device is a device that allows a human to interact with a computer through physical sensations and movements. Haptics refers to a human-computer interaction technology that includes haptic feedback or other physical sensations to perform actions or processes on a computing device. In some embodiments, the control interface can be the user interface such that the user can interact with the MR display system, for example, by providing user input to the system and the system responding by executing corresponding commands.

[0095] The user sensing system may include one or more sensors 762 that are operable to detect specific characteristics, features, or information related to a user 708 wearing the head-mounted system 760. For example, in some embodiments, the sensor 762 may include a camera or optical detection / scanning circuitry capable of detecting real-time optical characteristics / measurements of the user 708, such as one or more of the following optical characteristics / measurements: pupil constriction / dilation, angular measurement / location of each pupil, sphericity, eye shape (since eye shape changes over time), and other anatomical data. This data may provide or be used to calculate information (e.g., the user's visual focus) that can be used by the head-mounted system 760 to enhance the user's viewing experience.

[0096] The environment sensing system may include one or more sensors 764 for obtaining data from the user's physical environment 705. The objects or information detected by the sensors 764 may be provided as an input to the head-mounted system 760. In some embodiments, this input may represent the user's interaction with the virtual world. For example, a user (e.g., user 708) viewing a virtual keyboard on a desk (e.g., table 788) may make gestures with their fingers as if the user were typing on the virtual keyboard. The movement of the moving fingers may be captured by the sensors 764 and provided as an input to the head-mounted system 760, where the input may be used to change the virtual world or create new virtual objects.

[0097] The sensor 764 may include, for example, a generally outward-facing camera or scanner for interpreting scene information, such as by continuously and / or intermittently projected infrared structured light. The environment sensing system may be used to map one or more elements of the user's physical environment 705 around the user 708 by detecting and registering the local environment, including static objects, dynamic objects, people, gestures, and various lighting, atmospheric, and acoustic conditions. Thus, in some embodiments, the environment sensing system may include image-based 3D reconstruction software that is embedded in a local computing system (e.g., processor 770) and is operable to digitally reconstruct one or more objects or information detected by the sensors 764.

[0098] In one example embodiment, the environment sensing system provides one or more of the following: motion capture data (including gesture recognition), depth sensing, face recognition, object recognition, unique object feature recognition, voice / audio recognition and processing, sound source localization, noise reduction, infrared or similar laser projection, and monochrome and / or color CMOS sensors (or other similar sensors), field of view sensors, and various other optical enhancement sensors. It should be understood that the environment sensing system may include other components in addition to those discussed above.

[0099] As mentioned above, in some embodiments, the processor 770 may be integrated with other components of the head-mounted system 760, integrated with other components of the system of the environment 100, or may be a stand-alone device (wearable or separate from the user 708) as shown in Figure 7 . The processor 770 may be connected to various components of the head-mounted system 760 by a physical wired connection or by a wireless connection (such as, for example, a mobile network connection (including cellular phone and data network), Wi-Fi, Bluetooth, or any other wireless connection protocol). The processor 770 may include a storage module, an integrated and / or additional graphics processing unit, a wireless and / or wired Internet connection, and a codec and / or firmware capable of converting data from sources (such as, for example, the computing network 725, the user sensing system and the environment sensing system of the head-mounted system 760) into image and audio data, wherein the image / video and audio may be presented to the user 708 via a user interface (not shown).

[0100] The processor 770 processes data processing for various components of the head-mounted system 760 and data exchange between the head-mounted system 760 and 2D content from web pages displayed or accessed by the web browser 710 and the computing network 725. For example, the processor 770 may be used to buffer and process the data stream between the user 708 and the computing network 725, thereby enabling a smooth, continuous, and high-fidelity user experience.

[0101] Deconstructing the 2D content from a web page into elements and mapping these elements to be displayed on a surface in a 3D environment can be done in an intelligent and logical manner. A predefined set of rules may be used to recommend, suggest, or prescribe where to place specific types of elements / content identified within the 2D content / web page. For example, a particular type of 2D content element may have one or more content elements that may need to be mapped to the surface of a physical or virtual object suitable for storing and displaying the one or more elements, while other types of 2D content elements may be a single object, such as the main video or main article within a web page, in which case the single object may be mapped to the surface that makes the most sense for displaying the single object to the user.

[0102] Figure 8 An exemplary mapping of elements of 2D content to the user's 3D environment according to some embodiments is shown. The environment 800 depicts 2D content (such as, for example, a web page) displayed or accessed by the web browser 710 and the user's physical environment 705. The dashed lines with arrows depict elements (such as, for example, a particular type of content) from the 2D content (such as, for example, a web page) that are mapped to and displayed on the user's physical environment 705. Based on web designer cues or predefined browser rules, specific elements from the 2D content are mapped to specific physical or virtual objects in the user's physical environment 705.

[0103] As an example, the 2D content accessed or displayed by the web browser 710 can be a web page with multiple tabs, where the currently active tab 860 is displayed and the secondary tab 850 is currently hidden until it is selected to be displayed on the web browser 710. What is typically displayed within the active tab 860 is the web page. In this particular example, the active tab 860 is displaying a YOUTUBE page, which includes a main video 820, user comments 230, and recommended videos 840. As depicted in this example Figure 8 the main video 820 can be mapped to be displayed on the vertical surface 782, the user comments 230 can be mapped to be displayed on the horizontal surface 792, and the recommended videos 840 can be mapped to be displayed on a vertical surface 786 that is different from the vertical surface 782. Additionally, the secondary tab 850 can be mapped to be displayed on the virtual business card holder 790 and / or the multi-stack virtual object 794. In some embodiments, the specific content within the secondary tab 850 can be stored in the multi-stack virtual object 794. In other embodiments, all of the content residing within the secondary tab 850 can be stored and / or displayed on the multi-stack virtual object 794. Similarly, the virtual business card holder 790 can contain specific content from the secondary tab 850, or the virtual business card holder 790 can contain all of the content residing within the secondary tab 850.

[0104] The vertical surface 782 can be any type of structure that may be on the main wall 780 of a room (depicted as the user's physical environment 705), such as a window glass or a picture frame. In some embodiments, the vertical surface 782 can be a blank wall, where the head-mounted system 760 determines the optimal size of the frame of the vertical surface 782 that is suitable for the user 708 to view the main video 820. The determination of the size of the vertical surface 782 can be at least partially based on the distance of the user 708 from the main wall 780, the size and dimensions of the main video 820, the quality of the main video 820, the amount of uncovered wall space, and / or the posture of the user when looking at the main wall 780. For example, if the quality of the main video 820 is high definition, the size of the vertical surface 782 can be larger because the quality of the main video 820 will not be adversely affected by the vertical surface 782. However, if the video quality of the main video 820 is poor, having a large vertical surface 782 may greatly impede the video quality, in which case the methods and systems of the present disclosure can adjust / redefine the size of the vertical surface 782 to be smaller to minimize the poor video quality of pixilation.

[0105] Similar to the vertical surface 782, the vertical surface 786 is a vertical surface on an adjacent wall (e.g., the side wall 784) in the user's physical environment 705. In some embodiments, based on the orientation of the user 708, the side wall 784 and the vertical surface 786 may appear as inclined surfaces on an inclined plane. Vertically, an inclined surface on an inclined plane can be a type of surface orientation other than vertical and horizontal surfaces. The recommended video 840 from the YOUTUBE web page can be placed on the vertical surface 786 on the side wall 784 to allow the user 708 to view the recommended video in this example by simply moving their head slightly to the right.

[0106] The virtual business card holder 790 is a virtual object created and displayed to the user 708 by the head-mounted system 760. The virtual business card holder 790 can have the ability for the user 708 to cycle bidirectionally through a set of virtual pages. The virtual business card holder 790 can contain an entire web page, or it can contain individual articles or videos or audio. As shown in this example, the virtual business card holder 790 can contain a portion of the content from the secondary label 850, or in some embodiments, the virtual business card holder 790 can contain the entire page of the secondary label 850. The user 708 can cycle through the content within the virtual business card holder 790 bidirectionally by simply focusing on a specific label within the virtual business card holder 790, and one or more sensors (e.g., the sensor 762) within the head-mounted system 760 detect the eye focus of the user 708 and cycle through the labels within the virtual business card holder 790 accordingly to obtain relevant information for the user 708. In some embodiments, the user 708 can select relevant information from the virtual business card holder 790 and instruct the head-mounted system 760 to display the relevant information on an available surrounding surface or on another virtual object, such as a virtual display (not shown) very close to the user 708.

[0107] Similar to the virtual business card holder 790, the multi-stack virtual object 794 can contain content that the user 708 has bookmarked, saved for future viewing, or has already opened (e.g., inactive labels) ranging from complete content from one or more labels or specific content from various web pages or labels. The multi-stack virtual object 794 is also similar to a stack of newspapers in the real world. Each stack within the multi-stack virtual object 794 can belong to a specific newspaper article, page, magazine issue, recipe, etc. One of ordinary skill in the art can understand that there can be multiple types of virtual objects to achieve the same purpose of providing a surface for placing 2D content elements or content from 2D content sources.

[0108] One of ordinary skill in the art can understand that the 2D content accessed or displayed by the web browser 710 can be more than just a web page. In some embodiments, the 2D content can be a picture from an album, a video from a movie or TV show, a YOUTUBE video, an interactive form, etc. In other embodiments, the 2D content can be an e - book, or any electronic means of displaying a book. Finally, in other embodiments, the 2D content can be other types of content not yet described, as 2D content generally refers to how information is currently presented. If an electronic device can consume 2D content, the 2D content can be used by the head - mounted system 760 to deconstruct the 2D content and display it in a 3D setting (e.g., AR).

[0109] In some embodiments, mapping the accessed 2D content can include (e.g., from the browser) extracting the 2D content and placing it on a surface (such that the content is no longer in the browser and is only on the surface), and in some embodiments, mapping can include (e.g., from the browser) copying the content and placing it on a surface (such that the content is both in the browser and on the surface).

[0110] F. Network content in a bounded volume

[0111] In some embodiments, the extracted web content can be placed within a bounded volume in the user's 3D spatial environment. Advantageously, restricting the placement of the content within a bounded volume can allow the user to better control the placement of the content in their environment. In some examples, the placement of the content can be referenced to the volume surrounding a browser block that can be part of a web browser application. Figure 9A An example browser block 902 within a volume 904 of the 3D space is shown.

[0112] The browser block 902 can display 2D content, such as some or all of a web page, content associated with a web domain, or non - web - based text or graphical information. For example, the browser block 902 can include a graphical interface for accessing content (such as graphics or text). The browser block 902 can include one or more aspects of a web browser, such as multi - window or tab functionality, forward and backward interactive features for navigating content the user has previously interacted with, a refresh interactive feature for refreshing the currently viewed content, a cancel interactive feature for canceling the loading of content, a home interactive feature for returning to a preset content, a navigation feature for entering the local or web location of the content, or a search feature for searching local or web - based content. In some examples, the browser block 902 can include 2D content displayed in conjunction with one or more aspects of a web browser, such as those described above.

[0113] The AR system may allow a user to interact with the browser block 902 and / or the content displayed in the browser block 902. For example, the AR system may receive an indication to move or manipulate the browser block 902 via a user gesture, a hand motion, or actuation of a user input device. The AR system may respond by moving or manipulating the browser block 902 accordingly. In some examples, the relevant user gestures for moving or manipulating the browser block 902 may include hand movements, body positioning, etc. or some combination thereof. In another example, the AR system may receive an indication to manipulate the content displayed in the browser block 902 or other content associated with the browser block 902 via a user gesture or actuation of a user input device. The AR system may respond by manipulating the content associated with the browser block accordingly, such as by scrolling the content, clicking to find new content, selecting content, deleting content, extracting content, entering content, etc. or some combination thereof. Manipulation may include interacting with one or more browser features that may be included in the browser block 902. For example, manipulation may include finding content by entering information into an address or search feature, loading or aborting the loading of content, or other manipulations as a result of interacting with other interactive features.

[0114] The AR system may display the browser block 902 within a volume 904 of 3D space. The volume 904 may include a subset of the user's 3D environment. As described below, the volume 904 may be static or manipulable. For example, the volume 904 may be enlarged, shrunk, or otherwise transformed via a user indication and / or automatically by the AR system.

[0115] The AR system may display 3D content 906 associated with the 2D content displayed in the browser block 902. Figure 9B An example browser block 902 having 3D content 906 within the volume 904 is shown. The 3D content 906 may include one or more virtual objects.

[0116] In some embodiments, the 3D content may include one or more prisms, which generally describe containers, regions, or volumes associated with mixed reality content that may contain multiple virtual content items, such as representations of 3D objects. The content restricted within the prism may be controlled or placed in the user's environment by controlling or placing the prism that restricts the content. As used herein, a virtual object may be or include a prism. The various characteristics, uses, and implementations of prisms are described in U.S. Patent Publication No. 2019 / 0197785, published on June 27, 2019, the entire content of which is incorporated herein by reference.

[0117] The virtual object can be associated with the 2D content displayed in browser block 902. For example, the 2D content can be a website of a furniture company, and the virtual object can be a 3D virtual representation of the furniture described on the website. In some examples, the virtual object can be displayed near browser block 902, such as at a location within 3D volume 904 that is associated with the location of the graphics displayed on browser block 902. For example, the 2D content can include a 2D representation of a sofa. The 3D content can include a 3D representation of the sofa. In some examples, the AR system can display the 3D representation of the sofa in front of or instead of the 2D representation of the sofa, such as a representation of the sofa (or other 3D content) extending from browser block 902.

[0118] The placement and size of the virtual object can be changed or modified by the user. For example, the AR system can place the virtual object(s) based on a default location within volume 904 relative to browser block 902. In some examples, the AR system can apply transformations to one or more virtual objects, such as rotation, scaling, translation, more complex animations, etc., or some combination thereof.

[0119] As Figure 10A and Figure 10B shown, the display of the virtual object can be limited to the boundaries of 3D volume 1002. For example, the AR system can receive instructions, such as instructions from the user or executable instructions, to display a portion of virtual object 906 outside the boundaries of volume 1002. The AR system can determine not to display virtual object 906 or to display a portion of virtual object 906. Additionally or alternatively, as discussed below, the AR system can resize or reorient 3D volume 1002 to enclose virtual object 906, for example, by adjusting the position, volume, orientation, and / or location of virtual object 906.

[0120] In one example, as Figure 10A shown, virtual object 906A can be placed, sized, or oriented such that a portion 1004 of virtual object 906 extends outside the boundaries of 3D volume 1002. If portion 1004 exceeds a threshold volume or threshold percentage of 3D volume 1002 or virtual object 906A or other threshold associated with virtual object 906A or 3D volume 1002, the AR system can determine not to display the entire virtual object 906A. If portion 1004 is below the threshold, the AR system can determine to display all or a portion of virtual object 906A. For example, the AR system can display portion 1006 of virtual object 906 that falls within 3D volume 1002. In another example, the AR system can display the entire virtual object 906A even if it extends outside 3D volume 1002.

[0121] In another example, asFigure 10B As shown, the virtual object 906B can be placed, sized, and / or oriented such that the center 1008 of the virtual object 906B falls outside the 3D volume 1002. If the center 1008 of the virtual object 906 falls outside the 3D volume 1002, the AR system can determine not to display the entire virtual object 906B. In another example, the AR system can determine to display a portion of the virtual object 906B that falls within the 3D volume 1002.

[0122] G. Exemplary bounded volume

[0123] In some examples, the AR system can define a 3D spatial volume (also referred to as a stage) in the user's environment. As Figure 11 shown, the 3D volume 1102 can include a spatial region in which content 1104 can be created, displayed, or manipulated.

[0124] In some examples, the volume 1102 can have a default size or shape. The default size can be set to a predetermined size, can be set relative to the size of the content 1104, can be set relative to the user's environment or field of view, or can be set based on another factor.

[0125] For example, the default size can be a rectangular prism with sides of 0.75 meters, 0.8 meters, 0.86 meters, 1 meter, or other sizes. In another example, the default size can be a rectangular prism with sides of 1000 pixels, 1100 pixels, 1120 pixels, 1200 pixels, or other sizes. Advantageously, setting the default size based on a set of predetermined dimensions can allow for larger content without having to resize the volume 1102.

[0126] In another example, the default size can be a percentage greater than the content 1104 in at least one dimension. For example, the volume 1102 can be 20% wider and 20% taller than the content 1104, and the depth (or breadth) of the volume 904 can be equal to the width of the volume 1102. In another example, the volume 1102 can be 10% wider than the content 1104 and the same height as the content 1104. In another example, the volume 1102 can be 50% wider and 100% taller than the content 1104. However, other examples are possible. Advantageously, setting the default size based on the size of the content 1104 can allow for a smaller and more focused placement of new content related to the content 1104. For example, the content 1104 can be a browser block that allows access to a network domain. The network domain can request the placement of new content. Defining the default size relative to the size of the network domain can allow for the placement of small content related to the network domain near the browser block.

[0127] In some examples, the default size can be relative to the user's field of view such that volume 1102 fills a percentage of the user's field of view (FOV). For example, volume 1102 can be 10%, 25%, 50%, 100%, 150% or other percentage amounts of the FOV.

[0128] The shape of volume 1102 can be any 3D shape such as a cube, rectangular prism, sphere, ellipsoid, pyramid, cylinder, some combination thereof, etc. or other shapes. In some examples, the shape of volume 1102 can be the complementary shape of content 1104. For example, content 1104 can be a browser block having a rectangular prism shape. Volume 1102 can then have a rectangular prism shape. In another example, content 1104 can be a more complex geometric shape such as a couch or a person. Volume 1102 can be the shape that outlines the geometric shape of content 1104. Additionally or alternatively, volume 1102 can be a spherical or rectangular prism shape having dimensions that enclose content 1104.

[0129] In some examples, volume 1102 can have default dimensions based on one or more characteristics of content 1104. For example, content 1104 can be a browser block. Volume 1102 can have dimensions relatively close to the browser block since browser blocks are not designed to move without user input. In another example, content 1104 can be a virtual object (e.g., a butterfly) configured to fly around the user's environment. Volume 1102 can have dimensions that encompass a large portion of the user's FOV to allow content 1104 to move around.

[0130] In some examples, the volume can be a static volume. In other examples, the volume can be manipulable or changeable. As Figure 12A1 and Figure 12A2 shown, volume 1203 can expand to encompass one or more pieces of content 1204, 1206. Figure 12A1 A perspective view 1200 is shown, Figure 12A2 and a top view 1201 of a changeable volume 1202 is shown. For example, the initial size of volume 1203 can be a default size relative to the browser block. The user can interact with the network domain using browser block 1204. The network domain can request that the AR system display or otherwise cause the AR system to display relevant content 1206 within the user's environment. As a result, the AR system can expand volume 1203 in one or more directions to reach (e.g., become) volume 1202B that encompasses the relevant content 1206 and display content 1206, as Figure 12A2 shown.

[0131] The volume can be expanded or resized using defined ranges. Figure 12B1 and Figure 12B2Shows an example expansion of the usage range. The range can include an amount of distance from the definition of a piece of content, such as a browser block. In some examples, if the size of a piece of content is adjusted, one or more ranges around the piece of content can remain the same while the size of the volume around the content is adjusted. For example, an AR system can define a number of pixels in one or more dimensions to add to the volume around content 1224 (shown as 1224A in Figure 12B1 and shown as 1224B in Figure 12B2 ). In some examples, the range can include 100 pixels to the left, 100 pixels to the right, 150 pixels to the bottom, 250 pixels to the left, and / or 200 pixels forward. However, other sizes or combinations of ranges are possible. The range can be defined by the requester, the user, or the AR system.

[0132] In another example, the AR system can define a percentage change in one or more dimensions of the volume. For example, the AR system can increase the width by 20% to the left, 10% to the bottom, 50% to the top, or other combinations of sizes or percentages. In Figure 12B1 and Figure 12B2 the example shown, the AR system utilizes a set of defined ranges 1222 to expand volume 1205A to volume 1205B. For example, the defined range 1222 can be a number of pixels, such as 100 pixels. The AR system can initially determine that the size of volume 1205A corresponds to the volume of content 1224A plus the range on one or more sides of the volume of content 1224A. After resizing the volume of content 1224A to the volume of content 1224B, the AR system can maintain the same range and expand to volume 1205B, which includes the volume of content 1224B plus the range on one or more sides of the volume of content 1224B. In some examples, the content can include a browser block, and resizing can correspond to resizing the browser block or the web page associated with the browser block.

[0133] As discussed below, the AR system can control the dimensions of volume 1205A based on one or more of user input and third-party requests or other factors. For example, to prevent a third party (such as a network domain) from automatically expanding volume 1205A when displaying new content, the AR system can determine a set of permitted states, which can include user input. Advantageously, this process can allow the user or the AR system to exert better control over the use of space within the user's environment, but still allow the third party to recommend an optimized size for the content or volume.

[0134] H. Exemplary resizing authentication process

[0135] In some examples, the AR system can control the size of the adjustment volume through an authentication process. Figure 13A FIG. is a flowchart showing an example resizing process 1300 that may include an authentication process. Figure 13B FIG. shows a set of example steps that may be part of or a result of one or more steps in the resizing process 1300.

[0136] Referring Figure 13A , at block 1302, the AR system can receive a resizing request. The resizing request can include a request to resize the adjustment volume 1324 from an application, a network domain, a third party, or other source, such as Figure 13B shown. Resizing the adjustment volume 1324 can include expanding or contracting the volume 1324 in one or more directions from a point within the volume 1324, such as the center of the volume 1324, the center of the content 1322 within the volume 1324, or another point within the user's 3D environment. The resizing request can include a percentage or amount of change in one or more dimensions of the volume 1324. For example, the amount of change can include an amount in meters, a number of pixels, or other amount. In some examples, the amount of change can be based on the size or placement of the content added within the volume 1324. For example, the user can interact with a network domain via a browser application displayed within the volume 1324. Based on the user interaction, the network domain can request to display 3D content within the user's 3D environment. The network domain can send a resizing request (or the AR system can automatically initiate a resizing request, for example, based on the network domain content) based on the desired size and placement of the 3D content within the volume 1324 such that the 3D volume 1324 contains the added content.

[0137] At block 1304, the AR system can determine whether the resizing is authorized. For example, the AR system can determine whether the user has authorized or has already authorized changing the size of the volume 1324 to the requested size. The AR system can make this determination by displaying a prompt to the user, checking the authorization status of the request, checking the authorization status of the requestor, by some combination thereof, or by another method.

[0138] To determine authorization by displaying a prompt to the user, the AR system can graphically display a graphic or text requesting user input regarding the resizing request. For example, as Figure 13BAs shown, the AR system can output graphic 1328 to request approval for resizing volume 1324. Graphic 1328 can include text and / or one or more interactive components for user input. For example, graphic 1328 can include a prompt to the user regarding whether to allow resizing of the stage. Additionally or alternatively, graphic 1328 can include one or more buttons (such as a "Yes" button or a "No" button) or other interactive components for accepting user input. Additionally or alternatively, the AR system can display one or more temporary lines 1326 to show the new size of the proposed volume 1324. The one or more temporary lines 1326 can include arrows, lines, or other distance indicators. The lines 1326 can start from the current boundary of content 1322 (or the current boundary of the prism containing the content) or the current boundary of stage 1324 and extend to the boundary of the newly proposed stage 1324. In some examples, once user input is received, the lines can be removed. Additionally or alternatively for the temporary lines, the AR system can display the boundary of the proposed volume 1324, for example, by showing the outline of the boundary of the proposed volume 1324 or coloring the added volume for the size of the proposed volume 1324. If resizing is authorized, for example, by the user accepting the resizing request, the AR system can resize stage 1324. If resizing is not authorized, the AR system cannot resize the stage. For example, the AR system can display stage 1323 having the same or a similar volume as content 1322.

[0139] To determine authorization by checking the authorization status of the request, the AR system can determine whether the user has previously authorized resizing. For example, the user can authorize all resizing requests or some resizing requests during a particular interaction session. For example, during a network session, the user may have authorized a volume expansion of 100 pixels in each of six directions. However, the AR system may have expanded the volume by 25 pixels in each of the six directions when placing the content. Then, the AR system can determine whether the current resizing request falls within the limit of the original authorization of 100 pixels in each of the six directions. If the resizing falls within the previously authorized limit, at block 1306, the AR system can resize the stage. If the AR system finds that the resizing is not within the limit, the AR system can reject the resizing or display a prompt to the user to authorize the resizing. Additionally or alternatively, the AR system can determine the authorization status by checking whether the resizing falls within a threshold change (such as a default threshold change or a user-defined threshold change). If the threshold change is below the threshold amount, the AR system can authorize the resizing. If the threshold change exceeds the threshold change, the AR system can reject the resizing or display a prompt to the user to authorize the resizing.

[0140] To determine authorization by checking the authorization status of a requesting party, the AR system can determine whether the user has allowed the requesting party to resize volume 1324, typically during a limited time period and / or within specific limits. For example, the requesting party can be a trusted requesting party such that the AR system will approve all resize requests. In another example, the requesting party can have limited authorization such that the AR system will approve resize requests within set limits regarding time, volume change, and / or other parameters. For example, if the user has authorized the requesting party (such as a network domain) to resize volume 1324 during a network session, the AR system can resize the volume during the network session. However, if the user navigates away from the network domain, the AR system can either not resize the volume or prompt the user for authorization to resize.

[0141] At block 1306, the AR system can resize the stage. Resizing can include one or more processes such as those described with respect to FIGS. 12A and 12B. Referring Figure 13B to, the AR system can resize volume 1324 using a percentage change, a range, or another method. In some examples, the AR system can resize volume 1324 without showing the outline of volume 1324. In some examples, the AR system can show the outline of volume 1324 when resizing.

[0142] At block 1308, the AR system can determine whether to refresh the stage. The AR system can determine that the stage should be refreshed based on one or more refresh conditions. Refresh conditions can include one or more states of the content, content sources (such as network domains or applications), the AR system, resize authorization, or other relevant conditions. For example, the AR system can determine whether resize authorization has ended or whether resizing is no longer relevant, such as when the user is no longer interacting with content from the requesting party. In some examples, a refresh can occur when the user leaves the network domain that requested the resize. For example, the user can switch from a shopping website that requested a resize to display a product in the user's environment to a news website. In another example, a refresh can occur when a network session has ended or a new network session has started. In another example, a refresh can occur when an application is launched or restarted. In another example, the size of the stage can depend on or be bound to the content or application with which the user is interacting. If the user launches another application or loads content with its own stage size requirements, the AR system can refresh the stage. If the AR system determines that a refresh has occurred, at block 1310, the AR system can reset volume 1324. If the AR system has not determined that a refresh has occurred, the AR system can continue to use the current size of volume 1324.

[0143] At block 1310, the AR system can set volume 1324 to a previous size. For example, the AR system can set volume 1324 to the size of the AR system in the user before the resize request. In some examples, the previous size can include the default size of volume 1324, such as the size of a browser block or other content plus some range in one or more dimensions. In other examples, the previous size can include a size that is not the default size.

[0144] I. Exemplary page rotation

[0145] In some examples, the AR system can resize the volume to accommodate content rotation, such as the rotation of a web page within a browser block. Figure 14A An exemplary rotation process 1400 for rotating a web page within a browser block is shown, and FIG. 14B shows an example of resizing the volume rotation of a web page within a browser block.

[0146] Reference Figure 14A , at block 1402, the AR system can receive a rotation request. The rotation request can include a request from the user or a third-party application to rotate one or more components of the content displayed in the user's 3D environment. For example, as shown in FIG. 14B, the AR system can display a web page 1428 within a browser block 1424. Initially, the web page can be displayed in a vertical orientation. The user, a third party, or an application can request that the web page be oriented in a horizontal orientation. In other examples, the AR system can display multiple pieces of content. The AR system can receive a request to reorient or relocate one or more portions of the content within the user's 3D environment.

[0147] Continuing reference Figure 14A , at block 1404, the AR system can determine a new orientation of the content within the user's 3D environment. For example, the AR system can identify the new orientation based on the user's pose and / or the rotation request. Any number of orientations is possible. In some examples, the new orientation can be horizontal, vertical, diagonal, rotated around the x, y, or z axis, etc., or some combination thereof. For example, if the user is looking at the content, such as a browser block, the user can request the orientation of the content within the browser block such that the content faces (e.g., is perpendicular to) the user's line of sight. In another example, as shown in FIG. 14B, the content can be a web page 1428 within a browser block 1424, and the user can request that the web page 1428 be reoriented from a vertical orientation 1401 to a horizontal orientation 1403.

[0148] At block 1406, the AR system can determine a new volume size based on the new orientation. For example, as Figure 14B1As shown, the AR system can adjust the size of volume 1422A to accommodate page 1428 in the new orientation. The new volume 1422B can include the same, greater, or smaller lengths in one or more dimensions. For example, in the example shown in Figure 14B1 and Figure 14B2 , the height h and width w of the volume can remain the same (e.g., in volume 1422A and volume 1422B), but the depth d can be increased to the height of page 1428 in volume 1422B. However, other resizings are possible.

[0149] At block 1408, the AR system can determine a resizing authorization. The resizing authorization can include one or more steps for determining whether the resizing request is authorized by the user or the AR system. For example, the resizing authorization can include referring to one or more steps in the resizing process 1300 described in Figure 13A . In some examples, the AR system can display a prompt to the user to determine whether the resizing or orientation is authorized. If the AR system determines that the resizing is authorized, then at block 1410, the AR system can resize the volume. If the resizing is not authorized, then the AR system can either not resize the volume or not display the new content orientation.

[0150] At block 1410, the AR system can resize the volume based on the resizing authorization. The resizing can include one or more processes, such as the processes described with respect to FIGS. 12A and 12B. As applied to FIG. 14B, for example, the AR system can use a percentage change, a range, or another method to resize volume 1422. In some examples, the AR system can not display the outline of volume 1422A after resizing from volume 1422. In some examples, the AR system can display the outline of volume 1422A when resizing from volume 1422, for example, for a predetermined period of time and / or until a specific user interaction occurs.

[0151] At block 1412, the AR system can display the content in the resized volume in the new orientation. The content can be displayed in the requested orientation such that the position of the content falls within the resized volume 1422A.

[0152] J. Exemplary content processing

[0153] In some examples, the AR system can receive a request to position or place content in the user's 3D environment, where some or all of the content falls outside the boundaries of a defined spatial volume that permits the placement of content therein. Figure 15A An exemplary content handling process 1500 for placing such content is shown, and Figure 15BShows examples of content processing when the size of the volume is adjusted and when the size of the volume is not adjusted.

[0154] Reference Figure 15A , at block 1502, the AR system can receive a content placement request. The content placement request can include a request from an application, a network domain, a third party, or other sources to place content 1526 in an area of the user's 3D environment, such as Figure 15B shown. The content placement request can include the coordinates and / or dimensions of content 1526. The coordinates can be referenced to volume 1522, the user, and / or another origin within the user's 3D environment. In some examples, the request can include placing some or all of content 1526 outside of volume 1522. In some examples, the request can include placing some or all of content 1526 within volume 1522. For example, the user can interact with a network domain via a browser application displayed within volume 1522. Based on the user interaction, the network domain can request to display content 1524 within volume 1522. Additionally or alternatively, the network domain can send a content placement request based on the desired size and placement of content 1524 within volume 1522.

[0155] At block 1506, the AR system can determine whether a portion of the content will fall outside the boundaries of the volume based on the content placement request. For example, as Figure 15B shown in case 1501, the AR system can receive a request to place content 1526 entirely outside the boundaries of volume 1522. However, other cases are possible.

[0156] In another example, the AR system can receive a request to place a portion of content 1526 outside the boundaries of volume 1522. In some examples, if the portion exceeds a threshold volume or a threshold percentage of volume 1522 or other thresholds associated with content 1524, 1526, and / or 3D volume 1522, the AR system can determine that content 1526 falls outside the boundaries of volume 1522. In some examples, the threshold can include 5% of the volume, 10% of the volume, or any other percentage.

[0157] In some examples, the threshold can include 10 pixels, 15 pixels, or any other number of pixels. If the AR system determines that the content falls within the volume boundaries, at block 1518, the AR system can display the content within the volume. If the AR system determines that the content falls outside the volume boundaries, at block 1508, the AR system can determine whether to adjust the size of the volume.

[0158] At block 1508, the AR system can determine whether to resize the volume. For example, the AR system can determine whether to resize the content based on the status of the resize authorization. The resize authorization can include one or more steps for determining whether a resize request is authorized by the user or the AR system. For example, the resize authorization can include referring to Figure 13A one or more steps in the resize process 1300 described. In some examples, the AR system can display a prompt to the user to determine whether resizing or orientation is authorized. If the AR system determines that resizing is authorized, then at block 1516, the AR system can resize the volume. If resizing is not authorized, then at block 1510, the AR system can determine whether to duplicate the content.

[0159] At block 1516, the AR system can resize the volume. Resizing can include one or more processes, such as those described with respect to FIGS. 12A and 12B. Referring to Figure 15B , for example, the AR system can use a percentage change, a range, or another method to resize the volume 1522 to expand to the volume 1522A. In some examples, the AR system may not display the outline of the volume 1522A after resizing from the volume 1522. In some examples, the AR system may display the outline of the volume 1522A after resizing from the volume 1522. Once the AR system has resized the volume, the AR system can display the content within the resized volume 1522A at the requested location, as shown in Figure 15B case 1503 in.

[0160] At block 1510, the AR system can determine whether to duplicate the content. To determine whether a copy should be made and placed, the AR system can determine whether the user has authorized the display of a copy of the content 1526 outside the boundaries of the volume 1522. The copy authorization can include one or more steps for determining whether copying is authorized by the user or the AR system. In some examples, the AR system can display a prompt to the user to determine whether it is authorized to copy and display the content 1526 outside the volume 1522 at the requested placement. If the user indicates approval of the copy in response to the prompt, the AR system can determine that copying is authorized, and at block 1514, display a copy of the content outside the volume 1522. If the user indicates that copying is not approved, then at block 1512, the AR system can determine not to display a copy of the content at the requested placement location.

[0161] A copy of the content may include a visual representation of content 1526, which may or may not include metadata associated with the requesting party, and may or may not allow the requester to control one or more parameters associated with the copy. For example, the copy may include a visual representation of content 1526 that does not allow the party initiating content 1526 (such as a network domain) to control the movement, animation, or other aspects of the representation. In another example, the copy may include a representation of content 1526 that includes metadata connected to the requester, such as hyperlinks, animations, or other data associated with the requester.

[0162] At block 1512, the AR system may hide or not display some or all of content 1526, such as Figure 15B as shown in case 1501. For example, content 1526 may be sized or oriented such that the center of content 1526 falls outside of the 3D volume 1522. If the center of content 1526 falls outside of the 3D volume 1522, the AR system may determine not to display the entire content 1526. In another example, the AR system may determine to display the portion of content 1526 that falls within the 3D volume 1522.

[0163] K. Additional examples

[0164] The systems, methods, and devices described herein each have several aspects, no single one of which is solely responsible for its desirable attributes. Without limiting the scope of the present disclosure, several non-limiting features will now be briefly discussed. The following paragraphs describe various example implementations of the devices, systems, and methods described herein.

[0165] Example 1: A display system for displaying virtual content in a three-dimensional (3D) spatial environment, the display system comprising: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and circuitry communicatively coupled to the head-mounted display and configured to: receive a request to access network-based 3D content; identify parameters associated with the network-based 3D content, including at least one of the following: a location in the user's 3D spatial environment at which to display the network-based 3D content, an orientation of the 3D content, or a size of the 3D content; based on the parameter, determine whether the 3D content can be displayed in an authorized portion of the user's 3D spatial environment; and in response to determining that the 3D content cannot be displayed in the authorized portion of the 3D spatial environment, adjust the size of the authorized portion to allow the 3D content to be displayed in the resized authorized portion.

[0166] Example 2: The display system according to Example 1, wherein the circuit is configured to: display a browser block in a 3D spatial environment, wherein an authorized portion of the 3D spatial environment includes: a width greater than the width of the browser block; a height greater than the height of the browser block; and a depth greater than the depth of the browser block.

[0167] Example 3: The display system according to any one of Examples 1 or 2, wherein the circuit is configured to display a browser block in the authorized portion.

[0168] Example 4: The display system according to any one of Examples 1-3, wherein, in order to determine whether 3D content can be displayed in the authorized portion, the circuit is configured to determine whether a threshold amount of 3D content can be displayed within the authorized portion.

[0169] Example 5: The display system according to any one of Examples 1-4, wherein the circuit is configured to: in response to determining that 3D content can be displayed in the authorized portion of the 3D spatial environment, display the 3D content in the authorized portion.

[0170] Example 6: The display system according to any one of Examples 1-5, wherein, in order to adjust the size of the authorized portion, the circuit is configured to: determine a resize authorization status; and adjust the size of the authorized portion based on the resize authorization status.

[0171] Example 7: The display system according to any one of Examples 1-6, wherein, in order to determine the resize authorization status, the circuit is configured to identify authorization based on a user input.

[0172] Example 8: The display system according to any one of Examples 1-7, wherein the network-based 3D content is associated with a network domain, and wherein, in order to determine the resize authorization status, the circuit is configured to identify authorization based on whether the network domain is an authorized requester.

[0173] Example 9: The display system according to any one of Examples 1-8, wherein the resize authorization status includes authorization to adjust the size of the authorized portion during the current session of accessing the network domain associated with the network-based 3D content.

[0174] Example 10: The display system according to any one of Examples 1-9, wherein, in order to adjust the size of the authorized portion, the circuit is configured to: increase the width of the authorized portion by a first amount in at least one direction; increase the height of the authorized portion by a second amount in at least one direction; and increase the depth of the authorized portion by a third amount in at least one direction.

[0175] Example 11: The display system according to any one of Examples 1 - 10, wherein the circuit is configured to: determine a refresh condition associated with an authorized portion; and set the size of the authorized portion to a default size.

[0176] Example 12: The display system according to Example 11, wherein the refresh condition includes at least one of the following: user interaction with a network domain not associated with network - based 3D content, and an indication from the user to stop displaying network - based 3D content.

[0177] Example 13: A display system for displaying virtual content in a three - dimensional (3D) space environment, the display system comprising:

[0178] A head - mounted display configured to present virtual content to the eyes of a user of the display system; and a circuit in communication with the head - mounted display, the circuit being configured to: receive a request to access content; display the content in an authorized portion of the user's 3D space environment in a first orientation; receive a request to display the content in a second orientation in the user's 3D space environment; determine whether the content can be displayed in the authorized portion of the user's 3D space environment in the second orientation; and in response to determining that the content cannot be displayed in the authorized portion of the 3D space environment in the second orientation, adjust the size of the authorized portion to allow the content to be displayed in the resized authorized portion in the second orientation.

[0179] Example 14: The display system according to Example 13, wherein the content includes a web page.

[0180] Example 15: The display system according to Example 13 or 14, wherein the circuit is configured to: display the content within a virtual browser block in the 3D space environment, and wherein the authorized portion of the 3D space environment includes: a width greater than the width of the virtual browser block; a height greater than the height of the virtual browser block; and a depth greater than the depth of the virtual browser block.

[0181] Example 16: The display system according to Example 15, wherein the circuit is configured to display the virtual browser block within the authorized portion.

[0182] Example 17: The display system according to any one of Examples 13 - 16, wherein, in order to determine whether the content can be displayed within the authorized portion, the circuit is configured to determine whether a threshold amount of the content can be displayed within the authorized portion.

[0183] Example 18: The display system according to any one of Examples 13 - 17, wherein the circuit is configured to: in response to determining that the content can be displayed in the authorized portion of the 3D space environment in the second orientation, display the content in the authorized portion in the second orientation.

[0184] Example 19: The display system according to any one of Examples 13 - 18, wherein, in order to adjust the size of the authorized portion, the circuit is configured to: determine a resize authorization status; and adjust the size of the authorized portion based on the resize authorization status.

[0185] Example 20: The display system according to any one of Examples 13 - 19, wherein, in order to determine the resize authorization status, the circuit is configured to identify authorization based on a user input.

[0186] Example 21: The display system according to any one of Examples 13 - 20, wherein the content is associated with a network domain, and wherein, in order to determine the resize authorization status, the circuit is configured to identify authorization based on whether the network domain is an authorization requester.

[0187] Example 22: The display system according to any one of Examples 13 - 21, wherein the resize authorization status includes authorization to adjust the size of the authorized portion during a current session of accessing a network domain associated with 3D content.

[0188] Example 23: The display system according to any one of Examples 13 - 22, wherein, in order to adjust the size of the authorized portion, the circuit is configured to: increase the width of the authorized portion by a first amount in at least one direction; increase the height of the authorized portion by a second amount in at least one direction; and increase the depth of the authorized portion by a third amount in at least one direction.

[0189] Example 24: The display system according to any one of Examples 13 - 23, wherein the circuit is configured to: determine a refresh condition associated with the authorized portion; and set the size of the authorized portion to a default size.

[0190] Example 25: The display system according to Example 24, wherein the refresh condition includes at least one of the following: user interaction with a network domain not associated with the content, and an indication from the user to stop displaying the content.

[0191] Example 26: A display system for displaying virtual content in a three-dimensional (3D) space environment, the display system comprising: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and circuitry communicatively coupled to the head-mounted display, the circuitry being configured to: receive a request to access 3D content; identify parameters associated with the 3D content, the parameters including at least one of: a location in the user's 3D space environment at which to display the 3D content, an orientation of the 3D content, and a size of the 3D content; based on the parameters, determine whether the 3D content can be displayed in an authorized portion of the user's 3D space environment; and in response to determining that the 3D content cannot be displayed in the authorized portion of the 3D space environment, display a representation of the 3D content at the location in the 3D space environment.

[0192] Example 27: The display system of Example 26, wherein the circuitry is configured to display a browser block in the 3D space environment, and wherein the authorized portion of the 3D space environment includes: a width greater than a width of the browser block; a height greater than a height of the browser block; and a depth greater than a depth of the browser block.

[0193] Example 28: The display system of any one of Examples 26 or 27, wherein the circuitry is configured to display the browser block in the authorized portion.

[0194] Example 29: The display system of any one of Examples 26 - 28, wherein, to determine whether the 3D content can be displayed in the authorized portion, the circuitry is configured to determine whether a threshold amount of the 3D content can be displayed within the authorized portion.

[0195] Example 30: The display system of any one of Examples 26 - 28, wherein the circuitry is configured to: in response to determining that the 3D content can be displayed in the authorized portion of the 3D space environment, display the 3D content in the authorized portion.

[0196] Any of the above examples may be combined in any suitable combination.

[0197] L. Terms

[0198] To facilitate understanding of the systems and methods discussed herein, a number of terms are described below. The terms described below, as well as other terms used herein, should be construed to include the provided description, the ordinary and customary meaning of the term, and / or any other implied meaning for the corresponding term, where such construction is consistent with the context of the term. Accordingly, the following description does not limit the meaning of these terms, but merely provides exemplary descriptions.

[0199] Prism: A container, region, or volume associated with mixed reality content. For example, a prism can contain multiple virtual content items that can be selected by the user. A prism can be generated when an application is launched and then can generate sibling or child prisms to create a flexible layout. An application within a prism can be configured to control where these hierarchical prisms will appear, typically near the first prism and easily discoverable by the user.

[0200] A prism can provide feedback to the user. In some embodiments, the feedback can be a title that is only shown to the user when the prism is targeted by a head pose. In some embodiments, the feedback can be a glow around the prism. Prism glow (and / or other prism feedback) can also be used for sharing to give feedback to the user about the prism that is being shared.

[0201] Controller: A handheld controller, such as a totem.

[0202] Controller axis: An axis extending from the controller that defines the pointing direction of the controller.

[0203] Head pose: The head position and / or head orientation determined using sensors such as an inertial measurement unit (IMU), accelerometer, gyroscope, etc. A head pose ray extending in the direction of the head pose can be used to interact with virtual objects. For example, when the user is pointing at or looking at a prism or object, the object or prism intersects with the user's head pose ray.

[0204] Focus: A property of an object (such as a prism) that allows an interactive object to be selected.

[0205] Input focus: A property of an object (such as a prism or application) that causes the cursor of the object to be refreshed and rendered as the active system cursor. In some implementations, there can be multiple focus objects, but only one has the input focus.

[0206] Browser block: A content window that can be used to navigate, display, or otherwise interact with web-based content. A browser block can be displayed in the user's 3D environment as a 2D or 3D object that can be manipulated and interacted with.

[0207] 3D content: Virtual objects that can be displayed in the user's 3D environment. 3D content can be static, animated, manipulable, or otherwise interactive. 3D content can include web-based content generated through user interaction with a web domain or web page using, for example, a browser block.

[0208] Content volume (or content stage): The content volume can include the spatial volume of the user's environment within which 3D content can be displayed or manipulated.

[0209] M. Other considerations

[0210] Each of the processes, methods, and algorithms described herein or depicted in the figures can be embodied in, or fully or partially automated by, one or more physical computing systems, hardware computer processors, special-purpose circuits, or code modules executed by electronic hardware configured to execute specific and particular computer instructions. For example, a computing system can include a general-purpose computer (such as a server) programmed with specific computer instructions, or a special-purpose computer, special-purpose circuit, etc. The code modules can be compiled and linked into an executable program, installed in a dynamic link library, or written in an interpreted programming language. In some embodiments, specific operations and methods can be performed by circuits specific to a given function.

[0211] Furthermore, certain embodiments of the functions of the present disclosure are sufficiently complex mathematically, computationally, and technically that special-purpose hardware or one or more physical computing devices (utilizing appropriate special-purpose executable instructions) can be necessary for performing functions such as due to the volume or complexity of the computations involved or for providing results substantially in real time. For example, an animation or video can include many frames, each having millions of pixels, and requires specially programmed computer hardware to process the video data to provide the desired image processing tasks or applications within a commercially reasonable amount of time.

[0212] Code modules or any type of data can be stored on any type of non-transitory computer-readable medium, such as physical computer storage devices (including hard disk drives, solid-state memories, random access memories (RAM), read-only memories (ROM), optical discs), volatile or non-volatile storage devices, combinations thereof, etc. Methods and modules (or data) can also be sent as generated data signals (such as as part of a carrier wave or other analog or digital propagated signal) on various computer-readable transmission media (including wireless-based and wire / cable-based media), and can take many forms (such as as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps can be permanently or otherwise stored in any type of non-transitory tangible computer storage device, or can be transmitted via a computer-readable transmission medium.

[0213] Any process, block, state, step, or function in the flowcharts described herein or depicted in the figures should be understood as potentially representing code modules, code segments, or portions of code that include one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in the process. The various processes, blocks, states, steps, or functions may be combined, rearranged, added to the illustrative examples provided herein, deleted from the illustrative examples provided herein, modified, or otherwise changed. In some embodiments, additional or different computing systems or code modules may perform some or all of the functions described herein. The methods and processes described herein are also not limited to any particular order, and the associated blocks, steps, or states may be performed in a suitable other order (e.g., sequentially, in parallel, or in some other manner). Tasks or events may be added to or deleted from the disclosed example embodiments. Moreover, the separation of the various system components in the embodiments described herein is for illustrative purposes and should not be understood as required in all embodiments. It should be understood that the described program components, methods, and systems can generally be integrated together in a single computer product or packaged into multiple computer products. Many variations of the embodiments are possible.

[0214] Processes, methods, and systems may be implemented in a network (or distributed) computing environment. The network environment includes enterprise-wide computer networks, intranets, local area networks (LANs), wide area networks (WANs), personal area networks (PANs), cloud computing networks, crowdsourcing computing networks, the Internet, and the World Wide Web. The network may be a wired or wireless network or any other type of communication network.

[0215] The systems and methods of the present disclosure each have several innovative aspects, none of which alone is solely responsible for or requires the desired attributes disclosed herein. The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. Various modifications to the embodiments described in this disclosure may be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the spirit or scope of the present disclosure. Accordingly, the claims are not intended to be limited to the embodiments or examples shown herein, but rather will conform to the broadest scope consistent with the disclosure, principles, and novel features disclosed herein.

[0216] Certain features described in this specification in the context of separate embodiments or examples can also be implemented in combination in a single embodiment or example. Conversely, various features described in the context of a single embodiment or example can also be implemented separately or in any suitable sub-combination in multiple embodiments or examples. Moreover, although the features may be described above as acting in certain combinations and even initially so claimed, in some cases one or more features of the claimed combination can be removed from the combination, and the claimed combination can be directed to a sub-combination or a variant of a sub-combination. For each embodiment, no single feature or set of features is necessary or essential.

[0217] Conditional language used herein, such as, without limitation, "can," "could," "might," "may," "for example," etc., unless specifically stated otherwise, is generally understood within the context in which it is used to convey that certain embodiments include, while other embodiments do not include, certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that a feature, element, or step is in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether such a feature, element, or step is included or to be performed in any particular embodiment. The terms "comprising," "including," "having," etc. are synonymous and are used inclusively in an open-ended fashion, and do not exclude additional elements, features, acts, operations, etc. Moreover, the term "or" is used in its inclusive sense (and not in its exclusive sense) so that, for example, when used to connect a list of elements, the term "or" means one, some, or all of the elements in the list. Additionally, as used in this application and the appended claims, the articles "a," "an," and "the" are to be construed to mean "one or more" or "at least one" unless otherwise specified.

[0218] As used herein, the phrase "at least one" in reference to a list of items means any combination of those items, including a single member. For example, "at least one of A, B, or C" is intended to cover: A, B, C, A and B, A and C, B and C, and A, B, and C. Unless otherwise specifically stated, words such as the phrase "at least one of X, Y, and Z" are to be understood in the context as typically used to convey that items, terms, etc. may be at least one of X, Y, or Z. Thus, such conjunctive language is generally not intended to imply that certain embodiments require the presence of at least one of X, at least one of Y, and at least one of Z.

[0219] Similarly, although operations may be depicted in the drawings in a particular order, it should be recognized that such operations need not be performed in the particular order shown or in sequential order, or that all illustrated operations need to be performed to achieve the desired result. Further, the drawings may schematically depict one or more example processes in the form of a flowchart. However, other operations not depicted may be incorporated into the example methods and processes schematically shown. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the illustrated operations. Additionally, in other implementations, the operations may be rearranged or reordered. In some cases, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the above-described implementations should not be construed as requiring such separation in all implementations, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged into multiple software products. Additionally, other implementations are within the scope of the appended claims. In some cases, the acts recited in the claims may be performed in a different order and still achieve the desired result.

Claims

1. A display system for displaying virtual content in a three-dimensional (3D) space environment, the display system comprising: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and a circuit in communication with the head-mounted display, the circuit being configured to: receive a request to access 3D content, wherein the 3D content is network-based; identify parameters associated with the 3D content, the parameters including at least one of: a location in the 3D space environment at which to display the 3D content, an orientation of the 3D content, or a size of the 3D content; based on the parameters, determine whether a threshold amount of the 3D content can be displayed within a bounded volume of the 3D space environment, wherein the bounded volume is a portion of the 3D space environment defined by the display system; in response to determining that the threshold amount of the 3D content cannot be displayed within the bounded volume of the 3D space environment, automatically adjust the size of the bounded volume to allow the threshold amount of the 3D content to be displayed within the resized bounded volume.

2. The display system according to claim 1, wherein the circuit is configured to: display a browser block in the 3D space environment, wherein the bounded volume of the 3D space environment includes: a width greater than a width of the browser block; a height greater than a height of the browser block; and a depth greater than a depth of the browser block.

3. The display system according to claim 2, wherein the circuit is configured to display the browser block within the bounded volume.

4. The display system according to claim 1, wherein the circuit is configured to: in response to determining that the threshold amount of the 3D content can be displayed within the bounded volume of the 3D space environment, display the 3D content within the bounded volume.

5. The display system according to claim 1, wherein to adjust the size of the bounded volume, the circuit is configured to: determine a resize authorization status; and based on the resize authorization status, adjust the size of the bounded volume.

6. The display system according to claim 5, wherein to determine the resize authorization status, the circuit is configured to identify an authorization based on user input.

7. The display system according to claim 5, wherein the 3D content is associated with a network domain, and wherein to determine the resize authorization status, the circuit is configured to identify an authorization based on whether the network domain is an authorized requester.

8. The display system according to claim 5, wherein the resize authorization status includes authorization to adjust the size of the bounded volume during a current session accessing the network domain associated with the 3D content.

9. The display system according to claim 1, wherein to adjust the size of the bounded volume, the circuit is configured to: increase the width of the bounded volume by a first amount in at least one direction; increase the height of the bounded volume by a second amount in at least one direction; and Increase the depth of the bounded volume by a third amount in at least one direction.

10. The display system according to claim 1, wherein, the circuit is configured to: determine a refresh condition associated with the bounded volume; and set the size of the bounded volume to a default size.

11. The display system according to claim 10, wherein, the refresh condition includes at least one of the following: user interaction with a network domain not associated with the 3D content, and an indication from the user to stop displaying the threshold amount of the 3D content.

12. A display system for displaying virtual content in a three-dimensional (3D) space environment, the display system comprising: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and a circuit in communication with the head-mounted display, the circuit being configured to: receive a request to access content; display the content in a bounded volume of the user's 3D space environment in a first orientation; receive a request to display the content in the user's 3D space environment in a second orientation; determine whether a threshold amount of the 3D content can be displayed in the bounded volume of the user's 3D space environment in the second orientation, wherein the bounded volume is a portion of the 3D space environment defined by the display system; and in response to determining that the content cannot be displayed in the bounded volume of the 3D space environment in the second orientation, automatically adjust the size of the bounded volume to allow the content to be displayed in the resized bounded volume in the second orientation.

13. The display system according to claim 12, wherein, the content includes a web page.

14. The display system according to claim 12, wherein, the circuit is configured to: display the threshold amount of the 3D content within a virtual browser block in the 3D space environment, wherein the bounded volume of the 3D space environment includes: a width greater than the width of the virtual browser block; a height greater than the height of the virtual browser block; and a depth greater than the depth of the virtual browser block.

15. The display system according to claim 12, wherein, the circuit is configured to: in response to determining that the threshold amount of the 3D content can be displayed in the bounded volume of the 3D space environment in the second orientation, display the threshold amount of the 3D content in the bounded volume in the second orientation.

16. The display system according to claim 12, wherein, to determine a resize authorization status, the circuit is configured to identify authorization based on user input.

17. The display system according to claim 12, wherein, the content is associated with a network domain, and wherein to determine a resize authorization status, the circuit is configured to identify authorization based on whether the network domain is an authorized requester.

18. The display system according to claim 16, wherein, The resizing authorization status includes authorization to resize the bounded volume during a current session accessing a network domain associated with the content.

19. A display system for displaying virtual content in a three-dimensional (3D) spatial environment, the display system comprising: a head-mounted display configured to present virtual content to the eyes of a user of the display system; and a circuit in communication with the head-mounted display, the circuit being configured to: receive a request to access 3D content; identify parameters associated with the 3D content, the parameters including at least one of: a location in the 3D spatial environment of the user at which to display the 3D content, an orientation of the 3D content, and a size of the 3D content; based on the parameters, determine whether a threshold amount of the 3D content can be displayed within a bounded volume of the 3D spatial environment of the user, wherein the bounded volume is a portion of the 3D spatial environment defined by the display system; and in response to determining that the threshold amount of the 3D content cannot be displayed within the bounded volume of the 3D spatial environment, display a representation of the 3D content at the location in the 3D spatial environment.

Images