Artificial reality browser configured to trigger immersive experience
By preloading immersive resources in an artificial reality browser and configuring browser components using API calls, the problem that traditional browsers cannot provide a transformation of two-dimensional and three-dimensional experiences is solved, and a user-friendly immersive experience is achieved.
Patent Information
- Application Number
- CN202380034595.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-20
- Filing Date
- 2023-04-16
- Publication Date
- 2024-11-26
AI Technical Summary
Traditional browsers cannot effectively provide a transition between two-dimensional user experience and immersive user experience, and the lack of a consistent user interface makes users confused when triggering an immersive experience.
Configure browser chrome components by preloading immersive resources in an artificial reality browser and utilizing application programming interface (API) calls to configure browser chrome components, modifying their properties to trigger the transition from a 2D web page to a 3D immersive experience, providing consistent entry Click and user notification.
It realizes a smooth transition between providing a two-dimensional and three-dimensional experience in traditional browsers, eliminating user confusion and providing a consistent user interface that supports a unified entry point for multiple immersive experiences.
Smart Images

Figure CN119032355A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an artificial reality browser configured to trigger an immersive experience. Background Art
[0002] Display systems can display content to users in a variety of formats that match user preferences or use cases. However, content may have a variety of display configurations, and effectively displaying content based on user selection and / or user expectations remains a challenge. For example, web content including immersive experiences may have a variety of different configurations that are specific to design choices, implemented technologies, model selection, or other suitable configurations. Due to this diversity, users may encounter undesirable display artifacts and / or encounter navigation that does not achieve intuitive results. Summary of the invention
[0003] According to a first aspect of the present disclosure, a method for triggering an immersive experience at an artificial reality (XR) browser is provided, the method comprising: displaying a web page at an XR browser according to a received web resource defining a two-dimensional web page, wherein the XR browser comprises a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content according to the web resource received by the XR browser; preloading a plurality of immersive resources at the XR browser, the plurality of immersive resources comprising one or more JavaScript resources, graphic resources, model resources, binary resources, or any combination of the foregoing resources; receiving an application programming interface (API) call indicating that the plurality of immersive resources have been preloaded from the received web resource or the preloaded plurality of immersive resources; setting a property of the browser chrome component in response to the API call, wherein the set property modifies the display of the browser chrome component; and triggering a display transition from the two-dimensional web page to the immersive experience based on input related to the browser chrome component, wherein the immersive experience is displayed according to the preloaded plurality of immersive resources.
[0004] In some embodiments, the browser chrome component includes a button, and the input related to the browser chrome component includes a button press.
[0005] In some embodiments, the modification of the display of the browser chrome component by the set properties includes: a) transitioning the browser chrome component from hidden to displayed; or b) transitioning the browser chrome component from a non-interactive component that is not configured to receive user input to an interactive component that is configured to receive user input.
[0006] In some embodiments, the method further comprises: displaying a menu at the browser chrome based on input related to the browser chrome component, the menu comprising parameters for the immersive experience.
[0007] In some embodiments, at least one parameter of the menu includes a user presence parameter, and based on input related to the user presence parameter of the menu, the user avatar is included in the triggered immersive experience.
[0008] In some embodiments, in response to input related to a user presence parameter of a menu, a user avatar is dynamically loaded at the XR browser for an immersive experience.
[0009] In some embodiments, at least one parameter of the menu includes a position parameter, and the triggered display position within the immersive experience is defined by input related to the position parameter of the menu.
[0010] In some embodiments, the method further comprises:
[0011] Based on input related to the browser chrome component, one or more permission prompts are displayed at the browser chrome, wherein in response to the input related to the one or more permission prompts, a device implementing an XR browser at the client system is activated during the triggered immersive experience.
[0012] In some embodiments, the device includes one or more of: a microphone, a camera, a controller, a sensor, or any combination thereof.
[0013] In some embodiments, the immersive experience includes a three-dimensional environment.
[0014] According to a second aspect, the present disclosure provides a computer-readable storage medium storing instructions, which, when executed by a computing system, causes the computing system to execute a process for triggering an immersive experience at an artificial reality (XR) browser, the process comprising: displaying a web page at the XR browser based on a received web resource defining a two-dimensional web page, wherein the XR browser comprises a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content based on the web resource received by the XR browser; preloading a plurality of immersive resources at the XR browser; receiving an application programming interface call (API) indicating that the plurality of immersive resources have been preloaded from the received web resource or the preloaded plurality of immersive resources; setting a property of the browser chrome component in response to the API call, wherein the set property modifies the display of the browser chrome component; and triggering a display transition from a two-dimensional web page to an immersive experience based on input related to the browser chrome component, wherein the immersive experience is displayed based on the preloaded plurality of immersive resources.
[0015] In some embodiments, the browser chrome component includes a button, and the input related to the browser chrome component includes a button press.
[0016] In some embodiments, the modification of the display of the browser chrome component by the set properties includes: a) transitioning the browser chrome component from hidden to displayed; or b) transitioning the browser chrome component from a non-interactive component that is not configured to receive user input to an interactive component that is configured to receive user input.
[0017] In some embodiments, the process further includes displaying a menu at the browser chrome based on input related to the browser chrome component, the menu including parameters for the immersive experience.
[0018] In some embodiments, at least one parameter of the menu includes a user presence parameter, and based on input related to the user presence parameter of the menu, the user avatar is included in the triggered immersive experience.
[0019] In some embodiments, in response to input related to a user presence parameter of a menu, a user avatar is dynamically loaded at the XR browser for an immersive experience.
[0020] In some embodiments, at least one parameter of the menu includes a position parameter, and the triggered display position within the immersive experience is defined by input related to the position parameter of the menu.
[0021] In some embodiments, the process further includes: based on input related to the browser chrome component, displaying one or more permission prompts at the browser chrome, wherein, in response to input related to the one or more permission prompts, activating a device implementing an XR browser at the client system during the triggered immersive experience, wherein the device includes one or more of the following: a microphone, a camera, a controller, a sensor, or any combination thereof.
[0022] In some embodiments, the immersive experience includes a three-dimensional environment.
[0023] According to a third aspect, the present disclosure provides a computing system for triggering an immersive experience in an artificial reality (XR) browser, the computing system comprising: one or more processors; and one or more memories storing instructions, which, when executed by the one or more processors, cause the computing system to execute a process, the process comprising: displaying a web page at an XR browser based on a received web resource defining a two-dimensional web page, wherein the XR browser comprises a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content based on the web resource received by the XR browser; preloading multiple immersive resources at the XR browser; receiving an application programming interface call (API) indicating that the multiple immersive resources have been preloaded from the received web resource or the preloaded multiple immersive resources; in response to the API call, setting a property of the browser chrome component, wherein the set property modifies the display of the browser chrome component; and triggering a display transition from a two-dimensional web page to an immersive experience based on input related to the browser chrome component, wherein the immersive experience is displayed based on the preloaded multiple immersive resources.
[0024] It will be understood that any features described herein that are suitable for incorporation into one or more aspects or embodiments of the present disclosure are intended to be generalizable to any and all aspects and embodiments of the present disclosure. Other aspects of the present disclosure may be understood by those skilled in the art based on the specification, claims and drawings of the present disclosure. The foregoing general description and the following detailed description are exemplary and illustrative only, and are not intended to limit the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 is a block diagram showing an overview of a device on which some embodiments of the present technology may operate.
[0026] Figure 2A is a line diagram illustrating a virtual reality headset that may be used in some embodiments of the present technology.
[0027] Figure 2B is a line diagram illustrating a mixed reality head mounted viewer that may be used in some embodiments of the present technology.
[0028] Figure 2C is a line diagram illustrating a plurality of controllers that, in some implementations, a user may hold in one or both hands to interact with an artificial reality environment.
[0029] Figure 3 is a block diagram illustrating an overview of an environment in which some embodiments of the present technology may operate.
[0030] Figure 4 is a block diagram illustrating various components that may be used in a system employing the disclosed technology in some implementations.
[0031] Figure 5 is a system diagram illustrating components of an artificial reality browser configured to trigger an immersive experience.
[0032] Fig. 6A and Figure 6B is a diagram for configuring an immersive experience triggered using an artificial reality browser.
[0033] Fig. 7A , Figure 7B , Figure 7C and Fig.7D A series of schematic diagrams visualizing an artificial reality browser configured to trigger an immersive experience are shown.
[0034] Figure 8 is a flow chart illustrating a process for triggering an immersive experience at an artificial reality browser in some implementations of the present technology.
[0035] Fig. 9 is a flow chart illustrating a process for triggering an immersive experience configured with a menu at an artificial reality browser in some embodiments of the present technology.
[0036] The various techniques described herein may be better understood by referring to the following detailed description in conjunction with the accompanying drawings, in which like reference numbers refer to the same elements or functionally similar elements. DETAILED DESCRIPTION
[0037] Aspects of the present disclosure relate to an artificial reality browser configured to trigger an immersive experience. A web browser may, for example, display content from a public network (e.g., the Internet) via a control (host) of a web page, display content from a content distribution network, or content from any other network source of web content. An artificial reality browser may be configured to display both a two-dimensional display (e.g., a standard web page) and an immersive experience (e.g., a three-dimensional environment) to a user. A two-dimensional display, web page, or website may be a traditional two-dimensional (2D) panel or may include some three-dimensional (3D) content, such as a 3D model provided in association with a web page or a 3D model viewed with a parallax effect. A web browser may acquire / receive resources based on executing code and / or scripts (e.g., JavaScript) implemented as part of a web page / web content.
[0038] In some embodiments, after displaying a web page (e.g., a home page, a landing page, or any other suitable portion), the immersive resources are loaded (e.g., preloaded and / or dynamically loaded) by the artificial reality browser. For example, a landing page including limited web resources can load quickly, while an immersive experience of a three-dimensional environment may take more time to load. Therefore, the resources for the immersive experience can be loaded while the landing page (e.g., a two-dimensional web page) is displayed to the user. Any other suitable web page can be displayed while receiving, acquiring, launching, and / or configuring the resources for the immersive experience.
[0039] In some embodiments, the artificial reality browser includes a display panel that displays a two-dimensional web page / content obtained from a public network and a browser chrome that displays interface / control elements for configuring and interacting with the artificial reality browser itself. For example, buttons displayed in the display panel can trigger web page functions (e.g., navigation to the next web page, submission of a web form, etc.), while buttons displayed at the browser chrome can trigger browser functions (e.g., bookmark functions, user preferences, etc.).
[0040] When the immersive experience is loaded (e.g., when the immersive experience is available for experience by the user), each embodiment displays the element at the browser chrome of the artificial reality browser. For example, the user can select the displayed element (e.g., press the displayed button) and trigger the transition from the two-dimensional panel-based display to the three-dimensional immersive experience. The embodiment of the artificial reality browser includes an application programming interface (API) that supports the configuration of the browser chrome element through the files, scripts, resources, or other suitable components of the web page. For example, once the relevant resources for the immersive experience are loaded at the artificial reality browser, the components of the web page can make API calls. The API call can cause the browser chrome element to be displayed, the display properties of the browser chrome element to be changed (e.g., from a gray display button indicating an inactive state to a colored button indicating an active state), or the browser chrome element to be configured in any other suitable manner.
[0041] In some embodiments, in response to input at the browser chrome element (e.g., a button press), the XR browser may display one or more add-ons for configuring the immersive experience. In one example, the displayed add-ons may include a permissions panel in which the user may provide permissions to enter the immersive experience and / or access the user's system (e.g., microphone access, camera access, etc.) during the immersive experience. In another example, the displayed add-ons may include a configuration panel in which the user may provide input to configure the immersive experience, such as an initial location in the three-dimensional environment to which the user will be taken, the user's presence for the immersive experience (e.g., avatar selection), or any other suitable input for configuring the immersive experience.
[0042] Upon receiving input at the browser chrome element / when a trigger for an immersive experience is initiated, the artificial reality browser can transition from displaying a two-dimensional panel view of a web page supported by loaded web resources (e.g., hypertext transfer protocol (HTTP) pages, graphic images, etc.) to displaying a three-dimensional environment supported by preloaded immersive resources (e.g., three-dimensional models, three-dimensional graphic images, etc.). In some embodiments, the two-dimensional panel view of the web page is not displayed during the immersive experience.
[0043] Embodiments of the disclosed technology may include an artificial reality system, or may be implemented in conjunction with an artificial reality system. Artificial reality or extra reality (XR) is a form of reality that has been adjusted in some way before being presented to a user, and the artificial reality or extra reality may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content, or generated content combined with collected content (e.g., photos of the real world). Artificial reality content may include video, audio, tactile feedback, or some combination thereof, any of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that gives the viewer a three-dimensional effect). In addition, in some embodiments, artificial reality may be associated with an application, product, accessory, service, or some combination thereof, which is used, for example, to create content in artificial reality and / or to be used in artificial reality (e.g., to perform activities in artificial reality). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a stand-alone HMD, a mobile device or computing system, a "cave" environment projection system or other projection system, or any other hardware platform capable of providing artificial reality content to one or more viewers.
[0044] As used herein, "virtual reality" or "VR" refers to an immersive experience in which a user's visual input is controlled by a computing system. "Augmented reality" or "AR" refers to a system in which a user views real-world images after they have passed through a computing system. For example, a tablet with a camera on the back can capture images of the real world and then display the images on a screen on the side of the tablet opposite the camera. The tablet can process and adjust or "enhance" the images as they pass through the system (e.g., by adding virtual objects). "Mixed reality" or "MR" refers to a system in which the light entering the user's eyes is partially generated by a computing system and partially constitutes light reflected by objects in the real world. For example, an MR head-mounted viewer can be shaped as a pair of glasses with a see-through display that allows light from the real world to pass through a waveguide that simultaneously emits light from a projector in the MR head-mounted viewer, thereby allowing the MR head-mounted viewer to present virtual objects mixed with real objects that the user can see. As used herein, “artificial reality,” “hyperreality,” or “XR” refers to any of the following: VR, AR, MR, or any combination or hybrid thereof.
[0045] Traditional browsers cannot provide both a two-dimensional user experience and an immersive user experience and / or cannot provide an effective mechanism for transitioning between the two experiences. For example, traditional browsers that support immersive experiences typically rely on entry mechanisms (e.g., trigger buttons or interface components) to be defined and displayed by a web page. In other words, the entry point is variable based on where the website designer chooses to place the entry point and / or the way the web page is displayed in the browser. Due to this variability, users are often confused about how to trigger the immersive experience. In addition, traditional browsers lack a consistent mechanism to notify users that an immersive experience is available, loaded, and / or ready for interaction. Therefore, these browsers cannot provide an effective user interface.
[0046] Embodiments include an XR browser that permits configuration of the browser chrome component according to API calls by web code / web content. For example, once an immersive experience is loaded (e.g., once immersive resources defining the experience are preloaded), an API call to the XR browser may modify parameters of the browser chrome component. The modified parameters may modify the display of the browser chrome component, signal to a user that an immersive experience is available, and serve as an entry point to the immersive experience.
[0047] Embodiments of the configurable browser chrome component provide user notification that an immersive experience is part of a web page when the immersive experience is loaded, and embodiments of the configurable browser chrome component provide a consistent entry point for one or more immersive experiences. For example, multiple immersive experiences (e.g., across multiple third-party providers / designers) can support an entry point using the same (or similar) browser chrome component and functionality. This consistency enables an improved user interface and eliminates sources of user confusion.
[0048] Several embodiments are discussed in more detail below with reference to the accompanying drawings. Figure 1 is a block diagram showing an overview of devices on which some embodiments of the disclosed technology may run. These devices may include the following hardware components of the computing system 100: These hardware components trigger the immersive display at the XR browser. In various embodiments, the computing system 100 may include a single computing device 103 or multiple computing devices (e.g., computing device 101, computing device 102, and computing device 103), which communicate through a wired channel or a wireless channel to distribute processing and share input data. In some embodiments, the computing system 100 may include a standalone head-mounted viewer that is capable of providing a computer-created or enhanced experience to a user without the need for external processing or external sensors. In other embodiments, the computing system 100 may include multiple computing devices, such as a head-mounted viewer and a core processing component (e.g., a console, a mobile device, or a server system), wherein some processing operations are performed on the head-mounted viewer and other processing operations are offloaded to the core processing component. The following is combined with Figure 2A and Figure 2B An example head mounted view device is described. In some implementations, location data and environmental data may be collected solely by sensors incorporated into the head mounted view device, while in other implementations, one or more of the plurality of non-head mounted view computing devices may include sensor components that may track environmental data or location data.
[0049] The computing system 100 may include one or more processors 110 (e.g., a central processing unit (CPU), a graphical processing unit (GPU), a holographic processing unit (HPU), etc.). The processor 110 may be a single processing unit or multiple processing units, which are located in one device or distributed on multiple devices (e.g., distributed on two or more computing devices among the computing devices 101 to 103).
[0050] The computing system 100 may include one or more input devices 120 that provide input to the processors 110, thereby informing the processors of actions. These actions may be communicated by a hardware controller that interprets the signals received from the input devices and transmits the information to the processors 110 using a communication protocol. Each input device 120 may include, for example, a mouse, a keyboard, a touch screen, a touchpad, a wearable input device (e.g., a tactile glove, a bracelet, a ring, an earring, a necklace, a watch, etc.), a camera (or other light-based input device, such as an infrared sensor), a microphone, or other user input device.
[0051] The processor 110 can be coupled to other hardware devices, for example, by using an internal or external bus, or a wireless connection, such as a peripheral component interconnect standard (PCI) bus, a small computer system interface (SCSI) bus. The processor 110 can communicate with a hardware controller for each device (e.g., for a display 130). The display 130 can be used to display text and graphics. In some embodiments, the display 130 includes an input device, such as when the input device is a touch screen or is equipped with an eye movement direction monitoring system, the input device is part of the display. In some embodiments, the display is separated from the input device. Examples of display devices are: a liquid crystal display (LCD) display screen, a light emitting diode (LED) display screen, a projection display, a holographic display, or an augmented reality display (e.g., a head-up display device or a head-mounted device), etc. Other input / output (I / O) devices 140 may also be coupled to the processor, such as a network chip or network card, a video chip or video card, an audio chip or audio card, a universal serial bus (USB), FireWire or other external devices, a camera, a printer, a speaker, a compact disc read-only memory (CD-ROM) drive, a digital video disc (DVD) drive, a disk drive, etc.
[0052] In some embodiments, input from I / O devices 140 (e.g., cameras, depth sensors, inertial motion unit (IMU) sensors, GPS units, LiDAR or other time-of-flight sensors, etc.) can be used by computing system 100 to identify and map the user's physical environment and track the user's position within the environment. This simultaneous localization and mapping (SLAM) system can generate a map (e.g., topology, grid, etc.) of an area (which can be a room, building, outdoor space, etc.) and / or obtain a map previously generated by computing system 100 or another computing system that has mapped the area. The SLAM system can track users within the area based on factors such as GPS data, match identified objects and structures with mapped objects and structures, monitor acceleration and other position changes, etc.
[0053] The computing system 100 may include a communication device that can communicate wirelessly or wired with other local computing devices or network nodes. The communication device can communicate with another device or server over a network, for example, using a Transmission Control Protocol / Internet Protocol (TCP / IP) protocol. The computing system 100 can use the communication device to distribute operations across multiple network devices.
[0054] Processor 110 may access memory 150, which may be included in one of the computing devices of computing system 100, or may be distributed across the computing devices of computing system 100, or across multiple other external devices. Memory includes one or more hardware devices for volatile or non-volatile storage, and may include both read-only memory and writable memory. For example, memory may include one or more of the following: random access memory (RAM), various caches, CPU registers, read-only memory (ROM), and writable non-volatile memory, such as flash memory, hard drive, floppy disk, compact disk (CD), DVD, magnetic storage device, tape drive, etc. Memory is not a propagation signal that is separated from the underlying hardware; therefore, memory is non-transient. Memory 150 may include program memory 160 that stores programs and software, such as operating system 162, XR browser manager 164, and other applications 166. Memory 150 may also include data storage 170, which may include, for example, web content, JavaScript files, graphic files (e.g., bitmaps, portable network graphics (.png), joint photographic experts group (.jpeg), etc.), immersive content models (e.g., visual effects (VFX) models, three-dimensional models, mesh models, etc.), user data, avatar data, permission data and / or preference data, configuration data, settings, user options or preferences, etc., which may be provided to program memory 160 or any element of computing system 100.
[0055] Some embodiments may operate with many other computing system environments or configurations. Examples of computing systems, environments, and / or configurations suitable for use with the technology include, but are not limited to, XR headsets, personal computers, server computers, handheld or laptop devices, cellular phones, wearable electronic devices, game consoles, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers (PCs), minicomputers, mainframe computers, or distributed computing environments that include any of the above systems or devices, etc.
[0056] Figure 2A2 is a line diagram of a virtual reality head mounted display (HMD) 200 according to some embodiments. HMD 200 includes a front rigid body 205 and a band 210. Front rigid body 205 includes one or more electronic display elements of an electronic display 245, an inertial motion unit (IMU) 215, one or more position sensors 220, a locator 225, and one or more computing units 230. Position sensor 220, IMU 215, and computing unit 230 may be located inside HMD 200 and may not be visible to the user. In various embodiments, IMU 215, position sensor 220, and locator 225 may track the movement and position of HMD 200 in the real world and in an artificial reality environment with three degrees of freedom (3DoF) or six degrees of freedom (6DoF). For example, locator 225 may emit infrared beams that produce light spots on real objects around HMD 200. As another example, the IMU 215 may include, for example: one or more accelerometers; one or more gyroscopes; one or more magnetometers; one or more other non-camera-based position, force, or orientation sensors; or a combination thereof. One or more cameras (not shown) integrated with the HMD 200 may detect light points. The computing unit 230 in the HMD 200 may use the detected light points to infer the position and movement of the HMD 200, and to recognize the shape and position of real objects around the HMD 200.
[0057] The electronic display 245 can be integrated with the front rigid body 205 and can provide image light to the user as directed by the computing unit 230. In various embodiments, the electronic display 245 can be a single electronic display or multiple electronic displays (e.g., one display for each eye of the user). Examples of the electronic display 245 include: a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), a display including one or more quantum dot light-emitting diode (QOLED) sub-pixels, a projector unit (e.g., micro-LED, laser (LASER)), etc.), some other display, or some combination thereof.
[0058] In some embodiments, the HMD 200 can be coupled to a core processing component such as a personal computer (PC) (not shown) and / or one or more external sensors (not shown). The external sensors can monitor the HMD 200 (e.g., via light emitted from the HMD 200), and the PC can use the HMD in conjunction with the outputs from the IMU 215 and the position sensor 220 to determine the position and movement of the HMD 200.
[0059] Figure 2B 2 is a line diagram of a mixed reality HMD system 250 that includes a mixed reality HMD 252 and a core processing component 254. The mixed reality HMD 252 and the core processing component 254 can communicate via a wireless connection (e.g., a 60 Gigahertz (GHz) link) as indicated by link 256. In other embodiments, the mixed reality system 250 includes only a head mounted viewer without an external computing device, or includes other wired or wireless connections between the mixed reality HMD 252 and the core processing component 254. The mixed reality HMD 252 includes a see-through display 258 and a frame 260. The frame 260 can house various electronic components (not shown), such as a light projector (e.g., lasers, LEDs, etc.), a camera, an eye tracking sensor, a micro-electromechanical system (MEMS) component, a network component, etc.
[0060] The projector can be coupled to the see-through display 258, for example, via optical elements, to display media to the user. The optical elements can include one or more waveguide components, one or more reflectors, one or more lenses, one or more mirrors, one or more collimators, one or more gratings, etc. for directing light from the projector to the user's eyes. The image data can be transmitted from the core processing component 254 to the HMD 252 via the link 256. The controller in the HMD 252 can convert the image data into a plurality of light pulses from the projector, which can be transmitted to the user's eyes as output light via the optical elements. The output light can be mixed with the light passing through the display 258, thereby allowing the output light to present the following virtual objects: these virtual objects appear to exist in the real world.
[0061] Similar to HMD 200, HMD system 250 may also include motion and position tracking units, cameras, light sources, etc., which allow HMD system 250 to track itself, for example, with 3DoF or 6DoF, track multiple parts of the user (e.g., hands, feet, head, or other body parts), draw virtual objects to appear stationary when HMD 252 moves, and make virtual objects react to gestures and other real-world objects.
[0062] Figure 2C A plurality of controllers 270 (including controllers 276A and 276B) are shown, which in some embodiments can be held in one or both hands by a user to interact with an artificial reality environment presented by HMD 200 and / or HMD 250. Controllers 270 can communicate with the HMD directly or via an external device (e.g., core processing component 254). The controllers can have their own IMU units, position sensors, and / or can emit light points farther away. The HMD 200 or HMD 250, external sensors, or sensors in the controllers can track these controller light points to determine the position and / or orientation of the controllers (e.g., tracking the controllers with 3DoF or 6DoF). The computing unit 230 or core processing component 254 in the HMD 200 can combine the IMU output and the position output and use this tracking to monitor the position and movement of the user's hands. The controller may also include various buttons (e.g., buttons 272A-272F) and / or joysticks (e.g., joysticks 274A-274B) that a user may actuate to provide input and interact with objects.
[0063] In various embodiments, HMD 200 or HMD 250 may also include additional subsystems, such as an eye tracking unit, an audio system, various network components, etc., to monitor indications of user interaction and intent. For example, in some embodiments, instead of or in addition to a controller, one or more cameras included in HMD 200 or HMD 250 or from an external camera can monitor the position and posture of the user's hands to determine gestures and other hand and body movements. As another example, one or more light sources can illuminate one or both eyes of the user, and HMD 200 or HMD 250 can use eye-facing cameras to capture reflections of that light to determine eye position (e.g., based on a set of reflections around the user's cornea), model the user's eyes, and determine gaze direction.
[0064] Figure 3 305B may be a HMD 200 or a HMD system 250. The plurality of client computing devices 305 may be operated in a network environment using logical connections to one or more remote computers (e.g., server computing devices) through a network 330.
[0065] In some implementations, server 310 may be an edge server that receives multiple client requests and coordinates the fulfillment of these requests through other servers (e.g., servers 320A to 320C). Server computing devices 310 and 320 may include a computing system, such as computing system 100. Although each server computing device 310 and 320 is logically shown as a single server, each of these server computing devices may be a distributed computing environment that includes multiple computing devices located at the same physical location or at geographically different physical locations.
[0066] Client computing device 305, and server computing devices 310 and 320 can all act as servers or clients of one or more other servers / client devices. Server 310 can be connected to database 315. Servers 320A to 320C can each be connected to corresponding databases 325A to 325C. As discussed above, each server 310 or 320 can correspond to a group of servers, and each of these servers can share a database, or can have its own database. Although databases 315 and 325 are logically shown as a single unit, databases 315 and 325 can all be distributed computing environments that include multiple computing devices, can be located in their corresponding servers, or can be located at the same physical location or at geographically different physical locations.
[0067] The network 330 may be a local area network (LAN), a wide area network (WAN), a mesh network, a hybrid network, or other wired or wireless network. The network 330 may be the Internet or some other public or private network. The client computing device 305 may be connected to the network 330 via a network interface, for example, by wired or wireless communication. Although the connections between the server 310 and the plurality of servers 320 are shown as separate connections, these connections may be any type of local area network, wide area network, wired network, or wireless network including the network 330 or a separate public or private network.
[0068] Figure 44 is a block diagram showing a plurality of components 400 that can be used in a system that employs the disclosed technology in some embodiments. Each component 400 can be included in a device of the computing system 100, or can be distributed on multiple devices of the computing system 100. Component 400 includes hardware 410, middleware 420, and multiple dedicated components 430. As discussed above, a system implementing the disclosed technology can use various hardware, including a processing unit 412, a working memory 414, input and output devices 416 (e.g., a camera, a display, an IMU unit, a network connection, etc.) and a storage memory 418. In various embodiments, the storage memory 418 can be one or more of the following: a local device, an interface to a remote storage device, or a combination thereof. For example, the storage memory 418 can be one or more hard drives or flash drives accessible via a system bus, or can be a cloud storage provider (e.g., in memory 315 or 325) or other network memory accessible via one or more communication networks. In various implementations, component 400 may be implemented in a client computing device (eg, client computing device 305 ), or on a server computing device (eg, server computing devices 310 or 320 ).
[0069] Middleware 420 may include components that communicate resources between hardware 410 and a plurality of specialized components 430. For example, middleware 420 may include an operating system, services, drivers, a basic input output system (BIOS), controller circuits, or other hardware or software systems.
[0070] Multiple dedicated components 430 may include software or hardware that is configured to perform operations for triggering an immersive experience at an XR browser. Multiple dedicated components 430 may include: an XR browser 434; a preloader 436; a chrome controller 438; a panel controller 440; and components and APIs (e.g., interface 432) that can be used to provide a user interface, transfer data, and control dedicated components. In some embodiments, each component 400 may be in a computing system distributed across multiple computing devices, or may be an interface to a server-based application that executes one or more dedicated components 430. Although each dedicated component 430 is depicted as a separate module or code block.
[0071] The XR browser 434 can display web content, such as a two-dimensional web page or a three-dimensional immersive experience, at a user system. Embodiments of the XR browser 434 load web resources from a web host, a content distribution network, a cache, or any other suitable source of web content. The user system can be an XR system, a smartphone, or any other suitable client device capable of rendering an immersive experience (e.g., a three-dimensional environment).
[0072] The XR browser 434 may include at least two components. The first component may be a browser component including a browser chrome, and the second component may be a web display that displays web content. For example, the first component may include an interface for interacting with the browser itself and / or for configuring the browser itself. In some embodiments, the chrome controller 438 may control the browser chrome component of the XR browser 434. The second component may display received / acquired / stored web content at the XR browser 434. For example, the second component may include a panel window that displays a two-dimensional web page. In another example, the second component may include an immersive volume that displays a three-dimensional environment (e.g., a three-dimensional artificial reality environment) for an immersive experience.
[0073] In some embodiments, the XR browser 434 can run at least two parts of code. The first code portion can be browser code that implements functionality for the XR browser 434 itself (e.g., functionality of the first component of the XR browser 434), and the second code portion can be web code that implements web page / immersive experience functionality (e.g., functionality of the second component of the XR browser 434). The implementation of the browser code can be first-party code (e.g., local code at the user system, cloud-based code for the XR browser, or other suitable first-party code), and the implementation of the web code can be third-party code (e.g., code received / obtained from a web source). In some embodiments, the browser code can expose an API to the web code so that an API call from the web code can configure an element of the browser chrome (e.g., the first component of the XR browser 434). For example, after the web code loads resources that support an immersive experience, the web code can make an API call to configure an element (e.g., a button) at the browser chrome.
[0074] Preloader 436 can preload and / or dynamically load web resources, such as web resources that enable an immersive experience. In some embodiments, XR browser 434 can retrieve / receive one or more web pages from a web host / web source and display a web page, such as a login web page. For example, a user can navigate to a login web page (or any other suitable web page) while preloader 436 loads resources that support an immersive experience in the background.
[0075] The preloader 436 can be implemented by web code, a combination of browser code and web code, or any other suitable code. For example, one or more portions of the JavaScript code can load immersive resources that implement an immersive experience (e.g., obtaining immersive resources from a web source, configuring / initializing immersive resources using a script, etc.). Example immersive resources include JavaScript resources (e.g., files, scripts, etc.), graphic resources (.jpeg, .png, bitmaps, etc.), model resources (e.g., three-dimensional models, mesh models, vfx models, etc.), binary resources (e.g., executable files), or any other suitable immersive resources. Once the preloader 436 completes loading of a set of immersive resources associated with the immersive experience, portions of the web code can execute API calls to the browser code / chrome controller 438 to configure elements of the browser chrome, such as buttons or other suitable interface elements. For example, one or more "onload" methods implemented by the web code can return one or more indications that one or more resources have been loaded, and API calls can be executed in response to these one or more indications. In some implementations, the API call and resulting configuration of an element of the browser chrome (e.g., a button) can indicate to the user that the immersive experience is loaded and ready for user interaction.
[0076] The chrome controller 438 can control the chrome browser component of the XR browser 434. The browser chrome may include interface elements (e.g., buttons, menus, tabs, or other suitable interface elements) for configuring the XR browser 434. Example browser chrome elements include back, forward, and refresh buttons, an address bar, one or more bookmarks, one or more menus and / or panels (e.g., user preference interfaces, history interfaces, etc.), plugin elements / add-ons, and other suitable browser chrome elements. In some embodiments, the chrome controller 438 may expose an API so that web code can call the API and configure elements at the browser chrome.
[0077] For example, the chrome controller 438 may receive an API call and configure a button at the browser chrome based on the API call. In some embodiments, the web code may make an API call in response to the immersive resource being loaded and / or configured at the XR browser 434. The chrome controller 438 may set / modify the properties of the button according to the API call. In one example, the button may not have been previously displayed at the browser chrome, and the set / modified properties may cause the button to be displayed. In another example, the button may have been displayed in an inactive state (e.g., gray display, unable to receive push / press input), and the set / modified properties may cause the button to be activated (e.g., causing the color of the button to change, changing the function of the button so that push / press input can be received). In another case, the button may be visible when the website contains an immersive experience and may indicate the download progress of the immersive experience (e.g., a status bar), thereby becoming actionable to activate the immersive experience when the immersive experience content has been acquired. Any other suitable browser chrome elements and / or element properties may be configured by API calls.
[0078] In some embodiments, in response to input received at the browser chrome, the chrome controller 438 can cause an interface such as a menu or window to be displayed. The input received at the displayed interface can configure the immersive experience / three-dimensional environment. For example, the interface can request permissions from the user, such as permission to enter the immersive experience, permission to access one or more devices at the user system (e.g., microphones, cameras, controllers, etc.), or other suitable permissions. When the user grants permission, the web display controller 440 can transform the web display from a two-dimensional web page to an immersive experience, and the one or more devices can be accessed during the immersive experience according to the granted permissions.
[0079] In another example, the user can provide configuration input via the displayed interface. The configuration input can include a location within the immersive experience, a user presence (e.g., a user avatar) for the immersive experience, and other suitable configuration information. The web display controller 440 can transition the web display from a two-dimensional web page to an immersive experience based on the configuration input received at the interface. For example, the immersive experience can be displayed at a specified location within the three-dimensional environment, and / or the user presence during the immersive experience can be defined by a specified avatar / presence input.
[0080] The web display controller 440 can control the web display component of the XR browser 434. For example, the web display component can display a two-dimensional web page and / or a three-dimensional environment (e.g., an XR environment for an immersive experience). In some embodiments, the two-dimensional web page can be defined by received / acquired web page data (e.g., HTTP files, JavaScript files, images, etc.). In some embodiments, the three-dimensional environment can be defined by received / acquired immersive resources, such as one or more models (e.g., shells, vfx models, wireframes, etc.), graphic files (e.g., backgrounds, images scattered on wireframes, shells and / or models, etc.), code (e.g., JavaScript, binary files, etc.), and other suitable immersive resources.
[0081] The web display controller 440 can, for example, transition from displaying a two-dimensional web page to displaying a three-dimensional environment (for an immersive experience) at the web display component of the XR browser 434 in response to user input received at an element of the browser chrome (e.g., a button push / press, permission grant, etc.). In some implementations, the web display controller 440 can also transition from displaying a three-dimensional environment to displaying a two-dimensional display at the web display component of the XR browser 434 in response to user input.
[0082] Embodiments may improve transitions between two-dimensional display (e.g., web page display) and three-dimensional display (e.g., immersive experience) at an XR browser using an XR browser chrome component that may be configured through web code / web content via API calls to the XR browser. Figure 5 5 is a system diagram illustrating components of an artificial reality browser configured to trigger an immersive experience. System 500 includes an XR browser 502, a browser chrome 504, a web display component 506, one or more web resources 508, one or more immersive resources 510, and a chrome element 512.
[0083] The XR browser 502 may include a browser chrome 504 and a web display component 506. The browser chrome 504 may display interface elements (e.g., buttons, menus, forms, etc.) that configure the XR browser itself. The web display component 506 may display content received / obtained from a data network (e.g., the Internet), such as web pages, immersive experiences (e.g., three-dimensional environments), and / or other third-party web content.
[0084] In some implementations, the web display component 506 can display one or more web pages using one or more web resources 508 (e.g., HTTP files, JavaScript files, images, etc.). For example, one or more web resources 508 can be loaded at the XR browser 502 (e.g., received / obtained from a host service, a content delivery network, a cache, or any other suitable web content source), and a two-dimensional web page defined using the one or more web resources 508 can be displayed at the web display component 506.
[0085] While displaying one or more web pages at the web display component 506, embodiments of the XR browser 502 may preload (e.g., receive / acquire (from a host service, content delivery network, cache memory, or any other suitable source of web content), initialize, configure, or otherwise preload) one or more immersive resources 510. For example, in some instances where the one or more immersive resources 510 are larger in size than the one or more web resources 508 (and for other suitable reasons), the XR browser 502 may load the one or more web resources 508 faster than the one or more immersive resources 510. After loading the one or more web resources 508 and displaying the one or more web pages at the web display component 506, the XR browser 502 may preload the one or more immersive resources 510 in the background. In this example, the user may navigate and / or interact with the displayed web page while the one or more immersive resources 510 are loading.
[0086] Once the XR browser 502 has successfully loaded one or more immersive resources 510, the web code implementing the web page displayed at the web display component 506 can execute an API call to the XR browser 502. The API call can configure components of the browser chrome 504. For example, the API call can set properties of the chrome element 512. In some embodiments, the API call can set display properties of the chrome element 512 (e.g., change the display from hidden / not displayed to displayed), set the interaction settings of the chrome element (e.g., change an inactive button (i.e., a "greyed out" version) that cannot receive a press input to an active button), or configure the chrome element 512 in any other suitable manner. In some cases, the chrome element 512 can display an indication of the progress of loading the one or more immersive resources 510, such as by displaying the component 512 as a status bar indicating the number of preloaded one or more immersive resources 510.
[0087] Figure 5The chrome element 512 is shown as a button, however any other suitable element may be implemented. After the API call configures the chrome element 512, the display of the element may be modified. For example, the chrome element 512 may be displayed at the browser chrome 504 (where the element was not previously displayed), the chrome element 512 may undergo a color change (e.g., a grayed-out button to a colored button), the text at the chrome element 512 may be displayed / modified (e.g., blank text or text indicating that loading may be modified to text indicating that the immersive experience has loaded, such as "Loaded", "Go", or any other suitable text), or any other suitable display modification may occur. Modifications to the display of the chrome element 512 may indicate to the user that the immersive experience has loaded and is ready for user interaction.
[0088] In some embodiments, a user of the XR browser 502 may provide input at the chrome element 512 to trigger entry into the loaded immersive experience. For example, the chrome element 512 may be a button, and a button press action (after modification) may trigger the XR browser 502 to transition from a two-dimensional display of a web page to a three-dimensional display of an immersive experience. In some embodiments, the browser chrome 504 and the panel displaying the two-dimensional web page at the XR browser 502 may be hidden (e.g., not displayed) during the immersive experience / display of the three-dimensional environment.
[0089] In some implementations, a button press (or any other suitable input) at the (modified) chrome element 512 may cause the XR browser 502 to display one or more additional interfaces for configuring the immersive experience prior to entering. Fig. 6A and Figure 6B Schematic diagrams for configuring an immersive experience triggered using an artificial reality browser. Schematic diagrams 600A and 600B include an XR browser 502, a browser chrome 504, a web display component 506, a chrome element 512, a menu component 602, and a permission component 604.
[0090] Schematic 600A shows a menu component 602 displayed by the XR browser 502 in response to input received at the chrome element 512. The menu component 602 can display a plurality of menu items for configuring the immersive experience based on the user input. An example menu item can be a location within the immersive experience from which the immersive experience starts when loaded. In some embodiments, a drop-down menu (or other suitable pre-populated menu) including a set of locations within the immersive experience / three-dimensional environment can be displayed to the user, and the user can select one of these locations to configure the immersive experience.
[0091] Another example menu item may be user presence within the immersive experience. In some implementations, a drop-down menu (or other suitable pre-populated menu) including preconfigured / preloaded user presence options (e.g., avatars, default avatars for the immersive experience, custom avatars configured for / by the user, etc.) may be displayed to the user, and the user may select one of these user presence options to configure the immersive experience. In another implementation, the menu item may generate a web form that allows the user to select a file location (e.g., locally on the user's system, or in any other suitable location), and the selected file may define the user avatar. In this example, the user may upload a file that supports user presence in the immersive experience.
[0092] Diagram 600B shows that permissions component 604 is displayed by XR browser 502 in response to input received at chrome element 512. Permission component 604 can obtain input from a user to grant permissions related to an immersive experience. Example permissions include: a) permission to allow the user to enter an immersive experience; b) permission to access devices on the user's system (e.g., one or more cameras, one or more microphones, one or more sensors, or other suitable devices); c) permission to access the user's social media information; d) permission to allow other users within the immersive experience / three-dimensional environment to communicate and / or interact with the user; or e) any other suitable permissions.
[0093] Any other suitable one or more elements (e.g., display panels, prompts, etc.) may be implemented to receive configuration information and / or permissions from the user for an immersive experience. 7A to 7D As shown, once the immersive experience is configured with options and / or permissions, the XR browser 502 can transition from a two-dimensional display of a web page to a three-dimensional display of the immersive experience.
[0094] Fig. 7A , Figure 7B , Figure 7C and Fig.7DA series of schematic diagrams are shown that visualize an artificial reality browser configured to trigger an immersive experience. Each of the schematic diagrams 700A, 700B, and 700C depicts an artificial reality browser configured to trigger an immersive experience. Figure 5 The display at the XR browser 502 in FIG. Schematic diagram 700A, schematic diagram 700B and schematic diagram 700C include Figure 5 700A, a web page is displayed at the web display component 506. The browser chrome 504 displays an active chrome element 512, which in some embodiments indicates that the immersive resource has been preloaded by the XR browser 502.
[0095] Schematic 700B shows that a user interacting with a user system (e.g., an XR system) has selected / pressed chrome element 512. Schematic 700C shows that, in response, the XR browser 502 displays the permission component 604 in Figure 6. For example, the permission component can protect permissions from the user. Schematic 700D shows that in response to the user selecting the "forward" button at the permission component 604, a transition from a two-dimensional web page display to a three-dimensional immersive experience display is provided at the XR browser 502. For example, the XR browser 502 can display a three-dimensional environment to provide an immersive experience to the user. In some embodiments, the browser chrome 504 and / or the panel portion of the display component 506 (e.g., a panel display for a two-dimensional web page) is not displayed during the immersive experience.
[0096] It will be understood by those skilled in the art that the above Figures 1 to 5 , Fig. 6A , Figure 6B , Fig. 7A , Figure 7B , Figure 7C and Fig.7D The components shown in the flowcharts and the components shown in each of the flowcharts discussed below can be changed in various ways. For example, the logical order can be rearranged, multiple sub-steps can be performed in parallel, the logic shown can be omitted, other logic can be included, etc. In some embodiments, one or more of the above components can perform one or more of the processes described below.
[0097] Figure 8800 is a flow chart illustrating a process for triggering an immersive experience via an artificial reality browser in some embodiments of the present technology. In some embodiments, process 800 can be performed when a web page is loaded at an XR browser. Process 800 can load an immersive experience (e.g., a three-dimensional environment) and trigger a display transition from a two-dimensional web page to a three-dimensional immersive experience at the XR browser.
[0098] At block 802, process 800 may display the web page at the XR browser based on the received web resource defining the two-dimensional web page. For example, the XR browser may include a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content based on the web resource received / acquired by the XR browser. One or more web pages defined by web resources (e.g., HTTP pages, hypertext markup language (HTML), extensible markup language (XML), JavaScript, images, links, etc.) may be displayed, and these web resources are received / acquired by the XR browser, for example, from a host service, a content distribution network, a cache memory, or any other suitable web content source. In some embodiments, the displayed web page is a homepage or login page of a website.
[0099] At block 804, process 800 may preload immersive resources at the XR browser. In some embodiments, the immersive resources may include one or more JavaScript resources, graphics resources, model resources, binary resources, or any combination thereof. For example, web resources that define a displayed web page may be loaded faster than immersive resources that define an immersive experience / three-dimensional environment, so the immersive resources may be loaded in the background while the user is interacting with the web resources. The XR browser may preload immersive resources while the web page is being displayed to the user and / or while the user is interacting with or navigating the displayed web page.
[0100] At block 806, process 800 can determine whether an API call has been received. For example, a received web resource or preloaded immersive resource can issue the call to the XR browser, such as when a website begins loading an API call to indicate that the website has immersive content being preloaded, or once the immersive experience is loaded and / or initialized. In some implementations, the web page is implemented by web code, and once a set of immersive resources defining the immersive experience is identified or preloaded, the web code can issue the API call.
[0101] When an API call has been received, process 800 may proceed to block 810. When an API call has not been received, process 800 may proceed to block 808. At block 808, process 800 may continue to display the two-dimensional web page. For example, the XR browser may display the two-dimensional web page until process 800 determines at block 806 that an API call has been received (or the user navigates away from the current web page).
[0102] At block 810, process 800 may set properties of the browser chrome component in response to the API call. For example, the set properties may modify the display of the browser chrome component. In some embodiments, the modification of the display of the browser chrome component by the set properties transforms the browser chrome component from hidden to displayed. For example, the browser chrome component may be a button, and prior to the API call, the button may not be displayed by the browser chrome. In response to the API call and the set properties of the button, the XR browser may dynamically display the button at the browser chrome.
[0103] In some embodiments, the modification of the display of the browser chrome component by the set properties transforms the browser chrome component from a non-interactive component that is not configured to receive user input to an interactive component that is configured to receive user input. For example, the browser chrome component can be a button, and before the API call, the button can be displayed by the browser chrome in an inactive state (e.g., grayed out, no mouse hover display modification, etc.). In response to the API call and the set properties of the button, the XR browser can dynamically modify the display of the button to indicate an active state (e.g., modifying display color, text, activating mouse hover display modification, etc.). The browser chrome component associated with the API call can be any other suitable component.
[0104] At block 812, process 800 may determine whether input is received at the browser chrome component. For example, a user may interact with the browser chrome component (e.g., press a button, check a box, double-click, grab, etc.). When input is received at the browser chrome component, process 800 may proceed to block 814. When input is not received at the browser chrome component, process 800 may remain at block 812 until input is received at the browser chrome component. For example, the XR browser may continue to display a two-dimensional web page and a modified browser chrome component (e.g., a browser chrome component modified according to the properties set at block 810) until process 800 determines that input is received at the browser chrome component.
[0105] Although any block may be removed or rearranged in various embodiments, blocks 814 and 816 are shown in dashed lines to indicate that there is a specific instance of skipping blocks 814 and / or 816. At block 814, process 800 may display one or more permission prompts at the browser chrome. For example, these permission prompts may obtain user permissions for the immersive experience, such as permission to allow the user to enter the immersive experience, permission to access devices on the user's system (e.g., one or more cameras, one or more microphones, one or more sensors, or other suitable devices), permission to access the user's social media information, permission to allow other users within the immersive experience / three-dimensional environment to communicate and / or interact with the user, or any other suitable permissions.
[0106] At block 816, process 800 may determine whether permission to enter the immersive experience is received from the user. When permission is received from the user, process 800 may proceed to block 818. When permission is not received, process 800 may loop back to block 814 until user permission is received.
[0107] At block 818, process 800 may trigger a display transition from a two-dimensional web page to an immersive experience. The immersive experience may be displayed based on preloaded immersive resources. In some embodiments, the immersive experience includes a three-dimensional environment. In some embodiments, in response to input related to a permission prompt, a device implementing an XR browser at the client system is activated during the triggered immersive experience. For example, the device may include one or more of the following: a microphone, a camera, a controller, a sensor, or any combination thereof.
[0108] Fig. 9 900 is a flow chart illustrating a process for triggering an immersive experience configured with a menu at an artificial reality browser in some embodiments of the present technology. In some embodiments, process 900 can be performed when a web page is loaded at an XR browser. Process 900 can load and configure an immersive experience (e.g., a three-dimensional environment) and trigger a display transition from a two-dimensional web page to a three-dimensional immersive experience at the XR browser.
[0109] At block 902, process 900 may receive input at a browser chrome component. For example, the browser chrome component may be part of the browser chrome of an XR browser. An API call (made by web code implementing one or more web pages displayed at the XR browser) may modify the browser chrome component. For example, the API call may be performed when a set of immersive resources are preloaded at the XR browser. In some embodiments, the modification to the browser chrome component and the received input may be similar to Figure 8The browser chrome component and user input at blocks 810 and 812 in FIG.
[0110] At block 904, process 900 can display a menu in response to input received at the browser chrome component. For example, the menu can include configuration options for configuring the immersive experience / three-dimensional environment based on the user input. An example menu item can be an initial location within the immersive experience. In some implementations, a drop-down menu (or other suitable pre-populated menu) including a set of locations within the immersive experience / three-dimensional environment can be displayed to the user, and the user can select one of these locations to configure the immersive experience.
[0111] Another example menu item may be user presence within the immersive experience. In some implementations, a drop-down menu (or other suitable pre-populated menu) including preconfigured / preloaded user presence options (e.g., avatars, default avatars for the immersive experience, custom avatars configured for / by the user, etc.) may be displayed to the user, and the user may select one of these user presence options (or a set of avatar configuration options—e.g., style, face shape, skin color, etc.) to configure the immersive experience. In another implementation, the menu item may generate a web form (e.g., local to the user's system, cloud-based, or in any other suitable location) that allows the user to select an avatar-defining object location, and the selected avatar-defining object may control how the user appears in the immersive experience. In this example, the user's avatar is dynamically loaded at the XR browser for the immersive experience, and / or the user may upload an avatar-defining object that supports user presence in the immersive experience.
[0112] At block 906, process 900 may determine whether a menu response is received. For example, the user may select one or more options for an immersive experience, or choose not to make a selection at the displayed menu. When a menu response is received, process 900 may proceed to block 910. When a menu response is not received, process 900 may proceed to block 908.
[0113] At block 908, process 900 can select default configuration options for the immersive experience. For example, in the absence of a user selection, default location and / or default user presence options can configure the immersive experience. The default options can be generic default options for several users, or the default options can be default options stored for a particular user (e.g., based on stored historical data regarding the user's interactions with the immersive experience / three-dimensional environment).
[0114] At block 910, process 900 may configure the immersive experience according to the configuration options. For example, the user input may select the location of the immersive experience within the three-dimensional environment, the user presence for the immersive experience, or other suitable configuration options. In another example, when the user chooses not to make one or more selections, one or more default options may be selected for the immersive experience. Configuring the immersive experience may include loading / initializing the three-dimensional environment at the selected location, and / or loading / initializing the user presence selection / avatar. At block 912, process 900 may trigger a transition to the immersive experience according to the configuration. For example, the XR browser may display the user presence at a selected location in the three-dimensional environment that implements the immersive experience according to the selection.
[0115] References in this specification to "multiple embodiments" (e.g., "some embodiments," "various embodiments," "one embodiment," "an embodiment," etc.) mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present disclosure. The appearances of these phrases in different places in the specification do not necessarily all refer to the same embodiment, nor are they necessarily separate embodiments or alternative embodiments that are mutually exclusive of other embodiments. In addition, various features are described that may be rendered by some embodiments but not by other embodiments. Similarly, various requirements are described that may be requirements for some embodiments but not for other embodiments.
[0116] As used herein, being higher than a threshold value means that the value of a comparison item is higher than other values specified, the comparison item is located in a certain specified number of items with a maximum value, or the comparison item has a value within a specified maximum percentage value. As used herein, being lower than a threshold value means that the value of a comparison item is lower than other values specified, the comparison item is located in a certain specified number of items with a minimum value, or the comparison item has a value within a specified minimum percentage value. As used herein, being within a threshold value means that the value of a comparison item is between two other values specified, the comparison item is located in a middle specified number of items, or the comparison item has a value within a middle specified percentage range. When not otherwise defined, relative terms such as high or unimportant may be understood to assign a value and determine how the value is compared with the established threshold value. For example, the phrase "selecting a fast connection" may be understood to mean selecting a connection with a value corresponding to a connection speed assigned thereto that is higher than a threshold value.
[0117] As used herein, the word "or" refers to any possible arrangement of a group of items. For example, the phrase "A, B or C" refers to at least one of the following: A, B, C or any combination thereof, such as any of the following: A; B; C; A and B; A and C; B and C; A, B and C; or a plurality of any items (e.g., A and A; B, B and C; A, A, B, C and C, etc.).
[0118] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Specific embodiments and implementations have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of these embodiments and implementations, as defined in the appended claims. The above-mentioned specific features and actions are disclosed as example forms of implementing the appended claims. Therefore, the embodiments and implementations are not limited except in the appended claims.
Claims
1. A method for triggering an immersive experience at an artificial reality (XR) browser, the method include: Displaying the webpage at the XR browser according to the received web resource defining the two-dimensional webpage, wherein the XR browser includes a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content according to the web resource received by the XR browser; Preloading a plurality of immersive resources at the XR browser, the plurality of immersive resources comprising one or more JavaScript resources, graphic resources, model resources, binary resources, or any combination of the foregoing resources; receiving an application programming interface call (API) from the received web resource or the preloaded plurality of immersive resources indicating that the plurality of immersive resources have been preloaded; In response to the API call, setting a property of a browser chrome component, wherein the set property modifies a display of the browser chrome component; and Based on input related to the browser chrome component, triggering a display transition from the two-dimensional webpage to the immersive experience, wherein the immersive experience is displayed according to the preloaded plurality of immersive resources.
2. The method according to claim 1, in, The browser chrome component includes a button, and the input related to the browser chrome component includes a button press.
3. The method according to claim 1 or 2, in, The modification of the display of the browser chrome component by the set properties includes: a) changing the browser chrome component from hidden to displayed; or b) changing the browser chrome component from a non-interactive component not configured to receive user input to an interactive component configured to receive user input.
4. A method according to any preceding claim, further comprising: include: Based on the input related to the browser chrome component, displaying a menu at the browser chrome, the menu including parameters for the immersive experience; Optionally, at least one parameter of the menu includes a position parameter, and the triggered display position within the immersive experience is defined by input related to the position parameter of the menu.
5. The method according to claim 4, in, at least one parameter of the menu includes a user presence parameter, and based on input related to the user presence parameter of the menu, a user avatar is included in the triggered immersive experience; Optionally, wherein, in response to the input relating to the user presence parameter of the menu, the user avatar is dynamically loaded at the XR browser for the immersive experience.
6. A method according to any preceding claim, further comprising: include: Based on the input related to the browser chrome component, displaying one or more permission prompts at the browser chrome, wherein in response to the input related to the one or more permission prompts, activating a device implementing the XR browser at a client system during the triggered immersive experience; Optionally, the device includes one or more of the following: a microphone, a camera, a controller, a sensor, or any combination thereof.
7. A method according to any preceding claim, in, The immersive experience includes a three-dimensional environment.
8. A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for triggering an immersive experience at an artificial reality (XR) browser, the process include: Displaying the webpage at the XR browser according to the received web resource defining the two-dimensional webpage, wherein the XR browser includes a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content according to the web resource received by the XR browser; Preloading a plurality of immersive resources at the XR browser; receiving an application programming interface call (API) from the received web resource or the preloaded plurality of immersive resources indicating that the plurality of immersive resources have been preloaded; In response to the API call, setting a property of a browser chrome component, wherein the set property modifies a display of the browser chrome component; and Based on input related to the browser chrome component, triggering a display transition from the two-dimensional webpage to the immersive experience, wherein the immersive experience is displayed according to the preloaded plurality of immersive resources.
9. The computer-readable storage medium according to claim 8, in, The browser chrome component includes a button, and the input related to the browser chrome component includes a button press.
10. The computer-readable storage medium according to claim 8 or 9, in, The modification of the display of the browser chrome component by the set properties includes: a) changing the browser chrome component from hidden to displayed; or b) changing the browser chrome component from a non-interactive component not configured to receive user input to an interactive component configured to receive user input.
11. The computer-readable storage medium according to any one of claims 8 to 10, in, The process also includes: Based on the input related to the browser chrome component, displaying a menu at the browser chrome, the menu including parameters for the immersive experience; Optionally, at least one parameter of the menu includes a position parameter, and the triggered display position within the immersive experience is defined by input related to the position parameter of the menu.
12. The computer-readable storage medium according to claim 11, in, at least one parameter of the menu includes a user presence parameter, and based on input related to the user presence parameter of the menu, a user avatar is included in the triggered immersive experience; Optionally, wherein, in response to the input relating to the user presence parameter of the menu, the user avatar is dynamically loaded at the XR browser for the immersive experience.
13. The computer-readable storage medium according to any one of claims 8 to 12, in, The process also includes: Based on the input related to the browser chrome component, one or more permission prompts are displayed at the browser chrome, wherein in response to the input related to the one or more permission prompts, a device implementing the XR browser at the client system is activated during the triggered immersive experience, wherein the device includes one or more of the following: a microphone, a camera, a controller, a sensor, or any combination thereof.
14. The computer-readable storage medium according to any one of claims 8 to 13, in, The immersive experience includes a three-dimensional environment.
15. A computing system for triggering an immersive experience at an artificial reality (XR) browser, the computing system include: one or more processors; as well as One or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising: Displaying the webpage at the XR browser according to the received web resource defining the two-dimensional webpage, wherein the XR browser includes a browser chrome and a web component, the browser chrome displays an interface of the XR browser, and the web component displays web content according to the web resource received by the XR browser; Preloading a plurality of immersive resources at the XR browser; receiving an application programming interface call (API) from the received web resource or the preloaded plurality of immersive resources indicating that the plurality of immersive resources have been preloaded; In response to the API call, setting a property of a browser chrome component, wherein the set property modifies a display of the browser chrome component; and Based on input related to the browser chrome component, triggering a display transition from the two-dimensional webpage to the immersive experience, wherein the immersive experience is displayed according to the preloaded plurality of immersive resources.
