Selective boundaries for applications executing in artificial reality environment

Abstract

The present disclosure provides selective boundaries for applications executing in an artificial reality environment. Instead of having a constant boundary, an artificial reality (XR) application that can run in a borderless mode (e.g., in augmented reality or mixed reality) may be controlled by a selective boundary system on the XR system to customize partial boundaries. For example, when certain object types are in a real environment, an application may specify selection of a particular boundary. In another example, an application may transition to a different boundary mode when certain application events occur (e.g., when transitioning from virtual reality to mixed reality). The selective boundary system may provide an application programming interface (API) at the back-end that shows boundary information to an application. Based on these information, the application may use the API to create new boundaries, or may open or close certain boundaries based on the application's requests.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 585,450, filed on September 26, 2023, entitled “Selective Guardians for an Application Executing in an Artificial Reality Environment,” and U.S. Non-Provisional Application No. 18 / 810,209, filed on August 20, 2024, which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure is directed to providing selective control over the boundaries of an artificial reality (XR) environment for extra reality (XR) applications. Background Art

[0004] Artificial reality (XR) devices are becoming more and more common. As XR devices become more popular, the applications implemented on such devices are becoming more and more complex. Mixed reality (MR) applications and augmented reality (AR) applications can provide interactive three-dimensional (3D) experiences that combine real-world images with virtual objects, while virtual reality (VR) applications can provide a completely independent 3D computer environment. For example, MR applications or AR applications can be used to superimpose virtual objects on a real scene observed through a camera. Then, a real-world user in the scene can make a gesture that provides interactivity between the real-world user and the virtual object, which is captured by the camera. Users can observe AR, MR, and VR (collectively referred to as XR) experiences through a head-mounted display (HMD) (e.g., glasses or a head-mounted viewer (headset)). Some HMDs may have a pass-through display that allows light from the real world to pass through a lens to combine with light from a waveguide that simultaneously emits light from a projector in the HMD, allowing the HMD to present virtual objects mixed with real objects that the user can actually see. Summary of the invention

[0005] Various aspects of the present disclosure relate to selective boundaries for applications executed in an artificial reality (XR) environment. As used herein, a "boundary" (interchangeably referred to herein as a "guardian") can be an XR use space defined in a real-world environment. The XR use space can have one or more of such boundaries, which can trigger a warning or other system action if a user of the XR system crosses one or more of these boundaries when accessing an XR experience, as further described herein. In some embodiments, a boundary can be created by indicating an area (e.g., a line around a set of stairs), and the boundary can be added / triggered when located in a relevant area (e.g., three feet from the created line). BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a block diagram showing an overview of a device on which some embodiments of the present technology may operate.

[0007] Figure 2A is a line diagram illustrating a virtual reality head mounted viewer that may be used in some embodiments of the present technology.

[0008] Figure 2B is a line diagram illustrating a mixed reality head mounted viewer that may be used in some embodiments of the present technology.

[0009] Figure 2C is a line diagram illustrating a plurality of controllers that, in some implementations, a user may hold in one or both hands to interact with an artificial reality environment.

[0010] Figure 3 is a block diagram illustrating an overview of an environment in which some embodiments of the present technology may operate.

[0011] Figure 4 is a block diagram illustrating various components that may be used in a system employing the disclosed technology in some implementations.

[0012] Figure 5 is a flow chart illustrating a process for providing selective boundaries for applications executing in an artificial reality (XR) environment in some embodiments of the present technology.

[0013] Fig. 6A is a conceptual diagram illustrating an example view of a real-world environment surrounding a user of an XR device performing a virtual reality (VR) experience, indicating all enabled boundaries of the XR device.

[0014] Figure 6Bis a conceptual diagram illustrating an example view of a real-world environment surrounding a user of an artificial reality (XR) device performing a mixed reality (MR) experience or an augmented reality (AR) experience, indicating mandatory enabled boundaries of the XR device.

[0015] Figure 6C is a conceptual diagram illustrating an example view of a real-world environment surrounding a user of an artificial reality (XR) device performing a mixed reality (MR) experience or an augmented reality (AR) experience, indicating selectively enabled boundaries of the XR device.

[0016] The technology described herein may be better understood by referring to the following detailed description in conjunction with the accompanying drawings, in which like reference numbers refer to the same elements or elements with similar functions. DETAILED DESCRIPTION

[0017] Various aspects of the present disclosure relate to selective boundaries for applications executed in an artificial reality (XR) environment. As used herein, a "boundary" (interchangeably referred to herein as a "protected area") can be an XR use space defined in a real-world environment. The XR use space can have one or more of such boundaries, which can trigger a warning or other system action if a user of the XR system crosses one or more of these boundaries when accessing an XR experience, as further described herein. In some embodiments, a boundary can be created by indicating an area (e.g., a line around a set of stairs), and the boundary can be added / triggered when located in a relevant area (e.g., three feet from the created line).

[0018] According to some embodiments of the present technology, instead of having constant boundaries, XR applications can run in a borderless mode (e.g., mixed reality mode or augmented reality mode). Such applications can be given control by a selective boundary system on the XR device to customize partial boundaries. For example, when certain object types (e.g., stairs, walls, etc.) are in the real environment, the application can specify to establish or ignore partial boundaries. Thus, the selective boundary system can provide boundaries and types (including scene-based boundaries) for custom areas, which, for example, allow the application to ignore the boundaries of the table in an MR experience where a virtual object is placed on the table. In another example, when certain application events occur, the application can switch to a different boundary mode (e.g., from a full boundary mode to a partial boundary when changing from virtual reality to mixed reality).

[0019] The selective boundary system can provide an application programming interface (API) on the back end to the application that displays boundary information. Based on this information, the application can activate the option of the selective boundary system to open and close certain boundaries based on the application's request. However, the selective boundary system can act as the final authority on whether it is safe to close a specific boundary (i.e., the system can have some default mandatory boundaries), and in some embodiments, these specific boundaries can correspond to specific scene elements and / or specific boundary types (e.g., mandatory boundaries correspond to stairs).

[0020] Embodiments of the disclosed technology may include an artificial reality system, or may be implemented in conjunction with an artificial reality system. Artificial reality or hyperreality (XR) is a form of reality that has been adjusted in some way before being presented to a user, and artificial reality or hyperreality may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content, or generated content combined with collected content (e.g., real-world photos). Artificial reality content may include video, audio, tactile feedback, or some combination thereof, any of which may be presented in a single channel or multiple channels (e.g., stereoscopic video that gives the viewer a three-dimensional effect). In addition, in some embodiments, artificial reality may be associated with an application, product, accessory, service, or some combination thereof, which is used, for example, to create content in an artificial reality, and / or to be used in an artificial reality (e.g., to perform activities in an artificial reality). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a stand-alone HMD, a mobile device or computing system, a "cave-like" environment or other projection system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0021] As used herein, "virtual reality" or "VR" refers to an immersive experience in which the user's visual input is controlled by a computing system. "Augmented reality" or "AR" refers to a system in which a user views images of the real world after they have passed through a computing system. For example, a tablet with a camera on the back can capture images of the real world and then display them on a screen on the side of the tablet opposite the camera. The tablet can process and adjust or "enhance" the images as they pass through the system, for example by adding virtual objects. "Mixed reality" or "MR" refers to a system in which the light entering the user's eyes is generated in part by a computing system and in part from light reflected from objects in the real world. For example, an MR head-mounted viewer can be shaped as a pair of glasses with a transparent display that allows light from the real world to pass through a waveguide that simultaneously emits light from a projector in the MR head-mounted device, thereby allowing the MR head-mounted device to present virtual objects mixed with real objects that the user can see. As used herein, “artificial reality,” “hyperreality,” or “XR” refers to any of VR, AR, MR, or any combination or hybrid thereof.

[0022] The embodiments described herein provide specific technical improvements in the field of artificial reality. According to some embodiments, the XR system may scan the real-world space to automatically identify one or more boundaries corresponding to the real-world space (e.g., walls, physical objects, etc.), wherein the area within the boundary corresponds to an available area in which the XR system can access the XR environment. The XR system may display such boundaries to an XR application executed in the XR environment, and allow the application to request to selectively enable or disable at least some of the one or more boundaries. For example, in order to execute a VR environment, an application may request to enable all available boundaries, thereby preventing the user from colliding with physical objects when his or her field of view is blocked. In another example, in order to execute an MR environment or an AR environment, an application may request to disable some or all boundaries because the user runs the XR system in a transparent mode or a perspective mode and is able to visually see obstacles in the real-world environment. In some cases, the XR system may enforce one or more mandatory boundaries regardless of the application's request, such as those boundaries that exceed a threshold value and cause a risk of injury to the user (e.g., stairs going down, dark areas where objects may not be seen in transparent transmission, etc.). In some implementations, the XR system can seamlessly interact with applications in the background to selectively enable and disable such boundaries, without requiring user input or manual settings to select mandatory boundaries and / or optional boundaries.

[0023] Thus, the embodiments described herein can improve latency and the overall user experience by minimizing or eliminating manual boundary acquisition, setting, and / or specification, and reducing latency when rendering XR experiences. In addition, the XR system can save resources (e.g., processing power, battery power, etc.) required to activate and enforce all boundaries throughout the execution of the XR environment by allowing boundaries to be selectively enabled or disabled as needed or desired. By providing boundaries for the XR environment, and in some embodiments, by enforcing mandatory boundaries (regardless of any requests by the application to disable such boundaries), the XR system can reduce the likelihood, risk, and / or occurrence of collisions and / or other accidents (e.g., falling down stairs) between users of the XR system and physical objects in the real-world space.

[0024] Several embodiments are discussed in more detail below with reference to the accompanying drawings. Figure 1 is a block diagram illustrating an overview of various devices on which some embodiments of the disclosed technology may run. These devices may include multiple hardware components of a computing system 100 that may provide selective protection zones for applications executed in an artificial reality (XR) environment. In various embodiments, the computing system 100 may include a single computing device 103 or multiple computing devices (e.g., computing device 101, computing device 102, and computing device 103) that communicate via a wired channel or a wireless channel to distribute processing and share input data. In some embodiments, the computing system 100 may include a standalone head-mounted viewer that is capable of providing a computer-created or enhanced experience to a user without the need for external processing or external sensors. In other embodiments, the computing system 100 may include multiple computing devices, such as a head-mounted viewer and a core processing component (e.g., a console, a mobile device, or a server system), wherein some processing operations are performed on the head-mounted viewer and other processing operations are transferred to the core processing component. The following is combined with Figure 2A and Figure 2B An example head mounted view device is described. In some implementations, location data and environmental data may be collected solely by sensors incorporated into the head mounted view device, while in other implementations, one or more of a plurality of non-head mounted view computing devices may include sensor components that may track environmental data or location data.

[0025] The computing system 100 may include one or more processors 110 (e.g., a central processing unit (CPU), a graphical processing unit (GPU), a holographic processing unit (HPU), etc.). The processor 110 may be a single processing unit or multiple processing units, which are located in one device or distributed on multiple devices (e.g., distributed on two or more computing devices among the computing devices 101 to 103).

[0026] The computing system 100 may include one or more input devices 120 that provide input to the processors 110, thereby informing the processors of actions. These actions may be communicated by a hardware controller that interprets signals received from the input devices and transmits information to the processor 110 using a communication protocol. Each input device 120 may include, for example, a mouse, a keyboard, a touch screen, a touchpad, a wearable input device (e.g., a tactile glove, a bracelet, a ring, an earring, a necklace, a watch, etc.), a camera (or other light-based input device, such as an infrared sensor), a microphone, or other user input device.

[0027] The processor 110 can be coupled to other hardware devices, for example, by using an internal bus or an external bus or a wireless connection, such as a peripheral component interconnect standard (PCI) bus, a small computer system interface (SCSI) bus. The processor 110 can communicate with the hardware controller of each device (e.g., a display 130). The display 130 can be used to display text and graphics. In some embodiments, the display 130 includes an input device, such as when the input device is a touch screen or is equipped with an eye movement direction monitoring system, the input device is part of the display. In some embodiments, the display is separated from the input device. Examples of display devices include liquid crystal display (LCD) display screens, light emitting diode (LED) display screens, projection displays, holographic displays, or augmented reality displays (e.g., head-up display devices or head-mounted devices), etc. Other input / output (I / O) devices 140 may also be coupled to the processor, such as a network chip or card, a video chip or card, an audio chip or card, a universal serial bus (USB), FireWire or other external devices, a camera, a printer, a speaker, a compact disc read-only memory (CD-ROM) drive, a digital video disc (DVD) drive, a disk drive, etc.

[0028] In some embodiments, the computing system 100 can use input from the I / O device 140 (e.g., a camera, a depth sensor, an inertial motion unit (IMU) sensor, a GPS unit, a LiDAR or other time-of-flight sensor, etc.) to identify and map the user's physical environment and track the user's position within the environment. A simultaneous localization and mapping (SLAM) system can generate a map (e.g., topology, grid, etc.) of an area (which can be a room, a building, an outdoor space, etc.) and / or obtain a map previously generated by the computing system 100 or another computing system that has mapped the area. The SLAM system can track users in the area based on factors such as GPS data, match identified objects and structures with mapped objects and structures, monitor acceleration and other position changes, etc.

[0029] The computing system 100 may include such a communication device: the communication device can communicate with other local computing devices or network nodes wirelessly or based on wired communication. The communication device can communicate with another device or server through a network, for example, using the Transmission Control Protocol / Internet Protocol (TCP / IP) protocol. The computing system 100 can use the communication device to distribute operations on multiple network devices.

[0030] Processor 110 may access memory 150, which may be included in one of the computing devices of computing system 100, or may be distributed across the computing devices of computing system 100, or other external devices. Memory includes one or more hardware devices for volatile storage or non-volatile storage, and may include both read-only memory and writable memory. For example, memory may include one or more of the following: random access memory (RAM); various cache memories; CPU registers; read-only memory (ROM) and writable non-volatile memory, such as flash memory, hard drive, floppy disk, compact disk (CD), DVD, magnetic storage device, tape drive, etc. Memory is not a propagation signal that is separated from the underlying hardware; therefore, memory is non-transitory. Memory 150 may include program memory 160, which stores programs and software, such as operating system 162, selective boundary system 164, and other application programs 166. The memory 150 may also include a data memory 170, which may include, for example, protected area data, boundary data, restriction data, application data, request data, scene data, object type data, boundary type data, rendering data, configuration data, settings, user options or preferences, etc., which may be provided to the program memory 160 or any element of the computing system 100.

[0031] In various embodiments, the techniques described herein may include a non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform the steps shown and described herein. In various embodiments, the techniques described herein may include a computing system that includes one or more processors and one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform the steps shown and described herein.

[0032] Some embodiments may operate with many other computing system environments or configurations. Examples of computing systems, environments, and / or configurations suitable for use with the present technology include, but are not limited to, XR headsets, personal computers, server computers, handheld or laptop devices, cellular phones, wearable electronic devices, game consoles, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, networked personal computers (PCs), minicomputers, mainframe computers, or distributed computing environments that include any of the above systems or devices, etc.

[0033] Figure 2A 2 is a line diagram of a virtual reality head mounted display (HMD) 200 according to some embodiments. In this example, the virtual reality HMD 200 also includes augmented reality features that use a pass-through camera 225 to render portions of the real world, which augmented reality features may have computer-generated overlays. The virtual reality HMD 200 includes a front rigid body 205 and a strap 210. The front rigid body 205 includes one or more electronic display elements of one or more electronic displays 245, an inertial motion unit (IMU) 215, one or more position sensors 220, a camera and positioner 225, and one or more computing units 230. The position sensors 220, the IMU 215, and the computing unit 230 may be located internal to the virtual reality HMD 200 and may not be visible to the user. In various embodiments, the IMU 215, the position sensor 220, and the camera and positioner 225 can track the movement and position of the virtual reality HMD 200 in the real world and artificial reality environment with three degrees of freedom (3DoF) or six degrees of freedom (6DoF). For example, the positioner 225 can emit infrared beams that generate light spots on real objects around the virtual reality HMD 200, and / or the camera 225 captures images of the real world and positions the virtual reality HMD 200 within the real world environment. As another example, the IMU 215 can include, for example, one or more accelerometers, one or more gyroscopes, one or more magnetometers, one or more other non-camera-based position, force or orientation sensors, or a combination thereof, which can be used in the positioning process. One or more cameras 225 integrated with the virtual reality HMD 200 can detect the light spots. The calculation unit 230 in the virtual reality HMD 200 may use the detected light points and / or position points to infer the position and movement of the virtual reality HMD 200 and recognize the shapes and positions of real objects around the virtual reality HMD 200 .

[0034] One or more electronic displays 245 can be integrated with the front rigid body 205 and can provide image light to the user as directed by the computing unit 230. In various embodiments, the electronic display 245 can be a single electronic display or multiple electronic displays (e.g., one display for each eye of the user). Examples of electronic displays 245 include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a display including one or more quantum dot light-emitting diode (QOLED) sub-pixels, a projector unit (e.g., micro-LED, laser (LASER)), some other display, or some combination thereof.

[0035] In some embodiments, the virtual reality HMD 200 may be coupled to a core processing component such as a personal computer (PC) (not shown) and / or one or more external sensors (not shown). The external sensors may monitor the virtual reality HMD 200 (e.g., via light emitted from the virtual reality HMD 200), which the PC may use in conjunction with outputs from the IMU 215 and the position sensor 220 to determine the position and movement of the virtual reality HMD 200.

[0036] Figure 2B 2 is a line diagram of a mixed reality HMD system 250, which includes a mixed reality HMD 252 and a core processing component 254. The mixed reality HMD 252 and the core processing component 254 can communicate via a wireless connection (e.g., a 60 GHz link) as indicated by link 256. In other embodiments, the mixed reality HMD system 250 includes only a head-mounted viewer without an external computing device, or includes other wired or wireless connections between the mixed reality HMD 252 and the core processing component 254. The mixed reality HMD 252 includes a transparent display 258 and a frame 260. The frame 260 can accommodate various electronic components (not shown), such as light projectors (e.g., lasers, LEDs, etc.), cameras, eye tracking sensors, micro-electromechanical systems (MEMS) components, network components, etc.

[0037] The projector can be coupled to the transparent display 258, for example, via an optical element to display media to the user. The optical element may include one or more waveguide components, one or more reflectors, one or more lenses, one or more mirrors, one or more collimators, one or more gratings, etc. for guiding the light from the projector to the user's eyes. The image data from the core processing component 254 can be transmitted to the mixed reality HMD 252 via the link 256. The controller in the mixed reality HMD 252 can convert the image data into a plurality of light pulses from the projector, which can be transmitted to the user's eyes as output light via the optical element. The output light can be mixed with the light passing through the transparent display 258, thereby allowing the output light to present virtual objects as follows: these virtual objects look as if they exist in the real world.

[0038] Similar to the virtual reality HMD 200, the mixed reality HMD system 250 may also include motion and position tracking units, cameras, light sources, etc., which allow the mixed reality HMD system 250 to track itself, for example, with 3DoF or 6DoF, track multiple parts of the user (e.g., hands, feet, head, or other body parts), draw virtual objects to appear stationary when the mixed reality HMD 252 moves, and make virtual objects react to gestures and other real-world objects.

[0039] Figure 2C A plurality of controllers 270 (including controllers 276A and 276B) are shown, which in some embodiments can be held by a user with one or both hands to interact with an artificial reality environment presented by the virtual reality HMD 200 and / or the mixed reality HMD system 250. The controllers 270 can communicate directly with the HMD or can communicate with the HMD via an external device (e.g., the core processing component 254). The controllers can have their own IMU units, position sensors, and / or can emit additional light points. The virtual reality HMD 200 or the mixed reality HMD system 250, external sensors, or sensors in the controllers can track these controller light points to determine the controller position and / or orientation (e.g., to track the controllers with 3DoF or 6DoF). The computing unit 230 or the core processing component 254 in the virtual reality HMD 200 can combine the IMU output and the position output and use this tracking to monitor the user's hand position and movement. The controllers may also include various buttons (eg, buttons 272A-272F) and / or levers (eg, levers 274A and 274B) that a user may actuate to provide input and interact with objects.

[0040] In various embodiments, the virtual reality HMD 200 or mixed reality HMD system 250 may also include a number of additional subsystems for monitoring user interactions and indications of intent, such as an eye tracking unit, an audio system, various network components, etc. For example, in some embodiments, instead of or in addition to a controller, one or more cameras included in the virtual reality HMD 200 or mixed reality HMD system 250 or from an external camera can monitor the position and posture of the user's hands to determine gestures and other hand and body movements. As another example, one or more light sources can illuminate either or both of the user's eyes, and the virtual reality HMD 200 or mixed reality HMD system 250 can use eye-facing cameras to collect reflections of the light to determine eye position (e.g., based on a set of reflections around the user's cornea) to model the user's eyes and determine the gaze direction.

[0041] Figure 3 305B may be a virtual reality HMD 200 or a mixed reality HMD system 250. The client computing device 305 may be a network environment using a logical connection to one or more remote computers (e.g., a server computing device) through a network 330.

[0042] In some implementations, server computing device 310 may be an edge server that receives client requests and coordinates fulfillment of these requests through other servers (e.g., server computing devices 320A-320C). Server computing devices 310 and 320 may include a computing system, such as computing system 100. Although each server computing device 310 and 320 is logically shown as a single server, each of these server computing devices may be a distributed computing environment that includes multiple computing devices located at the same physical location or at geographically different physical locations.

[0043] Client computing device 305, and server computing devices 310 and 320 can all act as servers or clients of one or more other server / client devices. Server computing device 310 can be connected to database 315. Server computing devices 320A to 320C can each be connected to corresponding databases 325A to 325C. As described above, each server computing device 310 or 320 can correspond to a group of servers, and each of these servers can share a database or can have its own database. Although databases 315 and 325 are logically shown as a single unit, databases 315 and 325 can each be a distributed computing environment containing multiple computing devices, which can be located in their corresponding servers, or can be located at the same physical location or at geographically different physical locations.

[0044] The network 330 may be a local area network (LAN), a wide area network (WAN), a mesh network, a hybrid network, or other wired or wireless network. The network 330 may be the Internet or some other public or private network. The client computing device 305 may be connected to the network 330 via a network interface, for example, by wired or wireless communication. Although the connections between the server 310 and the plurality of servers 320 are shown as separate connections, these connections may be any type of local area network, wide area network, wired network, or wireless network including the network 330 or a separate public or private network.

[0045] Figure 44 is a block diagram showing a plurality of components 400 that may be used in a system employing the disclosed technology in some embodiments. The plurality of components 400 may be included in one device of the computing system 100, or may be distributed across multiple devices of the computing system 100. The plurality of components 400 include hardware 410, middleware 420, and a plurality of specialized components 430. As discussed above, a system implementing the disclosed technology may use various hardware including a processing unit 412, a working memory 414, input and output devices 416 (e.g., a camera, a display, an IMU unit, a network connection, etc.), and a storage memory 418. In various embodiments, the storage memory 418 may be one or more of the following: a local device; an interface to a remote storage device; or a combination thereof. For example, the storage memory 418 may be one or more hard drives or flash drives accessible via a system bus, or may be to a cloud storage provider (e.g., in a database 315 or 325) or other network storage device, which may be accessed via one or more communication networks. In various implementations, the plurality of components 400 may be implemented in a client computing device (eg, client computing device 305 ), or on a server computing device (eg, server computing devices 310 or 320 ).

[0046] Middleware 420 may include components that transfer resources between hardware 410 and a plurality of specialized components 430. For example, middleware 420 may include an operating system, services, drivers, a basic input output system (BIOS), controller circuits, or other hardware or software systems.

[0047] The plurality of specialized components 430 may include software or hardware configured to perform operations that provide selective boundaries for applications executed in an artificial reality (XR) environment. The plurality of specialized components 430 may include: a boundary identification module 434; a boundary display module 436; a selective boundary enforcement module 438; and components and APIs (e.g., interface 432) that may be used to provide a user interface, transfer data, and control the specialized components. In some embodiments, the plurality of components 400 may be located in a computing system distributed across multiple computing devices, or may be an interface to a server-based application that executes one or more of the plurality of specialized components 430. Although the plurality of specialized components 430 are depicted as separate components, the specialized components may be logical or other non-physical distinctions of functionality, and / or may be sub-modules or code blocks of one or more applications.

[0048] The boundary identification module 434 can identify one or more boundaries that can be used for XR applications. The one or more boundaries can be depicted in the real-world environment and correspond to the following physical objects in the real-world space: these physical objects will block the movement of users accessing the XR environment in the real-world space. These physical objects can include any stationary or movable objects in the real-world environment, such as walls, stairs, furniture, etc. In some embodiments, for example, when one or more boundaries are previously created by the user (or another user) through one or more XR systems and stored locally or in the cloud, the boundary identification module 434 can identify these boundaries by accessing pre-created and stored spatial data corresponding to the real-world space.

[0049] In some embodiments, boundary recognition module 434 can create one or more boundaries of the real-world space by one or more methods. For example, in some embodiments, boundary recognition module 434 can create one or more boundaries by automatically generating an XR space model or mesh corresponding to the real-world space, as further described herein. In some embodiments, boundary recognition module 434 can identify these boundaries based on one or more boundaries manually created by a user through the XR system, such as by using a controller (e.g., Figure 2C These boundaries are identified by using light cast by controllers 276A and / or 276B) to outline available space in the real-world environment (e.g., open ground areas that are surrounded by walls and do not include physical objects).

[0050] The one or more boundaries may define one or more limitations for the XR environment. In some embodiments, all of the one or more boundaries may define the same one or more limitations, while in other embodiments, some of the one or more boundaries may define limitations that are different from other boundaries of the one or more boundaries. The one or more limitations may include actions that may be taken when the XR system is inside or outside of the one or more boundaries. For example, from the one or more boundaries inward, a user may access and / or execute the XR environment without restriction. In another example, when approaching a threshold distance of one or more boundaries and / or crossing one or more boundaries, the XR system may limit access to the XR environment and / or limit functionality of the XR environment, for example, by pausing rendering of the XR environment, activating pass-through on the XR system, displaying a warning on the XR system, displaying at least a portion of one or more boundaries on the XR system, and the like. References herein Figure 5 Block 502 of FIG. 5 describes further details regarding identifying one or more boundaries that may be used for an application.

[0051] Boundary display module 436 can display the one or more boundaries identified by boundary identification module 434 to applications executing in the XR environment by making the one or more boundaries accessible to the application (e.g., via an application programming interface (API) call). By making such API calls and accessing such boundaries, the application can perform various functions related to the XR environment, such as by rendering virtual objects within these boundaries and / or relative to the one or more boundaries. Figure 5 Block 504 of describes further details regarding exposing one or more boundaries to an application executing in the XR environment.

[0052] The selective boundary enforcement module 438 can selectively enable or disable at least some of the boundaries based at least in part on one or more requests received by interacting with an application to which one or more boundaries are displayed via the boundary display module 436. In some embodiments, the application can make one or more API calls to the selective boundary enforcement module 438 to request to enable or disable at least some of the one or more boundaries. For example, when the application is running in mixed reality (MR) mode or augmented reality (AR) mode and the user can see the one or more boundaries, the selective boundary enforcement module 438 can disable these boundaries in the XR environment based on a request from the application. In another example, the selective boundary enforcement module 438 can selectively disable the boundary based on the application's request to allow the user to interact with a specific physical object. For example, the application can request to selectively disable the boundary corresponding to the physical table so as to allow the user to interact with the virtual object (e.g., virtual keyboard) rendered on the physical table without restriction in the MR mode or AR mode. In some embodiments, as further described herein, the application can request to selectively enable or disable the boundary based on the type of scene element corresponding to the physical object in the real world environment. In some embodiments, such as when the XR system is operating in VR mode and the user may not be able to see physical obstructions in the real-world environment, the selective boundary enforcement module 438 can enforce one or more mandatory boundaries of the XR environment regardless of any request by the application to disable such boundaries. Figure 5 Block 506 of further describes more details regarding selectively enabling or disabling at least some of the one or more boundaries based at least in part on one or more requests received through interaction with the application.

[0053] Those skilled in the art will appreciate that the above Figures 1 to 4The components shown in the flowcharts and in each of the flowcharts discussed below may be changed in various ways. For example, the order of the logic may be rearranged, sub-steps may be performed in parallel, the logic shown may be omitted, other logic may be included, etc. In some embodiments, one or more of the above-mentioned multiple components may perform one or more of the processes described below.

[0054] Figure 5 1 is a flow chart illustrating a process 500 for providing selective boundaries for an application executing in an artificial reality (XR) environment in some embodiments. In some embodiments, the process 500 can be performed in response to an application requesting a boundary corresponding to a real-world environment. In some embodiments, the process 500 can be performed automatically when an application is launched. In some embodiments, at least a portion of the process 500 can be performed by an XR system, which can include one or more XR devices and / or one or more other XR devices within the XR system, the one or more XR devices being, for example, an XR HMD (e.g., Figure 2A Virtual Reality HMD 200 and / or Figure 2B 252), the one or more other XR devices are, for example, external processing components, one or more controllers (e.g., Figure 2C In some embodiments, process 500 may be performed by controller 276A and / or 276B. Figure 1 The selective boundary system 164 is executed.

[0055] At block 502, process 500 may identify one or more boundaries available for an application. The one or more boundaries may be depicted in a real-world environment surrounding an XR system and may define one or more limits for the XR environment. In some implementations, process 500 may identify one or more boundaries available for an application based on a type of application (e.g., a virtual reality (VR) application or a mixed reality (MR) application / augmented reality (AR) application).

[0056] In some embodiments, process 500 may identify one or more boundaries available to an application by, for example, accessing previously created and stored boundaries associated with a real-world environment in which the XR system is located. In some embodiments, process 500 may create one or more boundaries for the application "on demand" when the application is executed. In some embodiments, process 500 may create one or more boundaries of the real-world environment by applying computer vision techniques to identify walls, ceilings, floors, etc. in one or more images captured by the XR system. In some embodiments, process 500 may also create one or more boundaries of the real-world environment by, for example, identifying physical objects (e.g., tables, chairs, etc.) in the real-world environment that should be included in or excluded from the boundaries by applying object recognition techniques and / or accessing previously labeled scene data associated with these physical objects. In some embodiments, process 500 may automatically create one or more boundaries by using a depth sensor and / or imaging device to identify the maximum boundaries of the real-world environment surrounding the XR device. Further details are described in U.S. patent application Ser. No. 18 / 771,009, filed on July 12, 2024, entitled “Automatic Boundary for an Artificial Reality Environment,” which is incorporated herein by reference in its entirety.

[0057] In some embodiments, process 500 can automatically determine one or more boundaries, for example, by applying a machine learning model to data collected by the XR system. For example, process 500 can identify one or more boundaries based on an XR space model generated for a real-world environment. The XR space model (interchangeably referred to herein as a "room box") can indicate where walls, floors, and / or ceilings exist in the real-world space. In some embodiments, process 500 can automatically obtain the XR space model. For example, as described above, a user of the XR system can use one or more cameras and / or one or more depth sensors to scan the real-world space by moving and / or looking around the real-world space using the XR system, and can use such images and / or depth data to automatically identify one or more flat surfaces (e.g., walls, floors, ceilings) in the real-world space. For example, process 500 can identify flat surfaces by analyzing images and / or depth data of larger areas that have the same color, have depths that continuously increase and / or decrease relative to the XR system, and / or have a specific orientation (e.g., above, below, or around the XR system), etc.

[0058] In some implementations, process 500 can be performed at least in part by detecting (e.g., via one or more cameras of the XR system and / or an external device) one or more controllers (e.g., Figure 2C The process 500 may identify one or more boundaries (e.g., manually or through an XR space model) based on the position of the controller 276A and / or controller 276B) and / or the position of a tracked hand or other body part. For example, a user of the XR system may move a controller or body part around in the real-world environment to, for example, outline a floor using light projected from a controller or finger. In another example, a user of the XR system may place a controller or body part on the floor to identify one or more boundaries based on the position of the controller or body part (e.g., detected by one or more cameras on the XR system, detected by one or more sensors of the IMU). In some embodiments, the process 500 may automatically identify walls, which may then be refined (if necessary) by one or more controllers or body parts, such as by clicking on a wall plane or pinching and dragging the wall plane toward or away from the user, or by placing a controller or hand on a wall creeper. In some embodiments, the user may manually adjust the boundaries of the created real-world environment, such as by clicking on a controller and dragging the controller relative to the boundary and / or pinching and grabbing the boundary with a hand detected by hand tracking technology on the XR system. In some embodiments, the application can create additional boundaries, for example, by indicating a specific location of the new boundary, a specific object type around which the new boundary should be placed, a specific user action or gesture to create a corresponding boundary, etc. More details on identifying and realigning an XR room (including wall planes) are described in U.S. patent application Ser. No. 18 / 346,379, filed on July 3, 2023, entitled “Artificial Reality Room Capture Realignment,” which is incorporated herein by reference in its entirety.

[0059] In some embodiments, process 500 can automatically create a guard zone by identifying a maximum boundary of the real-world environment around the XR device using a depth sensor and / or imaging device. Further details are described in U.S. patent application Ser. No. 63 / 585,360, filed on Sep. 26, 2023, entitled “AUTOMATIC GUARDIAN FOR AN ARTIFICIAL REALITY ENVIRONMENT,” which is incorporated herein by reference in its entirety.

[0060] In some embodiments, process 500 can automatically identify one or more boundaries based on a three-dimensional (3D) mesh generated by scanning a real-world environment. The mesh can be, for example, a three-dimensional (3D) model of the boundary of a real-world space (including one or more walls, ceilings, floors, and / or one or more physical objects, etc.). In some embodiments, process 500 can generate a mesh using one or more cameras, one or more depth sensors, or any combination thereof. However, in some embodiments, it is considered that depth data does not need to be collected, but can instead be predicted from one or more images, for example, by a machine learning model. In some embodiments, process 500 can also perform post-processing on the mesh to refine and / or simplify the mesh. More details about generating and using XR space models and meshes are described in U.S. patent application No. 18 / 454,349, entitled “Assisted Scene Capture for an Artificial Reality Environment,” filed on August 23, 2023, which is incorporated herein by reference in its entirety.

[0061] At block 504, process 500 may expose the one or more boundaries to an application executing in the XR environment. For example, process 500 may make the one or more boundaries accessible and usable by the application, such as through an application programming interface (API) call. Through the API call, the application may perform various functions with respect to the one or more boundaries, such as positioning a virtual object within and / or relative to the one or more boundaries.

[0062] At box 506, process 500 may selectively enable or disable at least some of the one or more boundaries based at least in part on one or more requests received through interaction with the application. For example, through an API call, the application may request to enable or disable a specific boundary based on the XR environment to be presented by the application, based on features of the real-world environment, etc. In some embodiments, the application may request to disable unnecessary boundaries and / or boundaries that may affect the user experience in the XR environment. For example, in some embodiments, process 500 may determine that an element boundary (i.e., a boundary in the one or more boundaries) corresponding to a scene element (e.g., a physical object marked in the real-world environment) is necessary or desired for the application to render the XR environment. Therefore, process 500 may disable the element boundary so that the application can render the XR environment relative to the scene element and / or allow interaction with the scene element.

[0063] In some embodiments, the application may request to enable element boundaries based on the type of scene element (e.g., enabling element boundaries around specific marker objects (e.g., doorways) that are not used for rendering and / or accessing the XR environment). In some embodiments, process 500 may selectively enable at least some of one or more boundaries based on the XR system being within a threshold distance of the element boundaries. For example, process 500 may cause physical objects outside the threshold distance to have no boundaries, but may activate boundaries around some or all physical objects within a threshold distance (e.g., 1 meter).

[0064] In some embodiments, selectively enabling or disabling at least a portion of one or more boundaries can be based at least in part on one or more events that occur during execution of the application. For example, in VR mode, process 500 can enable all of the above boundaries. However, based on the application switching from VR mode to MR mode, process 500 can allow the application to request to disable one or more boundaries, such as boundaries corresponding to physical objects that the user can visually see when transmitting (e.g., objects within the user's field of view). In some embodiments, selectively enabling or disabling at least a portion of a boundary can be based on an application-level event that modifies the rendering of an artificial reality environment, such as a virtual ball bouncing in a real-world environment, where the application can use enabled boundaries to display the virtual ball's interaction with physical objects and / or users in an otherwise blocked area.

[0065] In some embodiments, the process 500 may enforce at least one of the one or more boundaries. In the above example, when the application is executed in VR mode, the process 500 may force all of the above boundaries to be enabled because the user's entire view of the real world environment may be blocked by the VR environment. In another example, the process 500 may force specific boundaries necessary to ensure safety, such as boundaries corresponding to physical objects that may be damaged and / or cause harm when crossing the boundary. In some embodiments, the process 500 may enforce such a mandatory boundary regardless of any request received by the application to disable such a boundary. For example, the process 500 may enforce a mandatory boundary corresponding to a staircase because a virtual object rendered by the application in the XR environment may block the user's view of the danger, and / or the user's safety is more important than any improved user experience provided by disabling the boundary. In some embodiments, the process 500 may enforce all boundaries as mandatory boundaries by default when the application is launched, unless or until the application requests to disable one or more boundaries during execution.

[0066] As described above, the enabled boundaries may define one or more limits for the XR environment. For example, after the user of the XR system, together with the XR device and / or one or more detected body parts, is within a threshold distance from the boundary and / or crosses the boundary, the process 500 may take one or more actions. In some embodiments, these actions may include activating a warning overlaid on the XR environment (e.g., "You are too close to the wall!") or an instruction to move back away from one or more boundaries and move within the one or more boundaries (e.g., "Back off!"). In some embodiments, these actions may include pausing at least one of the execution, updating, rendering, or any combination of execution, updating, and rendering of the XR environment. For example, if the user leaves the boundary and enters a room unknown to the XR system and / or application, the process 500 may stop executing and / or rendering the XR environment. In some embodiments, these actions may include removing one or more virtual objects from the XR environment (e.g., stopping rendering virtual objects associated with the same or different applications outside the boundary). In some embodiments, these actions may include, for example, activating at least partial pass-through on the XR device when approaching one or more boundaries in VR mode. In some implementations, the actions may include displaying at least one of the one or more boundaries (e.g., the boundary being approached or all of the boundaries) on the XR device so that the user can see the location of the one or more boundaries and move away from the one or more boundaries.

[0067] Fig. 6A 6 is a conceptual diagram as follows: The conceptual diagram shows an example view 600A of a real-world environment 602 surrounding a user 604 of an artificial reality (XR) system 606 performing a virtual reality (VR) experience, indicating all enabled boundaries 608A-608C, 610, 614 (collectively forming a protected area) of the XR system 606. In some embodiments, the XR system 606 can enforce all of the enabled boundaries 608A-608C, 610, 614 based on the application executing in VR mode. For example, boundaries 608A-608C can correspond to walls in the real-world environment 602, boundary 610 can correspond to a sofa 612 in the real-world environment 602, and boundary 614 can correspond to a table 616 in the real-world environment 602. Therefore, although the user 604 of the XR system 606 is fully immersed in the computer-generated view, the user 604 can be prevented from colliding with the walls, sofa 612, and table 616 corresponding to boundaries 608A-608C.

[0068] Figure 6B6 is a conceptual diagram showing an example view 600B of a real-world environment 602 surrounding a user 604 of an artificial reality (XR) system 606 performing a mixed reality (MR) experience or an augmented reality (AR) experience, indicating mandatory enabled boundaries 608A-608C (together forming a protected area) of the XR system 606. In some embodiments, an application can be in VR mode (in VR mode, Fig. 6A In some embodiments, the application may request to disable all non-mandatory boundaries and / or explicitly disable boundaries 610, 614 without switching from VR mode to AR mode or MR mode (i.e., the application may start executing in AR mode or MR mode and / or execute only in AR mode or MR mode). In any embodiment, for the safety of the user 604, the XR system 606 may keep the boundaries 608A to 608C enabled. However, boundaries 610, 614 may be disabled. Therefore, in some embodiments, the application may render virtual objects relative to the sofa 612 and / or table 616 in the XR environment, and / or may allow the user 604 to access and view the XR environment from the sofa 612 and / or table 616 without restriction.

[0069] Figure 6C 6 is a conceptual diagram showing an example view 600C of a real-world environment 602 surrounding a user 604 of an artificial reality (XR) system 606 performing a mixed reality (MR) experience or an augmented reality (AR) experience, indicating selectively enabled boundaries 608A-608C, 610 (together forming a protected area) of the XR system 606. In some embodiments, an application executing on the XR system 606 can selectively enable or disable the protected area when switching from VR mode to MR mode or AR mode. Fig. 6A In some embodiments, the application can start in MR mode or AR mode without switching from VR mode and, after starting, selectively enable or disable one or more of the boundaries 608A to 608C, 610, 614. In one example, the boundaries 608A to 608C can be reference Figure 6B610 , 614 can be selectively enabled or disabled. In some embodiments, boundaries 608A to 608C, 610, 614 can be selectively enabled or disabled based on system-level events or application-level events. For example, boundary 610 of sofa 612 can be selectively enabled when the user of XR system 606 is within a threshold distance from boundary 610 and / or sofa 612. In another example, when an application wishes to render a virtual object that "touches" and "interacts" with table 616 in an artificial reality environment (e.g., a virtual chess board placed on table 616 that user 604 can interact with), boundary 614 can be selectively disabled upon request of the application.

[0070] Several embodiments of the disclosed technology are described above with reference to the accompanying drawings. The computing device on which the described technology can be implemented may include one or more central processing units, memories, input devices (e.g., keyboards and pointing devices), output devices (e.g., display devices), storage devices (e.g., disk drives), and network devices (e.g., network interfaces). The memories and storage devices are computer-readable storage media that can store instructions for implementing at least part of the technology. In addition, data structures and message structures can be stored or transmitted via data transmission media (e.g., signals on a communication link). Various communication links can be used, such as the Internet, a local area network, a wide area network, or a point-to-point dial-up connection. Therefore, computer-readable media may include computer-readable storage media (e.g., "non-transient" media) and computer-readable transmission media.

[0071] References in this specification to "multiple embodiments" (e.g., "some embodiments," "various embodiments," "one embodiment," "an embodiment," etc.) mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present disclosure. The appearances of these phrases in different places in the specification do not necessarily all refer to the same embodiment, nor do they necessarily refer to separate embodiments or alternative embodiments that are mutually exclusive of other embodiments. In addition, various features are described that may be present in some embodiments but not in other embodiments. Similarly, various requirements are described that may be requirements of some embodiments but not in other embodiments.

[0072] As used herein, being higher than a threshold value means that the value of the item being compared is higher than other values ​​specified, the item being compared is located in an item with a maximum value, a certain specified number, or the item being compared has a value within a specified maximum percentage value. As used herein, being lower than a threshold value means that the value of the item being compared is lower than other values ​​specified, the item being compared is located in an item with a minimum value, a certain specified number, or the item being compared has a value within a specified minimum percentage value. As used herein, being located within a threshold value means that the value of the comparison item is between two other values ​​specified, the comparison item is located in an item of a middle specified number, or the value of the comparison item is within a middle specified percentage range. In the absence of additional definition, relative terms (such as high or unimportant) can be understood as assigning a value and determining how the value is compared with a given threshold value. For example, the phrase "selecting a fast connection" can be understood to mean selecting a connection with a value corresponding to the connection speed assigned to it that is higher than a threshold value.

[0073] As used herein, the word "or" refers to any possible arrangement of a group of items. For example, the phrase "A, B or C" refers to at least one of A, B, C, or any combination thereof, such as any of the following: A; B; C; A and B; A and C; B and C; A, B and C; or any multiple of any item, such as A and A; B, B and C; A, A, B, C and C; etc.

[0074] Although the subject matter has been described in language specific to structural features and / or method actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Specific embodiments and implementations have been described herein for illustrative purposes, but various modifications may be made without departing from the scope of these embodiments and implementations. The specific features and actions described above are disclosed as example forms of implementing the appended claims. Therefore, the embodiments and implementations are not limited except as limited by the appended claims.

[0075] Any patent, patent application and other references mentioned above are incorporated herein by reference. If necessary, various aspects may be modified to provide further embodiments using the systems, functions and concepts of the various references mentioned above. If the statements or themes in the documents incorporated by reference conflict with the statements or themes of the present application, the present application shall prevail.

Claims

1. A method for providing a selective boundary for an application executed in an artificial reality environment, the method comprising: identifying, by the artificial reality system, one or more boundaries that can be used for the application, wherein the one or more boundaries are depicted in a real world environment and define one or more limits for the artificial reality environment; exposing the one or more boundaries to the application executing in the artificial reality environment; and selectively enabling or disabling at least some of the one or more boundaries based at least in part on one or more requests received through interaction with the application, Wherein, interacting with the application includes forcibly enabling at least one of the one or more boundaries.

2. The method according to claim 1, wherein: At least some of the one or more boundaries of the application are created in response to commands from the application indicating: a specific location for a new boundary; and / or a specific type of object around which the new boundary should be placed.

3. The method according to claim 1, further comprising: Switching from a mixed reality environment to a virtual reality environment; as well as In response to the transition, all of the one or more boundaries are enabled.

4. The method according to claim 1, further comprising: determining that at least one element boundary of the one or more boundaries corresponds to a scene element requested by the application for rendering the artificial reality environment; as well as The at least one element border is disabled, wherein the application renders the artificial reality environment relative to the scene element.

5. The method according to claim 1, further comprising: determining that at least one element boundary of the one or more boundaries corresponds to a scene element having an object type, Wherein, selectively enabling or disabling the at least part of the one or more boundaries comprises: enabling the at least one element boundary based at least in part on the object type.

6. The method according to claim 5, wherein: Enabling the at least one element boundary is also based at least in part on the position of the artificial reality system being within a threshold distance from the at least one element boundary.

7. The method according to claim 1, wherein: Selectively enabling or disabling at least a portion of the one or more boundaries includes enabling at least a portion of the one or more boundaries based at least in part on a position of the artificial reality system being within a threshold distance from at least a portion of the one or more boundaries.

8. The method according to claim 1, wherein: Selectively enabling or disabling the at least some of the one or more boundaries is based at least in part on one or more events occurring in the application during execution.

9. The method according to claim 1, wherein: The one or more restrictions include at least one of the following: activating a warning overlaid on the artificial reality environment; pausing at least one of execution, updating, rendering, or any combination of execution, updating, rendering of the artificial reality environment; removing one or more virtual objects from the artificial reality environment; activating at least partial pass-through on the artificial reality system; displaying at least one of the one or more boundaries; or Any combination of the above.

10. A computer-readable storage medium storing instructions for providing a selective protection zone for an application executed in an artificial reality environment, the instructions, when executed by a computing system, causing the computing system to: identifying, by the artificial reality system, one or more boundaries that can be used for the application, in, The one or more boundaries are depicted in a real-world environment and define one or more limits for the artificial reality environment; exposing the one or more boundaries to the application executing in the artificial reality environment; as well as At least some of the one or more boundaries are selectively enabled or disabled based at least in part on one or more requests received through interaction with the application.

11. The computer-readable storage medium of claim 10, wherein: Interacting with the application includes forcibly enabling at least one of the one or more boundaries.

12. The computer-readable storage medium of claim 10, wherein: At least some of the one or more boundaries of the application are created in response to commands from the application indicating: a specific location for a new boundary; and / or a specific type of object around which the new boundary should be placed.

13. The computer-readable storage medium of claim 10, wherein: When the instructions are executed by the computing system, the computing system further causes: Switching from a mixed reality environment to a virtual reality environment; and In response to the transition, all of the one or more boundaries are enabled.

14. The computer-readable storage medium of claim 10, wherein: When the instructions are executed by the computing system, the computing system further causes: determining that at least one element boundary of the one or more boundaries corresponds to a scene element requested by the application for rendering the artificial reality environment; as well as The at least one element border is disabled, wherein the application renders the artificial reality environment relative to the scene element.

15. A computing system for providing a selectively protected area for an application executing in an artificial reality environment, the computing system comprising: one or more processors; as well as one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to: identifying, by the artificial reality system, one or more boundaries that can be used for the application, wherein the one or more boundaries are depicted in a real world environment and define one or more limits for the artificial reality environment; exposing the one or more boundaries to the application executing in the artificial reality environment; as well as At least some of the one or more boundaries are selectively enabled or disabled based at least in part on one or more requests received through interaction with the application.

16. The computing system of claim 15, wherein: Interacting with the application includes forcibly enabling at least one of the one or more boundaries.

17. The computing system of claim 15, wherein: The instructions, when executed by the one or more processors, further cause the computing system to: determining that at least one element boundary of the one or more boundaries corresponds to a scene element having an object type, Wherein, selectively enabling or disabling the at least part of the one or more boundaries comprises: enabling the at least one element boundary based at least in part on the object type.

18. The computing system of claim 17, wherein: Enabling the at least one element boundary is also based at least in part on the position of the artificial reality system being within a threshold distance from the at least one element boundary.

19. The computing system of claim 15, wherein: Selectively enabling or disabling at least a portion of the one or more boundaries includes enabling at least a portion of the one or more boundaries based at least in part on a position of the artificial reality system being within a threshold distance from at least a portion of the one or more boundaries.

20. The computing system of claim 15, wherein: The one or more restrictions include at least one of the following: activating a warning overlaid on the artificial reality environment; pausing at least one of execution, updating, rendering, or any combination of execution, updating, rendering of the artificial reality environment; removing one or more virtual objects from the artificial reality environment; activating at least partial pass-through on the artificial reality system; displaying at least one of the one or more boundaries; or Any combination of the above.

Images