Virtual Private Space for Augmented Reality

Abstract

A system, method, and non-transitory medium for generating a virtual private space for an extended reality (XR) experience are provided. An exemplary method can include initiating a virtual session for presenting virtual content and identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content. The method can include outputting boundary information defining a boundary of the virtual private space and generating at least a portion of the virtual content for the virtual private space. At least a portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE

[0001] The present disclosure relates generally to generating virtual content in virtual private spaces for extended reality and other applications. [Background technology]

[0002] Augmented reality technology can be used to present virtual content to a user and / or combine real and virtual environments from the physical world to provide a user with an augmented reality experience. The term augmented reality can encompass virtual reality, augmented reality, mixed reality, etc. Each of these forms of augmented reality allows a user to experience or interact with an immersive virtual environment or content. For example, an augmented reality experience can allow a user to interact with a real or physical environment that has been augmented or augmented with virtual content.

[0003] Augmented reality technologies can be implemented to enhance user experiences in a wide range of contexts, including entertainment, healthcare, retail, education, and social media, among others. Summary of the Invention

[0004]

[0004] Systems, apparatus, methods, and computer-readable media for generating virtual content in a virtual private space for an augmented reality experience are disclosed. According to at least one example, a method for generating virtual content for one or more virtual private spaces is provided. The method includes initiating, by a device, a virtual session for presenting the virtual content; identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information defining a boundary of the virtual private space; generating at least a portion of the virtual content for the virtual private space; and configuring at least a portion of the virtual content to be viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

[0005] In another example, an apparatus for generating virtual content for one or more virtual private spaces is provided, the apparatus including a memory configured to store virtual content data and one or more processors coupled to the memory (e.g., implemented in a circuit). The processor is configured to initiate a virtual session for presenting the virtual content, identify, for the virtual session, a portion of a physical space to use as a virtual private space for presenting at least a portion of the virtual content, output boundary information defining a boundary of the virtual private space, generate at least a portion of the virtual content for the virtual private space, and configure the at least a portion of the virtual content to be viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

[0006]

[0006] In another example, a non-transitory computer-readable medium is provided that stores instructions that, when executed by one or more processors, cause the one or more processors to: initiate a virtual session for presenting virtual content; identify a portion of physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; output boundary information defining boundaries of the virtual private space; generate at least a portion of the virtual content for the virtual private space; and configure the at least a portion of the virtual content to be viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

[0007] In another example, an apparatus for generating virtual content for one or more virtual private spaces is provided, the apparatus including means for initiating, by a device, a virtual session for presenting the virtual content, means for identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content, means for outputting boundary information defining a boundary of the virtual private space, and means for generating at least a portion of the virtual content for the virtual private space, wherein at least a portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

[0008] In another example, a method for generating virtual content is provided. The method includes: identifying, by one or more processors, a portion of a physical space for use as a virtual private space for presenting virtual content for a virtual session; generating, by the one or more processors, boundary information defining a boundary of the virtual private space, the boundary information being associated with boundary virtual content that identifies the boundary for the virtual private space; generating, by the one or more processors, the virtual content for the virtual private space; displaying, by a first augmented reality viewing device, the virtual content for the virtual session, the first augmented reality viewing device being configured to receive and display the virtual content; and displaying, by a second augmented reality viewing device, the second augmented reality viewing device being configured to receive the boundary information and display the boundary virtual content. In some cases, the second augmented reality viewing device does not display the virtual content in the virtual private space.

[0009] In another example, an augmented reality system is provided. The augmented reality system includes one or more processors configured to: identify a portion of a physical space for use as a virtual private space for presenting virtual content for a virtual session; generate boundary information defining a boundary of the virtual private space, the boundary information being associated with boundary virtual content that identifies the boundary for the virtual private space; and generate virtual content for the virtual private space. The augmented reality system further includes a first augmented reality viewing device, the first augmented reality viewing device being authorized to display and / or view the virtual content for the virtual session, the first augmented reality viewing device being configured to receive and display the virtual content. The augmented reality system further includes a second augmented reality viewing device, the second augmented reality viewing device being not authorized to display and / or view the virtual content, the second augmented reality viewing device being configured to receive the boundary information and display the boundary virtual content, and not display the virtual content in the virtual private space. In some examples, the one or more processors are part of the first augmented reality viewing device.

[0010] In another example, a non-transitory computer-readable medium for an augmented reality system is provided having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to: identify a portion of a physical space for use as a virtual private space for presenting virtual content for a virtual session by the one or more processors; generate boundary information defining a boundary of the virtual private space, the boundary information being associated with boundary virtual content that identifies the boundary for the virtual private space; and generate virtual content for the virtual private space. A non-transitory computer-readable medium for a first augmented reality viewing device is provided having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to display virtual content for the virtual session, the first augmented reality viewing device being configured to receive and display the virtual content. A non-transitory computer-readable medium for a second augmented reality viewing device is provided, the non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to display virtual content, the second augmented reality viewing device being configured to receive boundary information and display boundary virtual content. In some cases, the second augmented reality viewing device does not display the virtual content in the virtual private space.

[0011] In another example, an augmented reality system is provided. The augmented reality system includes: means for identifying, by one or more processors, a portion of a physical space for use as a virtual private space for presenting virtual content for a virtual session; means for generating boundary information defining a boundary of the virtual private space, the boundary information being associated with boundary virtual content identifying the boundary for the virtual private space; means for generating the virtual content for the virtual private space; first means for displaying the virtual content for the virtual session, the first displaying means configured to receive and display the virtual content; and second means for displaying the virtual content, the second displaying means configured to receive the boundary information and display the boundary virtual content. In some cases, the second displaying means does not display the virtual content in the virtual private space.

[0012] In some aspects, at least a portion of the virtual content is not viewable by one or more unauthorized users based on boundary virtual content associated with the boundary information.

[0013]

[0013] In some aspects, the real-world volume defined within the boundary virtual content is not viewable by one or more unauthorized users based on the boundary virtual content associated with the boundary information.

[0014]

[0014] In some aspects, one or more of the methods, devices, and computer-readable media described above further comprise outputting boundary virtual content that identifies a boundary for the virtual private space, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session.

[0015]

[0015] In some aspects, the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate boundary virtual content that identifies a boundary for the virtual private space and world coordinates in the physical space.

[0016]

[0016] In some aspects, one or more of the methods, apparatus, and computer-readable media described above further comprise receiving an indication that an unauthorized user has entered the virtual private space, and, based on the indication that the unauthorized user has entered the virtual private space, occluding at least a portion of the unauthorized user from being viewable in the virtual private space by the one or more authorized users.

[0017]

[0017] In some aspects, one or more of the methods, apparatus, and computer-readable media described above further comprise receiving an indication that an unauthorized user has entered or is within a threshold distance of entering the virtual private space, and moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user based on the indication that the unauthorized user has entered or is within the threshold distance of entering the virtual private space.

[0018]

[0018] In some aspects, one or more of the methods, apparatus, and computer-readable media described above further comprise receiving an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space, and outputting a notification indicating the existence of the virtual private space based on the indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space.

[0019] In some aspects, outputting the notification includes at least one of sending a notification to the device of the unauthorized user and outputting an audio notification indicating the existence of the virtual private space. In some cases, the notification includes an outline of the unauthorized user.

[0020] In some aspects, one or more of the methods, apparatus, and computer-readable media described above further comprise receiving an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space, and, based on the indication that the unauthenticated user has entered or is within the threshold distance of entering a virtual private space, outputting a notification to one or more devices of one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering a virtual private space. In some cases, the notification includes a summary of the unauthenticated user.

[0021]

[0021] In some aspects, one or more of the devices or apparatuses are augmented reality devices, and one or more of the methods, apparatuses, and computer-readable media described above further comprise displaying at least a portion of the virtual content by the augmented reality device.

[0022]

[0022] In some aspects, one or more of the devices or apparatuses is a first augmented reality device, and one or more of the methods, apparatuses, and computer-readable media described above further comprise outputting at least a portion of the virtual content to a second augmented reality device.

[0023]

[0023] In some aspects, one or more of the devices or apparatuses is a first augmented reality device, and one or more of the methods, apparatuses, and computer-readable media described above further comprise displaying at least a portion of the virtual content by the first augmented reality device and outputting at least a portion of the virtual content to a second augmented reality device.

[0024]

[0024] In some aspects, one or more of the devices or apparatuses is a first augmented reality device, and one or more of the methods, apparatuses, and computer-readable media described above further comprise displaying at least a portion of the virtual content by the first augmented reality device, outputting at least a portion of the virtual content to a second augmented reality device, receiving boundary information by the second augmented reality device, and generating virtual boundary content from the received boundary information by the second augmented reality device.

[0025]

[0025] In some aspects, one or more of the devices or apparatuses are server devices, and one or more of the methods, apparatuses, and computer-readable media described above further comprise outputting at least a portion of the virtual content to the augmented reality device by the server device.

[0026] In some aspects, the apparatus comprises a camera, a mobile device (e.g., a mobile phone or so-called "smartphone" or other mobile device), a wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a personal computer, a laptop computer, a server computer, or other device. In some aspects, the apparatus includes one or more cameras for capturing one or more images. In some aspects, the apparatus further includes a display for displaying one or more images, notifications, and / or other displayable data. In some aspects, the apparatus described above may include one or more sensors.

[0027] This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used separately to determine the scope of the claimed subject matter, which subject matter should be understood by reference to the entire specification of this patent, any or all drawings, and appropriate portions of each claim.

[0028] The foregoing, together with other features and embodiments, will become more apparent with reference to the following specification, claims, and accompanying drawings.

[0029]

[0029] Exemplary embodiments of the present application are described in detail below with reference to the following drawings: [Brief explanation of the drawings]

[0030] [Figure 1]

[0030] A block diagram illustrating an exemplary augmented reality system, according to some examples of the present disclosure. [Figure 2]

[0031] FIG. 10 illustrates exemplary landmark points on a hand that may be used to track hand position and hand interaction with a virtual environment, according to some examples of the present disclosure. [Figure 3]

[0032] 1 illustrates an example of an augmented reality system being worn by a user, according to some examples of the present disclosure. [Figure 4]

[0033] 1 illustrates an example of an augmented reality virtual session with virtual content displayed in a virtual private space, according to some examples of the present disclosure. [Figure 5]

[0034] 5 illustrates another view of the virtual private space shown in FIG. 4, according to some examples of the present disclosure. [Figure 6]

[0035] 5 illustrates an example of a profile view of the table and bounded virtual content from FIG. 4, according to some examples of the present disclosure. [Figure 7A]

[0036] 1 illustrates an example of a person physically entering a virtual private space, according to some examples of the present disclosure. [Figure 7B] 1 illustrates an example of a person physically entering a virtual private space, according to some examples of the present disclosure. [Figure 7C] 1 illustrates an example of a person physically entering a virtual private space, according to some examples of the present disclosure. [Figure 8]

[0037] 1 is a flow diagram illustrating an example of a process for generating virtual content for one or more virtual private spaces, according to some examples of the present disclosure. [Figure 9]

[0038] FIG. 1 illustrates an exemplary computing system, according to some examples of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0031]

[0039] Some aspects and embodiments of the present disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments may be applied independently, and some of them may be applied in combination. In the following description, for purposes of explanation, specific details are set forth to provide a thorough understanding of the embodiments of the present application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and descriptions are not limiting.

[0032]

[0040] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments provides those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the present application, as set forth in the appended claims.

[0033]

[0041] Augmented reality (XR) systems can facilitate interaction with different types of XR environments, including virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or other XR environments. XR devices can be used by users to interact with the XR environments. Examples of XR devices include head-mounted displays (HMDs) and smart glasses, among others. For example, an AR system can overlay virtual content onto an image of a real-world environment, which can be viewed by a user through an AR device (e.g., an HMD, AR glasses, or other AR devices). The real-world environment can include physical objects, people, or other real-world objects. The XR device can track parts of a user (e.g., the user's hands and / or fingertips) to enable the user to interact with items of virtual content.

[0034]

[0042] Real-world objects may be supplemented with virtual content present in the XR environment. For example, a virtual coffee cup may be virtually anchored to (e.g., placed on) a real-world table in one or more images displayed during an AR session including the AR environment. A person may also directly affect the virtual content and / or other real-world objects in the environment. For example, a person in a space occupied by one or more items of virtual content may interact with one or more items of virtual content and / or one or more real-world objects in various ways. In one example, a person may interrupt an AR session by moving and / or blocking (or obstructing) the displayed virtual content. In another example, a person may move a real-world object virtually supporting an item of virtual content (e.g., by moving a physical table holding a virtual coffee cup).

[0035]

[0043] In some cases, a person may be unaware that items of virtual content are being rendered in a given environment during an AR session. For example, the person may not be using (e.g., not wearing) an AR device that allows the person to view the virtual content of the AR session. In another example, the person may have an AR device but not be authorized to view the virtual content of the AR session. If a person in a space is not using an AR device or is not authorized to view some virtual content, the person will not be able to see the virtual content and may inadvertently interfere with the virtual content in a way that adversely affects the user experience for users who are wearing AR devices and are authorized to view and interact with the virtual content of the AR session.

[0036]

[0044] Such interference can be problematic in some cases. For example, unexpected changes in the virtual environment can be frustrating for users involved in a virtual session (e.g., an AR session or other XR session). As described above, AR content can be anchored to one or more real-world objects that are observable and can be manipulated by users not using AR devices. In one illustrative example, a multiplayer AR game session may be initiated on a restaurant table containing plates, cups, cutlery, and other items. It can be frustrating for users observing the AR content of the AR session when the real-world objects used for anchoring are moved and / or when these objects otherwise block immersion (e.g., if cups are refilled by the server in a way that obstructs / interrupts the virtual content and / or affects the anchoring).

[0037]

[0045] It may also be desirable to maintain the privacy of a virtual session so that virtual content shown to one or more users involved in the virtual session cannot be viewed by others. However, users who are not authorized to view the virtual content of a virtual session may not realize that a virtual session is occurring or the scope of the session, even if using an AR device, and may inadvertently view the virtual content.

[0038]

[0046] This disclosure describes systems, apparatuses, methods, and computer-readable media for generating virtual private spaces. The techniques described herein provide users of XR devices or systems (e.g., HMDs, AR glasses, etc.) with the ability to define a virtual private space for a virtual session (e.g., an AR session), where the virtual content presented in the virtual private space can only be seen and interacted with by “authenticated” users who are authenticated to view the virtual content of the virtual private space (using an XR device such as AR glasses). Unauthenticated users cannot view the virtual content displayed in the virtual private space. As used herein, an unauthenticated user is a device user who is not authenticated to view the virtual content of the virtual session presented in the virtual private space. As described herein, users may or may not be authenticated using any suitable authorization technique. A virtual private space may be a large space (e.g., a configuration zone) or a small space (e.g., a game board displayed on a real-world table). A larger space may be larger than the area sensed by an individual user device. In some cases, a map corresponding to a large space may need to be redrawn based on input from a communication network.

[0039]

[0047] The virtual private space creates a "safe zone" where an authenticated AR user can engage in an activity (e.g., a virtual game, a virtual sports activity, a virtual presentation, etc.) and not be inadvertently disturbed by other people or objects in the physical environment when the virtual activity is being performed. Examples using AR sessions and AR devices are described herein for illustrative purposes. Those skilled in the art will appreciate that the present techniques can be applied to other XR environments, such as VR and / or MR environments.

[0040]

[0048] Data defining the virtual content to be included in the virtual private space may be provided to different AR devices of an authenticated user so that the AR devices can display the virtual content to be included in the virtual private space. In some examples, the virtual private space may be implemented by generating and displaying boundary virtual content that defines the boundary of the virtual private space and / or blocks the viewing of unauthorized users who may be in the same real-world environment as the authenticated user of the virtual private space. For example, the AR device and / or server may provide boundary information (e.g., world coordinates, feature points in an image, and / or other boundary information) to one or more AR devices of the unauthorized user and, in some cases, to one or more other AR devices of the authenticated user of the virtual session. The boundary information may be used by one or more AR devices to generate and display boundary virtual content that identifies the boundary for the virtual private space. In another example, the AR device and / or server may generate boundary virtual content that identifies the boundary for the virtual private space and provide the boundary virtual content to one or more AR devices of the unauthorized user and, in some cases, to one or more other AR devices of the authenticated user of the virtual session. In some cases, one or more AR devices of the unauthenticated user and / or the authenticated user can determine boundary information (e.g., world coordinates, feature points in an image, and / or other information) and use the information to generate boundary virtual content. In some cases, the AR devices of one or more authenticated users and / or the server can share a map of the real world with the AR devices of the unauthenticated users, so that the AR devices of the unauthenticated users can display boundaries or other visual indications of the virtual private space.In some cases, the virtual content defining the virtual private space may be generated based on the scope of the virtual content during the virtual session, generated based on user input, and / or inferred from application status provided by the user's AR device.

[0041]

[0049] The boundary virtual content allows the existence of a virtual private space to be observed by a user who is not authorized to view virtual content within the virtual private space. For example, the unauthorized user's AR device can display a virtual barrier that defines the virtual space. The display of the boundary virtual content can prevent unauthorized users from viewing virtual content in the virtual private space while limiting unintentional disruption of the virtual session. In one illustrative example, by observing the boundary virtual information of the virtual private space (e.g., using an AR device), a restaurant server can refrain from interfering with real-world objects contained within the virtual private space.

[0042]

[0050] Further details regarding the generation of virtual private spaces are provided herein with respect to various figures. FIG. 1 illustrates an exemplary augmented reality system 100 according to some aspects of the present disclosure. The augmented reality system 100 can operate (or execute) XR applications and implement XR operations. In some examples, the augmented reality system 100 can perform tracking and localization, mapping of the physical world (e.g., a scene), and placing and rendering of virtual content on a display 109 (e.g., a screen, a visible plane / area, and / or another display) as part of an XR experience. For example, the augmented reality system 100 can generate a map (e.g., a three-dimensional (3D) map) of a scene in the physical world, track the attitude (e.g., location and position) of the augmented reality system 100 relative to the scene (e.g., relative to the 3D map of the scene), place and / or anchor virtual content at specific locations on the map of the scene, and render the virtual content on the display 109 so that the virtual content appears to be at a location in the scene that corresponds to the specific location on the map of the scene where the virtual content is placed and / or anchored. The display 109 may include glasses, a screen, lenses, a projector, and / or other display mechanisms that allow a user to view a real-world environment and also allow XR content to be displayed thereon.

[0043]

[0051] In this illustrative example, augmented reality system 100 includes one or more image sensors 102, accelerometer 104, gyroscope 106, storage 107, computational components 110, XR engine 120, virtual private space management engine 122, image processing engine 124, and rendering engine 126. It should be noted that components 102-126 shown in FIG. 1 are non-limiting examples provided for illustrative and descriptive purposes, and that other examples may include more, fewer, or different components than those shown in FIG. 1. For example, in some cases, augmented reality system 100 may include one or more other sensors (e.g., one or more inertial measurement units (IMUs), radar, light detection and ranging (LIDAR) sensors, acoustic sensors, etc.), one or more display devices, one or more other processing engines, one or more other hardware components, and / or one or more other software and / or hardware components not shown in FIG. 1. An example architecture and example hardware components that may be implemented by the augmented reality system 100 are further described below with respect to FIG.

[0044]

[0052] Further, for simplicity and purposes of explanation, one or more image sensors 102 will be referred to herein (e.g., in the singular) as image sensor 102. However, those skilled in the art will recognize that augmented reality system 100 can include a single image sensor or multiple image sensors. Also, a reference to any of the components (e.g., 102-126) of augmented reality system 100 in the singular or plural should not be construed as limiting the number of such components implemented by augmented reality system 100 to one or more. For example, a reference to accelerometer 104 in the singular should not be construed as limiting the number of accelerometers implemented by augmented reality system 100 to one. Those skilled in the art will recognize that for any of components 102-126 shown in FIG. 1 , augmented reality system 100 can include only one such component or multiple such components.

[0045]

[0053] The augmented reality system 100 includes or is in communication (wired or wireless) with an input device 108. The input device 108 may include any suitable input device, such as a touchscreen, a pen or other pointer device, a keyboard, mouse buttons or keys, a microphone for receiving voice commands, a gesture input device for receiving gesture commands, any combination thereof, and / or other input devices. In some cases, the image sensor 102 may capture images that may be processed to interpret the gesture commands.

[0046]

[0054] Augmented reality system 100 may be part of or implemented by a single computing device or multiple computing devices. In some examples, augmented reality system 100 may be part of an electronic device such as an augmented reality head-mounted display (HMD) device, augmented reality glasses (e.g., augmented reality or AR glasses), a camera system (e.g., a digital camera, an IP camera, a video camera, a security camera, etc.), a telephone system (e.g., a smartphone, a cellular phone, a conferencing system, etc.), a desktop computer, a laptop or notebook computer, a tablet computer, a set-top box, a smart television, a display device, a game console, a video streaming device, an IoT (Internet of Things) device, and / or any other suitable electronic device.

[0047]

[0055] In some implementations, one or more of the image sensor 102, the accelerometer 104, the gyroscope 106, the storage 107, the computational component 110, the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and the rendering engine 126 may be part of the same computing device. For example, in some cases, the one or more of the image sensor 102, the accelerometer 104, the gyroscope 106, the storage 107, the computational component 110, the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and the rendering engine 126 may be integrated into an HMD, augmented reality glasses, a smartphone, a laptop, a tablet computer, a gaming system, and / or any other computing device. However, in some implementations, one or more of the image sensor 102, accelerometer 104, gyroscope 106, storage 107, computational component 110, XR engine 120, virtual private space management engine 122, image processing engine 124, and rendering engine 126 may be part of two or more separate computing devices. For example, in some cases, some of the components 102-126 may be part of or implemented by one computing device, and the remaining components may be part of or implemented by one or more other computing devices.

[0048]

[0056] The storage device 107 may be any storage device for storing data. Additionally, the storage device 107 may store data from any of the components of the augmented reality system 100. For example, the storage device 107 may store data (e.g., image or video data) from the image sensor 102, data (e.g., measurements) from the accelerometer 104, data (e.g., measurements) from the gyroscope 106, data (e.g., processing parameters, preferences, virtual content, rendering content, scene maps, tracking and localization data, object detection data, privacy data, XR application data, facial recognition data, occlusion data, etc.) from the computation component 110, data from the XR engine 120, data from the virtual private space management engine 122, data from the image processing engine 124, and / or data (e.g., output frames) from the rendering engine 126. In some examples, the storage device 107 may include a buffer for storing frames for processing by the computation component 110.

[0049]

[0057] The one or more computing components 110 may include a central processing unit (CPU) 112, a graphics processing unit (GPU) 114, a digital signal processor (DSP) 116, and / or an image signal processor (ISP) 118. The computing components 110 may perform various operations such as image processing, computer vision, graphics rendering, augmented reality (e.g., tracking, localization, pose estimation, mapping, content anchoring, content rendering, etc.), image / video processing, sensor processing, recognition (e.g., text recognition, face recognition, object recognition, feature recognition, tracking or pattern recognition, scene recognition, occlusion detection, etc.), machine learning, filtering, and any of the various operations described herein. In this example, the computing components 110 implement an XR engine 120, a virtual private space management engine 122, an image processing engine 124, and a rendering engine 126. In other examples, the computing components 110 may also implement one or more other processing engines.

[0050]

[0058] The image sensor 102 can include any image and / or video sensor or capture device. In some examples, the image sensor 102 can be part of a multiple camera assembly, such as a dual camera assembly. The image sensor 102 can capture images and / or video content (e.g., raw image and / or video data), which can then be processed by the computing component 110, the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and / or the rendering engine 126 described herein.

[0051]

[0059] In some examples, the image sensor 102 can capture image data, generate frames based on the image data, and / or provide the image data or frames to the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and / or the rendering engine 126 for processing. A frame can include a video frame of a video sequence or a still image. A frame can include a pixel array representing a scene. For example, a frame can be a red-green-blue (RGB) frame having red, green, and blue color components per pixel, a luma-chroma-red-chroma-blue (YCbCr) frame having a luma component and two chroma (color) components (chroma-red and chroma-blue) per pixel, or any other suitable type of color or monochrome picture.

[0052]

[0060] In some cases, the image sensor 102 (and / or other cameras of the augmented reality system 100) may also be configured to capture depth information. For example, in some implementations, the image sensor 102 (and / or other cameras) may include an RGB-depth (RGB-D) camera. In some cases, the augmented reality system 100 may include one or more depth sensors (not shown) that are separate from the image sensor 102 (and / or other cameras) and that can capture depth information. For example, such depth sensors may acquire depth information independently of the image sensor 102. In some examples, the depth sensor may be physically located in the same general location as the image sensor 102 but may operate at a different frequency or frame rate than the image sensor 102. In some examples, the depth sensor may take the form of a light source that can project a structured or textured light pattern, which may include one or more narrowband lights, onto one or more objects in a scene. The depth information is then obtained by exploiting geometric distortions of the projected pattern caused by the surface shape of the object. In one example, depth information may be obtained from a stereo sensor such as a combination of an infrared structured light projector and an infrared camera registered to a camera (e.g., an RGB camera).

[0053]

[0061] The augmented reality system 100 also includes one or more sensors. The one or more sensors may include one or more accelerometers (e.g., accelerometer 104), one or more gyroscopes (e.g., gyroscope 106), and / or other sensors. The one or more sensors may provide velocity, orientation, and / or other position-related information to the computational component 110. For example, the accelerometer 104 may detect acceleration by the augmented reality system 100 and generate an acceleration measurement based on the detected acceleration. In some cases, the accelerometer 104 may provide one or more translation vectors (e.g., up / down, left / right, forward / backward) that may be used to determine the position or attitude of the augmented reality system 100. The gyroscope 106 may detect and measure the orientation and angular velocity of the augmented reality system 100. For example, the gyroscope 106 may be used to measure the pitch, roll, and yaw of the augmented reality system 100. In some cases, the gyroscope 106 may provide one or more rotation vectors (e.g., pitch, yaw, roll). In some examples, the image sensor 102 and / or the XR engine 120 can use measurements obtained by the accelerometer 104 (e.g., one or more translation vectors) and / or the gyroscope 106 (e.g., one or more rotation vectors) to calculate the attitude of the augmented reality system 100. As mentioned above, in other examples, the augmented reality system 100 can also include other sensors such as an inertial measurement unit (IMU), a magnetometer, gaze and / or eye tracking sensors, machine vision sensors, smart scene sensors, voice recognition sensors, collision sensors, shock sensors, position sensors, tilt sensors, etc.

[0054]

[0062] In some cases, the one or more sensors may include at least one IMU. An IMU is an electronic device that measures specific forces, angular velocities, and / or orientations of the augmented reality system 100 using a combination of one or more accelerometers, one or more gyroscopes, and / or one or more magnetometers. In some examples, the one or more sensors may output measured information related to the capture of images captured by the image sensor 102 (and / or other cameras of the augmented reality system 100) and / or depth information obtained using one or more depth sensors of the augmented reality system 100.

[0055]

[0063] The output of one or more sensors (e.g., accelerometer 104, gyroscope 106, one or more IMUs, and / or other sensors) may be used by augmented reality engine 120 to determine the pose of augmented reality system 100 (also referred to as head pose) and / or the pose of image sensor 102 (or other camera of augmented reality system 100). In some cases, the pose of augmented reality system 100 and the pose of image sensor 102 (or other camera) may be the same. The pose of image sensor 102 refers to the position and orientation of image sensor 102 relative to a frame of reference (e.g., with respect to object 202). In some implementations, the pose of a camera may be determined in terms of six degrees of freedom (6DOF), which refers to three translational components (e.g., which may be given by X (horizontal), Y (vertical), and Z (depth) coordinates relative to a frame of reference, such as the plane of the image) and three angular components (e.g., roll, pitch, and yaw relative to the same frame of reference).

[0056]

[0064] In some cases, a device tracker (not shown) can use measurements from one or more sensors and image data from image sensor 102 to track the pose (e.g., 6DOF pose) of augmented reality system 100. For example, the device tracker can fuse visual data from the image data with inertial data from the measurements (e.g., using a visual tracking solution) to determine the position and movement of augmented reality system 100 relative to the physical world (e.g., scene) and a map of the physical world. As described below, in some examples, when tracking the pose of augmented reality system 100, the device tracker can generate a three-dimensional (3D) map of the scene (e.g., real world) and / or generate updates for the 3D map of the scene. 3D map updates can include, for example, but are not limited to, new or updated features and / or landmark points associated with the scene and / or the 3D map of the scene, location updates that identify or update the position of augmented reality system 100 within the scene and the 3D map of the scene, etc. The 3D map can provide a digital representation of a scene in the real / physical world. In some examples, the 3D map can anchor location-based objects and / or content to real-world coordinates and / or objects. The augmented reality system 100 can use the mapped scene (e.g., a scene in the physical world represented by and / or associated with the 3D map) to merge the physical world and the virtual world and / or to merge virtual content or objects with the physical environment.

[0057]

[0065] In some aspects, the pose of the image sensor 102 and / or the augmented reality system 100 as a whole may be determined and / or tracked by the computational component 110 using a visual tracking solution based on images captured by the image sensor 102 (and / or other cameras of the augmented reality system 100). For example, in some examples, the computational component 110 may perform tracking using computer vision-based tracking techniques, model-based tracking techniques, and / or simultaneous localization and mapping (SLAM) techniques. For example, the computational component 110 may perform SLAM or may be in communication (wired or wireless) with a SLAM engine (not shown). SLAM refers to a class of techniques in which a map of an environment (e.g., a map of the environment being modeled by the augmented reality system 100) is created while simultaneously tracking the pose of the camera (e.g., the image sensor 102) and / or the augmented reality system 100 relative to that map. The map may be referred to as a SLAM map and may be three-dimensional (3D). SLAM techniques may be implemented using color or grayscale image data captured by image sensor 102 (and / or other cameras of augmented reality system 100) and may be used to generate estimates of 6DOF pose measurements of image sensor 102 and / or augmented reality system 100. Such SLAM techniques configured to implement 6DOF tracking may be referred to as 6DOF SLAM. In some cases, the output of one or more sensors (e.g., accelerometer 104, gyroscope 106, one or more IMUs, and / or other sensors) may be used to estimate, correct, and / or possibly adjust the estimated pose.

[0058]

[0066] In some cases, 6DOF SLAM (e.g., 6DOF tracking) can associate observed features from several input images from the image sensor 102 (and / or other cameras) into a SLAM map. For example, 6DOF SLAM can use the association of feature points from the input images to determine the pose (position and orientation) of the image sensor 102 and / or the augmented reality system 100 with respect to the input images. 6DOF mapping can also be performed to update the SLAM map. In some cases, a SLAM map maintained using 6DOF SLAM can include 3D feature points triangulated from two or more images. For example, keyframes can be selected from the input images or video stream to represent the observed scene. For each keyframe, a pose of each 6DOF camera associated with the image can be determined. The pose of the image sensor 102 and / or the augmented reality system 100 can be determined by projecting features from the 3D SLAM map onto the image or video frame and updating the camera pose from the verified 2D-3D correspondence.

[0059]

[0067] In one illustrative example, the computational component 110 can extract feature points from every input image or from each keyframe. As used herein, a feature point (also referred to as a registration point) is a distinctive or identifiable part of an image, such as a hand, the edge of a table, or the like, among other things. Features extracted from a captured image can represent distinct feature points along three-dimensional space (e.g., coordinates on the X, Y, and Z axes), and every feature point can have an associated feature location. A feature point in a keyframe can match (be the same as or correspond to) or fail to match a feature point in a previously captured input image or keyframe. Feature detection can be used to detect feature points. Feature detection can include image processing operations used to examine one or more pixels of an image to determine whether a feature is present at a particular pixel. Feature detection can be used to process the entire captured image or portions of an image. For each image or keyframe, once a feature is detected, a local image patch around the feature can be extracted. Features may be extracted using any suitable technique, such as Scale Invariant Feature Transform (SIFT), Speed ​​Up Robust Features (SURF), Gradient Location-Orientation histogram (GLOH), Normalized Cross Correlation (NCC), or other suitable techniques (which locate features and generate their descriptions).

[0060]

[0068] In some cases, the augmented reality system 100 may also track the user's hands and / or fingers to enable the user to interact with and / or control virtual content in the virtual environment (e.g., virtual content displayed in a virtual private space). For example, the augmented reality system 100 may track the pose and / or movement of the user's hands and / or fingertips to identify or translate the user's interaction with the virtual environment. User interactions may include, for example, without limitation, moving items of virtual content, resizing items of virtual content and / or locations in the virtual private space, selecting input interface elements in a virtual user interface (e.g., a virtual representation of a mobile phone, a virtual keyboard, and / or other virtual interface), providing input through a virtual user interface, etc.

[0061]

[0069] 2 is a diagram illustrating example landmark points on a hand 200 that may be used to track the position of the hand 200 and its interaction with a virtual environment, such as virtual content displayed within a virtual private space as described herein. The landmark points shown in FIG. 2 correspond to different parts of the hand 200, including a landmark point 235 on the palm of the hand 200, a landmark point on the thumb 230 of the hand 200, a landmark point on the index finger 232 of the hand 200, a landmark point on the middle finger 234 of the hand 200, a landmark point on the ring finger 236 of the hand 200, and a landmark point on the little finger 238 of the hand 200. The palm of the hand 200 can move in three translational directions (e.g., measured in the X, Y, and Z directions relative to a plane, such as the image plane) and three rotational directions (e.g., measured in yaw, pitch, and roll relative to the plane), thus providing six degrees of freedom (6DOF) that may be used for registration and / or tracking. The 6 DOF movements of the palm are shown as squares in Figure 2, as indicated in legend 240.

[0062]

[0070] Different joints in the fingers of hand 200 allow for different degrees of movement, as indicated by legend 240. As indicated by the diamond shapes (e.g., diamond 233) in FIG. 2 , the base of each finger (corresponding to the metacarpophalangeal joint (MCP) between the proximal phalanx and metacarpal bone) has two degrees of freedom (2 DOF) corresponding to flexion and extension and abduction and adduction. As indicated by the circular shapes (e.g., circle 231) in FIG. 2 , each of the joints at the top of each finger (corresponding to the interphalangeal joints between the distal, middle, and proximal phalanges) has one degree of freedom (1 DOF) corresponding to flexion and extension. Thus, hand 200 provides 26 degrees of freedom (26 DOF) for tracking hand 200 and its interaction with virtual content rendered by augmented reality system 100.

[0063]

[0071] The augmented reality system 100 can use one or more of the landmark points on the hand 200 to track the hand 200 (e.g., track the posture and / or movement of the hand 200) and to track interaction with the virtual environment rendered by the augmented reality system 100. As described above, detection of one or more landmark points on the hand 200 can result in the establishment of a posture of the landmark (and thus the hand and fingers) in a physical position relative to the augmented reality system 100. For example, a landmark point (e.g., landmark point 235) on the palm of the hand 200 can be detected in an image, and the location of the landmark point can be determined relative to the image sensor 102 of the augmented reality system 100. A point (e.g., a central point such as a center of gravity or other central point) of an item of virtual content rendered by the augmented reality system 100 can be translated into a position on (or a rendering on) a display (e.g., display 109 of FIG. 1 ) of the augmented reality system 100 relative to the location determined for the landmark point on the palm of the hand 200.

[0064]

[0072] As described below, the augmented reality system 100 can also register the virtual content and / or hand 200 to points in the real world (detected in one or more images) and / or other parts of the user. For example, in some implementations, in addition to determining the physical pose of the hand 200 (or the augmented reality system 100) and / or items of virtual content relative to the augmented reality system 100, the augmented reality system 100 can determine the locations of other landmarks, such as characteristic points (called feature points) on a wall, one or more corners of an object, features on a floor, points on a human face, points on a nearby device, among others. In some cases, the augmented reality system 100 can place the virtual content within a position relative to feature points detected in the environment, which may correspond, for example, to objects and / or people detected in the environment.

[0065]

[0073] In some examples, the pose of the augmented reality system 100 (and / or the user's head) may be determined using, for example, image data from the image sensor 102 and / or measurements from one or more sensors, such as the accelerometer 104, the gyroscope 106, and / or one or more other sensors (e.g., one or more magnetometers, one or more inertial measurement units (IMUs), etc.). The head pose may be used to determine the position of virtual content, the hand 200, and / or objects and / or people in the environment.

[0066]

[0074] Operations for the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and the rendering engine 126 (and any image processing engines) may be implemented by any of the computing components 110. In one illustrative example, the operations of the rendering engine 126 may be implemented by the GPU 114, and the operations of the XR engine 120, the virtual private space management engine 122, and the image processing engine 124 may be implemented by the CPU 112, the DSP 116, and / or the ISP 118. In some cases, the computing component 110 may include other electronic circuitry or hardware, computer software, firmware, or any combination thereof to perform any of the various operations described herein.

[0067]

[0075] In some examples, the XR engine 120 can perform XR operations to generate an XR experience based on data from the image sensor 102, accelerometer 104, gyroscope 106, and / or one or more sensors on the augmented reality system 100, such as one or more IMUs, radars, etc. In some examples, the XR engine 120 can perform tracking, localization, pose estimation, mapping, content anchoring operations, and / or any other XR operations / functions. The XR experience can include use of the augmented reality system 100 to present XR content (e.g., virtual reality content, augmented reality content, mixed reality content, etc.) to a user during a virtual session. In some examples, the XR content and experiences can be provided by the augmented reality system 100 through XR applications (e.g., executed or implemented by the XR engine 120) that provide a particular XR experience, such as, for example, an XR gaming experience, an XR classroom experience, an XR shopping experience, an XR entertainment experience, an XR activity (e.g., a movement, a troubleshooting activity, etc.), among others. During an XR experience, a user can view and / or interact with virtual content using augmented reality system 100. In some cases, a user can view and / or interact with the virtual content while also being able to view and / or interact with the physical environment around the user, allowing the user to have an immersive experience between the physical environment and the virtual content mixed with or integrated with the physical environment.

[0068]

[0076] The virtual private space management engine 122 can perform various operations to determine (and manage) how, where, and / or when a virtual private space should be rendered during an XR experience. A virtual private space may be defined for a virtual session (e.g., an AR session) by a user of the augmented reality system 100 or automatically (e.g., based on user preferences, based on AR session requirements, etc.). A virtual private space may be generated to display virtual content over any region of a physical space. A virtual private space may be generated for a large space (e.g., a construction zone, a concert hall, a sports venue, among others) or for a small space (e.g., a virtual conference call with virtual content displayed on a real-world conference table, a game board displayed on a real-world table, among others). A larger virtual space may be larger than the area sensed by an individual user device. For example, as described below, crowdsourcing may be used to obtain features of a large space that can be used to generate the large virtual space. In some cases, a 3D map (e.g., a SLAM map) corresponding to a large space may need to be redrawn based on input obtained over a communications network. For example, a local map can be improved based on crowdsourcing information of the same geographic coordinates.

[0069]

[0077] A virtual private space may be defined by boundary information that defines the boundary of the virtual private space. The augmented reality system 100 (e.g., the virtual private space management engine 122 and / or other components) may generate the boundary information or may obtain the boundary information from a server and / or another augmented reality system. The boundary information is a data structure that represents the boundary of the virtual private space. The virtual content that defines the boundary of the virtual private space is referred to as boundary virtual content. The boundary virtual content identifies the boundary for the virtual private space. In some cases, the boundary information may include one or more feature points in at least one image of the physical space (e.g., captured by the image sensor 102) and / or may include world coordinates in the physical space. In some cases, the augmented reality system 100 may receive the boundary virtual content from a server and / or another XR system. In some cases, the augmented reality system 100 may receive the feature points and / or world coordinates from a server and / or another XR system. In one illustrative example, the augmented reality system 100 can receive boundary virtual content (and other virtual content for the virtual session) and feature points and / or world coordinates from a server. In another illustrative example, the augmented reality system 100 can receive boundary virtual content (and / or other virtual content for the virtual session) from a server and can receive feature points and / or world coordinates from another XR system. The augmented reality system 100 can use the feature points and / or world coordinates to generate the boundary virtual content (e.g., to determine where to display the boundary virtual content relative to items in the physical space). Further details regarding the use of feature points and / or world coordinates are described below.

[0070]

[0078] In some examples, boundary virtual content may be generated based on a user's line of sight and / or in response to a non-user's line of sight to the virtual content of the virtual session displayed within the virtual private space. For example, if an authenticated user's XR system detects that an unauthorized user is staring at the virtual private space, boundary virtual content may be generated to block the unauthorized user from viewing the virtual content of the virtual private space.

[0071]

[0079] Based on the boundary virtual content, the virtual content of the virtual session displayed within the virtual private space can be seen and interacted with only by authenticated users who are authenticated to view the virtual content of the virtual session (using an XR device such as an HMD or AR glasses). The boundary virtual content blocks the virtual content in the virtual private space from being viewed by unauthorized users who may be in the same real-world environment as the authenticated user, and therefore, unauthorized users cannot view the virtual content. The boundary virtual content also provides an indication of the existence of the virtual private space to unauthorized users located in the physical space where the virtual content is displayed. For unauthorized users using an XR device or system, the XR device can display boundary virtual content (e.g., as a virtual barrier) that defines the virtual space.

[0072]

[0080] In the case of an unauthorized user not using an XR device or system, the unauthorized user may be alerted regarding the location (e.g., boundaries) of the virtual private space. For example, when the unauthorized user does not have an XR device that enables the user to view virtual content, a notification may be sent to the unauthorized user's device (e.g., a mobile device, a wearable device such as a smartwatch or other wearable, or other device) to indicate the existence and / or location of the virtual private space. In one example, the server implementing the (authenticated user's) augmented reality system and / or virtual session can send a notification (e.g., a text-based message, an audible message, and / or other notification) to the unauthorized user's mobile device, wearable device, and / or other device indicating where the virtual private space is located. In another example, the augmented reality system can provide a notification (e.g., a visual notification, an audible notification, and / or other notification) in response to the augmented reality system detecting that an unauthorized user without an XR device (e.g., a user not using an XR device) is near or within a certain distance (e.g., 10 feet, 5 feet, 2 feet, and / or other distance) of the virtual private space. Notifications can allow unauthorized users to become aware of the location of the virtual private space and thus avoid interfering with the AR session taking place in the virtual private space. In such cases, interoperability can be provided between devices and networks. For example, protocols can be defined to keep the virtual space private and empower user devices to openly share XR capabilities and / or content between user devices (e.g., for devices from different vendors / implementations).

[0073]

[0081] Thus, a virtual private space creates an area of ​​physical space where an authenticated user can conduct an activity (e.g., virtual game, virtual sporting activity, virtual presentation, etc.) privately and not be inadvertently disturbed by other people or objects in the physical environment when the virtual activity is taking place. The display of bounded virtual content can prevent unauthorized users from viewing the virtual content in the virtual private space while simultaneously limiting unintentional disruption of the virtual session.

[0074]

[0082] Users of XR devices or systems may be authenticated to participate in virtual sessions associated with virtual private spaces using any suitable authorization technique. In some examples, users of XR devices may or may not be authenticated using electronic signatures in their respective XR devices. In one illustrative example, the hardware of the AR device may be recognized or verified (resulting in an authenticated user) or disabled (resulting in an unauthenticated user). In another example, authenticated users may be provided with login credentials used to authenticate the user. For example, multiple users may be authenticated to participate in a virtual poker game by entering their login credentials into a graphical interface of an application facilitating the virtual poker game. Users without login credentials may not be able to join the virtual session. In some examples, users of XR devices may or may not be authenticated using other authorization techniques, such as based on hand gestures, gaze, facial recognition, voice commands, and / or other authorization techniques.

[0075]

[0083] As described above, the boundary information may additionally or alternatively include world coordinates in the physical space and / or one or more feature points in at least one image of the physical space (e.g., captured by image sensor 102). In such cases, the boundary information (world coordinates and / or feature points) may be used by augmented reality system 100 to generate boundary virtual content that identifies a boundary for the virtual private space. Examples of using feature points and map data (e.g., using simultaneous localization and mapping (SLAM) techniques) have been described above. For example, world coordinates may indicate a location in a map of the real-world environment (e.g., a 3D map such as a SLAM map) of a surface on which the virtual content will be displayed (and thus on which the virtual private space will be located). An image of the surface may be obtained (e.g., captured by image sensor 102), and feature points of the surface may be extracted. The feature points may include points of characteristic features of the surface, such as lines, corners, and characteristic shapes, among other features.

[0076]

[0084] Items of virtual content may be registered or anchored to (e.g., located relative to) detected feature points in the scene. For example, virtual private space management engine 122 can coordinate with XR engine 120 and / or rendering engine 126 to anchor the virtual content of the virtual private space and boundary virtual content to feature points of a surface on which the virtual content will be displayed. In one illustrative example, a user may be looking at a table in a public space that the user wishes to use for a multi-user AR session (e.g., to play a card game). The user may provide input to augmented reality system 100 (e.g., using input device 108) indicating that the user wishes to conduct an AR session on the table and that the AR session should be associated with the virtual private space. Augmented reality engine 120, virtual private space management engine 122, image processing engine 124, and / or rendering engine 126 (implemented by computation component 110) can obtain an image of the table captured by image sensor 102 and identify features of the table from the image. The display 109 can display AR content related to the AR session on the table by anchoring the AR content to certain feature points of the table. The virtual private space management engine 122 can generate or receive (e.g., from a server) boundary virtual content to indicate the boundary of the virtual private space, and the boundary virtual content can be anchored to the feature points of the table. For example, the augmented reality engine 120 can register the AR content and the boundary virtual content to the feature points on the table so that the AR content and the boundary virtual content are displayed relative to the table.

[0077]

[0085] In another example, a user may be facing a restaurant across the street from where the user is standing. In response to identifying the restaurant from one or more images and obtaining AR content related to the restaurant, the augmented reality engine 120 may generate an AR object that provides information related to the restaurant. Feature points may be detected from a portion of the image that includes a sign for the restaurant. The augmented reality engine 120 may register the AR object to the feature points of the sign so that the AR object is displayed with respect to the sign (e.g., on top of the sign so that it is easily identifiable by the user associated with the restaurant).

[0078]

[0086] In some implementations, an authenticated user can prevent other authenticated users from viewing certain content in a virtual private zone. For example, a particular application running on a user device can prevent other users' devices running that application from displaying certain content. In one illustrative example, multiple users may be playing a multiplayer poker game, and a user device may prevent other users' cards from being displayed.

[0079]

[0087] In some examples, multiple virtual private spaces may be defined by the virtual private management engine 122 of the augmented reality system 100 and / or by other XR systems. In some cases, a new virtual private space may not be displayed over an existing virtual private space, which may prevent interference between the virtual private spaces. For example, virtual geofencing may be determined by selecting an object in space or by drawing a barrier around a private space. In some cases, each virtual private space may be defined relative to a common reference point in space. For example, a virtual space is a volume in space within which content is displayed. In an XR device, this volume of space may be defined relative to a tracked reference frame (e.g., a 3D coordinate frame with an origin and axis directions). Multiple virtual private spaces may be defined relative to the same reference frame.

[0080]

[0088] In some examples, an individual authenticated user of a virtual private space may invite one or more authenticated users of other virtual private spaces to merge different private spaces. For example, if two virtual sessions are taking place in separate virtual private spaces in close proximity to each other, the authenticated users of the two virtual spaces may agree to combine the two virtual sessions into a single virtual private space.

[0081]

[0089] In some cases, one or more authenticated users of the private space (e.g., a user of augmented reality system 100 authenticated for the private space and other users of the XR system) may be alerted when a disturbance or change occurs in the virtual private space. For example, the movement, removal, addition, and / or modification of real-world objects and / or unauthorized users present in the virtual private space may trigger a notification of the event to the authenticated user's device. In some examples, a visual indication may be displayed by display 109 and / or an audible indication may be output through a speaker (not shown) of augmented reality system 100 to indicate to the user that a disturbance or change has occurred in the virtual private space. In one illustrative example, multiple users may be participating in a multi-user virtual conference, where virtual content related to the virtual conference is displayed on tables in private rooms at a restaurant. In one example, a restaurant server may place drinks on the table in front of each of the users. The augmented reality system 100 may output a notification to users (including the user of augmented reality system 100) that an item has been placed within the boundary of the virtual private space. In another example, the augmented reality system 100 can output a notification to a user (including the user of the augmented reality system 100) about when a new virtual item appears in the virtual private space. In another example, using one or more images from the image sensor 102 and / or measurements from one or more sensors (e.g., the accelerometer 104, the gyroscope 106, one or more IMUs, and / or other sensors), the augmented reality system 100 can determine when a person or object is within a distance (e.g., 10 feet, 5 feet, 2 feet, and / or other distance) of interfering with the virtual private space. The augmented reality system 100 can output a notification to the user of the augmented reality system 100 that a person may be interfering with the virtual private space.In some cases, an alert may be triggered and given to an authenticated user's device based on the detection of a change to the map of the virtual private space. In some cases, the notification may include details about the interfering person or object.

[0082]

[0090] In some examples, virtual content in a virtual private space may be displayed with a higher priority than real-world content. For example, augmented reality system 100 and / or a server in communication with augmented reality system 100 may de-emphasize real-world objects and / or parts of an unauthorized user in the virtual private space. In some cases, any physical object that enters the private space may be fully or partially obstructed. In some examples, when an object or part of a person is fully obstructed, augmented reality system 100 and / or a server may provide an authenticated user with an indication (e.g., visually or audibly) of the object or person's location. In one illustrative example, augmented reality system 100 may fully or partially obstruct a body part (e.g., an arm) of an unauthorized user when that body part enters the virtual private space. In another example, if a physical object (e.g., a person's body part, a cup, and / or other physical object) is placed in a location where an item of virtual content is displayed in the virtual private space, the augmented reality system 100 can display the item of virtual content on top of the physical object or can obstruct the physical object so that it is not displayed by (or viewable through) the augmented reality system 100. If the object is obstructed, devices of authenticated users participating in a virtual session associated with the virtual private space may be alerted about the object's presence. In some examples, if unauthorized persons and / or other objects come into contact with virtual content in the virtual space, the virtual content may be moved to avoid the unauthorized persons and / or other objects. In some examples, if a virtual object depends on a real-world object (e.g., a virtual cup located on a real-world table) and the real-world object is removed from the physical space, the augmented reality system 100 can render a replica of the real-world object, maintaining the dependency.In some cases, the augmented reality system 100 may cause virtual objects to drop or move along with the real-world objects being moved.

[0083]

[0091] In some implementations, the augmented reality system 100 can perform object detection and / or recognition (e.g., human detection and / or recognition) to verify an authenticated user and / or to identify when a physical user or object enters or is within a certain distance of the virtual private space. The object detection and / or recognition can be image-based, audio-based, and / or using other techniques to detect / recognize objects. For example, an authenticated user may be verified by performing body, face, and / or voice recognition. Verifying a user before allowing an authenticated user to view and interact with virtual content in the virtual private space can prevent unauthorized persons from using the authenticated person's XR system to view and interact with private content.

[0084]

[0092] In some cases, crowdsourcing may be used to provide authenticated and / or unauthenticated users with enhanced details of their virtual private spaces. For example, image and sensor information from an XR system worn by multiple authenticated users (and, in some cases, unauthenticated users) can provide information that can be used to augment the virtual private space. In some examples, the authenticated users' (and, in some cases, unauthenticated users') computing devices other than the XR system (e.g., mobile devices, wearable devices, laptops, tablets, and / or other devices) can provide information used to augment the virtual space. For example, crowdsourced data from the authenticated users' devices and / or the unauthenticated users' devices can be used by the augmented reality system 100 (and / or a server in communication with the augmented reality system 100) to reliably map and track the virtual space relative to the real-world environment, which may enable the augmented reality system 100 (and / or the server) to more accurately monitor the virtual private space. Improved monitoring of the virtual private space may enable the augmented reality system 100 and / or the server to better notify the authenticated users about disturbances or changes to the virtual private space.

[0085]

[0093] In some implementations, all devices (of authenticated users) authenticated to participate in a virtual session implemented in a virtual private space can share information about the virtual private space to provide each device with the most accurate and up-to-date information about the space. Such information can include, among other information, intrusions by unauthorized users, the location of virtual content in the virtual private space, changes in the physical environment, and unseen angles and / or sides of the virtual content. Crowdsourcing of information related to unseen virtual content can help reduce latency and increase accuracy for rendering virtual objects in a virtual private space shared by multiple users. Such crowdsourcing can be important in virtual spaces that are larger than those that can be mapped using the sensors of a single XR system.

[0086]

[0094] As described above, crowdsourcing can be implemented using views (data) of a private space from unauthenticated users. For example, because the boundary of a virtual private space is virtual (as defined by boundary information and associated boundary virtual content) and not a fixed barrier between the unauthenticated user and the content of the virtual private space, an unauthenticated device (of an unauthenticated user) can share information about the virtual private space with an authenticated device (of an authenticated user) to provide accurate and up-to-date information about the virtual private space beyond the data shared between authenticated users' devices. In such an example, the unauthenticated user would not be able to see the private virtual content in the virtual private space, but data related to the space could still be shared with authenticated users.

[0087]

[0095] In some examples, authenticated users may broadcast information (e.g., game scores, food or drink orders, messages, and / or other information) and / or virtual content (e.g., virtual items for a virtual game, images of ordered food or drinks, images of messages, and / or other content) to non-authenticated users. The amount of information given to non-authenticated users' devices may be controlled by the authenticated user sending the information.

[0088]

[0096] The image processing engine 124 can perform one or more image processing operations related to the virtual content being presented in the virtual private space and / or related to other image content. For example, the image processing engine 124 can perform image processing operations based on data from the image sensor 102. In some cases, the image processing engine 124 can perform image processing operations such as filtering, demosaicing, scaling, color correction, color conversion, segmentation, noise reduction filtering, spatial filtering, artifact correction, etc. The rendering engine 126 can obtain image data generated and / or processed by the computation component 110, the image sensor 102, the XR engine 120, the virtual private space management engine 122, and / or the image processing engine 124 and can render video and / or image frames for presentation on a display device.

[0089]

[0097] While augmented reality system 100 is shown to include several components, those skilled in the art will appreciate that augmented reality system 100 can include more or fewer components than those shown in Figure 1. For example, augmented reality system 100, in some examples, can also include one or more memory devices (e.g., RAM, ROM, cache, etc.), one or more networking interfaces (e.g., wired and / or wireless communication interfaces, etc.), one or more display devices, and / or other hardware or processing devices not shown in Figure 1. Illustrative examples of computing systems and hardware components that may be implemented with augmented reality system 100 are described below with respect to Figure 9.

[0090]

[0098] FIG. 3 illustrates an example of an augmented reality system 100 worn by a user 301. While the augmented reality system 100 is illustrated in FIG. 3 as an AR device (e.g., AR glasses), the augmented reality system 100 may include any suitable type of XR device, such as an HMD or other XR device. The examples described below are described using AR for illustrative purposes. However, the aspects described below may be applied to other types of XR, such as VR and MR. The augmented reality system 100 illustrated in FIG. 3 may include a light-see-through AR device, which allows the user 301 to view the real world while wearing the augmented reality system 100. For example, the user 301 may view an object 302 in a real-world environment on a plane 304 at a distance from the user 301. As illustrated in FIG. 3 and described above, the augmented reality system 100 includes an image sensor 102 and a display 109. As described above, the display 109 may include glasses, a screen, lenses, and / or other display mechanisms that enable the user 301 to view the real-world environment and also enable AR content to be displayed thereon. AR content (e.g., images, videos, graphics, virtual or AR objects, or other AR content) may be projected or displayed on the display 109. In one example, the AR content may include an augmented version of the object 302. In another example, the AR content may include additional AR content related to the object 302 and / or related to one or more other objects in the real-world environment. While one image sensor 102 and one display 109 are shown in FIG. 3 , the augmented reality system 100, in some implementations, may include multiple cameras and / or multiple displays (e.g., a display for the right eye and a display for the left eye).

[0091]

[0099] As described above with respect to FIG. 1 , a virtual private space can be generated to present AR content for a particular AR session. FIG. 4 illustrates an example of an AR virtual session in which virtual content 458 is displayed by the augmented reality system 100 in the virtual private space 450. Boundary virtual content 451 is shown outlining the virtual private space 450. A group of authenticated users is shown, with a first authenticated user 452 represented by a first virtual representation of a hand, a second authenticated user 454 represented by a second virtual representation of a hand, and a third authenticated user 456 represented by a third virtual representation of a hand. The authenticated user 456 is using the augmented reality system 100, and each of the authenticated users 452 and 454 is using a respective XR device. The authenticated users 452, 454, and 456 are authorized to view the virtual content 458 of the virtual session using their respective XR devices.

[0092]

[0100] Authenticated users 452, 454, 456 may be located at a restaurant and interested in participating in a virtual poker game using a location in the restaurant. In some cases, authenticated users 452, 454, 456 may be authenticated by providing login credentials to a user interface of an application installed on their respective XR devices and associated with the virtual poker game (e.g., manually entering a username and password using an input mechanism, using a facial recognition-based login, and / or other login techniques). In some cases, the XR devices of authenticated users 452, 454, 456 may be authenticated using a hardwired signature as described above. Any other authorization or authentication technique may be used to authenticate authenticated users 452, 454, 456.

[0093]

[0101] An authenticated user 452, 454, 456 may identify a restaurant table 459 as a surface on which to display virtual content 458 for the virtual poker game. An authenticated user 456 (and / or other authenticated users 452 and / or 454) of the augmented reality system 100 may provide input to the augmented reality system 100 indicating that the user desires a virtual private space generated for the virtual poker game on the table 459. In some cases, the input requesting the virtual private space may be provided using a user interface of an application associated with the virtual poker game.

[0094]

[0102] In response to the input, the augmented reality system 100 can generate or receive (e.g., from a server) data defining the virtual content of the virtual poker game and boundary information that can be used to generate boundary virtual content 451 that defines the boundary of the virtual private space 450. Data defining the virtual content (e.g., virtual content 458) to be included in the virtual private space 450 can be provided to the different XR devices of the authenticated users 452, 454, 456 so that the XR devices can display the virtual content that defines the virtual private space 450. For example, a server associated with the virtual poker game can provide the virtual content (and, in some cases, boundary information) to the XR devices. In some cases, the server can provide the virtual content (and, in some cases, boundary information) to the augmented reality system 100 of the authenticated user 456, and the augmented reality system 100 can provide the virtual content (and, in some cases, boundary information) to the XR devices of the other users 452 and 454.

[0095]

[0103] In some cases, as described above, the boundary information may include boundary virtual content 451. In some cases, the boundary information may alternatively or additionally include coordinate information defining the location of the boundary virtual content 451 in a 3D map (e.g., a SLAM map) and feature points in a table 459 defining where the boundary virtual content 451 is to be anchored or registered. The augmented reality system 100 may provide the boundary information to XR devices of other users (non-authenticated users) who are not authenticated to view and / or interact with the virtual content 458. The XR devices of the other users may use the boundary information to generate and / or display the boundary virtual content 451. In some cases, the augmented reality system 100 may display the boundary virtual content 451 around the virtual content 458 so that the authenticated user 456 can view the boundary virtual content 451. In some cases, the XR devices of the authenticated users 452, 454, 456 do not display the boundary virtual content 451 for viewing by the authenticated users 452, 454, 456.

[0096]

[0104] In some examples, as shown in FIG. 4, boundary virtual content 451 may be a virtual boundary displayed around the top portion of a table where virtual content is displayed (e.g., as a three-dimensional box around the virtual content). In other examples, boundary virtual content 451 may be a virtual boundary displayed around the entire table 459 that is used as a backdrop for the virtual content. In some cases, an application associated with a particular virtual session may define how the boundary virtual content is displayed. For example, the virtual poker game shown in FIG. 4 may have a particular game board defined by the application used to facilitate the virtual poker game. Boundary virtual content 451 may be defined by the application such that it surrounds the game board of the virtual poker game. The boundary defined by boundary virtual content 451 may be configured in any other manner.

[0097]

[0105] The illustration in FIG. 4 is from the perspective of the content that user 456 is viewing through augmented reality system 100. As shown, user 456 can view virtual content 458 of virtual private space 450 and can also view an underlying physical table 459. In some examples, augmented reality system 100 can display boundary virtual content 451 for viewing by authenticated user 456. In some examples, augmented reality system 100 does not display boundary virtual content 451, for example, to avoid authenticated user 456 being distracted by boundary virtual content 451. Two other authenticated users 452 and 454 can also view the same virtual content 458 and table 459 through their respective XR devices, but from different perspectives or angles (corresponding to where each of authenticated users 452 and 454 is sitting at table 459). As described with respect to FIG. 5 below, XR devices or systems of unauthorized users will not have access to virtual content 458 and will not be able to display it. For example, any unauthorized user using an XR device will not be able to view the virtual content 458, but instead will only be able to view the boundary virtual content 451. An unauthorized user not using an XR device or system may be notified of the existence and / or location of the virtual private space 450.

[0098]

[0106] Figure 5 illustrates another view of the virtual private space 450 from Figure 4. The view in Figure 5 is from the perspective of an unauthenticated user and how such an unauthenticated user may view the virtual private space 450 when using an XR device. Actual (physical) authenticated users 452, 454, and 456 are shown seated around a table 459, each wearing a respective XR device (shown for purposes of illustration as AR glasses). As noted above, authenticated users 452, 454, 456 are authorized to use their respective XR devices to view the virtual content 458 and, in some cases, the boundary virtual content 451 of the virtual session shown in Figure 4.

[0099]

[0107] Unauthenticated user 560 and unauthenticated user 562 are also shown in FIG. 5. As previously mentioned, the unauthenticated user's XR device will not have access to and will not be able to display virtual content 458. For example, any unauthenticated user using an XR device will only be able to view boundary virtual content 451. Unauthenticated user 562 is a person sitting at table 559 near table 459 where authenticated users 452, 454, and 456 are sitting. As shown, unauthenticated user 562 is wearing XR system 564 (e.g., AR glasses) and may be viewing virtual content, but is not authorized to view virtual content 458 shown in virtual private space 450. Because unauthenticated user 562 is not authorized to view virtual content 458, when unauthenticated user 562 turns so that the field of view of XR system 564 is directed toward table 459, XR system 564 will display boundary virtual content 451. Boundary virtual content 451 blocks unauthorized users 562 from viewing virtual content 458 in virtual private space 450 .

[0100]

[0108] 6 is a diagram illustrating a profile view of table 459 and boundary virtual content 451. As shown in FIG. 6, boundary virtual content 451 may be displayed in three dimensions to an unauthenticated user, which may allow the virtual content within virtual private space 450 to remain private from all viewing angles. Boundary virtual content 451 may be displayed differently to an authenticated user. For example, boundary virtual content 451 may be displayed in three dimensions to an unauthenticated user and may be displayed in a modified manner or not at all to an authenticated user. In one example, boundary virtual content 451 may be displayed as a two-dimensional dotted outline to an authenticated user so as not to damage the view of virtual content 458.

[0101]

[0109] An unauthenticated user not using an XR device may be notified of the existence and / or location of the virtual private space 450. The unauthenticated user 560 shown in FIG. 5 is a service person employed by a restaurant. As shown, the unauthenticated user 560 is carrying a tray of drinks to a table 459 for authenticated users 452, 454, and 456. The unauthenticated user 560 is not using an XR device and therefore cannot view the boundary virtual content 451. The unauthenticated user 560 has a mobile device 561. Any one or more of the XR devices of the authenticated users 452, 454, and 456 (including the augmented reality system 100) can detect the presence of the mobile device 561 and cause the mobile device 561 to send a notification indicating the existence and / or location of the virtual private space 450. In some examples, the augmented reality system 100 (or the XR device of one of the users 452, 454) can send a message to the mobile device 561 with information indicating the existence and location of the virtual private space 450. In another example, a server providing virtual content 458 to the XR devices of the authenticated users 452, 454, 456 can send a message to the mobile device 561 of the unauthenticated user 560. As shown in FIG. 5 , the message can indicate that there is a “virtual private space at table 3,” which refers to the table 459 where the authenticated users 452, 454, 456 are seated. Based on the notification or message, the unauthenticated user 560 can be aware that the virtual private space 450 exists at the table 459 and can avoid placing a beverage in the physical space occupied by the virtual private space 450.

[0102]

[0110] If an unauthenticated user 560 places a beverage or other physical item within the virtual private space 450, one or more of the XR devices of the authenticated users 452, 454, 456 may output a notification to the authenticated users 452, 454, 456. For example, the augmented reality system 100 of the authenticated user 456 may display a notification on the display 109 alerting the authenticated user 456 that a physical item has been placed in the virtual private space 450.

[0103]

[0111] In some cases, one or more of the XR devices of the authenticated users 452, 454, 456 can detect when the unauthenticated user 560 is within a certain distance from the virtual private space 450. For example, a threshold distance such as 10 feet, 5 feet, 2 feet, and / or other distances may be defined. If the unauthenticated user 560 moves within the threshold distance from the virtual private space 450, one or more of the XR devices of the authenticated users 452, 454, 456 can output a notification to the authenticated user 452, 454, 456. For example, the augmented reality system 100 of the authenticated user 456 can display a notification on the display 109 alerting the authenticated user 456 that a person or object is approaching the virtual private space 450. In some examples, the threshold distance may be defined based on the physical environment in which the virtual private space is implemented. For example, in a restaurant environment, there may be many people moving throughout the physical environment. In such cases, the threshold distance may be smaller to avoid a large number of notifications being sent to the XR devices of authenticated users 452, 454, 456. In environments with fewer people or objects, the threshold distance may be larger.

[0104]

[0112] As previously described, in some cases, virtual content in a virtual private space may be displayed with a higher priority than real-world content. FIGS. 7A-7C illustrate an example of a person 760 physically entering a virtual private space 750 of a virtual session taking place in a restaurant (similar to the example of FIG. 4). As shown in FIG. 7A, virtual private space 750 includes virtual content 758 and boundary virtual content 751. Virtual private space 750 is displayed as an overlay on a physical object (not shown in FIG. 7A), such as a table (similar to table 459 shown in FIG. 4). Person 760 approaches virtual private space 750 and places beverages 763 and 764 on the physical object on which the virtual content of virtual private space 750 is displayed. Beverages 763 and 764 can interfere with the view of the virtual content by authenticated users of virtual private space 750.

[0105]

[0113] An XR device (e.g., augmented reality system 100) of an authenticated user of virtual private space 750 and / or a server in communication with the XR device can de-emphasize beverages 763 and 764 and / or portions of unauthenticated user 760 while they are located within virtual private space 750. For example, as shown in FIG. 7B , beverages 763 and 764 and the arm of unauthenticated user 760 are completely obstructed so that the objects are transparent from the perspective of an authenticated user viewing the virtual content in virtual private space 750. In AR systems, the display is generally an optical see-through display, in which case a user of the AR system can see the physical world through the display and can also see virtual objects on the display. The virtual objects appear to be superimposed on the physical world. Physical objects can be made transparent from the perspective of a user of the AR system by generating AR content that obstructs the physical objects. In one illustrative example, a background model that models the background of a given environment can be generated. For example, the background model may model a static portion of a restaurant where a virtual session related to virtual private space 750 is taking place. Drinks 763 and 764 and the arm of the unauthenticated user 760 are augmented (or obstructed) with portions of the background model that correspond to the portions of the restaurant that are behind drinks 763 and 764 and the arm of the unauthenticated user 760, and therefore those portions of the background model appear in front of drinks 763 and 764 and the arm of the unauthenticated user 760. Frames being captured by the XR device of an authenticated user viewing the virtual content of virtual private space 750 may be repainted with the content (e.g., pixels) of those portions of the background model.

[0106]

[0114] In some examples, an XR device of an authenticated user participating in a virtual session associated with virtual private space 750 may provide a visual, audible, or other type of indication of the location of beverages 763 and 764 and the arm of unauthenticated user 760. For example, the authenticated user's XR device may display a notification on the display of the XR device alerting the authenticated user that beverage 763 has been placed in virtual private space 450.

[0107]

[0115] 7C illustrates another example of de-emphasizing beverages 763 and 764 and the arm of the unauthenticated user 760 while the beverages 763 and 764 and the arm of the unauthenticated user 760 are located within virtual private space 750. As shown in FIG. 7C, the beverages 763 and 764 and the arm of the unauthenticated user 760 are displayed with a modified appearance, so that the objects have a less disruptive effect on authenticated users participating in a virtual session associated with virtual private space 750. In some cases, the authenticated user's XR device may first fully display the beverages 763 and 764 and the arm of the unauthenticated user 760 within the virtual private space 750, and then gradually fade away the display of the beverages 763 and 764 and the arm of the unauthenticated user 760 until the objects are fully obstructed.

[0108]

[0116] The virtual private spaces described herein can provide various benefits in AR and / or other XR environments. For example, as described above, a virtual private space can prevent the collapse of activity in a space containing virtual content. A virtual private space can also prevent unauthorized users from viewing the virtual content. In some cases, a user authorized to view content in the virtual private space (referred to as an authenticated user) may be alerted to the presence or proximity of one or more unauthorized users in the virtual private space. In some cases, an alert may be provided to an authorized user if the virtual content in the virtual private space is compromised in some way (e.g., an unauthorized user affecting the real-world space, causing some effect on the virtual content).

[0109]

[0117] 8 is a flowchart illustrating an example of a process 800 for generating virtual content for one or more virtual private spaces using the techniques described herein. At block 802, process 800 includes initiating, by a device, a virtual session for presenting the virtual content. At block 804, process 800 includes identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content. In one illustrative example, the virtual private space may include virtual private space 450 shown in FIGS. 4-6.

[0110]

[0118] At block 806, process 800 includes outputting boundary information defining a boundary of the virtual private space. As described above, the boundary information may be used to generate boundary virtual content defining the boundary of the virtual private space. In some examples, as described above, the boundary information includes world coordinates in the physical space and / or one or more feature points in at least one image of the physical space that can be used to generate boundary virtual content identifying a boundary for the virtual private space. In some examples, process 800 includes outputting boundary virtual content that identifies a boundary or extent for the virtual private space. The boundary virtual content is viewable by one or more unauthenticated users of the virtual session.

[0111]

[0119] At block 808, process 800 includes generating at least a portion of virtual content for the virtual private space. In some cases, the entire virtual content may be generated for the virtual private space. In some cases, a portion of the virtual content (less than the entire virtual content) may be generated for the virtual private space. For example, a first portion of the virtual content may extend beyond the boundary of the virtual private space, and more sensitive content (e.g., defined by an authenticated user, defined by a content provider associated with the virtual session, or defined as confidential) may be restricted within the boundary of the virtual private space. At least a portion of the virtual content viewable in the virtual private space by one or more authenticated users of the virtual session is not viewable by one or more unauthenticated users. For example, an augmented reality device or other device of each of the one or more authenticated users is authorized or authenticated to display at least a portion of the virtual content, and an augmented reality device or other device of each of the one or more unauthenticated users is not authorized or authenticated to display at least a portion of the virtual content. In some examples, at least a portion of the virtual content is not viewable by one or more unauthenticated users based on boundary virtual content associated with boundary information. In some examples, a real-world volume defined within the boundary virtual content is not viewable by one or more unauthorized users based on the boundary virtual content associated with the boundary information.

[0112]

[0120] In some examples, process 800 includes receiving an indication that an unauthorized user has entered the virtual private space. Based on the indication that an unauthorized user has entered the virtual private space, process 800 can prevent at least a portion of the unauthorized user from being viewable in the virtual private space by one or more authenticated users.

[0113]

[0121] In some examples, process 800 includes receiving an indication that an unauthorized user has entered or is within a threshold distance of entering the virtual private space. Based on the indication that an unauthorized user has entered or is within a threshold distance of entering the virtual private space, process 800 can move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0114]

[0122] In some examples, process 800 includes receiving an indication that an unauthorized user has entered or is within a threshold distance of entering a virtual private space. Based on the indication that an unauthorized user has entered or is within a threshold distance of entering a virtual private space, process 800 can output a notification indicating the existence of the virtual private space. In some cases, outputting the notification includes sending a notification to the unauthorized user's device, outputting an audio notification indicating the existence of the virtual private space, a combination thereof, and / or outputting another type of notification (such as those described herein). In some cases, the notification includes a summary of the unauthorized user.

[0115]

[0123] In some examples, process 800 includes receiving an indication that an unauthorized user has entered or is within a threshold distance of entering a virtual private space. Based on the indication that an unauthorized user has entered or is within a threshold distance of entering a virtual private space, process 800 can output a notification to one or more devices of one or more authorized users indicating that an unauthorized user has entered or is within a threshold distance of entering a virtual private space. In some cases, outputting the notification includes sending a notification to a device of at least one of the one or more authorized users, outputting an audio notification indicating the presence of the unauthorized user, a combination thereof, and / or outputting another type of notification (such as those described herein). In some cases, the notification includes a summary of the unauthorized user.

[0116]

[0124] In some examples, the device is an augmented reality device that includes one or more displays, one or more receivers (capable of receiving data and / or other information), one or more transmitters (capable of transmitting data and / or other information), and / or one or more transceivers (capable of receiving and transmitting data and / or other information). In some examples, process 800 includes displaying at least a portion of the virtual content by the augmented reality device (e.g., by a display of the augmented reality device).

[0117]

[0125] In some examples, the device is a first augmented reality device (including one or more displays, one or more receivers, one or more transmitters, and / or one or more transceivers). In some examples, process 800 includes outputting at least a portion of the virtual content to a second augmented reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers). The second augmented reality device can be used by an authenticated user.

[0118]

[0126] In some examples, the device is a first augmented reality device (including one or more displays, one or more receivers, one or more transmitters, and / or one or more transceivers). In some examples, process 800 includes displaying at least a portion of the virtual content by the first augmented reality device (e.g., by a display of the augmented reality device) and outputting at least a portion of the virtual content to a second augmented reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers).

[0119]

[0127] In some examples, process 800 includes displaying at least a portion of the virtual content by a first augmented reality device (e.g., by a display of the augmented reality device) and outputting at least a portion of the virtual content by the first augmented reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers) to a second augmented reality device. In some examples, process 800 includes receiving boundary information by the second augmented reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers of the second augmented reality device) and generating virtual boundary content from the received boundary information by the second augmented reality device (e.g., using one or more processors of the second augmented reality device).

[0120]

[0128] In some examples, one or more of the devices or apparatuses is a server device. In some examples, process 800 includes outputting, by the server device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers of the server device), at least a portion of the virtual content to the augmented reality device.

[0121]

[0129] In some examples, the processes described herein (e.g., process 800 and / or other processes described herein) may be performed by a computing device or apparatus. In one example, process 800 may be performed by the augmented reality system 100 of FIG. 1. In another example, process 800 may be performed by a computing device having the computing system 900 shown in FIG. 9. For example, a computing device having the computing architecture shown in FIG. 9 may include the components of the augmented reality system 100 of FIG. 1 and may implement the operations of FIG. 8.

[0122]

[0130] The computing device may include any suitable device, such as a mobile device (e.g., a mobile phone), a desktop computing device, a tablet computing device, a wearable device (e.g., a VR headset, an AR headset, AR glasses, a network-connected watch or smartwatch, or other wearable device), a server computer, an autonomous vehicle or a computing device of an autonomous vehicle, a robotic device, a television, and / or any other computing device, having the resource capabilities to perform the processes described herein, including process 800. In some cases, the computing device or device may include various components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and / or other components configured to perform the steps of the processes described herein. In some examples, the computing device may include a display, a network interface configured to communicate and / or receive data, any combination thereof, and / or other components. The network interface may be configured to communicate and / or receive Internet Protocol (IP)-based data or other types of data.

[0123]

[0131] Components of a computing device may be implemented with circuitry. For example, components may include and / or be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable electronic circuits (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuitry), and / or may include and / or be implemented using computer software, firmware, or any combination thereof, to perform various operations described herein.

[0124]

[0132] Process 800 is illustrated as a logical flow diagram, whose operations represent sequences of actions that may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the described actions. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular data types. The order in which the actions are described should not be construed as a limitation, and any number of the described actions may be combined in any order and / or in parallel to implement a process.

[0125]

[0133] Additionally, process 800 and / or other processes described herein may be performed under the control of one or more computer systems configured of executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that collectively execute on one or more processors, by hardware, or a combination thereof. As noted above, the code may be stored on a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-transitory.

[0126]

[0134] 9 illustrates an example of a system for implementing some aspects of the present technology. In particular, FIG. 9 illustrates an example of a computing system 900, which may be, for example, an internal computing system, a remote computing system, a camera, or any computing device making up any of these components, where the components of the system communicate with each other using a connection 905. The connection 905 may be a physical connection using a bus, such as in a chipset architecture, or a direct connection to a processor 910. The connection 905 may also be a virtual connection, a networked connection, or a logical connection.

[0127]

[0135] In some embodiments, computing system 900 is a distributed system in which the functions described in this disclosure may be distributed within a data center, multiple data centers, a peer network, etc. In some embodiments, one or more of the described system components represent many such components, each of which performs some or all of the described functions. In some embodiments, the components may be physical or virtual devices.

[0128]

[0136] The exemplary system 900 includes at least one processing unit (CPU or processor) 910 and connections 905 coupling various system components to the processor 910, including system memory 915 such as read-only memory (ROM) 920 and random access memory (RAM) 925. The computing system 900 may include a cache 912 of high-speed memory directly connected to the processor 910, in close proximity to the processor 910, or integrated as part of the processor 910.

[0129]

[0137] Processor 910 can include any general-purpose processor, as well as hardware or software services, such as services 932, 934, 936 stored in storage device 930, configured to control processor 910, and special-purpose processors in which software instructions are incorporated into the actual processor design. Processor 910 can essentially be a completely self-contained computing system, including multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors can be symmetric or asymmetric.

[0130]

[0138] To enable user interaction, computing system 900 includes input devices 945, which can represent any number of input mechanisms, such as a microphone for audio, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, voice, etc. Computing system 900 can also include output devices 935, which can be one or more of several output mechanisms. In some instances, a multimodal system can allow a user to provide multiple types of input / output to communicate with computing system 900. Computing system 900 can include a communication interface 940, which can generally govern and manage user input and system output.The communication interface may be an audio jack / plug, a microphone jack / plug, a Universal Serial Bus (USB) port / plug, an Apple® Lightning® port / plug, an Ethernet® port / plug, an optical fiber port / plug, a proprietary wired port / plug, a BLUETOOTH® wireless signal transmission, a BLUETOOTH Low Energy (BLE) wireless signal transmission, an IBEACON® wireless signal transmission, a Radio Frequency Identification (RFID) wireless signal transmission, a Near Field Communication (NFC) wireless signal transmission, a Dedicated Short Range Communication (DSRC) wireless signal transmission, an 802.11 Wi-Fi® wireless signal transmission, a Wireless Local Area Network (WLAN) wireless signal transmission, a Communications interface 940 may implement or facilitate the reception and / or transmission of wired or wireless communications using wired and / or wireless transceivers, including transceivers utilizing wireless LAN (LAN) signal transfer, visible light communications (VLC), Worldwide Interoperability for Microwave Access (WiMAX®), infrared (IR) communications wireless signal transfer, public switched telephone network (PSTN) signal transfer, integrated services digital network (ISDN) signal transfer, 3G / 4G / 5G / LTE® cellular data network wireless signal transfer, ad hoc network signal transfer, radio wave signal transfer, microwave signal transfer, infrared signal transfer, visible light signal transfer, ultraviolet light signal transfer, wireless signal transfer along the electromagnetic spectrum, or some combination thereof. Communications interface 940 may also include one or more Global Navigation Satellite System (GNSS) receivers or transceivers used to determine the location of computing system 900 based on reception of one or more signals from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the US-based Global Positioning System (GPS), the Russian-based Global Navigation Satellite System (GLONASS), the Chinese-based Beidou Navigation Satellite System (BDS), and the European-based Galileo GNSS.There is no restriction to operating on any particular hardware configuration, and therefore the basic features herein can be easily substituted for improved hardware or firmware configurations as they are developed.

[0131]

[0139] The storage device 930 may be a magnetic cassette, a flash memory card, a solid state memory device, a digital versatile disk, a cartridge, a floppy disk, a flexible disk, a hard disk, a magnetic tape, a magnetic strip / stripe, any other magnetic storage medium, a flash memory, a memristor memory, any other solid state memory, a compact disk read only memory (CD-ROM) optical disk, a rewritable compact disk (CD) optical disk, a digital video disk (DVD) optical disk, a Blu-ray Disc (BDD) optical disk, a holographic optical disk, another optical medium, a secure digital (SD) card, a micro secure digital (microSD) card, a memory Stick® card, smart card chip, EMV chip, subscriber identity module (SIM) card, mini / micro / nano / pico SIM card, another integrated circuit (IC) chip / card, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read only memory (ROM), programmable read only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM®), flash EPROM (FLASHEPROM), cache memory (L1 / L2 / L3 / L4 / L5 / L#), resistive random access memory (RRAM® / ReRAM), phase change memory (PCM), spin transfer torque RAM (STT-RAM), another memory chip or cartridge, and / or combinations thereof, may be a hard disk or other type of computer readable medium capable of storing data that is accessible by a computer.

[0132]

[0140] The storage device(s) 930 may include software services, servers, services, etc. that, when code defining such software is executed by the processor 910, cause the system to perform a function. In some embodiments, hardware services that perform a particular function may include software components stored on a computer-readable medium in conjunction with necessary hardware components, such as the processor 910, connections 905, output devices 935, etc., to perform that function. The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instruction(s) and / or data. Computer-readable media may include non-transitory media on which data may be stored and that do not include carrier waves and / or transitory electronic signals propagating wirelessly or over wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. A computer-readable medium may have stored thereon code and / or machine-executable instructions, which may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0133]

[0141] In some embodiments, computer-readable storage devices, media, and memories may include cable or wireless signals containing bitstreams, etc. However, when stated, non-transitory computer-readable storage media specifically excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0134]

[0142] Specific details are provided in the above description to provide a thorough understanding of the embodiments and examples provided herein. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For clarity of explanation, in some instances, the technology may be presented as including individual functional blocks comprising devices, device components, steps, or routines in methods implemented in software or a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may be used. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments.

[0135]

[0143] Individual embodiments may be described above as a process or method that is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Moreover, the order of operations may be rearranged. A process is terminated when its operations are completed but may have additional steps not included in the diagram. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to a calling function or a main function.

[0136]

[0144] The processes and methods according to the above-described examples may be implemented using computer-executable instructions stored on or otherwise available from a computer-readable medium. Such instructions may include, for example, instructions and data that cause a general-purpose computer, special-purpose computer, or processing device to perform a certain function or group of functions, or otherwise configure a general-purpose computer, special-purpose computer, or processing device to perform a certain function or group of functions. Portions of the computer resources used may be accessible over a network. The computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions during, information used in, and / or information created by, the methods according to the described examples include magnetic or optical disks, flash memory, USB devices with non-volatile memory, network-attached storage devices, etc.

[0137]

[0145] Devices implementing processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) to perform the necessary tasks may be stored on a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Typical examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mounted devices, standalone devices, etc. The functionality described herein may also be implemented in peripheral devices or add-in cards. Such functionality may also be implemented on a circuit board, in different chips, or different processes running within a single device, as further examples.

[0138]

[0146] The instructions, media for carrying such instructions, computing resources for executing them, and other structures for supporting such computing resources are exemplary means for providing the functionality described in this disclosure.

[0139]

[0147] In the foregoing description, aspects of the present application have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the present application is not limited thereto. Accordingly, while exemplary embodiments of the present application have been described in detail herein, it should be understood that the inventive concepts may, in some cases, be embodied and employed in various ways, and that the appended claims are intended to include such variations, except as limited by the prior art. Various features and aspects of the applications described above may be used individually or together. Furthermore, the embodiments may be utilized in any number of environments and applications other than those described herein without departing from the broader spirit and scope of the present specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive. For illustrative purposes, methods have been described in a particular order. It should be appreciated that in alternative embodiments, methods may be performed in an order different from that described.

[0140]

[0148] Those skilled in the art will appreciate that the less than ("<") and greater than (">") symbols or terminology used herein may be replaced with the less than or equal to ("≦") and greater than or equal to ("≧") symbols, respectively, without departing from the scope of this specification.

[0141]

[0149] When a component is described as being "configured to" perform an operation, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuit (e.g., a microprocessor or other suitable electronic circuitry) to perform the operation, or by any combination thereof.

[0142]

[0150] The phrase "coupled to" refers to any component that is directly or indirectly physically connected to another component and / or that is in direct or indirect communication with another component (e.g., connected to the other component over a wired or wireless connection and / or other suitable communication interface).

[0143]

[0151] Claim language or other language reciting "at least one of" a set and / or "one or more" of a set indicates that one member of the set or multiple members of the set (in any combination) satisfies the claim. For example, claim language reciting "at least one of A and B" or "at least one of A or B" means A, B, or A and B. As another example, claim language reciting "at least one of A, B, and C" or "at least one of A, B, or C" means A, B, C, or A and B, or A and C, or B and C, or A, B, and C. The language "at least one of" a set and / or "one or more" of a set does not limit the set to the items listed in the set. For example, claim language reciting "at least one of A and B" or "at least one of A or B" can mean A, B, or A and B, and can further include items not listed in the set of A and B.

[0144]

[0152] The various illustrative logic blocks, modules, circuits, and algorithm steps described in connection with the examples disclosed herein may be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability of hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled engineers may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0145]

[0153] The techniques described herein may be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general-purpose computer, a wireless communication device handset, or an integrated circuit device having multiple uses, including applications in wireless communication device handsets and other devices. Features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be realized at least in part by a computer-readable data storage medium comprising program code including instructions that, when executed, perform one or more of the methods, algorithms, and / or operations described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise a memory or data storage medium, such as a random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), a read-only memory (ROM), a nonvolatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), a FLASH memory, a magnetic or optical data storage medium, etc. The techniques may additionally or alternatively be realized at least in part by a computer-readable communications medium, such as a propagated signal or radio waves, that carries or communicates program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer.

[0146]

[0154] The program code may be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Such a processor may be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Accordingly, the term “processor” as used herein may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or apparatus suitable for implementing the techniques described herein.

[0147]

[0155] Example 1: A method for generating virtual content. The method includes initiating, by a device, a virtual session for presenting the virtual content, identifying, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content, outputting boundary information defining a boundary of the virtual private space, generating at least a portion of the virtual content for the virtual private space, wherein at least a portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users.

[0148]

[0156] Example 2: The method of Example 1, wherein at least a portion of the virtual content is not viewable by one or more unauthorized users based on boundary virtual content associated with the boundary information.

[0149]

[0157] Example 3: The method of any of Examples 1 or 2, wherein a real-world volume defined within the boundary virtual content is not viewable by one or more unauthorized users based on the boundary virtual content associated with the boundary information.

[0150]

[0158] Example 4: The method of any of Examples 1 to 3, further comprising outputting boundary virtual content that identifies a boundary for the virtual private space, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session.

[0151]

[0159] Example 5: A method according to any of Examples 1 to 4, wherein the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate boundary virtual content that identifies a boundary for the virtual private space, and world coordinates in the physical space.

[0152]

[0160] Example 6: The method of any of Examples 1 to 5, further comprising: receiving an indication that an unauthorized user has entered the virtual private space; and, based on the indication that an unauthorized user has entered the virtual private space, rendering at least a portion of the unauthorized user inaccessible to one or more authorized users in the virtual private space.

[0153]

[0161] Example 7: The method of any of Examples 1 to 6, further comprising: receiving an indication that an unauthenticated user has entered or is within a threshold distance of entering the virtual private space; and, based on the indication that the unauthenticated user has entered or is within a threshold distance of entering the virtual private space, moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthenticated user.

[0154]

[0162] Example 8: The method of any of Examples 1 to 7, further comprising: receiving an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space; and outputting a notification indicating the existence of the virtual private space based on the indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space.

[0155]

[0163] Example 9: The method of Example 8, wherein outputting the notification includes at least one of sending a notification to a device of the unauthenticated user and outputting an audio notification indicating the existence of the virtual private space.

[0156]

[0164] Example 10: A method as described in any of Examples 1 to 9, further comprising receiving an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space, and outputting a notification to one or more devices of one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering a virtual private space based on the indication that the unauthenticated user has entered or is within the threshold distance of entering a virtual private space.

[0157]

[0165] Example 11: The method of example 10, wherein the notification includes a summary of the unauthenticated user.

[0158]

[0166] Example 12: The method of any of Examples 1 to 11, wherein the device is an augmented reality device, and further comprising displaying at least a portion of the virtual content by the augmented reality device.

[0159]

[0167] Example 13: The method of any of Examples 1 to 12, wherein the device is a first augmented reality device, and further comprising: outputting at least a portion of the virtual content to a second augmented reality device.

[0160]

[0168] Example 14: The method of any of Examples 1 to 13, wherein the device is a first augmented reality device, further comprising displaying at least a portion of the virtual content by the first augmented reality device and outputting at least a portion of the virtual content to a second augmented reality device.

[0161]

[0169] Example 15: The method of any of Examples 1 to 14, wherein the device is a first augmented reality device, further comprising: displaying at least a portion of the virtual content by the first augmented reality device; outputting at least a portion of the virtual content to a second augmented reality device; receiving boundary information by the second augmented reality device; and generating, by the second augmented reality device, virtual boundary content from the received boundary information.

[0162]

[0170] Example 16: The method of any of Examples 1 to 15, wherein the device is a server device, and further comprising outputting, by the server device, at least a portion of the virtual content to the augmented reality device.

[0163]

[0171] Example 17: An apparatus for generating virtual content, the apparatus including: a memory configured to store the virtual content; and one or more processors coupled to the memory, the processor configured to: initiate a virtual session for presenting the virtual content; identify, for the virtual session, a portion of a physical space for use as a virtual private space for presenting at least a portion of the virtual content; output boundary information defining a boundary of the virtual private space; generate at least a portion of the virtual content for the virtual private space; and, wherein at least a portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and is not viewable by one or more unauthenticated users.

[0164]

[0172] Example 18: The apparatus of Example 17, wherein at least a portion of the virtual content is not viewable by one or more unauthorized users based on boundary virtual content associated with the boundary information.

[0165]

[0173] Example 19: An apparatus described in either Example 17 or 18, wherein a real-world volume defined within the boundary virtual content is not viewable by one or more unauthorized users based on the boundary virtual content associated with the boundary information.

[0166]

[0174] Example 20: An apparatus described in any of Examples 17 to 19, wherein one or more processors are configured to: output boundary virtual content that identifies a boundary for the virtual private space, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session.

[0167]

[0175] Example 21: An apparatus described in any of Examples 17 to 20, wherein the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate boundary virtual content that identifies a boundary for the virtual private space, and world coordinates in the physical space.

[0168]

[0176] Example 22: One or more processors receive an indication that an unauthorized user has entered a virtual private space; and, based on the indication that an unauthorized user has entered a virtual private space, 22. The apparatus of any of Examples 17 to 21, configured to: prevent at least a portion of unauthorized users from being viewed by one or more authorized users in the virtual private space.

[0169]

[0177] Example 23: An apparatus described in any of Examples 17 to 22, configured to receive an indication that an unauthorized user has entered or is within a threshold distance of entering the virtual private space, and, based on the indication that the unauthorized user has entered or is within the threshold distance of entering the virtual private space, move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0170]

[0178] Example 24: The device described in any of Examples 17 to 23, wherein the one or more processors are configured to receive an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space, and output a notification indicating the existence of a virtual private space based on the indication that an unauthenticated user has entered or is within the threshold distance of entering a virtual private space.

[0171]

[0179] Example 25: The device of Example 24, wherein outputting the notification includes at least one of sending a notification to a device of an unauthenticated user and outputting an audio notification indicating the existence of a virtual private space.

[0172]

[0180] Example 26: The apparatus of any of Examples 17 to 25, wherein the one or more processors are configured to: receive an indication that an unauthenticated user has entered or is within a threshold distance of entering a virtual private space; and, based on the indication that an unauthenticated user has entered or is within the threshold distance of entering a virtual private space, output a notification to one or more devices of one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering a virtual private space.

[0173]

[0181] Example 27: The apparatus of example 26, wherein the notification includes a summary of the unauthenticated user.

[0174]

[0182] Example 28: An apparatus described in any of Examples 17 to 27, wherein the apparatus is a first augmented reality device and the one or more processors are configured to cause at least a portion of the virtual content to be output to a second augmented reality device.

[0175]

[0183] Example 29: The apparatus of any of Examples 17 to 28, wherein the apparatus is an augmented reality device.

[0176]

[0184] Example 30: The device of any of Examples 17 to 29, further comprising a display.

[0177]

[0185] Example 31: The device of Example 30, wherein the display is configured to display at least a portion of the virtual content.

[0178]

[0186] Example 32: The device described in Example 30, wherein the device is a first augmented reality device, the display is configured to display at least a portion of the virtual content, and the one or more processors are configured to output at least a portion of the virtual content to a second augmented reality device.

[0179]

[0187] Example 33: The device described in Example 30, wherein the device is a first augmented reality device, the display is configured to display at least a portion of the virtual content, the one or more processors are configured to output at least a portion of the virtual content to a second augmented reality device, the second augmented reality device is configured to receive boundary information, and the second augmented reality device is configured to generate virtual boundary content from the received boundary information.

[0180]

[0188] Example 34: An apparatus described in any of Examples 17 to 33, wherein the apparatus is a server device and the one or more processors are configured to cause the server device to output at least a portion of the virtual content to the augmented reality device.

[0181]

[0189] Example 35: A non-transitory computer-readable medium for an augmented reality system having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform any of the operations of Examples 1 to 16.

[0182]

[0190] Example 36: An augmented reality system including one or more means for performing any of the operations of Examples 1 to 16. The inventions described in the claims of the present application as originally filed are set forth below. [C1] 1. A method for generating virtual content, comprising: initiating, by the device, a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information defining a boundary of the virtual private space; generating at least the portion of the virtual content for the virtual private space, wherein at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users; A method for providing [C2] The method of C1, wherein at least the portion of the virtual content is not viewable by the one or more unauthorized users based on the boundary virtual content associated with the boundary information. [C3] The method of claim 1, wherein a real-world volume defined within the boundary virtual content is not viewable by the one or more unauthenticated users based on the boundary virtual content associated with the boundary information. [C4] The method of C1, further comprising outputting boundary virtual content that identifies the boundary for the virtual private space, the boundary virtual content being viewable by the one or more unauthenticated users of the virtual session. [C5] The method of claim 1, wherein the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate the boundary virtual content that identifies the boundary for the virtual private space, and world coordinates in the physical space. [C6] receiving an indication that an unauthorized user has entered the virtual private space; based on the indication that the unauthorized user has entered the virtual private space, rendering at least a portion of the unauthorized user inaccessible to the one or more authorized users in the virtual private space; The method of C1, further comprising: [C7] receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized users based on the indication that the unauthorized users have entered or are within the threshold distance of entering the virtual private space; The method of C1, further comprising: [C8] receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; outputting a notification indicating the existence of the virtual private space based on the indication that the unauthorized user has entered or is within the threshold distance of entering the virtual private space; The method of C1, further comprising: [C9] The method of C8, wherein outputting the notification includes at least one of sending the notification to a device of the unauthenticated user and outputting an audio notification indicating the existence of the virtual private space. [C10] receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; Based on the indication that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space, outputting a notification to one or more devices of the one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space; The method of C1, further comprising: [C11] The method of C10, wherein the notification includes a summary of the unauthenticated user. [C12] the device is an augmented reality device; displaying, by the augmented reality device, at least the portion of the virtual content. The method described in C1. [C13] the device is a first augmented reality device; outputting at least the portion of the virtual content to a second augmented reality device. The method described in C1. [C14] the device is a first augmented reality device; displaying, by the first augmented reality device, at least the portion of the virtual content; outputting at least the portion of the virtual content to a second augmented reality device; The method of C1, further comprising: [C15] the device is a first augmented reality device; displaying, by the first augmented reality device, at least the portion of the virtual content; outputting at least the portion of the virtual content to a second augmented reality device; receiving, by the second augmented reality device, the boundary information; generating, by the second augmented reality device, the virtual boundary content from the received boundary information; The method of C1, further comprising: [C16] the device is a server device; outputting, by the server device, at least the portion of the virtual content to an augmented reality device. The method described in C1. [C17] 1. An apparatus for generating virtual content, comprising: a memory configured to store virtual content; one or more processors coupled to the memory; wherein the processor: initiating a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information defining a boundary of the virtual private space; generating at least the portion of the virtual content for the virtual private space, wherein at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users; An apparatus configured to: [C18] The device of C17, wherein at least the portion of the virtual content is not viewable by the one or more unauthorized users based on the boundary virtual content associated with the boundary information. [C19] The device of C17, wherein a real-world volume defined within the boundary virtual content is not viewable by the one or more unauthenticated users based on the boundary virtual content associated with the boundary information. [C20] The device described in C17, wherein the one or more processors are configured to output boundary virtual content that identifies the boundary for the virtual private space, the boundary virtual content being viewable by the one or more unauthenticated users of the virtual session. [C21] The device described in C17, wherein the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate the boundary virtual content that identifies the boundary for the virtual private space, and world coordinates in the physical space. [C22] the one or more processors: receiving an indication that an unauthorized user has entered the virtual private space; based on the indication that the unauthorized user has entered the virtual private space, rendering at least a portion of the unauthorized user inaccessible to the one or more authorized users in the virtual private space; 18. The apparatus of claim 17, configured to: [C23] receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized users based on the indication that the unauthorized users have entered or are within the threshold distance of entering the virtual private space; The apparatus of C17, further comprising: [C24] the one or more processors: receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; outputting a notification indicating the existence of the virtual private space based on the indication that the unauthorized user has entered or is within the threshold distance of entering the virtual private space; 18. The apparatus of claim 17, configured to: [C25] The device of C24, wherein outputting the notification includes at least one of sending the notification to a device of the unauthenticated user and outputting an audio notification indicating the existence of the virtual private space. [C26] the one or more processors: receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; Based on the indication that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space, outputting a notification to one or more devices of the one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space; 18. The apparatus of claim 17, configured to: [C27] The apparatus of C26, wherein the notification includes a summary of the unauthenticated user. [C28] the apparatus is a first augmented reality device, and the one or more processors: outputting at least the portion of the virtual content to a second augmented reality device. The apparatus described in C17. [C29] The apparatus of C17, wherein the apparatus is an augmented reality device. [C30] The device of C17, further comprising a display. [C31] The device of C30, wherein the display is configured to display at least the portion of the virtual content. [C32] the apparatus is a first augmented reality device; the display is configured to display at least the portion of the virtual content; the one or more processors are configured to output at least the portion of the virtual content to a second augmented reality device; The device described in C30. [C33] the apparatus is a first augmented reality device; the display is configured to display at least the portion of the virtual content; the one or more processors are configured to output at least the portion of the virtual content to a second augmented reality device; the second augmented reality device is configured to receive the boundary information; the second augmented reality device is configured to generate the virtual boundary content from the received boundary information. The device described in C30. [C34] The apparatus is a server device, and the one or more processors: outputting, by the server device, at least the portion of the virtual content to an augmented reality device. The apparatus described in C17. [C35] A non-transitory computer-readable medium that, when executed by one or more processors, causes the one or more processors to: initiating a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information defining a boundary of the virtual private space; generating at least the portion of the virtual content for the virtual private space, wherein at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and not viewable by one or more unauthenticated users; A non-transitory computer-readable medium having stored thereon instructions to cause a

Claims

1. 1. A method for generating virtual content, comprising: initiating, by the device, a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information identifying a boundary of the virtual private space, the boundary information being used to generate boundary virtual content, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session; generating at least the portion of the virtual content for the virtual private space, wherein the boundary virtual content is configured to block the portion of the physical space such that the at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and is not viewable by the one or more unauthenticated users of the virtual session; A method for providing the above.

2. The method of claim 1 , wherein a real-world volume defined within the boundary virtual content is not viewable by the one or more unauthorized users based on the boundary virtual content associated with the boundary information.

3. 2. The method of claim 1, wherein the boundary information includes at least one of one or more feature points in at least one image of the physical space that can be used to generate the boundary virtual content that identifies the boundary for the virtual private space, and world coordinates in the physical space.

4. receiving an indication that an unauthorized user has entered the virtual private space; based on the indication that the unauthorized user has entered the virtual private space, rendering at least a portion of the unauthorized user inaccessible to the one or more authorized users in the virtual private space; The method of claim 1 further comprising:

5. receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; outputting a notification indicating the existence of the virtual private space based on the indication that the unauthorized user has entered or is within the threshold distance of entering the virtual private space; The method of claim 1 further comprising:

6. 6. The method of claim 5, wherein outputting the notification includes at least one of sending the notification to a device of the unauthorized user and outputting an audio notification indicating the presence of the virtual private space.

7. receiving an indication that an unauthorized user has entered or is within a threshold distance of entering said virtual private space; based on the indication that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space, outputting a notification to one or more devices of the one or more authenticated users indicating that the unauthenticated user has entered or is within the threshold distance of entering the virtual private space; 10. The method of claim 1, further comprising: wherein the notification includes a summary of the unauthorized user.

8. the device is an augmented reality device; displaying, by the augmented reality device, at least the portion of the virtual content. The method of claim 1.

9. the device is a first augmented reality device; outputting the at least the portion of the virtual content to a second augmented reality device. The method of claim 1.

10. the device is a first augmented reality device; displaying, by the first augmented reality device, at least the portion of the virtual content; outputting the at least the portion of the virtual content to a second augmented reality device; The method of claim 1 further comprising:

11. the device is a first augmented reality device; displaying, by the first augmented reality device, at least the portion of the virtual content; outputting the at least the portion of the virtual content to a second augmented reality device; receiving, by the second augmented reality device, the boundary information; generating, by the second augmented reality device, the boundary virtual content from the received boundary information; The method of claim 1 further comprising:

12. the device is a server device; outputting, by the server device, the at least the portion of the virtual content to an augmented reality device. The method of claim 1.

13. 1. An apparatus for generating virtual content, comprising: a memory configured to store virtual content; one or more processors coupled to the memory; wherein the processor: initiating a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information identifying a boundary of the virtual private space, the boundary information being used to generate boundary virtual content, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session; generating at least the portion of the virtual content for the virtual private space, wherein the boundary virtual content is configured to block the portion of the physical space such that the at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and is not viewable by the one or more unauthenticated users of the virtual session; An apparatus configured to:

14. 14. Apparatus according to claim 13, further configured to perform the steps of the method according to any one of claims 2 to 12.

15. A non-transitory computer-readable medium that, when executed by one or more processors, causes the one or more processors to: initiating a virtual session for presenting virtual content; identifying a portion of a physical space for the virtual session to use as a virtual private space for presenting at least a portion of the virtual content; outputting boundary information identifying a boundary of the virtual private space, the boundary information being used to generate boundary virtual content, the boundary virtual content being viewable by one or more unauthenticated users of the virtual session; generating at least the portion of the virtual content for the virtual private space, wherein the boundary virtual content is configured to block the portion of the physical space such that the at least the portion of the virtual content is viewable in the virtual private space by one or more authenticated users of the virtual session and is not viewable by the one or more unauthenticated users of the virtual session; A non-transitory computer-readable medium having stored thereon instructions to cause a

Images