Virtual private space for extended reality

Abstract

Systems, methods, and non-transitory media for generating a virtual private space for an extended reality (XR) experience are provided. An example method can include initiating a virtual session for presenting virtual content and identifying, for the virtual session, a portion of a physical space to use as a virtual private space for presenting at least a portion of the virtual content. The method can include outputting boundary information defining a boundary of the virtual private space and generating, for the virtual private space, at least the portion of the virtual content. At least the portion of the virtual content can be viewable by one or more authorized users of the virtual session in the virtual private space and not viewable by one or more unauthorized users.

Description

Technical Field

[0001] In summary, this disclosure relates to generating virtual content in virtual private spaces for extended reality and other applications. Background Technology

[0002] Extended reality (ER) technologies can be used to present virtual content to users and / or combine real-world environments with virtual environments to provide users with extended reality experiences. The term extended reality can encompass virtual reality, augmented reality, mixed reality, and so on. Each of these forms of extended reality allows users to experience or interact with immersive virtual environments or content. For example, an extended reality experience can allow users to interact with a real or physical environment enhanced or augmented using virtual content.

[0003] It can enable extended reality technology to enhance the user experience in a wide variety of scenarios, such as entertainment, healthcare, retail, education, social media, etc. Summary of the Invention

[0004] Systems, apparatuses, methods, and computer-readable media for generating virtual content in virtual private spaces to extend real-world experiences are disclosed. According to at least one example, a method for generating virtual content for one or more virtual private spaces is provided. The method includes: initiating a virtual session by a device for presenting virtual content; identifying, for the virtual session, a portion of a physical space available for presenting at least a portion of the virtual content as a virtual private space; outputting boundary information defining the boundaries of the virtual private space; and generating at least the portion of the virtual content for the virtual private space, the at least the portion of the virtual content being viewable by one or more authorized users of the virtual session in the virtual private space and not viewable by one or more unauthorized users.

[0005] In another example, an apparatus is provided for generating virtual content for one or more virtual private spaces, comprising: a memory configured to store the virtual content; and one or more processors (e.g., implemented in a circuit) coupled to the memory. The processors are configured to: initiate a virtual session for presenting the virtual content; identify, for the virtual session, a portion of a physical space available for use as at least a portion of the virtual content for presenting the virtual content; output boundary information defining the boundaries of the virtual private space; and generate, for the virtual private space, at least the portion of the virtual content, which can be viewed by one or more authorized users of the virtual session within the virtual private space and cannot be viewed by one or more unauthorized users.

[0006] In another example, a non-transitory computer-readable medium is provided having instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to: initiate a virtual session for presenting virtual content; identify, for the virtual session, a portion of a virtual private space of physical space for use as at least a portion of the virtual content; output boundary information defining the boundaries of the virtual private space; and generate, for the virtual private space, at least the portion of the virtual content, which can be viewed in the virtual private space by one or more authorized users of the virtual session and cannot be viewed by one or more unauthorized users.

[0007] In another example, an apparatus is provided for generating virtual content for one or more virtual private spaces. The apparatus includes: a unit for initiating a virtual session for presenting virtual content; a unit for identifying, for the virtual session, a portion of a physical space designated as a virtual private space for presenting at least a portion of the virtual content; a unit for outputting boundary information defining the boundaries of the virtual private space; and a unit for generating at least the portion of the virtual content for the virtual private space, wherein the at least the portion of the virtual content can be viewed in the virtual private space by one or more authorized users of the virtual session and cannot be viewed by one or more unauthorized users.

[0008] In another example, a method for generating virtual content is provided. The method includes: one or more processors identifying a portion of a physical space for use as a virtual private space for presenting virtual content for a virtual session; the one or more processors generating boundary information defining the boundaries of the virtual private space, the boundary information being associated with boundary virtual content identifying the boundaries of the virtual private space; the one or more processors generating the virtual content for the virtual private space; displaying the virtual content of the virtual session by a first extended reality viewing device, the first extended reality viewing device being configured to receive and display the virtual content; and displaying the virtual content by a second extended reality viewing device, the second extended reality viewing device being configured to receive the boundary information and display the boundary virtual content. In some cases, the second extended reality viewing device does not display the virtual content in the virtual private space.

[0009] In another aspect, an extended reality system is provided. The extended reality system includes: one or more processors configured to: identify, for a virtual session, a portion of a physical space available for use as a virtual private space for presenting virtual content; generate boundary information defining the boundaries of the virtual private space, the boundary information being associated with boundary virtual content identifying the boundaries of the virtual private space; and generate the virtual content for the virtual private space. The extended reality system also includes a first extended reality viewing device authorized to display and / or view the virtual content of the virtual session, the first extended reality viewing device being arranged to receive and display the virtual content. The extended reality system further includes a second extended reality viewing device not authorized to display and / or view the virtual content, the second extended reality viewing device being arranged to receive the boundary information and display the boundary virtual content, and not display the virtual content in the virtual private space. In some examples, the one or more processors are part of the first extended reality viewing device.

[0010] In another example, a non-transitory computer-readable medium for an extended reality system is provided, having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: identify, for a virtual session, a portion of a physical space available for use as a virtual private space for presenting virtual content; generate boundary information defining the boundaries of the virtual private space, the boundary information being associated with boundary virtual content identifying the boundaries of the virtual private space; and generate the virtual content for the virtual private space. A non-transitory computer-readable medium for a first extended reality viewing device is provided, having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: display the virtual content of the virtual session, the first extended reality viewing device being configured to receive and display the virtual content. A non-transitory computer-readable medium for a second extended reality viewing device is provided, having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: display the virtual content, the second extended reality viewing device being configured to receive the boundary information and display the boundary virtual content. In some cases, the second extended reality viewing device does not display the virtual content in the virtual private space.

[0011] In another example, an extended reality system is provided. The extended reality system includes: units for identifying a portion of a physical space for use as a virtual private space for presenting virtual content, via one or more processors for a virtual session; units for generating boundary information defining the boundaries of the virtual private space, the boundary information being associated with boundary virtual content identifying the boundaries of the virtual private space; units for generating the virtual content for the virtual private space; a first unit for displaying the virtual content of the virtual session, the first unit for displaying being arranged to receive and display the virtual content; and a second unit for displaying the virtual content, the second unit for displaying being arranged to receive the boundary information and display the boundary virtual content. In some cases, the second unit for displaying does not display the virtual content in the virtual private space.

[0012] In some aspects, based on boundary virtual content associated with the boundary information, at least a portion of the virtual content cannot be viewed by the one or more unauthorized users.

[0013] In some aspects, based on the boundary virtual content associated with the boundary information, the real-world volume defined within the boundary virtual content cannot be viewed by the one or more unauthorized users.

[0014] In some aspects, one or more of the above-described methods, apparatuses, and computer-readable media further include: outputting boundary virtual content identifying the boundaries of the virtual private space, the boundary virtual content being viewable by the one or more unauthorized users of the virtual session.

[0015] In some aspects, the boundary information includes at least one of the following: world coordinates in the physical space, and one or more feature points in at least one image of the physical region, the one or more feature points being used to generate boundary virtual content identifying the boundary of the virtual private space.

[0016] In some aspects, one or more of the above-described methods, apparatuses, and computer-readable media further include: receiving an indication that an unauthorized user has entered the virtual private space; and based on the indication that the unauthorized user has entered the virtual private space, obscuring at least a portion of the unauthorized user to prevent it from being viewed by the one or more authorized users in the virtual private space.

[0017] In some aspects, one or more of the above-described methods, apparatuses, and computer-readable media further include: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0018] In some aspects, one or more of the above-described methods, apparatuses, and computer-readable media further include: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, outputting a notification indicating the existence of the virtual private space.

[0019] In some aspects, outputting the notification includes at least one of the following: sending the notification to the unauthorized user's device, and outputting an audio notification indicating the existence of the virtual private space. In some cases, the notification includes an outline of the unauthorized user.

[0020] In some aspects, one or more of the above-described methods, apparatus, and computer-readable media further include: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, outputting a notification to one or more devices of the one or more authorized users, the notification indicating that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space. In some cases, the notification includes an outline of the unauthorized user.

[0021] In some aspects, one or more of the devices or apparatuses are extended reality devices, and one or more of the methods, apparatuses, and computer-readable media described above further include: displaying at least a portion of the virtual content by the extended reality device.

[0022] In some aspects, one or more of the devices or apparatuses are first extended reality devices, and one or more of the methods, apparatuses, and computer-readable media described above further include: outputting at least a portion of the virtual content to a second extended reality device.

[0023] In some aspects, one or more of the devices or apparatuses are first extended reality devices, and one or more of the methods, apparatuses, and computer-readable media described above further include: displaying at least a portion of the virtual content by the first extended reality device; and outputting at least a portion of the virtual content to a second extended reality device.

[0024] In some aspects, one or more of the devices or apparatuses are first extended reality devices, and one or more of the methods, apparatuses, and computer-readable media described above further include: displaying at least a portion of the virtual content by the first extended reality device; outputting at least a portion of the virtual content to a second extended reality device; receiving the boundary information by the second extended reality device; and generating the virtual boundary content by the second extended reality device based on the received boundary information.

[0025] In some aspects, the device is a server device, and one or more of the above-described methods, apparatuses, and computer-readable media further include: outputting at least a portion of the virtual content from the server device to an extended reality device.

[0026] In some aspects, the device includes a camera, a mobile device (e.g., a mobile phone or so-called "smartphone" or other mobile device), a wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a personal computer, a laptop computer, a server computing device, or other devices. In some aspects, the device includes one or more cameras for capturing one or more images. In some aspects, the device also includes a display for displaying one or more images, notifications, and / or other displayable data. In some aspects, the device described above may include one or more sensors.

[0027] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to define the scope of the claimed subject matter. The subject matter should be understood by referring to appropriate portions of the entire specification, any or all of the drawings, and each claim.

[0028] The foregoing, as well as other features and embodiments, will become more apparent upon reference to the following description, claims, and drawings. Attached Figure Description

[0029] The illustrative embodiments of this application are described in detail below with reference to the accompanying drawings:

[0030] Figure 1 This is a block diagram illustrating an example extended reality system according to some examples of the present disclosure;

[0031] Figure 2 This is a schematic diagram illustrating example landmark points of a hand according to some examples of the present disclosure. These example landmark points can be used to track the position of the hand and the interaction between the hand and the virtual environment.

[0032] Figure 3 This is a schematic diagram illustrating examples of extended reality systems worn by users, according to some examples of this disclosure;

[0033] Figure 4 This is a schematic diagram illustrating some examples of augmented reality virtual sessions in which virtual content is displayed in a virtual private space, according to the present disclosure;

[0034] Figure 5 This is to illustrate some examples in accordance with this disclosure. Figure 4 A schematic diagram of another view of the virtual private space shown;

[0035] Figure 6 This is an example illustrating some aspects of this disclosure. Figure 4 An example illustration of a side view of a table and the virtual content of its boundaries;

[0036] Figure 7A , Figure 7B and Figure 7C This is a schematic diagram illustrating examples of a person physically entering a virtual private space, according to some examples of this disclosure;

[0037] Figure 8 This is a flowchart illustrating examples of a process for generating virtual content for one or more virtual private spaces, based on some examples of this disclosure; and

[0038] Figure 9 An example computing system is shown, which is based on some examples of this disclosure. Detailed Implementation

[0039] Certain aspects and embodiments of this disclosure are provided below. As will be apparent to those skilled in the art, some of these aspects and embodiments can be applied independently, and some can be applied in combination. In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of embodiments of this application. However, it will be apparent that various embodiments may be practiced without these specific details. The drawings and description are not intended to be limiting.

[0040] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the subsequent description of these exemplary embodiments will provide those skilled in the art with a feasible description for implementing the exemplary embodiments. It should be understood that various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of this application as set forth in the appended claims.

[0041] Extended reality (XR) systems facilitate interaction with different types of XR environments, including virtual reality (VR), augmented reality (AR), mixed reality (MR), and / or other XR environments. Users can interact with XR environments using XR devices. Examples of XR devices include head-mounted displays (HMDs), smart glasses, etc. For example, AR systems can overlay virtual content onto images of a real-world environment, which can be viewed by the user through AR devices (e.g., HMDs, AR glasses, or other AR devices). The real-world environment can include physical objects, people, or other real-world objects. XR devices can track parts of the user (e.g., the user's hands and / or fingertips) to allow the user to interact with virtual content items.

[0042] Virtual content present in an XR environment can be used to complement real-world objects. For example, in one or more images that include the AR environment and are displayed during an AR session, a virtual coffee cup can be virtually anchored to a real-world table (e.g., placed on it). People can also directly interact with virtual content and / or other real-world objects within the environment. For instance, a person in a space occupied by one or more virtual content items can interact with one or more virtual content items and / or one or more real-world objects in various ways. In one example, a person can interrupt an AR session by moving and / or blocking (or occluding) the virtual content being displayed. In another example, a person can move the real-world object that virtually supports the virtual content item (e.g., by moving the physical table holding the virtual coffee cup).

[0043] In some cases, a person may not be aware that virtual content items are being rendered in a given environment during an AR session. For example, a person may not be using (e.g., wearing) an AR device that allows that person to see the virtual content of the AR session. In another example, a person may have an AR device but may not be authorized to view the virtual content of the AR session. If a person in the space is not using an AR device or is not authorized to view certain virtual content, that person will not be able to view the virtual content and may unintentionally interfere with the virtual content in a way that adversely affects the user experience for a user who is wearing an AR device and is authorized to view and interact with the virtual content of the AR session.

[0044] In some cases, such interference can be problematic. For example, unexpected changes in the virtual environment can be frustrating for users participating in virtual sessions (e.g., AR sessions or other XR sessions). As mentioned above, AR content can be anchored to one or more observable real-world objects that can be manipulated by users not using AR devices. In an illustrative example, a multiplayer AR game session could be initiated on a restaurant table containing plates, cups, cutlery, and other items. Users observing the AR content of the AR session may experience frustration when the real-world objects used for anchoring are moved and / or otherwise obstruct immersion (e.g., if a waiter refills a cup in a way that obscures / interrupts the virtual content and / or affects the anchoring).

[0045] It may also be desirable to maintain the privacy of virtual sessions, preventing others from viewing the virtual content displayed to one or more users participating in the virtual session. However, users who are not authorized to view the virtual content of a virtual session may not be aware that a virtual session is taking place or the scope of the session, even if they are also using AR devices and may unintentionally view the virtual content.

[0046] This disclosure describes systems, apparatus, methods, and computer-readable media for generating virtual private spaces. The techniques described herein provide the ability for user-defined virtual private spaces for virtual sessions (e.g., AR sessions) using XR devices or systems (e.g., HMDs, AR glasses, etc.), wherein virtual content presented in the virtual private space can be seen and interacted with only by an "authorized" user authorized to view (using an XR device, such as AR glasses) the virtual content of the virtual private space. Unauthorized users cannot view the virtual content displayed in the virtual private space. As used herein, an unauthorized user is a user of a device that is not authorized to view the virtual content of a virtual session presented in the virtual private space. As described herein, any suitable authorization technique can be used for user authorization and unauthorization. The virtual private space can be a large space (e.g., an architectural area) or a small space (e.g., a game board displayed on a real-world table). A larger space can be larger than the area sensed by a single user device. In some cases, it may be necessary to redraw a map corresponding to the large space based on input from a communication network.

[0047] Virtual private spaces create "safe zones" where authorized AR users can engage in activities (e.g., virtual games, virtual sports events, virtual demonstrations, etc.) without being unintentionally disturbed by other people or objects in the physical environment where the virtual activity is taking place. For illustrative purposes, this document uses AR sessions and AR devices to describe the examples. Those skilled in the art will understand that these techniques can be applied to other XR environments, such as VR and / or MR environments.

[0048] Data defining virtual content to be included in a virtual private space can be provided to different AR devices of an authorized user, enabling the AR devices to display the virtual content to be included in that virtual private space. In some examples, a virtual private space can be implemented by generating and displaying boundary virtual content that demarcates the virtual private space and / or blocks the view of unauthorized users who may be in the same real-world environment as the authorized user of the virtual private space. For example, AR devices and / or servers can provide boundary information (e.g., world coordinates, feature points in an image, and / or other boundary information) to one or more AR devices of an unauthorized user and, in some cases, to one or more other AR devices of an authorized user in a virtual session. The boundary information can be used by one or more AR devices to generate and display boundary virtual content that identifies the boundaries of the virtual private space. In another example, AR devices and / or servers can generate boundary virtual content that identifies the boundaries of the virtual private space and can provide this boundary virtual content to one or more AR devices of an unauthorized user and, in some cases, to one or more other AR devices of an authorized user in a virtual session. In some cases, unauthorized users and / or one or more AR devices belonging to authorized users can determine boundary information (e.g., world coordinates, feature points in an image, and / or other information) and use this information to generate boundary virtual content. In some cases, one or more authorized users' AR devices and / or a server can share a real-world map with unauthorized users' AR devices, allowing the unauthorized users' AR devices to display the boundaries or other visual indicators of the virtual private space. In some cases, virtual content defining the virtual private space can be generated based on the extent of virtual content in a virtual session, based on user input, and / or inferred from the application state provided by the user's AR device.

[0049] Boundary virtual content allows unauthorized users (unauthorized users) to observe the existence of a virtual private space. For example, an unauthorized user's AR device could display a virtual barrier depicting the virtual space. The display of boundary virtual content prevents unauthorized users from viewing the virtual content within the virtual private space while simultaneously limiting accidental interruptions to the virtual session. In an illustrative example, by observing (e.g., using an AR device) the boundary virtual information of a virtual private space, restaurant staff can avoid interfering with real-world objects contained within the virtual private space.

[0050] This article provides further details about the generation of virtual private spaces in relation to the various diagrams. Figure 1This is a diagram illustrating an example extended reality system 100 according to some aspects of this disclosure. The extended reality system 100 can run (or execute) XR applications and implement XR operations. In some examples, as part of an XR experience, the extended reality system 100 can perform tracking and localization, mapping of the physical world (e.g., a scene), and localization and rendering of virtual content on a display 109 (e.g., a screen, a visible plane / area, and / or other display). For example, the extended reality system 100 can generate a map (e.g., a three-dimensional (3D) map) of a scene in the physical world, track the pose (e.g., location and position) of the extended reality system 100 relative to the scene (e.g., relative to a 3D map of the scene), localize and / or anchor virtual content at a specific location on the map of the scene, and render the virtual content on the display 109 such that the virtual content appears to be located in the scene at a location corresponding to the specific location on the map of the scene where the virtual content is localized and / or anchored. The display 109 may include glasses, a screen, a lens, a projector, and / or other display mechanisms that allow users to see a real-world environment and also allow XR content to be displayed thereon.

[0051] In this illustrative example, the extended reality system 100 includes one or more image sensors 102, accelerometers 104, gyroscopes 106, storage devices 107, computing components 110, an XR engine 120, a virtual private space management engine 122, an image processing engine 124, and a rendering engine 126. It should be noted that... Figure 1 The components 102-126 shown are non-limiting examples provided for illustrative and explanatory purposes, and other examples may include those with similar characteristics. Figure 1 The components shown may be more, fewer, or different than those shown. For example, in some cases, the extended reality system 100 may include one or more other sensors (e.g., one or more inertial measurement units (IMUs), radar, light detection and ranging (LIDAR) sensors, audio sensors, etc.), one or more display devices, one or more other processing engines, one or more other hardware components, and / or Figure 1 One or more other software and / or hardware components not shown. See below for reference. Figure 9 The example architecture and example hardware components that can be implemented by the extended reality system 100 are further described.

[0052] Furthermore, for simplicity and explanation purposes, one or more image sensors 102 will be referred to herein as image sensor 102 (e.g., in the singular). However, those skilled in the art will recognize that the extended reality system 100 may include a single image sensor or multiple image sensors. Furthermore, references to any component of the extended reality system 100 (e.g., 102-126) in the singular or plural form should not be construed as limiting the number of such components implemented by the extended reality system 100 to one or more. For example, references to accelerometer 104 in the singular form should not be construed as limiting the number of accelerometers implemented by the extended reality system 100 to one. Those skilled in the art will recognize that for... Figure 1 Any of the components 102-126 shown, the extended reality system 100 may include only one such component or more than one such component.

[0053] The extended reality system 100 includes an input device 108 or communicates with an input device 108 (wired or wirelessly). The input device 108 may include any suitable input device, such as a touchscreen, pen or other pointing device, keyboard, mouse, buttons or keys, microphone for receiving voice commands, gesture input device for receiving gesture commands, any combination thereof, and / or other input devices. In some cases, the image sensor 102 may capture images that can be processed to interpret gesture commands.

[0054] The extended reality system 100 may be a single computing device or part of or implemented therewith. In some examples, the extended reality system 100 may be part of electronic devices (or multiple electronic devices), such as an extended reality head-mounted display (HMD) device, extended reality glasses (e.g., augmented reality or AR glasses), camera systems (e.g., digital cameras, IP cameras, video cameras, security cameras, etc.), telephone systems (e.g., smartphones, cellular phones, conferencing systems, etc.), desktop computers, laptops or notebook computers, tablet computers, set-top boxes, smart TVs, display devices, game consoles, video streaming devices, IoT (Internet of Things) devices, and / or any other suitable electronic devices.

[0055] In some implementations, one or more image sensors 102, accelerometers 104, gyroscopes 106, storage devices 107, computing components 110, XR engines 120, virtual private space management engines 122, image processing engines 124, and rendering engines 126 may be part of the same computing device. For example, in some cases, one or more image sensors 102, accelerometers 104, gyroscopes 106, storage devices 107, computing components 110, XR engines 120, virtual private space management engines 122, image processing engines 124, and rendering engines 126 may be integrated into HMDs, extended reality glasses, smartphones, laptops, tablets, gaming systems, and / or any other computing device. However, in some implementations, one or more image sensors 102, accelerometers 104, gyroscopes 106, storage devices 107, computing components 110, XR engines 120, virtual private space management engines 122, image processing engines 124, and rendering engines 126 may be part of two or more separate computing devices. For example, in some cases, some of the components 102-126 may be part of or implemented by a computing device, while the remaining components may be part of or implemented by one or more other computing devices.

[0056] Storage device 107 can be any storage device used for storing data. Furthermore, storage device 107 can store data from any component of the extended reality system 100. For example, storage device 107 can store data from image sensor 102 (e.g., image or video data), data from accelerometer 104 (e.g., measurement results), data from gyroscope 106 (e.g., measurement results), data from computing component 110 (e.g., processing parameters, preferences, virtual content, rendered content, scene maps, tracking and positioning data, object detection data, privacy data, XR application data, facial recognition data, occlusion data, etc.), data from XR engine 120, data from virtual private space management engine 122, data from image processing engine 124, and / or data from rendering engine 126 (e.g., output frames). In some examples, storage device 107 may include a buffer for storing frames processed by computing component 110.

[0057] One or more computing components 110 may include a central processing unit (CPU) 112, a graphics processing unit (GPU) 114, a digital signal processor (DSP) 116, and / or an image signal processor (ISP) 118. Computing components 110 can perform various operations, such as image enhancement, computer vision, graphics rendering, extended reality (e.g., tracking, localization, pose estimation, mapping, content anchoring, content rendering, etc.), image / video processing, sensor processing, recognition (e.g., text recognition, face recognition, object recognition, feature recognition, tracking or pattern recognition, scene recognition, occlusion detection, etc.), machine learning, filtering, and any of the various operations described herein. In this example, computing component 110 implements an XR engine 120, a virtual private space management engine 122, an image processing engine 124, and a rendering engine 126. In other examples, computing component 110 may also implement one or more other processing engines.

[0058] Image sensor 102 may include any image and / or video sensor or capture device. In some examples, image sensor 102 may be part of a multi-camera assembly, such as a dual-camera assembly. Image sensor 102 may capture image and / or video content (e.g., raw image and / or video data), which may then be processed by computing component 110, XR engine 120, virtual private space management engine 122, image processing engine 124, and / or rendering engine 126, as described herein.

[0059] In some examples, image sensor 102 can capture image data and generate frames based on the image data, and / or provide the image data or frames to XR engine 120, virtual private space management engine 122, image processing engine 124, and / or rendering engine 126 for processing. Frames can include video frames in a video sequence or still images. Frames can include pixel arrays representing a scene. For example, a frame can be: a red-green-blue (RGB) frame with red, green, and blue components per pixel; a luminance, chrominance, chrominance (YCbCr) frame with a luminance component and two chrominance (color) components (chrominance red and chrominance blue) per pixel; or any other suitable type of color or monochrome image.

[0060] In some cases, image sensor 102 (and / or other cameras of extended reality system 100) can be configured to also capture depth information. For example, in some implementations, image sensor 102 (and / or other cameras) may include an RGB depth (RGB-D) camera. In some cases, extended reality system 100 may include one or more depth sensors (not shown) that are separate from image sensor 102 (and / or other cameras) and can capture depth information. For example, such depth sensors may acquire depth information independently of image sensor 102. In some examples, depth sensors may be physically mounted in the same general location as image sensor 102, but may operate at a different frequency or frame rate than image sensor 102. In some examples, depth sensors may take the form of a light source that can project a structured or textured light pattern (which may include one or more narrowband lights) onto one or more objects in a scene. Depth information can then be obtained by utilizing the geometric deformation of the projected pattern caused by the surface shape of the objects. In one example, depth information may be obtained from a stereo sensor, such as a combination of an infrared structured light projector and an infrared camera registered to a camera (e.g., an RGB camera).

[0061] The extended reality system 100 also includes one or more sensors. These sensors may include one or more accelerometers (e.g., accelerometer 104), one or more gyroscopes (e.g., gyroscope 106), and / or other sensors. The one or more sensors may provide velocity, orientation, and / or other position-related information to the computing component 110. For example, accelerometer 104 may detect the acceleration of the extended reality system 100 and may generate an acceleration measurement based on the detected acceleration. In some cases, accelerometer 104 may provide one or more translation vectors (e.g., up / down, left / right, forward / backward) that can be used to determine the position or pose of the extended reality system 100. Gyroscope 106 may detect and measure the orientation and angular velocity of the extended reality system 100. For example, gyroscope 106 may be used to measure the pitch, roll, and yaw of the extended reality system 100. In some cases, gyroscope 106 may provide one or more rotation vectors (e.g., pitch, yaw, roll). In some examples, the image sensor 102 and / or the XR engine 120 may use measurements obtained by the accelerometer 104 (e.g., one or more translation vectors) and / or measurements obtained by the gyroscope 106 (e.g., one or more rotation vectors) to calculate the pose of the extended reality system 100. As previously mentioned, in other examples, the extended reality system 100 may also include other sensors such as an inertial measurement unit (IMU), a magnetometer, a gaze and / or eye-tracking sensor, a machine vision sensor, a smart scene sensor, a voice recognition sensor, a collision sensor, an impact sensor, a position sensor, a tilt sensor, etc.

[0062] In some cases, one or more sensors may include at least one IMU. An IMU is an electronic device that uses a combination of one or more accelerometers, one or more gyroscopes, and / or one or more magnetometers to measure specific forces, angular velocities, and / or orientations of the extended reality system 100. In some examples, one or more sensors may output measured information associated with the capture of images by image sensor 102 (and / or other cameras of the extended reality system 100) and / or depth information obtained using one or more depth sensors of the extended reality system 100.

[0063] The extended reality engine 120 can use the output of one or more sensors (e.g., accelerometer 104, gyroscope 106, one or more IMUs and / or other sensors) to determine the pose of the extended reality system 100 (also referred to as head pose) and / or the pose of the image sensor 102 (or other cameras of the extended reality system 100). In some cases, the pose of the extended reality system 100 and the pose of the image sensor 102 (or other cameras) can be the same. The pose of the image sensor 102 refers to the position and orientation of the image sensor 102 relative to a reference frame (e.g., relative to object 202). In some implementations, the camera pose can be determined for 6 degrees of freedom (6DOF), which refers to three translational components (e.g., which can be given by X (horizontal), Y (vertical), and Z (depth) coordinates relative to a reference frame (such as an image plane)) and three angular components (e.g., roll, pitch, and yaw relative to the same reference frame).

[0064] In some cases, a device tracker (not shown) can use measurements from one or more sensors and image data from image sensor 102 to track the pose (e.g., 6DOF pose) of the extended reality system 100. For example, the device tracker can fuse visual data from the image data (e.g., using a visual tracking solution) with inertial data from the measurements to determine the position and motion of the extended reality system 100 relative to the physical world (e.g., a scene) and a map of the physical world. As described below, in some examples, when tracking the pose of the extended reality system 100, the device tracker can generate a three-dimensional (3D) map of the scene (e.g., the real world) and / or generate updates to the 3D map of the scene. 3D map updates may include (e.g., but not limited to) new or updated features and / or feature points or markers associated with the scene and / or the 3D map of the scene, localization updates identifying or updating the position of the extended reality system 100 within the scene and the 3D map of the scene, etc. The 3D map can provide a digital representation of the scene in the real / physical world. In some examples, 3D maps can anchor location-based objects and / or content to real-world coordinates and / or objects. Extended reality system 100 can use map-rendered scenes (e.g., scenes in the physical world represented by and / or associated with 3D maps) to merge the physical and virtual worlds and / or merge virtual content or objects with the physical environment.

[0065] In some aspects, computing component 110 may use a visual tracking solution to determine and / or track the pose of image sensor 102 and / or the extended reality system 100 as a whole, based on images captured by image sensor 102 (and / or other cameras of extended reality system 100). For example, in some examples, computing component 110 may perform tracking using computer vision-based tracking, model-based tracking, and / or simultaneous localization and mapping (SLAM) techniques. For example, computing component 110 may perform SLAM or may communicate (wired or wirelessly) with a SLAM engine (not shown). SLAM refers to a class of techniques that create a map of an environment (e.g., a map of the environment modeled by extended reality system 100) while simultaneously tracking the pose of the camera (e.g., image sensor 102) and / or extended reality system 100 relative to that map. This map may be called a SLAM map and may be three-dimensional (3D). SLAM technology can be performed using color or grayscale image data captured by image sensor 102 (and / or other cameras of extended reality system 100) and can be used to generate estimates of 6DOF pose measurements of image sensor 102 and / or extended reality system 100. Such SLAM technology configured to perform 6DOF tracking can be referred to as 6DOF SLAM. In some cases, the output of one or more sensors (e.g., accelerometer 104, gyroscope 106, one or more IMUs and / or other sensors) can be used to estimate, correct, and / or otherwise adjust the estimated pose.

[0066] In some cases, 6DOF SLAM (e.g., 6DOF tracking) can associate features observed from certain input images from image sensor 102 (and / or other cameras) with a SLAM map. For example, 6DOF SLAM can use feature point association from the input images to determine the pose (position and orientation) of image sensor 102 and / or extended reality system 100 relative to the input images. 6DOF map rendering can also be performed to update the SLAM map. In some cases, the SLAM map maintained using 6DOF SLAM can contain 3D feature points triangulated from two or more images. For example, keyframes can be selected from the input images or video stream to represent the observed scene. For each keyframe, the corresponding 6DOF camera pose associated with the image can be determined. The pose of image sensor 102 and / or extended reality system 100 can be determined by projecting features from the 3D SLAM map onto the image or video frame and updating the camera pose according to a verified 2D-3D correspondence.

[0067] In an illustrative example, computational component 110 can extract feature points from each input image or from each keyframe. Feature points (also referred to as registration points) as used herein are unique or identifiable parts of an image, such as a part of a hand, the edge of a table, etc. Features extracted from a captured image can represent different feature points along three-dimensional space (e.g., coordinates on the X, Y, and Z axes), and each feature point can have an associated feature location. Feature points in a keyframe may match (be identical to or correspond to) feature points in a previously captured input image or keyframe, or may not match. Feature detection can be used to detect feature points. Feature detection can include image processing operations that examine one or more pixels of an image to determine if a feature exists at a particular pixel. Feature detection can be used to process the entire captured image or parts of an image. For each image or keyframe, once a feature has been detected, a local image patch around that feature can be extracted. Any suitable technique can be used to extract features, such as Scale Invariant Feature Transform (SIFT) (which locates features and generates their descriptions), Speed-Up Robust Feature Transform (SURF), Gradient Location Orientation Histogram (GLOH), Normalized Cross-Correlation (NCC), or other suitable techniques.

[0068] In some cases, the extended reality system 100 can also track a user's hands and / or fingers to allow the user to interact with and / or control virtual content in the virtual environment (e.g., virtual content displayed in a virtual private space). For example, the extended reality system 100 can track the pose and / or movement of a user's hands and / or fingertips to identify or translate the user's interaction with the virtual environment. User interaction may include, but is not limited to, moving virtual content items, resizing the location of virtual content items and / or virtual private spaces, selecting input interface elements in the virtual user interface (e.g., a virtual representation of a mobile phone, a virtual keyboard, and / or other virtual interfaces), providing input through the virtual user interface, etc.

[0069] Figure 2 This is a diagram showing example markers for hand 200, which can be used to track the position of hand 200 and its interaction with a virtual environment, such as virtual content displayed within a virtual private space as described herein. Figure 2The markers shown correspond to different parts of hand 200, including marker 235 on the palm of hand 200, marker 230 on the thumb of hand 200, marker 232 on the index finger of hand 200, marker 234 on the middle finger of hand 200, marker 236 on the ring finger of hand 200, and marker 238 on the little finger of hand 200. The palm of hand 200 can move in three translational directions (e.g., measured in the X, Y, and Z directions relative to a plane (such as an image plane)) and three rotational directions (e.g., measured in terms of yaw, pitch, and roll relative to that plane), and thus provides six degrees of freedom (6DOF) that can be used for registration and / or tracking. The 6DOF movement of the palm in... Figure 2 The center is shown as a square, as indicated in Figure 240.

[0070] Different joints of the fingers in hand 200 allow for varying degrees of movement, as shown in Figure 240. Figure 2 As shown in the rhomboid shape (e.g., rhomboid 233), the base of each finger (corresponding to the metacarpophalangeal joint (MCP) between the proximal phalanges and metacarpals) has two degrees of freedom (2DOF), corresponding to flexion and extension, and abduction and adduction. Figure 2 As shown in the circular shape (e.g., circle 231), each upper joint of each finger (corresponding to the interphalangeal joints between the distal, intermediate, and proximal phalanges) has one degree of freedom (2DOF) corresponding to flexion and extension. As a result, hand 200 provides 26 degrees of freedom (26DOF) for tracking hand 200 and its interaction with virtual content rendered by extended reality system 100.

[0071] The extended reality system 100 can use one or more landmarks on the hand 200 to track the hand 200 (e.g., based on the pose and / or movement of the hand 200) and to track interactions with the virtual environment rendered by the extended reality system 100. As described above, as a result of the detection of one or more landmarks on the hand 200, the pose of the landmarks (and therefore the hand and fingers) in relative physical positions with respect to the extended reality system 100 can be established. For example, landmarks on the palm of the hand 200 (e.g., landmark 235) can be detected in an image, and the location of the landmarks can be determined relative to the image sensor 102 of the extended reality system 100. Points of virtual content items rendered by the extended reality system 100 (e.g., center points, such as centroids or other center points) can be translated to the display of the extended reality system 100 with respect to the location determined relative to the landmarks on the palm of the hand 200 (e.g., ...). Figure 1 The position on the monitor (109) (or rendered on the monitor).

[0072] As described below, the Extended Reality system 100 can also register virtual content and / or hand 200 to points in the real world (such as those detected in one or more images) and / or other parts of the user. For example, in some implementations, in addition to determining the physical pose of hand 200 relative to the Extended Reality system 100 (or the Extended Reality system 100) and / or the virtual content item, the Extended Reality system 100 can determine the location of other landmarks, such as unique points on walls (referred to as feature points), one or more corners of an object, features on the floor, points on a person's face, points on nearby devices, etc. In some cases, the Extended Reality system 100 can place virtual content within a specific location relative to feature points detected in the environment, which may correspond to, for example, objects and / or people detected in the environment.

[0073] In some examples, the pose of the extended reality system 100 (and / or the user's head) can be determined using, for example, image data from image sensor 102 and / or measurements from one or more sensors (such as accelerometer 104, gyroscope 106, and / or one or more other sensors (e.g., one or more magnetometers, one or more inertial measurement units (IMUs), etc.)). Head pose can be used to determine the position of virtual content, hands 200, and / or objects and / or people in the environment.

[0074] The operations of the XR engine 120, the virtual private space management engine 122, the image processing engine 124, and the rendering engine 126 (and any other image processing engine) can be implemented by any of the computing components 110. In an illustrative example, the operation of the rendering engine 126 can be implemented by the GPU 114, and the operation of the XR engine 120, the virtual private space management engine 122, and the image processing engine 124 can be implemented by the CPU 112, the DSP 116, and / or the ISP 118. In some cases, the computing components 110 may include other electronic circuitry or hardware, computer software, firmware, or any combination thereof to perform any of the various operations described herein.

[0075] In some examples, XR engine 120 can perform XR operations to generate XR experiences based on data from one or more sensors (such as one or more IMUs, radar, etc.) on image sensor 102, accelerometer 104, gyroscope 106, and / or extended reality system 100. In some examples, XR engine 120 can perform tracking, localization, pose estimation, mapping, content anchoring operations, and / or any other XR operations / functions. XR experiences may include using extended reality system 100 to present XR content (e.g., virtual reality content, augmented reality content, mixed reality content, etc.) to a user during a virtual session. In some examples, XR content and experiences may be provided by extended reality system 100 through XR applications (e.g., performed or implemented by XR engine 120) that provide specific XR experiences, such as, for example, XR gaming experiences, XR classroom experiences, XR shopping experiences, XR entertainment experiences, XR activities (e.g., operation, maintenance activities, etc.). During an XR experience, users can use the extended reality system 100 to view and / or interact with virtual content. In some cases, users can view and / or interact with virtual content while also viewing and / or interacting with the physical environment around them, thus allowing for an immersive experience between the physical environment and virtual content that is mixed with or integrated with it.

[0076] The virtual private space management engine 122 can perform various operations to determine (and manage) how, where, and / or when virtual private spaces are rendered during an XR experience. Virtual private spaces can be defined by the user of the extended reality system 100 or automatically (e.g., based on user preferences, AR session requirements, etc.) for a virtual session (e.g., an AR session). Virtual private spaces can be generated for displaying virtual content in any area of ​​a physical space. Virtual private spaces can be generated for large spaces (e.g., architectural areas, concert halls, sports venues, etc.) or for small spaces (e.g., a virtual teleconference with virtual content displayed on a real-world conference table, a game board displayed on a real-world table, etc.). Larger virtual spaces can be larger than the area sensed by a single user device. For example, crowdsourcing can be used to obtain features of large spaces, which can be used to generate large virtual spaces, as described below. In some cases, it may be necessary to redraw a 3D map (e.g., a SLAM map) corresponding to a large space based on input obtained through a communication network. For example, a local map can be refined based on crowdsourced information with the same geographic coordinates.

[0077] A virtual private space can be depicted by boundary information defining the boundaries of that virtual private space. Extended Reality System 100 (e.g., Virtual Private Space Management Engine 122 and / or other components) can generate the boundary information, or it can obtain it from a server and / or other Extended Reality systems. Boundary information is a data structure representing the boundaries of a virtual private space. The virtual content defining the boundaries of the virtual private space is called boundary virtual content. Boundary virtual content identifies the boundaries of the virtual private space. In some cases, boundary information may include one or more feature points in at least one image of the physical space (e.g., captured by image sensor 102) and / or may include world coordinates in that physical space. In some cases, Extended Reality System 100 may receive boundary virtual content from a server and / or from another XR system. In some cases, Extended Reality System 100 may receive feature points and / or world coordinates from a server and / or from another XR system. In an illustrative example, Extended Reality System 100 may receive boundary virtual content (and other virtual content for virtual sessions) and feature points and / or world coordinates from a server. In another illustrative example, the extended reality system 100 may receive boundary virtual content (and / or other virtual content for virtual sessions) from a server, and may receive feature points and / or world coordinates from another XR system. The extended reality system 100 may use the feature points and / or world coordinates to generate the boundary virtual content (e.g., determining where the boundary virtual content is displayed relative to an object in physical space). Further details regarding the use of feature points and / or world coordinates are described below.

[0078] In some examples, boundary virtual content can be generated based on a user's gaze and / or in response to a non-user's gaze toward virtual content of a virtual session displayed within a virtual private space. For instance, if an authorized user's XR system detects that an unauthorized user is looking toward the virtual private space, boundary virtual content can be generated to block the unauthorized user's view of the virtual content within the virtual private space.

[0079] Based on boundary virtual content, the virtual content of a virtual session displayed within a virtual private space can only be seen and interacted with by authorized users (using XR devices such as HMDs, AR glasses, etc.). Boundary virtual content prevents unauthorized users who might be in the same real-world environment as authorized users from viewing the virtual content in the virtual private space, thus preventing unauthorized users from viewing the virtual content. Boundary virtual content also provides unauthorized users located in the physical space where the virtual content is displayed with an indication of the existence of the virtual private space. For unauthorized users using XR devices or systems, the XR device can display boundary virtual content depicting the virtual space (e.g., as a virtual barrier).

[0080] For unauthorized users not using XR devices or systems, the location (e.g., boundaries) of the virtual private space can be notified. For example, when an unauthorized user does not have an XR device that allows them to view virtual content, a notification can be sent to the unauthorized user's device (e.g., a mobile device, a wearable device such as a smartwatch or other wearable device) to indicate the presence and / or location of the virtual private space. In one example, the extended reality system (of the authorized user) and / or the server implementing the virtual session can send notifications (e.g., text-based messages, audible messages, and / or other notifications) to the unauthorized user's mobile device, wearable device, and / or other device indicating where the virtual private space is located. In another example, in response to the extended reality system detecting that an unauthorized user without an XR device (e.g., a user not using an XR device) is approaching or within a specific distance (e.g., 10 feet, 5 feet, 2 feet, and / or other distances) to the virtual private space, the extended reality system can provide notifications (e.g., visual notifications, audible notifications, and / or other notifications). This notification can allow unauthorized users to know the location of the virtual private space, thereby preventing interference with AR sessions taking place within the virtual private space. In such cases, interoperability can be provided between devices and the network. For example, protocols can be defined to keep the virtual space private and to provide the ability for user devices to publicly share XR capabilities and / or access to and from content (e.g., for devices from different vendors / implementations).

[0081] Therefore, a virtual private space creates a region of physical space where authorized users can engage in activities privately (e.g., virtual games, virtual sports activities, virtual demonstrations, etc.) without being unintentionally disturbed by other people or objects in the physical environment where the virtual activity is taking place. Displaying virtual content at the boundary prevents unauthorized users from viewing virtual content within the virtual private space, while simultaneously limiting the accidental interruption of virtual sessions.

[0082] Any suitable authorization technology can be used to authorize users of XR devices or systems to participate in virtual sessions associated with virtual private spaces. In some examples, an electronic signature within the XR device can be used to authorize or deauthorize users of that XR device. In one illustrative example, the hardware of the AR device can be identified and verified (resulting in user authorization) or invalidated (resulting in user deauthorization). In another example, login credentials can be provided to authorized users for authorization. For example, multiple users can be authorized to participate in a virtual poker game by entering login credentials into the graphical interface of an application facilitating the game. Users without login credentials cannot join the virtual session. In some examples, other authorization technologies, such as gesture-based, eye-gazing, facial recognition, voice commands, and / or other authorization technologies, can be used to authorize or deauthorize users of XR devices.

[0083] As described above, boundary information may additionally or alternatively include world coordinates in physical space and / or one or more feature points in at least one image of physical space (e.g., captured by image sensor 102). In such a case, extended reality system 100 can use the boundary information (world coordinates and / or feature points) to generate boundary virtual content that identifies the boundaries of the virtual private space. An example of using feature points and map data (e.g., using Simultaneous Localization and Mapping (SLAM) techniques) has been described above. For example, world coordinates may indicate locations in a map (e.g., a 3D map such as a SLAM map) of the real-world environment on which virtual content will be displayed (and therefore the virtual private space will be located). An image of the surface (e.g., captured by image sensor 102) can be obtained, and feature points of the surface can be extracted. Feature points may include unique features of the surface, such as lines, corners, unique shapes, and other features.

[0084] Virtual content items can be registered or anchored to feature points detected in the scene (e.g., positioned relative to them). For example, the virtual private space management engine 122 can coordinate with the XR engine 120 and / or the rendering engine 126 to anchor virtual content and boundary virtual content of the virtual private space to feature points on the surface on which the virtual content will be displayed. In an illustrative example, a user may be looking at a table in a public space that the user wishes to use for a multi-user AR session (e.g., playing a card game). The user may provide input to the extended reality system 100 (e.g., using input device 108) indicating that the user wishes to have an AR session at the table and that the AR session will be associated with a virtual private space. The extended reality engine 120, the virtual private space management engine 122, the image processing engine 124, and / or the rendering engine 126 (implemented by computing component 110) may acquire an image of the table captured by the image sensor 102 and may identify features of the table from the image. The display 109 may display AR content associated with the AR session on the table by anchoring the AR content to certain feature points of the table. The virtual private space management engine 122 can (e.g., from a server) generate or receive boundary virtual content to indicate the boundaries of the virtual private space, and the boundary virtual content can be anchored to features on the table. For example, the extended reality engine 120 can register AR content and boundary virtual content to feature points on the table, such that the AR content and boundary virtual content are displayed relative to the table.

[0085] In another example, a user can be facing a restaurant across the street from where they are standing. In response to identifying the restaurant from one or more images and obtaining AR content associated with it, the Extended Reality engine 120 can generate an AR object that provides information related to the restaurant. Feature points can be detected from a portion of an image including the restaurant's signboard. The Extended Reality engine 120 can register the AR object to the feature points of the signboard, such that the AR object is displayed relative to the signboard (e.g., above the signboard, making it easily recognizable to the user as being associated with the restaurant).

[0086] In some implementations, an authorized user can prevent other authorized users from viewing certain content in a virtual private area. For example, a specific application running on a user's device can prevent other users' devices running that application from displaying specific content. In an illustrative example, multiple users can play a multiplayer poker game, and it's possible to prevent user devices from displaying other users' cards.

[0087] In some examples, the virtual privacy management engine 122 of the extended reality system 100 and / or other XR systems can define multiple virtual private spaces. In some cases, a new virtual private space cannot be displayed on top of an existing virtual private space, which prevents interference between virtual private spaces. For example, a virtual geofence can be determined by selecting objects in the space or drawing barriers around the private space. In some cases, each virtual private space can be defined relative to a common reference point in the space. For example, a virtual space is a volume of space containing content displayed within it. In an XR device, this volume of space can be defined relative to a tracking reference frame (e.g., a 3D coordinate system with an origin and axis directions). Multiple virtual private spaces can be defined relative to the same reference frame.

[0088] In some examples, a single authorized user in a virtual private space can invite one or more authorized users in other virtual private spaces to merge the different private spaces. For example, if two virtual sessions are taking place in separate virtual private spaces that are close to each other, the authorized users of the two virtual spaces can agree to combine the two virtual sessions into a single virtual private space.

[0089] In some cases, when a disturbance or change occurs in a virtual private space, alerts can be sent to one or more authorized users of the private space (e.g., users of Extended Reality System 100 and other users of the XR system authorized for the private space). For example, the movement, removal, addition, and / or alteration of real-world objects present in the virtual private space and / or by unauthorized users can trigger notifications of the event to authorized users' devices. In some examples, visual cues can be displayed on display 109 and / or audible cues can be output via speakers (not shown) of Extended Reality System 100 to indicate to users that a disturbance or change has occurred in the virtual private space. In an illustrative example, multiple users may be participating in a multi-user virtual meeting where virtual content associated with the virtual meeting is displayed on tables in a private room of a restaurant. In one example, restaurant staff may place drinks on the table in front of each user. Extended Reality System 100 can output notifications to users (including users of Extended Reality System 100) that items have been placed within the boundaries of the virtual private space. In another example, when a virtual item is newly displayed in the virtual private space, the extended reality system 100 may output a notification to a user (including the user of the extended reality system 100). In another example, using one or more images from image sensor 102 and / or measurements from one or more sensors (e.g., accelerometer 104, gyroscope 106, one or more IMUs and / or other sensors), the extended reality system 100 may determine when a person or object is within a certain distance (e.g., 10 feet, 5 feet, 2 feet, and / or other distances) that is interfering with the virtual private space. The extended reality system 100 may output a notification to the user of the extended reality system 100 informing them that a person may be interfering with the virtual private space. In some cases, the alert may be triggered and provided to an authorized user's device based on the detection of changes to the map of the virtual private space. In some cases, the notification may include details about the person or object causing the disturbance.

[0090] In some examples, virtual content in a virtual private space may be displayed with higher priority than real-world content. For example, the Extended Reality system 100 and / or a server communicating with it may not emphasize real-world objects and / or portions of unauthorized users in the virtual private space. In some cases, any physical object entering the private space may be completely or partially occluded. In some examples, when a portion of an object or person is completely occluded, the Extended Reality system 100 and / or the server may provide authorized users with (e.g., visually or audibly) indications of the object or person's location. In an illustrative example, if a portion of an unauthorized user's body (e.g., an arm) enters the virtual private space, the Extended Reality system 100 may completely or partially occlude that body portion. In another example, if a physical object (e.g., a part of a person's body, a cup, and / or other physical object) is placed where a virtual content item is being displayed in a virtual private space, the extended reality system 100 can display the virtual content item on top of the physical object, or it can occlude the physical object so that it is not displayed (or viewable) by the extended reality system 100. In the case of occlusion, the presence of the object can be alerted to the devices of authorized users participating in a virtual session associated with the virtual private space. In some examples, if an unauthorized person and / or other object comes into contact with virtual content in the virtual space, the virtual content can be moved to avoid the unauthorized person and / or other object. In some examples, if a virtual object depends on a real-world object (e.g., a virtual cup placed on a real-world table) and that real-world object is removed from the physical space, the extended reality system 100 can render a copy of the real-world object and maintain this dependency. In some cases, the extended reality system 100 can cause the virtual object to fall or move with a moving real-world object.

[0091] In some implementations, Extended Reality system 100 may perform object detection and / or recognition (e.g., human detection and / or recognition) to verify authorized users and / or identify when a physical user or object has entered the virtual private space or is within a certain distance of the virtual private space. Object detection and / or recognition may be image-based, voice-based, and / or use other techniques for detecting / recognizing objects. For example, authorized users may be verified by performing body, facial, and / or voice recognition. Verifying authorized users before allowing users to view and interact with virtual content in the virtual private space can prevent unauthorized personnel from using the authorized person's XR system and viewing and interacting with private content.

[0092] In some cases, crowdsourcing can be used to provide enhanced details of a virtual private space to authorized and / or unauthorized users. For example, image and sensor information from XR systems worn by multiple authorized users (and, in some cases, unauthorized users) can provide information that can be used to enhance the virtual private space. In some examples, in addition to XR systems, the computing devices of authorized users (and, in some cases, unauthorized users) (e.g., mobile devices, wearable devices, laptops, tablets, and / or other devices) can provide information for enhancing the virtual space. For example, crowdsourced data from authorized and / or unauthorized user devices can be used by Extended Reality System 100 (and / or by a server communicating with Extended Reality System 100) to reliably map and track the virtual space relative to the real-world environment, which can allow Extended Reality System 100 (and / or the server) to monitor the virtual private space more accurately. Improved monitoring of the virtual private space allows Extended Reality System 100 and / or the server to better notify authorized users of disturbances or alterations to the virtual private space.

[0093] In some implementations, all devices of authorized users participating in virtual sessions within a virtual private space can share information about the virtual private space to provide each device with the most accurate and up-to-date information about that space. This information may include unauthorized user disturbances, the location of virtual content within the virtual private space, alterations to the physical environment, the invisible angles and / or sides of the virtual content, and other information. Crowdsourcing the sharing of information related to invisible virtual content can help reduce latency and improve the accuracy of rendering virtual objects in a virtual private space shared by multiple users. This type of crowdsourcing can be important in large virtual spaces (larger than the space that can be mapped using sensors from a single XR system).

[0094] As described above, crowdsourcing can be achieved using the view (data) of a private space from an unauthorized user. For example, because the boundaries of a virtual private space are virtual (as defined by boundary information and associated virtual content) and not a rigid barrier between the unauthorized user and the content of the virtual private space, the unauthorized device (of the unauthorized user) can share information about the virtual private space with the authorized device (of the authorized user) to provide accurate and up-to-date information about the virtual private space in addition to the data shared between the authorized user's devices. In such an example, the unauthorized user will not be able to see the private virtual content within the virtual private space, but data related to that space can be shared with the authorized user.

[0095] In some examples, an authorized user may broadcast information (e.g., game scores, food or beverage orders, messages, and / or other information) and / or virtual content (e.g., virtual items in a virtual game, images of food or beverages being ordered, images of messages, and / or other content) to an unauthorized user. The amount of information provided to the unauthorized user's device can be controlled by the authorized user sending the information.

[0096] Image processing engine 124 can perform one or more image processing operations related to virtual content presented in the virtual private space and / or other image content. For example, image processing engine 124 can perform image processing operations based on data from image sensor 102. In some cases, image processing engine 124 can perform image processing operations such as filtering, depigmentation, scaling, color correction, color conversion, segmentation, noise reduction filtering, spatial filtering, artifact correction, etc. Rendering engine 126 can obtain image data generated and / or processed by computing component 110, image sensor 102, XR engine 120, virtual private space management engine 122, and / or image processing engine 124, and can render video and / or image frames for presentation on a display device.

[0097] Although the extended reality system 100 is shown to include certain components, those skilled in the art will understand that the extended reality system 100 may include more than just [the components shown in the original text]. Figure 1 The components shown may include more or fewer components. For example, in some cases, the extended reality system 100 may also include one or more memory devices (e.g., RAM, ROM, cache, etc.), one or more networking interfaces (e.g., wired and / or wireless communication interfaces, etc.), one or more display devices, and / or... Figure 1 Other hardware or processing devices not shown. See below for reference. Figure 9 An illustrative example of a computing system and hardware components that can be implemented using the extended reality system 100 is described.

[0098] Figure 3 This is a schematic diagram illustrating an example of an extended reality system 100 worn by user 301. Although in Figure 3 The extended reality system 100 is shown as an AR device (e.g., AR glasses), but the extended reality system 100 can include any suitable type of XR device, such as an HMD or other XR device. For illustrative purposes, AR will be used to describe the examples described below. However, the aspects described below can be applied to other types of XR, such as VR and MR. Figure 3The extended reality system 100 shown may include an optical see-through AR device that allows user 301 to view the real world while wearing the extended reality system 100. For example, user 301 can view objects 302 in the real world environment on a plane 304 at a distance from user 301. Figure 3 As shown and as described above, the extended reality system 100 has an image sensor 102 and a display 109. As described above, the display 109 may include glass, a screen, a lens, and / or other display mechanisms that allow the user 301 to see the real-world environment and also allow AR content to be displayed thereon. AR content (e.g., images, videos, graphics, virtual or AR objects, or other AR content) may be projected or otherwise displayed on the display 109. In one example, the AR content may include an enhanced version of object 302. In another example, the AR content may include additional AR content associated with object 302 and / or associated with one or more other objects in the real-world environment. Although in Figure 3 The image sensor 102 and the display 109 are shown, but in some implementations, the extended reality system 100 may include multiple cameras and / or multiple displays (e.g., a display for the right eye and a display for the left eye).

[0099] As mentioned above Figure 1 The described space can generate virtual private spaces for presenting AR content for a specific AR session. Figure 4 This is a schematic diagram illustrating an example of an AR virtual session, where virtual content 458 is displayed by an extended reality system 100 in a virtual private space 450. Boundary virtual content 451 is shown to outline the virtual private space. A group of authorized users is shown, where a first authorized user 452 is represented by a first virtual representation of his hand, a second authorized user 454 by a second virtual representation of his hand, and a third authorized user 456 by a third virtual representation of his hand. Authorized user 456 is using the extended reality system 100, and each of authorized users 452 and 454 is using a corresponding XR device. Authorized users 452, 454, and 456 are authorized to view the virtual content 458 of the virtual session using their respective XR devices.

[0100] Authorized users 452, 454, and 456 may be located in the restaurant and interested in participating in a virtual poker game using the restaurant's location. In some cases, authorized users 452, 454, and 456 may be authorized by providing login credentials to the user interface of the application installed on the corresponding XR device and associated with the virtual poker game (e.g., manually entering a username and password using an input mechanism, using facial recognition login, and / or other login technologies). In each case, the aforementioned hardware signature may be used to authorize the XR devices of authorized users 452, 454, and 456. Any other authorization or authentication technology may be used to authorize authorized users 452, 454, and 456.

[0101] Authorized users 452, 454, and 456 can recognize table 459 in the dining area as a surface on which virtual content 458 for a virtual poker game is to be displayed. Authorized user 456 (and / or other authorized users 452 and / or 454) of the Extended Reality System 100 can provide input to the Extended Reality System 100 indicating the virtual private space the user desires for the virtual poker game on table 459. In some cases, the user interface of an application associated with the virtual poker game can be used to provide input requesting the virtual private space.

[0102] In response to this input, the Extended Reality system 100 can generate or receive (e.g., from a server) data defining virtual content for a virtual poker game, as well as boundary information that can be used to generate boundary virtual content 451 defining the boundaries of the virtual private space 450. Data defining virtual content (e.g., virtual content 458) to be included in the virtual private space 450 can be provided to the different XR devices of authorized users 452, 454, and 456, enabling the XR devices to display the virtual content defining the virtual private space 450. For example, a server associated with the virtual poker game can provide virtual content (and in some cases, boundary information) to the XR devices. In some cases, the server can provide virtual content (and in some cases, boundary information) to the Extended Reality system 100 of authorized user 456, and the Extended Reality system 100 can provide virtual content (and in some cases, boundary information) to the XR devices of other users 452 and 454.

[0103] In some cases, as described above, boundary information may include boundary virtual content 451. In some cases, boundary information may alternatively or additionally include coordinate information defining the location of boundary virtual content 451 in a 3D map (e.g., a SLAM map) and feature points on table 459 defining where boundary virtual content 451 will be anchored or registered. Extended Reality System 100 may provide boundary information to XR devices of unauthorized users (unauthorized users) who are not authorized to view virtual content 458 and / or interact with it. Other users' XR devices may use the boundary information to generate and / or display boundary virtual content 451. In some cases, Extended Reality System 100 may display boundary virtual content 451 around virtual content 458, allowing authorized user 456 to view boundary virtual content 451. In some cases, the XR devices of authorized users 452, 454, and 456 may not display boundary virtual content 451 intended for viewing by authorized users 452, 454, and 456.

[0104] In some examples, such as Figure 4 As shown, the boundary virtual content 451 can be a virtual boundary displayed around a portion of the top of the table on which the virtual content is displayed (e.g., as a 3D frame around the virtual content). In other examples, the boundary virtual content 451 can be a virtual boundary displayed around the entire table 459, which serves as a background for the virtual content. In some cases, the application associated with a particular virtual session can define how the boundary virtual content is displayed. For example, in Figure 4 The virtual poker game shown can have a specific game board defined by the application used to facilitate the virtual poker game. Boundary virtual content 451 can be defined by the application such that it surrounds the game board of the virtual poker game. The boundary defined by boundary virtual content 451 can be configured in any other way.

[0105] Figure 4 The illustration is from the perspective of content being viewed by user 456 through extended reality system 100. As shown, user 456 can view virtual content 458 in the virtual private space 450 and also the physical table 459 below. In some examples, extended reality system 100 may display boundary virtual content 451 for authorized user 456 to view. In some examples, extended reality system 100 does not display boundary virtual content 451, for example, to avoid distracting authorized user 456 from boundary virtual content 451. Two other authorized users 452 and 454 can also view the same virtual content 458 and table 459 through their respective XR devices, but from different perspectives or angles (corresponding to where each of authorized users 452 and 454 is seated at table 459). See below for more details. Figure 5As described, an unauthorized user's XR device or system will not be able to access and will not be able to display virtual content 458. For example, any unauthorized user using an XR device will not be able to view virtual content 458, but will only be able to view boundary virtual content 451. The existence and / or location of virtual private space 450 may be notified to unauthorized users who are not using XR devices or systems.

[0106] Figure 5 It is shown Figure 4 A schematic diagram of another view of the virtual private space 450. Figure 5 The illustration shows how an unauthorized user, and such an unauthorized user, can view the virtual private space 450 from the perspective of using an XR device. Actual (physical) authorized users 452, 454, and 456 are shown sitting around table 459, each wearing a corresponding XR device (shown as AR glasses for illustrative purposes). As described above, authorized users 452, 454, and 456 are authorized to use their respective XR devices to view... Figure 4 The virtual content 458 of the virtual session is shown, and in some cases, the boundary virtual content 451 is viewed.

[0107] exist Figure 5 The diagram also illustrates unauthorized users 560 and 562. As previously stated, the XR devices of unauthorized users will not be accessible and will not be able to display virtual content 458. For example, any unauthorized user using an XR device will only be able to view the boundary virtual content 451. Unauthorized user 562 is the person sitting at table 559, which is close to the table 459 where authorized users 452, 454, and 456 are sitting. As shown, unauthorized user 562 is wearing XR system 564 (e.g., AR glasses) and may be viewing virtual content, but is not authorized to view the virtual content 458 shown in the virtual private space 450. Because unauthorized user 562 is not authorized to view virtual content 458, when unauthorized user 562 turns so that the XR system 564's field of view is directed toward table 459, the XR system 564 will display the boundary virtual content 451. Boundary Virtual Content 451 blocks unauthorized users 562 from viewing virtual content 450 in the Virtual Private Space 458.

[0108] Figure 6 This is a schematic diagram showing a side view of table 459 and boundary virtual content 451. (As shown) Figure 6As shown, the boundary virtual content 451 can be displayed to unauthorized users in three dimensions, which allows the virtual content within the virtual private space 450 to remain private from all perspectives. The boundary virtual content 451 can be displayed to authorized users in different ways. For example, the boundary virtual content 451 can be displayed to unauthorized users in three dimensions, and can be displayed to authorized users in a modified manner or not displayed to authorized users at all. In one example, the boundary virtual content 451 can be displayed as a dashed outline to authorized users in two dimensions so as not to impair the view of the virtual content 458.

[0109] The existence and / or location of the virtual private space 450 can be notified to unauthorized users who are not using XR devices. Figure 5 The unauthorized user 560 shown is a restaurant employee. As depicted, the unauthorized user 560 is bringing a beverage tray to table 459 for authorized users 452, 454, and 456. The unauthorized user 560 is not using an XR device and therefore cannot view the boundary virtual content 451. The unauthorized user 560 does possess a mobile device 561. Any one or more of the XR devices of authorized users 452, 454, and 456 (including the extended reality system 100) can detect the presence of mobile device 561 and can cause a notification indicating the presence and / or location of the virtual private space 450 to be sent to mobile device 561. In some examples, the extended reality system 100 (or one of the XR devices of users 452 and 454) can send a message to mobile device 561 containing information indicating the presence and location of the virtual private space 450. In another example, a server providing virtual content 458 to the XR devices of authorized users 452, 454, and 456 can send messages to the mobile device 561 of unauthorized user 560. For example... Figure 5 As shown, the message can indicate the existence of a "virtual private space at table 3," which refers to table 459 where authorized users 452, 454, and 456 are seated. Based on this notification or message, unauthorized user 560 can be informed that a virtual private space 450 exists at table 459 and can avoid placing beverages in the physical space occupied by the virtual private space 450.

[0110] If an unauthorized user 560 actually places a beverage or other physical item within the virtual private space 450, one or more XR devices of authorized users 452, 454, and 456 can output a notification to authorized users 452, 454, and 456. For example, the extended reality system 100 of authorized user 456 can display a notification on display 109 reminding authorized user 456 that a physical item has been placed in the virtual private space 450.

[0111] In some cases, one or more XR devices among the authorized users 452, 454, and 456 can detect when the unauthorized user 560 is within a certain distance of the virtual private space 450. For example, a threshold distance, such as 10 feet, 5 feet, 2 feet, and / or other distances, can be defined. If the unauthorized user 560 moves within the threshold distance of the virtual private space 450, one or more XR devices among the authorized users 452, 454, and 456 can output a notification to the authorized user 452, 454, and 456. For example, the extended reality system 100 of the authorized user 456 can display a notification on display 109 alerting the authorized user 456 that a person or object is approaching the virtual private space 450. In some examples, the threshold distance can be defined based on the physical environment in which the virtual private space is implemented. For example, in a restaurant environment, many people may move throughout the physical environment. In such cases, the threshold distance can be reduced to avoid sending a large number of notifications to the XR devices of authorized users 452, 454, and 456. In environments with fewer people or objects, the threshold distance can be increased.

[0112] As mentioned earlier, in some cases, virtual content in a virtual private space can be displayed with a higher priority than real-world content. Figures 7A-7C This is a schematic diagram illustrating an example of a person (unauthorized user 760) physically entering a virtual private space 750 where a virtual session is taking place in a restaurant (similar to...). Figure 4 (Example). Figure 7A As shown, the virtual private space 750 includes virtual content 758 and boundary virtual content 751. The virtual private space 750 is displayed as a physical object (in... Figure 7A (not shown in the image) (such as a table (similar to a table in the image) Figure 4 The table 459 shown is overlaid on top of each other. A person (unauthorized user 760) approaches the virtual private space 750 and places beverages 763 and 764 on the physical objects on which virtual content is displayed in the virtual private space 750. Beverages 763 and 764 may interfere with the viewing of virtual content by authorized users of the virtual private space 750.

[0113] When beverages 763 and 764, as well as portions of unauthorized user 760, are located within virtual private space 750, the authorized user's XR device (e.g., extended reality system 100) and / or the server communicating with the XR device in virtual private space 750 may not emphasize beverages 763 and 764 and / or portions of unauthorized user 760. For example, as Figure 7BAs shown, beverages 763 and 764, as well as the arm of unauthorized user 760, are completely occluded, making the objects appear transparent from the perspective of an authorized user viewing the virtual content in virtual private space 750. In AR systems, the display is typically an optical perspective display, in which case the user of the AR system can see the physical world through the display and also see the virtual content on the display. Virtual objects behave as if they are superimposed on the physical world. AR content that occludes physical objects can be generated to make the physical objects appear transparent from the perspective of the AR system user. In an illustrative example, a background model can be generated that models the background of a given environment. For example, the background model can model a static part of a restaurant where a virtual session associated with virtual private space 750 is taking place. Beverages 763 and 764, as well as the arm of unauthorized user 760, can be enhanced (or occluded) using portions of the background model that correspond to the portions of the restaurant behind the arms of beverages 763 and 764, as well as the arm of unauthorized user 760, so that those portions of the background model are displayed in front of the arms of beverages 763 and 764, as well as the arm of unauthorized user 760. Frames captured by an authorized user’s XR device viewing virtual content in the Virtual Private Space 750 can be redrawn using the content (e.g., pixels) of those parts of the background model.

[0114] In some examples, the XR device of an authorized user participating in a virtual session associated with virtual private space 750 can provide visual, audible, or other types of indications regarding the locations of beverages 763 and 764, as well as the arm of an unauthorized user 760. For example, the authorized user's XR device can display a notification on the XR device's display reminding the authorized user that beverage 763 has been placed in virtual private space 450.

[0115] Figure 7C This is a schematic diagram illustrating another example where beverages 763 and 764, and the arm of the unauthorized user 760, are not emphasized when they are located within the virtual private space 750. Figure 7C As shown, beverages 763 and 764, as well as the arm of the unauthorized user 760, are displayed with a modified appearance to minimize the disruptive impact on authorized users participating in the virtual session associated with the virtual private space 750. In some cases, the authorized user's XR device may initially display beverages 763 and 764, as well as the arm of the unauthorized user 760, fully within the virtual private space 750, and then gradually fade the display of beverages 763 and 764, as well as the arm of the unauthorized user 760, until the objects are completely obscured.

[0116] The virtual private spaces described herein can provide various benefits in AR and / or other XR environments. For example, as mentioned above, virtual private spaces can prevent disruption of activity within spaces containing virtual content. Virtual private spaces can also prevent unauthorized users from viewing virtual content. In some cases, authorized users (referred to as authorized users) can be alerted to the presence or proximity of one or more unauthorized users to the virtual private space. In some cases, authorized users can be alerted if the virtual content within the virtual private space is compromised in some way (e.g., an unauthorized user affects the real-world space, resulting in some impact on the virtual content).

[0117] Figure 8 This is a flowchart illustrating an example of a process 800 for generating virtual content for one or more virtual private spaces using the techniques described herein. At block 802, process 800 includes a virtual session initiated by a device for presenting virtual content. At block 804, process 800 includes identifying a portion of a virtual private space for the virtual session, which is a physical space available for use as at least a portion of the virtual content. In an illustrative example, the virtual private space may include... Figures 4-6 The virtual private space 450 shown is illustrated.

[0118] At box 806, process 800 includes outputting boundary information defining the boundaries of the virtual private space. As described above, the boundary information can be used to generate boundary virtual content defining the limits of the virtual private space. In some examples, as described above, the boundary information includes world coordinates in physical space and / or one or more feature points in at least one image of physical space, which can be used to generate boundary virtual content identifying the boundaries of the virtual private space. In some examples, process 800 includes outputting boundary virtual content identifying the boundaries or limits of the virtual private space. The boundary virtual content can be viewed by one or more unauthorized users of the virtual session.

[0119] At box 808, process 800 includes generating at least that portion of the virtual content for a virtual private space. In some cases, the entire virtual content may be generated for the virtual private space. In some cases, a portion (less than the entire virtual content) of the virtual content may be generated for the virtual private space. For example, a first portion of the virtual content may extend beyond the boundaries of the virtual private space, and more sensitive content (e.g., content defined by an authorized user, a content provider associated with the virtual session, or otherwise defined as sensitive) may be confined within the boundaries of the virtual private space. At least that portion of the virtual content may be viewed in the virtual private space by one or more authorized users of the virtual session, but not by one or more unauthorized users. For example, each of the one or more authorized users may have an extended reality device or other device authenticated or otherwise authorized to display at least that portion of the virtual content, while each of the one or more unauthorized users may have an extended reality device or other device unauthenticated or unauthorized to display at least that portion of the virtual content. In some examples, based on boundary virtual content associated with boundary information, at least that portion of the virtual content may not be viewed by one or more unauthorized users. In some cases, based on boundary virtual content associated with boundary information, the real-world volume defined within the boundary virtual content cannot be viewed by one or more unauthorized users.

[0120] In some examples, process 800 includes receiving an indication that an unauthorized user has entered the virtual private space. Based on the indication that an unauthorized user has entered the virtual private space, process 800 may obscure at least a portion of the unauthorized user's information to prevent it from being viewed by one or more authorized users within the virtual private space.

[0121] In some examples, process 800 receives an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space. Based on the indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space, process 800 may move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0122] In some examples, process 800 includes receiving an indication that an unauthorized user has entered a virtual private space or is within a threshold distance for entering a virtual private space. Based on the indication that an unauthorized user has entered a virtual private space or is within a threshold distance for entering a virtual private space, process 800 may output a notification indicating the presence of a virtual private space. In some cases, outputting a notification includes sending a notification to the unauthorized user's device, outputting an audio notification indicating the presence of a virtual private space, a combination thereof, and / or outputting another type of notification (such as the notification described herein). In some cases, the notification includes an outline of the unauthorized user.

[0123] In some examples, process 800 includes receiving an indication that an unauthorized user has entered a virtual private space or is within a threshold distance for entering the virtual private space. Based on the indication that an unauthorized user has entered a virtual private space or is within a threshold distance for entering the virtual private space, process 800 may output a notification to one or more devices of one or more authorized users, indicating that the unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space. In some cases, outputting a notification includes sending a notification to the device of at least one of the one or more authorized users, outputting an audio notification indicating the presence of the unauthorized user, a combination thereof, and / or outputting another type of notification (such as the notification described herein). In some cases, the notification includes an outline of the unauthorized user.

[0124] In some examples, the device is an extended reality device, which includes one or more displays, one or more receivers (which can receive data and / or other information), one or more transmitters (which can transmit data and / or other information), and / or one or more transceivers (which can receive and transmit data and / or other information). In some examples, process 800 includes displaying at least that portion of virtual content by the extended reality device (e.g., via a display of the extended reality device).

[0125] In some examples, the device is a first extended reality device (including one or more displays, one or more receivers, one or more transmitters, and / or one or more transceivers). In some examples, process 800 includes at least that portion of outputting virtual content to a second extended reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers). The second extended reality device may be used by an authorized user.

[0126] In some examples, the device is a first extended reality device (including one or more displays, one or more receivers, one or more transmitters, and / or one or more transceivers). In some examples, process 800 includes: displaying at least that portion of the virtual content by the first extended reality device (e.g., via a display of the extended reality device); and (e.g., outputting at least that portion of the virtual content to a second extended reality device using one or more receivers, one or more transmitters, and / or one or more transceivers).

[0127] In some examples, process 800 includes: displaying at least a portion of the virtual content by a first extended reality device (e.g., via a display of the extended reality device); and outputting at least a portion of the virtual content to a second extended reality device by the first extended reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers). In some examples, process 800 includes: receiving boundary information by the second extended reality device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers of the second extended reality device); and generating virtual boundary content by the second extended reality device (e.g., using one or more processors of the second extended reality device) based on the received boundary information.

[0128] In some examples, one or more of the devices or apparatuses are server devices. In some examples, process 800 includes at least that portion of outputting virtual content to an extended reality device by the server device (e.g., using one or more receivers, one or more transmitters, and / or one or more transceivers of the server device).

[0129] In some examples, the processes described herein (e.g., process 800 and / or other processes described herein) may be performed by a computing device or apparatus. In one example, process 800 may be performed by... Figure 1 The extended reality system 100 is used to execute this process. In another example, the process 800 can be performed by a system with... Figure 9 The computing device of the computing system 900 shown is used to perform the operation. For example, it has a computing device in... Figure 9 The computing device of the computing architecture shown may include Figure 1 The components of the extended reality system 100, and can achieve Figure 8 The operation.

[0130] Computing devices may include any suitable device, such as mobile devices (e.g., mobile phones), desktop computing devices, tablet computing devices, wearable devices (e.g., VR headsets, AR headsets, AR glasses, network-connected watches or smartwatches, or other wearable devices), server computers, autonomous vehicles or computing devices for autonomous vehicles, robotic devices, televisions, and / or any other computing devices with the resource capability to perform the processes described herein (including process 800). In some cases, a computing device or apparatus may include various components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and / or other components configured to perform the steps of the processes described herein. In some examples, a computing device may include a display, a network interface configured to transmit and / or receive data, any combination thereof, and / or other components. The network interface may be configured to transmit and / or receive Internet Protocol (IP) based data or other types of data.

[0131] Components of a computing device can be implemented in circuitry. For example, components may include and / or may be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable circuits (e.g., a microprocessor, graphics processing unit (GPU), digital signal processor (DSP), central processing unit (CPU), and / or other suitable circuitry) and / or may include and / or be implemented using computer software, firmware, or any combination thereof to perform the various operations described herein.

[0132] Process 800 is shown as a logic flowchart, the operations of which represent a series of operations that can be implemented using hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media, which, when executed by one or more processors, perform the described operations. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific data type. The order in which the operations are described is not intended to be construed as limiting, and any number of the described operations can be combined in any order and / or in parallel to implement these processes.

[0133] Furthermore, process 800 and / or other processes described herein may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that executes jointly on one or more processors, implemented in hardware, or a combination thereof. As mentioned above, the code may be stored, for example, in the form of a computer program comprising multiple instructions executable by one or more processors, on a computer-readable or machine-readable storage medium. The computer-readable or machine-readable storage medium may be non-transitory.

[0134] Figure 9 This is a schematic diagram illustrating an example of a system used to implement certain aspects of the techniques described herein. Specifically, Figure 9 An example of a computing system 900 is shown. The computing system 900 can be any computing device, such as constituting an internal computing system, a remote computing system, a camera, or any component thereof, wherein the components of the system communicate with each other using connection 905. Connection 905 can be a physical connection using a bus, or a direct connection to processor 910 (such as in a chipset architecture). Connection 905 can also be a virtual connection, a network connection, or a logical connection.

[0135] In some embodiments, the computing system 900 is a distributed system, wherein the functions described herein may be distributed across a data center, multiple data centers, a peer-to-peer network, etc. In some embodiments, one or more of the system components described represent a plurality of such components, each performing some or all of the functions described for that component. In some embodiments, these components may be physical or virtual devices.

[0136] Example system 900 includes at least one processing unit (CPU or processor) 910 and connection 905, which couples various system components, including system memory 915 (such as read-only memory (ROM) 920 and random access memory (RAM) 925), to processor 910. Computing system 900 may include cache 912 of high-speed memory, which is directly connected to, close to, or integrated into processor 910.

[0137] Processor 910 may include any general-purpose processor and hardware or software services configured to control processor 910 (such as services 932, 934, and 936 stored in storage device 930), as well as dedicated processors in which software instructions are incorporated into the actual processor design. Processor 910 may essentially be a fully self-contained computing system, containing multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors may be symmetric or asymmetric.

[0138] To enable user interaction, the computing system 900 includes an input device 945 that can represent any number of input mechanisms, such as a microphone for voice, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, and voice input. The computing system 900 may also include an output device 935, which can be one or more of a plurality of output mechanisms. In some cases, a multimodal system allows the user to provide multiple types of input / output to communicate with the computing system 900. The computing system 900 may include a communication interface 940, which typically governs and manages user input and system output. The communication interface may use wired and / or wireless transceivers to perform or facilitate the reception and / or transmission of wired or wireless communications, including those utilizing the following: audio jacks / plugs, microphone jacks / plugs, Universal Serial Bus (USB) ports / plugs, Apple® Lightning® ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, proprietary wired ports / plugs, BLUETOOTH® wireless signal transmission, BLUETOOTH® Low Energy (BLE) wireless signal transmission, IBEACON® wireless signal transmission, Radio Frequency Identification (RFID) wireless signal transmission, Near Field Communication (NFC) wireless signal transmission, Dedicated Short Range Communication (DSRC) wireless signal transmission, and 802.11. The communication interface 940 may include Wi-Fi wireless signal transmission, wireless local area network (WLAN) signal transmission, visible light communication (VLC), microwave access global interoperability (WiMAX), infrared (IR) wireless signal transmission, public switched telephone network (PSTN) signal transmission, integrated services digital network (ISDN) signal transmission, 3G / 4G / 5G / LTE cellular data network wireless signal transmission, ad hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or some combination thereof. The communication interface 940 may also include one or more Global Navigation Satellite System (GNSS) receivers or transceivers for determining the location of the computing system 900 based on receiving one or more signals from one or more satellites associated with one or more GNSS systems. GNSS includes, but is not limited to, the US-based Global Positioning System (GPS), the Russian-based Global Navigation Satellite System (GLONASS), the Chinese-based BeiDou Navigation Satellite System (BDS), and the European-based Galileo GNSS system. There are no restrictions on operation for any particular hardware layout, and therefore the basic functionality here can be easily replaced with an improved hardware or firmware layout when it is developed.

[0139] Storage device 930 may be a non-volatile and / or non-transitory and / or computer-readable storage device, and may be a hard disk or other type of computer-readable medium capable of storing data accessible by a computer, such as cartridges, flash memory cards, solid-state storage devices, digital multifunction disks, magnetic tape cassettes, floppy disks, hard disks, magnetic tapes, strips, any other magnetic storage media, flash memory, memristor memory, any other solid-state storage, CD-ROM, rewritable CD, DVD, Blu-ray Disc, holographic disc, another optical medium, secure digital (SD) cards, microSD cards, Memory Stick® cards, smart... Card chip, EMV chip, Subscriber Identity Module (SIM) card, mini / micro / nano / micro SIM card, another integrated circuit (IC) chip / card, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash EPROM, cache memory (L1 / L2 / L3 / L4 / L5 / L#), resistive random access memory (RRAM / ReRAM), phase change memory (PCM), spin-transfer torque RAM (STT-RAM), another memory chip or cassette, and / or combinations thereof.

[0140] Storage device 930 may include software services, servers, services, etc., which cause the system to perform functions when the code defining such software is executed by processor 910. In some embodiments, hardware services performing a particular function may include software components stored in a computer-readable medium connected to necessary hardware components (such as processor 910, connection 905, output device 935, etc.) for performing that function. The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. Computer-readable media may include non-transitory media in which data can be stored and excludes carrier waves and / or transient electronic signals propagating wirelessly or over a wired connection. Examples of non-transitory media may include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. Computer-readable media may have code and / or machine-executable instructions stored thereon, which may represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. A code segment can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or sent via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0141] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0142] Specific details are provided in the foregoing description to provide a thorough understanding of the embodiments and examples provided herein. However, those skilled in the art will understand that these embodiments can be practiced without these specific details. For clarity, in some instances, the techniques described herein may be presented as comprising individual functional blocks, including devices, device components, steps or routines in a software-embodied method, or a combination of hardware and software. Additional components may be used in addition to those shown in the figures and / or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure these embodiments with unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring these embodiments.

[0143] The various embodiments described above may be presented as processes or methods, depicted as flowcharts, schematic diagrams, data flow diagrams, structural diagrams, or block diagrams. While a flowchart may describe operations as a sequential process, many of these operations may be performed in parallel or concurrently. Furthermore, the order of operations may be rearranged. A process terminates when its operations are completed, but may have additional steps not included in the diagram. A process may correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0144] The processes and methods described in the examples above can be implemented using computer-executable instructions, which are stored in or otherwise made available from a computer-readable medium. Such instructions may include, for example, instructions or data that cause a general-purpose computer, special-purpose computer, or processing device to perform or otherwise configure it to perform a particular function or a particular set of functions. The portion of the computer resources used may be accessible via a network. Computer-executable instructions may be, for example, binary files, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that may be used to store instructions, information used, and / or information created during the methods according to the described examples include hard disks or optical disks, flash memory, USB devices equipped with non-volatile memory, network storage devices, etc.

[0145] Devices implementing the processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of a variety of form factors. When implemented with software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) for performing the necessary tasks may be stored in a computer-readable or machine-readable medium. A processor may perform the necessary tasks. Typical examples of form factors include laptop computers, smartphones, mobile phones, tablet devices, or other small form factor personal computers, personal digital assistants, rack-mount devices, standalone devices, etc. The functionality described herein may also be embodied in peripheral devices or plug-in cards. By further example, such functionality may also be implemented on a circuit board between different chips or different processes executed in a single device.

[0146] Instructions, media for transmitting such instructions, computing resources for executing them, and other structures for supporting such computing resources are example modules for providing the functionality described in this disclosure.

[0147] In the foregoing description, various aspects of this application have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that this application is not limited thereto. Therefore, although illustrative embodiments of this application have been described in detail herein, it will be understood that the inventive concept may be embodied and employed differently in other ways, and the appended claims are intended to be interpreted as including such variations, in addition to those limited by the prior art. Various features and aspects of the above-described applications may be used individually or in combination. Furthermore, the embodiments may be utilized in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Therefore, the specification and drawings are to be considered illustrative rather than restrictive. For illustrative purposes, the methods have been described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in a different order than that described.

[0148] Those skilled in the art will understand that, without departing from the scope of this specification, the less than ("<") and greater than (">") symbols or terms used herein can be replaced with less than or equal to (">") respectively. ") and greater than or equal to (" Replace with the ") symbol.

[0149] When a component is described as being “configured” to perform certain operations, such configuration can be achieved, for example, by designing electronic circuits or other hardware to perform the operation, programming programmable electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform the operation, or any combination thereof.

[0150] The phrase “coupled to” refers to any component that is physically connected to another component, directly or indirectly, and / or any component that communicates with another component, directly or indirectly (e.g., connected to another component via a wired or wireless connection and / or other suitable communication interface).

[0151] The language of a claim that states "at least one" and / or "one or more" in a set indicates that one or more members of the set (in any combination) satisfy the claim. For example, the language of a claim stating "at least one of A and B" or "at least one of A or B" means A, B, or A and B. In another example, the language of a claim stating "at least one of A, B, and C" or "at least one of A, B, or C" means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The use of "at least one" and / or "one or more" in a language set does not limit the set to items listed in that set. For example, the language of a claim stating "at least one of A and B" or "at least one of A and B" may mean A, B, or A and B, and may additionally include items not listed in the set of A and B.

[0152] The various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the examples disclosed herein can be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been generally described above in relation to their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this application.

[0153] The techniques described herein can also be implemented using electronic hardware, computer software, firmware, or any combination thereof. Such techniques can be implemented in any of a variety of devices, such as general-purpose computers, mobile phones with wireless communication devices, or integrated circuit devices with multiple uses (including applications in mobile phones with wireless communication devices and other devices). Any feature described as a module or component can be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices. If implemented in software, the techniques can be implemented at least in part by a computer-readable data storage medium comprising program code that, when executed, performs one or more of the methods, algorithms, and / or operations described above. The computer-readable data storage medium can form part of a computer program product, which may include packaging material. The computer-readable medium may include memory or data storage media, such as random access memory (RAM) (such as synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, etc. Alternatively or concurrently, the technology may be implemented, at least in part, by a computer-readable communication medium (such as a propagating signal or wave) that carries or transmits program code in the form of instructions or data structures and can be accessed, read, and / or executed by a computer.

[0154] The program code can be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor can be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Therefore, as used herein, the term "processor" may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or apparatus suitable for implementing the techniques described herein.

[0155] Example 1: A method for generating virtual content. The method includes: initiating a virtual session by a device for presenting virtual content; identifying, for the virtual session, a portion of a virtual private space of physical space for use as at least a portion of the virtual content; outputting boundary information defining the boundaries of the virtual private space; and generating, for the virtual private space, at least the portion of the virtual content, which can be viewed by one or more authorized users of the virtual session in the virtual private space and cannot be viewed by one or more unauthorized users.

[0156] Example 2: According to the method of Example 1, wherein, based on the boundary virtual content associated with the boundary information, at least a portion of the virtual content cannot be viewed by the one or more unauthorized users.

[0157] Example 3: The method according to any one of Examples 1 or 2, wherein, based on the boundary virtual content associated with the boundary information, the real-world volume defined within the boundary virtual content cannot be viewed by the one or more unauthorized users.

[0158] Example 4: The method according to any one of Examples 1 to 3 further includes: outputting boundary virtual content that identifies the boundary of the virtual private space, the boundary virtual content being viewable by the one or more unauthorized users of the virtual session.

[0159] Example 5: The method according to any one of Examples 1 to 4, wherein the boundary information includes at least one of the following: world coordinates in the physical space, and one or more feature points in at least one image of the physical region, the one or more feature points being used to generate boundary virtual content identifying the boundary of the virtual private space.

[0160] Example 6: According to any one of Examples 1 to 5, receive an indication that an unauthorized user has entered the virtual private space; and based on the indication that the unauthorized user has entered the virtual private space, obscure at least a portion of the unauthorized user to prevent it from being viewed by the one or more authorized users in the virtual private space.

[0161] Example 7: The method according to any one of Examples 1 to 6 further includes: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0162] Example 8: The method according to any one of Examples 1 to 7 further includes: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and outputting a notification indicating the existence of the virtual private space based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

[0163] Example 9: According to the method of Example 8, the output of the notification includes at least one of the following: sending the notification to the device of the unauthorized user, and outputting an audio notification indicating the existence of the virtual private space.

[0164] Example 10: The method according to any one of Examples 1 to 9 further includes: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, outputting a notification to one or more devices of the one or more authorized users, the notification indicating that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

[0165] Example 11: The method according to Example 10, wherein the notification includes the profile of the unauthorized user.

[0166] Example 12: The method according to any one of Examples 1 to 11, wherein the device is an extended reality device, the method further comprising: displaying at least a portion of the virtual content by the extended reality device.

[0167] Example 13: The method according to any one of Examples 1 to 12, wherein the device is a first extended reality device, and the method further includes: outputting at least a portion of the virtual content to a second extended reality device.

[0168] Example 14: The method according to any one of Examples 1 to 13, wherein the device is a first extended reality device, and the method further includes: displaying at least a portion of the virtual content by the first extended reality device; and outputting at least a portion of the virtual content to a second extended reality device.

[0169] Example 15: The method according to any one of Examples 1 to 14, wherein the device is a first extended reality device, and the method further includes: displaying at least a portion of the virtual content by the first extended reality device; outputting at least a portion of the virtual content to a second extended reality device; receiving the boundary information by the second extended reality device; and generating the virtual boundary content by the second extended reality device based on the received boundary information.

[0170] Example 16: The method according to any one of Examples 1 to 15, wherein the device is a server device, and the method further includes: outputting at least a portion of the virtual content from the server device to an extended reality device.

[0171] Example 17: An apparatus for generating virtual content. The apparatus includes: a memory configured to store virtual content; and one or more processors coupled to the memory and configured to: initiate a virtual session for presenting the virtual content; identify, for the virtual session, a portion of a virtual private space of physical space for use as at least a portion of the virtual content; output boundary information defining the boundaries of the virtual private space; and generate, for the virtual private space, at least the portion of the virtual content, which can be viewed in the virtual private space by one or more authorized users of the virtual session and cannot be viewed by one or more unauthorized users.

[0172] Example 18: The apparatus according to Example 17, wherein, based on boundary virtual content associated with the boundary information, at least a portion of the virtual content cannot be viewed by the one or more unauthorized users.

[0173] Example 19: An apparatus according to any one of Examples 17 or 18, wherein a real-world volume defined within the boundary virtual content, based on boundary virtual content associated with the boundary information, cannot be viewed by the one or more unauthorized users.

[0174] Example 20: An apparatus according to any one of Examples 17 to 19, wherein the one or more processors are configured to: output boundary virtual content identifying the boundary of the virtual private space, the boundary virtual content being viewable by the one or more unauthorized users of the virtual session.

[0175] Example 21: An apparatus according to any one of Examples 17 to 20, wherein the boundary information includes at least one of the following: world coordinates in the physical space, and one or more feature points in at least one image of the physical region, the one or more feature points being used to generate boundary virtual content identifying the boundary of the virtual private space.

[0176] Example 22: An apparatus according to any one of Examples 17 to 21, wherein the one or more processors are configured to: receive an indication that an unauthorized user has entered the virtual private space; and based on the indication that the unauthorized user has entered the virtual private space, obscure at least a portion of the unauthorized user so that it can be viewed by the one or more authorized users in the virtual private space.

[0177] Example 23: The apparatus according to any one of Examples 17 to 22 further includes: receiving an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, moving one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

[0178] Example 24: An apparatus according to any one of Examples 17 to 23, wherein the one or more processors are configured to: receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and output a notification indicating the existence of the virtual private space based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

[0179] Example 25: The apparatus according to Example 24, wherein outputting the notification includes at least one of the following: sending the notification to the device of the unauthorized user, and outputting an audio notification indicating the existence of the virtual private space.

[0180] Example 26: An apparatus according to any one of Examples 17 to 25, wherein the one or more processors are configured to: receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance for entering the virtual private space; and, based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, output a notification to one or more devices of the one or more authorized users, the notification indicating that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

[0181] Example 27: The apparatus according to Example 26, wherein the notification includes the profile of the unauthorized user.

[0182] Example 28: An apparatus according to any one of Examples 17 to 27, wherein the apparatus is a first extended reality device, and wherein the one or more processors are configured to cause at least a portion of the virtual content to be output to a second extended reality device.

[0183] Example 29: An apparatus according to any one of Examples 17 to 28, wherein the apparatus is an extended reality device.

[0184] Example 30: The apparatus according to any one of Examples 17 to 29 further includes a display.

[0185] Example 31: The apparatus according to Example 30, wherein the display is configured to display at least a portion of the virtual content.

[0186] Example 32: The apparatus according to Example 30, wherein the apparatus is a first extended reality device, and wherein: the display is configured to display at least said portion of the virtual content; and the one or more processors are configured to: output at least said portion of the virtual content to a second extended reality device.

[0187] Example 33: An apparatus according to Example 30, wherein the apparatus is a first extended reality device, and wherein: the display is configured to display at least a portion of the virtual content; the one or more processors are configured to output at least a portion of the virtual content to a second extended reality device; the second extended reality device is configured to receive the boundary information; and the second extended reality device is configured to generate the virtual boundary content based on the received boundary information.

[0188] Example 34: An apparatus according to any one of Examples 17 to 33, wherein the apparatus is a server device, and wherein the one or more processors are configured to output at least a portion of the virtual content to an extended reality device via the server device.

[0189] Example 35: A non-transitory computer-readable medium for an extended reality system having instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform any of the operations described in Examples 1 to 16.

[0190] Example 35: An extended reality system comprising one or more units for performing any of the operations described in Examples 1 to 16.

Claims

1. A method for generating virtual content, the method comprising: A virtual session initiated by the device to present virtual content; For the virtual session, identify a portion of the physical space that is used for presenting at least a portion of the virtual content; Output boundary virtual content that identifies the boundaries of the virtual private space, and the boundary virtual content can be viewed by one or more unauthorized users of the virtual session; as well as At least a portion of the virtual content is generated for the virtual private space, wherein the boundary virtual content is configured to block the portion of the physical space such that at least a portion of the virtual content can be viewed in the virtual private space by one or more authorized users of the virtual session and cannot be viewed by one or more unauthorized users of the virtual session.

2. The method according to claim 1, wherein, Based on the boundary virtual content, the real-world volume defined within the boundary virtual content cannot be viewed by the one or more unauthorized users.

3. The method according to claim 1, further comprising: Output at least one of the following: world coordinates in the physical space, or one or more feature points of at least one physical object in at least one image in the physical space, the one or more feature points being used to generate boundary virtual content identifying the boundary of the virtual private space.

4. The method according to claim 1, further comprising: Receive an instruction regarding an unauthorized user entering the virtual private space; as well as Based on the indication that the unauthorized user has entered the virtual private space, at least a portion of the unauthorized user is obscured to prevent it from being viewed by one or more authorized users in the virtual private space.

5. The method according to claim 1, further comprising: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; as well as Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance of entering the virtual private space, move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

6. The method according to claim 1, further comprising: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; as well as Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance to enter the virtual private space, a notification indicating the existence of the virtual private space is output.

7. The method according to claim 6, wherein, The notification output includes at least one of the following: sending the notification to the unauthorized user's device, or outputting an audio notification indicating the existence of the virtual private space.

8. The method according to claim 1, further comprising: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; as well as Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, a notification is output to one or more devices of the one or more authorized users, the notification indicating that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

9. The method according to claim 8, wherein, The notification includes a profile of the unauthorized user.

10. The method according to claim 1, wherein, The device is an extended reality device, and the method further includes: At least a portion of the virtual content is displayed by the extended reality device.

11. The method according to claim 1, wherein, The device is a first extended reality device, and the method further includes: Output at least a portion of the virtual content to a second extended reality device.

12. The method according to claim 1, wherein, The device is a first extended reality device, and the method further includes: The virtual content is displayed by the first extended reality device at least a portion thereof; and Output at least a portion of the virtual content to a second extended reality device.

13. The method according to claim 1, wherein, The device is a first extended reality device, and the method further includes: At least a portion of the virtual content is displayed by the first extended reality device; Output at least a portion of the virtual content to a second extended reality device; Boundary information is received by the second extended reality device; and The second extended reality device generates the boundary virtual content based on the received boundary information.

14. The method according to claim 1, wherein, The device is a server device, and the method further includes: The server device outputs at least a portion of the virtual content to the extended reality device.

15. An apparatus for generating virtual content, comprising: Memory, configured to store virtual content; as well as One or more processors coupled to the memory and configured to: Initiate a virtual session for presenting virtual content; For the virtual session, identify a portion of the physical space that is used for presenting at least a portion of the virtual content; Output boundary virtual content that identifies the boundaries of the virtual private space, and the boundary virtual content can be viewed by one or more unauthorized users of the virtual session; as well as At least a portion of the virtual content is generated for the virtual private space, wherein the boundary virtual content is configured to block the portion of the physical space such that at least a portion of the virtual content can be viewed in the virtual private space by one or more authorized users of the virtual session and cannot be viewed by one or more unauthorized users of the virtual session.

16. The apparatus according to claim 15, wherein, Based on the boundary virtual content, the real-world volume defined within the boundary virtual content cannot be viewed by the one or more unauthorized users.

17. The apparatus according to claim 15, wherein, The one or more processors are configured to: Output at least one of the following: world coordinates in the physical space, or one or more feature points of at least one physical object in at least one image of the physical space, the one or more feature points being used to generate boundary virtual content that identifies the boundary of the virtual private space.

18. The apparatus according to claim 15, wherein, The one or more processors are configured to: Receive an instruction regarding an unauthorized user entering the virtual private space; and Based on the indication that the unauthorized user has entered the virtual private space, at least a portion of the unauthorized user is obscured to prevent it from being viewed by one or more authorized users in the virtual private space.

19. The apparatus of claim 15, further comprising: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; as well as Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance of entering the virtual private space, move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

20. The apparatus according to claim 15, wherein, The one or more processors are configured to: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; and Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance to enter the virtual private space, a notification indicating the existence of the virtual private space is output.

21. The apparatus according to claim 20, wherein, The notification output includes at least one of the following: sending the notification to the unauthorized user's device, or outputting an audio notification indicating the existence of the virtual private space.

22. The apparatus according to claim 15, wherein, The one or more processors are configured to: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; and Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space, a notification is output to one or more devices of the one or more authorized users, the notification indicating that the unauthorized user has entered the virtual private space or is within the threshold distance for entering the virtual private space.

23. The apparatus according to claim 22, wherein, The notification includes a profile of the unauthorized user.

24. The apparatus according to claim 15, wherein, The device is a first extended reality device, and wherein the one or more processors are configured to: This causes at least a portion of the virtual content to be output to a second extended reality device.

25. The apparatus according to claim 15, wherein, The device is an augmented reality device.

26. The apparatus of claim 15, further comprising a display.

27. The apparatus according to claim 26, wherein, The display is configured to display at least a portion of the virtual content.

28. The apparatus according to claim 26, wherein, The device is a first extended reality device, and wherein: The display is configured to display at least a portion of the virtual content; and The one or more processors are configured to output at least a portion of the virtual content to a second extended reality device.

29. The apparatus according to claim 26, wherein, The device is a first extended reality device, and wherein: The display is configured to display at least a portion of the virtual content; The one or more processors are configured to output at least a portion of the virtual content to the second extended reality device; The second extended reality device is configured to receive boundary information; and The second extended reality device is configured to generate the boundary virtual content based on the received boundary information.

30. The apparatus according to claim 15, wherein, The apparatus is a server device, and wherein the one or more processors are configured to: The server device outputs at least a portion of the virtual content to the extended reality device.

31. A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, causing the one or more processors to perform the following operations: Initiate a virtual session for presenting virtual content; For the virtual session, identify a portion of the physical space that is used for presenting at least a portion of the virtual content; Output boundary virtual content that identifies the boundaries of the virtual private space, and this boundary virtual content can be viewed by one or more unauthorized users of the virtual session; and At least a portion of the virtual content is generated for the virtual private space, wherein, The boundary virtual content is configured to block the portion of the physical space such that at least the portion of the virtual content can be viewed by one or more authorized users of the virtual session in the virtual private space and cannot be viewed by one or more unauthorized users of the virtual session.

32. The non-transitory computer-readable medium of claim 31, further comprising, when executed by the one or more processors, instructions that cause the one or more processors to perform the following operations: Output at least one of the following: world coordinates in the physical space, or one or more feature points of at least one physical object in at least one image in the physical space, the one or more feature points being used to generate boundary virtual content identifying the boundary of the virtual private space.

33. The non-transitory computer-readable medium according to claim 31, wherein, Based on the boundary virtual content, the real-world volume defined within the boundary virtual content cannot be viewed by the one or more unauthorized users.

34. The non-transitory computer-readable medium of claim 31, further comprising, when executed by the one or more processors, instructions that cause the one or more processors to perform the following operations: Receive an instruction regarding an unauthorized user entering the virtual private space; and Based on the indication that the unauthorized user has entered the virtual private space, at least a portion of the unauthorized user is obscured to prevent it from being viewed by one or more authorized users in the virtual private space.

35. The non-transitory computer-readable medium of claim 31, further comprising, when executed by the one or more processors, instructions that cause the one or more processors to perform the following operations: Receive an indication that an unauthorized user has entered the virtual private space or is within a threshold distance to enter the virtual private space; and Based on the indication that the unauthorized user has entered the virtual private space or is within the threshold distance of entering the virtual private space, move one or more virtual objects in the virtual private space to avoid at least a portion of the unauthorized user.

Images