Virtual space, mixed reality space and combined mixed reality space for improved interaction and collaboration

Abstract

The invention relates to a virtual space, a mixed reality space, and a combined mixed reality space for improved interaction and collaboration. In one embodiment, a method includes displaying first virtual content to a first user in a virtual area, the virtual area including one or more second virtual content, inferring an intent of the first user to interact with the first virtual content based on one or more of a first user action or contextual information, and displaying the first virtual content to the first user based on the inferred intent. And adjusting one or more configurations associated with one or more of the second virtual content based on an inference of an intent of the first user to interact with the first virtual content.

Description

[0001] This application is a divisional application of an application with an application date of February 14, 2019, application number 201980093248.8, and invention name “Virtual space, mixed reality space and combined mixed reality space for improving interaction and collaboration”. Technical Field

[0002] The present disclosure relates generally to artificial reality environments, including virtual reality environments and mixed virtual reality environments. background

[0003] Artificial reality is a form of reality that has been adjusted in some way before being presented to the user, which may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with captured content (e.g., photos of the real world). Artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of them may be presented in a single channel or multiple channels (e.g., stereoscopic video that produces a three-dimensional effect to the viewer). Artificial reality may be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality and / or to be used in artificial reality (e.g., to perform activities in artificial reality). Artificial reality systems that provide artificial reality content may be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a stand-alone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0004] Overview of Certain Embodiments

[0005] Certain embodiments described herein relate to a method of creating a system for modifying a VR environment using contextual information and user intent to create an isolated experience for the user. The system may first determine whether the user wants to focus on interacting with a particular object based on the user's movement, interaction with the object, and eye movement. Once the system determines that the user wants to interact with the particular object, the system may then modify the environment to maximize the user's experience with the particular object.

[0006] Certain embodiments described herein relate to creating a system for merging realities between various user locations to create a joint VR space using the free space of each user. The system may first determine that the free space of a particular user is large enough to accommodate the joint VR space. The system may then map out or retrieve a map of the free space of the particular user and coordinate with other users who also want to participate in the joint VR space by determining and creating a framework for the joint VR space that accommodates the free space constraints of each user and maximizes overlap between the free spaces of the users.

[0007] Certain embodiments described herein relate to synchronizing content and objects from real life with content and objects in a digital / VR environment to enhance a user's interaction, communication, and cooperation with other users (e.g., to collaborate on a project). The system may first determine what objects within the user's real life environment the user may want to use to collaborate with other users. The system may then replicate and present the real life objects in real time within the UI of the VR shared space so that other users in the VR shared space can view and interact with them.

[0008] Embodiments of the present invention may include an artificial reality system or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some way before being presented to a user, which may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with captured content (e.g., photos of the real world). Artificial reality content may include video, audio, tactile feedback, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that produces a three-dimensional effect to the viewer). In addition, in some embodiments, artificial reality may be associated with an application, product, accessory, service, or some combination thereof, which is used, for example, to create content in artificial reality and / or to be used in artificial reality (e.g., to perform activities in artificial reality). Artificial reality systems that provide artificial reality content can be implemented on various platforms, including a head-mounted display (HMD) connected to a host computer system, an independent HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0009] The embodiments disclosed herein are merely examples, and the scope of the present disclosure is not limited to them. Specific embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed herein. Embodiments according to the present invention are specifically disclosed in the attached claims relating to methods, storage media, systems, and computer program products, wherein any feature mentioned in one claim category (e.g., method) may also be claimed in another claim category (e.g., system). Dependencies or back references in the attached claims are selected only for formal reasons. However, any subject matter generated by intentional back references to any previous claim (particularly multiple dependencies) may also be claimed, so that any combination of claims and their features is disclosed and may be claimed, regardless of the dependencies selected in the attached claims. The subject matter that may be claimed includes not only the combination of features as set forth in the attached claims, but also any other combination of features in the claims, wherein each feature mentioned in the claims may be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein may be claimed in a separate claim and / or in any combination with any embodiment or feature described or depicted herein or in any combination with any features of the appended claims.

[0010] In an embodiment, a method may include, by a computing system:

[0011] Displaying the first virtual content to the first user in a virtual area, the virtual area including one or more second virtual contents;

[0012] Inferring the first user's intent to interact with the first virtual content based on one or more of the first user action or the context information; and

[0013] Based on the inference of the first user's intent to interact with the first virtual content, one or more configurations associated with one or more of the second virtual content are adjusted.

[0014] The first user action may include one or more of the following:

[0015] The user's eye movement focused on the first virtual content,

[0016] A verbal request from the first user,

[0017] a user input associated with the first virtual content, or

[0018] A user input associated with one or more of the second virtual contents.

[0019] The context information may include one or more of the following:

[0020] location information associated with the first user,

[0021] movement information associated with the first user,

[0022] time information associated with the first user,

[0023] a preset action associated with the first virtual content;

[0024] a content type associated with the first virtual content, or

[0025] A service type associated with the first virtual content.

[0026] The time information associated with the first user may include a predetermined time period of user inactivity.

[0027] Inferring the first user's intent may be based at least in part on the perspective of a hypothetical user, which may be based at least in part on one or more users of an associated social network.

[0028] It is assumed that users can base their decisions, at least in part, on:

[0029] Each user of a social network, or

[0030] One or more subsets of the users of a social network.

[0031] Adjusting one or more configurations associated with the second virtual content may include one or more of the following:

[0032] adjusting one or more visual attributes of one or more of the second virtual contents,

[0033] adjusting one or more audio properties of one or more of the second virtual content, or

[0034] One or more social network attributes of one or more of the second virtual content are adjusted.

[0035] The adjustment of the visual attribute or the audio attribute of one or more of the second virtual content may be determined based at least in part on a content type associated with the second virtual content or a service type associated with the second virtual content.

[0036] Adjusting the social network attributes may include temporarily limiting or removing all notifications from the social network associated with the second virtual content.

[0037] The virtual area may reside in a virtual reality environment, and the first user may be a virtual user in the virtual reality environment.

[0038] In an embodiment, one or more computer-readable non-transitory storage media may embody software that, when executed, is operable to perform a method according to any of the embodiments herein, or is operable to:

[0039] displaying first virtual content to a first user in a virtual area, wherein the virtual area may include one or more second virtual content;

[0040] Inferring the first user's intent to interact with the first virtual content based on one or more of the first user action or the context information; and

[0041] Based on the inference of the first user's intent to interact with the first virtual content, one or more configurations associated with one or more of the second virtual content are adjusted.

[0042] The first user action may include one or more of the following:

[0043] The user's eye movement focused on the first virtual content,

[0044] A verbal request from the first user,

[0045] a user input associated with the first virtual content, or

[0046] A user input associated with one or more of the second virtual contents.

[0047] The context information may include one or more of the following:

[0048] location information associated with the first user,

[0049] movement information associated with the first user,

[0050] time information associated with the first user,

[0051] one or more preset actions associated with the first virtual content;

[0052] a content type associated with the first virtual content, or

[0053] A service type associated with the first virtual content.

[0054] Inferring the first user's intent may be based at least in part on the perspective of a hypothetical user, which may be based at least in part on one or more users of an associated social network.

[0055] Adjusting one or more configurations associated with the second virtual content may include one or more of the following:

[0056] adjusting one or more visual attributes of one or more of the second virtual contents,

[0057] adjusting one or more audio properties of one or more of the second virtual content, or

[0058] One or more social network attributes of one or more of the second virtual content are adjusted.

[0059] In an embodiment, a system may include: one or more processors; and a memory coupled to the processor, the memory including instructions executable by the processor, when executing the instructions, the processor is operable to perform a method according to any embodiment of the present invention or is operable to:

[0060] Displaying the first virtual content to the first user in a virtual area, the virtual area including one or more second virtual contents;

[0061] Inferring the first user's intent to interact with the first virtual content based on one or more of the first user action or the context information; and

[0062] Based on the inference of the first user's intent to interact with the first virtual content, one or more configurations associated with one or more of the second virtual content are adjusted.

[0063] The first user action may include one or more of the following:

[0064] The user's eye movement focused on the first virtual content,

[0065] A verbal request from the first user,

[0066] a user input associated with the first virtual content, or

[0067] A user input associated with one or more of the second virtual contents.

[0068] The context information may include one or more of the following:

[0069] location information associated with the first user,

[0070] movement information associated with the first user,

[0071] time information associated with the first user,

[0072] one or more preset actions associated with the first virtual content;

[0073] a content type associated with the first virtual content, or

[0074] A service type associated with the first virtual content.

[0075] Inferring the first user's intent may be based at least in part on the perspective of a hypothetical user, which may be based at least in part on one or more users of an associated social network.

[0076] Adjusting one or more configurations associated with the second virtual content may include one or more of the following:

[0077] adjusting one or more visual attributes of one or more of the second virtual contents,

[0078] adjusting one or more audio properties of one or more of the second virtual content, or

[0079] One or more social network attributes of one or more of the second virtual content are adjusted.

[0080] In an embodiment, a method, in particular a method according to any embodiment herein, may include, by a computing system:

[0081] receiving a request from a first user to create a joint virtual space for use with one or more second users;

[0082] determining a first area in a first room based at least in part on a spatial constraint associated with the first room and a location of one or more items in the first room, the first room being associated with a first user;

[0083] retrieving information associated with one or more second rooms for each of the second users;

[0084] creating a joint virtual space based on the first area of ​​the first room and the information associated with each of the second rooms; and

[0085] Access to the joint virtual space is provided to each of the first user and the one or more second users.

[0086] In an embodiment, a method may include, prior to retrieving information associated with one or more second rooms:

[0087] A determination is made as to whether a first area in a first room is equal to or greater than a predetermined minimum area.

[0088] The first area may be determined by calculating a maximum free space associated with the first room after evaluating the spatial constraints and the location of the one or more items in the first room.

[0089] The retrieved information associated with the second room may include at least:

[0090] a spatial constraint associated with each of the second rooms for each of the second users, and

[0091] The location of the one or more items in each of the second rooms for each of the second users.

[0092] In an embodiment, a method may include:

[0093] determining a second area for each of the one or more second rooms based at least in part on the spatial constraint and the location of the one or more items,

[0094] Therein, the second area is determined by calculating a maximum free space associated with each of the one or more second rooms after evaluating the spatial constraints and the location of the one or more items in each of the one or more second rooms.

[0095] The joint virtual space may be created by determining a maximum overlap between a maximum free space associated with the first room and a maximum free space associated with each of the one or more second rooms.

[0096] Providing access to the joint virtual space may include notifying each of the first user and the one or more second users that the joint virtual space is available for use.

[0097] Notifying each of the first user and the one or more second users may include sending an instruction to generate a portal object to each of the first user and the one or more second users, the portal object allowing each of the first user and the one or more second users to virtually access the joint virtual space.

[0098] Generating a portal object may include:

[0099] sending an instruction to the first user to draw a virtual doorway within a first area in the first room, the virtual doorway allowing the first user to virtually access the joint virtual space; and

[0100] Instructions are sent to each of the second users to draw a virtual doorway in each of the second rooms that allows each of the second users to virtually access the joint virtual space.

[0101] The joint virtual space may reside in a virtual reality environment, and each of the first user and the one or more second users may be a virtual user in the virtual reality environment.

[0102] In an embodiment, one or more computer-readable non-transitory storage media, in particular a medium according to any embodiment herein, may embody software which, when executed, is operable to perform a method according to any embodiment herein or is operable to:

[0103] receiving a request from a first user to create a joint virtual space for use with one or more second users;

[0104] determining a first area in a first room based at least in part on a spatial constraint associated with the first room and a location of one or more items in the first room, the first room being associated with a first user;

[0105] retrieving information associated with one or more second rooms for each of the second users;

[0106] creating a joint virtual space based on the first area of ​​the first room and the information associated with each of the second rooms; and

[0107] Access to the joint virtual space is provided to each of the first user and the one or more second users.

[0108] The first area may be determined by calculating a maximum free space associated with the first room after evaluating the spatial constraints and the location of the one or more items in the first room.

[0109] The retrieved information associated with the second room may include at least:

[0110] a spatial constraint associated with each of the second rooms for each of the second users, and

[0111] The location of the one or more items in each of the second rooms for each of the second users.

[0112] In an embodiment, a medium may include:

[0113] determining a second area for each of the one or more second rooms based at least in part on the spatial constraint and the location of the one or more items,

[0114] Therein, the second area is determined by calculating a maximum free space associated with each of the one or more second rooms after evaluating the spatial constraints and the location of the one or more items in each of the one or more second rooms.

[0115] The joint virtual space may be created by determining a maximum overlap between a maximum free space associated with the first room and a maximum free space associated with each of the one or more second rooms.

[0116] In an embodiment, a system, in particular a system according to any embodiment herein, may include: one or more processors; and a memory coupled to the processor, the memory including instructions executable by the processor, when executing the instructions, the processor is operable to perform a method according to any embodiment herein or is operable to:

[0117] receiving a request from a first user to create a joint virtual space for use with one or more second users;

[0118] determining a first area in a first room based at least in part on a spatial constraint associated with the first room and a location of one or more items in the first room, the first room being associated with a first user;

[0119] retrieving information associated with one or more second rooms for each of the second users;

[0120] creating a joint virtual space based on the first area of ​​the first room and the information associated with each of the second rooms; and

[0121] Access to the joint virtual space is provided to each of the first user and the one or more second users.

[0122] The first area may be determined by calculating a maximum free space associated with the first room after evaluating the spatial constraints and the location of the one or more items in the first room.

[0123] The retrieved information associated with the second room may include at least:

[0124] a spatial constraint associated with each of the second rooms for each of the second users, and

[0125] The location of the one or more items in each of the second rooms for each of the second users.

[0126] In an embodiment, a system may include:

[0127] determining a second area for each of the one or more second rooms based at least in part on the spatial constraint and the location of the one or more items,

[0128] Therein, the second area is determined by calculating a maximum free space associated with each of the one or more second rooms after evaluating the spatial constraints and the location of the one or more items in each of the one or more second rooms.

[0129] The joint virtual space may be created by determining a maximum overlap between a maximum free space associated with the first room and a maximum free space associated with each of the one or more second rooms.

[0130] In an embodiment, a method, in particular a method according to any embodiment herein, may include, by a computing system:

[0131] receiving a request to share a display of a first interactive item with one or more users;

[0132] generating the first virtual item as a copy of the first interactive item; and

[0133] displaying a first virtual item to a subset of the one or more users in a virtual reality environment,

[0134] Wherein, if a change made to the first interactive item is received, the display of the first virtual item in the virtual reality environment is updated to include the same change as the first interactive item.

[0135] The request to share the display of the first interactive item may come from a first user of the one or more users who is currently interacting with the first interactive item.

[0136] The request to share the display of the first interactive item may come from one or more second users that are virtual users associated with the virtual reality environment.

[0137] In an embodiment, a method may include, prior to receiving a request to share display of a first interactive item,

[0138] identifying one or more interactive items in the environment; and

[0139] A list of interactive items is sent to a first user for selection of at least one of the interactive items for display in a virtual reality environment.

[0140] The one or more subsets of users may include virtual users in a virtual reality environment.

[0141] The first interactive item may be located in the real world environment.

[0142] In an embodiment, a method may include:

[0143] accessing a position of the first interactive item relative to one or more other items in the real world environment surrounding the first interactive item;

[0144] generating one or more second virtual items as copies of one or more other items; and

[0145] A first virtual object and one or more second virtual objects are displayed in the virtual reality environment based on a position of the first interactive object relative to one or more other objects in the real world environment.

[0146] In an embodiment, a method may include:

[0147] accessing an orientation of the first interactive item in the real world environment; and

[0148] A first virtual object is displayed in the virtual reality environment based on an orientation of the first interactive object in the real world environment.

[0149] In an embodiment, a method may include:

[0150] receiving a review associated with the first virtual item from one or more users in the subset of users; and

[0151] The comment to be displayed is sent to a first user of the one or more users who is currently interacting with the first interactive item in the real-world environment.

[0152] The comments may include one or more of audio comments, video comments, or written comments.

[0153] In an embodiment, one or more computer-readable non-transitory storage media, in particular a medium according to any embodiment herein, may embody software which, when executed, is operable to perform a method according to any embodiment herein or is operable to:

[0154] receiving a request to share a display of a first interactive item with one or more users;

[0155] generating the first virtual item as a copy of the first interactive item; and

[0156] displaying a first virtual item to a subset of the one or more users in a virtual reality environment;

[0157] Wherein, if a change made to the first interactive item is received, the display of the first virtual item in the virtual reality environment is updated to include the same change as the first interactive item.

[0158] When executed, the software may be operable to, prior to receiving a request to share display of a first interactive item:

[0159] identifying one or more interactive items in the environment; and

[0160] A list of interactive items is sent to a first user for selection of at least one of the interactive items for display in a virtual reality environment.

[0161] The first interactive item may be located in the real world environment.

[0162] The software, when executed, is operable to:

[0163] accessing a position of the first interactive item relative to one or more other items in the real world environment surrounding the first interactive item;

[0164] generating one or more second virtual items as copies of the one or more other items; and

[0165] A first virtual object and one or more second virtual objects are displayed in the virtual reality environment based on a position of the first interactive object relative to one or more other objects in the real world environment.

[0166] When executed, the software may be operable to:

[0167] accessing an orientation of the first interactive item in the real world environment; and

[0168] A first virtual object is displayed in the virtual reality environment based on an orientation of the first interactive object in the real world environment.

[0169] In an embodiment, a system, in particular a system according to any embodiment herein, may include: one or more processors; and a memory coupled to the processor, the memory including instructions executable by the processor, the processor being operable to perform a method according to any embodiment herein or operable to:

[0170] receiving a request to share a display of a first interactive item with one or more users;

[0171] generating the first virtual item as a copy of the first interactive item; and

[0172] displaying a first virtual item to a subset of the one or more users in a virtual reality environment;

[0173] Wherein, if a change made to the first interactive item is received, the display of the first virtual item in the virtual reality environment is updated to include the same change as the first interactive item.

[0174] When executing the instructions, the processor may be operable to, prior to receiving a request to share display of the first interactive item:

[0175] identifying one or more interactive items in the environment; and

[0176] A list of interactive items is sent to a first user for selection of at least one of the interactive items for display in a virtual reality environment.

[0177] The first interactive item may be located in the real world environment.

[0178] When executing the instructions, the processor may be operable to:

[0179] accessing a position of the first interactive item relative to one or more other items in the real world environment surrounding the first interactive item;

[0180] generating one or more second virtual items as copies of the one or more other items; and

[0181] A first virtual object and one or more second virtual objects are displayed in the virtual reality environment based on a position of the first interactive object relative to one or more other objects in the real world environment.

[0182] When executing the instructions, the processor may be operable to:

[0183] accessing an orientation of the first interactive item in the real world environment; and

[0184] A first virtual object is displayed in the virtual reality environment based on an orientation of the first interactive object in the real world environment.

[0185] In an embodiment, one or more computer-readable non-transitory storage media may embody software that, when executed, is operable to perform a method according to or within any of the embodiments described above.

[0186] In an embodiment, a system may include: one or more processors; and at least one memory coupled to the processor and including instructions executable by the processor, when executing the instructions, the processor is operable to perform a method according to or within any of the above embodiments.

[0187] In an embodiment, a computer program product, preferably comprising a computer readable non-transitory storage medium, when executed on a data processing system, may be operable to perform a method according to or in any of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0188] Figure 1 An example artificial reality system is shown.

[0189] Figure 2 An example 3D eye tracking system is shown.

[0190] Figure 3 An example artificial reality space is shown.

[0191] Figure 4An example VR environment is shown.

[0192] Figure 5 Another example of a VR environment is shown.

[0193] Figure 6 An example method for updating a VR environment based on user intent and contextual information is shown.

[0194] Figure 7 An example first user is shown specifying a space for merging with other users into a joint VR space.

[0195] Figure 8 An example second user specifies a space for merging with other users into a joint VR space is shown.

[0196] Fig. 9 An example merged VR environment is shown.

[0197] Fig. 10A and 10B An example of creating a portal object for accessing a merged VR environment is shown.

[0198] Fig.11 An example method for specifying a space for merging an artificial reality system with other users is shown.

[0199] Fig.12 An object designated by a first user for sharing with other users in an artificial reality system is shown.

[0200] Fig.13 An object designated by a second user for sharing with other users in the artificial reality system is shown.

[0201] Fig.14A and 14B Various environments are shown for various users to view shared objects.

[0202] Fig.15 Example methods for specifying various objects in an artificial reality system and sharing those objects with other users are shown.

[0203] Fig.16 An example network environment associated with a social networking system is shown.

[0204] Fig.17 An example social graph is shown.

[0205] Fig.18 An example computer system is shown.

[0206] Description of Example Embodiments

[0207] Artificial Reality Overview

[0208] Figure 1 An example artificial reality system 100 is shown. In certain embodiments, the artificial reality system 100 may include a headset 104, a controller 106, and a computing system 108. A user 102 may wear the headset 104, which may display visual artificial reality content to the user 102. The headset 104 may include an audio device that may provide audio artificial reality content to the user 102. The headset 104 may include one or more cameras that may capture images and video of the environment. The headset 104 may include an eye tracking system for determining the convergence distance of the user 102. The convergence distance may be the distance from the user's eyes to an object (e.g., a real-world object or a virtual object in a virtual space) that the user's eyes converge on. The headset 104 may be referred to as a head-mounted display (HMD). The controller 106 may include a touchpad and one or more buttons. The controller 106 may receive input from the user 102 and relay the input to the computing system 108. The controller 206 may also provide tactile feedback to the user 102. The computing system 108 may be connected to the head mounted device 104 and the controller 106 via a cable or wireless connection. The computing system 108 may control the head mounted device 104 and the controller 106 to provide artificial reality content to the user 102 and receive input from the user 102. The computing system 108 may be a stand-alone host computer system, an on-board computer system integrated with the head mounted device 104, a mobile device, or any other hardware platform capable of providing artificial reality content to the user 102 and receiving input from the user 102.

[0209] In certain embodiments, the artificial reality system may include an eye tracking system for tracking the user's eyes in real time. The eye tracking system may be a 3D eye tracking system that tracks the user's eye movements (e.g., gaze direction, gaze angle, convergence) and determines where the user is looking (e.g., vergence distance or gaze point). Figure 2An example 3D eye tracking system 200 is shown. The 3D eye tracking system 200 can track eye movements in three dimensions to determine a user's convergence distance or gaze point. The user's convergence distance can be the distance from the user's eyes to the point where the user's eyes converge. The user's gaze point can be the point at which the user is gazing. The eye tracking system 200 can include a lens 210, a plurality of infrared light sources (e.g., 212A-H), a hot mirror 220, and an infrared camera 240. The light sources 212A-H can be infrared light emitting diodes (LEDs) mounted on the lens 210. The hot mirror 220 can be a dichroic filter that reflects infrared light while allowing visible light to pass. Infrared light (e.g., 214) emitted by one or more light sources 212A-H can reach an eye 250 and be reflected away from the eye 250. Reflected light 216 can be further reflected by the hot mirror 220 and reach an infrared camera 240. The camera 240 can be an infrared camera that captures an image of the eye 250 using the reflected infrared light. The eye tracking system 200 can capture images of the user's two eyes (e.g., pupils) and process the images using computer vision techniques. The eye tracking system 200 can measure the angle of the two eyes and use geometric relationships to determine the user's convergence distance and gaze point. For example, the 3D eye tracking system 200 can measure the user's eye angle with an accuracy of 1 degree. Visible light 232 from the display screen 230 can reach the eye 250 through the hot mirror 220 and the lens 210, allowing the user to see the content rendered by the display screen 230. In a specific embodiment, the 3D eye tracking system 200 can use ambient light 260 from the environment to capture eye images. Ambient light 240 can reach the eye 250 and can be reflected away from the eye 250. The reflected light can pass through the lens 210 and reach the hot mirror 220 and the camera 240. The camera 240 can capture an image of the eye 250 based on the ambient light reflected away from the eye 250. In certain embodiments, the 3D eye tracking system may use a hybrid approach utilizing light sources (eg, 212A- 212H) and ambient light 260 to capture eye images and track eye movements.

[0210] In certain embodiments, 308 may use a machine learning (ML) based approach for eye tracking. The head mounted device system may capture a sequence of images of the eyes of the user wearing the head mounted device (e.g., using a camera of a 3D eye tracking system), and use a machine learning (ML) algorithm to process the images and output vergence information. For example, the machine learning (ML) algorithm may include an inference model to determine the vergence distance and gaze point of the user. In certain embodiments, the head mounted device system may include a hybrid approach combining 3D eye tracking and ML based eye tracking.

[0211] In certain embodiments, the artificial reality system may use a combination of methods to determine the user's vergence distance and gaze point. These methods may include, for example, but not limited to, eye-tracking-based methods (e.g., 3D eye tracking, ML-based eye tracking), body-based methods (e.g., head position / movement, hand position / movement, body position / movement), and content-based methods (e.g., Z-buffer, facial / object recognition, developer-provided information). U.S. Patent Application No. 16 / 132,153, entitled “Vergence Determination,” filed on September 14, 2018, discloses examples of determining vergence distance or gaze point using different combinations of methods, which application is incorporated by reference as an example and not by limitation.

[0212] Figure 3 An example artificial reality space 300 is shown. In certain embodiments, the artificial reality space 300 may include a first user 302, a second user 304, and a third user 306. In certain embodiments, the artificial reality space 300 may include a virtual reality scene rendered by a head mounted device in a virtual space and in the field of view of each of the first user 302, the second user 304, and the third user 306 wearing head mounted devices 308, 310, and 312, respectively. As discussed in more detail below, the first user 302, the second user 304, and the third user 306 may access the artificial reality space 300 to meet and collaborate with each other in addition to interacting with one or more objects or items located in the artificial reality space 300.

[0213] trigger

[0214] In certain embodiments, the VR environment may be changed or updated based on analysis of the user's movement in the VR environment and / or interaction with one or more items in the VR environment to better suit the user's needs or provide a better user experience. The certain embodiments described herein are directed to using contextual information and the user's intent to modify the VR environment to create an artificial reality for an isolated experience for the user. As described in more detail below, in addition to various social network information (discussed below), the system may first determine whether the user wants to focus on interacting with a particular object based on the user's movement, interaction with the object, and eye movement. Once the system determines that the user wants to interact with a particular object, the system may then modify the environment to maximize the user's experience with that particular object.

[0215] In particular embodiments, the system may infer a user's intent to focus on interacting with a particular object based on the user's movement or position. Figure 4 An example VR environment 400 is shown. Figure 4As shown, VR environment 400 includes screen 410, speaker 412, lamp 414, alarm clock 416, radio 418 and sofa 420. In addition, Figure 5 Another example of a VR environment 500 is shown. Figure 5 As shown, VR environment 500 includes sofa 510, bookshelf 512, coffee table 514, chandelier 516, floor lamp 518, window covered by curtain 520, and radio 522. In certain embodiments, the artificial reality system can infer the user's intention based on contextual information and one or more "triggers" in the VR environment, which can include determining that the user's movement and / or position is within a predetermined distance or position relative to one or more objects in the VR environment (e.g., virtual screen, bookshelf). These triggers can include instructions associated with one or more objects in the VR environment, which are triggered when specific conditions associated with the user are met (e.g., the user is within a predetermined position or distance to one or more objects, the user moves with respect to one or more objects, etc.).

[0216] As an example, Figure 4 As shown, if the artificial reality system determines that the user wants to watch the program on screen 410 in the VR environment based on the user's movement toward a position close to the position of screen 410 (e.g., virtual TV screen, virtual projection screen, etc.) in the VR environment (e.g., by sitting in a sofa 420 in front of screen 410, by standing next to screen 410, etc.), the system can determine that the user wants to focus their attention on watching the program on screen 410 and minimize all other distractions. In this way, the system can modify the VR environment around the user to create an isolated experience for the user, thereby maximizing the user's experience. As an example and not limitation, the system can determine that in order to isolate the user's experience of watching the program on screen 410, the system can remove any and all distractions by dimming or turning off lights 414. In addition, the system can remove objects that become distractions, such as turning off alarm clocks 416, radios 418, and other related objects. In addition, the system can temporarily limit interactions with other users (e.g., users of social networks, as discussed in more detail below).

[0217] As another example, Figure 5As shown, if the artificial reality system determines that the user wants to read a book from bookshelf 512 in the VR environment based on the user's movement toward a position close to the position of bookshelf 512 in the VR environment (e.g., by standing in front of bookshelf 512, by sitting on sofa 510, etc.), the system can determine that the user wants to focus their attention on reading books from bookshelf 512 and minimize all other distractions. In this way, the system can modify the VR environment around the user to create an isolated experience for the user, thereby maximizing the experience. As an example and not limitation, the system can remove any and all distractions by brightening the ceiling light 516 and also dimming the floor lamp 518 to reduce unnecessary light in the room. In addition, the system can turn on the radio 522 to provide quiet music. In addition, the system can remove objects that become distractions and temporarily limit interactions with other users.

[0218] In certain embodiments, the artificial reality system may infer the user's intent to focus on interacting with a particular object based on the user's interaction with the object. In certain embodiments, the system may infer the user's intent based on contextual information and one or more triggers in the VR environment, including determining the user's level of interaction with one or more objects in the VR environment (e.g., a virtual screen, a bookshelf). These triggers may include instructions associated with one or more objects in the VR environment that are triggered when specific conditions associated with the user's interaction with the one or more objects are met. As an example, Figure 4 As shown, if the artificial reality system determines that the user wants to watch the program on screen 410 in the VR environment based on the user's interaction with screen 410 in the VR environment (e.g., by using a controller to control the program on screen 410, by verbally requesting screen 410 to turn on or play a program, etc.) or interaction with sofa 420 (e.g., by sitting in sofa 420 in front of screen 410), the system can determine that the user wants to focus their attention on watching the program on screen 410 and minimize all other distractions (e.g., using methods of adjusting visual attributes, audio attributes, and / or social network attributes as discussed above). As another example, if Figure 5 As shown, if the artificial reality system determines that the user wants to read a book from bookshelf 512 in the VR environment based on the user's interaction with books on bookshelf 512 in the VR environment (e.g., by removing books from bookshelf 512) or interaction with sofa 510 (e.g., by sitting on sofa 510), the system can determine that the user wants to focus their attention on reading the book from bookshelf 512 and minimize all other distractions (e.g., using methods as discussed above).

[0219] In certain embodiments, the artificial reality system may be based on tracking the user's eye movements (e.g., using the method described above with respect to Figure 2 The method described in the description) can be used to infer the user's intention to focus on interacting with a specific object. In certain embodiments, the system can infer the user's intention based on contextual information and one or more triggers in the VR environment, including determining the user's eye movements related to one or more objects in the VR environment (e.g., a virtual screen, a bookshelf). These triggers can include instructions associated with one or more objects in the VR environment, and these instructions are triggered when specific conditions associated with tracking the user's eye movements are met. In addition, the inference of the user's intention can be determined based on the tracking of the user's eye movements combined with a time component (e.g., a predetermined time period of inaction of the user). As an example, if Figure 4 As shown, if the artificial reality system tracks the user's eye movements and determines that the user is looking at screen 410 in the VR environment for a predetermined amount of time (e.g., by determining that the user's eyes are focused on point 422 on screen 410), the system can determine that the user wants to focus their attention on watching the program on screen 410 and minimize all other distractions (e.g., using methods of adjusting visual attributes, audio attributes, and / or social network attributes as discussed above). This determination can be made by further determining that a predetermined period of user inaction has passed. As another example, if Figure 5 As shown, if the artificial reality system tracks the user's eye movement and determines that the user wants to read a book from bookshelf 512 in the VR environment (e.g., by determining that the user's eyes are focused on point 524 on bookshelf 512), the system can determine that the user wants to focus their attention on reading the book from bookshelf 512 and minimize all other distractions (e.g., using methods as discussed above). For both examples, this determination can be made by further accessing time information associated with the user (e.g., determining that a predetermined period of inaction has passed).

[0220] In certain embodiments, modifications to the VR environment may be encoded into or associated with various objects based on the type of object that the user wants to interact with (e.g., screen 410, books from bookshelf 512, etc.), based on the type of activity the user wants to engage in (e.g., watching a show, reading a book, etc.), or based on the type of service associated with a particular object (e.g., screen 410 is associated with a video application, books on bookshelf 512 are associated with a reading application). Furthermore, modifications to the VR environment may be user-specified or user-input, or may be based on preset settings (e.g., factory settings) that may be changed or updated by the user.

[0221] In certain embodiments, the artificial reality system may infer the user's intent to focus on interacting with a particular object based on one or more information items stored by or available to the social networking system (discussed in more detail below). Examples of information items stored by the social networking system may include social graph information associated with a target user (i.e., a user interacting with one or more objects). Examples of information items available to the social networking system may include information items accessible by the social networking system and stored on one or more client systems, one or more third-party systems, one or more networks, or any combination thereof. In certain embodiments, the information items from which the intent may be inferred may include social graph information (e.g., nodes and edges, affinity and separation), content objects, posts, text data, location information, media, user profile information, time information, and privacy settings. In certain embodiments, one or more information items may belong to multiple categories. For example, one or more information items may be classified as social graph information, posts, and media. Alternatively, in certain embodiments, one or more information items may belong to only one category.

[0222] In certain embodiments, the artificial reality system may infer the user's intent to focus on interacting with a particular object based on one or more perspectives of one or more users of a social network. For example, the inferred intent may be based on the perspective of a hypothetical user, which is based on one or more users of the social network. In certain embodiments, the hypothetical user may be based on each user of the social network. By way of example and not limitation, by standing in front of a user, the user may be able to focus on interacting with a particular object. Figure 4 By being near, interacting with, and / or looking at screen 410 (e.g., determined based on tracking eye movements), it can be inferred that the hypothetical user of each user based on the social network will have an intention to watch the program. As another example, by interacting with or looking at books on bookshelf 524 or sitting in sofa 510 next to bookshelf 524, it can be inferred that the hypothetical user of each user based on the social network will have an intention to read the book, such as Figure 5 shown.

[0223] As another example, in certain embodiments, the inferred intent may be based on the perspective of a hypothetical user based on a subset of users of a social network. In certain embodiments, the subset of users may be determined by any suitable means, including but not limited to one or more numerical limits, one or more time limits, one or more location-based limits, one or more degrees of separation, one or more affiliation coefficients between the target user and the users constituting the subset of users, one or more commonalities between the target user and the users constituting the subset of users, or any combination thereof. In certain embodiments, commonalities may include any features or characteristics shared between the target user and the users constituting the subset of users, including but not limited to location, age, religion or religious beliefs, education, political affiliation or political beliefs, or common interests (e.g., interest in food, books, movies, or music). For example, it may be inferred that a hypothetical user who shares common interests with the target user, a sample set of one hundred users within two degrees of separation of the target user, and who has been standing nearby, interacting with screen 410, and / or watching (e.g., determined based on tracking eye movement) screen 410 will have the intent to watch the program on screen 410. As another example, it can be inferred that a hypothetical user based on a sample set of twenty-five users who are within three years of the target user, within one degree of separation of the target user, who are currently reading books of a similar type as the target user, and who have been interacting with or looking at books on bookshelf 524 or sitting in sofa 510 next to bookshelf 524 will have an intent to read the books on bookshelf 524.

[0224] Figure 6An example method 600 for updating a VR environment based on a user's intent and contextual information is shown. The method may begin at step 610, where an artificial reality system may display first virtual content to a first user in a virtual area that includes one or more second virtual content. At step 620, the artificial reality system may infer the first user's intent to interact with the first virtual content based on one or more of the first user action or contextual information. At step 630, the artificial reality system may adjust one or more configurations associated with one or more of the second virtual content based on the inference of the first user's intent to interact with the first virtual content. In a particular embodiment, the first user action may include one or more of a user eye movement focused on the first virtual content, a verbal request of the first user, a user input associated with the first virtual content, or a user input associated with one or more of the second virtual content. In a particular embodiment, the contextual information may include one or more of location information associated with the first user, movement information associated with the first user, time information associated with the first user, a preset action associated with the first virtual content, a content type associated with the first virtual content, or a service type associated with the first virtual content. As an example, the time information associated with the first user may include a predetermined time period of inaction by the user.

[0225] In certain embodiments, inferring the intent of the first user may be based at least in part on the perspective of a hypothetical user, which is based at least in part on one or more users of an associated social network. As an example, the hypothetical user may be based at least in part on each user of the social network, or one or more subsets of the users of the social network. In certain embodiments, adjusting one or more configurations associated with the second virtual content includes adjusting one or more visual attributes of one or more of the second virtual content, adjusting one or more audio attributes of one or more of the second virtual content, or adjusting one or more of one or more social network attributes of one or more of the second virtual content. As an example, the adjustment of one or more visual attributes or audio attributes of the second virtual content may be determined based at least in part on the content type associated with the second virtual content or the service type associated with the second virtual content. As another example, the adjustment of the social network attribute may include temporarily limiting or removing all notifications associated with the second virtual content from the social network. In certain embodiments, the virtual area may reside in a virtual reality environment, and the first user may be a virtual user in the virtual reality environment

[0226] Specific examples may be repeated where appropriate. Figure 6 Although the present disclosure will Figure 6The specific steps of the method are described and shown as occurring in a particular order, but the present disclosure contemplates Figure 6 Any suitable steps of the method may occur in any suitable order. In addition, although the present disclosure describes and illustrates the method including Figure 6 The present disclosure contemplates any suitable method for updating a VR environment based on a user's intent and contextual information including any suitable steps, and any suitable steps may include, where appropriate, Figure 6 In addition, although the present disclosure describes and illustrates the implementation of Figure 6 specific components, devices or systems for specific steps of the method, but the present disclosure contemplates performing Figure 6 Any suitable combination of any suitable components, devices, or systems of any suitable steps of the method.

[0227] Portal

[0228] In certain embodiments, one or more users may want to create a joint VR space that users can access and use to interact with each other. Certain embodiments described herein are directed to creating an artificial reality system for merging realities between the locations of various users to create a joint VR space using the free space of each user. The system may first determine that the free space of a particular user is large enough to accommodate the joint VR space. The system may then draw out or retrieve a map of the free space of the particular user, and coordinate with other users who also want to participate in the joint VR space by determining and creating a framework of the joint VR space that accommodates the free space constraints of each user and maximizes the overlap between the free spaces of the users. In certain embodiments, once the joint VR space is created, the system will allow users to participate in the joint VR space by, for example, drawing a doorway (e.g., a "portal") that can be used to enter the joint VR space. Once users enter the joint VR space, they can interact and collaborate with other users in the space. In certain embodiments, a guard box may be used to limit the movement of users of the joint VR space by blocking off areas that one or more users may not be able to access because they are beyond the users' available free space.

[0229] In certain embodiments, to generate a joint VR space, the artificial reality system may first determine whether the user has enough free space to accommodate the joint VR space. As an example, the artificial reality system may determine whether the user has a minimum amount of free space by requesting the user to scan the space using a system controller (e.g., controller 106) to take measurements. Figure 7An example first user specifies a space 700 for merging with other users into a joint VR space. Figure 7 As shown, space 700 includes various items that can block free space in room 710, including table 712, chair 714, bed 716, and wardrobe 718. In the middle of room 710 is an empty area that can be used to create a joint VR space. User 720 can use controller 722 (e.g., controller 106) to scan room 710 to measure the empty area. As an example, user 720 can use controller 722 to scan room 710 by using a straight line (e.g., as shown in FIG. 1 ). Figure 7 As another example, the artificial reality system may request a measurement from the user that may be manually input by the user. By accessing the measurement obtained by the controller or input by the user, the artificial reality system may calculate the area of ​​the free space in the room 710. If the system determines that the area is greater than a predetermined minimum area, the system will allow the user to continue to designate the area for the joint VR space. On the other hand, if the system determines that the area is insufficient because it is less than the predetermined minimum area required to participate in the joint VR space, the system will output an error message to the user and notify the user that a larger area is required.

[0230] In certain embodiments, once the system determines that the area is greater than a predetermined minimum area required to participate in the joint VR space, the system may require user 720 to scan room 710 using controller 722 to map out the empty areas in the room. Figure 7 As shown, user 720 can use controller 722 to specify area 724 (as shown by the dashed line) as the area to be used for the joint VR space. As an example, the system can ask user 720 to specify the largest possible area that can be used for the joint VR space so that the maximum amount of space can be used to determine the overlap between the spaces of the respective users. As another example, the system can ask user 720 to specify only the area that the user wants the joint VR space to cover, while excluding other areas from the evaluation (e.g., corridor areas, walkway areas, etc.). In certain embodiments, such a scan of room 710 may only be required when user 720 first requests to participate in the joint VR space. Once the system maps out the empty areas in the room, the system can store this information for future use. In certain embodiments, before asking user 720 to map out the empty areas in the room, the system can first check the stored information to find a previously stored room map. If the system finds a previously stored room map, the system can first ask user 720 whether he / she wants to use the previously stored room map, and / or before taking further measurements, the system can ask user 720 whether the room configuration has changed since the time the previously stored room map was stored.

[0231] In a particular embodiment, after the artificial reality system determines that user 720 wants to create a joint VR space with one or more other users and determines that user 720 has enough free space in room 710, the artificial reality system can then determine one or more other users to participate in the joint VR space. As an example, the artificial reality system can ask user 720: which other users user 720 wants to invite to participate in the joint VR space. As another example, the artificial reality system can send a list of other users to user 720 for selection. As yet another example, the artificial reality system can maintain a list of other users who have expressed interest in joining the joint VR space to interact with other users. The other users can be users from the user's social network as determined by social graph information (e.g., nodes and edges, affinities, and separations), as discussed in more detail below. In particular embodiments, other users may be determined by any suitable means, including, but not limited to, one or more time constraints (e.g., other users using the artificial reality system during the same time period as user 720), one or more location-based constraints (e.g., other users within a geographic distance or within a particular geographic area), one or more degrees of separation, one or more affiliation coefficients between user 720 and the other users, one or more commonalities between user 720 and the other users, or any combination thereof.

[0232] In certain embodiments, after the artificial reality system determines one or more other users participating in the joint VR space, the system can retrieve information about the spatial limitations of each other user's free space. For example, the system can request and receive spatial limitation information from each other user. If the spatial limitation information is not available, the system can request the other user's system to determine the spatial limitation information (similar to the method described above). Figure 8 An example second user-specified space 800 is shown for merging with other users into a joint VR space. Figure 8 As shown, space 800 includes various items that may block free space in room 810, including a TV cabinet 812, a bookshelf 814, and a sofa 816. In the middle of room 810 is an empty area that may be used to create a joint VR space. User 818 may use controller 820 (e.g., controller 106) to scan room 810 to measure the empty area. As an example, user 818 may use controller 820 to measure the empty area by using a straight line (e.g., Figure 8 ), lines that follow the outline of the object, other suitable methods, or any combination thereof to outline the boundaries of the blank area, thereby mapping out the blank area. Once the system determines that the area is larger than the predetermined minimum area required to participate in the joint VR space, the system can ask the user 818 to scan the room 810 using the controller 820 to map out the blank area in the room. Figure 8 As shown, user 818 may use controller 820 to designate area 822 (shown as a dashed line) as the area to be used for the joint VR space.

[0233] Upon receiving the spatial constraint information of all other users who want to participate in the joint VR space, the artificial reality system can determine and create a framework of the joint VR space that accommodates the free space constraints of each user and maximizes the overlap between the users' free spaces. Fig. 9 An example merged VR environment 900 is shown. Fig. 9 As shown, a merged VR environment 900 is created by maximizing the overlap between the free spaces of user 720 and user 818. Fig. 9 As shown in solid lines in FIG. 7 , the free space of the room 710 of the user 720 is blocked by various items in the room 710, including a table 712, a chair 714, a bed 716, and a wardrobe 718. Fig. 9 As shown in FIG. 8 using light dashed lines, the free space of the room 810 of the user 818 is blocked by various items in the room 810, including a TV cabinet 812, a bookshelf 814, and a sofa 816. Fig. 9 As shown, room 710 has a larger area than room 810 and also has a larger free space area, so the combined VR environment 900 is limited by the free space area of ​​room 810. In determining the maximum overlap area 910 between rooms 710 and 810, the artificial reality system can determine the maximum overlap between the maximum free space associated with room 710 and the maximum free space associated with room 810. This maximum overlap area 910 is then used as a joint VR space in which users 720 and 818 can use together to interact with each other and with other users. In certain embodiments, the maximum overlap area used as a joint VR space can be a square, rectangle, circle, polygon, or other suitable area that maximizes the available free space.

[0234] In certain embodiments, the maximum overlap area used as the joint VR space may include certain areas that are blocked off so that the user does not run into or move close to certain areas in the user's room (e.g., blocking off the area around the stove even though the stove is located in the free space area). As an example, these areas may be blocked off using a protective frame displayed in the merged VR environment. The protective frame may be a visual cue that tells the user in the merged VR environment that the area is restricted from access.

[0235] In certain embodiments, once the joint VR space is created, the artificial reality system may notify users 720 and 818 that the joint VR space is available and accessible for interaction with each other. As an example, notifying users 720 and 818 may include sending instructions to users 720 and 818 to generate a portal object that allows users 720 and 818 to virtually access the joint virtual space. Fig. 10A and Fig. 10B An example of creating a portal object in space 1000 to access a merged VR environment is shown. In certain embodiments, generating a portal object may include the artificial reality system sending instructions to users 720 and 818 to draw a virtual doorway within an area in their respective rooms 710 and 810, which allows each of users 720 and 818 to virtually access the joint virtual space. Fig. 10A As shown, space 1000 includes various items that may block free space in room 1010, including a TV cabinet 1012, a bookshelf 1014, and a sofa 1016. In the middle of room 1010 is a blank area 1018 that is used to create a joint VR space. Once the artificial reality system notifies user 1020 that the joint VR space is available, the artificial reality system may send an instruction to generate a portal object to user 1020 in order to access the joint VR space. As an example, Fig. 10A As shown, the artificial reality system may send an instruction to the user 1020 to generate a portal object 1022 (e.g., a door) to access the joint VR space. The portal object 1022 may be generated by the user 1020 using a controller 1024 (e.g., controller 106) by outlining the portal object 1022, selecting the portal object 1022 from a selection list, or any other suitable means. Once the portal object 1022 is generated, as shown in FIG. Fig. 10B As shown, user 1020 may interact with portal object 1022 (e.g., open a door and walk through doorway 1026) in order to access the joint VR space.

[0236] Fig.11An example method for specifying a space for merging an artificial reality system with other users is shown. The method may begin at step 1110, where the artificial reality system may receive a request from a first user to create a joint virtual space for use with one or more second users. At step 1120, the artificial reality system may determine a first area in a first room associated with the first user based at least in part on a spatial constraint associated with the first room and a location of one or more items in the first room. At step 1130, the artificial reality system may retrieve information associated with one or more second rooms for each of the second users. At step 1140, the artificial reality system may create a joint virtual space based on the first area of ​​the first room and the information associated with each of the second rooms. At step 1150, the artificial reality system may provide access to the joint virtual space to each of the one or more second users and the first user.

[0237] In certain embodiments, the artificial reality system may determine whether a first area in the first room is equal to or greater than a predetermined minimum area before retrieving information associated with the one or more second rooms. In certain embodiments, the first area may be determined by calculating a maximum free space associated with the first room after evaluating the spatial constraints and the location of one or more items in the first room. The retrieved information associated with the second room may include at least a spatial constraint associated with each of the second rooms for each of the second users, and the location of one or more items in each of the second rooms for each of the second users. The artificial reality system may determine a second area for each of the one or more second rooms based at least in part on the spatial constraints and the location of the one or more items, wherein the second area is determined by calculating a maximum free space associated with each of the one or more second rooms after evaluating the spatial constraints and the location of the one or more items in each of the one or more second rooms. The joint virtual space may be created by determining a maximum overlap between the maximum free space associated with the first room and the maximum free space associated with each of the one or more second rooms.

[0238] In a particular embodiment, providing access to the joint virtual space includes notifying the first user and each of the one or more second users that the joint virtual space is available for use. Notifying each of the first user and the one or more second users may include sending an instruction to generate a portal object to each of the first user and the one or more second users, the portal object allowing each of the first user and the one or more second users to virtually access the joint virtual space. In a particular embodiment, generating the portal object may include sending an instruction to the first user to draw a virtual doorway in a first area in a first room that allows the first user to virtually access the joint virtual space, and sending an instruction to each of the second users to draw a virtual doorway in each of the second rooms that allows each of the second users to virtually access the joint virtual space. In a particular embodiment, the joint virtual space may reside in a virtual reality environment, and each of the first user and the one or more second users may be a virtual user in the virtual reality environment.

[0239] Where appropriate, specific examples may be repeated Fig.11 Although the present disclosure will Fig.11 The specific steps of the method are described and shown as occurring in a particular order, but the present disclosure contemplates Fig.11 Any suitable steps of the method may occur in any suitable order. In addition, although the present disclosure describes and illustrates the method including Fig.11 The example method for specifying a space for merging an artificial reality system with other users includes specific steps of the method, but the present disclosure contemplates any suitable method for specifying a space for merging an artificial reality system with other users including any suitable steps, where appropriate, any suitable steps may include Fig.11 In addition, although the present disclosure describes and illustrates the implementation of Fig.11 specific components, devices or systems for specific steps of the method, but the present disclosure contemplates performing Fig.11 Any suitable combination of any suitable components, devices, or systems of any suitable steps of the method.

[0240] Shared Space

[0241] In certain embodiments, one or more users may want to collaborate with each other using content that all users can view and dynamically interact with. Certain embodiments described herein are directed to an artificial reality system that synchronizes content and objects from real life with content and objects in a digital / VR environment to enhance a user's interaction, communication, and cooperation with other users (e.g., to collaborate on a project). The system can first determine what objects within the user's real-life environment a user may want to use to collaborate with other users. The system can then replicate and present the real-life objects in real time within the UI of the VR shared space so that other users in the VR shared space can view and interact with them.

[0242] In certain embodiments, the artificial reality system may first determine whether the user is interacting with an object that can be used to communicate and collaborate with other users. As an example, the system may detect that the object the user is interacting with can be viewed and shared with other users, or has been designated as an object that can be viewed and shared with other users. Alternatively, the user may request from the system to share the object with other users by presenting an image, video, real-time view, or any other suitable presentation of the object to the other users. Fig.12 4 shows an object specified by a first user for sharing with other users in an artificial reality system, such as Fig.12 As shown, user 1210 is using device 1214 to interact with independent screen 1212. As an example, screen 1212 can be a screen that is not connected to the virtual reality system (e.g., a blackboard, a whiteboard, etc.). As another example, screen 1212 can alternatively be a screen connected to a virtual reality system, in which any information written on the screen will be copied, saved, and accessible via the system. In addition, as an example, device 1214 can be a physical device (e.g., a writing device, a pointer device, etc.) that allows user 1210 to interact with screen 1212 in the real world. As another example, device 1214 can be an electronic device (e.g., an electronic pointer device) that allows user 1210 to interact with screen 1212 in the real world or an electronic device (e.g., controller 106) that allows user 1210 to interact with screen 1212 in the virtual world.

[0243] In a specific embodiment, if Fig.12As shown, in the area around the user 1210, there are also bookshelves 1216, beds 1218, and desks 1220. Among these objects in the room, the artificial reality system can determine that bookshelves 1216 and beds 1218 are not objects that will be designated as objects for sharing information with other users in the artificial reality system, while desks 1220 can be designated as objects for sharing information with other users in the artificial reality system. In addition, the user 1210 can designate screen 1212 as an object for sharing information with other users. In a particular embodiment, the artificial reality system can access the environment around the user 1210 to determine the position, location, and / or orientation of bookshelves 1216, beds 1218, and desks 1220, and include these objects in the virtual reality display together with screen 1212. This can provide the user with a more realistic scene and context about the location of screen 1212, so that the user is not just watching a screen floating in 3D space.

[0244] Fig.13 An object designated by a second user for sharing with other users in the artificial reality system is shown. Fig.13 As shown, user 1310 is interacting with screen 1312 located on the wall of the room. As an example, screen 1312 may be a physical screen located on the wall. Alternatively, screen 1312 may be a projection screen that is positioned on the wall and is projected from a component of an artificial reality system or other projection system. In addition, similar to the above example, screen 1312 may be a screen that is not connected to a virtual reality system (e.g., in the case of a physical screen), or may be a screen that is connected to a virtual reality system in which any information written on the screen will be copied, saved, and accessible via the system (e.g., in the case of a projection screen). User 1310 may interact with screen 1312 using device 1314, which may be a physical device that allows user 1310 to interact with screen 1312 in the real world, or an electronic device that allows user 1310 to interact with screen 1312 in the real world (e.g., an electronic pointer device), or an electronic device that allows user 1310 to interact with screen 1312 in the virtual world (e.g., controller 106).

[0245] In a specific embodiment, if Fig.13As shown, in the area around user 1210, there are also bookshelves 1316, TV cabinets 1318, and sofas 1320. Among these objects in the room, the artificial reality system can determine that bookshelves 1316 and sofas 1320 are not objects that will be designated as objects for sharing information with other users in the artificial reality system, while TV cabinets 1318 (with associated TVs) can be designated as objects for sharing information with other users in the artificial reality system. In addition, user 1310 can designate screen 1312 as an object for sharing information with other users and / or for receiving shared information from other users. In a particular embodiment, the artificial reality system can access the environment around user 1310 to determine the location, position, and / or orientation of bookshelves 1316, TV cabinets 1318, and sofas 1320, and include these objects in the virtual reality display together with screen 1312. As discussed above, this can provide users with more realistic scenes and contexts about the location of screen 1312, so that users are not just watching screens floating in 3D space.

[0246] In a particular embodiment, once the artificial reality system determines that the user is interacting with an object that can be used to communicate and collaborate with other users, the artificial reality system can send a query to the user to determine whether the user wants to share the display of the interactive object with one or more other users. Alternatively, the artificial reality system can receive a request from the user to share the display of the interactive object with one or more other users. In both cases, the artificial reality system can ask the user which other users the user wants to invite to participate in the sharing of the display of the interactive object. As an example, the artificial reality system can send a list of other users to the user for selection. As another example, the artificial reality system can maintain a list of other users who have expressed interest in participating in the sharing of the display of the interactive object. Other users can be users from the user's social network as determined by social graph information (e.g., nodes and edges, affinities, and degrees of separation), as discussed in more detail below. In particular embodiments, other users may be determined by any suitable means, including, but not limited to, one or more time constraints (e.g., other users using the artificial reality system during the same time period as the user), one or more location-based constraints (e.g., other users within a geographic distance or within a particular geographic area), one or more degrees of separation, one or more affiliation coefficients between the user and the other users, one or more commonalities between the user and the other users, or any combination thereof.

[0247] Fig.14A and Fig. 14BVarious environments 1400, 1450 are shown for various users to view shared objects. In certain embodiments, a user can write on a real-life whiteboard, and the artificial reality system can synchronize the writing on the whiteboard with a virtual whiteboard so that other users in virtual reality (e.g., in a VR shared space similar to the VR shared space described above) can view and interact with the user and the content on the whiteboard. Fig.14A As shown, user 1410 may be a user who is interacting with a real-life whiteboard and using the whiteboard to write and display content to collaborate with other users 1414 and 1416. Other users 1414 and 1416 may be virtual users using head-mounted devices 1418 and 1420, respectively, to view content displayed on a virtual whiteboard 1412 that is a copy of the real-life whiteboard. Users 1414 and 1416 may access the displayed whiteboard 1412 via a VR shared space (as discussed above). In certain embodiments, in order to display a whiteboard in virtual reality, the artificial reality system may generate a virtual whiteboard 1412 (e.g., a virtual item) as a copy of the real-life whiteboard, and then display the virtual whiteboard 1412 to users 1414 and 1416 in a virtual reality environment (e.g., a VR shared space). In addition, the artificial reality system may also create a copy of user 1410, or use an avatar associated with user 1410, to be displayed in the virtual reality environment together with the virtual whiteboard 1412. In certain embodiments, if a change is made to the real-life whiteboard (e.g., user 1410 writes more content on the whiteboard), the artificial reality system will update the virtual whiteboard 1412 to include the same changes as the real-life whiteboard. This allows content and objects from real life to be synchronized with their representative versions in the virtual reality environment to create a mixed reality environment that enhances interaction, communication, and collaboration between users. As an example, users 1414 and 1416 can provide comments associated with the content on the virtual whiteboard 1412, which can be sent individually to user 1410 or visible to all users viewing the virtual whiteboard 1412.

[0248] In certain embodiments, when a user writes on a real-life whiteboard and the artificial reality system synchronizes the writing on the whiteboard with the virtual whiteboard so that other users in virtual reality (e.g., in a VR shared space similar to the VR shared space described above) can view and interact with the user and the content on the whiteboard, only a copy of the real-life whiteboard is displayed in virtual reality (without user 1410). In contrast, users 1414 and 1416 and additional new user 1452 can view only the virtual whiteboard 1412 through head mounted devices 1418, 1420, and 1454, respectively, and interact only with the content on the virtual whiteboard 1412. Similar to the embodiments discussed above, if changes are made to the real-life whiteboard (e.g., user 1410 writes more content on the whiteboard), the artificial reality system will update the virtual whiteboard 1412 to include the same changes as the real-life whiteboard. This allows content and objects from real life to be synchronized with their representative versions in the virtual reality environment to create a mixed reality environment that enhances interaction, communication, and collaboration between users. As an example, users 1414 , 1416 , and 1454 may provide comments associated with content on virtual whiteboard 1412 , which may be sent individually to user 1410 , or visible to all users viewing virtual whiteboard 1412 .

[0249] In a particular embodiment, as another example, a user may have questions about something on their computer screen, and may request the system to synchronize the computer screen with a virtual computer screen in order to invite another user in the VR shared space to help solve the problem or answer questions about the content on the computer screen. Similar to the situation discussed above, the artificial reality system may receive a request from a first user to share the display of a computer screen with one or more second users. As an example, this may be due to a computer problem encountered by the first user on the computer, and the fastest way to solve the problem may be to ask another user to visually evaluate the computer (rather than explaining the problem over the phone). As another example, this may be due to the first user wanting to share the content on the computer screen with another user (e.g., to collaborate on a project). In this way, the artificial reality system can generate a copy of the computer screen to display in the virtual environment, and then allow the second user to view the virtual computer screen in the virtual environment. If the first user makes changes to the content displayed on the computer screen, the virtual computer screen will be updated to show these changes. Similar to the embodiment discussed above, this allows content and objects from real life to be synchronized with their representative versions in the virtual reality environment to create a mixed reality environment that enhances interaction, communication, and collaboration between users.

[0250] Fig.15An example method for specifying various objects in an artificial reality system and sharing these objects with other users is shown. The method may begin at step 1510, where the artificial reality system may receive a request to share the display of a first interactive item with one or more users. At step 1520, the artificial reality system may generate a first virtual item as a copy of the first interactive item. At step 1530, the artificial reality system may display the first virtual item to a subset of one or more users in a virtual reality environment, wherein if a change is received to the first interactive item, the display of the first virtual item in the virtual reality environment is updated to include the same change as the first interactive item. In a particular embodiment, the request to share the display of the first interactive item may come from a first user of the one or more users who is currently interacting with the first interactive item. In a particular embodiment, the request to share the display of the first interactive item may come from one or more second users, the one or more second users being virtual users associated with the virtual reality environment. In a particular embodiment, before receiving the request to share the display of the first interactive item, the artificial reality system may determine one or more interactive items in the environment and send a list of interactive items to the first user for selecting at least one of the interactive items to be displayed in the virtual reality environment. In certain embodiments, the subset of one or more users may include virtual users in a virtual reality environment.

[0251] In a particular embodiment, the first interactive item may be located in a real-world environment. The artificial reality system may access the position of the first interactive item relative to one or more other items surrounding the first interactive item in the real-world environment, generate one or more second virtual items as copies of the one or more other items, and display the first virtual item and the one or more second virtual items in the virtual reality environment based on the position of the first interactive item relative to the one or more other items in the real environment. In addition, the artificial reality system may access the orientation of the first interactive item in the real-world environment, and display the first virtual item in the virtual reality environment based on the orientation of the first interactive item in the real-world environment. In a particular embodiment, the artificial reality system may receive a comment associated with the first virtual item from one or more users in the user subset, and send the comment to be displayed to a first user of the one or more users who is currently interacting with the first interactive item in the real-world environment. The comment may include one or more of an audio comment, a video comment, or a written comment.

[0252] Where appropriate, specific examples may be repeated Fig.15 Although the present disclosure will Fig.15The specific steps of the method are described and shown as occurring in a particular order, but the present disclosure contemplates Fig.15 Any suitable steps of the method may occur in any suitable order. In addition, although the present disclosure describes and illustrates the method including Fig.15 The example method for specifying various objects in an artificial reality system and sharing these objects with other users includes specific steps of the method, but the present disclosure contemplates any suitable method for specifying various objects in an artificial reality system and sharing these objects with other users including any suitable steps, and any suitable steps may include, where appropriate, Fig.15 In addition, although the present disclosure describes and illustrates the implementation of Fig.15 specific components, devices or systems for specific steps of the method, but the present disclosure contemplates performing Fig.15 Any suitable combination of any suitable components, devices, or systems of any suitable steps of the method.

[0253] System Overview

[0254] Fig.16 An example network environment 1600 associated with a social networking system is shown. The network environment 1600 includes users 1601, client systems 1630, social networking systems 1660, and third-party systems 1670 connected to each other via a network 1610. Fig.16 A particular arrangement of users 1601, client systems 1630, social networking systems 1660, third-party systems 1670, and networks 1610 is shown, but this disclosure contemplates any suitable arrangement of users 1601, client systems 1630, social networking systems 1660, third-party systems 1670, and networks 1610. As an example and not by way of limitation, two or more of client systems 1630, social networking systems 1660, and third-party systems 1670 may be directly connected to each other, bypassing network 1610. As another example, two or more of client systems 1630, social networking systems 1660, and third-party systems 1670 may be physically or logically co-located with each other, in whole or in part. Furthermore, although Fig.16 A particular number of users 1601, client systems 1630, social networking systems 1660, third-party systems 1670, and networks 1610 are shown, but this disclosure contemplates any suitable number of users 1601, client systems 1630, social networking systems 1660, third-party systems 1670, and networks 1610. By way of example and not limitation, network environment 1600 may include a plurality of users 1601, client systems 1630, social networking systems 1660, third-party systems 1670, and networks 1610.

[0255] In certain embodiments, user 1601 may be an individual (human user), entity (e.g., a company, enterprise, or third-party application), or group (e.g., a group of individuals or entities) that interacts or communicates with or through social networking system 1660. In certain embodiments, social networking system 1660 may be a network-addressable computing system that hosts an online social network. Social networking system 1660 may generate, store, receive, and send social network data (e.g., user profile data, concept profile data, social graph information, or other suitable data related to an online social network). Social networking system 1660 may be accessed by other components of network environment 1600 directly or via network 1610. In certain embodiments, social networking system 1660 may include an authorization server (or other suitable component) that allows user 1601 to opt-in or opt-out of having their actions recorded by social networking system 1660 or shared with other systems (e.g., third-party systems 1670), for example, by setting appropriate privacy settings. The user's privacy settings may determine what information associated with the user may be recorded, how information associated with the user may be recorded, when information associated with the user may be recorded, who may record information associated with the user, with whom information associated with the user may be shared, and for what purposes information associated with the user may be recorded or shared. The authorization server may be used to implement one or more privacy settings of users of the social networking system 30 through blocking, data hashing, anonymization, or other suitable techniques as needed. In a particular embodiment, the third-party system 1670 may be a network addressable computing system. The third-party system 1670 may be accessed by other components of the network environment 1600 directly or via the network 1610. In a particular embodiment, one or more users 1601 may use one or more client systems 1630 to access the social networking system 1660 or the third-party system 1670, send data to it, and receive data from it. The client system 1630 may access the social networking system 1660 or the third-party system 1670 directly, via the network 1610, or via a third-party system. By way of example and not limitation, client system 1630 may access third party system 1670 via social networking system 1660. Client system 1630 may be any suitable computing device, such as a personal computer, laptop, cellular phone, smartphone, tablet computer, or augmented / virtual reality device.

[0256] The present disclosure contemplates any suitable network 1610. By way of example and not limitation, one or more portions of network 1610 may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of a public switched telephone network (PSTN), a cellular telephone network, or a combination of two or more of these. Network 1610 may include one or more networks 1610.

[0257] Links 1650 may connect client systems 1630, social networking systems 1660, and third-party systems 1670 to communication network 1610 or to each other. The present disclosure contemplates any suitable links 1650. In particular embodiments, one or more links 1650 include one or more wired links (such as, for example, digital subscriber lines (DSL) or cable data service interface specifications (DOCSIS)), wireless links (such as, for example, Wi-Fi or Worldwide Interoperability for Microwave Access (WiMAX)), or optical links (such as, for example, synchronous optical networks (SONET) or synchronous digital hierarchies (SDH)). In particular embodiments, one or more links 1650 each include an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, a portion of the Internet, a portion of the PSTN, a network based on cellular technology, a network based on satellite communication technology, another link 1650, or a combination of two or more such links 1650. Links 1650 need not necessarily be the same throughout network environment 1600. The one or more first links 1650 may differ from the one or more second links 1650 in one or more aspects.

[0258] Social Graph

[0259] Fig.17 An example social graph 1700 is shown. In certain embodiments, social networking system 1660 may store one or more social graphs 1700 in one or more data stores. In certain embodiments, social graph 1700 may include a plurality of nodes, which may include a plurality of user nodes 1702 or a plurality of concept nodes 1704, and a plurality of edges 1706 connecting the nodes. For teaching purposes, Fig.17The illustrated example social graph 1700 is shown in a two-dimensional visual map representation. In certain embodiments, social networking system 1660, client system 1630, or third-party system 1670 may access social graph 1700 and related social graph information for use in appropriate applications. The nodes and edges of social graph 1700 may be stored as data objects, for example, in a data store (e.g., a social graph database). Such a data store may include one or more searchable or queryable indexes of the nodes or edges of social graph 1700.

[0260] In certain embodiments, user nodes 1702 may correspond to users of social networking system 1660. As an example and not by way of limitation, users may be individuals (human users), entities (e.g., businesses, companies, or third-party applications), or groups (e.g., individuals or entities) that interact or communicate with social networking system 1660 or through social networking system 1660. In certain embodiments, when a user registers an account with social networking system 1660, social networking system 1660 may create a user node 1702 corresponding to the user and store the user node 1702 in one or more data stores. Users and user nodes 1702 described herein may refer to registered users and user nodes 1702 associated with registered users, where appropriate. Additionally or as an alternative, users and user nodes 1702 described herein may refer to users who are not registered with social networking system 1660, where appropriate. In certain embodiments, user nodes 1702 may be associated with information provided by a user or information collected by various systems (including social networking system 1660). As an example and not by way of limitation, a user may provide his or her name, profile picture, contact information, date of birth, gender, marital status, family status, occupation, educational background, preferences, interests, or other demographic information. In certain embodiments, user node 1702 may be associated with one or more data objects corresponding to information associated with the user. In certain embodiments, user node 1702 may correspond to one or more web pages.

[0261] In certain embodiments, concept nodes 1704 may correspond to concepts. By way of example and not limitation, a concept may correspond to a place (such as, for example, a movie theater, a restaurant, a landmark, or a city); a website (such as, for example, a website associated with social networking system 1660 or a third-party website associated with a web application server); an entity (such as, for example, a person, a business, a group, a sports team, or a celebrity); a resource (such as, for example, an audio file, a video file, a digital photo, a text file, a structured document, or an application), which may be located within social networking system 1660 or on an external server (e.g., a web application server); real estate or intellectual property (such as, for example, a sculpture, a painting, a movie, a game, a song, an idea, a photo, or a written work); a game; an activity; an idea or theory; another suitable concept; or two or more such concepts. Concept node 1704 may be associated with information about the concept provided by a user or information collected by various systems including social networking system 1660. By way of example and not limitation, information about a concept may include a name or title; one or more images (e.g., an image of a book cover); a location (e.g., an address or geographic location); a website (which may be associated with a URL); contact information (e.g., a phone number or email address); other suitable concept information; or any suitable combination of such information. In particular embodiments, a concept node 1704 may be associated with one or more data objects that correspond to the information associated with the concept node 1704. In particular embodiments, a concept node 1704 may correspond to one or more web pages.

[0262] In certain embodiments, a node in social graph 1700 may represent a web page (which may be referred to as a "profile page") or be represented by a web page. A profile page may be hosted by social networking system 1606 or accessible to social networking system 1606. A profile page may also be hosted on a third-party website associated with third-party server 1670. As an example and not as a limitation, a profile page corresponding to a particular external web page may be a particular external web page, and the profile page may correspond to a particular concept node 1704. The profile page may be viewable by all or a selected subset of other users. As an example and not as a limitation, user node 1702 may have a corresponding user profile page, where the corresponding user may add content, make a statement, or otherwise express him or herself. As another example and not as a limitation, a concept node 1704 may have a corresponding concept profile page, where one or more users may add content, make a statement, or express themselves, particularly with respect to the concept corresponding to concept node 1704.

[0263] In certain embodiments, concept node 1704 may represent a third-party web page or resource hosted by third-party system 1670. The third-party web page or resource may include content representing an action or activity, selectable icons or other icons or other interactable objects (which may be implemented, for example, with JavaScript, AJAX, or PHP code), and other elements. As an example and not by way of limitation, the third-party web page may include selectable icons (e.g., "Like," "Check-in," "Eat," "Recommend," or other suitable actions or activities). A user viewing the third-party web page may perform an action by selecting one of the icons (e.g., "Check-in"), causing client system 130 to send a message to social-networking system 1660 indicating the user's action. In response to the message, social-networking system 1660 may create an edge (e.g., a check-in-type edge) between user node 1702 corresponding to the user and concept node 1704 corresponding to the third-party web page or resource, and store edge 1706 in one or more data stores.

[0264] In particular embodiments, a pair of nodes in the social graph 1700 may be connected to each other via one or more edges 1706. An edge 1706 connecting a pair of nodes may represent a relationship between the pair of nodes. In particular embodiments, an edge 1706 may include or represent one or more data objects or attributes corresponding to the relationship between a pair of nodes. As an example and not by way of limitation, a first user may indicate that a second user is a "friend" of the first user. In response to the indication, the social networking system 1660 may send a "friend request" to the second user. If the second user confirms the "friend request," the social networking system 1660 may create an edge 1706 in the social graph 1700 connecting the first user's user node 1702 to the second user's user node 1702, and store the edge 1706 as social graph information in one or more data stores 1664. Fig.17In the example of , the social graph 1700 includes an edge 1706 indicating a friend relationship between the user nodes 1702 of user "A" and user "B", and an edge indicating a friend relationship between the user nodes 1702 of user "C" and user "B". Although the present disclosure describes or shows a specific edge 1706 with a specific attribute connecting a specific user node 1702, the present disclosure contemplates any suitable edge 1706 with any suitable attribute connecting the user nodes 1702. As an example and not limitation, the edge 1706 can represent a friendship, a family relationship, a business or employment relationship, a fan relationship (including, for example, likes, etc.), a follower relationship, a visitor relationship (including, for example, visits, views, check-ins, sharing, etc.), a subscriber relationship, a superior / subordinate relationship, a reciprocal relationship, a non-reciprocal relationship, another suitable type of relationship, or two or more such relationships. In addition, although the present disclosure generally describes nodes as being connected, the present disclosure also describes users or concepts as being connected. Herein, references to connected users or concepts may refer to nodes corresponding to those users or concepts connected by one or more edges 1706 in the social graph 1700 , where appropriate.

[0265] In certain embodiments, edges 1706 between user node 1702 and concept node 1704 may represent specific actions or activities performed by a user associated with user node 1702 on a concept associated with concept node 1704. By way of example and not limitation, Fig.17 As shown, a user can "Like," "Attend," "Play," "Listen to," "Cook," "Work at," or "Watch" a concept, each of which can correspond to an edge type or subtype. The concept profile page corresponding to concept node 1704 may include, for example, a selectable "Check-in" icon (e.g., a clickable "Check-in" icon) or a selectable "Add to Favorites" icon. Similarly, after the user clicks on these icons, social networking system 1660 can create a "Favorites" edge or a "Check-in" edge in response to the user action corresponding to the corresponding action. As another example and not by way of limitation, a user (user "C") may use a particular application (SPOTIFY, which is an online music application) to listen to a particular song ("Imagine"). In this case, social networking system 1660 may create a "Listen" edge 1706 and a "Use" edge (e.g., a clickable "Check-in" icon) between user node 1702 corresponding to the user and concept nodes 1704 corresponding to the song and application. Fig.17 ) to indicate that the user listened to the song and used the application. In addition, social networking system 1660 can create a "play" edge 1706 (as shown) between concept nodes 1704 corresponding to the song and the application. Fig.17) to indicate that the particular song was played by a particular application. In this case, the "played" edge 1706 corresponds to an action performed by an external application (SPOTIFY) on an external audio file (the song "Imagine"). Although this disclosure describes a particular edge 1706 having particular attributes connecting a user node 1702 and a concept node 1704, this disclosure contemplates any suitable edge 1706 having any suitable attributes connecting a user node 1702 and a concept node 1704. Furthermore, although this disclosure describes an edge between a user node 1702 and a concept node 1704 representing a single relationship, this disclosure contemplates edges between a user node 1702 and a concept node 1704 representing one or more relationships. As an example and not by way of limitation, an edge 1706 may indicate that a user liked and used a particular concept. Optionally, another edge 1706 may represent a relationship between a user node 1702 and a concept node 1704 (e.g., Fig.17 As shown, each type of relationship (or multiples of a single relationship) between a user node 1702 for user "E" and a concept node 1704 for "SPOTIFY".

[0266] In certain embodiments, social networking system 1660 may create edge 1706 between user node 1702 and concept node 1704 in social graph 1700. As an example and not by way of limitation, a user viewing a concept profile page (e.g., using a web browser or dedicated application hosted by a user's client system 1630) may indicate that he or she likes the concept represented by concept node 1704 by clicking or selecting a "Like" icon, which may cause the user's client system 1630 to send a message to social networking system 1660 indicating that the user likes the concept associated with the concept profile page. In response to the message, social networking system 1660 may create edge 1706 between user node 1702 associated with the user and concept node 1704, as shown by "Like" edge 1706 between the user and concept node 1704. In certain embodiments, social networking system 1660 may store edge 1706 in one or more data stores. In certain embodiments, edge 1706 may be automatically formed by social networking system 1660 in response to a particular user action. As an example and not by way of limitation, if a first user uploads a picture, watches a movie, or listens to a song, an edge 1706 may be formed between the user node 1702 corresponding to the first user and the concept nodes 1704 corresponding to those concepts. Although this disclosure describes forming particular edges 1706 in particular manners, this disclosure contemplates forming any suitable edges 1706 in any suitable manner.

[0267] Social graph affinity and coefficient

[0268] In particular embodiments, social networking system 1660 may determine social graph affinities (which may be referred to herein as "affinity") of various social graph entities for each other. Affinity may represent the strength of a relationship or level of interest between specific objects (e.g., users, concepts, content, actions, advertisements) associated with an online social network, other objects associated with an online social network, or any suitable combination thereof. Affinity may also be determined with respect to objects associated with third-party system 1670 or other suitable system. An overall affinity for a social graph entity may be established for each user, topic, or content type. The overall affinity may change based on ongoing monitoring of actions or relationships associated with the social graph entities. Although this disclosure describes determining particular affinities in a particular manner, this disclosure contemplates determining any suitable affinity in any suitable manner.

[0269] In certain embodiments, social networking system 1660 may measure or quantify social graph affinity using affinity coefficients (which may be referred to herein as "coefficients"). The coefficients may represent or quantify the strength of relationships between specific objects associated with an online social network. The coefficients may also represent a probability or function that measures the predicted probability that a user will perform a specific action based on the user's interest in the action. In this way, a user's future actions may be predicted based on the user's previous actions, where the coefficients may be calculated based at least in part on the user's history of actions. The coefficients may be used to predict any number of actions that may be within or outside of an online social network. By way of example and not limitation, these actions may include various types of communications, such as sending messages, posting content, or commenting on content; various types of observation actions, such as visiting or viewing a profile page, media, or other suitable content; various types of consistent information about two or more social graph entities, such as being in the same group, being tagged in the same photo, checking in at the same location, or attending the same event; or other suitable actions. Although the present disclosure describes measuring affinity in a particular manner, the present disclosure contemplates measuring affinity in any suitable manner.

[0270] In certain embodiments, the social networking system 1660 may use various factors to calculate coefficients. These factors may include, for example, user actions, types of relationships between objects, location information, other suitable factors, or any combination thereof. In certain embodiments, different factors may be weighted differently when calculating coefficients. The weight of each factor may be static, or the weight may be changed according to, for example, the user, the type of relationship, the type of action, the location of the user, etc. The ratings of these factors may be combined according to the weights of the factors to determine the total coefficient of the user. As an example and not by way of limitation, a particular user action may be assigned a rating and a weight, while a relationship associated with the particular user action is assigned a rating and an associated weight (e.g., so that the weights total 100%). To calculate a user's coefficient for a particular object, the rating assigned to the user action may include, for example, 60% of the total coefficient, while the relationship between the user and the object may include 40% of the total coefficient. In certain embodiments, when determining the weights of various factors used to calculate coefficients, the social networking system 1660 may consider various variables, such as the time since the information was accessed, the decay factor, the frequency of access, the relationship to the information or to the object to which the information was accessed, the relationship to the social graph entity connected to the object, the short-term or long-term average of user actions, user feedback, other suitable variables, or any combination thereof. As an example and not by way of limitation, the coefficients may include a decay factor that decays the strength of the signal provided by a particular action over time, making more recent actions more relevant when calculating the coefficients. The rankings and weights may be continuously updated based on the ongoing tracking of the actions on which the coefficients are based. Any type of process or algorithm may be employed to assign, combine, average, etc. the ranking of each factor and the weights assigned to these factors. In certain embodiments, the social networking system 1660 may determine the coefficients using a machine learning algorithm trained on historical actions and past user responses, or data obtained from users by exposing them to various options and measuring responses. Although the present disclosure describes calculating the coefficients in a particular manner, the present disclosure contemplates calculating the coefficients in any suitable manner.

[0271] In a particular embodiment, the social networking system 1660 can calculate coefficients based on the user's actions. The social networking system 1660 can monitor such actions on online social networks, third-party systems 1670, other suitable systems, or any combination thereof. Any suitable type of user action can be tracked or monitored. Typical user actions include viewing profile pages, creating or publishing content, interacting with content, marking in images or being marked in images, joining groups, listing and confirming event attendance, signing in at different locations, liking specific pages, creating pages, and performing other tasks that contribute to social actions. In a particular embodiment, the social networking system 1660 can calculate coefficients based on the user's actions on specific types of content. Content can be associated with an online social network, a third-party system 1670, or another suitable system. Content can include users, profile pages, posts, news stories, titles, instant messages, chat room sessions, emails, advertisements, pictures, videos, music, other suitable objects, or any combination thereof. The social networking system 1660 can analyze the user's actions to determine whether one or more of these actions indicate affinity for a topic, content, other users, etc. As an example and not by way of limitation, if the user is likely to frequently post content related to "coffee" or variations thereof, the social networking system 1660 may determine that the user has a high coefficient with respect to the concept "coffee". Certain actions or types of actions may be assigned higher weights and / or rankings than other actions, which may affect the overall calculated coefficient. As an example and not by way of limitation, if a first user sends an email to a second user, the weight or ranking of that action may be higher than if the first user simply views the user profile page of the second user.

[0272] In certain embodiments, the social networking system 1660 may calculate coefficients based on the type of relationships between certain objects. Referring to the social graph 1700, when calculating coefficients, the social networking system 1660 may analyze the number and / or type of edges 1706 connecting certain user nodes 1702 and concept nodes 1704. As an example and not by way of limitation, user nodes 1702 connected by spouse-type edges (indicating that two users are married) may be assigned higher coefficients than user nodes 1702 connected by friend-type edges. In other words, based on the weights assigned to the actions and relationships of a certain user, it may be determined that the overall affinity for content about the user's spouse is higher than the overall affinity for content about the user's friends. In certain embodiments, a user's relationship with another object may affect the weight and / or ranking of the user's actions for calculating the coefficients for that object. As an example and not by way of limitation, if a user is tagged in a first photo, but only likes a second photo, the social networking system 1660 may determine that the user has a higher coefficient with respect to the first photo than the second photo, because having a tagged-in-type relationship with the content may be assigned a higher weight and / or rating than having a like-type relationship with the content. In certain embodiments, the social networking system 1660 may calculate the coefficient of the first user based on the relationship that one or more second users have with a particular object. In other words, the connections and coefficients of other users with an object may affect the coefficient of the first user with respect to the object. As an example and not by way of limitation, if the first user is connected to one or more second users or has a high coefficient for one or more second users, and those second users are connected to a particular object or have a high coefficient for a particular object, the social networking system 1660 may determine that the first user should also have a relatively high coefficient for the particular object. In certain embodiments, the coefficient may be based on the degree of separation between particular objects. A lower coefficient may indicate a reduced likelihood that the first user will share an interest in the content object with users indirectly connected to the first user in the social graph 1700. As an example and not by way of limitation, social graph entities that are closer together (ie, less separated) in social graph 1700 may have a higher coefficient than entities that are farther apart in social graph 1700 .

[0273] In certain embodiments, social networking system 1660 may calculate coefficients based on location information. Objects that are geographically closer to each other may be considered more relevant or interesting to each other than objects that are farther away. In certain embodiments, a user's coefficient for a particular object may be based on the proximity of the object's location to the current location associated with the user (or the location of the user's client system 1630). A first user may be more interested in other users or concepts that are closer to the first user. As an example and not by way of limitation, if a user is one mile from an airport and two miles from a gas station, social networking system 1660 may determine that the user has a higher coefficient for the airport than for the gas station based on the proximity of the airport to the user.

[0274] In certain embodiments, the social networking system 1660 may perform a specific action about a user based on coefficient information. The coefficient may be used to predict whether a user will perform the action based on the user's interest in the specific action. The coefficient may be used when generating or presenting any type of object (e.g., advertisement, search results, news, media, message, notification, or other suitable object) to a user. The coefficient may also be used to properly rank and order such objects. In this way, the social networking system 1660 may provide information related to the user's interests and current environment, increasing the likelihood that they will find such interesting information. In certain embodiments, the social networking system 1660 may generate content based on coefficient information. Content objects may be provided or selected based on coefficients specific to the user. As an example and not as a limitation, the coefficient may be used to generate media for a user, wherein the user may be presented with media with a high total coefficient about a media object. As another example and not as a limitation, the coefficient may be used to generate advertisements for a user, wherein the user may be presented with advertisements with a high overall coefficient about an advertisement object. In certain embodiments, the social networking system 1660 may generate search results based on coefficient information. Search results for a particular user may be scored or ranked based on coefficients associated with search results for the querying user. As an example and not by way of limitation, search results corresponding to objects with higher coefficients may be ranked higher on a search results page than results corresponding to objects with lower coefficients.

[0275] In certain embodiments, the social networking system 1660 may calculate coefficients in response to a coefficient request from a specific system or process. In order to predict the possible actions that a user may take (or may be its subject) in a given situation, any process may request the calculated coefficients for the user. The request may also include a set of weights for various factors used to calculate the coefficients. The request may come from a process running on the online social network, from a third-party system 1670 (e.g., via an API or other communication channel), or from another suitable system. In response to the request, the social networking system 1660 may calculate the coefficients (or access the coefficient information if the coefficient information has been previously calculated and stored). In certain embodiments, the social networking system 1660 may measure the affinity of a specific process. Different processes (inside and outside the online social network) may request coefficients for a specific object or set of objects. The social networking system 1660 may provide a measure of affinity associated with a specific process that has requested a measure of affinity. In this way, each process receives an affinity measure customized for a different context, where the process will use the affinity measure.

[0276] In conjunction with social graph affinities and affinity coefficients, particular embodiments may utilize one or more systems, components, elements, functions, methods, operations, or steps disclosed in U.S. patent application Ser. No. 11 / 503,093, filed Aug. 11, 2006, U.S. patent application Ser. No. 12 / 977,027, filed Dec. 22, 2010, U.S. patent application Ser. No. 12 / 978,265, filed Dec. 23, 2010, and U.S. patent application Ser. No. 13 / 632,869, filed Oct. 1, 2012 (each of which is incorporated by reference).

[0277] privacy

[0278] In certain embodiments, one or more content objects of an online social network may be associated with a privacy setting. The privacy setting (or "access setting") of an object may be stored in any suitable manner, such as, for example, associated with the object, indexed on an authorization server, in another suitable manner, or any combination thereof. The privacy setting of an object may specify how an object (or specific information associated with an object) may be accessed (e.g., viewed or shared) using an online social network. In the case where the privacy setting of an object allows a particular user to access the object, the object may be described as being "visible" relative to the user. As an example and not by way of limitation, a user of an online social network may specify a privacy setting for a user profile page that identifies a set of users that may access work experience information on a user profile page, thereby excluding other users from accessing the information. In certain embodiments, the privacy setting may specify a "blocked list" of users that should not be allowed to access certain information associated with the object. In other words, the blacklist may specify one or more users or entities for whom the object is not visible. As an example and not by way of limitation, a user may specify a set of users that may not access an album associated with the user, thereby excluding those users from accessing the album (while also potentially allowing certain users not in the set of users to access the album). In particular embodiments, privacy settings may be associated with particular social graph elements. Privacy settings for social graph elements (e.g., nodes or edges) may specify how the social graph element, information associated with the social graph element, or content objects associated with the social graph element may be accessed using an online social network. As an example and not by way of limitation, a particular concept node 1704 corresponding to a particular photo may have a privacy setting that specifies that the photo can only be accessed by the user tagged in the photo and their friends. In particular embodiments, privacy settings may allow a user to opt-in or opt-out of having their actions recorded by the social networking system 1660 or shared with other systems (e.g., third-party systems 1670). In particular embodiments, privacy settings associated with an object may specify any suitable granularity of access granted or access denied. By way of example and not limitation, access may be specified or denied for specific users (e.g., only me, my roommate, and my boss), users within a particular degree of separation (e.g., friends or friends of friends), groups of users (e.g., a gaming club, my family), networks of users (e.g., employees of a particular employer, students or alumni of a particular university), all users ("public"), no users ("private"), users of third-party system 170, specific applications (e.g., third-party applications, external websites), other suitable users or entities, or any combination thereof. Although this disclosure describes using particular privacy settings in particular ways, this disclosure contemplates using any suitable privacy settings in any suitable way.

[0279] In certain embodiments, one or more servers 1662 may be authorization / privacy servers for implementing privacy settings. In response to a request from a user (or other entity) for a particular object stored in data storage 1664, social networking system 1660 may send a request for the object to data storage 1664. The request may identify a user associated with the request, and the request may be sent to the user (or the user's client system 1630) only if the authorization server determines that the user is authorized to access the object based on the privacy settings associated with the object. If the requesting user is not authorized to access the object, the authorization server may prevent the requested object from being retrieved from data storage 1664, or may prevent the requested object from being sent to the user. In the context of a search query, an object may be generated as a search result only if the querying user is authorized to access the object. In other words, the object must have visibility visible to the querying user. If the object has visibility that is not visible to the user, the object may be excluded from the search results. Although the present disclosure describes implementing privacy settings in a particular manner, the present disclosure contemplates implementing privacy settings in any suitable manner.

[0280] System and method

[0281] Fig.18 An example computer system 1800 is shown. In certain embodiments, one or more computer systems 1800 perform one or more steps of one or more methods described or shown herein. In certain embodiments, one or more computer systems 1800 provide functionality described or shown herein. In certain embodiments, software running on one or more computer systems 1800 performs one or more steps of one or more methods described or shown herein, or provides functionality described or shown herein. Certain embodiments include one or more portions of one or more computer systems 1800. Herein, references to computer systems may include computing devices, and vice versa, where appropriate. Furthermore, references to computer systems may include one or more computer systems, where appropriate.

[0282] The present disclosure contemplates any suitable number of computer systems 1800. The present disclosure contemplates that computer system 1800 takes any suitable physical form. By way of example and not limitation, computer system 1800 may be an embedded computer system, a system on a chip (SOC), a single board computer system (SBC) (e.g., a computer on a module (COM) or a system on a module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these systems. Where appropriate, computer system 1800 may include one or more computer systems 1800; be monolithic or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems 1800 may perform one or more steps of one or more methods described or illustrated herein without substantial spatial or temporal limitations. As an example and not as a limitation, one or more computer systems 1800 may perform one or more steps of one or more methods described or shown herein in real time or in batch mode. Where appropriate, one or more computer systems 1800 may perform one or more steps of one or more methods described or shown herein at different times or at different locations.

[0283] In a particular embodiment, computer system 1800 includes a processor 1802, a memory 1804, a storage device 1806, an input / output (I / O) interface 1808, a communication interface 1810, and a bus 1812. Although this disclosure describes and illustrates a particular computer system having particular numbers of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable numbers of any suitable components in any suitable arrangement.

[0284] In a particular embodiment, the processor 1802 includes hardware for executing instructions (e.g., those that constitute a computer program). As an example and not by way of limitation, to execute instructions, the processor 1802 may retrieve (or fetch) instructions from an internal register, an internal cache, a memory 1804, or a storage device 1806; decode them and execute them; and then write one or more results to an internal register, an internal cache, a memory 1804, or a storage device 1806. In a particular embodiment, the processor 1802 may include one or more internal caches for data, instructions, or addresses. Where appropriate, the present disclosure contemplates that the processor 1802 includes any suitable number of any suitable internal caches. As an example and not by way of limitation, the processor 1802 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). The instructions in the instruction cache may be copies of the instructions in the memory 1804 or the storage device 1806, and the instruction cache may speed up the retrieval of those instructions by the processor 1802. The data in the data cache may be: a copy of data in the memory 1804 or storage device 1806 for operating the instructions executed at the processor 1802; the result of a previous instruction executed at the processor 1802 for access by a subsequent instruction executed at the processor 1802 or for writing to the memory 1804 or storage device 1806; or other suitable data. The data cache may speed up read or write operations performed by the processor 1802. The TLB may speed up virtual address translations for the processor 1802. In a particular embodiment, the processor 1802 may include one or more internal registers for data, instructions, or addresses. Where appropriate, the present disclosure contemplates that the processor 1802 includes any suitable number of any suitable internal registers. Where appropriate, the processor 1802 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 1802. Although the present disclosure describes and illustrates a particular processor, the present disclosure contemplates any suitable processor.

[0285] In a particular embodiment, the memory 1804 includes a main memory for storing instructions for the processor 1802 to execute or data for the processor 1802 to operate. As an example and not by way of limitation, the computer system 1800 can load instructions from the storage device 1806 or another source (e.g., another computer system 1800) to the memory 1804. The processor 1802 can then load the instructions from the memory 1804 to an internal register or an internal cache. In order to execute the instructions, the processor 1802 can retrieve the instructions from the internal register or the internal cache and decode them. During or after the execution of the instructions, the processor 1802 can write one or more results (which can be intermediate results or final results) to the internal register or the internal cache. The processor 1802 can then write one or more of these results to the memory 1804. In a particular embodiment, the processor 1802 executes only instructions in one or more internal registers or internal caches or in the memory 1804 (rather than the storage device 1806 or elsewhere), and operates only on data in one or more internal registers or internal caches or in the memory 1804 (rather than the storage device 1806 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple the processor 1802 to the memory 1804. As described below, the bus 1812 may include one or more memory buses. In a particular embodiment, one or more memory management units (MMUs) reside between the processor 1802 and the memory 1804 and facilitate access to the memory 1804 requested by the processor 1802. In a particular embodiment, the memory 1804 includes a random access memory (RAM). Where appropriate, the RAM may be a volatile memory. Where appropriate, the RAM may be a dynamic RAM (DRAM) or a static RAM (SRAM). In addition, where appropriate, the RAM may be a single-port RAM or a multi-port RAM. The present disclosure contemplates any suitable RAM. Where appropriate, memory 1804 may include one or more memories 1804. Although this disclosure describes and illustrates particular memories, this disclosure contemplates any suitable memories.

[0286] In a particular embodiment, storage device 1806 includes a large-capacity storage device for data or instructions. As an example and not as a limitation, storage device 1806 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, storage device 1806 may include a removable or non-removable (or fixed) medium. Where appropriate, storage device 1806 may be inside or outside computer system 1800. In a particular embodiment, storage device 1806 is a non-volatile solid-state memory. In a particular embodiment, storage device 1806 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically variable ROM (EAROM), or a flash memory, or a combination of two or more of these. The present disclosure contemplates a large-capacity storage device 1806 in any suitable physical form. Where appropriate, storage 1806 may include one or more storage control units that facilitate communications between processor 1802 and storage 1806. Where appropriate, storage 1806 may include one or more storage 1806. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

[0287] In a particular embodiment, I / O interface 1808 includes hardware, software, or both of one or more interfaces for communication between computer system 1800 and one or more I / O devices. Where appropriate, computer system 1800 may include one or more of these I / O devices. One or more of these I / O devices may enable communication between a person and computer system 1800. As an example and not as a limitation, I / O devices may include a keyboard, a keypad, a microphone, a monitor, a mouse, a printer, a scanner, a speaker, a static camera, a stylus, a tablet computer, a touch screen, a tracking ball, a video camera, another suitable I / O device, or a combination of two or more of these. I / O devices may include one or more sensors. The present disclosure contemplates any suitable I / O device and any suitable I / O interface 1808 for them. Where appropriate, I / O interface 1808 may include one or more devices or software drivers that enable processor 1802 to drive one or more of these I / O devices. Where appropriate, I / O interface 1808 may include one or more I / O interfaces 1808. Although this disclosure describes and illustrates a particular I / O interface, this disclosure contemplates any suitable I / O interface.

[0288] In a particular embodiment, the communication interface 1810 includes hardware, software, or both that provide one or more interfaces for communication (e.g., packet-based communication) between the computer system 1800 and one or more other computer systems 1800 or one or more networks. As an example and not by way of limitation, the communication interface 1810 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network (e.g., a WI-FI network). The present disclosure contemplates any suitable network and any suitable communication interface 1810 for it. As an example and not by way of limitation, the computer system 1800 may communicate with one or more parts of an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or a combination of two or more of these. One or more parts of one or more of these networks may be wired or wireless. As an example, the computer system 1800 may communicate with a wireless PAN (WPAN) (e.g., a Bluetooth WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (e.g., a Global System for Mobile Communications (GSM) network), or other suitable wireless networks, or a combination of two or more of these. Where appropriate, the computer system 1800 may include any suitable communication interface 1810 for any of these networks. Where appropriate, the communication interface 1810 may include one or more communication interfaces 1810. Although this disclosure describes and illustrates particular communication interfaces, this disclosure contemplates any suitable communication interface.

[0289] In a particular embodiment, bus 1812 includes hardware, software, or both that couples components of computer system 1800 to each other. By way of example and not limitation, bus 1812 may include an accelerated graphics port (AGP) or other graphics bus, an extended industry standard architecture (EISA) bus, a front side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an industry standard architecture (ISA) bus, an INFINIBAND interconnect, a low pin count (LPC) bus, a memory bus, a micro channel architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a video electronics standard association local (VLB) bus, or any other suitable bus, or a combination of two or more of these. Where appropriate, bus 1812 may include one or more buses 1812. Although the present disclosure describes and illustrates a particular bus, the present disclosure contemplates any suitable bus or interconnect.

[0290] Herein, where appropriate, one or more computer-readable non-transitory storage media may include one or more semiconductor-based or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical disk drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital (SECURE DIGITAL) cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these. Where appropriate, the computer-readable non-transitory storage media may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0291] As used herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or indicated by context. Thus, as used herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or indicated by context. Furthermore, "and" is both conjunctive and several, unless expressly indicated otherwise or indicated by context. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly indicated otherwise or indicated by context.

[0292] The scope of the present disclosure includes all changes, substitutions, variations, alterations and modifications to the example embodiments described or shown herein that will be understood by those of ordinary skill in the art. The scope of the present disclosure is not limited to the example embodiments described or shown herein. In addition, although the present disclosure describes and illustrates the corresponding embodiments herein as including specific components, elements, features, functions, operations or steps, any of these embodiments may include any combination or permutation of any components, elements, features, functions, operations or steps described or shown anywhere herein that will be understood by those of ordinary skill in the art. In addition, references to devices or systems or components of devices or systems that are suitable for, arranged to, capable of, configured to, implemented to, operable to, or operated to perform specific functions in the appended claims include the device, system, component, whether it or that specific function is activated, turned on or unlocked, as long as the device, system or component is adjusted, arranged, enabled, configured, implemented, operable, or operated in this way.

Claims

1. A method comprising: receiving a request from a first user to create a joint virtual space for use with one or more second users; determining a first area in a first room based at least in part on a spatial constraint associated with the first room and a location of one or more items in the first room, the first room being associated with the first user; retrieving information associated with one or more second rooms for each of the second users; creating the joint virtual space based on the first area of ​​the first room and the information associated with each of the second rooms; and Access to the joint virtual space is provided to each of the one or more second users and the first user.

2. The method of claim 1 , further comprising, before retrieving information associated with the one or more second rooms: A determination is made as to whether the first area in the first room is equal to or larger than a predetermined minimum area.

3. The method according to claim 1, wherein: The first area is determined by calculating a maximum free space associated with the first room after evaluating the spatial constraints and the location of one or more items in the first room.

4. The method according to claim 3, wherein: The retrieved information associated with the second room includes at least: a spatial constraint associated with each of the second rooms for each of the second users, and The location of one or more items in each of the second rooms for each of the second users.

5. The method according to claim 4, further comprising: determining a second area for each of the one or more second rooms based at least in part on the spatial constraint and the location of the one or more objects, wherein the second area is determined by calculating a maximum free space associated with each of the one or more second rooms after evaluating the spatial constraints and the location of one or more items in each of the one or more second rooms.

6. The method according to claim 5, wherein: The joint virtual space is created by determining a maximum overlap between a maximum free space associated with the first room and a maximum free space associated with each of the one or more second rooms.

7. The method according to claim 1, wherein: Providing access to the joint virtual space includes notifying each of the first user and the one or more second users that the joint virtual space is available for use.

8. The method according to claim 7, wherein: Notifying each of the first user and the one or more second users includes sending an instruction to generate a portal object to each of the first user and the one or more second users, the portal object allowing each of the first user and the one or more second users to virtually access the joint virtual space.

9. The method according to claim 1, wherein: Generate portal objects including: sending an instruction to the first user to draw a virtual doorway within the first area in the first room that allows the first user to virtually access the joint virtual space; and An instruction is sent to each of the second users to draw a virtual doorway in each of the second rooms that allows each of the second users to virtually access the joint virtual space.

10. The method according to claim 1, wherein: The joint virtual space resides in a virtual reality environment, and each of the first user and the one or more second users is a virtual user in the virtual reality environment.

Images