Method for starting or joining a visual computing session, system for participating in a visual computing session, and storage medium

Abstract

A head-mounted device system includes one or more cameras, one or more display devices, and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to perform operations for initiating or joining a joint visual computing session. The method may include receiving user input to initiate a joint session of a visual computing experience, monitoring a short-range data transmission including data indicating the existence of a current session of the visual computing experience, and providing a user input option to join the current session of the visual computing experience based on determining that the current session is in progress.

Description

[0001] Related application data

[0002] This application claims priority to U.S. patent application serial number 17 / 544,496, filed on December 7, 2021, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to augmented reality and other display devices and interfaces, and to creating and joining shared augmented reality or other visual computing sessions. Background Art

[0004] A head-mounted device may be implemented with a transparent or translucent display through which a user of the head-mounted device can view the surrounding environment. Such a device enables the user to look through the transparent or translucent display to view the surrounding environment, and also to see objects (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) generated for display that appear to be part of and / or superimposed on the surrounding environment. This is generally referred to as "augmented reality."

[0005] A head-mounted device can also completely obscure the user's field of view and display a virtual environment through which the user can move or be moved. This is often referred to as "virtual reality." As used herein, unless the context indicates otherwise, the term "augmented reality" or "AR" refers to both augmented reality and virtual reality as they are traditionally understood.

[0006] A user of the head mounted device can access a messaging application or a social networking application to view content or share content with other users of the application. In some cases, the user can view and enhance or modify real-time content or stored content. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] To easily identify the discussion of any particular element or act, the highest digit or digits in a reference number refer to the figure number in which the element is first introduced.

[0008] Figure 1 is a perspective view of a head-mounted device according to some examples.

[0009] Figure 2 Shown according to some examples Figure 1 Additional views of the head-mounted device.

[0010] Figure 3 is shown according to some examples including Figure 1 A block diagram of a networking system 300 showing details of a head mounted device.

[0011] Figure 4is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, according to some examples.

[0012] Figure 5A Aspects of the subject matter according to one embodiment are shown.

[0013] Figure 5B Aspects of the subject matter according to one embodiment are shown.

[0014] Figure 5C Aspects of the subject matter according to one embodiment are shown.

[0015] Figure 6 is a flowchart illustrating operations performed by a head mounted device system for shared AR session creation according to some examples.

[0016] Figure 7 A collaborative AR session is shown viewed by participants wearing head-mounted devices, according to some examples.

[0017] Figure 8 is a block diagram illustrating a software architecture within which the present disclosure may be implemented according to some examples.

[0018] Figure 9 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed, for causing the machine to perform any one or more of the methodologies discussed. DETAILED DESCRIPTION

[0019] Known head-mounted devices, such as AR glasses, include a transparent or translucent display, which enables a user to look through the transparent or translucent display to view the surrounding environment. Additional information or objects (e.g., virtual objects, such as 3D renderings, images, videos, text, etc.) are displayed on the display and appear as part of the surrounding environment and / or overlaid on the surrounding environment to provide an augmented reality experience for the user. The display may, for example, include a waveguide that receives a light beam from a projector, but any suitable display for presenting augmented or virtual content to the wearer may be used.

[0020] AR glasses can be used to create a shared environment in which virtual objects are positioned in fixed locations in the real world and are visible from the perspective of each of the users participating in the shared environment. Participants can then collaborate or compete in the shared environment. In some examples, a shared AR session in which multiple users can participate via client devices is hosted on a system including multiple head-mounted devices, associated mobile devices, and networked resources. The experience is synchronized and shared between users, so that the actions of one user in the shared AR session can be synchronized and broadcast to other users.

[0021] In some cases, it may be beneficial to provide a means for notifying physically present or nearby users that a shared AR session is already in progress, and offering them the option to join the already ongoing shared AR session. This can be accomplished by announcing the type and session ID of the already ongoing shared AR session via short-range data transmission from the current participants' headsets or other computing devices.

[0022] In some examples, a method of launching or joining a visual computing session performed by one or more processors is provided, the method comprising: receiving user input to launch a session of a visual computing experience; detecting a short-range data transmission including data indicating the existence of a current session of the visual computing experience; determining that the current session of the visual computing experience is in progress based on the data indicating the existence of the current session of the visual computing experience; and providing a user input option to join the current session of the visual computing experience based on the determination that the current session is in progress.

[0023] The method may also include receiving user input to begin a new session of the visual computing experience, and transmitting a short-range data transmission including data indicating the existence of a new session of the visual computing experience. The data indicating the existence of a current session of the visual computing experience may include a session identifier, and the method may also include transmitting a request to a remote server to join the current session, the request including the session identifier.

[0024] The user input option to join the current session may be selectively provided based on additional requirements being met. The additional requirements may include a degree of relationship between the user from whom the user input to initiate the session is received and the other participants in the current session.

[0025] The data indicating the existence of a current session or a new session of the visual computing experience may include a session identifier and an identifier corresponding to the visual computing experience.

[0026] In some examples, a system including one or more cameras, one or more display devices, and one or more processors is provided. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to perform operations corresponding to the method disclosed above for initiating or joining a visual computing session, including but not limited to: receiving user input to initiate a session of a visual computing experience; detecting a short-range data transmission including data indicating the existence of a current session of the visual computing experience; determining that the current session of the visual computing experience is in progress based on the data indicating the existence of the current session of the visual computing experience; and providing a user input option to join the current session of the visual computing experience based on the determination that the current session is in progress.

[0027] In some examples, a non-transitory computer-readable storage medium is provided that includes instructions that, when executed by a system that includes one or more display devices, cause the system to perform operations for initiating or joining a visual computing session corresponding to the methods disclosed above, including but not limited to: receiving user input to initiate a session of a visual computing experience; detecting a short-range data transmission including data indicating the existence of a current session of the visual computing experience; determining that the current session of the visual computing experience is in progress based on the data indicating the existence of the current session of the visual computing experience; and providing a user input option to join the current session of the visual computing experience based on the determination that the current session is in progress.

[0028] Other technical features may be readily apparent to those skilled in the art from the accompanying drawings, descriptions, and claims.

[0029] As referred to herein, the phrase "augmented reality experience" includes or refers to various image processing operations corresponding to image modification, filtering, media overlays, transformations, and the like, as further described herein. In some examples, these image processing operations provide an interactive experience of a real-world environment in which objects, surfaces, backgrounds, lighting, and the like in the real world are augmented by computer-generated perceptual information. In this context, an "augmented reality effect" includes the collection of data, parameters, and other assets required to apply a selected augmented reality experience to an image or video feed. In some examples, augmented reality effects are provided by Snap Inc. under the registered trademark LENSES.

[0030] In some examples, the augmented reality effect includes augmented reality (or "AR") content that is configured to modify or transform image data presented within the GUI of the head-mounted device in some way. For example, AR effect data can be used to perform complex additions or transformations to content images, such as adding rabbit ears to a person's head, adding a floating heart with a background color, changing the proportions of a person's features, adding augmentations to landmarks in a scene viewed on the head-mounted device, or many other such transformations. This includes both real-time modifications that modify an image as it is captured using a camera associated with the head-mounted device, which image is then displayed by the head-mounted device with the AR effect modification, as well as modifications to stored content (e.g., video clips in a gallery) that can be modified using the AR effect. Similarly, real-time video capture can be used with AR effects to show a user of the head-mounted device how a video image currently being captured by the device's sensors will modify the captured data. Such data may be displayed merely on the screen without being stored in memory, content captured by the device sensors may be recorded and stored in memory with or without AR effect modification (or both), or content captured by the device sensors may be transmitted over a network to a server or another device with AR effect modification.

[0031] Thus, AR effects and associated systems and modules for modifying content using AR effects can involve: detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.), tracking of such objects as they leave, enter, and move around the field of view in a video frame, and modification or transformation of such objects as they are tracked. In various examples, different methods for implementing such transformations can be used. For example, some examples can involve generating a 3D mesh model of one or more objects and implementing the transformation using transformations and animated textures of the models within a video. In other examples, tracking of points on an object can be used to place an image or texture (which can be two-dimensional or three-dimensional) at the tracked locations. In another example, neural network analysis of a video frame can be used to place an image, model, or texture in content (e.g., an image or video frame). Thus, AR effect data can include both the images, models, and textures used to create the transformation in the content, as well as the additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement.

[0032] Although described herein with reference to AR systems, the concepts and methods disclosed herein also apply to other joint visual computing experiences such as virtual reality or metaverse sessions.

[0033] Figure 1is a perspective view of a head-mounted device (e.g., glasses 100) according to some examples. The glasses 100 may include a frame 102 made of any suitable material, such as plastic or metal, including any suitable shape memory alloy. In one or more examples, the frame 102 includes a first or left optical element holder 104 (e.g., a display or lens holder) and a second or right optical element holder 106 connected by a bridge 112. A first or left optical element 108 and a second or right optical element 110 may be disposed within the left optical element holder 104 and the right optical element holder 106, respectively. Each of the right optical element 110 and the left optical element 108 may be a lens, a display, a display assembly, or a combination of the foregoing. Any suitable display assembly may be disposed in the glasses 100.

[0034] Frame 102 additionally includes a left arm or temple piece 120 and a right arm or temple piece 122. In some examples, frame 102 can be formed from a single piece of material to have a unitary or unitary construction.

[0035] The glasses 100 can include a computing device such as a computer 118, which can be of any suitable type to be carried by the frame 102 and, in one or more examples, can be of a suitable size and shape to be at least partially disposed in one of the temple pieces 120 or 122. The computer 118 can include one or more processors as well as memory, wireless communication circuitry, and a power source. As discussed below, the computer 118 includes low-power circuitry, high-speed circuitry, and a display processor. Various other examples can include these elements in different configurations or integrated together in different ways. Additional details of various aspects of the computer 118 can be implemented as shown in the data processor 302 discussed below.

[0036] The computer 118 additionally includes a battery 116 or other suitable portable power supply. In some examples, the battery 116 is disposed in the left temple piece 120 and is electrically coupled to the computer 118 disposed in the right temple piece 122. The glasses 100 may include a connector or port (not shown) suitable for charging the battery 116, a wireless receiver, transmitter, or transceiver (not shown), or a combination of such devices.

[0037] The glasses 100 include a camera 114. Although two cameras are depicted, other examples contemplate the use of a single camera or additional (i.e., more than two) cameras. In one or more examples, the glasses 100 include any number of input sensors or other input / output devices in addition to the camera 114. Such sensors or input / output devices may additionally include biometric sensors, position sensors, motion sensors, and the like.

[0038] The glasses 100 may also include a touchpad 124 mounted to or integrated with one or both of the left and right temple pieces 120, 122. The touchpad 124 is generally vertically aligned, in some examples approximately parallel to the user's temple. As used herein, generally vertically aligned means that the touchpad is at least more vertical than horizontal, although potentially more vertical than vertical. Additional user input may be provided via one or more buttons 126, which, in the example shown, are located on the outer upper edges of the left and right optical element holders 104, 106. The one or more touchpads 124 and buttons 126 provide a means by which the glasses 100 can receive input from the user of the glasses 100.

[0039] Figure 2 The eyeglasses 100 are shown from the wearer's perspective. Figure 1 Several elements shown in FIG have been omitted. Figure 1 As described, Figure 2 The illustrated eyewear 100 includes a left optical element 108 and a right optical element 110 secured within each of a left optical element holder 104 and a right optical element holder 106, respectively.

[0040] The glasses 100 include a front optical assembly 202 including a right projector 204 and a right near-eye display 206 , and a front optical assembly 210 including a left projector 212 and a left near-eye display 216 .

[0041] In some examples, the near-eye display is a waveguide. The waveguide includes a reflective structure or a diffractive structure (e.g., a grating and / or an optical element such as a mirror, lens, or prism). Light 208 emitted by projector 204 encounters the diffractive structure of the waveguide of near-eye display 206, which directs the light toward the user's right eye to provide an image on or in the right optical element 110 that is superimposed on the view of the real world seen by the user. Similarly, light 214 emitted by projector 212 encounters the diffractive structure of the waveguide of near-eye display 216, which directs the light toward the user's left eye to provide an image on or in the left optical element 108 that is superimposed on the view of the real world seen by the user.

[0042] However, it should be understood that other display technologies or configurations can be provided that can display images to the user in a forward field of view. For example, instead of providing a projector 204 and a waveguide, an LCD, LED or other display panel or surface can be provided instead.

[0043] In use, the wearer of the glasses 100 will be presented with information, content, and various user interfaces on the near-eye display. As described in more detail below, in addition to the associated devices such as Figure 3 In addition to providing voice input or touch input on the client device 328 as shown, the user can then interact with the glasses 100 using the touchpad 124 and / or buttons 126.

[0044] Figure 3 is a block diagram illustrating a networked system 300 including details of glasses 100 according to some examples.

[0045] The networked system 300 includes the glasses 100, a client device 328, and a server system 332. The client device 328 can be a smartphone, a tablet computer, a tablet phone, a laptop computer, an access point, or any other such device capable of connecting to the glasses 100 using both a low-power wireless connection 336 and a high-speed wireless connection 334. The client device 328 is connected to the server system 332 via a network 330. The network 330 can include any combination of wired and wireless connections. The server system 332 can be one or more computing devices that are part of a service or network computing system. The client device 328 and any elements of the server system 332 and network 330 can use Figure 8 and Figure 9 The details of the software architecture 804 or machine 900 described in are implemented.

[0046] The glasses 100 include a data processor 302, a display 310, one or more cameras 308, and additional input / output elements 316. The input / output elements 316 may include a microphone, an audio speaker, a biometric sensor, additional sensors, or additional display elements integrated with the data processor 302. Figure 8 and Figure 9 Examples of input / output elements 316 are further discussed. For example, input / output elements 316 may include any I / O component 906, including output component 928, motion component 936, etc. Figure 2 Examples of display 310 are discussed in . In the particular examples described herein, display 310 includes a display for each of the user's left and right eyes.

[0047] Data processor 302 includes image processor 306 (eg, video processor), GPU and display driver 338, tracking module 340, interface 312, low power circuitry 304, and high speed circuitry 320. The components of data processor 302 are interconnected by bus 342.

[0048] Interface 312 refers to any source of user commands provided to data processor 302. In one or more examples, interface 312 is a physical button that, when pressed, transmits a user input signal from interface 312 to low-power processor 314. Low-power processor 314 can process pressing such a button and then immediately releasing it as a request to capture a single image, or vice versa. Low-power processor 314 can process pressing such a button for a first period of time as a request to capture video data while the button is pressed and to stop video capture when the button is released, with the video captured while the button is pressed being stored as a single video file. Alternatively, pressing the button for a longer period of time can capture a still image. In some examples, interface 312 can be any mechanical switch or physical interface capable of accepting user input associated with a data request from camera 308. In other examples, interface 312 can have a software component or be associated with a command received wirelessly from another source, such as client device 328.

[0049] Image processor 306 includes circuitry for receiving signals from camera 308 and processing those signals from camera 308 into a format suitable for storage in memory 324 or transmission to client device 328. In one or more examples, image processor 306 (e.g., a video processor) includes a microprocessor integrated circuit (IC) customized for processing sensor data from camera 308, and volatile memory used by the microprocessor during operation.

[0050] The low power circuitry 304 includes a low power processor 314 and a low power wireless circuitry 318. These elements of the low power circuitry 304 can be implemented as separate elements or can be implemented on a single IC as part of a single system on a chip. The low power processor 314 includes logic for managing the other elements of the glasses 100. As described above, for example, the low power processor 314 can accept user input signals from the interface 312. The low power processor 314 can also be configured to receive input signals or instruction communications from the client device 328 via the low power wireless connection 336. The low power wireless circuitry 318 includes circuit elements for implementing a low power wireless communication system. Bluetooth TM Smart, also known as Bluetooth TM Low power consumption is a standard implementation of a low power wireless communication system that may be used to implement the low power wireless circuitry 318. In other examples, other low power communication systems may be used.

[0051] High-speed circuitry 320 includes a high-speed processor 322, memory 324, and high-speed wireless circuitry 326. High-speed processor 322 can be any processor capable of managing high-speed communications and operations of any general-purpose computing system required by data processor 302. High-speed processor 322 includes the processing resources required to manage high-speed data transmission over high-speed wireless connection 334 using high-speed wireless circuitry 326. In some examples, high-speed processor 322 executes an operating system such as the LINUX operating system or a program such as the UNIX operating system. Figure 8 The high-speed processor 322, which executes the software architecture of the data processor 302, manages data transmission with the high-speed wireless circuit system 326, in addition to any other responsibilities. In a specific example, the high-speed wireless circuit system 326 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, which is also referred to herein as Wi-Fi. In other examples, the high-speed wireless circuit system 326 can implement other high-speed communication standards.

[0052] Memory 324 includes any storage device capable of storing camera data generated by camera 308 and image processor 306. Although memory 324 is shown as being integrated with high-speed circuitry 320, in other examples, memory 324 may be a separate, independent element of data processor 302. In certain such examples, electrical wiring may provide a connection from image processor 306 or low-power processor 314 to memory 324 through a chip that includes high-speed processor 322. In other examples, high-speed processor 322 may manage addressing of memory 324 so that low-power processor 314 will direct high-speed processor 322 whenever a read or write operation involving memory 324 is required.

[0053] The tracking module 340 estimates the pose of the glasses 100. For example, the tracking module 340 uses image data and corresponding inertial data from the camera 308 and the positioning component 940, as well as GPS data, to track the position and determine the pose of the glasses 100 relative to a reference frame (e.g., a real-world environment). The tracking module 340 continuously collects and uses updated sensor data describing the movement of the glasses 100 to determine an updated three-dimensional pose of the glasses 100, which indicates changes in relative position and orientation relative to physical objects in the real-world environment. The tracking module 340 allows the glasses 100 to visually place virtual objects relative to physical objects within the user's field of view via the display 310.

[0054] The GPU and display driver 338 can use the pose of the glasses 100 to generate frames of virtual content or other content to be presented on the display 310 when the glasses 100 are operating in a conventional augmented reality mode. In this mode, the GPU and display driver 338 generate updated frames of virtual content based on the updated three-dimensional pose of the glasses 100, which reflects changes in the user's position and orientation relative to physical objects in the user's real environment.

[0055] One or more functions or operations described herein may also be performed in an application resident on the glasses 100 or on the client device 328 or on a remote server. For example, one or more functions or operations described herein may be performed by one of the applications 806, such as the messaging application 846.

[0056] Figure 4is a block diagram illustrating an example messaging system 400 for exchanging data (e.g., messages and associated content) over a network. The messaging system 400 includes multiple instances of a client device 328, each of which hosts several applications including a messaging client 402 and other applications 404. Each messaging client 402 is communicatively coupled to other instances of the messaging client 402 (e.g., hosted on respective other client devices 328), a messaging server system 406, and a third-party server 408 via a network 330 (e.g., the Internet). The messaging client 402 can also communicate with the locally hosted application 404 using an application program interface (API).

[0057] The messaging clients 402 are able to communicate and exchange data with other messaging clients 402 and with a messaging server system 406 via the network 330. The data exchanged between the messaging clients 402 and between the messaging clients 402 and the messaging server system 406 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0058] The messaging server system 406 provides server-side functionality to certain messaging clients 402 via the network 330. Although certain functionality of the messaging system 400 is described herein as being performed by either the messaging client 402 or the messaging server system 406, the location of certain functionality within the messaging client 402 or within the messaging server system 406 may be a matter of design choice. For example, it may be technically preferable to initially deploy certain technologies and functionality within the messaging server system 406 but to later migrate such technologies and functionality to a messaging client 402 where the client device 328 has sufficient processing power.

[0059] The messaging server system 406 supports various services and operations provided to the messaging clients 402. Such operations include transmitting data to the messaging clients 402, receiving data from the messaging clients 402, and processing data generated by the messaging clients 402. By way of example, this data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the messaging system 400 is activated and controlled by functionality available through the user interface (UI) of the messaging clients 402.

[0060] Turning now specifically to the messaging server system 406, an application program interface (API) server 410 is coupled to an application server 414 and provides a programming interface to the application server 414. The application server 414 is communicatively coupled to a database server 416, which facilitates access to a database 420 that stores data associated with messages processed by the application server 414. Similarly, a web server 424 is coupled to the application server 414 and provides a web-based interface to the application server 414. To this end, the web server 424 handles incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0061] The application program interface (API) server 410 receives and transmits message data (e.g., commands and message payloads) between the client device 328 and the application server 414. Specifically, the application program interface (API) server 410 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client 402 to activate the functionality of the application server 414. The application program interface (API) server 410 exposes various functions supported by the application server 414, including: account registration; login functionality; sending messages from a particular messaging client 402 to another messaging client 402 via the application server 414, sending media files (e.g., images or videos) from the messaging client 402 to the messaging server 412, and possible access by another messaging client 402; setting up collections of media data (e.g., stories); retrieving a friend list of a user of the client device 328; retrieving such collections; retrieving messages and content; adding and removing entities (e.g., friends) to an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to the messaging client 402).

[0062] The application server 414 hosts several server applications and subsystems, including, for example, a messaging server 412, an image processing server 418, and a social network server 422. The messaging server 412 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 402. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to the messaging client 402. Given the hardware requirements for such processing, other processor- and memory-intensive processing of data can also be performed on the server side by the messaging server 412.

[0063] The application server 414 also includes an image processing server 418 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 412.

[0064] The social network server 422 supports various social networking functions and services and makes these functions and services available to the messaging server 412. To do so, the social network server 422 maintains and accesses an entity graph within the database 420. Examples of functions and services supported by the social network server 422 include identifying other users in the messaging system 400 with whom a particular user has relationships or who the particular user is "following," and also identifying interests and other entities of a particular user.

[0065] The messaging client 402 can notify the user of the client device 328 or other users related to such user (e.g., "friends") of activities occurring in a shared or shareable session. For example, the messaging client 402 can provide participants in a conversation (e.g., a chat session) in the messaging client 402 with notifications related to current or recent use of a game by one or more members of a user group. One or more users can be invited to join an active session or initiate a new session. In some examples, a shared session can provide a shared augmented reality experience in which multiple people can collaborate or participate.

[0066] Figure 5A A user interface 500 is shown that is displayed to a user of glasses 100 according to some examples. User interface 500 includes a carousel 502 of icons 504 corresponding to augmented reality experiences that can be selected by the user. Carousel 502 can be scrolled left or right to move one of icons 504 to a central position where the augmented reality experience corresponding to that icon can be activated, for example, by a tap input on one of touchpads 124. Scrolling of carousel 502 can be achieved, for example, by swiping forward or backward on one of touchpads 124.

[0067] In the illustrated user interface 500, a selection icon 506 in a central position in the carousel 502 causes its title 510 to be displayed below the carousel, identifying the corresponding augmented reality experience. The fact that the selected augmented reality experience can be provided in a shared session with other users is indicated by a display element 508, such as a symbol, showing the head and shoulders of more than one person. Activating an augmented reality experience that can be provided in a shared session will transition the user interface to either a shared session or a short-range announcement, depending on whether an existing shared session is occurring nearby (as determined by detecting short-range announcements as discussed below). Figure 5B Medium or Figure 5C The user interface shown in .

[0068] Figure 5B A user interface 512 is shown that is displayed to a user of glasses 100 when a shared session augmented reality experience is selected by the user but no instance of a shared session for the same augmented reality experience exists nearby. The title 510 of the augmented reality experience is displayed, followed by a "Select a shared room to join" prompt 516. Below prompt 516 is a single option, a new room selection option 514. A visual enhancement 518, such as a highlight or surrounding brackets, may be provided to indicate that the option will be selected upon receiving user input, such as a tap on one of the touchpads 124.

[0069] Figure 5B A user interface 520 is shown that is displayed to the user of glasses 100 when a shared session augmented reality experience is selected by the user and a shared session instance of the same augmented reality experience exists nearby. The title 510 of the augmented reality experience is displayed, followed by a "Select a shared room to join" prompt 516. Below prompt 516 is a new room selection option 514, as before, and one or more additional options corresponding to nearby shared sessions of the same augmented reality experience. In the example shown, a join an existing session selection option 522 for a shared session initiated by user Terek is available for selection. A visual enhancement 518, such as a highlight or surrounding brackets, may be provided to indicate that the option will be selected upon receiving user input, such as a tap on one of the touchpads 124. The user can scroll through the available options, for example, by swiping forward or backward on one of the touchpads 124.

[0070] Figure 6 6 is a flowchart 600 illustrating operations performed by glasses 100 to provide or facilitate a shared augmented reality session, according to some examples. For illustrative purposes, the operations of flowchart 600 are described herein as occurring serially or linearly. However, multiple operations of flowchart 600 may occur in parallel. Additionally, the operations of flowchart 600 need not be performed in the order shown and / or one or more blocks of flowchart 600 need not be performed and / or may be replaced by other operations.

[0071] Figure 6The operations shown in the flowchart 600 will generally be performed on the data processor 302 and related hardware in or associated with the glasses 100. For the purpose of clarity, the flowchart 600 is discussed herein with reference to such an example. Various implementations are of course possible in which some of the operations described occur in an application such as the messaging application 846 in the client device 328, on the server system 332, or in which one application on the client device 328 calls another application or SDK to obtain the desired functionality. In some examples, the operations are performed in conjunction with the messaging application 846 running on the client device 328 and the related hardware and data processor 302 in or associated with the glasses 100.

[0072] The method begins at operation 602 by receiving a user input requesting that an augmented reality experience allowing multiple participants be initiated by the corresponding glasses 100 or client device 328, e.g., as described with reference to Figure 5A As discussed. In operation 604, the glasses or client device 328 begins monitoring transmissions of existing sessions of the same AR experience occurring nearby. This is achieved by the glasses 100 or client device 328 monitoring short-range transmissions that include relevant identification information, such as an identifier for the AR experience and a session identifier. Any potentially relevant identification information is extracted from any detected short-range transmissions. The short-range transmissions can be Bluetooth or Bluetooth LE broadcasts or beacons, but any other suitable short-range data transmission method (IR, ultrasonic, RF, etc.) can be used. The range of the data transmission method and protocol should be sufficient to cover a reasonable area around the glasses 100 or client device 328 within which potential participants are likely to be located or physically present, or can become physically present reasonably quickly.

[0073] In operation 606, the glasses 100 or the client device 328 determines whether an existing shared session exists based on the presence or absence of short-range transmissions identifying an existing session of the same AR experience. The presence of an existing session is determined by verifying that the identification information in any received and potentially relevant short-range transmission is the same as the identification information of the AR experience for which the user input was received in operation 602. If no potentially relevant short-range transmission is received, or if the identification information in any received short-range transmission does not match the identification information of the AR experience for which the user input was received in operation 602, then it is determined that there is no existing session.

[0074] If there is no existing session nearby, the method proceeds to operation 608 where an option to start a new sharing session is displayed, e.g., Figure 5B After receiving input to start a new session in operation 610 , a new session of AR experience is started in operation 612 .

[0075] Then, in operation 614, the glasses 100 or the client device 328 begins transmitting short-range transmissions including relevant identifying information such as an identifier for the AR experience and a session identifier for the new session. These transmissions continue until the user exits the sharing session.

[0076] The creation and management of the actual session itself occurs in the application server 414 and includes the assignment of a unique ID to the shared session, which is provided to the client device 328 initiating the new session. Communications between the participants' client devices 328, with or without the application server 414 acting as an intermediary, also primarily occur over the network 330. Thus, broadcasting identification information using short-range transmissions provides for out-of-band session initiation that exploits the short-range nature of the transmissions to identify the existence of a shared session that can be joined by nearby potential participants.

[0077] If it is determined in operation 606 that an existing session is occurring nearby, the method proceeds to operation 616 where an option to start a new sharing session or to join one or more existing sessions is displayed, e.g., Figure 5C As shown in . As determined in operation 620, after receiving the input to start a new session in operation 618, a new session of the AR experience is started in operation 622. Then, in operation 624, the glasses 100 or the client device 328 begins transmitting short-range transmissions including relevant identification information such as an identifier of the AR experience and a session identifier of the new session. These transmissions continue until the user exits the new session.

[0078] Upon receiving input to join an existing session in operation 618 , as determined in operation 620 , the selected existing session is joined in operation 626 .

[0079] In some examples, the name of the creator of the sharing session and other potentially relevant information, such as permissions or restrictions regarding the joint session, are included in the short-range transmission. In other examples, upon receiving the session identifier, additional information can be retrieved from the messaging server system 406. This information may include the name of the creator of the sharing session and the permissions or requirements.

[0080] For example, the additional information may specify that a certain degree of relationship exists between the creator or other participant in the joint session and the user of client device 328 in order to provide the user of client device 328 with the option to join an existing session. For example, the degree of relationship may specify that only friends of the creator of the shared session are allowed, or friends of the creator (or other current participants) and friends of friends. In another example, a certain level of hardware may be required to join the session. In the event that there are any restrictions on who or what can join the shared session, this information is extracted from the short-range transmission using the session identifier or retrieved from the messaging server system 406, and any restrictions or limitations are checked as part of operation 606 to ensure that only sessions that can be joined are displayed in operation 616.

[0081] Figure 7 The sharing session 700 is shown as seen by a first participant wearing a head mounted device such as glasses 100. As discussed above, the sharing session 700 has been initiated and joined. Figure 7 Items in the real world seen by the first participant are included, including the second participant 706, the third participant 710, the table 714, and the registration marker 718. Superimposed on the real-world elements on the display 310 of the first participant's glasses 100 is a shared AR object 702, which includes a bottom AR ring 704, a middle AR ring 708, a top AR ring 712, and an AR interface 716.

[0082] Each participant has a view of the shared AR object 702 that depends on the position of their respective glasses 100. Compared to the shared session 700 viewed by the first participant as shown, the second participant 706 is looking at the right-hand side of the shared AR object 702, while the third participant 710 is looking at the left-hand side of the shared AR object 702. If the first participant moves to the position of the third participant 710, the shared AR object 702 will rotate in the first participant's field of view to appear fixed relative to the table 714, as if the shared AR object 702 were real rather than a virtual object. Depending on the implementation, each participant can manipulate the shared AR object 702 for a variety of purposes, such as music composition, game playing, or other collaborative or participatory AR activities.

[0083] The creation of such a shared reference frame can be accomplished in a variety of ways. In some examples, registration markers (such as registration marker 718) are placed in relevant locations, such as where the shared AR object 702 will be used (in Figure 7714 in the table). One of the glasses 100 (e.g., the glasses of the creator of the sharing session) captures an image of the registration mark, which is transmitted to the glasses being worn by the other participants. The other glasses also each detect and capture an image of the registration mark. The other glasses 100 then each determine a transformation between the first image and the image captured by each of the glasses 100. The transformation is then used to determine a common coordinate system, and the common coordinate system is used to generate the sharing session 700. The registration mark remains visible to each of the glasses 100 and can be used as a reference to update the pose of each of the glasses 100 as the participants move around in the environment. The view of the shared AR object 702 is then updated to reflect each new pose of the glasses 100.

[0084] Other methods can be used to generate a shared reference frame, including, for example, 3D reconstruction and visual inertial odometry based on images captured by each pair of glasses 100 and based on signals or data received from inertial sensors or other positioning sensors located in each pair of glasses. Features identified using the 3D reconstruction can also be compared with existing point cloud models to locate and determine the pose of each pair of glasses in the real-world environment.

[0085] Figure 8 800 is a block diagram illustrating a software architecture 804 that can be installed on any one or more of the devices described herein. The software architecture 804 is supported by hardware, such as a machine 802, which includes a processor 820, a memory 826, and I / O components 838. In this example, the software architecture 804 can be conceptualized as a stack of layers, each of which provides specific functionality. The software architecture 804 includes layers such as an operating system 812, libraries 808, a framework 810, and applications 806. In operation, the applications 806 invoke API calls 850 through the software stack and receive messages 852 in response to the API calls 850.

[0086] The operating system 812 manages hardware resources and provides common services. The operating system 812 includes, for example, a kernel 814, services 816, and drivers 822. The kernel 814 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 814 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. Services 816 can provide other common services to other software layers. Drivers 822 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 822 may include display drivers, camera drivers, or Low-power drivers, Flash drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), drivers, audio drivers, power management drivers, etc.

[0087] The libraries 808 provide a low-level common infrastructure used by the applications 806. The libraries 808 may include system libraries 818 (e.g., C standard libraries) that provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the libraries 808 may include API libraries 824, such as media libraries (e.g., libraries for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., OpenGL framework for rendering graphical content on a display in two dimensions (2D) and three dimensions (3D), database libraries (e.g., SQLite providing various relational database functions), web libraries (e.g., WebKit providing web browsing functions), etc. The libraries 808 may also include various other libraries 828 to provide many other APIs to the applications 806.

[0088] The framework 810 provides a high-level common infrastructure used by the applications 806. For example, the framework 810 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. The framework 810 can provide a wide range of other APIs that can be used by the applications 806, some of which may be specific to a particular operating system or platform.

[0089] In an example, the applications 806 may include a home application 836, a contacts application 830, a browser application 832, a book reader application 834, a location application 842, a media application 844, a messaging application 846, a game application 848, and a variety of other applications such as third-party applications 840. The applications 806 are programs that perform functions defined in the program. Various programming languages ​​may be used to create one or more of the applications 806 structured in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C or assembly language). In a specific example, the third-party applications 840 (e.g., those written by entities other than the vendor of a particular platform using ANDROID) may be used to create a third-party application 840. TM or IOS TM Software Development Kit (SDK) can be used to develop applications on platforms such as IOS TM ANDROID TM 、 Mobile software running on the mobile operating system of the phone or another mobile operating system. In this example, third-party applications 840 can call API calls 850 provided by the operating system 812 to facilitate the functions described herein.

[0090] Figure 9 The illustrative embodiment of the present invention is a diagrammatic representation of a machine 900 in which instructions 910 (e.g., software, programs, applications, applet, apps, or other executable code) may be executed to cause the machine 900 to perform any one or more of the methodologies discussed herein. For example, the instructions 910 may cause the machine 900 to perform any one or more of the methodologies described herein. The instructions 910 transform a general-purpose, unprogrammed machine 900 into a specialized machine 900 that is programmed to perform the functions described and illustrated in the manner described. The machine 900 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 900 may operate in the capacity of a server or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular phone, a smart phone, a mobile device, a head-mounted device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing, sequentially or otherwise, the instructions 910 specifying actions to be taken by the machine 900. Furthermore, while only a single machine 900 is shown, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute the instructions 910 to perform any one or more of the methodologies discussed herein.

[0091] The machine 900 may include a processor 902, a memory 904, and an I / O component 906 that may be configured to communicate with each other via a bus 944. In an example, the processor 902 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 908 that executes instructions 910 and a processor 912. The term "processor" is intended to include a multi-core processor that may include two or more independent processors (sometimes referred to as "cores") that may execute instructions concurrently. Although Figure 9Multiple processors 902 are shown, but the machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0092] The memory 904 includes a main memory 914, a static memory 916, and a storage unit 918, all of which are accessible by the processor 902 via the bus 944. The main memory 904, the static memory 916, and the storage unit 918 store instructions 910 that implement any one or more of the methods or functions described herein. During execution of the instructions 910 by the networked system 300, the instructions 910 may also reside, in whole or in part, within the main memory 914, within the static memory 916, within the machine-readable medium 920, within the storage unit 918, within at least one of the processors 902 (e.g., within a cache memory of the processor), or within any suitable combination thereof.

[0093] The I / O components 906 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 906 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It is understood that the I / O components 906 may include Figure 9 Many other components are not shown in the drawings. In various examples, the I / O components 906 may include output components 928 and input components 932. The output components 928 may include visual components (e.g., displays such as plasma display panels (PDPs), light emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. The input components 932 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing instruments), tactile input components (e.g., physical buttons, touch screens that provide location and / or force of touches or touch gestures, or other tactile input components), audio input components (e.g., microphones), etc.

[0094] In another example, the I / O component 906 may include a biometric component 934, a motion component 936, an environmental component 938, or a positioning component 940, as well as various other components. For example, the biometric component 934 includes a component for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component 936 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environment component 938 includes, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting concentrations of hazardous gases or measuring pollutants in the atmosphere for safety purposes), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The positioning component 940 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting air pressure that can provide altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0095] A variety of technologies can be used to achieve communication. The I / O component 906 also includes a communication component 942 that is operable to couple the networked system 300 to the network 922 or the device 924 via coupling 930 and coupling 926, respectively. For example, the communication component 942 may include a network interface component or another suitable device that interfaces with the network 922. In other examples, the communication component 942 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low power consumption), Device 924 may be another machine or any of a variety of peripheral devices (eg, a peripheral device coupled via USB).

[0096] In addition, the communication component 942 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 942 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting the following: a one-dimensional barcode, such as a universal product code (UPC) barcode; a multi-dimensional barcode, such as a Quick Response (QR) code, an Aztec code, a data matrix, a data glyph, a MaxiCode, a PDF417, a Hypercode, a UCC RSS-2D barcode, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 942, such as a location via Internet Protocol (IP) geolocation, a location via Internet Protocol (IP), ... Location of signal triangulation, location of NFC beacon signals that can indicate a specific location via detection, etc.

[0097] Various memories (e.g., memory 904, main memory 914, static memory 916, and / or memory of processor 902) and / or storage unit 918 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 910), when executed by processor 902, cause various operations to implement the disclosed examples.

[0098] Instructions 910 may be transmitted or received over network 922 via a network interface device (e.g., a network interface component included in communications component 942) using a transmission medium and using any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 910 may be transmitted or received to device 924 via coupling 926 (e.g., a peer-to-peer coupling) using a transmission medium.

[0099] "Carrier signal" refers to any intangible medium that can store, encode, or carry instructions for execution by a machine and includes digital or analog communication signals or other intangible media to facilitate communication of such instructions. Instructions may be transmitted or received over a network using a transmission medium via a network interface device.

[0100] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, desktop computer, laptop computer, portable digital assistant (PDA), smartphone, tablet computer, ultrabook, netbook, laptop computer, multiprocessor system, microprocessor-based or programmable consumer electronics, game console, set-top box, or any other communications device that a user may use to access a network.

[0101] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a Plain Old Telephone Service (POTS) network, a cellular telephone network, a wireless network, The coupling may be a network, another type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) networks, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long distance protocols, or other data transmission technologies.

[0102] "Component" refers to a device, physical entity or logic with boundaries defined by function or subroutine calls, branch points, APIs or other technologies provided for partitioning or modularizing specific processing or control functions. A component can be combined with other components via its interface to perform machine processing. A component can be a packaged functional hardware unit designed for use with other components, and is generally part of a program that performs a specific function of the associated function. A component can constitute a software component (e.g., a code implemented on a machine-readable medium) or a hardware component. A "hardware component" is a tangible unit that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., an independent computer system, a client computer system or a server computer system) or one or more hardware components (e.g., a processor or a processor group) of a computer system can be configured to operate to perform certain operations as described herein by software (e.g., an application or an application portion). Hardware components can also be implemented mechanically, electronically or in any suitable combination thereof. For example, a hardware component can include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component can be a special-purpose processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The hardware component can also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or specific component of a machine) that is uniquely customized to perform the configured function, rather than a general-purpose processor. It will be understood that the decision to mechanically implement the hardware component in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., configured by software) circuit system can be driven due to cost and time considerations. Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to include tangible entities, i.e., entities that are physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an example in which a hardware component is temporarily configured (e.g., programmed), each of the hardware components does not need to be configured or instantiated at any one time. For example, in the case where a hardware component includes a general-purpose processor that is configured by software to become a special-purpose processor, the general-purpose processor can be configured as a different special-purpose processor (e.g., including different hardware components) at different times. The software configures one or more specific processors accordingly, such as to constitute a specific hardware component at one time and to constitute different hardware components at different times. A hardware component can provide information to other hardware components and receive information from other hardware components. Thus, the described hardware components can be considered to be communicatively coupled.In the case of having multiple hardware components at the same time, communication can be achieved by signal transmission (for example, by appropriate circuits and buses) between two or more hardware components or among two or more hardware components. In the example that multiple hardware components are configured or instantiated at different times, the communication between such hardware components can be achieved, for example, by storing information in a memory structure that multiple hardware components can access and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to its communication ground. Then, other hardware components can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (for example, the collection of information). The various operations of the example methods described herein can be performed at least in part by temporary configuration (for example, by software) or permanently configured to perform one or more processors of the related operations. Whether it is temporary configuration or permanent configuration, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein specific one or more processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors or the parts implemented by the processor. In addition, one or more processors can also operate to support the execution of the related operations in the " cloud computing " environment or operate as " software as a service " (SaaS). For example, at least some of the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (for example, the Internet) and via one or more appropriate interfaces (for example, API). The execution of certain operations in the operation can be distributed between the processors, not only resides in a single machine, but also deployed across multiple machines. In some examples, the parts implemented by the processor or the processor can be located in a single geographical location (for example, in a home environment, an office environment or a server farm). In other examples, the parts implemented by the processor or the processor can be distributed across multiple geographical locations.

[0103] "Computer-readable media" refers to both machine storage media and transmission media. Thus, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and are used interchangeably in this disclosure.

[0104] An "ephemeral message" is a message that can be accessed for a limited duration. An ephemeral message can be text, an image, a video, or the like. The access period for an ephemeral message can be set by the sender. Alternatively, the access period can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is ephemeral.

[0105] “Machine storage medium” refers to a single or multiple storage devices and / or media (e.g., a centralized or distributed database, and / or associated caches and servers) that store executable instructions, routines, and / or data. Thus, the term should be taken to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage medium,” “device storage medium,” and “computer storage medium” mean the same thing and may be used interchangeably in this disclosure. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are encompassed by the term “signal media.”

[0106] A "processor" refers to any circuit or virtual circuit (a physical circuit emulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine code," etc.) and produces corresponding output signals that are applied to operate a machine. For example, a processor may be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.

[0107] "Signal medium" refers to any intangible medium that is capable of storing, encoding, or carrying instructions for execution by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be construed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0108] Changes and modifications may be made to the disclosed examples without departing from the scope of the present disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure as expressed in the following claims.

Claims

1. A method for initiating or joining a visual computing session, performed by one or more processors, comprising: receiving user input to initiate a session of a visual computing experience; detecting a short-range data transmission including data indicative of the presence of a current session of the visual computing experience; determining, based on the data indicating that a current session of the visual computing experience exists, that the current session of the visual computing experience is ongoing; as well as Based on determining that the current session is ongoing, providing a user input option to join the current session of the visual computing experience.

2. The method according to claim 1, further comprising: receiving user input to begin a new session of the visual computing experience; as well as The transmission includes a short-range data transmission of data indicating the existence of a new session of the visual computing experience.

3. The method according to claim 2, wherein: The data indicating the existence of a current session or a new session of the visual computing experience includes an identifier corresponding to the visual computing experience.

4. The method according to claim 3, wherein: The data indicating the existence of a current session or a new session of the visual computing experience includes a session identifier.

5. The method according to claim 1, wherein The data indicating that a current session of the visual computing experience exists includes a session identifier, the method further comprising: A request to join the current session is transmitted to a remote server, the request including the session identifier.

6. The method according to claim 1, further comprising: The user input option to join the current session is selectively provided based on additional requirements being met.

7. The method according to claim 6, wherein: The additional requirements include the degree of relationship between the user from whom the user input initiating the session was received and the other participants in the current session.

8. A system for participating in a visual computing session, comprising: one or more camera devices; one or more display devices; one or more processors; as well as a memory storing instructions that, when executed by the one or more processors, configure the system to perform operations of initiating or joining a visual computing session, the operations comprising: receiving user input to initiate a session of a visual computing experience; detecting a short-range data transmission including data indicative of the presence of a current session of the visual computing experience; Based on the data indicating that a current session of the visual computing experience exists, determining that a current session of the visual computing experience is ongoing; and Based on determining that the current session is ongoing, providing a user input option to join the current session of the visual computing experience.

9. The system according to claim 8, wherein: The operations further include: receiving user input to begin a new session of the visual computing experience; and The transmission includes a short-range data transmission of data indicating the existence of a new session of the visual computing experience.

10. The system according to claim 9, wherein: The data indicating the existence of a current session or a new session of the visual computing experience includes an identifier corresponding to the visual computing experience.

11. The system according to claim 9, wherein: The data indicating the existence of a current session or a new session of the visual computing experience includes a session identifier.

12. The system according to claim 8, wherein: The data indicating that a current session of the visual computing experience exists includes a session identifier, the operations further comprising: A request to join the current session is transmitted to a remote server, the request including the session identifier.

13. The system according to claim 8, wherein: The operations further include: The user input option to join the current session is selectively provided based on additional requirements being met.

14. The system according to claim 13, wherein: The additional requirements include the degree of relationship between the user from whom the user input initiating the session was received and the other participants in the current session.

15. A non-transitory computer-readable storage medium comprising instructions that, when executed by a system comprising one or more display devices, cause the system to perform operations of launching or joining a visual computing session, the operations comprising: receiving user input to initiate a session of a visual computing experience; detecting a short-range data transmission including data indicative of the presence of a current session of the visual computing experience; determining, based on the data indicating that a current session of the visual computing experience exists, that the current session of the visual computing experience is ongoing; as well as Based on determining that the current session is ongoing, providing a user input option to join the current session of the visual computing experience.

16. The non-transitory computer-readable storage medium of claim 15, wherein: The operations further include: receiving user input to begin a new session of the visual computing experience; and The transmission includes a short-range data transmission of data indicating the existence of a new session of the visual computing experience.

17. The non-transitory computer-readable storage medium of claim 16, wherein: The data indicating the existence of a current session or a new session of the visual computing experience includes a session identifier.

18. The non-transitory computer-readable storage medium of claim 15, wherein: The data indicating that a current session of the visual computing experience exists includes a session identifier, the operations further comprising: A request to join the current session is transmitted to a remote server, the request including the session identifier.

19. The non-transitory computer-readable storage medium of claim 15, wherein: The operations further include: The user input option to join the current session is selectively provided based on additional requirements being met.

20. The non-transitory computer-readable storage medium of claim 19, wherein: The additional requirements include the degree of relationship between the user from whom the user input initiating the session was received and the other participants in the current session.

Images