Method of initiating or joining visual computing session, system participating in visual computing session, and storage medium

Abstract

The invention discloses a method for starting or joining a visual computing session, a system for participating in the visual computing session and a storage medium. A method performed by one or more processors of initiating or joining a collaborative computing session on a first device includes: receiving, at the first device, a user input to initiate a new collaborative computing session; determining that a current collaborative computing session of the same type is proceeding on a physically existing second device; and based on determining that the current collaborative computing session is proceeding on the physically existing second device, providing, at the first device, a user input option to join the current collaborative computing session using the first device instead of starting a new collaborative computing session.

Description

[0001] This application is a divisional application of Chinese patent application No. 202280080744.1, entitled "Method for initiating or joining a visual computing session and system for participating in a visual computing session, and storage medium". The international filing date of the parent application is November 30, 2022, and the international application number is PCT / US2022 / 080702.

[0002] Relevant application data

[0003] This application claims priority to U.S. Patent Application Serial No. 17 / 544,496, filed on December 7, 2021, which is incorporated herein by reference in its entirety. Technical Field

[0004] This disclosure generally relates to augmented reality and other display devices and interfaces, and to the creation and joining of shared augmented reality or other visual computing sessions. Background Technology

[0005] Head-mounted devices can be implemented with transparent or semi-transparent displays through which users can view their surroundings. Such devices allow users to view their surroundings through these displays, and also to see objects generated for display that appear as part of and / or superimposed on the surrounding environment (e.g., virtual objects such as 3D renderings, images, videos, text, etc.). This is often referred to as "augmented reality."

[0006] Headsets can also completely obscure the user's field of view and display a virtual environment that the user can move or be moved by. This is often referred to as "virtual reality." As used herein, unless the context otherwise indicates, the terms "augmented reality" or "AR" refer to both augmented reality and virtual reality as conventionally understood.

[0007] Users of head-mounted devices can access messaging or social networking applications to view or share content with other users within the application. In some cases, users may be able to view, enhance, or modify live or stored content. Summary of the Invention

[0008] According to one aspect of this disclosure, a method for initiating or joining a collaborative computing session on a first device, executed by one or more processors, is provided, comprising: receiving user input at the first device to initiate a new collaborative computing session; determining that a current collaborative computing session of the same type is taking place on a physically existing second device; and, based on the determination that the current collaborative computing session is taking place on the physically existing second device, providing at the first device a user input option to join the current collaborative computing session using the first device instead of initiating the new collaborative computing session.

[0009] According to another aspect of this disclosure, a system is provided, comprising: a first device including one or more camera devices and one or more display devices; one or more processors; and a memory storing instructions, which, when executed by the one or more processors, configure the system to perform operations of initiating or joining a collaborative computing session, the operations including: receiving user input at the first device to initiate a new collaborative computing session; determining that a current collaborative computing session of the same type is taking place on a physically existing second device; and, based on the determination that the current collaborative computing session is taking place on the physically existing second device, providing at the first device a user input option to join the current collaborative computing session using the first device instead of initiating the new collaborative computing session.

[0010] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided, the instructions, when executed by a system including a first device comprising one or more display devices, causing the system to perform operations of initiating or joining a collaborative computing session, the operations including: receiving user input at the first device to initiate a new collaborative computing session; determining that a current collaborative computing session of the same type is taking place on a physically existing second device; and, based on the determination that the current collaborative computing session is taking place on the physically existing second device, providing at the first device an option to join the current collaborative computing session using the first device instead of initiating the new collaborative computing session. Attached Figure Description

[0011] To facilitate the identification of any particular element or action in discussion, one or more of the highest-order digits in the reference numerals indicate the drawing number in which the element was first introduced.

[0012] Figure 1 These are perspective views of some example head-mounted devices.

[0013] Figure 2 The following are examples. Figure 1 Another view of the head-mounted device.

[0014] Figure 3 This is shown based on some examples, including Figure 1 A block diagram of the detailed networking system of the head-mounted device 300.

[0015] Figure 4 It is a diagrammatic representation of a networked environment in which the content of this disclosure can be deployed, based on some examples.

[0016] Figure 5A An aspect of the subject matter according to one implementation is shown.

[0017] Figure 5B An aspect of the subject matter according to one implementation is shown.

[0018] Figure 5C An aspect of the subject matter according to one implementation is shown.

[0019] Figure 6 This is a flowchart illustrating operations performed by a head-mounted device system for creating a shared AR session, based on some examples.

[0020] Figure 7 The illustration shows a collaborative AR session viewed by participants wearing head-mounted devices, based on some examples.

[0021] Figure 8 It is a block diagram illustrating a software architecture that can be implemented within this disclosure based on some examples.

[0022] Figure 9 It is a graphical representation of a machine in the form of a computer system, based on some examples, in which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed. Detailed Implementation

[0023] Known head-mounted devices, such as AR glasses, include transparent or semi-transparent displays, allowing users to view their surroundings through these displays. Additional information or objects (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) are displayed on the display and appear as part of and / or overlaid on the surrounding environment to provide the user with an augmented reality experience. The display may, for example, include a waveguide that receives a beam of light from a projector, but any suitable display used to present augmented or virtual content to the wearer can be used.

[0024] AR glasses can be used to create shared environments where virtual objects are positioned at fixed locations in the real world, visible from the perspective of each user participating in the shared environment. Participants can then collaborate or compete within this shared environment. In some examples, shared AR sessions that multiple users can participate in via client devices are hosted on a system that includes multiple head-mounted displays, associated mobile devices, and networked resources. The experience is synchronized and shared among users, allowing actions by one user in a shared AR session to be synchronized and broadcast to other users.

[0025] In some situations, it may be beneficial to provide a means to notify physically present or nearby users of an ongoing shared AR session and to offer them the option to join the session. This can be achieved by transmitting short-range data from the current participant's head-mounted device or other computing device, announcing the type and session ID of the ongoing shared AR session.

[0026] In some examples, a method for initiating or joining a visual computing session, executed by one or more processors, is provided, the method comprising: receiving user input for initiating a session of a visual computing experience; detecting short-range data transmission including data indicating the existence of a current session of the visual computing experience; determining, based on the data indicating the existence of a current session of the visual computing experience, that the current session of the visual computing experience is in progress; and, based on determining that the current session is in progress, providing user input options for joining the current session of the visual computing experience.

[0027] The method may further include receiving user input to initiate a new session of the visual computing experience, and transmitting short-range data including data indicating the existence of the new session of the visual computing experience. The data indicating the existence of the current session of the visual computing experience may include a session identifier, and the method may further include transmitting a request to a remote server to join the current session, the request including the session identifier.

[0028] The option to selectively provide user input to join the current session can be based on additional requirements that can be met. These additional requirements may include the degree of relationship between the user who received the user input to initiate the session and other participants in the current session.

[0029] Data indicating the existence of a current or new session of a visual computing experience may include a session identifier and an identifier corresponding to the visual computing experience.

[0030] In some examples, a system is provided that includes one or more camera devices, one or more display devices, and one or more processors. The system also includes a memory storing instructions that, when executed by one or more processors, configure the system to perform operations corresponding to the methods disclosed above, including but not limited to: receiving user input to initiate a session of visual computing experience; detecting short-range data transmission including data indicating the existence of a current session of visual computing experience; determining, based on the data indicating the existence of a current session of visual computing experience, that a current session of visual computing experience is in progress; and, based on determining that a current session is in progress, providing user input options to join the current session of visual computing experience.

[0031] In some examples, a non-transitory computer-readable storage medium is provided that includes instructions, when executed by a system including one or more display devices, causing the system to perform operations corresponding to the methods disclosed above, including but not limited to: receiving user input to initiate a session of visual computing experience; detecting short-range data transmission including data indicating the existence of a current session of visual computing experience; determining, based on the data indicating the existence of a current session of visual computing experience, that a current session of visual computing experience is in progress; and, based on determining that a current session is in progress, providing user input options to join the current session of visual computing experience.

[0032] Other technical features may be readily apparent to those skilled in the art from the accompanying drawings, description and claims.

[0033] As mentioned herein, the phrase "augmented reality experience" includes or refers to various image processing operations corresponding to image modification, filtering, media overlay, transformation, etc., as further described herein. In some examples, these image processing operations provide an interactive experience of a real-world environment, where real-world objects, surfaces, backgrounds, lighting, etc., are enhanced through computer-generated perceptual information. In this context, "augmented reality effect" includes the set of data, parameters, and other assets required to apply a selected augmented reality experience to an image or video feed. In some examples, augmented reality effects are provided by Snap Inc. under the registered trademark LENSES.

[0034] In some examples, augmented reality effects include augmented reality (or “AR”) content configured to modify or transform image data presented within the GUI of a head-mounted device in some way. For example, AR effect data can be used to perform complex additions or transformations to content images, such as adding rabbit ears to a person’s head, adding a floating heart with a background color, changing the proportions of a person’s features, adding augmentations to landmarks in a scene viewed on the head-mounted device, or many other such transformations. This includes both real-time modifications and modifications to stored content (such as video clips in a gallery), where real-time modifications modify images as they are captured using a camera device associated with the head-mounted device, and then displayed by the head-mounted device with AR effect modifications, allowing the stored content to be modified using AR effects. Similarly, real-time video capture can be used in conjunction with AR effects to show the user of the head-mounted device how the video images currently captured by the device’s sensors will modify the captured data. Such data can simply be displayed on the screen without being stored in memory. The content captured by the device's sensors can be recorded and stored in memory with or without AR effects (or both), or the content captured by the device's sensors can be transmitted over a network to a server or another device with AR effects.

[0035] Therefore, AR effects, and the associated systems and modules for modifying content using AR effects, can involve: the detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.), tracking such objects as they leave and enter the field of view of a video frame and move around within the field of view of a video frame, and modifying or transforming such objects while they are being tracked. In various examples, different methods can be used to achieve such transformations. For example, some examples may involve generating 3D mesh models of one or more objects, and using transformations and animated textures of models within the video to achieve the transformation. In other examples, tracking points on an object can be used to place an image or texture (which can be two-dimensional or three-dimensional) at the tracked location. In still other examples, neural network analysis of video frames can be used to place images, models, or textures into content (e.g., images or video frames). Therefore, AR effect data can include both: images, models, and textures used to create transformations in the content, and additional modeling and analysis information required to achieve such transformations using object detection, tracking, and placement.

[0036] Although this paper describes AR systems, the ideas and methods disclosed herein can also be applied to other synthetic visual computing experiences such as virtual reality or metaverse sessions.

[0037] Figure 1This is a perspective view based on some examples of a head-mounted device (e.g., glasses 100). Glasses 100 may include a frame 102 made of any suitable material, such as plastic or metal, including any suitable shape memory alloy. In one or more examples, frame 102 includes a first optical element holder or left optical element holder 104 (e.g., a display or lens holder) and a second optical element holder or right optical element holder 106 connected by a bridging portion 112. The first optical element or left optical element 108 and the second optical element or right optical element 110 may be disposed within the left optical element holder 104 and the right optical element holder 106, respectively. Each of the right optical element 110 and the left optical element 108 may be a lens, a display, a display assembly, or a combination thereof. Any suitable display assembly may be disposed in glasses 100.

[0038] Frame 102 additionally includes a left arm or left temple 120 and a right arm or right temple 122. In some examples, frame 102 may be formed from a single piece of material to have a uniform or monolithic construction.

[0039] The eyeglasses 100 may include a computing device such as a computer 118, which may be of any suitable type for being carried by the frame 102, and in one or more examples, the computing device may have a suitable size and shape to be at least partially housed in one of the temple pieces 120 or 122. The computer 118 may include one or more processors, as well as memory, wireless communication circuitry, and a power supply. As discussed below, the computer 118 includes low-power circuitry, high-speed circuitry, and a display processor. Various other examples may include these elements in different configurations or integrated in different ways. Additional details of the aspects of the computer 118 may be implemented as shown in the data processor 302 discussed below.

[0040] The computer 118 additionally includes a battery 116 or other suitable portable power supply. In some examples, the battery 116 is disposed in the left temple 120 and electrically coupled to the computer 118 disposed in the right temple 122. The glasses 100 may include a connector or port (not shown) for charging the battery 116, a wireless receiver, a transmitter or transceiver (not shown), or a combination of such devices.

[0041] The glasses 100 include a camera device 114. Although two camera devices are depicted, other examples contemplate the use of a single or additional (i.e., more than two) camera devices. In one or more examples, in addition to the camera device 114, the glasses 100 also includes any number of input sensors or other input / output devices. Such sensors or input / output devices may additionally include biometric sensors, position sensors, motion sensors, etc.

[0042] The glasses 100 may also include a touchpad 124, which is mounted to or integrated with one or both of the left temple 120 and the right temple 122. The touchpad 124 is generally arranged vertically, and in some examples is approximately parallel to the user's temple. As used herein, generally vertical alignment means that the touchpad is at least more vertical than horizontal, although potentially more vertical than said vertical. Additional user input may be provided via one or more buttons 126, which, in the illustrated example, are located on the outer upper edges of the left optics retainer 104 and the right optics retainer 106. The one or more touchpads 124 and buttons 126 provide means by which the glasses 100 can receive input from the user of the glasses 100.

[0043] Figure 2 The glasses 100 are shown from the wearer's perspective. For clarity, Figure 1 Several components shown have been omitted. For example... Figure 1 As described, Figure 2 The eyeglasses 100 shown include a left optical element 108 and a right optical element 110 respectively fixed in each of the left optical element holder 104 and the right optical element holder 106.

[0044] The glasses 100 include: a forward optics 202, which includes a right projector 204 and a right near-eye display 206; and a forward optics 210, which includes a left projector 212 and a left near-eye display 216.

[0045] In some examples, the near-eye display is a waveguide. The waveguide includes reflective or diffractive structures (e.g., gratings and / or optical elements such as mirrors, lenses, or prisms). Light 208 emitted by projector 204 encounters the diffractive structure of the waveguide of near-eye display 206, which directs the light toward the user's right eye to provide an image superimposed on or within the right optical element 110, representing a view of the real world as seen by the user. Similarly, light 214 emitted by projector 212 encounters the diffractive structure of the waveguide of near-eye display 216, which directs the light toward the user's left eye to provide an image superimposed on or within the left optical element 108, representing a view of the real world as seen by the user.

[0046] However, it should be understood that other display technologies or configurations that can display images to the user in the forward field of view can be provided. For example, instead of providing the projector 204 and the waveguide, an LCD, LED, or other display panel or surface can be provided.

[0047] When in use, the wearer of glasses 100 will have information, content, and various user interfaces displayed on a near-eye monitor. As described in more detail below, in addition to the associated devices such as… Figure 3 In addition to providing voice or touch input on the client device 328 shown, the user can then interact with the glasses 100 using the touchpad 124 and / or button 126.

[0048] Figure 3 This is a block diagram showing details of a networking system 300, including glasses 100, according to some examples.

[0049] The networking system 300 includes glasses 100, client device 328, and server system 332. Client device 328 can be a smartphone, tablet computer, tablet phone, laptop computer, access point, or any other such device capable of connecting to glasses 100 using both low-power wireless connection 336 and high-speed wireless connection 334. Client device 328 is connected to server system 332 via network 330. Network 330 can include any combination of wired and wireless connections. Server system 332 can be one or more computing devices as part of a service or network computing system. Client device 328, as well as any components of server system 332 and network 330, can be used... Figure 8 and Figure 9 The software architecture described in the document is implemented in detail for the 804 or 900 machine.

[0050] The glasses 100 include a data processor 302, a display 310, one or more camera devices 308, and additional input / output elements 316. The input / output elements 316 may include a microphone, an audio speaker, a biometric sensor, additional sensors, or an additional display element integrated with the data processor 302. (About...) Figure 8 and Figure 9 Examples of input / output element 316 are further discussed. For example, input / output element 316 may include any I / O component 906, which may include output component 928, motion component 936, etc. Figure 2 An example of display 310 is discussed herein. In the specific example described herein, display 310 includes displays for each of the user's left and right eyes.

[0051] The data processor 302 includes an image processor 306 (e.g., a video processor), a GPU and a display driver 338, a tracking module 340, an interface 312, a low-power circuit system 304, and a high-speed circuit system 320. The components of the data processor 302 are interconnected via a bus 342.

[0052] Interface 312 refers to any source of user commands provided to data processor 302. In one or more examples, interface 312 is a physical button that, when pressed, sends a user input signal from interface 312 to low-power processor 314. Low-power processor 314 can process pressing such a button and then immediately releasing it as a request to capture a single image, and vice versa. Low-power processor 314 can process pressing such a button for a first time period as a request to capture video data while the button is pressed and to stop video capture when the button is released, wherein the video captured while the button is pressed is stored as a single video file. Alternatively, pressing the button for a longer time period can capture a still image. In some examples, interface 312 can be any mechanical switch or physical interface capable of accepting user input associated with data requests from camera device 308. In other examples, interface 312 can have software components or can be associated with commands received wirelessly from another source, such as client device 328.

[0053] Image processor 306 includes circuitry for receiving signals from camera device 308 and processing those signals into a format suitable for storage in memory 324 or for transmission to client device 328. In one or more examples, image processor 306 (e.g., video processor) includes a microprocessor integrated circuit (IC) customized for processing sensor data from camera device 308, and volatile memory used by the microprocessor in operation.

[0054] The low-power circuit system 304 includes a low-power processor 314 and a low-power wireless circuit system 318. These elements of the low-power circuit system 304 can be implemented as separate components or as part of a single-chip system on a single IC. The low-power processor 314 includes logic for managing other elements of the glasses 100. As described above, for example, the low-power processor 314 can accept user input signals from interface 312. The low-power processor 314 can also be configured to receive input signals or command communication from client device 328 via low-power wireless connection 336. The low-power wireless circuit system 318 includes circuit elements for implementing a low-power wireless communication system, Bluetooth. TM Smart, also known as Bluetooth TM Low power consumption is a standard implementation method for low-power wireless communication systems that can be used to implement the low-power wireless circuit system 318. In other examples, other low-power communication systems can be used.

[0055] The high-speed circuit system 320 includes a high-speed processor 322, a memory 324, and a high-speed wireless circuit system 326. The high-speed processor 322 can be any processor capable of managing the high-speed communication and operation of any general-purpose computing system required by the data processor 302. The high-speed processor 322 includes the processing resources required to manage high-speed data transmission over the high-speed wireless connection 334 using the high-speed wireless circuit system 326. In some examples, the high-speed processor 322 executes an operating system such as LINUX or other operating systems. Figure 8 The operating system 812 is one of other such operating systems. In addition to any other duties, a high-speed processor 322, using the software architecture of the execution data processor 302, manages data transmission with the high-speed wireless circuit system 326. In a particular example, the high-speed wireless circuit system 326 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also referred to herein as Wi-Fi. In other examples, the high-speed wireless circuit system 326 may implement other high-speed communication standards.

[0056] Memory 324 includes any storage device capable of storing camera data generated by camera device 308 and image processor 306. While memory 324 is shown as integrated with high-speed circuitry 320, in other examples, memory 324 may be a separate, independent element of data processor 302. In a particular such example, electrical wiring may provide a connection from image processor 306 or low-power processor 314 to memory 324 via a chip including high-speed processor 322. In other examples, high-speed processor 322 may manage addressing of memory 324 such that low-power processor 314 will bootstrap high-speed processor 322 whenever a read or write operation involving memory 324 is required.

[0057] The tracking module 340 estimates the pose of the glasses 100. For example, the tracking module 340 uses image data and corresponding inertial data from the camera device 308 and the positioning component 940, as well as GPS data, to track the position and determine the pose of the glasses 100 relative to a reference frame (e.g., the real-world environment). The tracking module 340 continuously collects and uses updated sensor data describing the motion of the glasses 100 to determine an updated three-dimensional pose of the glasses 100, which indicates changes in the relative position and orientation of the glasses 100 with respect to physical objects in the real-world environment. The tracking module 340 allows the glasses 100 to visually position virtual objects relative to physical objects within the user's field of view via the display 310.

[0058] The GPU and display driver 338 can use the pose of the glasses 100 to generate frames of virtual content or other content to be displayed on the display 310 when the glasses 100 is running in a conventional augmented reality mode. In this mode, the GPU and display driver 338 generate updated frames of virtual content based on the updated 3D pose of the glasses 100, which reflects changes in the user's position and orientation relative to physical objects in the user's real environment.

[0059] One or more functions or operations described herein can also be performed in an application residing on glasses 100, client device 328, or a remote server. For example, one or more functions or operations described herein can be performed by application 806, such as messaging application 846.

[0060] Figure 4This is a block diagram illustrating an example messaging system 400 for exchanging data (e.g., messages and associated content) over a network. The messaging system 400 includes multiple instances of client devices 328, each hosting several applications including a messaging client 402 and other applications 404. Each messaging client 402 is communicatively coupled to other instances of the messaging client 402 (e.g., hosted on corresponding other client devices 328), a messaging server system 406, and a third-party server 408 via a network 330 (e.g., the Internet). The messaging client 402 can also communicate with the locally hosted applications 404 using an application programming interface (API).

[0061] The messaging client 402 can communicate and exchange data with other messaging clients 402 and with the messaging server system 406 via the network 330. The data exchanged between messaging clients 402 and between messaging clients 402 and the messaging server system 406 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0062] The message transceiver server system 406 provides server-side functionality to a specific message transceiver client 402 via network 330. Although the specific functionality of the message transceiver system 400 is described herein as being performed by either the message transceiver client 402 or the message transceiver server system 406, the location of this specific functionality within the message transceiver client 402 or the message transceiver server system 406 can be a design choice. For example, it might be technically preferred that the specific technology and functionality are initially deployed within the message transceiver server system 406, but then migrated to the message transceiver client 402 of the client device 328 with sufficient processing power.

[0063] The messaging server system 406 supports various services and operations provided to the messaging client 402. Such operations include transmitting data to and receiving data from the messaging client 402, and processing data generated by the messaging client 402. As an example, this data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and on-site event information. Data exchange within the messaging system 400 is activated and controlled through functions available via the user interface (UI) of the messaging client 402.

[0064] Specifically, turning to message transceiver server system 406, application programming interface (API) server 410 is coupled to application server 414 and provides a programming interface to application server 414. Application server 414 is communicatively coupled to database server 416, which facilitates access to database 420, which stores data associated with messages processed by application server 414. Similarly, web server 424 is coupled to application server 414 and provides a web-based interface to application server 414. For this purpose, web server 424 handles incoming network requests via Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0065] Application Programming Interface (API) server 410 receives and transmits message data (e.g., commands and message payloads) between client device 328 and application server 414. Specifically, API server 410 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by messaging client 402 to activate the functionality of application server 414. Application Programming Interface (API) server 410 exposes various functions supported by application server 414, including: account registration; login functionality; sending messages from a specific messaging client 402 to another messaging client 402 via application server 414, sending media files (e.g., images or videos) from messaging client 402 to messaging server 412, and providing possible access for another messaging client 402; setting up collections of media data (e.g., stories); retrieving the friend list of the user of client device 328; retrieving such collections; retrieving messages and content; adding and deleting entities (e.g., friends) in an entity graph (e.g., a social graph); locating friends in the social graph; and opening application events (e.g., related to messaging client 402).

[0066] Application server 414 hosts several server applications and subsystems, including, for example, message transceiver server 412, image processing server 418, and social networking server 422. Message transceiver server 412 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of message transceiver client 402. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to message transceiver client 402. Given the hardware requirements for such processing, additional processor- and memory-intensive data processing can also be performed by message transceiver server 412 on the server side.

[0067] Application server 414 also includes image processing server 418, which is dedicated to performing various image processing operations, typically relative to the images or videos within the payload of messages sent from or received at message transceiver server 412.

[0068] Social networking server 422 supports various social networking functions and services, and makes these functions and services available to messaging server 412. To this end, social networking server 422 maintains and accesses an entity graph within database 420. Examples of functions and services supported by social networking server 422 include identifying other users that a particular user in messaging system 400 has a relationship with or that the particular user is "following," and also including identifying the interests and other entities of a particular user.

[0069] The messaging client 402 can notify users of client device 328 or other users (e.g., "friends") associated with such users of a shared or shareable session. For example, the messaging client 402 can provide participants in a conversation (e.g., a chat session) within the messaging client 402 with notifications related to current or recent use of the game by one or more members of a user group. One or more users can be invited to join an active session or a new session can be initiated. In some examples, shared sessions can provide a shared augmented reality experience that multiple people can collaborate on or participate in.

[0070] Figure 5A A user interface 500 displayed to a user of glasses 100 is shown, according to some examples. The user interface 500 includes a carousel 502 with icons 504 corresponding to augmented reality experiences selectable by the user. The carousel 502 can scroll left or right to move one of the icons 504 to a central position where the augmented reality experience corresponding to that icon can be activated, for example, by tapping on one of the touchpads 124. Scrolling the carousel 502 can be achieved, for example, by swiping forward or backward on one of the touchpads 124.

[0071] In the user interface 500 shown, the selection icon 506 in the center of the carousel 502 displays its title 510 below the carousel to identify the corresponding augmented reality experience. The fact that the selected augmented reality experience can be provided in a shared session with other users is indicated by display elements 508, such as symbols, showing the heads and shoulders of more than one person. Activating the augmented reality experience that can be provided in a shared session will transition the user interface based on whether an existing shared session is occurring nearby (e.g., by detecting short-range announcements as discussed below). Figure 5B or Figure 5C The user interface shown.

[0072] Figure 5B The user interface 512 shown is presented to the user of glasses 100 when a user selects a shared session augmented reality experience but no other shared session with the same augmented reality experience is nearby. A title 510 for the augmented reality experience is displayed, followed by a prompt 516 stating "Select a shared room to join." Below prompt 516 is only one option, a new room selection option 514. Visual enhancements 518, such as highlighting or surrounding brackets, may be provided to indicate that this option will be selected upon receiving user input, such as a tap on one of the touchpads 124.

[0073] Figure 5B The user interface 520 shown to the user of glasses 100 is illustrated when a user selects a shared session augmented reality experience and an instance of a shared session with the same augmented reality experience exists nearby. A title 510 for the augmented reality experience is displayed, followed by a prompt 516 that reads "Select a shared room to join." Below prompt 516 are new room selection options 514, as before, and one or more additional options corresponding to nearby shared sessions with the same augmented reality experience. In the example shown, the option 522 to join an existing session initiated by user Terek is available. Visual enhancements 518, such as highlighting or surrounding brackets, may be provided to indicate that the option will be selected upon receiving user input, such as a tap on one of the touchpads 124. The user can scroll through the available options, for example, by swiping forward or backward on one of the touchpads 124.

[0074] Figure 6 This is a flowchart 600 illustrating operations performed by glasses 100 to provide or facilitate a shared augmented reality session, according to some examples. For illustrative purposes, the operations of flowchart 600 are described herein as occurring sequentially or linearly. However, multiple operations of flowchart 600 may occur in parallel. Furthermore, the operations of flowchart 600 do not need to be performed in the order shown, and / or one or more boxes of flowchart 600 need not be performed and / or may be replaced by other operations.

[0075] Figure 6The operations shown are typically performed on the data processor 302 and associated hardware in or associated with glasses 100. For clarity, flowchart 600 is discussed herein with reference to such an example. Various implementations are, of course, possible, in which some operations occur in an application on client device 328 such as messaging application 846, on server system 332, or in which an application on client device 328 calls another application or SDK to obtain the desired functionality. In some examples, the messaging application 846 running on client device 328 performs operations jointly with the associated hardware and data processor 302 in or associated with glasses 100.

[0076] The method begins at operation 602 with receiving a request initiated by the corresponding glasses 100 or client device 328 to allow for augmented reality experiences involving multiple participants, for example, as referenced. Figure 5A As discussed, in operation 604, the glasses or client device 328 begins monitoring for transmissions of existing sessions of the same AR experience occurring nearby. This is achieved by the glasses 100 or client device 328 monitoring short-range transmissions including relevant identifying information such as the identifier of the AR experience and the session identifier. Any potentially relevant identifying information is extracted from any detected short-range transmission. The short-range transmission can be Bluetooth or Bluetooth LE broadcast or beacon, but any other suitable short-range data transmission method (IR, ultrasound, RF, etc.) can be employed. The range of the data transmission method and protocol should be sufficient to cover a reasonable area around the glasses 100 or client device 328 where a potential participant is likely to be located or physically present, or can reasonably and quickly become physically present within the reasonable area.

[0077] In operation 606, the glasses 100 or client device 328 determines whether an existing shared session exists based on the presence or absence of short-range transmissions that identify an existing session of the same AR experience. The existence of an existing session is determined by verifying that the identification information in any received and potentially relevant short-range transmission is identical to the identification information of the AR experience for which user input has already been received in operation 602. If no potentially relevant short-range transmission is received, or if the identification information in any received short-range transmission does not match the identification information of the AR experience for which user input has already been received in operation 602, it is determined that an existing session is missing.

[0078] If no existing session is available nearby, the method proceeds to operation 608, where options for starting a new shared session are displayed, such as... Figure 5B As shown in the diagram. After receiving input to start a new session in operation 610, a new session of the AR experience is started in operation 612.

[0079] Then, in operation 614, the glasses 100 or the client device 328 begins short-range transmissions including relevant identification information such as the identifier of the AR experience and the session identifier of the new session. These transmissions continue until the user exits the shared session.

[0080] The actual creation and management of the sessions occur within application server 414, and include assigning a unique ID to the shared session, which is provided to the client device 328 initiating the new session. Communication between the participating client devices 328, with or without the application server 414 acting as an intermediary, also primarily occurs on network 330. Therefore, out-of-band session initiation is provided using short-range transmission broadcast identification information, which leverages the short-range nature of the transmission to identify the existence of a shared session that can be joined by nearby potential participants.

[0081] If it is determined in operation 606 that an existing session is occurring nearby, the method proceeds to operation 616, where options are displayed for starting a new shared session or joining one or more existing sessions, for example, such as... Figure 5C As shown in operation 620, after receiving input to start a new session in operation 618, a new session of the AR experience is started in operation 622. Then, in operation 624, the glasses 100 or the client device 328 begins short-range transmissions including relevant identification information such as the identifier of the AR experience and the session identifier of the new session. These transmissions continue until the user exits the new session.

[0082] As determined in operation 620, after receiving input to join an existing session in operation 618, the selected existing session is joined in operation 626.

[0083] In some examples, the name of the creator of the shared session and other potentially relevant information, such as permissions or restrictions regarding the federated session, are included in the short-range transmission. In other examples, upon receiving the session identifier, additional information can be retrieved from the messaging server system 406. This information may include the name of the creator of the shared session and any permissions or requirements.

[0084] For example, additional information could specify a certain level of relationship between the creator or other participant in the federated session and the user of client device 328, in order to provide the user of client device 328 with the option to join an existing session. For instance, the level of relationship could specify only friends of the creator of the shared session, or friends of the creator (or other current participants) and friends of friends. In another example, joining a session might require certain hardware levels. Where there are any restrictions on who or what can join a shared session, this information is extracted from short-range transmissions using a session identifier or retrieved from the messaging server system 406, and any limitations or restrictions are checked as part of operation 606 to ensure that only sessions that can be joined are displayed in operation 616.

[0085] Figure 7 A shared session 700 is shown as seen by a first participant wearing a head-mounted device such as glasses 100. As discussed above, the shared session 700 has been initiated and joined. Figure 7 This includes real-world items seen by the first participant, such as the second participant 706, the third participant 710, a table 714, and a registration marker 718. A shared AR object 702 is overlaid on the real-world elements on the display 310 of the first participant's glasses 100. This shared AR object 702 includes a bottom AR ring 704, a middle AR ring 708, a top AR ring 712, and an AR interface 716.

[0086] Each participant has a view of the shared AR object 702, which depends on the position of each participant's corresponding glasses 100. The second participant 706 is looking at the right side of the shared AR object 702, while the third participant 710 is looking at the left side, compared to the shared session 700 viewed by the first participant as shown. If the first participant moves to the position of the third participant 710, the shared AR object 702 will rotate in the first participant's field of vision so that it appears fixed relative to the table 714, as if the shared AR object 702 were a real rather than a virtual object. Depending on the implementation, each participant can manipulate the shared AR object 702 for various purposes such as music creation, playing games, or other collaborative or participatory AR activities.

[0087] This shared reference frame can be created in several ways. In some examples, registration markers (such as registration marker 718) are placed in relevant locations, such as where the shared AR object 702 will be used (in... Figure 7(On top of table 714). One of the glasses 100 (e.g., the glasses of the creator of the shared session) captures an image of a registration marker, which is then transmitted to glasses worn by other participants. The other glasses also each detect and capture images of the registration marker. The other glasses 100 then each determine a transformation between the first image and the image captured by each of the glasses 100. The transformation is then used to determine a common coordinate system, and the shared session 700 is generated using this common coordinate system. The registration marker remains visible to each of the glasses 100 and can be used as a reference to update the pose of each of the glasses 100 as participants move around in the environment. The view of the shared AR object 702 is then updated to reflect each new pose of the glasses 100.

[0088] Other methods can be used to generate a shared reference frame, including, for example, 3D reconstruction and visual-inertial ranging based on images captured by each pair of glasses 100 and signals or data received from inertial sensors or other positioning sensors located in each pair of glasses. Features identified using 3D reconstruction can also be compared with existing point cloud models to locate and determine the pose of each pair of glasses in a real-world environment.

[0089] Figure 8 This is a block diagram 800 illustrating a software architecture 804 that can be installed on any one or more of the devices described herein. The software architecture 804 is supported by hardware such as a machine 802 including a processor 820, memory 826, and I / O components 838. In this example, the software architecture 804 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 804 includes layers such as an operating system 812, libraries 808, frameworks 810, and applications 806. Operationally, application 806 invokes API calls 850 through the software stack and receives messages 852 in response to API calls 850.

[0090] Operating system 812 manages hardware resources and provides public services. Operating system 812 includes, for example, a kernel 814, services 816, and drivers 822. Kernel 814 serves as an abstraction layer between hardware and other software layers. For example, kernel 814 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. Services 816 can provide other public services to other software layers. Drivers 822 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 822 may include display drivers, camera drivers, etc. or Low-power drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc.

[0091] Library 808 provides low-level common infrastructure used by application 806. Library 808 may include system libraries 818 (e.g., the C standard library) that provide functions such as memory allocation, string manipulation, and mathematical functions. Additionally, library 808 may include API libraries 824, such as media libraries (e.g., libraries for supporting the rendering and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codecs, Joint Picture Experts Group (JPEG or JPG) or Portable Web Graphics (PNG)), graphics libraries (e.g., OpenGL frameworks for rendering graphic content on a display in two-dimensional (2D) and three-dimensional (3D) formats), database libraries (e.g., SQLite providing various relational database functions), web libraries (e.g., WebKit providing web browsing capabilities), etc. Library 808 may also include various other libraries 828 to provide many other APIs to application 806.

[0092] Framework 810 provides high-level common infrastructure for use by application 806. For example, framework 810 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. Framework 810 can provide a wide range of other APIs that can be used by application 806, some of which may be specific to a particular operating system or platform.

[0093] In the example, application 806 may include a home application 836, a contacts application 830, a browser application 832, a book reader application 834, a location application 842, a media application 844, a messaging application 846, a game application 848, and a variety of other applications such as third-party application 840. Application 806 is a program that performs the functions defined in the program. One or more applications 806 can be created using various programming languages, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C or assembly language). In a particular example, third-party application 840 (e.g., an entity other than the vendor of a particular platform using Android) TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM , Mobile software running on the phone's mobile operating system or another mobile operating system. In this example, a third-party application 840 can call API calls 850 provided by the operating system 812 to facilitate the functions described herein.

[0094] Figure 9 This is a schematic representation of machine 900, in which instructions 910 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause machine 900 to perform any or more of the methods discussed herein. For example, instructions 910 can cause machine 900 to perform any or more of the methods described herein. Instructions 910 transform a general, unprogrammed machine 900 into a specific machine 900 programmed to perform the described and illustrated functions in the described manner. Machine 900 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 900 can operate as a server machine or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 900 may include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), PDAs, entertainment media systems, cellular phones, smartphones, mobile devices, head-mounted devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, web devices, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instructions 910 specifying actions to be taken by machine 900. Furthermore, although only a single machine 900 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute instructions 910 to perform any one or more of the methods discussed herein.

[0095] Machine 900 may include processor 902, memory 904, and I / O components 906, which may be configured to communicate with each other via bus 944. In the example, processor 902 (e.g., a central processing unit (CPU), a simplified instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 908 and 912 that execute instruction 910. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although... Figure 9Multiple processors 902 are shown, but machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0096] Memory 904 includes main memory 914, static memory 916, and storage cells 918, all of which are accessible by processor 902 via bus 944. Main memory 904, static memory 916, and storage cells 918 store instructions 910 that implement any one or more of the methods or functions described herein. During execution of instructions 910 by networked system 300, instructions 910 may also reside wholly or partially in main memory 914, in static memory 916, in machine-readable medium 920, in storage cells 918, in at least one of processors 902 (e.g., in the processor's cache memory), or in any suitable combination thereof.

[0097] I / O component 906 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurement results, etc. The specific I / O component 906 included in a particular machine will depend on the machine type. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It is to be understood that I / O component 906 may include... Figure 9 Many other components are not shown. In various examples, I / O component 906 may include output component 928 and input component 932. Output component 928 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tube (CRT) displays), acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input component 932 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens providing position and / or force for touch or touch gestures, or other haptic input components), audio input components (e.g., microphones), etc.

[0098] In another example, I / O component 906 may include: biometric component 934, motion component 936, environmental component 938, or positioning component 940, and various other components. For example, biometric component 934 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 936 includes accelerometer components (e.g., accelerometer), gravity sensor components, rotation sensor components (e.g., gyroscope), etc. Environmental component 938 includes, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of hazardous gases or measuring pollutants in the atmosphere for safety purposes), or other components that can provide indications, measurement results, or signals corresponding to the surrounding physical environment. Positioning component 940 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting air pressure to obtain altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0099] A variety of technologies can be used to implement communication. I / O component 906 also includes communication component 942, which is operable to couple network system 300 to network 922 or device 924 via coupling 930 and coupling 926, respectively. For example, communication component 942 may include a network interface component or another suitable device that interfaces with network 922. In further examples, communication component 942 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, etc. Components (e.g.) (low power consumption) Components and other communication components that provide communication via other modes. Device 924 can be another machine or any peripheral device of a wide variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0100] Furthermore, the communication component 942 can detect identifiers or include components operable to detect identifiers. For example, the communication component 942 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., for detecting optical sensors such as one-dimensional barcodes, such as Universal Product Code (UPC) barcodes; multi-dimensional barcodes, such as Quick Response (QR) codes, Aztec codes, data matrices, dataglyphs, MaxiCodes, PDF417, hypercodes, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying audio signals from tags). Additionally, various information can be obtained via the communication component 942, such as location via Internet Protocol (IP) geolocation, etc. The location of signal triangulation, the location of NFC beacon signals that can be detected to indicate a specific location, etc.

[0101] Various memories (e.g., memory 904, main memory 914, static memory 916, and / or the memory of processor 902) and / or storage units 918 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instruction 910) cause various operations to implement the disclosed examples when executed by processor 902.

[0102] Instructions 910 can be transmitted or received over network 922 via a network interface device (e.g., a network interface component included in communication component 942), using a transmission medium and any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 910 can be transmitted or received to device 924 via a transmission medium through coupling 926 (e.g., peer-to-peer coupling).

[0103] "Carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions can be transmitted or received over a network using a transmission medium via a network interface device.

[0104] "Client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, tablet computers, ultrabooks, netbooks, laptop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0105] "Communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS (Plain Old-Style Telephone Service) network, a cellular telephone network, a wireless network, etc. A network, another type of network, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any data transmission technology of various types, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rate Evolution (EDGE) technology of GSM, the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long-distance protocols, or other data transmission technologies.

[0106] A “component” means a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies provided for partitioning or modularizing specific processing or control functions. A component can be combined with other components via its interface to perform machine processing. A component can be a packaged functional hardware unit designed for use with other components and is part of a program that typically performs a specific function related to that function. A component can constitute a software component (e.g., code implemented on a machine-readable medium) or a hardware component. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various examples, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component can include a dedicated circuit system or logic permanently configured to perform certain operations. Hardware components can be dedicated processors, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). Hardware components can also include programmable logic or circuit systems that are temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function, and is no longer a general-purpose processor. It will be understood that the decision to implement hardware components mechanically in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., software-configured) circuit system can be driven by cost and time considerations. Accordingly, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. Consider the example of hardware components being temporarily configured (e.g., programmed), where each hardware component does not need to be configured or instantiated at any given time. For example, in cases where the hardware components include a general-purpose processor that is configured as a dedicated processor via software, this general-purpose processor can be configured at different times as its respective dedicated processor (e.g., including different hardware components). The software accordingly configures one or more specific processors to constitute a specific hardware component at one time and different hardware components at different times. Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicatively coupled.In the presence of multiple hardware components, communication can be achieved through signal transmission (e.g., via appropriate circuitry and buses) between or within two or more hardware components. In examples where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving information from the memory structure. For example, a hardware component can perform an operation and store the output of that operation in a memory device communicatively coupled to it. Another hardware component can then access the memory device at a subsequent time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and can operate on resources (e.g., information collection). Various operations of the example methods described herein can be performed, at least in part, by one or more processors configured, either temporarily (e.g., via software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented at least in part by processors, where one or more specific processors are examples of hardware. For example, at least some of the operations of the methods can be executed by one or more processors or processor-implemented components. Furthermore, one or more processors can also operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some of the operations can be executed by a group of computers (as an example of a machine including processors), where these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of some operations can be distributed among processors, not only residing within a single machine but also deployed across multiple machines. In some examples, the processors or processor-implemented components can reside in a single geographic location (e.g., within a home environment, office environment, or server cluster). In other examples, the processors or processor-implemented components can be distributed across multiple geographic locations.

[0107] "Computer-readable medium" refers to both machine storage media and transmission media. Therefore, these terms encompass both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" refer to the same thing and can be used interchangeably in this disclosure.

[0108] A "brief message" is a message that can be accessed for a limited time. Brief messages can be text, images, videos, etc. The access time for a brief message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting method, the message is temporary.

[0109] "Machine storage medium" refers to one or more storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions, routines, and / or data. Therefore, this term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The terms "machine storage medium," "device storage medium," and "computer storage medium" refer to the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium."

[0110] A "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and generates corresponding output signals that are applied to operate a machine. For example, a processor can be a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Radio Frequency Integrated Circuit (RFIC), or any combination thereof. A processor can also be a multi-core processor having two or more independent processors (sometimes called "cores") capable of executing instructions simultaneously.

[0111] "Signal medium" means any intangible medium capable of storing, encoding, or carrying instructions executable by a machine, and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal whose characteristics are set or altered in a manner that encodes information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0112] Changes and modifications may be made to the disclosed examples without departing from the scope of this disclosure. Such and other changes or modifications are intended to be included within the scope of this disclosure as set forth in the appended claims.

[0113] Regarding the implementation methods including the above embodiments, the following technical solutions are also disclosed:

[0114] Option 1. A method for initiating or joining a visual computing session, executed by one or more processors, comprising:

[0115] Receive user input for initiating a visual computing experience session;

[0116] Detection includes short-range data transmission of data that indicates the presence of the current session in the visual computing experience;

[0117] Based on the data indicating the existence of the current session of the visual computing experience, it is determined that the current session of the visual computing experience is in progress; and

[0118] Based on the determination that the current session is in progress, user input options are provided for joining the current session of the visual computing experience.

[0119] Option 2. The method according to Option 1 further includes:

[0120] Receive user input to begin a new session of the visual computing experience; and

[0121] The transmission includes short-range data transmission of data indicating the existence of a new session of the visual computing experience.

[0122] Option 3. According to the method of Option 2, wherein the data indicating the existence of the current session or new session of the visual computing experience includes an identifier corresponding to the visual computing experience.

[0123] Option 4. The method according to Option 3, wherein the data indicating the existence of the current session or a new session of the visual computing experience includes a session identifier.

[0124] Option 5. The method according to Option 1, wherein the data indicating the existence of the current session of the visual computing experience includes a session identifier, and the method further includes:

[0125] A request to join the current session is transmitted to a remote server, the request including the session identifier.

[0126] Option 6. The method according to Option 1 further includes:

[0127] The user input option to join the current session is selectively provided based on whether additional requirements are met.

[0128] Option 7. The method according to Option 6, wherein the additional requirement includes the degree of relationship between the user who receives the user input that initiates the session and other participants in the current session.

[0129] Option 8. A system comprising:

[0130] One or more camera devices;

[0131] One or more display devices;

[0132] One or more processors; and

[0133] A memory storing instructions that, when executed by the one or more processors, configure the system to perform operations such as initiating or joining a visual computing session, the operations including:

[0134] Receive user input for initiating a visual computing experience session;

[0135] Detection includes short-range data transmission of data that indicates the presence of the current session in the visual computing experience;

[0136] Based on the data indicating the existence of the current session of the visual computing experience, it is determined that the current session of the visual computing experience is in progress; and

[0137] Based on the determination that the current session is in progress, user input options are provided for joining the current session of the visual computing experience.

[0138] Option 9. The system according to Option 8, wherein the operation further includes:

[0139] Receive user input to begin a new session of the visual computing experience; and

[0140] The transmission includes short-range data transmission of data indicating the existence of a new session of the visual computing experience.

[0141] Option 10. The system according to Option 9, wherein the data indicating the existence of a current session or a new session of the visual computing experience includes an identifier corresponding to the visual computing experience.

[0142] Option 11. The system according to Option 9, wherein the data indicating the existence of a current session or a new session of the visual computing experience includes a session identifier.

[0143] Option 12. The system according to Option 8, wherein the data indicating the existence of the current session of the visual computing experience includes a session identifier, and the operation further includes:

[0144] A request to join the current session is transmitted to a remote server, the request including the session identifier.

[0145] Option 13. The system according to Option 8, wherein the operation further includes:

[0146] The user input option to join the current session is selectively provided based on whether additional requirements are met.

[0147] Option 14. The system according to Option 13, wherein the additional requirement includes the degree of relationship between the user who receives user input from which the session is initiated and other participants in the current session.

[0148] Option 15. A non-transitory computer-readable storage medium including instructions that, when executed by a system including one or more display devices, cause the system to perform operations of initiating or joining a visual computing session, the operations including:

[0149] Receive user input for initiating a visual computing experience session;

[0150] Detection includes short-range data transmission of data that indicates the presence of the current session in the visual computing experience;

[0151] Based on the data indicating the existence of the current session of the visual computing experience, it is determined that the current session of the visual computing experience is in progress; and

[0152] Based on the determination that the current session is in progress, user input options are provided for joining the current session of the visual computing experience.

[0153] Option 16. The non-transitory computer-readable storage medium according to Option 15, wherein the operation further includes:

[0154] Receive user input to begin a new session of the visual computing experience; and

[0155] The transmission includes short-range data transmission of data indicating the existence of a new session of the visual computing experience.

[0156] Option 17. The non-transitory computer-readable storage medium according to Option 16, wherein the data indicating the existence of a current session or a new session of the visual computing experience includes a session identifier.

[0157] Option 18. The non-transitory computer-readable storage medium according to Option 15, wherein the data indicating the existence of the current session of the visual computing experience includes a session identifier, and the operation further includes:

[0158] A request to join the current session is transmitted to a remote server, the request including the session identifier.

[0159] Option 19. The non-transitory computer-readable storage medium according to Option 15, wherein the operation further includes:

[0160] The user input option to join the current session is selectively provided based on whether additional requirements are met.

[0161] Option 20. The non-transitory computer-readable storage medium according to Option 19, wherein the additional requirement includes the degree of relationship between the user who receives user input from which the session is initiated and other participants in the current session.

Claims

1. A method for initiating or joining a collaborative computing session on a first device, executed by one or more processors, comprising: Receive user input to initiate a new collaborative computing session at the first device; It was determined that a current collaborative computing session of the same type was taking place on a physically existing second device; as well as Based on the determination that the current collaborative computing session is taking place on the physically existing second device, a user input option is provided at the first device to join the current collaborative computing session using the first device instead of starting the new collaborative computing session.

2. The method according to claim 1, further comprising: Further user input is received at the first device to initiate the new session of the collaborative computing session; as well as Short-range data transmission is transmitted from the first device, including data indicating that the new collaborative computing session is taking place on the first device.

3. The method according to claim 2, wherein, The data indicating that the new collaborative computing session is in progress includes an identifier corresponding to the collaborative computing session.

4. The method according to claim 3, wherein, The data indicating that the new collaborative computing session is in progress also includes a session identifier.

5. The method according to claim 1, wherein, Determining that the current collaborative computing session is taking place on the physically present second device includes receiving data from the physically present second device indicating that the current collaborative computing session is in progress, the data including a session identifier, the method further includes: The first device transmits a request to the remote server to join the current collaborative computing session, the request including the session identifier.

6. The method according to claim 1, further comprising: Based on the fulfillment of additional requirements, the user input option to join the current collaborative computing session is selectively provided, wherein the additional requirements include the degree of relationship between the user of the first device and other participants in the current collaborative computing session, and wherein the user input to initiate the new collaborative computing session is received from that user.

7. The method according to claim 1, wherein, Determining that the current collaborative computing session is being conducted on the physically present second device includes detecting short-range data transmission from the physically present second device, the short-range data transmission including data indicating that the current collaborative computing session is being conducted on the physically present second device.

8. A system comprising: The first device includes one or more camera devices and one or more display devices; One or more processors; as well as A memory storing instructions that, when executed by the one or more processors, configure the system to perform operations such as initiating or joining a collaborative computing session, the operations including: Receive user input to initiate a new collaborative computing session at the first device; Determine that a current collaborative computing session of the same type is taking place on a physically existing second device; and Based on the determination that the current collaborative computing session is taking place on the physically existing second device, a user input option is provided at the first device to join the current collaborative computing session using the first device instead of starting the new collaborative computing session.

9. The system according to claim 8, wherein, The operation also includes: Receive further user input at the first device to start the collaborative computing session; and Short-range data transmission is transmitted from the first device, including data indicating that the new collaborative computing session is taking place on the first device.

10. The system according to claim 9, wherein, The data indicating that the new collaborative computing session is in progress includes an identifier corresponding to the collaborative computing session.

11. The system according to claim 9, wherein, The data indicating that the new collaborative computing session is in progress also includes a session identifier.

12. The system according to claim 8, wherein, Determining that the current collaborative computing session is taking place on the physically present second device includes receiving data from the physically present second device indicating that the current collaborative computing session is in progress, the data including a session identifier, and the operation further includes: The first device transmits a request to the remote server to join the current collaborative computing session, the request including the session identifier.

13. The system according to claim 8, wherein, The operation also includes: Based on the fulfillment of additional requirements, the user input option to join the current collaborative computing session is selectively provided, wherein the additional requirements include the degree of relationship between the user of the first device and other participants in the current collaborative computing session, and wherein the user input to initiate the new collaborative computing session is received from that user.

14. The system according to claim 13, wherein, Determining that the current collaborative computing session is being conducted on the physically present second device includes detecting short-range data transmission from the physically present second device, the short-range data transmission including data indicating that the current collaborative computing session is being conducted on the physically present second device.

15. A non-transitory computer-readable storage medium comprising instructions, said instructions, when executed by a system including a first device comprising one or more display devices, causing said system to perform operations of initiating or joining a collaborative computing session, said operations comprising: Receive user input to initiate a new collaborative computing session at the first device; It was determined that a current collaborative computing session of the same type was taking place on a physically existing second device; as well as Based on the determination that the current collaborative computing session is taking place on the physically existing second device, a user input option is provided at the first device to join the current collaborative computing session using the first device instead of starting the new collaborative computing session.

16. The non-transitory computer-readable storage medium according to claim 15, wherein, The operation also includes: Receive further user input at the first device to start the collaborative computing session; and Short-range data transmission is transmitted from the first device, including data indicating that the new collaborative computing session is taking place on the first device.

17. The non-transitory computer-readable storage medium according to claim 16, wherein, The data indicating that the new collaborative computing session is in progress also includes a session identifier.

18. The non-transitory computer-readable storage medium according to claim 15, wherein, Determining that the current collaborative computing session is taking place on the physically present second device includes receiving data from the physically present second device indicating that the current collaborative computing session is in progress, the data including a session identifier, and the operation further includes: The first device transmits a request to the remote server to join the current collaborative computing session, the request including the session identifier.

19. The non-transitory computer-readable storage medium according to claim 15, wherein, The operation also includes: Based on the fulfillment of additional requirements, the user input option to join the current collaborative computing session is selectively provided, wherein the additional requirements are...

Images