Virtual interactive sessions for facilitating time-limited augmented reality-based communication between multiple users

Abstract

Aspects of the disclosure relate to a system, method, and user interface including a computer readable storage medium having at least one program stored thereon, for facilitating augmented reality based communication between a plurality of users over a network. Session configuration data is received from a first device, the session configuration data including configuration parameters of a virtual interaction session with a first user. The configuration parameters include an identifier of a second user allowed to join the virtual interaction session and a micro-chat duration defining a time limit of a real-time communication link between the first user and the second user during the virtual interaction session. The real-time communication link between the first user and the second user is established by causing display, by a second device, of a live camera feed generated at the first device. The real-time communication link between the first user and the second user is terminated upon expiration of the micro-chat duration.

Description

[0001] Priority requirements

[0002] This application claims priority to U.S. Patent Application Serial No. 17 / 207,174, filed March 19, 2021, and U.S. Provisional Application Serial No. 62 / 994,671, filed March 25, 2020, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally relates to mobile and wearable computing technologies. In particular, example embodiments of this disclosure address systems, methods, and user interfaces for facilitating augmented reality-based communication among multiple users over a network. Background Technology

[0004] Many wearable and mobile devices, such as “smart” glasses, include embedded camera devices. Virtual rendering systems implemented using these types of devices can be used to create engaging and fun augmented reality experiences in which three-dimensional (3D) graphical content appears to exist in the real world. Attached Figure Description

[0005] To facilitate identification of any particular element or action being discussed, one or more of the highest significant digits in the figure references refer to the figure number in which the element or action is first introduced.

[0006] Figure 1 This is a diagrammatic representation of a networked environment in which the present disclosure can be deployed, according to some example implementations.

[0007] Figure 2 It is a graphical representation of a communication system based on some example implementations.

[0008] Figure 3 This is a diagram illustrating a wearable device for use in a virtual interactive session, according to some example implementations.

[0009] Figure 4 This is a block diagram illustrating various aspects of a wearable device according to some example implementations.

[0010] Figures 5A to 5H This is a conceptual diagram illustrating the flow of an example virtual interaction session between a first user and a second user according to some example implementations.

[0011] Figure 6A and Figure 6B This is an interaction diagram illustrating an example interaction between components of a communication system when executing a method for facilitating a virtual interactive session with a first user, according to an example embodiment.

[0012] Figures 7 to 9 This is a flowchart illustrating the operation of a communication system when performing a method for facilitating a virtual interactive session with a first user, according to an example embodiment.

[0013] Figure 10 This is a block diagram illustrating a representative software architecture according to an example implementation, which can be used in conjunction with various hardware architectures described herein.

[0014] Figure 11 This is a block diagram illustrating components of a machine according to an example embodiment, the machine being capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any or more of the methods discussed herein. Detailed Implementation

[0015] The following description includes systems, methods, techniques, instruction sequences, and computer program products for implementing illustrative embodiments of the present disclosure. In this description, numerous specific details are set forth for illustrative purposes in order to provide an understanding of various embodiments of the subject matter of the invention. However, it will be apparent to those skilled in the art that embodiments of the subject matter of the invention can be practiced without these specific details. Generally, well-known examples of instructions, protocols, structures, and techniques are not necessarily shown in detail.

[0016] This disclosure includes systems, methods, techniques, instruction sequences, and computer program products for facilitating virtual interactive sessions between users. In a virtual interactive context, one or more users can establish a real-time communication link with a host user. When a real-time communication link is established between a first user and a second user, a live camera feed depicting the real-world environment at the first user's location is presented to the second user. As an example, the first user can wear a wearable device including a camera and optics with an optical element showing the real-world environment visible to the first user. During an active virtual interactive session with the first user, a real-time communication link can be established between the first user and the second user, and as part of the real-time communication link, the second user can view the first user's real-world environment via a live camera feed generated by the camera of the first user's wearable device and displayed on the second user's device. The first user (host user) can initiate a real-time communication link by inviting the second user to join the virtual interactive session, and the real-time communication link is established based on the second user accepting the invitation.

[0017] When the real-time communication link is active, the first and second users can communicate verbally using the audio communication link between their devices and also using augmented reality-based communication methods. For example, in addition to the live video feed from the first user's device, the second user's device can also display a set of selectable virtual content items that can be applied to the real-world environment visible to the first user. Each virtual content item includes one or more media objects. Following the example above, the virtual content items selected by the second user can be sent to the first user's wearable device. By displaying one or more media objects on the real-world environment display of the first user's wearable device, the virtual content items can be applied to the real-world environment visible to the first user. In this way, one or more media objects appear to the first user as if they exist in the real-world environment, thereby enhancing the first user's sense of reality.

[0018] A single virtual interaction session can include multiple virtual interaction sessions, and multiple users can interact with a first user (host user) in one or more virtual interaction sessions during a real-time communication session. To provide control and privacy mechanisms to the first user of a virtual interaction session, a configuration interface can be provided, displayed to the first user via a wearable device or a companion device coupled to the wearable device. The first user can use the configuration interface to define configuration parameters for the virtual interaction session. Configuration parameters include the session duration, which defines the time period of the virtual interaction session. No additional real-time communication link can be established with the first user when the session duration expires.

[0019] Configuration parameters may also include the micro-chat duration, which defines the time limit for the real-time communication link established with the first user during the virtual interactive session. The first user can use the configuration interface to define the micro-chat duration of the virtual interactive session. When a real-time communication link is established between the first user and the second user, both users' devices can display a countdown timer based on the session duration, and the real-time communication link between the two users is terminated when the session duration expires. That is, the display fed to the first user's live camera device is terminated on the second user's device, and the audio communication link between the two users' devices is disabled. While the virtual interactive session is still active, the second user can request to establish a second real-time communication link with the first user, or a third user authorized to join the virtual interactive session can request to establish a real-time communication link with the first user.

[0020] Figure 1This is a block diagram illustrating an example communication system 100 for exchanging data (e.g., messages and associated content) over a network. Communication system 100 includes multiple instances of client devices 102 (102-1 and 102-2). Each instance of client device 102 hosts multiple applications, including a communication client application 104. Each communication client application 104 is communicatively coupled to other instances of the communication client application 104 and a communication server system 108 via a network 106 (e.g., the Internet).

[0021] Communication client application 104 is able to communicate and exchange data with another communication client application 104 and communication server system 108 via network 106. The data exchanged between communication client applications 104 and between communication client application 104 and communication server system 108 includes functions (e.g., commands for invoking functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0022] The communication server system 108 provides server-side functionality to a specific communication client application 104 via network 106. While some functions of the communication system 100 are described herein as being performed by either the communication client application 104 or the communication server system 108, the location of a specific function within either the communication client application 104 or the communication server system 108 is a design choice. For example, it may be technically preferred that specific technologies and functions be initially deployed within the communication server system 108, but that technology and functions are later migrated to the communication client application 104 on the client device 102, which has sufficient processing power.

[0023] The communication server system 108 supports providing various services and operations to the communication client application 104. These operations include sending data to and receiving data from the communication client application 104, and processing data generated by the communication client application 104. As an example, this data may include message content, client device information, geolocation information, media comments and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the communication system 100 is invoked and controlled via functions available through the user interface (UI) of the communication client application 104.

[0024] Specifically, turning to communication server system 108, application programming interface (API) server 110 is coupled to application server 112 and provides a program interface to application server 112. Application server 112 is communicatively coupled to database server 118, which facilitates access to database 120, in which data associated with messages processed by application server 112 is stored.

[0025] Application Programming Interface (API) server 110 receives and sends message data (e.g., commands and message payloads) between client device 102 and application server 112. Specifically, API server 110 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by communication client application 104 to invoke functions of application server 112. API server 110 exposes various functions supported by application server 112, including account registration, login functionality, sending messages from one communication client application 104 to another communication client application 104 via application server 112, sending media files (e.g., images or videos) from one communication client application 104 to the other, and setting up media data (e.g., story) sets, retrieving the friend list of the user of client device 102, retrieving such sets, retrieving messages and content, adding and deleting friends to and from the social graph, locating friends within the social graph, and opening application events (e.g., related to communication client application 104).

[0026] Application server 112 hosts multiple applications and subsystems, including communication server application 114, image processing system 116, and social networking system 122. Communication server application 114 implements multiple message processing techniques and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of communication client application 104. As will be described in further detail, text and media content from multiple sources can be aggregated into content collections (e.g., referred to as stories or galleries). Communication server application 114 then makes these collections available to communication client application 104. Given the hardware requirements for additional processor- and memory-intensive processing of data, communication server application 114 can also perform such data processing on the server side.

[0027] Communication server application 114 also facilitates virtual interactive sessions between users. In the context of a virtual interactive session, one or more users can establish real-time communication links with a host user. To establish a real-time communication link between user 103-1 on client device 102-1 and user 103-2 on client device 102-2, communication server application 114 causes client device 102-2 to display a live camera feed depicting the real-world environment at user 103-1's location. In the example, client device 102-1 is a wearable device (e.g., smart glasses) worn by user 103-1, which includes a camera and optics with an optical element that allows the real-world environment to be seen through it by user 103-1. When the active real-time communication link between users 103-1 and 103-2 is active, user 103-2 can view user 103-1's real-world environment via a live camera feed generated by the camera of client device 102-1 and displayed by client device 102-2. The communication server application 114 also enables users 103-1 and 103-2 to communicate verbally using audio communication modalities and augmented reality-based communication modalities.

[0028] As an example, in addition to the live camera feed generated by client device 102-1, communication server application 114 can cause client device 102-1 to display a set of selectable virtual content items, which can be applied to the real-world environment visible to user 103-1. Each virtual content item includes one or more media objects. In the example, the virtual content items selected by user 103-2 can be applied to the real-world environment visible to user 103-1 by causing one or more media objects to be displayed by a transparent display in the optical elements of client device 102-1. In this way, one or more media objects appear to the first user as if they exist in the real-world environment. More details about virtual interactive sessions are discussed below.

[0029] Application server 112 also includes an image processing system 116 dedicated to performing various image processing operations, typically relating to images or videos generated and displayed by an instance of client device 102.

[0030] Social networking system 122 supports various social networking features and services, and makes these features and services available to communication server application 114. To this end, social networking system 122 maintains and accesses an entity graph within database 120. Examples of features and services supported by social networking system 122 include identifying other users with whom a specific user of communication system 100 has a relationship or "follows," as well as other entities and interests of that specific user.

[0031] Application server 112 is communicatively coupled to database server 118, which facilitates access to database 120, in which data associated with messages processed by communication server application 114 is stored.

[0032] Figure 2 This is a block diagram illustrating further details of a communication system 100 according to an example implementation. Specifically, the communication system 100 is shown to include a communication client application 104 and an application server 112, which in turn include multiple subsystems, namely a temporary timer system 202, a collection management system 204, and a virtual rendering system 206.

[0033] Temporary timer system 202 is responsible for enforcing temporary access to content permitted by client application 104 and application server 112. To this end, temporary timer system 202 incorporates multiple timers that selectively display and implement access to messages and associated content via client application 104 based on duration and display parameters associated with messages or message sets (e.g., stories).

[0034] The collection management system 204 is responsible for managing collections of media (e.g., collections of text, images, video, and audio data). In some examples, collections of content (e.g., messages including images, videos, text, and audio) can be organized into “event galleries” or “event stories.” Such collections can be available for a specified time period, such as the duration of an event related to the content. For example, content related to a concert can be available as a “story” for the duration of that concert.

[0035] The collection management system 204 also includes a curation interface 208, which enables the collection manager to manage and curate specific collections of content. For example, the curation interface 208 enables an event organizer to curate collections of content related to a specific event (e.g., removing inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections.

[0036] Virtual rendering system 206 provides various functions that enable users to enhance or otherwise modify or edit media content (e.g., media content including image data and / or audio data). For example, virtual rendering system 206 provides functions related to applying virtual content items to a real-world environment, whether by displaying media objects on a transparent display (through which the real-world environment is visible) or by enhancing image data to include media objects overlaid on the real-world environment described therein. Virtual content items may include one or more media objects. Media objects may include audio and visual content as well as visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Audio and visual content or visual effects may be applied to media data (e.g., a live image stream). Virtual content items may be stored in database 120 and accessed through database server 132.

[0037] Figure 3 This is a diagram illustrating a wearable device of an example form of glasses 331 for use in a virtual interactive session, according to some exemplary embodiments. The glasses 331 may include a frame 332 made of any suitable material such as plastic or metal (including any suitable shape memory alloy). The frame 332 may have a front element 333, which may include a first or left lens, display, or optical element holder 336 and a second or right lens, display, or optical element holder 337 connected by a bridge 338. The front element 333 additionally includes a left end portion 341 and a right end portion 342. A first or left optical element 344 and a second or right optical element 343 may be disposed within corresponding left optical element holders 336 and right optical element holders 337. Each of the optical elements 343, 344 may be a lens, a display (e.g., a transparent display or video display), a display assembly, or a combination of the foregoing components. In some embodiments, for example, the glasses 331 are provided with an integrated near-eye display mechanism that is capable of displaying, for example, a preview image of visual media captured by the camera device 367 of the glasses 331 to the user. In some embodiments, the integrated near-eye display mechanism allows the display of media objects such that the media objects are overlaid on the real-world environment that can be viewed through the optical elements 343 and 344.

[0038] Frame 332 additionally includes a left temple or arm piece 346 and a right temple piece 347, which are coupled to the respective left end 341 and right end 342 of the front piece 333 by any suitable means such as hinges (not shown) to couple to the front piece 333, or to be rigidly or fixably attached to the front piece 333 to be integral with the front piece 333. Each of the temple pieces 346 and 347 may include a first portion 351 coupled to the respective end 341 or 342 of the front piece 333 and any suitable second portion 352 for coupling to the user's ear, such as a bend or bow. In one embodiment, the front piece 333 may be formed from a single piece of material to have a monolithic or one-piece construction. In one embodiment, the entire frame 332 may be formed from a single piece of material to have a monolithic or one-piece construction.

[0039] The eyeglasses 331 may include a device such as a computer 361, which may be of any suitable type for being carried by the frame 332, and in one embodiment, the device may have a suitable size and shape for being at least partially disposed in one of the temple members 346 and 347. In one embodiment, the computer 361 has a size and shape similar to that of one of the temple members 346 and 347, and is therefore disposed, if not completely, almost completely within the structure and constraints of such temple members 346 and 347. In one embodiment, the computer 361 may be disposed in both temple members 346 and 347. The computer 361 may include one or more processors with memory, a wireless communication circuitry, and a power supply. The computer 361 includes low-power circuitry, high-speed circuitry, and a display processor. Various other embodiments may include these elements in different configurations or integrated in different ways.

[0040] The computer 361 additionally includes a battery 362 or other suitable portable power supply device. In one embodiment, the battery 362 is disposed in one of the temple members 346 or 347. Figure 3 In the illustrated glasses 331, battery 362 is shown disposed in the left temple 346 and electrically coupled to the remainder of computer 361 disposed in the right temple 347 via connection 374. One or more I / O devices may include connectors or ports (not shown) suitable for charging battery 362, which is accessible from outside frame 332, wireless receivers, transmitters or transceivers (not shown), or combinations of such devices. Given the limited size of glasses 331 and computer 361, resource-intensive operations such as video streaming may rapidly deplete battery 362 and may stress one or more processors of computer 361, potentially leading to overheating.

[0041] The glasses 331 include a digital camera device 367. Although two camera devices 367 are depicted, other embodiments are contemplated to use a single or additional (i.e., more than two) camera devices. For ease of description, various features associated with a camera device 367 will be further described with reference only to a single camera device 367; however, it should be understood that in suitable embodiments, these features may be applied to both camera devices 367.

[0042] Consistent with some implementations, glasses 331 are an example instance of client device 102 and can be worn by user 103-1. Furthermore, in these implementations, user 103-2 can view a live camera feed generated by camera device 367 and interact with user 103-1 by adding virtual content items to the real-world environment visible to user 103-1 via glasses 331. That is, one or more media objects corresponding to virtual content items selected by user 103-2 can be displayed via an integrated near-eye display mechanism that overlays the media objects onto the real-world environment visible through optical elements 343 and 344.

[0043] In various embodiments, in addition to the camera device 367, the glasses 331 may include any number of input sensors or peripheral devices. The front member 333 is provided with: an outward-facing, forward-facing front or outer surface 366 that faces forward or away from the user when the glasses 331 are mounted on the user's face; and an opposing inward-facing, rearward-facing rear or inner surface 369 that faces the user's (e.g., user 103-1) face when the glasses 331 are mounted on the user's face. Such sensors may include: an inward-facing video sensor or digital imaging module, for example, which may be mounted or disposed within the inner surface 369 of the front member 333 or in other locations on the frame 332 to face the camera device; and an outward-facing video sensor or digital imaging module, for example, which may be mounted or disposed within the outer surface 366 of the front member 333 or in other locations on the frame 332 to face away from the camera device 367. Such sensors, peripherals, or peripheral devices may additionally include biosensors, position sensors, accelerometers, or any other such sensors.

[0044] The glasses 331 also include, in an example embodiment, a camera device control mechanism or user input mechanism, which includes a camera device control button mounted on the frame 332 for tactile or manual engagement by the user. The camera device control button provides a bimodal or single-action mechanism in which the user can use it only between two states: an engaged state and a disengaged state. In this example embodiment, the camera device control button is a button that is in the disengaged state by default, and can be pressed by the user to set it to the engaged state. When the pressed camera device control button is released, it automatically returns to the disengaged state.

[0045] In other embodiments, the single-action input mechanism may alternatively be provided by, for example, a touch-sensitive button including a capacitive sensor mounted adjacent to the surface of the touch-sensitive button on the frame 332, for detecting the presence of a user's finger so that the touch-sensitive button is set to an engaged state when the user touches a corresponding point on the outer surface of the frame 332. It should be understood that the camera device control button and the capacitive touch button described above are merely two examples of tactile input mechanisms for single-action control of the camera device 367, and other embodiments may employ different single-action tactile control arrangements.

[0046] Figure 4 This is a block diagram illustrating aspects of a wearable device in an exemplary form of glasses 331 according to some exemplary embodiments. The computer 361 of the glasses 331 includes a central processing unit 421 communicating with onboard memory 426. The central processing unit 421 may be a CPU and / or a graphics processing unit (GPU). The memory 426 in this exemplary embodiment includes a combination of flash memory and random access memory.

[0047] The glasses 331 also include a camera device controller 414 that communicates with the central processing unit 421 and the camera device 367. The camera device controller 414 includes a circuit system configured to: control the recording of photographic or video content based on processing of control signals received from a single-action input mechanism including camera device control buttons; and provide automatic adjustment of one or more image capture parameters related to image data captured by the camera device 367 and onboard processing of the image data before persistent storage and / or presenting the image data to the user for viewing or preview.

[0048] In some embodiments, the camera device controller 414 includes a permanently configured circuitry, such as firmware or an application-specific integrated circuit (ASIC) configured to perform the various functions described herein. In other embodiments, the camera device controller 414 may include a dynamically reconfigurable processor that executes instructions that temporarily configure the processor to perform the various functions described herein.

[0049] The camera device controller 414 interacts with the memory 426 to store, organize, and present image content in the form of photographic and video content. For this purpose, the memory 426 in this example embodiment includes a photographic content memory 428 and a video content memory 442. Therefore, the camera device controller 414 cooperates with a central processing unit 421, which is configured to: receive image data representing digital images generated by the camera device 367 according to some image capture parameters; process the image data according to some image capture parameters; and store the processed image data in one of the appropriate photographic content memory 428 and video content memory 442.

[0050] The camera device controller 414 is also configured to cooperate with the display controller 449 to display selected photos and videos from the memory 426 on a display mechanism incorporated in the glasses 331, and thus provide a preview of the captured photos and videos. In some embodiments, the camera device controller 414 will manage the processing of images captured using automatic bracketing parameters for inclusion in a video file.

[0051] A single-action input mechanism 435 is communicatively coupled to a central processing unit 421 and a camera device controller 414 to transmit a signal representing the current state of a camera device control button, thereby transmitting to the camera device controller 414 regardless of whether the camera device control button is currently being pressed. The camera device controller 414 also communicates with the central processing unit 421 regarding input signals received from the single-action input mechanism 435. In one embodiment, the camera device controller 414 is configured to process input signals received via the single-action input mechanism 435 to determine whether a specific user engagement with the camera device control button results in the recording of video or photographic content and / or dynamically adjust one or more image capture parameters based on the processing of the input signal. For example, a press of a camera device control button for a duration longer than a predetermined threshold causes the camera device controller 414 to automatically apply relatively less stringent video processing to the captured video content before persistent storage and display of the captured video content. Conversely, in such an implementation, pressing the camera control button for a duration shorter than the threshold causes the camera controller 414 to automatically apply relatively more stringent photo stabilization processing to the image data representing one or more still images.

[0052] Glasses 331 can be a standalone client device capable of independent operation or a companion device 108 that works with a main device to offload intensive processing and / or exchange data with a communication server system via network 106. Glasses 331 may also include various components common to mobile electronic devices such as smart glasses or smartphones (e.g., a display controller for controlling the display of visual media (including photographs and video content captured by camera device 367) on a display mechanism incorporated into the device). Note that... Figure 4 The schematic diagram is not a detailed representation of all the components that make up the eyeglasses 331.

[0053] Figures 5A to 5H This is a conceptual diagram illustrating the flow of an example virtual interaction session between a first user (e.g., user 103-1) and a second user (e.g., user 103-2) according to some example implementations. In the context of the virtual interaction session, one or more users can establish a real-time communication link with the first user based on the first user's permission and invitation to establish a real-time communication link with one or more other users. When a real-time communication link is established between the first user and the second user, the second user can access their own device—device 502 (as described herein and...) Figures 5A to 5H In this context, it is referred to as the "second device"; for example, the display of client device 102-2 shows the device viewed by the first user—device 500 (as described in this document). Figures 5A to 5H The device is referred to as the "first device"; for example, the client device 102-1 generates a live camera feed (e.g., including real-time image data) and also interacts with the first user using verbal and augmented reality communication modalities.

[0054] In the example implementations discussed below, the first device is a wearable device (e.g., glasses 331) having an embedded camera and optical elements including a display (e.g., a transparent display). Therefore, the following discussion refers to such a wearable device. Figures 5A to 5H However, while the examples described below may refer to an implementation in which information is presented by a display of a first device, it should be understood that such information may alternatively or additionally be presented by a main device coupled to the wearable device. The wearable device may be a standalone device capable of independent operation, or it may be a companion device that works with the main device to offload intensive processing.

[0055] Consistent with these implementations, a second user can utilize a client application (e.g., client application 104) running on the second device to view live camera feeds generated by the first device while interacting with the first user. Similarly, the first user can utilize a client application (e.g., client application 104) running on the first device to configure and initiate virtual interactive sessions.

[0056] Reference Figure 5A The diagram illustrates a session configuration interface 504 displayed by a first device. The session configuration interface 504 allows a first user to configure session parameters for a virtual interactive session. To this end, the session configuration interface includes multiple interface elements that the first user can use to input configuration parameters.

[0057] As shown, the first user can specify configuration parameters, such as an identifier corresponding to the user allowed to join the virtual interactive session—regardless of whether the first user approves it before the real-time communication link is established (“Call in” vs. “Drop in”), session duration, micro-chat duration, and ambiguity level. The session duration defines the time limit of the virtual interactive session, and the micro-chat duration defines the time limit of the real-time communication link established during the virtual interactive session. Multiple micro-chat sessions can exist within a single virtual interactive session. Real-time communication links with time limits may be referred to herein as “time-limited real-time communication links” or simply “micro-chats.”

[0058] Session configuration data, including one or more configuration parameters specified by the first user via session configuration interface 504, is provided to the application server (e.g., application server 112). In response to receiving the session configuration information, the application server initiates a virtual interactive session with the first user. Upon initiating the virtual interactive session, the application server provides an indicator of the virtual interactive session with the first user to the second device for display. This indicator may include interactive elements that allow the second user to join the virtual interactive session by establishing a real-time communication link with the first user. For example, the indicator may include a button that, when selected by the second user, triggers the sending of a request to join the virtual interactive session. More specifically, this request may include a request to establish a real-time communication link between the first user and the second user.

[0059] In some implementations, the indicator may be displayed within a feed of an available virtual interactive session or within a similar user interface element provided by a client application running on a second device. In some implementations, the indicator may be provided for display on the second device as a notification, pop-up window, or other such user interface element.

[0060] The application server provides an indicator for display on the second device until the session duration expires. Therefore, when a virtual interactive session is started, the application server starts a timer to measure the elapsed time and compares it to the session duration. The session duration expires when the elapsed time reaches the end of the session duration.

[0061] Based on a request received from the second device to establish a real-time communication link between the first user and the second user, the application server can cause an instruction to be displayed on the first device and / or the second device to start the real-time communication link. For example, as Figure 5B As shown, a countdown timer 506 is displayed on the second device (device 502) to notify the second user that a real-time communication link with the first user is about to begin. Although Figure 5B Only instructions provided for display on a second device are shown; however, it should be understood that in other examples, these instructions are also provided to the first device (device 500).

[0062] Continue to refer to Figure 5B The real-world environment 508 is visible to the first user via the optical elements of the first device. For example... Figure 5C As shown, as part of establishing a real-time communication link between the first user and the second user, the application server enables the second user to view the real-world environment 508 by displaying a live camera feed 510 generated by the first device on the second device. The live camera feed 510 depicts the real-world environment 508, as shown. Furthermore, as shown, the application server causes a countdown timer 512 to be displayed on both the first and second devices to provide an indicator of the remaining time in the real-time communication link. In this example, the countdown timer 512 provides a measurement of the remaining time in the real-time communication link until the micro-chat duration expires. In other example implementations, the countdown timer 512 may correspond to the remaining time in a virtual interactive session.

[0063] As described above, when a real-time communication link is established between the first user and the second user, the second user can control the content presented to the first user via the display of the first device. More specifically, the second user can view the real-world environment 508 visible to the first user and interact with the first user via augmented reality and verbal modality. To this end, the application server enables an audio communication link between the first device and the second device that allows verbal communication between the first user and the second user, and as... Figure 5DAs shown, the application server also enables a set of selectable virtual content items 514 to be displayed on the second device for transmission to the first device and applied to the real-world environment 508 visible to the first user. Each virtual content item includes one or more media objects (two-dimensional or three-dimensional media objects) that will be overlaid on the view of the real-world environment 508 visible to the first user via the first device.

[0064] like Figure 5E As shown, based on the second user selecting a virtual content item 516 from the set of optional virtual content items 514, the application server causes one or more media objects to be displayed on both the first and second devices, overlaid on the real-world environment 508. Figure 5E In the specific example shown, the virtual content item includes snow, and the virtual content item is applied to the real-world environment 508 so that it appears to the first user as if it is snowing at the first user's location.

[0065] In an implementation where the real-world environment 508 is visible to the user via a transparent display, such as using a first device in this example, the application server applies virtual content items to the real-world environment 508 by causing the transparent display to show one or more media objects. In an implementation where the real-world environment 508 is fed to the user via a live camera device, such as using a second device in this example, the application server applies virtual content items to the real-world environment 508 by enhancing the image data generated by the live camera feed 510 from the first device to include one or more media objects overlaid on the real-world environment 508.

[0066] like Figure 5F As shown, while the real-time communication link between the first user and the second user remains active, the second user can continue to select virtual content items to apply to the real-world environment 508 visible to the first user via the optical elements of the first device. Figure 5F In the example shown, the second user selects a second virtual content item including Santa Claus on a sleigh pulled by reindeer, and in response to this selection, the application server applies the second virtual content item together with the first virtual content item to the real-world environment 508 visible to the first user.

[0067] like Figure 5G As shown, before the micro-chat duration expires, the application server can provide the first user with the ability to extend the time limit on the real-time communication link by displaying a notification 518 that includes interactive elements that the first user can use to trigger an extension of the time limit.

[0068] If the first user does not extend the time limit, the countdown timer 506 will continue, and when the micro-chat duration expires, the application server will terminate the real-time link between the first and second users. Figure 5H As shown. When terminating the real-time link between the first user and the second user, the application server terminates the live camera feed 510 on the second device and disables the audio communication link between the first device and the second device. Furthermore, as... Figure 5H As shown, the application server can also display an indication 520 that the real-time communication link has been terminated on the second device, and can also display an interactive element 522 that enables the second user to trigger the sending of a request for an additional real-time communication link with the first user.

[0069] Figure 6A and Figure 6B This is an interaction diagram illustrating an example interaction between a first device, a second device, and a server (application server 112) of a communication system in a method 600 for performing a virtual interaction session to facilitate a first user associated with a first device (client device 102-1) and a second user associated with a second device (client device 102-2), according to an example implementation.

[0070] like Figure 6A As shown, method 600 begins at operation 602, in which application server 112 provides instructions to client device 102-1 for displaying a session configuration interface (e.g., session configuration interface 504). Client device 102-1 displays the session configuration interface at operation 604 and receives input via the session configuration interface at operation 606. The input defines one or more configuration parameters for a virtual interactive session with a first user. For example, the input may specify the duration of the micro-chat, defining the time limit of the real-time communication link between client devices 110-1 and 110-2. Client device 102-1 provides configuration data, including one or more configuration parameters, to application server 112, which is received at operation 608.

[0071] In response to receiving session configuration data, at operation 610, application server 112 initiates a virtual interactive session with the first user. Upon initiating the virtual interactive session, application server 112 triggers the activation of the embedded camera device of client device 102-1 (operation 612) and causes client device 102-1 to send live video feeds generated by the camera device back to application server 112 (operation 614). Based on the session configuration data indicating that the first user has allowed the second user to join the virtual interactive session, at operation 616, application server 112 further causes an indicator of the virtual interactive session to be displayed on client device 102-2. This indicator is provided to client device 102-2 to display the session duration and allows the second user to join the virtual interactive session by sending a request to establish a real-time communication link between client devices 110-1 and 110-2 (operation 618).

[0072] Based on a received request to establish a real-time communication link between client devices 110-1 and 110-2, at operation 620, application server 112 enables the real-time communication link between client devices 102-1 and 110-2. When establishing the real-time communication link, application server 112 enables an audio communication link between the devices, allowing user 106-1 and 106-2 to communicate verbally in real time. Application server 112 also causes client device 102-2 to display a live camera feed provided by client device 102-1 (at operation 622), thereby allowing user 103-2 to view the real-world environment from user 103-1's location and control the presentation of virtual content items to the user. For example, the live camera feed can be displayed on client device 102-2 as part of a user interface that includes a set of selectable virtual content items that can be applied to the real-world environment visible to user 103-1 via client device 102-1.

[0073] like Figure 6B As shown, at operation 624, the client device 102-2 receives the user's selection of a virtual content item and instructs the client device 102-2 to provide the input data of the user's selection to the application server 112, the input data being received at operation 626.

[0074] Based on the received input data, at operation 628, application server 112 instructs client devices 110-1 and 110-2 to display virtual content items overlaid on the real-world environment visible to user 103-1 at operations 630 and 632, respectively. Instructions are provided to client device 102-1 to display virtual content items on a display visible to the real-world environment (e.g., embedded in glasses 331). In embodiments where the display is transparent, the instructions cause client device 102-1 to display virtual content items, and because the display is transparent, the virtual content items appear to be overlaid on the real-world environment. In embodiments where the display is a non-transparent video display, client device 102-1 enhances the live camera feed presented to the user to include virtual content items overlaid on the real-world environment. Instructions are provided to client device 102-2 to enhance the live camera feed to include virtual content items overlaid on the real-world environment.

[0075] At operation 634, application server 112 detects the expiration of the micro-chat duration specified in the session configuration data. At operation 636, based on the detected expiration of the micro-chat duration, application server 112 terminates the real-time communication link between client devices 102-1 and 110-2. Upon terminating the real-time communication link, application server 112 terminates the live camera feed displayed on client device 102-2 and disables the audio communication link between the devices.

[0076] Figures 7 to 9 This is a flowchart illustrating the operation of a communication system when executing a method 700 for facilitating a virtual interactive session with a first user, according to an exemplary embodiment. Method 700 can be implemented in computer-readable instructions executable by one or more processors, such that the operation of method 700 can be performed partially or entirely by functional components of the communication system 100; therefore, method 700 will be described below by way of example with reference to it. However, it should be understood that at least some operations of method 700 can be deployed on various other hardware configurations besides the communication system 100.

[0077] At operation 705, application server 112 causes a session configuration interface (e.g., session configuration interface 504) to be displayed on a first device (e.g., client device 102-1). The first device is associated with a first user (e.g., user 103-1). The session configuration interface includes one or more elements for defining session configuration parameters associated with the virtual interaction session. As described above, in the context of a virtual interaction session, one or more users invited by the first user can establish a real-time communication link with the first user to view the real-world environment at the first user's location and interact with the first user using a communication modality that is both verbal and augmented. The session configuration interface may include a combination of input fields, toggles, and other user interface input elements that can be used to specify configuration parameters, including: a user identifier associated with other users authorized to participate in the virtual interaction session; whether the first user wants to approve it before the real-time communication link is established; a session duration defining the time limit of the virtual interaction session; a micro-chat duration defining the time limit of the real-time communication link established during the virtual interaction session; and a fuzzy level.

[0078] At operation 710, application server 112 receives session configuration data, including one or more session parameters specified by the first user, via a session configuration interface. As described above, the one or more configuration parameters may include identifiers associated with other users authorized to participate in the virtual interaction session, session duration, and micro-chat duration. In some implementations, the one or more configuration parameters may include parameters indicating whether micro-chat is enabled for the virtual interaction session. As described above, micro-chat is a time-limited real-time communication link between the first user and the second user. When the real-time communication link is established, the second user can view the real-world environment from the first user's location, and the first user and the second user can communicate verbally via an audio communication link enabled between the users' devices.

[0079] At operation 715, application server 112 initiates a virtual interaction session with the first user. Upon initiating the virtual interaction session, application server 112 activates a camera device coupled to the first device and causes the first device to send live video feeds generated by the camera device back to application server 112. In some embodiments, the camera device is an embedded camera device of the first device. In some embodiments, the camera device is an embedded camera device of an accessory to the first device, such as a wearable device (e.g., glasses 331).

[0080] At operation 720, application server 112 receives a request from the second device to establish a real-time communication link with the first device. The second device is associated with a second user. Application server 112 enables the second user to participate in the real-time communication link based on session configuration data specifying that the second user is authorized by the first user to participate in a virtual interactive session. In some embodiments, an invitation may be sent to the second device based on the session configuration data specifying that the second user is authorized by the first user to participate in a virtual interactive session. This invitation may include interactive elements (e.g., buttons) that enable the second user to submit a request to establish a real-time communication link with the first device.

[0081] At operation 725, application server 112 enables the second user to view the real-world environment at the location of the first user and interact with the first user by establishing a real-time communication link between the first and second devices. The real-world environment is the environment at the location of the first user that is visible within the field of view of the camera device coupled to the first device. Therefore, when establishing the real-time communication link, application server 112 causes the live camera feed (generated by the camera device and provided by the first device) to be displayed on the monitor of the second device.

[0082] At operation 730, application server 112 terminates the real-time communication link. In some instances, application server 112 terminates the real-time communication link based on user input received from one of the two devices (e.g., input corresponding to a request to terminate the real-time communication link). In some instances, application server 112 may terminate the real-time communication link based on the expiration of a session duration defined by session configuration data. Consistent with some implementations, application server 112 may terminate the real-time communication link based on the expiration of a micro-chat duration defined by session configuration data.

[0083] like Figure 8 As shown, in some embodiments, method 700 may include operations 805, 810, 815, 820, and 825. Consistent with these embodiments, operations 805, 810, 815, 820, and 825 may be performed after operation 720, in which application server 112 establishes a real-time communication link between the first device and the second device, and before operation 725, in which application server 112 terminates the real-time communication link. Additionally, consistent with some embodiments, operations 805, 810, 815, 820, and 825 may be performed as part of a method independent of the operations of method 700.

[0084] At operation 805, application server 112 causes a set of selectable virtual content items to be displayed on the second device (e.g., to be applied to a real-world environment depicted in a live camera feed provided by the first device at the location of the first user). Each virtual content item includes one or more media objects. The media objects can be two-dimensional or three-dimensional.

[0085] At operation 810, application server 112 receives user input from the second device, which instructs the second user to select a virtual content item (e.g., virtual content item 516) from a set of virtual content items (e.g., a set of selectable virtual content items 514) to apply to the real-world environment depicted in the live camera feed.

[0086] At operation 815, application server 112 verifies that the second user is permitted to apply virtual content items to the real-world environment depicted in the live camera feed. This is in contrast to an implementation where virtual content item selection is performed within a virtual interaction session with the first user (e.g., ...). Figure 8 Consistent with the implementation shown (performed as part of method 700), application server 112 verifies that the second user is permitted to apply virtual content items based on session configuration data. This is similar to some implementations, such as those that perform virtual content item selection as an independent experience separate from the virtual interaction session with the first user (e.g., ...). Figure 8 The operation shown is consistent with the implementation of the operation of method 700, which is independent of the operation execution of method 700. The application server 112 can verify that the second user is allowed to apply virtual content items based on the user profile data associated with the first user.

[0087] At operation 820, application server 112 causes both the first device and the second device to present one or more media objects overlaid on a real-world environment depicted in a live camera feed provided by the first device, based on selected virtual content items. Application server 112 may maintain object definition data defining the display of virtual content items (i.e., one or more media objects), and when causing the display of one or more media objects, application server 112 may provide the first device and the second device with the virtual content item definition data and a set of instructions that cause the first device and the second device to display one or more media objects overlaid on the real-world environment according to the virtual content item definition data.

[0088] In some embodiments, the first device is a wearable device worn by a first user, including optical elements with a transparent display device. Consistent with these embodiments, the application server 112 causes the transparent display device to display one or more media objects while allowing the first user to continue viewing the real-world environment through the device. In this way, one or more media objects are presented by a transparent display device overlaid on the real-world environment.

[0089] In some embodiments, the first device is a wearable device worn by a first user, including a non-transparent video display device. Consistent with these embodiments, the application server 112 causes the display device to enhance image data depicting the real-world environment overlaid thereon.

[0090] Instructions provided to the second device cause the second device to display an enhanced live camera feed that includes one or more media objects overlaid on a real-world environment. In some embodiments, the application server 112 may work with a client application running on the second device to enhance the image data from the live camera feed to include one or more media objects overlaid on a real-world environment.

[0091] Consistent with some implementations, when displaying the enhanced live camera feed at the second device, application server 112 can, for example, blur one or more portions of the image or otherwise blur them based on a blur level specified by configuration parameters. As an example, the background of the live camera feed presented by the second device can be blurred based on the blur level, while the foreground of the live camera feed remains sharp. In another example, the entire image can be uniformly blurred based on the blur level.

[0092] At operation 825, application server 112 causes an indicator to be displayed on the first device indicating that the second user is viewing a live camera feed provided by the first device. This indicator may include at least an identifier associated with the second user.

[0093] like Figure 9 As shown, in some embodiments, method 700 may include operations 905, 910, 915, 920, 925, 930, 935, 940, and 945. Consistent with these embodiments, operations 905 and 910 may be performed as part of operation 710, in which application server 112 receives session configuration information. At operation 905, application server 112 receives the session duration. The session duration defines the time limit of the virtual interactive session. At operation 910, application server 112 receives the user-specified micro-chat duration. The user-specified micro-chat duration defines the time limit of the real-time communication link between the first user and the second user.

[0094] Consistent with some implementations, operation 915 can be performed as part of operation 715, in which application server 112 initiates a virtual interactive session. At operation 910, application server 112 provides an indicator of the virtual interactive session to the second device for display until the session duration expires. This indicator includes an interactive element (e.g., a button) that can be used by the second user to trigger a request to application server 112 to establish a real-time communication link with the first user.

[0095] Consistent with these implementations, operations 920, 925, and 930 can be performed as part of operation 720, in which application server 112 enables a real-time communication link between the first user and the second user. In operation 920, application server 112 causes the live camera feed generated by the first device to be displayed on the second device.

[0096] At operation 925, application server 112 enables a real-time audio communication link between the first device and the second device. This audio communication link allows the first user and the second user to communicate verbally in real time.

[0097] At operation 930, application server 112 causes indicators of remaining duration to be displayed on both the first and second devices. The remaining duration may correspond to the remaining time in the virtual interaction session or the remaining time in the real-time communication link between the first and second users, depending on whether the session configuration data specifies that a timed real-time communication link is enabled. In some implementations, application server 112 causes both indicators of the remaining time in the virtual interaction session and indicators of the remaining time in the real-time communication link to be displayed.

[0098] Consistent with some implementations, operation 935 can be performed before operation 730, in which application server 112 terminates the real-time communication link between the first user and the second user. At operation 935, application server 112 detects the expiration of the micro-chat duration. In these implementations, application server 112 terminates the real-time link between the first user and the second user based on the expiration of the micro-chat duration.

[0099] Consistent with these implementations, operations 940 and 945 can be performed as part of operation 720, in which application server 112 terminates the real-time communication link between the first user and the second user. At operation 940, application server 112 terminates the display of the live camera feed (generated by the first device) on the second device. Upon termination of the live camera feed, the second user can no longer view the first user's real-world environment. At operation 945, application server 112 disables the audio communication link between the first and second devices, thereby eliminating the ability for real-time verbal communication between the first and second users. In some implementations, application server 112 may also cause an indicator indicating that the real-time communication link has been terminated to be displayed on the first and / or second devices.

[0100] Software Architecture

[0101] Figure 10 This is a block diagram illustrating an example software architecture 1006, which can be used in conjunction with various hardware architectures described herein. Figure 10 This is a non-limiting example of a software architecture, and it should be understood that many other architectures can be implemented to facilitate the functionality described herein. Software Architecture 1006 can be implemented in, for example... Figure 11 The execution is performed on the hardware of machine 1100, which includes processor 1104, memory / storage device 1106, and I / O components 1118, etc. A representative hardware layer 1052 is shown and can represent, for example... Figure 11 The machine 1100. A representative hardware layer 1052 includes a processing unit 1054 having associated executable instructions 1004. The executable instructions 1004 represent executable instructions of the software architecture 1006, including implementations of the methods, components, etc., described herein. Hardware layer 1052 also includes a memory and / or storage device module 1056 that also has executable instructions 1004. Hardware layer 1052 may also include other hardware 1058.

[0102] exist Figure 10In the example architecture, software architecture 1006 can be conceptualized as a stack of layers, where each layer provides specific functionality. For example, software architecture 1006 may include layers such as operating system 1002, library 1020, framework / middleware 1018, application 1016, and presentation layer 1014. Operationally, application 1016 and / or other components within a layer can call APIs via the software stack, invoking API call 1008 and receiving responses to API call 1008 as message 1012. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide framework / middleware 1018, while others may provide such a layer. Other software architectures may include additional or different layers.

[0103] Operating system 1002 can manage hardware resources and provide public services. Operating system 1002 may include, for example, kernel 1022, services 1024, and drivers 1026. Kernel 1022 can act as an abstraction layer between hardware and other software layers. For example, kernel 1022 can be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 1024 can provide other public services to other software layers. Drivers 1026 are responsible for controlling the underlying hardware or interfacing with the underlying hardware. For example, drivers 1026 include display drivers, camera drivers, etc. Drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc., depend on the hardware configuration.

[0104] Library 1020 provides common infrastructure used by application 1016 and / or other components and / or layers. Library 1020 provides functionality that allows other software components to perform tasks more easily than by directly interfacing with the functions of the underlying operating system 1002 (e.g., kernel 1022, services 1024, and / or drivers 1026). Library 1020 may include system libraries 1044 (e.g., the C standard library), which provide functions such as memory allocation, string manipulation, mathematical functions, etc. Furthermore, library 1020 may include API libraries 1046, such as media libraries (e.g., libraries supporting the rendering and manipulation of various media formats such as MPEG4, H.294, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., OpenGL frameworks for rendering 2D and 3D graphics content on a display), database libraries (e.g., SQLite providing various relational database functions), web libraries (e.g., WebKit providing web browsing functionality), and so on. Library 1020 may also include a wide variety of other libraries 1048 to provide many other APIs to application 1016 and other software components / modules.

[0105] Framework / Middleware 1018 provides a higher level of common infrastructure that can be used by application 1016 and / or other software components / modules. For example, framework / middleware 1018 can provide various GUI functions, advanced resource management, advanced location services, and so on. Framework / middleware 1018 can provide a wide range of other APIs that can be used by application 1016 and / or other software components / modules, some of which may be specific to a particular operating system 1002 or platform.

[0106] Application 1016 includes built-in application 1038 and / or third-party application 1040. Examples of representative built-in applications 1038 may include, but are not limited to, contact applications, browser applications, book reader applications, location applications, media applications, messaging applications, and / or game applications. Third-party applications 1040 may include those used by entities other than the platform-specific vendor using Android. TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM , Mobile software running on the phone's mobile operating system or other mobile operating systems. Third-party application 1040 may call API calls 1008 provided by the mobile operating system (e.g., operating system 1002) to facilitate the functions described herein.

[0107] Application 1016 can use built-in operating system functions (e.g., kernel 1022, service 1024, and / or driver 1026), libraries 1020, and frameworks / middleware 1018 to create a user interface to interact with the system's user. Alternatively or additionally, in some systems, interaction with the user can occur through a presentation layer such as presentation layer 1014. In these systems, the application / component "logic" can be separated from the application / component's user-interactive aspects.

[0108] Figure 11 This is a block diagram illustrating components of a machine 1100 according to some example embodiments, which is capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and executing any or more of the methods discussed herein. Specifically, Figure 11 A graphical representation of a machine 1100 in an example form of a computer system is shown, in which instructions 1110 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause machine 700 to perform any or more of the methods discussed herein. Therefore, instructions 1110 can be used to implement the modules or components described herein. Instructions 1110 transform a general, unprogrammed machine 1100 into a specific machine 1100 programmed to perform the described and illustrated functions in the described manner. In alternative embodiments, machine 1100 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 1100 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1100 may include, but is not limited to, server computers, client computers, PCs, tablet computers, laptop computers, netbooks, set-top boxes (STBs), PDAs, entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart home appliances), other smart devices, web devices, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instructions 1110, which specify the actions to be taken by machine 1100. Furthermore, although only a single machine 1100 is shown, the term "machine" should also be considered as a collection of machines that individually or jointly execute instructions 1110 to perform any or more of the methods discussed herein.

[0109] Machine 1100 may include processor 1104, memory / storage device 1106, and I / O components 1118, which may be configured to communicate with each other, for example, via bus 1102. In an example embodiment, processor 1104 (e.g., CPU, Reduced Instruction Set Computing (RISC) processor, Complex Instruction Set Computing (CISC) processor, GPU, Digital Signal Processor (DSP), ASIC, Radio Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 1108 and 1109 capable of executing instructions 1110. Although Figure 11 Multiple processors 1104 are shown, but machine 1100 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0110] Memory / storage device 1106 may include memory 1112, such as main memory or other memory storage devices, and storage cells 1114, which processor 1104 can access, for example, via bus 1102. Storage cells 1114 and memory 1112 store instructions 1110 embodying any one or more of the methods or functions described herein. Instructions 1110 may also reside wholly or partially within memory 1112, storage cells 1114, at least one of processor 1104 (e.g., the processor's cache memory), or any suitable combination thereof during execution by machine 1100. Therefore, memory 1112, storage cells 1114, and the memory of processor 1104 are examples of machine-readable media.

[0111] I / O component 1118 may include a wide variety of components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O component 1118 included in a particular machine 1100 will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O component 1118 may include... Figure 11Many other components are not shown. The I / O components 1118 are grouped by function only for the sake of simplicity in the following discussion, and this grouping is by no means limiting. In various example embodiments, the I / O components 1118 may include output components 1126 and input components 1128. Output components 1126 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tube (CRT) displays), auditory components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, and so on. Input components 1128 may include alphanumeric input components (e.g., keyboards, touchscreen displays configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreen displays or other haptic input components that provide touch position and / or touch force or touch gesture), audio input components (e.g., microphones), and so on.

[0112] In another example implementation, I / O component 1118 may include various other components such as biometric component 1130, motion component 1134, environmental component 1136, or positioning component 1138. For example, biometric component 1130 may include components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1134 may include accelerometer components (e.g., accelerometer), gravity sensor components, rotation sensor components (e.g., gyroscope), and so on. Environmental component 1136 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers to detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an hearing sensor component (e.g., one or more microphones to detect background noise), a proximity sensor component (e.g., an infrared sensor to detect nearby objects), a gas sensor (e.g., a gas sensor to detect the concentration of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Positioning component 1138 may include a position sensor component (e.g., a Global Positioning System (GPS) receiver component), an altitude sensor component (e.g., an altimeter or barometer to detect air pressure from which altitude can be obtained), an orientation sensor component (e.g., a magnetometer), etc.

[0113] A wide variety of technologies can be used to implement communication. I / O component 1118 may include communication component 1140, which is operable to couple machine 1100 to network 1132 or device 1120 via coupling 1124 and coupling 1122, respectively. For example, communication component 1140 may include a network interface component or other suitable device for interfacing with network 1132. In further examples, communication component 1140 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, etc. Components (e.g.) (low power consumption) Components and other communication components for providing communication via other means. Device 1120 can be another machine or any variety of peripheral devices (e.g., peripheral devices coupled via USB).

[0114] Furthermore, the communication component 1140 can detect identifiers or may include components operable to detect identifiers. For example, the communication component 1140 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) bars, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, data matrices, data symbols, MaxiCode, PDF4114, Ultra Code, UCC RSS-2D barcodes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying audio signals from tags). Additionally, various information can be obtained via the communication component 1140, such as location via Internet Protocol (IP) geolocation, etc. Location can be obtained through signal triangulation or by detecting NFC beacon signals that indicate a specific location.

[0115] Glossary

[0116] In this context, "carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of these instructions. Instructions can be sent or received over a network using a transmission medium via a network interface device and using any of a number of well-known transmission protocols.

[0117] In this context, "client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, PDAs, smartphones, tablet computers, ultrabooks, netbooks, laptops, multiprocessor systems, microprocessor-based or programmable consumer electronics systems, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0118] In this context, "communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), wireless wide area network (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS (Plain Old-Style Telephone Service) network, a cellular telephone network, a wireless network, etc. A network, another type of network, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and coupling to the network may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, coupling can implement any of a variety of data transmission technologies, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other standards defined by various standards-setting organizations, other telematics protocols, or other data transmission technologies.

[0119] In this context, "machine-readable medium" refers to a component, device, or other tangible medium capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EPROM)) and / or any suitable combination thereof. The term "machine-readable medium" should be considered to include a single medium or multiple media capable of storing instructions (e.g., a centralized or distributed database or associated cache and server). The term "machine-readable medium" should also be considered to include any medium or combination of media capable of storing machine-executable instructions (e.g., code) such that, when executed by one or more processors of the machine, the instructions cause the machine to perform any or more methods described herein. Therefore, "machine-readable medium" refers to a single storage device or apparatus, and a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" does not include signals themselves.

[0120] In this context, "component" refers to a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies that provide partitioning or modularity for specific processing or control functions. A component can be combined with other components via its interface to perform machine processing. A component can be a packaged functional hardware unit designed for use with other components and can be part of a program that typically performs a specific function within a related function. A component can constitute a software component (e.g., code embodied on a machine-readable medium) or a hardware component.

[0121] A “hardware component” is a tangible unit capable of performing certain operations and can be physically configured or arranged. In various example implementations, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or a group of processors) of a computer system may be configured by software (e.g., an application or an application portion) to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic permanently configured to perform certain operations. A hardware component may be a dedicated processor, such as a field-programmable gate array (FPGA) or an ASIC. A hardware component may also include programmable logic or circuitry temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor.

[0122] Once configured via such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function, rather than a general-purpose processor. It should be understood that decisions to implement hardware components mechanically in dedicated and permanently configured circuit systems or in temporarily configured circuit systems (e.g., configured by software) can be driven by cost and time considerations. Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to include tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform some of the operations described herein.

[0123] Considering the implementation where hardware components are temporarily configured (e.g., programmed), it is not necessary to configure or instantiate each hardware component at any given time. For example, in cases where the hardware components include a general-purpose processor configured by software as a dedicated processor, the general-purpose processor can be configured as different dedicated processors (e.g., including different hardware components) at different times. The software accordingly configures one or more specific processors to constitute a specific hardware component at one time and different hardware components at different times.

[0124] Hardware components can provide information to and receive information from other hardware components. Accordingly, the described hardware components can be considered communicatively coupled. In the presence of multiple hardware components, communication can be achieved through signal transmission between two or more hardware components (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving information from that memory structure. For example, a hardware component can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. Other hardware components can then access the memory device at a subsequent time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and can operate on resources (e.g., collections of information).

[0125] The various operations of the example methods described herein can be performed, at least in part, by one or more processors configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute components of a processor implementation that perform the operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented, at least in part, by processors, where a particular processor or one or more processors are examples of hardware. For example, at least some of the operations of the methods can be performed by one or more processors or processor-implemented components.

[0126] Furthermore, one or more processors may operate to support the execution of related operations in a “cloud computing” environment or as a “Software as a Service” (SaaS) operation. For example, at least some of the operations may be executed by a group of computers (as an example of a machine including processors), wherein these operations are accessible via a network (e.g., the Internet) and via one or more suitable interfaces (e.g., application programming interfaces (APIs)). The execution of some operations may be distributed among processors, not residing within a single machine, but deployed across multiple machines. In some example implementations, the processor or processor-implemented components may reside in a single geographic location (e.g., in a home environment, office environment, or server cluster). In other example implementations, the processor or processor-implemented components may be distributed across several geographic locations.

[0127] In this context, "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and generates corresponding output signals used to operate the machine. For example, a processor can be a CPU, RISC processor, CISC processor, GPU, DSP, ASIC, RFIC, or any combination thereof. A processor can also be a multi-core processor with two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously.

[0128] In this context, a "timestamp" refers to a series of characters or encoded information that identifies when an event occurred (for example, given a date and time of day), sometimes accurate to a fraction of a second.

Claims

1. A method for facilitating virtual interactive sessions between users, comprising: This allows a session configuration interface to be displayed on a first user's first device to specify session configuration data for a virtual interaction session with the first user. The session configuration interface includes multiple interface elements for specifying configuration parameters for the virtual interaction session. These multiple interface elements include: A first interface element for specifying the identifier of a second user allowed to join the virtual interactive session; and A second interface element for specifying the duration of a micro-chat, the duration of which is defined as the time limit of the real-time communication link between the first user and the second user during the virtual interaction session; The session configuration data is received from the first user's first device. The session configuration data includes configuration parameters for a virtual interaction session with the first user. The configuration parameters include the identifier of the second user who is allowed to join the virtual interaction session and the duration of the micro-chat. Establishing a first real-time communication link between the first user and the second user, the establishment of the first real-time communication link includes feeding a live camera device generated at the first device and displayed by the second device, the live camera device feeding including an image of the real-world environment visible to the first user; When the duration of the micro-chat expires, the first real-time communication link between the first user and the second user is terminated. Terminating the first real-time communication link includes terminating the live video feed displayed through the second device. When terminating the first real-time communication link between the first user and the second user, an interactive element is provided to the second device to trigger the transmission of a request for a second real-time communication link between the first user and the second user; and While the virtual interaction session remains active, in response to receiving the request, a second real-time communication link is established between the first user and the second user.

2. The method according to claim 1, wherein, Establishing the first real-time communication link between the first user and the second user also includes enabling the audio communication link between the first device and the second device.

3. The method according to claim 2, wherein, Terminating the first real-time communication link also includes disabling the audio communication link between the first device and the second device.

4. The method according to claim 1, further comprising: This enables the display of an indicator of the remaining time in the first real-time communication link via the first device and the second device.

5. The method according to claim 1, further comprising: This causes the first device to display an indicator that the second user is viewing the live camera feed.

6. The method according to claim 1, further comprising: When the first real-time communication link is established between the first user and the second user, the first user is able to extend the duration of the micro-chat.

7. The method according to claim 6, wherein, Enabling the first user to extend the duration of the micro-chat includes displaying interactive elements that trigger the extension of the micro-chat duration via the first device.

8. The method according to claim 1, further comprising: This allows a set of selectable virtual content items to be displayed via the first device for application to the real-world environment visible to the second user.

9. The method according to claim 8, further comprising: The second user receives input data from the second device, the input data instructing the second user to select virtual content items from the set of selectable virtual content items to apply to the real-world environment visible to the first user, the virtual content items comprising one or more media objects; as well as Based on the received input data, the one or more media objects overlaid on the live camera feed generated by the first device are displayed via the first device and the second device.

10. The method according to claim 9, wherein, Displaying the one or more media objects via the second device includes enhancing the feed from the live camera device to include the one or more media objects overlaid on the real-world environment.

11. The method according to claim 9, wherein: The first device is a wearable device worn by the second user; The wearable device includes a transparent display; and Displaying the one or more media objects overlaid on the real-world environment by the first device includes causing the transparent display to render the one or more media objects.

12. The method according to claim 1, further comprising: Based on the session configuration data, an indicator of the virtual interaction session with the first user is displayed on the second device.

13. The method according to claim 12, wherein, The configuration parameters also include session duration, wherein the indicator is provided to the second device for display until the session duration expires.

14. A system comprising: A memory that stores instructions; as well as One or more processors, said one or more processors being configured by the instructions to perform operations including: This allows a session configuration interface to be displayed on a first user's first device to specify session configuration data for a virtual interaction session with the first user. The session configuration interface includes multiple interface elements for specifying configuration parameters for the virtual interaction session. These multiple interface elements include: A first interface element for specifying the identifier of a second user allowed to join the virtual interactive session; and A second interface element for specifying the duration of a micro-chat, the duration of which is defined as the time limit of the real-time communication link between the first user and the second user during the virtual interaction session; The session configuration data is received from the first user's first device. The session configuration data includes configuration parameters for a virtual interaction session with the first user. The configuration parameters include the identifier of the second user who is allowed to join the virtual interaction session and the duration of the micro-chat. Establishing a first real-time communication link between the first user and the second user, the establishment of the first real-time communication link includes feeding a live camera device generated at the first device and displayed by the second device, the live camera device feeding including an image of the real-world environment visible to the first user; When the duration of the micro-chat expires, the first real-time communication link between the first user and the second user is terminated. Terminating the first real-time communication link includes terminating the live video feed displayed through the second device. When terminating the first real-time communication link between the first user and the second user, an interactive element is provided to the second device to trigger the transmission of a request for a second real-time communication link between the first user and the second user; and While the virtual interaction session remains active, in response to receiving the request, a second real-time communication link is established between the first user and the second user.

15. The system according to claim 14, wherein: Establishing the first real-time communication link between the first user and the second user also includes enabling the audio communication link between the first device and the second device; as well as Terminating the first real-time communication link also includes disabling the audio communication link between the first device and the second device.

16. The system of claim 14, further comprising: This enables the display of an indicator of the remaining time in the first real-time communication link via the first device and the second device; as well as This causes interactive elements that trigger an extension of the duration of the micro-chat to be displayed via the first device.

17. The system according to claim 14, wherein, The operation also includes: This enables the display of a set of selectable virtual content items via the first device to be applied to the real-world environment visible to the second user; The second user receives input data from the second device, the input data instructing the second user to select virtual content items from the set of selectable virtual content items to apply to the real-world environment visible to the first user, the virtual content items comprising one or more media objects; and Based on the received input data, the first device and the second device are used to display the one or more media objects overlaid on the live camera feed generated by the first device, wherein displaying the one or more media objects via the second device includes enhancing the display of the live camera feed to include the one or more media objects overlaid on the real-world environment.

18. The system according to claim 17, wherein: The first device is a wearable device worn by the second user; The wearable device includes a transparent display; and Displaying the one or more media objects overlaid on the real-world environment by the first device includes causing the transparent display to render the one or more media objects.

19. The system according to claim 14, wherein, The operation also includes: Based on the session configuration data, an indicator of the virtual interaction session with the first user is displayed on the second device. The configuration parameters also include session duration, and The indicator is provided to the second device for display until the session duration expires.

20. A machine-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations including: This allows a session configuration interface to be displayed on a first user's first device to specify session configuration data for a virtual interaction session with the first user. The session configuration interface includes multiple interface elements for specifying configuration parameters for the virtual interaction session. These multiple interface elements include: A first interface element used to specify the identifier of a second user allowed to join the virtual interactive session; as well as A second interface element for specifying the duration of a micro-chat, the duration of which is defined as the time limit of the real-time communication link between the first user and the second user during the virtual interaction session; The session configuration data is received from the first user's first device. The session configuration data includes configuration parameters for a virtual interaction session with the first user. The configuration parameters include the identifier of the second user who is allowed to join the virtual interaction session and the duration of the micro-chat. Establishing a first real-time communication link between the first user and the second user, the establishment of the first real-time communication link includes feeding a live camera device generated at the first device and displayed by the second device, the live camera device feeding including an image of the real-world environment visible to the first user; When the duration of the micro-chat expires, the first real-time communication link between the first user and the second user is terminated. Terminating the first real-time communication link includes terminating the live video feed displayed through the second device. When terminating the first real-time communication link between the first user and the second user, an interactive element is provided to the second device to trigger the transmission of a request for a second real-time communication link between the first user and the second user; as well as While the virtual interaction session remains active, in response to receiving the request, a second real-time communication link is established between the first user and the second user.

Images