Interactive viewing system

By providing an improved human-machine interface in a video conferencing session, allowing users to zoom in or out of detail in real time and manipulate rendering positions, solving the problem of inefficient interaction in the prior art and improving productivity and computing resource utilization efficiency.

CN113302581BActive Publication Date: 2024-12-06MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980084195.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-12-20
Filing Date
2019-12-11
Publication Date
2024-12-06
Estimated Expiration
2039-12-11

AI Technical Summary

Technical Problem

The inefficient interaction of existing collaboration environments leads to inefficiency of user productivity and computing resources, making it difficult for users to interact with content in video conferencing sessions quickly and easily, lacking the necessary tools to amplify areas of interest and maintain fidelity.

Method used

Provides an improved human-machine interface (HCI) that allows users to zoom in or out of details of stage canvas in real time during video conferencing sessions and manipulates their rendering position to interact through a freely floating zoom lens or window, supporting voice and gesture commands , allowing users to edit within the zoom window and improve fidelity.

Benefits of technology

Improves the interaction efficiency between users and the collaborative environment, optimizes the use of computing resources, such as processor cycles, memory and network bandwidth utilization, and achieves higher productivity and better perspective control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113302581B_ABST
    Figure CN113302581B_ABST
Patent Text Reader

Abstract

A tool for interacting with a rendered environment is configured to render a representation of a real-world environment and receive input indicating a location of a zoom window to be placed within the representation. The zoom window is rendered to have a size determined based on one or more criteria. An enlarged view of a portion of the representation proximate the location of the zoom window is rendered. Input data indicating a gesture is received, the gesture indicating a new location of the zoom window. The zoom window is repositioned on the UI, and the size of the zoom window is maintained during the repositioning. Within the zoom window, an enlarged view of a portion of the representation proximate the new location of the zoom window is rendered. The zoom window may be moved to any rendered portion of the representation.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Some computing systems provide collaborative environments to facilitate communication between two or more participants. Systems that provide collaborative environments may allow participants to exchange real-time video, real-time audio, and other forms of data in a communication session. The collaborative environment may employ any suitable communication session format, including but not limited to private chat sessions, multi-user editing sessions, group conferences, broadcasts, and the like.

[0002] Some existing systems have many shortcomings in promoting user participation. Inefficiencies in interacting with a collaborative environment can be detrimental to user productivity and the use of computing resources. When software applications fail to optimize user participation, production losses and inefficiencies in computing resources are exacerbated if participants cannot quickly and easily view the collaborative environment and selectively interact with the rendered content. For example, a live meeting may involve images showing participants discussing topics, presenting materials on a display within the meeting environment, and drawing graphics on a whiteboard. Existing systems lack the tools necessary to quickly and easily interact with such meeting activities. Such systems require users to perform many menu-driven tasks to zoom in on areas of interest. For example, when a participant needs more details about a portion of a rendered meeting environment, the participant must increase the zoom scale of the image. However, the entire rendered image is enlarged around the center of the image, and the desired zoom area may or may not be within the new enlarged image. In addition, fidelity is often lost. With respect to these and other considerations, the disclosure made herein is proposed. Summary of the invention

[0003] An improved human-computer interface ("HCI") is disclosed herein for interacting with representations of various environments, such as three-dimensional ("3D") real-world environments, and in some embodiments, involving scenes of video conferencing sessions. In various embodiments, a system for interacting with a communication environment utilizing video and other content is described. Such a system may be referred to as an interactive viewing system. The interactive viewing system may be provided in conjunction with a video conferencing session. The interactive viewing system may facilitate a shared communication environment that facilitates collaboration and other activities. For example, the interactive viewing system may be presented on each participant's device in a group control state that provides group control of viewing and editing functions and has the option of granting control to individual users. The rendered environment may include views of various viewing and editing tools that each participant may view while using the tools. In some embodiments, the rendered video conferencing session may be referred to as a stage canvas. In one embodiment, the viewing system allows a user to zoom in or out of details in the stage canvas video conferencing experience in real time and manipulate where and how the stage canvas is rendered. The viewing system can interact with, but is not limited to, videos, images, 3D models, office applications, captured environments / objects, annotations, demonstrations, shared locations, notes, expressions, or other shared activities. In some embodiments, a duplicate stage canvas layer visible to a user or interaction group can be implemented on the active stage canvas and operated as an independent annotation, picture, or other productivity file type. Therefore, the system allows any activity to be provided outside of a video conference session. In addition, the system can allow any activity to be recorded and available at different times.

[0004] In some embodiments, a free-floating zoom lens or window can be moved on the stage canvas, allowing the user to zoom in on a selected portion of the rendered screen experience and mark the selected portion for further review and manipulation by the interactive viewing system. In some embodiments, voice and / or gesture commands can be used to control the actions of the interactive viewing system. In addition, the zoom window can be moved to any area of ​​the depicted environment. In some embodiments, the zoom window can be attached to a specific object in the rendered environment, and the zoom window can follow the object as the object moves or as the rendered perspective changes. During the repositioning of the zoom window, the zoom ratio inside the zoom window can remain constant. In addition, during the repositioning of the zoom window, the size of the zoom window can remain unchanged.

[0005] In some embodiments, the zoom window can be configured to provide controls for interacting with the rendered environment. A user can, for example, interact with an image rendered within the zoom window. The image can be saved and edited as a screen summary or other multimedia object. The controls can also include editing the image source of the rendered environment and changing the perspective of the rendered environment, thereby allowing for improved fidelity and perspective of the image within the zoom window.

[0006] Existing systems for allowing users to manually interact with representations require users to perform many menu-driven tasks. Users may spend a lot of time searching available items to find and change settings, calling other applications to perform functions that the rendering application does not have, and looking for content related to a specific part of the rendered activity. This can result in a lot of unnecessary consumption of computing resources.

[0007] The examples described herein are provided in the context of a collaborative environment, e.g., a private chat session, a multi-user editing session, a group meeting, a live broadcast, etc. For purposes of illustration, it will be appreciated that a computer managing a collaborative environment refers to any type of computer managing a communication session in which two or more computers share data. For purposes of illustration, an "event" is a specific instance of a communication session that may have a start time, an end time, and other parameters for controlling how data is shared and displayed to users participating in the communication session.

[0008] The technology disclosed herein can enable users to efficiently manage rendered views, such as 3-D representations of real-world collaborative environments. This can allow for more efficient use of computing resources, such as processor cycles, memory, network bandwidth, and power consumption, compared to previous solutions. Other technical benefits not specifically mentioned herein can also be achieved through implementations of the disclosed subject matter.

[0009] It should be understood that various aspects of the subject matter briefly described above and described in further detail below may be implemented as hardware devices, computer-implemented methods, computer-controlled apparatus or devices, computing systems, or articles of manufacture, such as computer storage media. Although the subject matter described herein is presented in the general context of program modules executed on one or more computing devices, those skilled in the art will recognize that other implementations may be performed in conjunction with other types of program modules. Typically, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types.

[0010] Those skilled in the art will also appreciate that aspects of the subject matter described herein may be practiced on or with other computer system configurations than those specifically described herein, including multi-processor systems, microprocessor-based or programmable consumer electronics, AR, VR and MR devices, video gaming devices, PDAs, smartphones, smart TVs, self-driving cars, smart watches, e-readers, tablet computing devices, dedicated hardware devices, network devices, and the like.

[0011] By reading the following detailed description and reviewing the associated drawings, in addition to the features and technical benefits explicitly described above, other features and technical benefits will become apparent. This summary is provided to introduce some concepts in a simplified form, which are further described in the following detailed description. This summary is neither intended to identify key or essential features of the claimed subject matter nor to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a computing system diagram illustrating aspects of an operating environment for embodiments disclosed herein.

[0013] Figure 2 An example display according to one embodiment disclosed herein is shown.

[0014] Figure 3 An example user control according to one embodiment disclosed herein is shown.

[0015] Figure 4 An example display according to one embodiment disclosed herein is shown.

[0016] Figure 5 An illustrative display with a zoom window is shown in accordance with one embodiment disclosed herein.

[0017] Fig. 6A shows an illustrative display with a repositioned zoom window according to one embodiment disclosed herein;

[0018] Figure 6B shows an illustrative display with a repositioned zoom window according to one embodiment disclosed herein;

[0019] Figure 7 shows an illustrative display with a modified zoom window according to one embodiment disclosed herein;

[0020] Figure 8 shows an illustrative display with a modified zoom window according to one embodiment disclosed herein;

[0021] Fig. 9A shows an illustrative display with a selected zoom window according to one embodiment disclosed herein;

[0022] Fig. 9B shows an illustrative display with a selected zoom window according to one embodiment disclosed herein;

[0023] Fig.10 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0024] Fig.11 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0025] Fig.12 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0026] Fig.13 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0027] Fig.14 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0028] Fig.15 shows an illustrative display with a changing view according to one embodiment disclosed herein;

[0029] Fig.16 shows an illustrative display with a zoom window according to one embodiment disclosed herein;

[0030] Fig.17 shows an illustrative display with a zoom window according to one embodiment disclosed herein;

[0031] Fig.18 illustrates aspects of a routine according to one embodiment disclosed herein;

[0032] Fig.19 illustrates aspects of a routine according to one embodiment disclosed herein;

[0033] Fig. 20 illustrates aspects of a routine according to one embodiment disclosed herein;

[0034] Fig.21 is a computing system diagram showing aspects of an illustrative operating environment for the techniques disclosed herein;

[0035] Fig. 22is a computing device diagram illustrating various aspects of the configuration and operation of a device that may implement various aspects of the disclosed technology, according to one embodiment disclosed herein. DETAILED DESCRIPTION

[0036] The following detailed description describes an improved HCI for viewing and editing objects in a representation of an environment (e.g., a 3D representation of a real-world environment). This can result in more efficient utilization of computing resources such as processor cycles, memory, network bandwidth, and power compared to previous solutions that rely on inefficient interaction, selection, and editing of a rendered environment and objects in the rendered environment. Technical benefits in addition to those specifically described herein can also be realized by implementation of the disclosed technology.

[0037] Networked conferencing represents a popular form of electronic collaboration using applications (e.g., CISCO WEBEX provided by CISCO SYSTEMS, Inc. of San Jose, California, GOTOMEETING provided by CITRIX SYSTEMS, Inc. of Santa Clara, California, ZOOM provided by ZOOM Video Communications, Inc. of San Jose, California, GOOGLE HANGOUTS provided by ALPHABET, Inc. of Mountain View, California, SKYPE and SKYPE FOR BUSINESS and TEAMS provided by MICROSOFT CORPORATION of Redmond, Washington) to facilitate communication between two or more participants who are present in separate physical locations. Participants in a communication session in a networked conference are able to exchange real-time video, audio, and other types of content to view, listen to, and share information. Participants can also view a common space, such as a whiteboard or shared application, where ideas can be exchanged. Viewing of the common space can be supplemented with video and audio conferencing, instant messaging sessions, or any combination thereof, so that a network conference can serve as a close substitute for a face-to-face meeting.

[0038] Various types of computing devices may be utilized to participate in web conferences, including but not limited to smartphones, tablet computing devices, set-top boxes, smart televisions, video game systems, and AR, VR, and MR devices.

[0039] While meeting participants can view a real environment such as a meeting space, the ability to interact with the rendered environment is limited. As a result, remote participants typically must accept image and video feeds provided by the rendering application, regardless of the quality and fidelity of the feed. In addition, participants typically must use offline resources to supplement their access to meeting materials, e.g., requesting copies of presented documents and files, requesting meeting participants to take photos of items of interest, and so on. The disclosed HCI addresses the technical considerations set forth above, as well as potentially other technical considerations, thereby providing technical benefits to computing systems that implement the disclosed technology.

[0040] In various embodiments, a viewing and editing system that can be used in conjunction with a collaborative activity such as a web conference is disclosed. Such a system may also be referred to herein as a tool, but should not be construed as having different or less functionality than the system. In one embodiment, the viewing and editing tool includes a window or lens rendered on a representation of a live meeting. The window or lens can be moved to any part of the representation and can be further resized and / or scaled to zoom an area near the window or lens. In some embodiments, the zoom window / lens can be rotated or translated based on a rotation input gesture or a lateral scroll gesture. Other functionality can be implemented to enable users to better interact with such functionality. For example, in some embodiments, the thickness or other properties of the border of the window or lens can be changed to indicate the amount of zoom.

[0041] In some embodiments, the contents of a window or lens may be captured, saved, and edited. In some embodiments, options for further action may be determined based on the context of the window contents, the current meeting state and activity, and the user's role. Options for further action may include, for example, sending the content to a participant or other recipient.

[0042] The viewing and editing tools may include functions that facilitate the user to interact with the content and with other participants of the rendering activity. In the context of live video streaming, the user may interact with the video stream itself, such as interacting with the frames of the video stream. In addition, the viewing and editing tools may provide the ability to interact with the depicted content of the video stream (such as presented documents and files). The viewing and editing tools may further provide the ability to interact with various aspects of the environment depicted in the video stream, such as the device depicted in the video stream or the device that can provide input to the video stream. The user interactions enabled may include updating content, sharing content, and interacting with other participants via content.

[0043] In some embodiments, the depicted environment (e.g., a meeting) may be represented as an object that may be sent to a user who may access the meeting to interact with the object. For example, the recipient may be able to click on the object to join the meeting or view detailed information about the meeting.

[0044] In some embodiments, the viewing and editing tool can facilitate the detection of the object or other content being rendered and the source of the object or content. The object can be a document that can be identified, searched, accessed, downloaded and edited by the user. For example, the currently rendered slide of a Powerpoint presentation can be used to identify the source file of the presentation. In another example, if the rendered object is a device such as an electronic whiteboard, the device can be identified, and the user can be provided with options (if authorized) to control or provide input to the device, such as entering annotations to the whiteboard. In other embodiments, a virtual whiteboard can be instantiated and rendered, which can be edited by the participant via the viewing and editing tool. If the device is a camera, the user can be provided with the ability to change the focus of the camera or change other parameters of the camera. The viewing and editing tool can continuously identify the original source data of the content that is part of the rendered environment. This allows the user to quickly access the original content, rather than viewing the image of the content and searching for the content as a separate task. Therefore, the viewing and editing tool provides a centralized view of the content that the context can be used for the group during the communication session process.

[0045] In some embodiments, when multiple video sources are available, the viewing and editing tool can determine which video source can provide the best fidelity based on the position of the zoom window within the rendering environment. For example, the primary video feed used for the main video feed may not have the highest available resolution. When the user selects a position for the zoom window, a higher resolution image source (if available) can be used to provide a higher fidelity zoom of the selected area.

[0046] In some embodiments, a captured image of an activity, such as a meeting, may be linked to a time window. For example, an image selected for editing may be associated with a default time window, such as 30 seconds. The viewing and editing tools may provide a timeline tool to traverse the time range over which the activity occurred and over which the recording is available. In this way, the user may view various times of the activity from the perspective of the zoom window, and may also be provided with various editing options within the available time range.

[0047] In some embodiments, actions and features activated by a user during an editing / viewing session may be recorded and may be replayed by the user.

[0048] In some embodiments, the actions available to the viewing and editing tool can be dynamically updated based on the activity detected in the room. For example, if a presentation is rendered on a display within the rendering environment, the viewing and editing tool can update the user options to include actions that can be used to access and edit the display and / or presentation source.

[0049] In some embodiments, the actions available on the viewing and editing tools may be based on the roles assigned to the user. For example, some users may be assigned a producer role and may be allowed to edit content before sharing it on the network. Other users may have a contributor role or a group role and may be allowed to control their own settings or jointly control the settings of the group.

[0050] Turning now to the accompanying drawings (which may be referred to herein as "figures" or "figures"), additional details about the improved HCI disclosed herein will be provided with reference to the accompanying drawings. These figures show specific configurations or examples by way of illustration. In all figures, the same reference numerals represent the same or similar elements. In the figures, the leftmost digit of a reference numeral generally identifies the figure in which the reference numeral first appears. References to individual items of a plurality of items may use a reference number with another numeral in parentheses (and / or a letter without parentheses) to reference each individual item. General references to items may use a specific reference number without a letter sequence. The figures are not drawn to scale.

[0051] Figure 1 An example scenario is shown involving a system 100 associated with a tool for interacting with a rendering environment such as a collaborative environment. The rendering environment may include a workspace 120, which may be an office, conference room, auditorium, or other space configured to allow individuals to gather and collaborate. The workspace 120 may include cameras 111A and 111B. The environment may include other devices such as smart beacons 131. Other input sources ( Figure 1 11A, 111B, and 11C and microphones (not shown). Figure 1 ) and other input devices receive data and send the collected data as input data 107 to the collaboration server 101.

[0052] The collaboration server 101 may process the input data 107 and send the interaction data 106 to one or more user devices, such as the laptop 102 and the VR display 103. The interaction data 106 may include data that can be used to render an interactive display 150 that may include a representation of the workspace 120. The interactive data 106 may include any images, documents, video data, audio data, or any other information that can be used as data for rendering a representation of the workspace 120 and the activities occurring in the workspace 120 captured by the cameras 111A and 111B. The interactive data 106 may also include other forms of data, such as a meeting request, which may identify multiple attendees, titles associated with each attendee, and other relevant information. The interactive data 106 may also indicate parameters of the event, such as a start time, an end time, and a location. For example, the interactive data 106 may include meeting information that indicates a list of attendees, the role of each attendee, the date, time, and location.

[0053] The interactive data 106 may include any information that conveys viewing and editing preferences for parameters or settings related to the collaborative environment. For example, the interactive data 106 may define the user interface configuration, volume level, camera angle, or other parameters that a particular user has used. The interactive data 106 may also include historical information. For example, the interactive data 106 may include a list of meetings, the attendees of each meeting, and the UI layout used in each meeting.

[0054] Input data 107 may include a description of hardware available to computing device 121. For example, input data 107 may describe aspects of various input devices, sensors, lights, microphones, sound attenuation devices, and other hardware available to computing device 121. Input data 107 may also describe the specifications of a display screen or the specifications of a computer in communication with system 100.

[0055] Input data 107 may also describe the specifications of available hardware, such as, but not limited to, sensitivity level, zoom level, etc. Input data 107 may also describe the location of each device and the range of each device. For example, input data 107 may describe the location, positioning, and viewing area of ​​a particular camera, such as a camera that can capture a speaker on a particular podium, stage, etc. In another example, input data 107 may identify the location of a room microphone and the coordinates defining the microphone range. In this example, input data 107 indicates the availability of two cameras 111 (111A-11B). The interactive display 150 may use a device list to provide editing and viewing options, as further described herein. Input data 107 may also indicate that a first camera 111A is directed to a first area, and a second camera 111B is directed to a second area.

[0056] Figure 2 2 is a UI diagram illustrating aspects of an example UI 201 that enables computationally efficient interaction with a 3D representation of a real-world environment 202 according to one embodiment disclosed herein. The UI 201 may correspond to Figure 1 As briefly discussed above, the technology disclosed herein can be used in conjunction with an application that provides functionality for holding a web conference. The UI 201 presented by such an application is Figure 2-17 and described below.

[0057] UI 201 may include, for example, Figure 1 A rendering of the real environment 202 generated by the computing device 102 or VR device 103. In this way, the user of the computing device 102 or VR device 103 can see a view of the real environment 202 as well as the whiteboard 210, the display 200, and the participants 230. UI controls 220 can also be presented, which allow users to interact with the UI 201.

[0058] More details of UI control 220 are in Figure 3 In one embodiment, UI control 220 may include a circular wheel with selectable options 310. In one example, selectable options 310 may be configured to cause placement of a zoom window in UI 201. Although UI control 220 shown in this example is shown as a circular wheel, the interface may be implemented in a variety of forms, such as a rectangular list of options, a drop-down menu, and other forms.

[0059] The user may interact with UI 201 using mouse input, touch input, or other types of input. Figure 4 In the example shown, for example, the user has moved the mouse cursor 221 onto the UI control 220 to initiate the Figure 5 When the user completes the selection action using the mouse button, the zoom window 500 is rendered at the specified position, the previous position, or the default position on the UI 201. Figure 5 , zoom window 500 is shown as being close to display 200, and zoom window 500 is shown as rendering zoom details of the bottom of display 200. In some embodiments, zoom window 500 may include a plurality of selective rings or regions 501, 502, and 503, each of which may be associated with a function or action. For example, ring 501 may be associated with a reposition function, ring 502 may be associated with a resize function, and ring 503 may be associated with additional menu options.

[0060] exist Fig. 6AIn the illustrated embodiment, the user can tap and drag the outer ring of zoom window 500 (or other ring associated with the function) and move the position of zoom window 500. In one embodiment, the zoom ratio of the content within zoom window 500 can remain unchanged when zoom window 500 is repositioned. Fig. 6A As shown, zoom window 500 has been repositioned to the right of display 200, and the content of zoom window 500 is now rendering the text portion of the current content being rendered on display 200. Available imaging data, such as imaging data from a camera that can be used as an image source, can be used to provide the highest possible fidelity to the content within zoom window 500 when it is repositioned. For example, if rendering environment 201 is typically provided with multiple camera sources located at various locations within the rendering environment, the content of zoom window 500 can use different camera sources to provide the best available fidelity when the position of zoom window 500 is changed.

[0061] In some embodiments, the user can pan the content within zoom window 500 without resizing the window or changing the zoom ratio. Figure 6B In the illustrated embodiment, a user may input a pan gesture by touching an area within zoom window 500 and sliding the user's finger in a selected direction. Figure 6B As shown, the user has panned toward the right side of display 200. In some embodiments, the user can rotate the content within zoom window 500 without resizing the window or changing the zoom ratio. Figure 6B In the illustrated embodiment, a user may input a rotation gesture by touching an area within zoom window 500 to rotate the user's finger in a selected rotation direction. Available imaging data (e.g., from a camera that may be an image source for the rendering environment) may be used to provide the highest possible fidelity for panning content in zoom window 500. For example, if rendering environment 201 typically has a 1K camera source but a 4K camera is available, the content of zoom window 500 may use the 4K camera source to provide more clarity when panning the content of zoom window 500.

[0062] In some embodiments, the user can change the zoom factor within zoom window 500 without resizing the window. Figure 7In the illustrated embodiment, a user may input a pinch gesture by touching the zoom window 500 at two points 700 and changing the distance between the two points 700 to change the zoom scale factor within the zoom window 500. In one embodiment, the size of the zoom window 500 may remain constant as the zoom scale within the zoom window 500 changes. Available imaging data (e.g., from a camera that may be an image source for the rendering environment) may be used to provide the highest possible fidelity for the magnified content in the zoom window 500. For example, if the rendering environment 201 typically has a 4K camera available, although a 1K camera source is available, the content of the zoom window 500 may use the 4K camera source to provide more clarity when the content of the zoom window 500 is magnified.

[0063] like Figure 8 As shown, the distance between the two points 700 has increased, and the zoom scale factor within the zoom window 500 has increased, while the size of the zoom window 500 remains unchanged.

[0064] exist Fig. 9A In the illustrated embodiment, a user can input a touch gesture by, for example, touching zoom window 500 at inner ring 900 and changing the size of zoom window 500. In one embodiment, the user can expand outward or inward after touching ring 900 to adjust the size of the window. Fig. 9B As shown, the user has enlarged the size of the zoom window 500. The zoom window 500 therefore displays more zoom content without changing the zoom ratio. In other embodiments, the user can use other input actions to adjust the size of the window. For example, the zoom window 500 can have an anchor point of the resizing that the user can select and expand to adjust the window size. In one embodiment, when the zoom window 500 is resized, the zoom ratio of the content in the zoom window 500 can remain unchanged. When the content in the zoom window 500 is resized, the available imaging data such as from the camera that can be used as the image source can be used to provide the highest possible fidelity for the content in the zoom window 500. For example, if the rendering environment 201 is usually provided with multiple camera sources at various positions in the rendering environment, the content of the zoom window 500 can use different camera sources to provide the best available fidelity when the size of the zoom window 500 is changed.

[0065] Additional controls may also be provided. For example, Fig.10 As shown, UI 201 may also include a UI editing window 1000 for controlling various aspects of the web conference, such as but not limited to initiating or ending the web conference, sharing content with other participants in the web conference, changing capture devices, and selecting and editing content presented in the web conference. In other embodiments, other UI controls may be provided on the editing window 1000.

[0066] Likewise Fig.10 As shown, the editing window 1000 may also include UI controls for performing other tasks related to the web conference. For example, but not limited to, the UI editing window 1000 may provide the following functions: display notifications, display a list of users and related chat sessions, display available user groups or user teams, display meetings within a day or other time period, and display any recently shared or used files. Other UI controls for performing other types of functions may be provided in other embodiments. Fig.10 In the example shown, UI control window 1000 includes two zoom details within content box 1002, one of which displays Figures 3 to 9B The contents of zoom window 500 in FIG. 5 are indicated by zoom detail 1. In this example, zoom detail 2 depicts an image of participant 230. In one embodiment, the user can annotate the captured image with notes. The user can also send the captured image to one or more recipients.

[0067] like Fig.11 As shown, UI editing window 1000 can be expanded to provide additional UI controls for performing other tasks related to the rendered web conference. Fig.11 As shown, the UI editing window 1000 can be expanded to provide additional zoom details 1101 in addition to the two zoom details of the content box 1002. In this example, the additional zoom details 1101 show greater details of the zoom detail 2 of the content box 1002. In one embodiment, the user can mark the content as a video or a still image. The editing window 1000 can provide access to various filter effects and other menu tools. For example, the editing window 1000 can allow the user to make further changes to the selected image by selecting a file type, sharpening the image, changing the color balance, changing the brightness, and vectorizing the image, to name a few examples. In one embodiment, a timeline 1110 can be provided to the user to move forward and backward from the current time of the captured content rendered in the zoom detail 1101. For example, the image can be associated with a default time window such as 30 seconds. The timeline 1110 for the editing window 1000 can provide options for traversing the time range in which the activity occurred and was recorded. In this way, the user can view various times of the activity from the perspective of the zoom window 500, and various editing options can also be provided to the user within the time range. In some embodiments, the time window can be synchronized with the smart transcript, chat history and record.At any particular timestamp, other activities associated with that timestamp can be provided to the user.

[0068] Reference Fig.12, viewing and editing tools can facilitate detection of the source of content or the object being rendered. In one example, a user can choose to zoom in on the content of detail 1. The source object can be a document that can be identified, searched, accessed, and downloaded by the user. For example, the slide currently rendered on display 200 can be used to identify the source file of the underlying presentation. As shown in source information window 1210, information of the source document is shown. If the user wishes to edit and manipulate the currently rendered image rather than the source document, the user can also be provided with the option of using the content on the screen. The user can further be provided with the option of searching for additional or related content. As shown in source information window 1210, the user can also be provided with the option of searching for additional or related content. Fig.13 As shown, if the user selects a source document (in this example, "group / meeting / presentation.ppt"), the content box 1002 provides other options available for the source document, such as opening the document, sending the document, and saving the document.

[0069] Reference Fig.14 , viewing and editing tools can facilitate remote interaction with the device depicted in UI 201. In one example, a user can select a whiteboard object 210. The device can be identified, and if the user has permission to control the device or provide input to the device, options can be provided to perform operations, such as entering annotations into the whiteboard. In some embodiments, a virtualized whiteboard or whiteboard application can be instantiated and rendered and can be edited by a participant via viewing and editing tools. If the device is a camera, the user can be provided with the ability to change the camera focal length or other parameters of the camera. As shown in the content box 1002 window, source information is shown, which includes the whiteboard device and the content rendered on the whiteboard device. In some embodiments, the user can also be provided with the option to edit the whiteboard content on the screen. As shown in the content box 1002 window, the source information is shown, which includes the whiteboard device and the content rendered on the whiteboard device. In some embodiments, the user can also be provided with the option to edit the whiteboard content on the screen. Fig.14 As shown, the content box 1002 provides additional options, such as controlling the whiteboard and / or editing the image on the whiteboard or editing the whiteboard image independently of the content currently rendered within the whiteboard.

[0070] In some embodiments, editing tool 1000 can facilitate the formation of additional groups and conferences. For example, many participants can form a breakout session to discuss a specific topic and then rejoin the larger group.

[0071] Reference Fig.15 When multiple video sources are available, viewing and editing tools may allow selection of different perspectives of the rendered environment. For example, a user may be able to view and select the camera used for the video feed (e.g., Figure 1 If the user is using a zoom window, the zoom window can be automatically and persistently positioned in the same area when changing perspectives. Fig.15As shown in the example of , the user can select the central camera or one of the two side cameras. In addition, the user can use a companion device such as a camera on his own computing device as an input imaging device.

[0072] Viewing and editing tools may be configured to select and edit the audio portion of a rendered session, respectively.

[0073] refer to Fig.16 and 17 , zoom window 500 is shown as being located near participant 230. As participant 230 moves within the rendering environment, as Fig.17 As shown, in one embodiment, zoom window 500 can move as the participant moves within the rendered space. For example, if zoom window 500 is placed close to a participant who moves during a meeting, zoom window 500 can move. In some embodiments, the perspective of zoom window 500 can remain constant even if the user changes the perspective. For example, in this example, if the user selects a different camera view that results in the environment being rendered from a different angle, zoom window 500 can be positioned to continue to provide a magnified view of participant 230. In some embodiments, even when the perspective of the rendered environment changes, the focus in the environment can be marked and the zoom window can remain in the current position. This allows the user to continuously follow the marked activity.

[0074] The technology disclosed herein can enable a user to interact with and control a 3D representation of a real-world environment based on user gestures. In some embodiments, based on the timing and direction of the input gesture, the computing device can determine the position and orientation of the zoom window 500. For example, a first type of user gesture can include a tap on a button, such as holding a mouse button for less than a threshold time. A second type of user gesture can include a press and hold action, such as if the mouse button is held for more than a threshold time period. Based on the detected gestures, the user can be allowed to perform different actions on the zoom window 500, such as resizing, repositioning, and changing magnification.

[0075] Although the above examples relate to input devices with buttons, such as a mouse, it is understood that the technology disclosed herein can utilize any other suitable input device. For example, the technology disclosed herein can utilize a computing device with a touch screen. In such an implementation, once the user first selects UI control 220 for placement of zoom window 500, the user can track a finger or pen on the touch screen, allowing the computing device to monitor the direction of movement. When the user performs an input action, for example, the user lifts his or her finger or pen from the touch surface or provides a voice command, the computing device can determine the position based on the position of the contact point between the touch screen and the finger or pen. The position of the virtual object can be at the position where the user lifts his or her finger or pen, and the orientation of the object can be based on the direction movement before the input action.

[0076] Fig.18 1 is a diagram illustrating aspects of a routine 1800 for interacting with a rendering environment according to one embodiment disclosed herein. It should be understood by those of ordinary skill in the art that the operations of the methods disclosed herein are not necessarily presented in any particular order, and that it is possible and contemplated to perform some or all of the operations in an alternative order. For ease of description and illustration, the operations have been presented in the order presented. Operations may be added, omitted, performed together, and / or performed simultaneously without departing from the scope of the appended claims.

[0077] It should also be understood that the method shown can be terminated at any time and does not need to be performed as a whole. As defined herein, some or all of the operations of the method and / or substantially equivalent operations can be performed by executing computer-readable instructions included on a computer storage medium. The term "computer-readable instructions" and its variants as used in the specification and claims are widely used herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on various system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, etc.

[0078] Therefore, it should be understood that the logical operations described herein are implemented: (1) as a sequence of computer-implemented actions or program modules running on a computing system such as those described herein) and / or (2) as interconnected machine logic circuits or circuit modules within a computing system. The implementation is a matter of choice depending on the performance and other requirements of the computing system. Therefore, the logical operations may be implemented in software, firmware, dedicated digital logic, and any combination thereof.

[0079] In addition, you can Figures 1 to 17 The example presentation GUI described herein is associated with implementing the Fig.18For example, the various devices and / or modules described herein may generate, send, receive and / or display data associated with the content of a communication session (e.g., real-time content, recorded content, etc.) and / or present a GUI that includes images of one or more participants 30 (e.g., users 230), avatars, channels, chat sessions, video streams, images, virtual objects and / or applications associated with the communication session.

[0080] Reference Fig.18 , operation 1801 shows rendering a representation of an environment on a user interface (UI), the environment indicating an interactive communication session between multiple users. Operation 1801 may be followed by operation 1803. Operation 1803 shows receiving input data indicating a location of a zoom window to be placed within the representation of the environment. Operation 1803 may be followed by operation 1805. Operation 1805, in response to the input data, shows rendering a zoom window on the UI at a location within the representation, the zoom window having a size determined based on one or more criteria and having a plurality of selectable areas available for receiving user input. In one embodiment, the zoom window has a size determined based on one or more criteria. Operation 1805 may be followed by operation 1807. Operation 1807 shows rendering an enlarged view of a portion of the representation proximate to the location of the zoom window within the zoom window. In one embodiment, the zoom window is configured to translate the enlarged view in response to input received via the zoom window indicating a change in the portion of the representation. Operation 1807 may be followed by operation 1809. Operation 1809 shows receiving input data indicating a first gesture applied to the zoom window. In one embodiment, the first gesture indicates a new location of the zoom window in the representation. Operation 1809 may be followed by operation 1811. In response, a first operation 1811 illustrates repositioning the zoom window at a new location on the UI in response to the first gesture. In one embodiment, the size of the zoom window is maintained during the repositioning. Operation 1811 may be followed by operation 1813. Operation 1813 illustrates rendering within the zoom window an enlarged view of a portion of the representation proximate the new location of the zoom window. In one embodiment, the zoom window may be moved to any rendered portion of the representation.

[0081] Fig.19 1 is a diagram illustrating aspects of a routine 1900 for interacting with a rendered environment according to one embodiment disclosed herein. Fig.19, operation 1901 shows rendering a representation of an environment on a user interface (UI) indicating an interactive communication session between multiple users. Operation 1901 may be followed by operation 1903. Operation 1903 shows receiving input data indicating a location of a zoom window to be placed within the representation. Operation 1903 may be followed by operation 1905. Operation 1905 shows rendering a zoom window at a location within the representation on the UI in response to the input data. In one embodiment, the zoom window has a size determined based on one or more criteria. Operation 1905 may be followed by operation 1907. Operation 1907 shows rendering an enlarged view of a portion of the representation near the location of the zoom window within the zoom window. Operation 1907 may be followed by operation 1909. Operation 1909 shows receiving input data indicating a first gesture applied to the zoom window. In one embodiment, the first gesture indicates a size adjustment of the zoom window. Operation 1909 may be followed by operation 1911. Operation 1911 shows, in response to a first gesture, adjusting the size of a zoom window on the UI according to the first gesture, wherein, as the zoom window is resized, the scale of the magnified view within the zoom window is maintained. Operation 1911 may be followed by operation 1913. Operation 1913 shows receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in the scale of the content within the zoom window. Operation 1913 may be followed by operation 1915. Operation 1915 shows, in response to the second gesture, updating the magnified view of a portion of the representation near the position of the zoom window on the UI. In one embodiment, the size of the zoom window is maintained when the magnified view is updated. Operation 1915 may be followed by operation 1917. Operation 1917 shows identifying a source file or document of content rendered within the zoom window. Operation 1917 may be followed by operation 1919. Operation 1919 shows allowing access to the source file or document during the interactive communication session. Operation 1919 may be followed by operation 1921. Operation 1921 shows identifying additional source files or documents for new content rendered in the representation.

[0082] Fig. 20 2 is a diagram illustrating aspects of a routine 2000 for interacting with a rendering environment according to one embodiment disclosed herein. Fig. 20, operation 2001 shows rendering a simultaneous representation of an environment indicating an interactive communication session between multiple users on a user interface (UI). Operation 2001 may be followed by operation 2003. Operation 2003 shows receiving first input data indicating a location for a zoom window placed in the representation. Operation 2003 may be followed by operation 2005. Operation 2005 shows rendering a zoom window at a location within the representation on the UI in response to the first input data. Operation 2005 may be followed by operation 2007. Operation 2007 shows rendering an enlarged view of a portion of the representation near the location of the zoom window within the zoom window. Operation 2007 may be followed by operation 2009. Operation 2009 shows receiving second input data indicating an interaction with the zoom window. Operation 2009 may be followed by operation 2011. Operation 2011 shows presenting an editing pane on the UI in response to the interaction. In one embodiment, the editing pane includes a representation of the content of the zoom window. In addition, the editing pane includes one or more selectable options for applying an action to the content. Operation 2011 may be followed by operation 2013. Operation 2013 illustrates receiving third input data indicating a selection of one of the selectable options. Operation 2013 may be followed by operation 2015. Operation 2015 illustrates performing an editing action on the content in response to the selection. In one embodiment, the editing pane is configured to send data indicating the action. The data may be used to render a shared and simultaneously updated view of the action to the interactive communication session.

[0083] It should be understood that the above subject matter can be implemented as a computer-controlled device, a computer process, a computing system, or as an article such as a computer-readable storage medium. The operations of the example methods are shown in separate boxes and summarized with reference to those boxes. The method is illustrated as a logical flow of boxes, each of which can represent one or more operations that can be implemented in hardware, software, or a combination thereof. In the context of software, these operations represent computer-executable instructions stored on one or more computer-readable media that enable one or more processors to perform the described operations when executed by one or more processors.

[0084] Typically, computer executable instructions include routines, programs, objects, modules, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the operations may be performed in any order, combined in any order, subdivided into multiple sub-operations, and / or performed in parallel to implement the described processes. The described processes may be performed by resources associated with one or more devices (e.g., one or more internal or external CPUs or GPUs) and / or one or more hardware logic (e.g., field programmable gate arrays ("FPGAs"), digital signal processors ("DSPs"), or other types of accelerators).

[0085] All of the methods and processes described above can be embodied and fully automated by software code modules executed by one or more general-purpose computers or processors. The code modules can be stored in any type of computer-readable storage medium or other computer storage device, such as those described below. Some or all of these methods can alternatively be embodied in dedicated computer hardware, such as described below.

[0086] Any conventional description, element or box in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, code segment or part of a code, which includes one or more executable instructions for implementing a specific logical function or element in a routine. Alternative implementations are included within the scope of the examples described herein, where elements or functions may be deleted from or executed out of the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved as understood by those skilled in the art, including substantially simultaneous or in reverse order.

[0087] Fig.21 21 is a diagram illustrating an example environment 2100 in which a system 2102 may operate to populate an HCI disclosed herein with images 108, virtual objects 216, and / or other types of presentation content. In some implementations, an agent implemented by the system may function to collect and / or analyze data associated with the example environment 2100. For example, the agent may function to collect and / or analyze data exchanged between participants involved in a communication session 2104 linked to a GUI disclosed herein.

[0088] As shown, a communication session 2104 (where N is a positive integer having a value of two or greater) may be implemented between a plurality of client computing devices 2106(1)-2106(N) associated with or part of the system 2102. The client computing devices 2106(1)-2106(N) enable users (also referred to as individuals) to participate in the communication session 2104.

[0089] In this example, the communication session 2104 is hosted by the system 2102 on one or more networks 2108. That is, the system 2102 can provide services that enable users of client computing devices 2106(1) to 2106(N) to participate in the communication session 2104 (e.g., through real-time viewing and / or recorded viewing). Thus, "participants" of the communication session 2104 can include users and / or client computing devices (e.g., multiple users can be in a communication room participating in a communication session through the use of a single client computing device), each of which can communicate with other participants. Alternatively, the communication session 2104 can be hosted by one of the client computing devices 2106(1) to 2106(N) using peer-to-peer technology. The system 2102 can also host chat conversations and other team collaboration functions (e.g., as part of an application suite).

[0090] In some implementations, such chat conversations and other team collaboration functions are considered to be external communication sessions distinct from the communication session 2104. A computerized agent that collects participant data in the communication session 2104 can be linked to such an external communication session. Thus, the computerized agent can receive information such as date, time, session details, etc. that enables connection to such an external communication session. In one example, a chat conversation can be conducted in accordance with the communication session 2104. In addition, the system 2102 can host a communication session 2104 that includes at least a plurality of participants co-located at a meeting location, such as a conference room or auditorium, or located at different locations.

[0091] In the examples described herein, client computing devices 2106(1) to 2106(N) participating in a communication session 2104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data may include a collection of various instances or streams of real-time content and / or recorded content. The collection of various instances or streams of real-time content and / or recorded content may be provided by one or more cameras (e.g., video cameras). For example, a single stream of live content or recorded content may include media data associated with a video feed provided by a video camera (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). In some implementations, the video feed may include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images may also include one or more avatars.

[0092] Another example of a separate live or recorded content stream may include media data including avatars of users participating in a communication session and audio data capturing the user's voice. Another example of a single stream of live or recorded content may include media data including files displayed on a display screen and audio data capturing the user's voice. Thus, various streams of live content or recorded content in the communication data enable remote conferencing to be facilitated between a group of people and sharing content among a group of people. In some implementations, various streams of live content or recorded content within the communication data may originate from multiple juxtaposed cameras located in a space such as a room to record or stream a presentation including one or more presenting individuals and one or more individuals consuming the presented content.

[0093] Participants or attendees can view the content of the communication session 2104 in real time as the activity occurs, or alternatively, view it by recording it at a later time after the activity occurs. In the example described herein, the client computing devices 2106 (1) to 2106 (N) participating in the communication session 2104 are configured to receive and render to display communication data on a user interface of a display screen. The communication data may include a collection of various instances or streams of live and / or recorded content. For example, a single content stream may include media data associated with a video feed (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). Another example of a single content stream may include media data including an avatar of a user participating in a conference session and audio data capturing the user's voice. Yet another example of a single content stream may include media data including content items displayed on a display screen and / or audio data capturing the user's voice. Therefore, the various content streams within the communication data enable a conference or broadcast presentation to be facilitated between a group of people dispersed in a remote location.

[0094] A participant or attendee of a communication session is a person within range of a camera or other image and / or audio capture device such that the person's movements and / or sounds generated while the person is viewing and / or listening to content shared via the communication session can be captured (e.g., recorded). For example, a participant may be sitting in a crowd, viewing shared content live from a broadcast location of a live performance. Alternatively, a participant may be sitting in an office conference room, viewing shared content of a communication session with other colleagues via a display screen. Even further, a participant may be sitting or standing in front of a personal device (e.g., a tablet, smartphone, computer, etc.), viewing shared content of a communication session alone in their office or at home.

[0095] System 2102 includes device 2110. Device 2110 and / or other components of system 2102 may include distributed computing resources that communicate with each other and / or with client computing devices 2106(1) to 2106(N) via one or more networks 2108. In some examples, system 2102 may be a stand-alone system responsible for managing various aspects of one or more communication sessions (e.g., communication session 2104). System 2102 may be managed by an entity such as Slack, WEBEX, GOTOMEETING, GOOGLE Hangouts, etc.

[0096] The network 2108 may include, for example, a public network (e.g., the Internet), a private network (e.g., an institutional and / or personal intranet), or some combination of a private network and a public network. The network 2108 may also include any type of wired and / or wireless network, including but not limited to a local area network ("LAN"), a wide area network ("WAN"), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. The network 2108 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as the Internet Protocol ("IP"), the Transmission Control Protocol ("TCP"), the User Datagram Protocol ("UDP"), or other types of protocols. In addition, the network 2108 may also include many devices that contribute to network communication and / or form the basis of network hardware, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, and the like.

[0097] In some examples, network 2108 may further include devices that enable connection to a wireless network, such as a wireless access point ("WAP"). Examples support connectivity to WAPs that send and receive data over various electromagnetic frequencies (e.g., radio frequency frequencies) through the WAP, including support for the Institute of Electrical and Electronics Engineers (IEEE) 802.21 standards (e.g., 802.21g, 802.21h, 802.21ac, etc.) and other standards.

[0098] In various examples, device 2110 may include one or more computing devices that operate in a cluster or other grouping configuration to share resources, balance loads, improve performance, provide failover support or redundancy, or for other purposes. For example, device 2110 may belong to various types of devices, such as traditional server-type devices, desktop computer-type devices, and / or mobile devices. Therefore, although shown as a single type of device or a server-type device, device (one or more) 2110 may include various types of devices, and is not limited to a particular type of device. Device 2110 may represent, but is not limited to, a server computer, a desktop computer, a web server computer, a personal computer, a mobile computer, a laptop computer, a tablet computer, or any other type of computing device.

[0099] The client computing device (e.g., one of client computing devices 2106(1)-2106(N)) can be of a variety of types of devices, which can be the same as or different from device 2110, such as a traditional client-type device, a desktop computer-type device, a mobile-type device, a dedicated-type device, an embedded-type device, and / or a wearable-type device. Thus, the client computing device can include, but is not limited to, a desktop computer, a game console and / or a gaming device, a tablet computer, a personal data assistant ("PDA"), a mobile phone / tablet hybrid device, a laptop computer, a telecommunications device, a computer navigation-type client computing device (e.g., a satellite-based navigation system, including a Global Positioning System ("GPS") device), a wearable device, a virtual reality ("VR") device, an augmented reality ("AR") device, an implanted computing device, an automotive computer, a web-enabled television, a thin client, a terminal, an Internet of Things ("IoT") device, a workstation, a media player, a personal video recorder ("PVR"), a set-top box, a camera, an integrated component (e.g., a peripheral device) for inclusion in a computing device, an appliance, or any other kind of computing device. Furthermore, the client computing device may include a combination of the earlier listed examples of client computing devices, such as a combination of a desktop-type device or a mobile-type device with a wearable device, and the like.

[0100] The various classes and device types of client computing devices 2106(1) to 2106(N) may represent any type of computing device having one or more data processing units 2112 operably connected to a computer-readable medium 2184, for example, via a bus 2116, which in some cases may include a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and one or more of any of a variety of local, peripheral, and / or independent buses.

[0101] Executable instructions stored on the computer-readable medium 2194 may include, for example, an operating system 2119, a client module 2120, a profile module 2122, and other modules, programs, or applications 2192 that may be loaded and executed by the data processing unit.

[0102] The client computing devices 2106(1)-2106(N) may also include one or more interfaces 2124 to enable communication between the client computing devices 2106(1)-2106(N) and other networked devices such as device 2110 via the network device 2110. Such network interfaces 2124 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over a network. In addition, one or more of the client computing devices 2106(1)-2106(N) may include input / output (“I / O”) interfaces 2126 to enable communication with input / output devices (e.g., including peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices (such as microphones), cameras for acquiring and providing video feeds and / or still images, touch input devices, gesture input devices) and / or output devices (including peripheral output devices (e.g., displays, printers, audio speakers, tactile output devices, etc.)). Fig.21 The client computing device 2106(1) is shown connected in some manner to a display device (e.g., display screen 2128(1)) that can display a GUI in accordance with the techniques described herein.

[0103] exist Fig.21 In the exemplary environment 2100 of FIG. 2100 , client computing devices 2106 ( 1 ) to 2106 (N) can use their respective client modules 2120 to connect to each other and / or other external devices to participate in a communication session 2104 or contribute activities to a collaborative environment. For example, a first user can utilize a client computing device 2106 ( 1 ) to communicate with a second user of another client computing device 2106 ( 2 ). When executing the client modules 2120 , the users can share data, which can result in the client computing device 2106 ( 1 ) connecting to the system 2102 and / or other client computing devices 2106 ( 2 ) to 2106 (N) via a network 2108 .

[0104] The client computing devices 2106(1) to 2106(N) may use their respective profile modules 2122 to generate participant profiles ( Fig.212102), and provide the participant profile to other client computing devices and / or devices 2110 of the system 2102. The participant profile may include one or more of an identity of a user or a group of users (e.g., name, unique identifier (“ID”), etc.), user data (e.g., personal data), machine data (such as location (e.g., IP address, room in a building, etc.), and technical capabilities, etc. The participant profile may be used to register participants for a communication session.

[0105] like Fig.21 As shown, the device 2110 of the system 2102 includes a server module 2130 and an output module 2132. In this example, the server module 2130 is configured to receive media streams 2134(1) to 2134(N) from various client computing devices such as client computing devices 2106(1) to 2106(N). As described above, the media streams may include video feeds (e.g., audio and visual data associated with a user), audio data to be output with the presentation of the user's avatar (e.g., an audio-only experience in which the user's video data is not sent), text data (e.g., a text message), file data, and / or screen sharing data (e.g., a document, slide, image, video, etc. displayed on a display screen), etc. Thus, the server module 2130 is configured to receive a collection of various media streams 2134(1) to 2134(N) (the collection is referred to herein as "media data 2134") during the real-time viewing of the communication session 2104. In some cases, not all client computing devices participating in the communication session 2104 provide media streams. For example, the client computing device may be a consuming device or a “listening” device only, such that it only receives content associated with the communication session 2104 but does not provide any content to the communication session 2104 .

[0106] In various examples, the server module 2130 can select various aspects of the media stream 2134 to be shared with various ones of the participating client computing devices 2106(1)-2106(N). Accordingly, the server module 2130 can be configured to generate session data 2136 based on the stream 2134 and / or to communicate the session data 2136 to the output module 2132. The output module 2132 can then transmit the communication data 2138 to the client computing devices (e.g., the client computing devices 2106(1)-2106(3) participating in the real-time viewing of the communication session). The communication data 2138 can include video, audio, and / or other content data provided by the output module 2132 based on the content 2150 associated with the output module 2132 and based on the received session data 2136.

[0107] As shown, output module 2132 sends communication data 2138(1) to client computing device 2106(1), and sends communication data 2138(2) to client computing device 2106(2), and sends communication data 2138(3) to client computing device 2106(3), etc. The communication data 2138 sent to the client computing devices may be the same or different (e.g., the location of the content stream in the user interface may differ from one device to the next).

[0108] In various implementations, the device 2110 and / or the client module 2120 may include a GUI presentation module 2140. The GUI presentation module 2140 may be configured to analyze the communication data 2138 that is communicated to one or more client computing devices 2106. Specifically, the GUI presentation module 2140 at the device 2110 and / or the client computing device 2106 may analyze the communication data 2138 to determine an appropriate manner for displaying the video, image, and / or content on the display screen 2128 of the associated client computing device 2106. In some implementations, the GUI presentation module 2140 may provide the video, image, and / or content to a presentation GUI 2146 that is presented on the display screen 2128 of the associated client computing device 2106. The GUI presentation module 2140 may cause the presentation GUI 2146 to be rendered on the display screen 2128. The presentation GUI 2146 may include the video, image, and / or content analyzed by the GUI presentation module 2140.

[0109] In some implementations, the presentation GUI 2146 may include multiple sections or grids that may render or include video, images, and / or content for display on the display screen 2128. For example, a first section of the presentation GUI 2146 may include a video feed of a presenter or individual, and a second section of the presentation GUI 2146 may include a video feed of an individual consuming the conference information provided by the presenter or individual. The GUI presentation module 2140 may populate the first and second sections of the presentation GUI 2146 in a manner that appropriately mimics an environment experience that the presenter and individual may share.

[0110] In some implementations, the GUI presentation module 2140 can zoom in or provide a zoomed view of the individual represented by the video feed to highlight the individual's reactions to the presenter, such as facial features. In some implementations, the presentation GUI 2146 can include video feeds of multiple participants associated with a meeting (such as a general communication session). In other implementations, the presentation GUI 2146 can be associated with a channel such as a chat channel, an enterprise team channel, etc. Thus, the presentation GUI 2146 can be associated with an external communication session that is different from a general communication session.

[0111] Fig. 22 A diagram is shown showing example components of an example device 2200 configured to populate an HCI disclosed herein, which may include one or more portions or grids that may render or include video, images, virtual objects 116, and / or content for display on a display screen 1228. The device 2200 may represent one of the devices 102 or 104. Additionally, or alternatively, the device 2200 may represent one of the client computing devices 1106.

[0112] As shown, device 2200 includes one or more data processing units 2202, computer readable media 2204, and communication interface 2206. The components of device 2200 may be operatively connected, for example, via a bus, which may include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of a variety of local, peripheral, and / or independent buses.

[0113] As utilized herein, a data processing unit such as data processing unit 2202 and / or data processing unit 1182 may represent, for example, a CPU-type data processing unit, a GPU-type data processing unit, a field programmable gate array (“FPGA”), another type of DSP, or other hardware logic components that may be driven by a CPU in some cases. For example, but not limited to, exemplary types of hardware logic components that may be utilized include application specific integrated circuits (“ASICs”), application specific standard products (“ASSPs”), systems on chips (“SOCs”), complex programmable logic devices (“CPLDs”), and the like.

[0114] As used herein, computer-readable media such as computer-readable media 2204 and computer-readable media 1194 can store instructions that can be executed by a data processing unit. The computer-readable medium can also store instructions that can be executed by an external data processing unit (e.g., by an external CPU, an external GPU) and / or by an external accelerator (e.g., an FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated into the computing device, while in some examples, one or more of the CPU, GPU, and / or accelerator is external to the computing device.

[0115] Computer-readable media, which may also be referred to herein as computer-readable media, may include computer storage media and / or communication media. Computer storage media may include one or more of volatile memory, non-volatile memory and / or other persistent and / or secondary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Thus, computer storage media includes media in tangible and / or physical form included in devices and / or hardware components that are part of a device or external to a device, including, but not limited to, random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), phase change memory (“PCM”), read only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, compact disk read-only memory (“CD-ROM”), digital versatile disk (“DVD”), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid-state storage devices, storage arrays, network attached storage, storage area network, hosted computer storage, or any other storage memory, storage device, and / or storage medium that can be used to store and maintain information for access by a computing device.

[0116] In contrast to computer storage media, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism. As defined herein, computer storage media does not include communication media. That is, computer storage media itself does not include communication media consisting only of a modulated data signal, carrier wave, or propagation signal.

[0117] The communication interface 2206 may represent, for example, a network interface controller ("NIC") or other type of transceiver device to send and receive communications over a network. In addition, the one or more communication interfaces 2206 may include one or more cameras and / or audio devices 2222 to enable the generation of video feeds and / or still images, etc.

[0118] In the example shown, computer-readable medium 2204 includes data store 2208. In some examples, data store 2208 includes a data store, such as a database, a data warehouse, or other type of structured or unstructured data store. In some examples, data store 2208 includes a corpus and / or a relational database having one or more tables, indexes, stored procedures, etc. to enable data access including, for example, one or more of a Hypertext Markup Language (“HTML”) table, a Resource Description Framework (“RDF”) table, a Web Ontology Language (“OWL”) table, and / or an Extensible Markup Language (“XML”) table.

[0119] The data storage area 2208 can store data for the operation of processes, applications, components, and / or modules stored in the computer-readable medium 2204 and / or executed by the data processing unit 2202 and / or accelerator. For example, in some examples, the data storage area 2208 can store session data 2210 (e.g., session data 836), profile data 2222 (e.g., associated with participant profiles), and / or other data. The session data 2210 can include the total number of participants (e.g., users and / or client computing devices) in the communication session, the activities that occurred in the communication session, the list of invitees for the communication session, and / or other data related to when and how the communication session is conducted or hosted. The data storage area 2208 can also include content data 2214, such as content 850 including video, audio, or other content, for rendering and display on one or more display screens 828.

[0120] Alternatively, some or all of the above reference data may be stored on a separate memory 2216 on one or more data processing units 2202, such as a memory on a CPU-type processor, a GPU-type processor, an FPGA-type accelerator, a DSP-type accelerator, and / or other accelerator. In this example, the computer-readable medium 2204 also includes an operating system 2218 and an application programming interface 2210 (API) configured to disclose functions and data of the device 2200 to other devices. In addition, the computer-readable medium 2204 includes one or more modules, such as a server module 2230, an output module 2232, and a GUI rendering module 2240, but the number of modules shown is only an example, and the number may be higher or lower. That is, the functions associated with the modules shown described herein may be performed by a smaller number of modules or a larger number of modules on one device or distributed on multiple devices.

[0121] It should be understood that conditional language used herein, such as "will," "may," "might," or "could," is understood in context, unless otherwise specifically stated, to indicate that certain features, elements, and / or steps are included in certain examples but not in other examples. Thus, such conditional language is generally not intended to imply that one or more examples require certain features, elements, and / or steps in any way, or that one or more examples must include a method for determining, with or without user input or prompting, whether certain features, elements, and / or steps are included or to be performed in any particular example. Unless otherwise expressly stated, combinational language such as the phrase "at least one of X, Y, or Z" should be understood to indicate that an item, term, etc. can be X, Y, or Z, or a combination thereof.

[0122] It should also be appreciated that many variations and modifications may be made to the above examples, and that the elements of these examples and examples should be understood as being in other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the accompanying claims.

[0123] Finally, although various configurations have been described using language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended representations is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter.

[0124] Sample Clauses

[0125] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0126] Example Item A, a system comprising:

[0127] one or more data processing units; and

[0128] A computer readable medium having computer executable instructions encoded thereon to cause the one or more data processing units to perform the following operations:

[0129] rendering a representation of an environment on a user interface (UI) that is indicative of an interactive communication session between a plurality of users;

[0130] receiving input data indicating a location of a zoom window to be placed in a representation of an environment;

[0131] Rendering the zoom window on the UI at a location within the representation in response to the input data, the zoom window having a size determined based on one or more criteria and having a plurality of selectable areas operable to receive user input;

[0132] rendering in the zoom window an enlarged view of a portion of the representation proximate the zoom window, wherein the zoom window is configured to translate the enlarged view in response to input received via the zoom window indicating a change of the portion of the representation;

[0133] receiving, via the selectable region, input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation;

[0134] In response to the first gesture, repositioning the zoom window at a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0135] A magnified view of a portion of the representation proximate to a new position of the zoom window is rendered in the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

[0136] Example clause B, the system of Example clause A, wherein the instructions further cause the one or more data processing units to:

[0137] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window;

[0138] In response to the second gesture, rendering the zoom window at a new size on the UI at a location within the three-dimensional representation of the real environment; and

[0139] An enlarged view of an updated portion of the three-dimensional representation is rendered within the zoom window, the updated portion being proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

[0140] Example clause C, the system of any of example clauses A-B, wherein the second gesture is a resize gesture in which a two finger input is applied to one of the selectable areas via the touch-sensitive surface.

[0141] Example clause D. The system of any one of example clauses A to C, wherein the instructions further cause the one or more data processing units to:

[0142] receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicating a new scale factor for a portion of the three-dimensional representation proximate a location of the zoom window; and

[0143] In response to the second gesture, the magnification within the zoom window is updated based on the new scale factor.

[0144] Example clause E. The system of any one of Example clauses A to D, wherein the second gesture is a pinch gesture applied to the touch-sensitive element within the zoom window.

[0145] Example clause F, the system of any one of example clauses A to E, wherein the instructions further cause the one or more data processing units to modify a boundary of the zoom window to indicate the new scale factor.

[0146] Example clause G, the system of any of Example clauses A to F, wherein the input data indicative of the first gesture is a voice command.

[0147] Example clause H. The system of any one of example clauses A to G, wherein the instructions further cause the one or more data processing units to:

[0148] receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicative of a rotation of a portion of the three-dimensional representation proximate a location of the zoom window; and

[0149] In response to the second gesture, rendered content within the zoom window is updated based on the rotation.

[0150] Example clause I, the system of any of example clauses A to H, wherein the size of the zoom window is based on a distance between a first point and a second point applied to the representation.

[0151] Example clause J, a method for interacting with a rendered environment, the method comprising:

[0152] rendering a representation of an environment on a user interface (UI), the environment indicating a communication session between the plurality of users;

[0153] receiving input data indicating a location of a zoom window to be placed in the representation;

[0154] responsive to the input data, rendering a zoom window on the UI at a location within the representation, the zoom window having a plurality of selectable areas operable to receive user input;

[0155] rendering within the zoom window an enlarged view of a portion of the representation proximate to the location of the zoom window;

[0156] receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation;

[0157] In response to the first gesture, repositioning the zoom window at a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0158] Within the zoom window, an enlarged view of a portion of the representation proximate to a new position of the zoom window is rendered; wherein the zoom window is movable to any rendered portion of the representation.

[0159] Example clause K, the method of example clause J also includes:

[0160] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window;

[0161] In response to the second gesture, rendering a zoom window on the UI at the location in the representation with a new size; and

[0162] An enlarged view of an updated portion of the representation is rendered within the zoom window, the updated portion being proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

[0163] Example Clause L, the method of any of Example Clauses I to K, wherein the second gesture is a resize gesture having two finger inputs applied to the touch-sensitive surface at an edge of the zoom window.

[0164] Example clause M, the method of any of example clauses I to L, further comprising:

[0165] receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicative of a new scale factor for a portion of the representation proximate a location of the zoom window; and

[0166] In response to the second gesture, a magnification within the zoom window is updated based on the new scale factor.

[0167] Example clause N, the method of any of Example clauses I to M, wherein the second gesture is a pinch gesture applied to the touch-sensitive element within the zoom window.

[0168] Example clause O, the method of any one of example clauses I to N, further comprising modifying a boundary of the zoom window to indicate the new scale factor.

[0169] Example clause P, the method of any one of example clauses I to O, further comprising: receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a translation of a portion of the representation proximate to the location of the zoom window; and

[0170] In response to the second gesture, rendered content within the zoom window is updated based on the translation.

[0171] Example Item Q, a system comprising:

[0172] means for rendering a representation of an environment on a user interface (UI), the environment being indicative of a communication session between a plurality of users;

[0173] a unit for receiving input data indicating locations of zoom windows to be placed in the three representations;

[0174] means for rendering the zoom window at a location within the representation on the UI, the zoom window having a size determined based on one or more criteria, the zoom window having a plurality of selectable areas operable to receive user input; and

[0175] means for rendering within the zoom window an enlarged view of a portion of the representation proximate a zoom window position;

[0176] means for receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window in the representation;

[0177] means for repositioning the zoom window at the new location on the UI in response to the first gesture, wherein a size of the zoom window is maintained during the repositioning; and

[0178] Means for rendering, within the zoom window, a magnified view of a portion of the representation proximate the new position of the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

[0179] Example clause R, the system of example clause Q, further comprising:

[0180] means for receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window;

[0181] In response to the second gesture, rendering the cell of the zoom window at the new size on the UI at the location within the three-dimensional representation of the real environment; and

[0182] Means for rendering an enlarged view of an updated portion of the three-dimensional representation within the zoom window, the updated portion being proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

[0183] Example clause S, the system of any of example clauses Q to R, wherein the input data indicative of the first gesture is a voice command.

[0184] Example clause T, the system of any of example clauses Q to S, wherein a size of the zoom window is based on a distance between a first point and a second point applied to the representation.

[0185] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0186] Exemplary clause AA, a system comprising:

[0187] one or more data processing units; and

[0188] A computer-readable medium having computer-executable instructions encoded thereon to cause one or more data processing units to perform the following operations:

[0189] rendering a representation of an environment on a user interface (UI) that is indicative of an interactive communication session between a plurality of users;

[0190] receiving input data indicating a zoom window position to be placed in the representation;

[0191] Rendering a zoom window on the UI at a location within the representation in response to the input data;

[0192] rendering in the zoom window an enlarged view of the portion of the representation proximate the zoom window location;

[0193] receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating a resize of the zoom window;

[0194] In response to the first gesture, resizing a zoom window on the UI according to the first gesture, wherein a proportion of the magnified view within the zoom window is maintained as the zoom window is resized;

[0195] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom ratio of content within the zoom window;

[0196] In response to the second gesture, updating the enlarged view of the portion of the representation near the zoom window position on the UI according to the second gesture, wherein as the enlarged view is updated, the size of the zoom window is maintained;

[0197] Identify the source file or document to obtain the content rendered in the zoom window;

[0198] Allow access to source files or documents during an interactive communications session; and

[0199] Identify additional source files or documents for new content rendered in representations.

[0200] Example clause BB, the system of example clause AA, wherein the instructions further cause the one or more data processing units to:

[0201] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position for the zoom window within the representation;

[0202] In response to the third gesture, repositioning the zoom window at a new location on the UI, wherein a size of the zoom window is maintained during the repositioning;

[0203] Within the zoom window, an updated magnified view of the updated portion of the representation is rendered proximate to the new position of the zoom window.

[0204] Example Clause CC: The system of any of Example Clauses AA through BB, wherein the first gesture is a resize gesture in which a two finger input is applied to the touch-sensitive surface at an edge of the zoom window.

[0205] Example clause DD, the system of any of example clauses AA through CC, wherein the second gesture is a pinch gesture applied to the touch sensitive member within the zoom window.

[0206] Example clause EE, the system of any one of example clauses AA to DD, wherein the instructions further cause the one or more data processing units to modify a border of the zoom window to indicate the changed zoom ratio.

[0207] Example clause FF, the system of any of example clauses AA through EE, wherein the representation is a video feed of the collaborative work environment.

[0208] Example clause GG, the system of any of example clauses AA to FF, wherein the instructions further cause the one or more data processing units to:

[0209] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating scrolling of a portion of the representation proximate a location of the zoom window; and

[0210] In response to the third gesture, the rendered content within the zoom window is updated based on the scrolling.

[0211] Example clause HH, the system of any of example clauses AA through GG, wherein the instructions further cause the one or more data processing units to:

[0212] Automatically reposition the zoom window to stay close to the participant's new position as they move.

[0213] Example clause II, the system of any of example clauses AA to HH, wherein the instructions further cause the one or more data processing units to:

[0214] receiving input data indicative of a change in perspective of the representation; and

[0215] In response to changes in viewing angle, the zoom window is automatically repositioned to maintain view of the portion of the representation.

[0216] Example Clause JJ, a method for interacting with a rendering environment, the method comprising:

[0217] rendering a representation on a user interface (UI) indicating an interactive communication session between the plurality of users;

[0218] receiving input data indicating a location of a zoom window to be placed in the representation;

[0219] In response to the input data, presenting a zoom window on the UI at a location within the representation, the size of the zoom window being determined based on one or more criteria;

[0220] rendering in the zoom window an enlarged view of the portion of the representation proximate the zoom window location;

[0221] receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating a resize of the zoom window;

[0222] In response to the first gesture, resizing the zoom window on the UI according to the first gesture, wherein a proportion of the magnified view within the zoom window is maintained when the zoom window is resized;

[0223] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window;

[0224] In response to the second gesture, updating the enlarged view of the portion of the representation on the UI near the location of the zoom window according to the second gesture, wherein the size of the zoom window is maintained when the enlarged view is updated;

[0225] Identify the source file or document to obtain the content rendered within the zoom window;

[0226] Allow access to source files or documents during an interactive communications session; and

[0227] Identify additional source files or documents for new content rendered in representations.

[0228] The method of Example Clause KK, Example Clause JJ also includes:

[0229] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window in the representation;

[0230] In response to the third gesture, repositioning the zoom window at a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0231] Within the zoom window, an updated magnified view of the updated portion of the representation is rendered, the updated portion being proximate to the new position of the zoom window.

[0232] Example clause LL, the method of any of example clauses II to KK, wherein the first gesture is a resize gesture in which a two finger input is applied to the touch-sensitive surface at an edge of the zoom window.

[0233] Example clause MM, the method of any one of example clauses II to LL, wherein the second gesture is a pinch gesture applied to the touch-sensitive element within the zoom window.

[0234] Example clause NN, a method according to any of example clauses II to MM, wherein the representation is a video feed of the collaborative work environment.

[0235] Example clause OO, the method of any one of example clauses II to NN, further comprising:

[0236] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating scrolling of a portion of the representation proximate to the location of the zoom window; and

[0237] In response to the third gesture, the rendered content within the zoom window is updated based on the scrolling.

[0238] Example Clause PP, A system comprising:

[0239] A unit for presenting a representation of a real-world environment on a user interface (UI);

[0240] means for receiving input data indicating a location of a zoom window to be placed within a representation;

[0241] means for rendering, in response to the input data, a zoom window at a location in the representation on the UI, the zoom window having a size determined based on one or more criteria;

[0242] means for rendering in the zoom window an enlarged view of a portion of the representation proximate to the zoom window position;

[0243] means for receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicative of a resizing of the zoom window;

[0244] means for resizing a zoom window on the UI in response to the first gesture according to the first gesture, wherein a proportion of the magnified view within the zoom window is maintained when the zoom window is resized;

[0245] means for receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window;

[0246] means for, in response to the second gesture, updating a magnified view of a portion of the representation on the UI proximate to the location of the zoom window according to the second gesture, wherein a size of the zoom window is maintained as the magnified view is updated;

[0247] Unit for identifying a source file or document for obtaining content rendered within a zoom window;

[0248] A unit for allowing access to a source file or document during an interactive communication session;

[0249] A unit used to identify additional source files or documents for new content rendered in a representation.

[0250] The system of sample clauses QQ and PP also includes:

[0251] means for receiving input data indicative of a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window in the representation;

[0252] means for repositioning the zoom window at a new location on the UI in response to the third gesture, wherein a size of the zoom window is maintained during the repositioning; and

[0253] Means for rendering within the zoom window an updated, magnified view of an updated portion of the representation, the updated portion being proximate to a new position of the zoom window.

[0254] Example clause RR, the system of any one of example clauses PP to QQ, further comprising means for modifying a border of the zoom window to indicate a changed zoom ratio.

[0255] Example clause SS, the system of any of example clauses PP to RR, wherein the portion of the representation proximate the location of the zoom window includes the participant, further comprising:

[0256] Unit for automatically repositioning a zoom window to remain close to a participant's new position as the participant moves.

[0257] Example clause TT, the system of any of example clauses PP to SS, further comprising:

[0258] means for receiving input data indicative of a change in viewing angle of a representation; and

[0259] Means for automatically repositioning the zoom window to maintain view of the portion of the three-dimensional representation in response to a change in viewing angle.

[0260] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0261] Exemplary Clauses AAA, a system comprising:

[0262] one or more data processing units; and

[0263] A computer-readable medium having computer-executable instructions encoded thereon to cause one or more data processing units to:

[0264] rendering a contemporaneous representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0265] receiving first input data indicating a location of a zoom window to be placed in the representation;

[0266] responsive to the first input data, rendering a zoom window on the UI at a location within the representation;

[0267] rendering within the zoom window an enlarged view of the portion of the representation proximate the location of the zoom window;

[0268] receiving second input data indicative of an interaction with the zoom window;

[0269] In response to the interaction, an editing pane is rendered on the UI, wherein the editing pane includes:

[0270] a representation of the contents of the zoom window; and

[0271] one or more selectable options for actions to be applied to the content; and

[0272] receiving third input data indicating a selection of one of the selectable options;

[0273] In response to the selection, performing an editing action on the content;

[0274] Therein, the editing pane is configured to send data indicative of the action, which can be used to render a shared and contemporaneously updated view of the action to the interactive communication session.

[0275] Example Clause BBB, the system of Example Clause AAA, wherein the selectable options are determined based on the context of the content of the zoom window.

[0276] Example clause CCC, the system of any of example clauses AAA through BBB, wherein the selectable option includes storing the contents of the zoom window.

[0277] Example clause DDD, the system of any of example clauses AAA to CCC, wherein the instructions further cause the one or more data processing units to:

[0278] A source file is identified to obtain the content, wherein the selectable option includes accessing the source file.

[0279] Example clause EEE, the system of any one of example clauses AAA to DDD, wherein the instructions further cause the one or more data processing units to:

[0280] A rendering device is identified to obtain the content, wherein the selectable options include inputting one or more commands to the rendering device to update the rendered content.

[0281] Example clause FFF, the system of any of example clauses AAA to EEE, wherein the instructions further cause the one or more data processing units to:

[0282] identifying available image capture devices in a real-world environment; and

[0283] Based on the content of the zoom window, a current image capture device is changed to provide an image of the three-dimensional representation to improve the image quality of the rendered content.

[0284] Example clause GGG, the system of any of example clauses AAA through FFF, wherein the selectable option includes sending the content or source file to a selected recipient.

[0285] Example clause HHH, the system of any of example clauses AAA through GGG, wherein the selectable option includes sending the content or source file to a selected recipient.

[0286] Example clause III, the system of any of example clauses AAA through HHH, wherein the editing pane includes a timeline navigable to points in time of events captured from the environment.

[0287] Example clause JJJ, any of example clauses AAA through III, wherein the selectable option includes sending an object representing a real-world environment to a selected user, wherein the object can be used to join a current collaboration session in the environment.

[0288] Example Clause KKK, a method for interacting with a rendered environment, the method comprising:

[0289] rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0290] receiving first input data indicating a location of a zoom window to be placed within the representation;

[0291] responsive to the first input data, rendering a zoom window on the UI at a location within the representation;

[0292] rendering within the zoom window a magnified view of a portion of the representation proximate to a location of the zoom window;

[0293] receiving second input data indicating an interaction with the zoom window;

[0294] In response to the interaction, an editing pane is rendered on the UI, wherein the editing pane includes:

[0295] a representation of the contents of the zoom window; and

[0296] One or more selectable options for the action to be applied to the content;

[0297] receiving third input data indicating a selection of one of the selectable options; and

[0298] In response to the selection, performing an editing action on the content;

[0299] Therein, the editing pane is configured to transmit data indicative of the action, which can be used to present a shared and contemporaneously updated view of the action to users of the interactive communication session.

[0300] Example clause LLL, the method of example clause KKK, wherein the selectable option is determined based on the context of the content of the zoom window.

[0301] Example clause MMM, the method of any of example clauses KKK to LLL, wherein the selectable option includes storing the contents of the zoom window.

[0302] The method of any one of Example Clauses NNN, Example Clauses KKK to MMM, further comprising:

[0303] A source file is identified to obtain the content, wherein optionally, the source file is accessed.

[0304] Example clause OOO, the method of any one of example clauses KKK to NNN, further comprising:

[0305] A rendering device is identified to obtain content, wherein optionally one or more commands are input to the rendering device to update the rendered content.

[0306] Sample Provisions PPP, a system comprising:

[0307] A unit for presenting a representation of a real environment on a user interface (UI);

[0308] means for receiving first input data indicating a position of a zoom window to be placed in the three-dimensional representation of the real environment;

[0309] means for rendering a zoom window on the UI at a location within the representation of the real-world environment in response to the first input data;

[0310] means for rendering within the zoom window an enlarged view of a portion of the representation proximate to a position of the zoom window;

[0311] means for receiving second input data indicative of an interaction with the zoom window;

[0312] Means for rendering an editing pane on a UI in response to an interaction, wherein the editing pane comprises:

[0313] a representation of the contents of the zoom window; and

[0314] One or more selectable options for actions to be applied to the content; and

[0315] receiving third input data indicating a selection of one of the selectable options; and

[0316] In response to the selection, performing an editing action on the content; and

[0317] A unit for sending data indicative of an action, which can be used to render a shared and contemporaneously updated view of the action.

[0318] Example Clause QQQ, a system of Example Clause PPP, wherein the content of an editing pane and permissions for allowed actions are determined based on a role of a user of the editing pane.

[0319] A system of any of the example terms RRR, example terms PPP to QQQ, further comprising:

[0320] means for identifying image capture devices available in a real-world environment; and

[0321] Means for changing a current image capture device used to provide images for a three-dimensional representation to improve image quality of rendered content based on content of a zoom window.

[0322] Example Clause SSS, any of Example Clauses PPP to RRR, wherein the selectable option includes sending the content or source file to selected recipients.

[0323] Example Clause TTT, any of Example Clauses PPP to SSS, wherein the content of the editing pane and permissions for allowed actions are determined based on a role of a user of the editing pane.

[0324] Among many other technical benefits, the techniques herein can more efficiently utilize computing resources such as processor cycles, memory, network bandwidth, and power than previous solutions that rely on inefficient manual placement of virtual objects in a 3D environment. Other technical benefits not specifically mentioned herein may also be realized by implementations of the disclosed subject matter.

[0325] Although the technology has been described in language specific to structural features and / or methodological acts, it should be understood that the appended claims are not necessarily limited to the described features or acts. Rather, the features and acts are described as example implementations of the technology.

Claims

1. A system for interacting with a rendering environment, include: one or more data processing units; as well as A computer readable medium having computer executable instructions encoded thereon to cause the one or more data processing units to perform the following operations: rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users; receiving input data indicating a location of a zoom window to be placed within the representation of the environment; Rendering the zoom window at the location within the representation on the UI in response to the input data, the zoom window having a size determined based on one or more criteria and having a plurality of selectable areas operable to receive user input; rendering within the zoom window an enlarged view of a portion of the representation proximate a location of the zoom window, wherein the zoom window is configured to translate the enlarged view in response to input received via the zoom window indicating a change of the portion of the representation; receiving, via the selectable area, input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation; In response to the first gesture, repositioning the zoom window at the new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and An enlarged view of a portion of the representation proximate the new position of the zoom window is rendered within the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

2. The system according to claim 1, in, The instructions further cause the one or more data processing units to: receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window; In response to the second gesture, rendering the zoom window at the new size on the UI at the location within the representation of the environment; as well as An enlarged view of an updated portion of the representation is rendered within the zoom window, the updated portion being proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

3. The system according to claim 2, in, The second gesture is a resize gesture in which a two finger input is applied to one of the selectable areas via the touch-sensitive surface.

4. The system according to claim 1, in, The instructions further cause the one or more data processing units to: receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicating a new scale factor for the portion of the representation proximate the location of the zoom window; as well as In response to the second gesture, the magnification within the zoom window is updated based on the new scale factor.

5. The system according to claim 4, in, The second gesture is a pinch gesture applied to the touch-sensitive element within the zoom window.

6. The system according to claim 4, in, The instructions further cause the one or more data processing units to modify the boundaries of the zoom window to indicate the new scale factor.

7. The system according to claim 1, in, The input data indicative of the first gesture is a voice command.

8. The system according to claim 1, in, The instructions further cause the one or more data processing units to: receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicative of a rotation of the portion of the representation proximate the location of the zoom window; as well as In response to the second gesture, rendered content within the zoom window is updated based on the rotation.

9. The system according to claim 1, in, The size of the zoom window is based on a distance between a first point and a second point applied to the representation.

10. A method for interacting with a rendering environment, the method include: rendering a representation of an environment on a user interface (UI), the environment being indicative of a communication session between a plurality of users; receiving input data indicating a location of a zoom window to be placed within the representation; Rendering the zoom window at the location within the representation on the UI in response to the input data, the zoom window having a plurality of selectable areas operable to receive user input; rendering within the zoom window an enlarged view of a portion of the representation proximate the location of the zoom window; receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation; In response to the first gesture, repositioning the zoom window at the new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and Within the zoom window, an enlarged view of a portion of the representation proximate the new position of the zoom window is rendered; wherein the zoom window can be moved to any rendered portion of the representation.

11. The method according to claim 10, further comprising: include: receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window; In response to the second gesture, rendering the zoom window at the new size on the UI at the location within the representation; as well as An enlarged view of an updated portion of the representation is rendered within the zoom window, the updated portion being proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

12. The method according to claim 10, further comprising: include: receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicating a new scale factor for the portion of the representation proximate the location of the zoom window; as well as In response to the second gesture, the magnification within the zoom window is updated based on the new scale factor.

13. The method of claim 12, further comprising modifying a boundary of the zoom window to indicate the new scale factor.

14. The method according to claim 10, further comprising: include: receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicative of a translation of the portion of the representation proximate the location of the zoom window; as well as In response to the second gesture, rendered content within the zoom window is updated based on the translation.

15. A system for interacting with a rendering environment, include: means for rendering a representation of an environment on a user interface (UI), the environment being indicative of a communication session between a plurality of users; means for receiving input data indicating a location of a zoom window to be placed within the representation; means for rendering the zoom window at the location within the representation on the UI, the zoom window having a size determined based on one or more criteria, the zoom window having a plurality of selectable areas operable to receive user input; means for rendering, within the zoom window, an enlarged view of a portion of the representation proximate the location of the zoom window; means for receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation; means for repositioning the zoom window at the new location on the UI in response to the first gesture, wherein a size of the zoom window is maintained during the repositioning; and Means for rendering, within the zoom window, a magnified view of a portion of the representation proximate the new position of the zoom window; wherein the zoom window is movable to any rendered portion of the representation.