Interactive viewing and editing system

Through an interactive viewing system, real-time zoom-in, reduction and position manipulation of stage canvas in video conferencing, solving the problem that users have difficulty in interacting effectively and improving productivity and computing resources utilization efficiency.

CN113196221BActive Publication Date: 2024-11-05MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980084169.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-12-20
Filing Date
2019-12-12
Publication Date
2024-11-05
Estimated Expiration
2039-12-12

AI Technical Summary

Technical Problem

The lack of effective tools in existing systems in real-time meetings makes it difficult for users to interact quickly and easily with the rendered environment, resulting in loss of productivity and inefficient utilization of computing resources.

Method used

Provide an interactive viewing system that allows users to zoom in or out of details of stage canvas in real time in video conferencing sessions, and manipulate the position and rendering methods of stage canvas, and control the size and position of the zoom window through gestures to achieve efficient rendering environment Interaction.

Benefits of technology

Improves users' interaction efficiency in a collaborative environment, reduces the consumption of computing resources, such as the use of processor cycles, memory and network bandwidth, and enhances control over the rendering environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113196221B_ABST
    Figure CN113196221B_ABST
Patent Text Reader

Abstract

A tool is configured to render a representation of a real-world environment. Input data is received for a zoom window position. The zoom window is rendered at the position, and a size of the zoom window is determined based on one or more criteria. Within the zoom window, a magnified view is rendered proximate to the position of the zoom window. Input data is received for a first gesture, and the first gesture indicates adjusting the size of the zoom window. The size of the zoom window is adjusted, and a scale of the magnified view is maintained when the size of the zoom window is adjusted. Input data is received indicating a second gesture, and the second gesture indicates changing a scale of content within the zoom window. The magnified view is updated, and the size of the zoom window is maintained when the magnified view is updated.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Some computing systems provide collaborative environments that facilitate communication between two or more participants. Systems that provide collaborative environments may allow participants to exchange real-time video, real-time audio, and other forms of data within a communication session. The collaborative environment may employ any suitable communication session format, including but not limited to private chat sessions, multi-user editing sessions, group conferences, broadcasts, and the like.

[0002] Inefficient interaction with a collaborative environment can be detrimental to user productivity and use of computing resources. When software applications fail to optimize user engagement, productivity losses and inefficiencies with respect to computing resources are exacerbated if participants cannot quickly and easily view the collaborative environment and selectively interact with rendered content.

[0003] A real-time conference may involve displaying an environment with multiple participants and multiple information sources. For example, one or more participants may participate in a discussion, a primary presenter may present material on a display, and graphics may be drawn on a whiteboard. Existing systems lack the necessary tools to quickly and easily interact with such conference activities. Existing systems typically require users to perform multiple tasks on the same application or call another application to obtain more details about aspects of interest when rendering a real-time conference.

[0004] For example, when a participant desires more details about the activities of the rendered conference environment, the participant must increase the magnification ratio of the image. However, the entire user screen is typically magnified, and the desired zoomed area may or may not be within the new magnified image. The user must then pan the image to find the area of ​​interest. When this occurs, other areas of the rendered conference may not be within the view, and the user may not be able to view important aspects of the conference while the conference continues. If it is desired to view more details of the area of ​​interest, the user must further zoom the image and search for the area of ​​interest. During this time, more details of the real-time conference may be lost from the view. It is with respect to these and other considerations that the disclosure presented herein is presented. Summary of the invention

[0005] An improved human-computer interface ("HCI") is disclosed herein for interacting with representations of various environments (e.g., three-dimensional ("3D") representations of real-world environments, and in some embodiments, scenes involving video conferencing sessions). In various embodiments, a system for interacting with a communication environment that utilizes video and other content is described. Such a system may be referred to as an interactive viewing system. The interactive viewing system may be provided in conjunction with a video conferencing session. The interactive viewing system may facilitate a shared communication environment that facilitates collaboration and other activities. For example, the interactive viewing system may be presented on each participant's device in a group control state that provides group control of viewing and editing functions and provides the option to grant control to a single user. The rendered environment may include views of various viewing and editing tools that each participant may view as they are using the tools. In some embodiments, the rendered video conferencing session may be referred to as a stage canvas. In one embodiment, the viewing system allows a user to zoom in or out of details in the stage canvas video conferencing experience in real time and manipulate where and how the stage canvas is rendered. The interactive viewing system can interact with, but is not limited to, videos, images, 3D models, office applications, captured environments / objects, annotations, demonstrations, shared locations, notes, expressions, or other shared activities. In some embodiments, a replicated stage canvas layer visible to a user or interactive group can be implemented on the active stage canvas and manipulated as an independent annotation, picture, or other productivity file type. Thus, the system allows any activity to be available outside of a video conference session. Additionally, the system can allow any activity to be recorded and available at different times.

[0006] The interactive viewing system can be configured to allow users to view, manipulate, save, send, and share details of the collaboration and conversation experience in real time or asynchronously to the current group or a new / alternative chat, channel, or collaboration conversation. In some embodiments, a free-floating zoom lens or window can be moved on the stage canvas, allowing the user to zoom in on a portion of the screen experience, rescale the content of the zoom lens or window, change or translate the content rendered within the zoom lens or window, and mark and save the portion for further inspection and manipulation via the viewing system. In some embodiments, voice and gesture commands can be used to control the actions of the viewing and editing system.

[0007] In an embodiment, the size of the zoom window can be adjusted by applying gestures to an optional area such as the outer border of the zoom window. For example, the size of the zoom window can be adjusted by pinching and expanding the outer ring of the zoom window with the user's fingers using a touch-sensitive display. During the resizing, the zoom ratio inside the zoom window can remain unchanged. Therefore, a larger zoom window size may result in more content of the currently zoomed image being displayed within the zoom window. Additionally, the zoom ratio within the zoom window can be changed without changing the size of the zoom window. For example, a pinch gesture entered within the inner lens of the zoom window may result in an increase or decrease in the zoom percentage, while the outer ring size of the zoom window remains unchanged. In addition, the zoom window can be moved to any area of ​​the depicted environment.

[0008] In some embodiments, the zoom window can be configured to provide controls for interacting with the rendered three-dimensional environment. A user can, for example, interact with an image being rendered within the zoom window. The image can be saved and edited as a screen clip or other multimedia object. In some embodiments, the image can be saved and edited at a corresponding magnification level.

[0009] Existing tools for allowing users to manually interact with representations require users to perform multiple menu-driven tasks. Users can spend a lot of time searching available items to find and change settings, calling additional applications to perform functions that are not native to the rendering application, and finding content related to a specific part of the rendered activity. This can result in a large and unnecessary consumption of computing resources.

[0010] The examples described herein are provided within the context of a collaborative environment (e.g., a private chat session, a multi-user editing session, a group conference, a real-time broadcast, etc.). For illustrative purposes, it will be appreciated that a computer managing a collaborative environment involves any type of computer managing a communication session in which two or more computers share data. For illustrative purposes, an "event" is a specific instance of a communication session that may have a start time, an end time, and other parameters for controlling how data is shared and displayed to users participating in the communication session.

[0011] The technology disclosed herein can enable users to efficiently manage rendered views, such as 3D representations of real-world collaborative environments. This can allow for more efficient use of computing resources, such as processor cycles, memory, network bandwidth, and power, compared to previous solutions. Other technical benefits not specifically mentioned herein can also be achieved through implementations of the disclosed subject matter.

[0012] It should be appreciated that various aspects of the subject matter briefly described above and described in further detail below may be implemented as hardware devices, computer-implemented methods, computer-controlled apparatus or devices, computing systems, or articles of manufacture, e.g., computer storage media. Although the subject matter described herein is presented in the general context of program modules executed on one or more computing devices, those skilled in the art will recognize that other implementations may be performed in conjunction with other types of program modules. Typically, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types.

[0013] Those skilled in the art will also recognize that aspects of the subject matter described herein may be practiced on or in conjunction with other computer system configurations than those specifically described herein, including multiprocessor systems, microprocessor-based or programmable consumer electronics, augmented reality or virtual reality devices, video gaming devices, handheld computers, smart phones, smart televisions, self-driving vehicles, smart watches, e-readers, tablet computing devices, dedicated hardware devices, networked devices, and the like.

[0014] Features and technical benefits other than those explicitly described above will be apparent from reading the following detailed description and viewing the associated drawings. This summary is provided to introduce some concepts in a simplified form, which are further described in the following detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a computing system diagram illustrating aspects of an operating environment for embodiments disclosed herein;

[0016] Figure 2 shows an example display according to one embodiment disclosed herein;

[0017] Figure 3 An example user control according to one embodiment disclosed herein is shown;

[0018] Figure 4 shows an example display according to one embodiment disclosed herein;

[0019] Figure 5 shows an illustrative display with a zoom window according to one embodiment disclosed herein;

[0020] Fig. 6A shows an illustrative display with a repositioned zoom window according to one embodiment disclosed herein;

[0021] Figure 6B shows an illustrative display with a repositioned zoom window according to one embodiment disclosed herein;

[0022] Figure 7 shows an illustrative display with a modified zoom window according to one embodiment disclosed herein;

[0023] Figure 8 shows an illustrative display with a modified zoom window according to one embodiment disclosed herein;

[0024] Fig.9A shows an illustrative display with a selected zoom window according to one embodiment disclosed herein;

[0025] Fig. 9B shows an illustrative display with a selected zoom window according to one embodiment disclosed herein;

[0026] Fig.10 shows an illustrative display with an edit window according to one embodiment disclosed herein;

[0027] Fig.11 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0028] Fig.12 shows an illustrative display with an edit window according to one embodiment disclosed herein;

[0029] Fig.13 shows an illustrative display with an editing window according to one embodiment disclosed herein;

[0030] Fig.14 shows an illustrative display with an edit window according to one embodiment disclosed herein;

[0031] Fig.15 shows an illustrative display with a changed view according to one embodiment disclosed herein;

[0032] Fig.16 shows an illustrative display with a zoom window according to one embodiment disclosed herein;

[0033] Fig.17 shows an illustrative display with a zoom window according to one embodiment disclosed herein;

[0034] Fig.18 illustrates aspects of a routine according to one embodiment disclosed herein;

[0035] Fig.19illustrates aspects of a routine according to one embodiment disclosed herein;

[0036] Fig. 20 illustrates aspects of a routine according to one embodiment disclosed herein;

[0037] Fig.21 is a computing system diagram showing aspects of an illustrative operating environment for the techniques disclosed herein;

[0038] Fig. 22 is a computing device diagram illustrating aspects of the configuration and operation of a device that may implement aspects of the disclosed technology according to one embodiment disclosed herein. DETAILED DESCRIPTION

[0039] The following detailed description describes an improved HCI for viewing and editing objects in a representation of an environment (e.g., a 3D representation of a real-world environment). This can result in more efficient use of computing resources, such as processor cycles, memory, network bandwidth, and power, compared to previous solutions that rely on inefficient interaction, selection, and editing of rendered environments and objects within the rendered environments. Technical benefits in addition to those specifically described herein can also be achieved through implementations of the disclosed technology.

[0040] A networked meeting refers to a popular form of electronic collaboration that utilizes an application (e.g., CISCO WEBEX provided by CISCO SYSTEMS, Inc., of San Jose, California; GOTOMEETING provided by CITRIX SYSTEMS, Inc., of Santa Clara, California; ZOOM provided by ZOOM VIDEO COMMUNCATIONS, of San Jose, California; GOOGLE HANGOUTS provided by ALPHABET, Inc., of Mountain View, California; SKYPE, SKYPE FOR BUSINESS, and TEAMS provided by MICROSOFT CORPORATION, of Redmond, Washington) to facilitate communication between two or more participants who are located at separate physical locations. Participants in a communication session in a networked meeting are able to exchange real-time video, audio, and other types of content to view, listen to, and otherwise share information. Participants may also view a common space, such as a whiteboard or a shared application, through which ideas may be exchanged. Viewing of the common space may be supplemented with video and audio conferencing, instant messaging sessions, or any combination thereof, so that a networked meeting may be used as a close substitute for a face-to-face meeting.

[0041] Various types of computing devices may be utilized to participate in networked conferences, including, but not limited to, smartphones, tablet computing devices, set-top boxes, smart televisions, video game systems, and AR, VR, and MR devices.

[0042] While conference participants can view the real-world environment (e.g., the conference space), the ability to interact with the rendered environment has been limited. As a result, remote participants typically must make do with the image and video feeds provided by the rendering application, regardless of the quality and fidelity of the feed. In addition, participants typically must use offline resources to supplement their access to conference materials, e.g., requesting copies of presented documents and files, requesting conference participants to take photos of items of interest, etc. The disclosed HCI addresses the technical considerations set forth above, as well as potential other technical considerations, thereby providing technical benefits to computing systems that implement the disclosed technology.

[0043] In various embodiments, a viewing and editing system that can be used in conjunction with collaborative activities such as networked meetings is disclosed. Such a system may also be referred to as a tool herein, but should not be construed as having different or less functionality than the system. In one embodiment, the viewing and editing tool includes a window or lens rendered on a representation of a real-time meeting. The window or lens can be moved to any part of the representation and can be further resized and / or scaled to zoom in on an area close to the window or lens. In some embodiments, the zoom window / lens can be rotated or translated based on a rotation input gesture or a lateral scroll gesture. Additional functionality can be implemented to enable users to better interact with such features. For example, in some embodiments, the thickness or other properties of the border of the window or lens can be changed to indicate the amount of zoom.

[0044] In some embodiments, the content of a window or lens can be captured, saved, and edited. In some embodiments, options for further actions can be determined based on the context of the window content, the current meeting state and activity, and the user's role. Options for further actions can include, for example, sending the content to a participant or other recipient.

[0045] Viewing and editing tools can include the ability to promote the interaction of users with content and with other participants of the activity of rendering. In the context of real-time video streaming, users can interact with the video stream itself, for example, interact with the frames of the video stream. Additionally, viewing and editing tools can provide the ability to interact with the depicted content (for example, presented documents and files) of the video stream. Viewing and editing tools can also provide the ability to interact with the aspects of the environment depicted in the video stream, for example, the device depicted in the video stream or the device that can provide input to the video stream. The user interaction realized can include updating content, sharing content and interacting with other participants via content.

[0046] In some embodiments, the depicted environment (e.g., a meeting) may be represented as an object that may be sent to a user who may access the meeting to interact with the object. For example, the recipient may be able to click on the object to join the meeting or view details about the meeting.

[0047] In some embodiments, the viewing and editing tool can facilitate the detection of the object or other content being rendered and the source of the object or content. The object can be a document that can be identified, searched, accessed, downloaded and edited by the user. For example, the slides of the currently rendered Powerpoint presentation can be used to identify the source file of the presentation. In another example, if the object being rendered is a device such as an electronic whiteboard, the device can be identified, and the user can be provided with options (if authorized to control or provide input to the device), for example, to enter annotations to the whiteboard. In other embodiments, a virtual whiteboard can be instantiated and rendered, which can be edited by the participant via the viewing and editing tool. If the device is a camera, the user can be provided with the ability to change the focus of the camera or change other parameters of the camera. The viewing and editing tool can continuously identify the original source data of the content that is part of the rendered environment. This allows the user to quickly access the original content, rather than viewing the image of the content and searching for the content as a separate task. Therefore, the viewing and editing tool provides a centralized view of the content, and its context is available to the group during the process of the communication session.

[0048] In some embodiments, when multiple video sources are available, the viewing and editing tool can determine which video source can provide the best fidelity based on the position of the zoom window within the rendered environment. For example, the primary video feed for the primary video feed may not have the highest available resolution. When the user selects the position of the zoom window, a higher resolution image source (if available) can be used to provide a higher fidelity zoom of the selected area.

[0049] In some embodiments, captured images of activities such as meetings can be linked to time windows. For example, images selected for editing can be associated with a default time window of, for example, 30 seconds. The viewing and editing tools can provide a timeline tool to traverse the time range of durations in which an activity occurred and its recording is available. In this way, a user can view various times of an activity from the perspective of a zoom window, and can also provide the user with various editing options over the available time range.

[0050] In some embodiments, actions and features activated by a user during an editing / viewing session may be recorded and may be replayed by the user.

[0051] In some embodiments, the actions available to the viewing and editing tool can be dynamically updated based on activity detected in the room. For example, if a presentation is rendered on a display within a rendered environment, the viewing and editing tool can update the user options to include actions available for accessing and editing the display and / or presentation source.

[0052] In some embodiments, the actions available to the viewing and editing tools may be based on the assigned roles for the users. For example, some users may be assigned a producer role and may be allowed to edit content before sharing it on the network. Other users may have a contributor role or a group role and may be allowed to control their own settings or jointly control settings for the group.

[0053] Turning now to the accompanying drawings (which may be referred to herein as "FIG." or "FIGS."), additional details about the improved HCI disclosed herein will be provided with reference to the accompanying drawings. By way of illustration, these figures show specific configurations or examples. In all figures, the same reference numerals represent the same or similar elements. In the figures, the leftmost digit of a reference numeral identifies the figure in which the reference numeral first appears. References to individual items of a plurality of items may reference each individual item using a reference numeral enclosed in parentheses (and / or a letter without parentheses). General references to items may use a specific reference numeral without a letter sequence. The figures are not drawn to scale.

[0054] Figure 1 An example scenario is shown involving a system 100 associated with a tool for interacting with a rendered environment, such as a collaborative environment. The rendered environment may include a workspace 120, which may be an office, conference room, auditorium, or other space configured to allow individuals to gather and collaborate. Workspace 120 may include cameras 111A and 111B. The environment may include other devices, such as smart beacons 131. Other input sources ( Figure 1111B and 111C and microphone (not shown) may include sensors and other devices. In some embodiments, computing device 121 can obtain information from cameras 111A, 111B and 111C and microphones ( Figure 1 ) and other input devices receive data, and send the collected data as input data 107 to the collaboration server 101.

[0055] The collaboration server 101 may process the input data 107 and send interactive data 106 to one or more user devices, such as the laptop 102 and the VR device 103. The interactive data 106 may include data operable to render an interactive display 150, which may include a representation of the workspace 120. The interactive data 106 may include any images, documents, video data, audio data, or any other information that may be used as data for rendering a representation of the workspace 120 and activities occurring within the workspace 120 as captured by the cameras 111A and 111B. The interactive data 106 may also include other forms of data, such as a meeting request, which may identify the number of attendees, a title associated with each attendee, and other relevant information. The interactive data 106 may also indicate parameters of the event, such as a start time, an end time, and a location. For example, the interactive data 106 may include meeting information indicating a list of attendees, the role of each attendee, the date, time, and location.

[0056] The interactive data 106 may include any information that conveys viewing and editing preferences for parameters or settings related to the collaborative environment. For example, the interactive data 106 may define user interface configurations, volume levels, camera angles, or other parameters that have been utilized by a particular user. The interactive data 106 may also include historical information. For example, the interactive data 106 may include a list of meetings, the attendees of each meeting, and the UI layout used in each meeting.

[0057] Input data 107 may include a description of hardware available to computing device 121. For example, input data 107 may describe aspects of various input devices, sensors, lights, microphones, sound suppression devices, and other hardware available to computing device 121. Input data 107 may also describe the specifications of a display screen or the specifications of a computer that communicates with system 100.

[0058] The input data 107 may also describe the specifications of the available hardware, such as, but not limited to, sensitivity level, zoom level, etc. The input data 107 may also describe the location of each device and the range of each device. For example, the input data 107 may describe the location, position, and viewing area of ​​a particular camera, such as a camera that can capture a speaker at a particular podium, on a stage, etc. In another example, the input data 107 may identify the location of a room microphone and the coordinates defining the range of the microphone. In this example, the input data 107 indicates the availability of two cameras 111 (111A-111B). The interactive display 150 may use a device list to provide editing and viewing options, as further described herein. The input data 107 may also indicate that the first camera 111A is directed toward the first area, and the second camera 111B is directed toward the second area.

[0059] Figure 2 2 is a UI diagram illustrating aspects of an example UI 201 that enables computationally efficient interaction with a 3D representation of a real-world environment 202 according to one embodiment disclosed herein. The UI 201 may correspond to Figure 1 As briefly discussed above, the technology disclosed herein can be utilized in conjunction with an application that provides functionality for holding a networked conference. The UI 201 presented by such an application is Figure 2-17 and described below.

[0060] UI 201 may include, for example, Figure 1 A rendering of the real world environment 202 generated by the computing device 102 or VR device 103. In this way, the user of the computing device 102 or VR device 103 can see a view of the real world environment 202 as well as the whiteboard 210, the display 200, and the participants 230. A control interface 220 that allows interaction with the UI 201 can also be presented.

[0061] More details of the control interface 220 are given in Figure 3 In one embodiment, the control interface 220 may include a circular wheel with selectable options 310. In one example, the selectable options 310 may be configured to cause placement of a zoom window in the UI 201. Although the control interface 220 shown is shown as a circular wheel in this example, the interface may be implemented in various forms, such as a rectangular list of options, a drop-down menu, and other forms.

[0062] The user may interact with UI 201 using mouse input, touch input, or other types of input. Figure 4 In the example shown in FIG. 2 , for example, the user has moved the mouse cursor 221 onto the UI control 220 to initiate the following operation: Figure 5 When the user completes the selection action using the mouse button, the zoom window 500 is rendered on the UI 201 at a specified position, at a previous position, or in a default position. Figure 5 , zoom window 500 is shown proximate to display 200, and zoom window 500 is shown rendering zoomed details of the bottom portion of display 200. In some embodiments, zoom window 500 may include a plurality of selective rings or regions 501, 502, and 503, which may each be associated with a function or action. For example, ring 501 may be associated with a reposition function, ring 502 may be associated with a resize function, and ring 503 may be associated with additional menu options.

[0063] exist Fig. 6A In the embodiment shown in , the user can tap and drag the outer ring of the zoom window 500 (or other rings associated with the function) and move the position of the zoom window 500. In some embodiments, the zoom window 500 may include multiple selective rings or areas 501, 502 and 503, which can each be associated with a function or action. For example, ring 501 can be associated with a reposition function, ring 502 can be associated with a resize function, and ring 503 can be associated with additional menu options. In one embodiment, when the zoom window 500 is repositioned, the zoom ratio of the content within the zoom window 500 can remain unchanged. Fig. 6A , zoom window 500 has been repositioned to the right side of display 200, and the content of zoom window 500 is now rendering the text portion of the current content being rendered on display 200. Available imaging data (e.g., from a camera that can be used as an image source) can be used to provide the highest possible fidelity for the content within zoom window 500 when zoom window 500 is repositioned. For example, if rendered environment 201 is typically provided with multiple camera sources located at various locations within the rendered environment, when the position of zoom window 500 is changed, the content of zoom window 500 can use different camera sources to provide the best available fidelity.

[0064] In some embodiments, the user can pan the content within zoom window 500 without resizing the window or changing the zoom ratio. Figure 6B In the embodiment shown in , the user can input a pan gesture by touching an area within the zoom window 500 and sliding the user's finger in a selected direction. Figure 6B , the user has panned toward the right side of display 200. In some embodiments, the user can rotate the content within zoom window 500 without resizing the window or changing the zoom ratio. Figure 6BIn the embodiment shown in , a user can input a rotation gesture by touching an area within zoom window 500 to rotate the user's finger in a selected rotation direction. Available imaging data (e.g., from a camera that can be used as an image source for the rendered environment) can be used to provide the highest possible fidelity for the panned content within zoom window 500. For example, if rendered environment 201 is typically provided with a 1K camera source, but a 4K camera is available, the content of zoom window 500 can use the 4K camera source to provide more clarity when panning the content of zoom window 500.

[0065] In some embodiments, the user can change the zoom factor within zoom window 500 without resizing the window. Figure 7 , a user may input a pinch gesture by touching the zoom window 500 at two points 700 and changing the distance between the two points 700 to change the zoom scale factor within the zoom window 500. In one embodiment, the size of the zoom window 500 may remain constant when changing the zoom scale within the zoom window 500. Available imaging data (e.g., from a camera that may be used as an image source for the rendered environment) may be used to provide the highest possible fidelity for the magnified content within the zoom window 500. For example, if the rendered environment 201 is typically provided with a 1K camera source, but a 4K camera is available, the content of the zoom window 500 may use the 4K camera source to provide more clarity when magnifying the content of the zoom window 500.

[0066] like Figure 8 As shown in FIG. 5 , the distance between the two points 700 has increased, and the zoom factor within the zoom window 500 has increased, while the size of the zoom window 500 remains unchanged.

[0067] exist Fig.9A In the embodiment shown in FIG. 1 , a user can input a touch gesture by, for example, touching zoom window 500 at inner ring 900 and changing the size of zoom window 500. In one embodiment, the user can expand outward or inward after touching ring 900 to adjust the size of the window. Fig. 9BAs shown in , the user has enlarged the size of the zoom window 500. Therefore, the zoom window 500 shows more zoomed content without changing the zoom ratio. In other embodiments, the user can use other input actions to adjust the size of the window. For example, the zoom window 500 can have a resize anchor point that the user can select and expand to adjust the size of the window. In one embodiment, when adjusting the size of the zoom window 500, the zoom ratio of the content in the zoom window 500 can remain unchanged. Available imaging data (for example, it comes from a camera that can be used as an image source) can be used to provide the highest possible fidelity for the content in the zoom window 500 when adjusting the size of the zoom window 500. For example, if the rendered environment 201 is generally provided with multiple camera sources located at various positions within the rendered environment, when changing the size of the zoom window 500, the content of the zoom window 500 can use different camera sources to provide the best available fidelity.

[0068] Additional controls may also be provided. For example, Fig.10 As shown in FIG. 1 , UI 201 may also include a UI editing window 1000 to control aspects of the networking conference, such as, but not limited to: initiating or ending a networking conference, sharing content with other participants in the networking conference, changing capture devices, and selecting and editing content presented in the networking conference. In other embodiments, other UI controls may be provided on the editing window 1000.

[0069] Also like Fig.10 As shown in , the editing window 1000 may also include UI controls for performing other tasks related to the online meeting. For example, but not limited to, the UI editing window 1000 may provide functionality for displaying notifications, displaying a list of users and associated chat sessions, displaying available user groups or teams, displaying meetings during a day or other time period, and displaying any recently shared or used files. In other embodiments, other UI controls for performing other types of functions may be provided. Fig.10 In the example shown in FIG. 1 , the UI control window 1000 includes two zoom details in the content bin 1002, one of which shows Figures 3 to 9B The contents of the zoom window 500 are shown in FIG. 5 , which is indicated by zoom detail 1. In this example, zoom detail 2 depicts an image of participant 230. In an embodiment, the user can annotate the captured image with notes. The user can also send the captured image to one or more recipients.

[0070] like Fig.11 As shown in FIG. 1 , UI 1000 can be extended to provide additional UI controls for performing other tasks related to the rendered networking conference. For example, Fig.11As shown in , the UI editing window 1000 can be expanded, and in addition to the two zoom details of the content warehouse 1002, the UI editing window 1000 also provides additional zoom details 1101, in this example, the additional zoom details 1101 shows more details of the zoom detail 2 of the content warehouse 1002. In one embodiment, the user can mark the content as a video or a still image. The editing window 1000 can provide access to various filter effects and other menu tools. For example, the editing window 1000 can allow the user to make further changes to the selected image by selecting a file type, sharpening the image, changing the color balance, changing the brightness and vectorizing the image, etc. In an embodiment, a timeline 1110 can be provided to the user to move forward and backward from the current time of the captured content rendered in the zoom detail 1101. For example, an image can be associated with a default time window of, for example, 30 seconds. The timeline 1110 for the editing window 1000 can provide an option to traverse the time range in which the activity occurred and was recorded. In this way, the user can view various times of activity from the perspective of the zoom window 500, and various editing options can also be provided to the user over the time range. In some embodiments, the time window can be synchronized with the smart transcript, chat history, and recording. At any particular timestamp, other activities associated with that timestamp can be provided to the user.

[0071] refer to Fig.12 , viewing and editing tool 1000 can facilitate detection of the source of content or the object being rendered. In one example, the user can choose to zoom in on the content of detail 1. The source object can be a document that can be identified, searched, accessed, and downloaded by the user. For example, the slide currently being rendered on display 200 can be used to identify the source file of the underlying presentation. As shown in source information window 1210, information about the source document is shown. If the user wishes to edit and manipulate the currently rendered image instead of the source document, the user can also be provided with the option of using the content on the screen. The user can also be provided with the option of searching for additional or related content. Fig.13 As shown in , if the user selects a source document (Group / Meetings / presentation.ppt in this example), the content warehouse 1002 provides additional options available for the source document, such as opening the document, sending the document, and saving the document.

[0072] refer to Fig.14, the viewing and editing tool 1000 can facilitate remote interaction with the device depicted in the UI 201. In one example, a user can select a whiteboard object 210. The device can be identified, and the user can be provided with options (if authorized to control or provide input to the device) to perform operations, such as entering annotations to the whiteboard. In some embodiments, a virtual whiteboard or whiteboard application can be instantiated and rendered, which can be edited by a participant via the viewing and editing tool 1000. If the device is a camera, the user can be provided with the ability to change the focus of the camera or other parameters of the camera. As shown in the content warehouse 1002 window, source information is shown, which includes the whiteboard device and the content rendered on the whiteboard device. In some embodiments, the user can also be provided with options to edit the whiteboard content on the screen. Fig.14 As shown in FIG. 1 , content pod 1002 provides additional options, such as controlling a whiteboard and / or editing an image on the whiteboard or editing the whiteboard image independently of the content currently rendered within the whiteboard.

[0073] In some embodiments, editing tool 1000 can facilitate the formation of additional groups and conferences. For example, multiple participants can form a breakout meeting to discuss a specific topic and then rejoin the larger group.

[0074] refer to Fig.15 When multiple video sources are available, the viewing and editing tool 1000 may allow selection of different perspectives of the rendered environment. For example, a user may be able to view and select a camera for a video feed (e.g., Figure 1 If the user is using a zoom window, the zoom window can be automatically and persistently positioned to the same area when the viewing angle changes. Fig.15 As shown in the example of , the user can select the central camera or one of the two side cameras. Additionally, the user can use a companion device such as a camera as an input imaging device on his own computing device.

[0075] The viewing and editing tool 1000 may be configured to select and edit, respectively, the audio portion of the rendered session.

[0076] refer to Fig.16 and Fig.17 , zoom window 500 is shown as being positioned near participant 230. Fig.17As shown in , in one embodiment, zoom window 500 can move as the participant moves within the rendered space. For example, if zoom window 500 is placed close to a participant who moves during the meeting, zoom window 500 can move. In some embodiments, the perspective of zoom window 500 can remain constant even if the user changes the perspective view. For example, in this example, if the user selects a different camera view that results in the environment being rendered from a different angle, zoom window 500 can be positioned so as to continue to provide a magnified view of participant 230. In some embodiments, even when the perspective of the rendered environment changes, the focus in the environment can be marked and the zoom window can remain in the current position. This allows the marked activity to be continuously followed by the user.

[0077] The technology disclosed herein can enable a user to interact with and control a 3D representation of a real-world environment based on user gestures. In some embodiments, based on the timing and direction of the input gestures, the computing device can determine the position and orientation of the zoom window 500. For example, a first type of user gesture may include a short tap on a button, such as holding down a mouse button for less than a threshold time period. A second type of user gesture may include a press and hold action, such as holding down a mouse button for more than a threshold time period. Based on the detected gestures, the user may be allowed to perform different actions on the zoom window 500, such as resizing, repositioning, and changing magnification.

[0078] Although the examples described above relate to input devices (e.g., mice) with buttons, it is recognized that the technology disclosed herein can utilize any other suitable input device. For example, the technology disclosed herein can utilize a computing device with a touch screen. In such an implementation, once the user first selects UI control 220 to place zoom window 500, the user can track a finger or pen on the touch screen, thereby allowing the computing device to monitor the direction of movement. When the user performs an input action (e.g., the user lifts his or her finger or pen from the touch surface or provides a voice command), the computing device can determine the position based on the position of the contact point between the touch screen and the finger or pen. The position of the virtual object can be located at the position of the point where the user lifts his or her finger or pen, and the orientation of the object can be based on the direction of movement before the input action.

[0079] Fig.181 is a diagram illustrating aspects of a routine 1800 for interacting with a rendered environment according to one embodiment disclosed herein. It should be understood by those of ordinary skill in the art that the operations of the methods disclosed herein are not necessarily presented in any particular order, and that it is possible and contemplated to perform some or all of the operations in an alternative order. For ease of description and illustration, the operations have been presented in the order shown. Operations may be added, omitted, performed together, and / or performed simultaneously without departing from the scope of the appended claims.

[0080] It should also be understood that the illustrated method may be terminated at any time and need not be performed in its entirety. As defined herein, some or all of the operations of the method and / or substantially equivalent operations may be performed by executing computer-readable instructions included on a computer storage medium. The term "computer-readable instructions" and variations thereof used in the specification and claims are used broadly herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions may be implemented on a variety of system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, and the like.

[0081] It should be appreciated, therefore, that the logical operations described herein are implemented as: (1) a sequence of computer-implemented actions or program modules running on a computing system such as those described herein; and / or (2) interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice depending on the performance and other requirements of the computing system. Thus, the logical operations may be implemented in software, firmware, dedicated digital logic units, and any combination thereof.

[0082] In addition, it can be combined with the above Figures 1 to 17 The described example presentation GUI is implemented in association with Fig.18 For example, the various devices and / or modules described herein may generate, send, receive, and / or display data associated with the content (e.g., real-time content, recorded content, etc.) of a communication session and / or present a GUI that includes images of one or more participants 30 (e.g., users 230), avatars, channels, chat sessions, video streams, images, virtual objects, and / or applications associated with the communication session.

[0083] refer to Fig.18, operation 1801 shows rendering a representation of an environment on a user interface (UI), the environment indicating an interactive communication session between multiple users. Operation 1801 may be followed by operation 1803. Operation 1803 shows receiving input data indicating a location where a zoom window is to be placed within the representation of the environment. Operation 1803 may be followed by operation 1805. Operation 1805 shows, in response to the input data, rendering a zoom window at a location within the representation on the UI, the size of the zoom window being determined based on one or more criteria, and having a plurality of selectable areas available for receiving user input. In an embodiment, the size of the zoom window is determined based on one or more criteria. Operation 1805 may be followed by operation 1807. Operation 1807 shows rendering an enlarged view of a portion of the representation proximate to the location of the zoom window within the zoom window. In an embodiment, the zoom window is configured to pan the enlarged view in response to input received via the zoom window indicating a change in a portion of the representation. Operation 1807 may be followed by operation 1809. Operation 1809 shows receiving input data indicating a first gesture applied to the zoom window. In an embodiment, the first gesture indicates a new position of the zoom window within the representation. Operation 1809 may be followed by operation 1811. Operation 1811 illustrates repositioning the zoom window to the new position on the UI in response to the first gesture. In an embodiment, the size of the zoom window is maintained during the repositioning. Operation 1811 may be followed by operation 1813. Operation 1813 illustrates rendering an enlarged view of a portion of the representation proximate to the new position of the zoom window within the zoom window. In an embodiment, the zoom window may be moved to any rendered portion of the representation.

[0084] Fig.19 1 is a diagram illustrating aspects of a routine 1900 for interacting with a rendered environment according to one embodiment disclosed herein. Fig.19, operation 1901 shows rendering a representation of an environment on a user interface (UI) indicating an interactive communication session between multiple users. Operation 1901 may be followed by operation 1903. Operation 1903 shows receiving input data indicating a location where a zoom window is to be placed within the representation. Operation 1903 may be followed by operation 1905. Operation 1905 shows rendering a zoom window at a location within the representation on the UI in response to the input data. In an embodiment, the size of the zoom window is determined based on one or more criteria. Operation 1905 may be followed by operation 1907. Operation 1907 shows rendering an enlarged view of a portion of the representation near the location of the zoom window within the zoom window. Operation 1907 may be followed by operation 1909. Operation 1909 shows receiving input data indicating a first gesture applied to the zoom window. In an embodiment, the first gesture indicates adjusting the size of the zoom window. Operation 1909 may be followed by operation 1911. Operation 1911 shows that in response to the first gesture, the size of the zoom window is adjusted on the UI according to the first gesture, wherein the scale of the enlarged view within the zoom window is maintained when the size of the zoom window is adjusted. Operation 1911 may be followed by operation 1913. Operation 1913 shows receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in the scale of the content within the zoom window. Operation 1913 may be followed by operation 1915. Operation 1915 shows that in response to the second gesture, the enlarged view of a portion of the position of the zoom window represented near the second gesture is updated on the UI. In an embodiment, the size of the zoom window is maintained when the enlarged view is updated. Operation 1915 may be followed by operation 1917. Operation 1917 shows identifying a source file or document of the content rendered within the zoom window. Operation 1917 may be followed by operation 1919. Operation 1919 shows allowing access to the source file or document during the interactive communication session. Operation 1919 may be followed by operation 1921. Operation 1921 shows identifying additional source files or documents for new content to be rendered in the representation.

[0085] Fig. 20 2000 is a diagram illustrating aspects of a routine for interacting with a rendered environment according to one embodiment disclosed herein. Fig. 20, operation 2001 shows rendering a contemporaneous representation of an environment on a user interface (UI), the environment indicating an interactive communication session between multiple users. Operation 2001 may be followed by operation 2003. Operation 2003 shows receiving first input data indicating a location where a zoom window is to be placed within the representation. Operation 2003 may be followed by operation 2005. Operation 2005 shows rendering a zoom window at a location within the representation on the UI in response to the first input data. Operation 2005 may be followed by operation 2007. Operation 2007 shows rendering an enlarged view of a portion of the representation near the location of the zoom window within the zoom window. Operation 2007 may be followed by operation 2009. Operation 2009 shows receiving second input data indicating an interaction with the zoom window. Operation 2009 may be followed by operation 2011. Operation 2011 shows rendering an editing pane on the UI in response to the interaction. In an embodiment, the editing pane includes a representation of the content of the zoom window. Additionally, the editing pane includes one or more selectable options for actions to be applied to the content. Operation 2011 may be followed by operation 2013. Operation 2013 illustrates receiving third input data indicating a selection of one of the selectable options. Operation 2013 may be followed by operation 2015. Operation 2015 illustrates performing an editing action on the content in response to the selection. In an embodiment, the editing pane is configured to send data indicating the action. The data may be used to render a shared and contemporaneously updated view of the action to the interactive communication session.

[0086] It should be appreciated that the subject matter described above may be implemented as a computer-controlled device, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. The operations of the example methods are shown in separate boxes and are summarized with reference to those boxes. The methods are shown as a logical flow of boxes, each of which may represent one or more operations that may be implemented in hardware, software, or a combination thereof. In the context of software, these operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, enable the one or more processors to perform the recited operations.

[0087] Typically, computer executable instructions include routines, programs, objects, modules, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be performed in any order, combined in any order, subdivided into multiple sub-operations, and / or performed in parallel to implement the described processes. The described processes may be performed by resources associated with one or more devices (e.g., one or more internal or external CPUs or GPUs) and / or one or more hardware logic units (e.g., field programmable gate arrays ("FPGAs"), digital signal processors ("DSPs"), or other types of accelerators).

[0088] All methods and processes described above can be embodied in a software code module executed by one or more general-purpose computers or processors and fully automated via the software code module. The code module can be stored in any type of computer-readable storage medium or other computer storage device, as described below. Some or all of the methods can alternatively be embodied in dedicated computer hardware, as described below.

[0089] Any conventional description, element or box in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code, which includes one or more executable instructions for implementing a specific logical function or element in the routine. Alternative implementations are included within the scope of the examples described herein, wherein, depending on the functions involved as understood by those skilled in the art, elements or functions may be deleted from those shown or discussed or performed in a different order, including substantially simultaneously or in reverse order.

[0090] Fig.21 21 is a diagram illustrating an example environment 2100 in which a system 2102 may operate to populate an HCI disclosed herein with images 108, virtual objects 216, and / or other types of rendered content. In some implementations, a system-implemented agent may be used to collect and / or analyze data associated with the example environment 2100. For example, the agent may be used to collect and / or analyze data exchanged between participants involved in a communication session 2104 linked to a GUI disclosed herein.

[0091] As shown, a communication session 2104 may be implemented between a plurality of client computing devices 2106(1)-2106(N) (where N is a positive integer having a value greater than 2 or greater) associated with or as part of the system 2102. The client computing devices 2106(1)-2106(N) enable users (also referred to as individuals) to participate in the communication session 2104.

[0092] In this example, the communication session 2104 is hosted by the system 2102 on one or more networks 2108. That is, the system 2102 can provide services that enable users of client computing devices 2106(1) to 2106(N) to participate in the communication session 2104 (e.g., via real-time viewing and / or recorded viewing). Thus, a "participant" of the communication session 2104 can include users and / or client computing devices (e.g., multiple users can participate in a communication session in a communication room via the use of a single client computing device), each of which can communicate with other participants. Alternatively, the communication session 2104 can be hosted by one of the client computing devices 2106(1) to 21069(N) using peer-to-peer technology. The system 2102 can also host chat conversations and other team collaboration functions (e.g., as part of an application suite).

[0093] In some implementations, such chat conversations and other team collaboration functions are considered to be external communication sessions distinct from the communication session 2104. A computerized agent that collects participant data in the communication session 2104 may be able to link to such an external communication session. Thus, the computerized agent may receive information such as date, time, session details, etc. that enables connection to such an external communication session. In one example, a chat conversation may be conducted in accordance with the communication session 2104. Additionally, the system 2102 may host a communication session 2104 that includes at least a plurality of participants co-located at a meeting location (e.g., a conference room or auditorium) or located at different locations.

[0094] In the examples described herein, client computing devices 2106(1) to 2106(N) participating in a communication session 2104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data may include a collection of various instances or streams of real-time content and / or recorded content. The collection of various instances or streams of real-time content and / or recorded content may be provided by one or more cameras (e.g., video cameras). For example, a single stream of real-time content or recorded content may include media data associated with a video feed provided by a video camera (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). In some implementations, the video feed may include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images may also include one or more avatars.

[0095] Another example of a single stream of real-time content or recorded content may include media data including avatars of users participating in a communication session and audio data capturing the users' voices. Yet another example of a single stream of real-time content or recorded content may include media data including files displayed on a display screen and audio data capturing the users' voices. Thus, various streams of real-time content or recorded content within the communication data enable remote conferences to be facilitated between a group of people and sharing of content within a group of people. In some implementations, various streams of real-time content or recorded content within the communication data may originate from multiple co-located cameras positioned in a space such as a room to record or stream in real time a presentation of content including one or more individual presentations and one or more individual consumption of the presentation.

[0096] Participants or attendees can view the content of the communication session 2104 in real time as the activity occurs, or alternatively view it via recording at a later time after the activity occurs. In the example described herein, the client computing devices 2106 (1) to 2106 (N) participating in the communication session 2104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data may include a collection of various instances or streams of real-time content and / or recorded content. For example, a single stream of content may include media data associated with a video feed (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). Another example of a single stream of content may include media data including an avatar of a user participating in the conference session and audio data capturing the user's voice. Yet another example of a single stream of content may include media data including content items displayed on a display screen and audio data capturing the user's voice. Therefore, the various streams of content within the communication data enable a conference or broadcast presentation to be facilitated between a group of people dispersed across remote locations.

[0097] A participant or attendee of a communication session is a person within range of a camera or other image and / or audio capture device, such that the person's movements and / or sounds generated while the person is viewing and / or listening to content shared via the communication session can be captured (e.g., recorded). For example, a participant may be sitting in a crowd, viewing shared real-time content at a broadcast location where a stage presentation is taking place. Or, a participant may be sitting in an office conference room, viewing shared content of a communication session with other colleagues via a display screen. Even further, a participant may be sitting or standing in front of a personal device (e.g., a tablet computer, smart phone, computer, etc.), viewing shared content of a communication session alone in their office or at home.

[0098] System 2102 includes device(s) 2110. Device(s) 2110 and / or other components of system 2102 may include distributed computing resources that communicate with each other and / or with client computing devices 2106(1) to 2106(N) via one or more networks 2108. In some examples, system 2102 may be a stand-alone system responsible for managing aspects of one or more communication sessions (e.g., communication session 2104). As an example, system 2102 may be managed by an entity such as Slack, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0099] The network(s) 2108 may include, for example, a public network (e.g., the Internet), a private network (e.g., an institutional and / or personal intranet), or some combination of private and public networks. The network(s) 2108 may also include any type of wired and / or wireless network, including, but not limited to, a local area network (“LAN”), a wide area network (“WAN”), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. The network(s) 2108 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as the Internet Protocol (“IP”), the Transmission Control Protocol (“TCP”), the User Datagram Protocol (“UDP”), or other types of protocols. In addition, the network(s) 2108 may also include a plurality of devices that facilitate network communications and / or form the hardware foundation of the network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, and the like.

[0100] In some examples, network(s) 2108 may also include devices that enable connection to wireless networks, such as wireless access points (“WAPs”). Examples support connectivity via WAPs that send and receive data over various electromagnetic frequencies (e.g., radio frequencies), including WAPs that support the Institute of Electrical and Electronics Engineers (IEEE) 802.21 standards (e.g., 802.21g, 802.21n, 802.21ac, etc.) and other standards.

[0101] In various examples, (multiple) devices 2110 may include one or more computing devices that operate in a cluster or other grouping configuration to share resources, balance loads, improve performance, provide failover support or redundancy, or for other purposes. For example, (multiple) devices 2110 may belong to various categories of devices, such as traditional server-type devices, desktop-type devices, and / or mobile-type devices. Therefore, although (multiple) devices 2110 are shown as single-type devices or server-type devices, (multiple) devices 2110 may include a variety of device types and are not limited to specific types of devices. (Multiple) devices 2110 may represent, but are not limited to, server computers, desktop computers, web server computers, personal computers, mobile computers, laptop computers, tablet computers, or any other type of computing devices.

[0102] The client computing device (e.g., one of client computing devices 2106(1)-2106(N)) can belong to various categories of devices, which can be the same as or different from device(s) 2110, such as, for example, a traditional server-type device, a desktop-type device, a mobile-type device, a dedicated-type device, an embedded-type device, and / or a wearable-type device. Thus, the client computing device can include, but is not limited to, a desktop computer, a game console and / or a gaming device, a tablet computer, a personal data assistant ("PDA"), a mobile phone / tablet hybrid device, a laptop computer, a telecommunications device, a computer navigation-type client computing device (e.g., a satellite-based navigation system, including a Global Positioning System ("GPS") device), a wearable device, a virtual reality ("VR") device, an augmented reality ("AR") device, an implanted computing device, an automotive computer, a web-enabled television, a thin client, a terminal, an Internet of Things ("IoT") device, a workstation, a media player, a personal video recorder ("PVR"), a set-top box, a camera, an integrated component (e.g., a peripheral device) for inclusion in a computing device, a home appliance, or any other kind of computing device. Furthermore, the client computing device may include a combination of the earlier listed examples of client computing devices, such as a combination of a desktop computer-type device or a mobile-type device with a wearable device, and the like.

[0103] The various classes and device types of client computing devices 2106(1) to 2106(N) may represent any type of computing device having one or more data processing units 2112 operably connected to a computer-readable medium 2184 (e.g., via a bus 2116), which in some instances may include a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and one or more of any of a variety of local, peripheral, and / or independent buses.

[0104] The executable instructions stored on the computer-readable medium 2194 may include, for example, an operating system 2119 , a client module 2120 , a profile module 2122 , and other modules, programs, or applications that may be loaded and executed by the data processing unit(s) 2192 .

[0105] The client computing devices 2106(1)-2106(1)N may also include one or more interfaces 2124 to enable communications between the client computing devices 2106(1)-2106(N) and other networked devices (e.g., device(s) 2110) over the network(s) 2108. Such network interfaces 2124 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. In addition, the client computing devices 2106(1)-2106(N) may include an input / output (“I / O”) interface 2126 that enables communication with input / output devices (e.g., user input devices including peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices such as microphones, cameras for obtaining and providing video feeds and / or still images, touch input devices, gesture input devices, etc.), and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, touch output devices, etc.). Fig.21 The client computing device 2106(1) is shown connected in some manner to a display device (e.g., display screen 2128(1)) that can display a GUI in accordance with the techniques described herein.

[0106] exist Fig.21 In the example environment 2100 of FIG. 1 , client computing devices 2106(1) to 2106(N) can use their respective client modules 2120 to connect to each other and / or to other external devices (multiple) to participate in a communication session 2104 or contribute to an activity for a collaborative environment. For example, a first user can utilize a client computing device 2106(1) to communicate with a second user of another client computing device 2106(2). When executing the client modules 2120, the users can share data, which can result in the client computing device 2106(1) connecting to the system 2102 and / or other client computing devices 2106(2) to 2106(N) via the network(s) 2108.

[0107] The client computing devices 2106(1) to 2106(N) may use their respective profile modules 2122 to generate participant profiles ( Fig.212102) and provide the participant profile to other client computing devices and / or device(s) 2110 of the system 2102. The participant profile may include one or more of the following: an identity of a user or a group of users (e.g., name, unique identifier (“ID”), etc.), user data (e.g., personal data), machine data such as location (e.g., IP address, room in a building, etc.), and technical capabilities, etc. The participant profile may be utilized to register a participant for a communication session.

[0108] like Fig.21 , the device(s) 2110 of the system 2102 includes a server module 2130 and an output module 2132. In this example, the server module 2130 is configured to receive media streams 2134(1) to 2134(N) from various client computing devices (e.g., client computing devices 2106(1) to 2106(N)). As described above, the media streams may include video feeds (e.g., audio and visual data associated with a user), audio data to be output with the presentation of the user's avatar (e.g., an audio-only experience in which the user's audio data is not sent), text data (e.g., a text message), file data, and / or screen sharing data (e.g., a document, slide layout, image, video, etc. displayed on a display screen), etc. Thus, the server module 2130 is configured to receive a collection of various media streams 2134(1) to 2134(N) (the collection is referred to herein as "media data 2134") during the real-time viewing communication session 2104. In some scenarios, not all client computing devices participating in the communication session 2104 provide media streams. For example, a client computing device may be a consuming device or a "listening" device only, such that it only receives content associated with the communication session 2104, but does not provide any content to the communication session 2104.

[0109] In various examples, the server module 2130 can select aspects of the media stream 2134 to be shared with individual devices of the participating client computing devices 2106(1) through 2106(N). Accordingly, the server module 2130 can be configured to generate session data 2136 based on the stream 2134 and / or pass the session data 2136 to the output module 2132. The output module 2132 can then transmit communication data 2138 to the client computing devices (e.g., the client computing devices 2106(1) through 2106(3) participating in the real-time viewing communication session). The communication data 2138 can include video, audio, and / or other content data provided by the output module 2132 based on the content 2150 associated with the output module 2132 and based on the received session data 2136.

[0110] As shown, output module 2132 sends communication data 2138(1) to client computing device 2106(1), and sends communication data 2138(2) to client computing device 2106(2), and sends communication data 2138(3) to client computing device 2106(3), etc. The communication data 2138 sent to the client computing devices may be the same or may be different (e.g., the positioning of the content stream within the user interface may vary between devices).

[0111] In various implementations, the device(s) 2110 and / or the client module 2120 may include a GUI rendering module 2140. The GUI rendering module 2140 may be configured to analyze the communication data 2138 that is delivered to one or more client computing devices 2106. Specifically, the GUI rendering module 2140 at the device(s) 2110 and / or the client computing device 2106 may analyze the communication data 2138 to determine an appropriate manner for displaying the video, image, and / or content on the display screen 2128 of the associated client computing device 2106. In some implementations, the GUI rendering module 2140 may provide the video, image, and / or content to a presentation GUI 2146 that is rendered on the display screen 2128 of the associated client computing device 2106. The GUI rendering module 2140 may cause the presentation GUI 2146 to be rendered on the display screen 2128. The presentation GUI 2146 may include the video, image, and / or content analyzed by the GUI rendering module 2140.

[0112] In some implementations, the presentation GUI 2146 may include multiple sections or grids that may render or include video, images, and / or content for display on the display screen 2128. For example, a first section of the presentation GUI 2146 may include a video feed of a presenter or individual, and a second section of the presentation GUI 2146 may include a video feed of an individual consuming the conference information provided by the presenter or individual. The GUI presentation module 2140 may populate the first section and the second section of the presentation GUI 2146 in a manner that appropriately mimics an environmental experience that the presenter and individual may share.

[0113] In some implementations, the GUI presentation module 2140 can zoom in or provide a scaled view of an individual presented by a video feed to highlight the individual's reactions to the presenter, e.g., facial features. In some implementations, the presentation GUI 2146 can include video feeds of multiple participants associated with a conference (e.g., a general communication session). In other implementations, the presentation GUI 2146 can be associated with a channel (e.g., a chat channel, an enterprise team channel, etc.). Thus, the presentation GUI 2146 can be associated with an external communication session that is different from a general communication session.

[0114] Fig. 22 A diagram illustrating example components of an example device 2200 configured to populate an HCI disclosed herein, which may include one or more portions or grids that may render or include video, images, virtual objects 116, and / or content for display on a display screen 1228. The device 2200 may represent one of the device(s) 102 or 104. Additionally or alternatively, the device 2200 may represent one of the client computing devices 1106.

[0115] As shown, device 2200 includes one or more data processing units 2202, computer readable media 2204, and communication interface(s) 2206. The components of device 2200 are operably connected, for example, via a bus, which may include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of a variety of local, peripheral, and / or independent buses.

[0116] As utilized herein, data processing unit(s) (e.g., data processing unit(s) 2202 and / or data processing unit(s) 1182) may represent, for example, a CPU-type data processing unit, a GPU-type data processing unit, a field programmable gate array (“FPGA”), another class of DSPs, or other hardware logic components (which may be driven by a CPU in some instances). For example, but not limitation, illustrative types of hardware logic components that may be utilized include application specific integrated circuits (“ASICs”), application specific standard products (“ASSPs”), systems on chips (“SOCs”), complex programmable logic devices (“CPLDs”), and the like.

[0117] As used herein, computer-readable media (e.g., computer-readable media 2204 and computer-readable media 1194) can store instructions that can be executed by (multiple) data processing units. The computer-readable medium can also store instructions that can be executed by an external data processing unit (e.g., by an external CPU, an external GPU) and / or can be executed by an external accelerator (e.g., an FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated into the computing device, while in some examples, one or more of the CPU, GPU, and / or accelerator is external to the computing device.

[0118] Computer readable media (media) (which may also be referred to herein as computer readable media (medium)) may include computer storage media and / or communication media. Computer storage media may include one or more of volatile memory, non-volatile memory and / or other persistent and / or secondary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storing information (e.g., computer readable instructions, data structures, program modules or other data). Thus, computer storage media includes media in tangible and / or physical form included in devices and / or hardware components that are part of a device or external to a device, including, but not limited to, random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), phase change memory (“PCM”), read only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, compact disk read-only memory (“CD-ROM”), digital versatile disk (“DVD”), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage devices, magnetic cards or other magnetic storage devices or media, solid-state memory devices, storage arrays, network attached storage devices, storage area networks, hosted computer storage devices, or any other storage memory, storage devices, and / or storage media that can be used to store and maintain information for access by a computing device.

[0119] In contrast to computer storage media, communication media may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism. As defined herein, computer storage media does not include communication media. That is, computer storage media does not include communication media that consists only of a modulated data signal, a carrier wave, or a propagation signal itself.

[0120] The communication interface(s) 2206 may represent, for example, a network interface controller ("NIC") or other type of transceiver device to send and receive communications over a network. In addition, the communication interface(s) 2206 may include one or more cameras and / or audio devices 2222 to enable the generation of video feeds and / or still images, etc.

[0121] In the example shown, computer-readable medium 2204 includes data repository 2208. In some examples, data repository 2208 includes a data storage device, such as a database, a data warehouse, or other type of structured or unstructured data storage device. In some examples, data repository 2208 includes a corpus and / or a relational database with one or more tables, indexes, stored procedures, etc. to enable data access, including, for example, one or more of the following: Hypertext Markup Language ("HTML") tables, Resource Description Framework ("RDF") tables, Web Ontology Language ("OWL") tables, and / or Extensible Markup Language ("XML") tables.

[0122] The data repository 2208 may store data for the operation of processes, applications, components, and / or modules stored in the computer-readable medium 2204 and / or executed by the (multiple) data processing units 2202 and / or (multiple) accelerators. For example, in some examples, the data repository 2208 may store session data 2210 (e.g., session data 836), profile data 2212 (e.g., which is associated with a participant profile), and / or other data. The session data 2210 may include the total number of participants (e.g., users and / or client computing devices) in the communication session, the activities that occurred in the communication session, the list of invitees for the communication session, and / or other data related to when and how the communication session is conducted or hosted. The data repository 2208 may also include content data 2214, for example, content 850 including video, audio, or other content for rendering and displaying on one or more of the display screens 828.

[0123] Alternatively, some or all of the data cited above may be stored on a separate memory 2216 on board of one or more data processing units 2202, for example, on a CPU-type processor, a GPU-type processor, an accelerator of an FPGA type, an accelerator of a DSP type, and / or other accelerator. In this example, the computer-readable medium 2204 also includes an operating system 2218 and (multiple) application programming interfaces 2220 (APIs) configured to expose the functions and data of the device 2200 to other devices. Additionally, the computer-readable medium 2204 includes one or more modules (e.g., a server module 2230, an output module 2232, and a GUI rendering module 2240), but the number of illustrated modules is only an example, and the number may become higher or lower. That is, the functions described in association with the illustrated modules herein may be performed by a smaller number of modules or a larger number of modules on one device or across multiple devices.

[0124] It should be appreciated that, unless specifically stated otherwise, conditional language used herein (e.g., "can," "could," "might," or "may," etc.) is understood within the context to express that certain examples include and other examples do not include certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that one or more examples in any way require certain features, elements, and / or steps, or that one or more examples must include logic for determining (with or without user input or prompting) whether certain features, elements, and / or steps are included or to be performed in any particular example. Unless specifically stated otherwise, conjunction language such as the phrase "at least one of X, Y, or Z" should be understood to mean that an item, term, etc. may be X, Y, or Z, or a combination thereof.

[0125] It should also be recognized that various changes and modifications may be made to the examples described above, and the elements therein should be understood to be among other acceptable examples. All such changes and modifications are intended to be included herein within the scope of the present disclosure and protected by the accompanying claims.

[0126] Finally, although various configurations have been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended representations is not necessarily limited to the specific features or acts described. Rather, the specific features and acts disclosed serve as example forms of implementing the claimed subject matter.

[0127] Sample Clauses

[0128] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0129] Example Item A, a system comprising:

[0130] one or more data processing units; and

[0131] A computer-readable medium having encoded thereon computer-executable instructions for causing one or more data processing units to:

[0132] rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0133] receiving input data indicating a location within the representation of the environment at which a zoom window is to be placed;

[0134] Rendering, in response to the input data, a zoom window on the UI at a location within the representation, the zoom window having a size determined based on one or more criteria and having a plurality of selectable areas operable to receive user input;

[0135] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window, wherein the zoom window is configured to translate the magnified view in response to input received via the zoom window indicating a change to the portion of the representation;

[0136] receiving, via the selectable region, input data indicating a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation;

[0137] In response to the first gesture, repositioning the zoom window to a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0138] An enlarged view of a portion of the representation proximate the new position of the zoom window is rendered within the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

[0139] Example clause B, the system of example clause A, wherein the instructions further cause the one or more data processing units to perform the following operations:

[0140] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window;

[0141] In response to the second gesture, rendering a zoom window on the UI at a new size at a location within the three-dimensional representation of the real-world environment; and

[0142] An enlarged view of an updated portion of the three-dimensional representation proximate the location of the zoom window is rendered within the zoom window, wherein the updated portion is determined based on the new size.

[0143] Example clause C, the system of any of Example clauses A-B, wherein the second gesture is a resize gesture inputted with two fingers applied to one of the selectable areas via the touch-sensitive surface.

[0144] Example clause D. The system of any one of example clauses A to C, wherein the instructions further cause the one or more data processing units to perform the following operations:

[0145] Receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new zoom factor for a portion of the three-dimensional representation proximate to the location of the zoom window; and in response to the second gesture, updating a magnification within the zoom window based on the new zoom factor.

[0146] Example clause E, the system of any of Example clauses A to D, wherein the second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

[0147] Example clause F, the system of any of example clauses A to E, wherein the instructions further cause the one or more data processing units to modify the boundaries of the zoom window to indicate the new zoom factor.

[0148] Example clause G, the system of any of Example clauses A-F, wherein the input data indicative of the first gesture is a voice command.

[0149] Example clause H. The system of any one of example clauses A to G, wherein the instructions further cause the one or more data processing units to:

[0150] receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicative of rotating a portion of the three-dimensional representation proximate to the zoom window; and

[0151] In response to the second gesture, the rendered content within the zoom window is updated based on the rotation.

[0152] Example clause I, the system of any of example clauses A to H, wherein the size of the zoom window is based on a distance between a first point applied to the representation and a second point.

[0153] Example clause J, a method for interacting with a rendered environment, the method comprising:

[0154] rendering a representation of an environment on a user interface (UI), the environment representing a communication session between a plurality of users;

[0155] receiving input data indicating a location within the representation where a zoom window is to be placed;

[0156] Rendering, in response to the input data, a zoom window on the UI at a location within the representation, the zoom window having a plurality of selectable areas operable to receive user input;

[0157] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window;

[0158] receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation;

[0159] In response to the first gesture, repositioning the zoom window to a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0160] An enlarged view of a portion of the representation proximate the new position of the zoom window is rendered within the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

[0161] Example clause K, the method of example clause J, further comprising:

[0162] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new size of the zoom window;

[0163] In response to the second gesture, rendering a zoom window on the UI at the location within the representation at the new size; and

[0164] An enlarged view of an updated portion of the representation proximate the location of the zoom window is rendered within the zoom window, wherein the updated portion is determined based on the new size.

[0165] Example clause L, the method of any of Example clauses I to K, wherein the second gesture is a resize gesture input with two fingers applied to the touch-sensitive surface at an edge of the zoom window.

[0166] Example clause M, the method of any of example clauses I to L, further comprising:

[0167] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a new zoom factor for a portion of the representation proximate to the location of the zoom window; and

[0168] In response to the second gesture, the magnification within the zoom window is updated based on the new zoom factor.

[0169] Example clause N, the method of any of example clauses I to M, wherein the second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

[0170] Example clause O, the method of any one of example clauses I to N, further comprising modifying the boundaries of the zoom window to indicate the new zoom factor.

[0171] Example clause P, the method of any one of example clauses I to O, further comprising: receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a portion of the position of the pan representation proximate to the zoom window; and

[0172] In response to the second gesture, the rendered content within the zoom window is updated based on the panning.

[0173] Example Item Q, a system comprising:

[0174] means for rendering a representation of an environment on a user interface (UI), the environment being indicative of a communication session between a plurality of users;

[0175] means for receiving input data indicating a location within the three-dimensional representation where a zoom window is to be placed;

[0176] means for rendering a zoom window on the UI at a location within the representation, the zoom window having a size determined based on one or more criteria, the zoom window having a plurality of selectable areas operable to receive user input;

[0177] means for rendering within the zoom window an enlarged view of a portion of the representation proximate a location of the zoom window;

[0178] means for receiving input data indicative of a first gesture applied to the zoom window, the first gesture indicating a new position of the zoom window within the representation;

[0179] means for repositioning the zoom window to a new location on the UI in response to the first gesture, wherein a size of the zoom window is maintained during the repositioning; and

[0180] Means for rendering within a zoom window an enlarged view of a portion of a representation proximate a new position of the zoom window; wherein the zoom window is movable to any rendered portion of the representation.

[0181] Example clause R, the system of example clause Q, further comprising:

[0182] means for receiving input data indicative of a second gesture applied to the zoom window, the second gesture indicating a new size for the zoom window;

[0183] means for rendering, on the UI, a zoom window at a new size at a location within the three-dimensional representation of the real-world environment in response to the second gesture; and

[0184] Means for rendering, within the zoom window, an enlarged view of an updated portion of the three-dimensional representation proximate to the location of the zoom window, wherein the updated portion is determined based on the new size.

[0185] Example clause S, the system of any of example clauses Q to R, wherein the input data indicative of the first gesture is a voice command.

[0186] Example clause T, the system of any of example clauses Q to S, wherein the size of the zoom window is based on a distance between the first point applied to the representation and the second point.

[0187] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0188] Example Clause AA, A system comprising:

[0189] one or more data processing units; and

[0190] A computer-readable medium having encoded thereon computer-executable instructions for causing one or more data processing units to:

[0191] rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0192] receiving input data indicating a location within the representation where a zoom window is to be placed;

[0193] Rendering a zoom window on the UI at a location within the representation in response to the input data;

[0194] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window;

[0195] receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating resizing the zoom window;

[0196] In response to the first gesture, adjusting the size of the zoom window according to the first gesture on the UI, wherein a proportion of the enlarged view within the zoom window is maintained when the size of the zoom window is adjusted;

[0197] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window;

[0198] In response to the second gesture, updating, on the UI, a magnified view of a portion of the representation proximate to the location of the zoom window according to the second gesture, wherein a size of the zoom window is maintained when updating the magnified view;

[0199] Identifying a source file or document of content rendered within the zoom window;

[0200] Allow access to source files or documents during an interactive communications session; and

[0201] Identifies additional source files or documents for new content rendered in a representation.

[0202] Example clause BB, the system of example clause AA, wherein the instructions further cause the one or more data processing units to perform the following operations:

[0203] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window within the representation;

[0204] In response to the third gesture, repositioning the zoom window to a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0205] An updated magnified view of the updated portion of the representation proximate the new position of the zoom window is rendered within the zoom window.

[0206] Example clause CC, the system of any of example clauses AA through BB, wherein the first gesture is a resize gesture input with two fingers applied to the touch-sensitive surface at an edge of the zoom window.

[0207] Example clause DD, the system of any of example clauses AA through CC, wherein the second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

[0208] Example clause EE, the system of any one of example clauses AA to DD, wherein the instructions further cause the one or more data processing units to modify a border of the zoom window to indicate the changed zoom ratio.

[0209] Example clause FF, the system of any of example clauses AA through EE, wherein the representation is a video feed of the collaborative work environment.

[0210] Example clause GG, the system of any of example clauses AA to FF, wherein the instructions further cause the one or more data processing units to:

[0211] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a portion of the scroll representation proximate to the location of the zoom window; and

[0212] In response to the third gesture, the rendered content within the zoom window is updated based on the scrolling.

[0213] Example clause HH, the system of any of example clauses AA to GG, wherein the instructions further cause the one or more data processing units to:

[0214] As the participant moves, the zoom window is automatically repositioned to stay close to the participant's new position.

[0215] Example clause II, the system of any one of example clauses AA to HH, wherein the instructions further cause the one or more data processing units to:

[0216] receiving input data indicating a change in perspective of a representation; and

[0217] In response to a change in viewing angle, the zoom window is automatically repositioned to maintain a view of a portion of the representation.

[0218] Example Clause JJ, A method for interacting with a rendered environment, the method comprising:

[0219] rendering a representation on a user interface (UI) indicating an interactive communication session between a plurality of users;

[0220] receiving input data indicating a location within the representation where a zoom window is to be placed;

[0221] Rendering a zoom window on the UI at a location within the representation in response to the input data, the zoom window having a size determined based on the one or more criteria;

[0222] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window;

[0223] receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating resizing the zoom window;

[0224] In response to the first gesture, adjusting the size of the zoom window according to the first gesture on the UI, wherein a proportion of the enlarged view within the zoom window is maintained when the size of the zoom window is adjusted;

[0225] receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window;

[0226] In response to the second gesture, updating, on the UI, a magnified view of a portion of the representation proximate to the location of the zoom window according to the second gesture, wherein a size of the zoom window is maintained when updating the magnified view;

[0227] Identifying a source file or document of content rendered within the zoom window;

[0228] Allow access to source files or documents during an interactive communications session; and

[0229] Identifies additional source files or documents for new content rendered in a representation.

[0230] Example Clause KK, the method of Example Clause JJ, further comprising:

[0231] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window within the representation;

[0232] In response to the third gesture, repositioning the zoom window to a new location on the UI, wherein a size of the zoom window is maintained during the repositioning; and

[0233] An updated magnified view of the updated portion of the representation proximate the new position of the zoom window is rendered within the zoom window.

[0234] Example clause LL, the method of any of example clauses II to KK, wherein the first gesture is a resize gesture input with two fingers applied to the touch-sensitive surface at an edge of the zoom window.

[0235] Example clause MM, the method of any of example clauses II to LL, wherein the second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

[0236] Example clause NN, the method of any of example clauses II to MM, wherein the representation is a video feed of the collaborative work environment.

[0237] Example clause OO, the method of any one of example clauses II to NN, further comprising:

[0238] receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a portion of the scroll representation proximate to the location of the zoom window; and

[0239] In response to the third gesture, the rendered content within the zoom window is updated based on the scrolling.

[0240] Example Clause PP, A system comprising:

[0241] a unit for rendering a representation of a real-world environment on a user interface (UI);

[0242] means for receiving input data indicating a location within the representation where a zoom window is to be placed;

[0243] means for rendering, on the UI, a zoom window at a location within the representation in response to the input data, the size of the zoom window being determined based on one or more criteria;

[0244] means for rendering within the zoom window an enlarged view of a portion of the representation proximate a location of the zoom window;

[0245] means for receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating resizing the zoom window;

[0246] means for adjusting, on the UI, a size of the zoom window according to the first gesture in response to the first gesture, wherein a proportion of the magnified view within the zoom window is maintained when the zoom window is adjusted in size;

[0247] means for receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window;

[0248] means for updating, in response to the second gesture, on the UI, a magnified view of a portion of the representation proximate to the location of the zoom window according to the second gesture, wherein a size of the zoom window is maintained while updating the magnified view;

[0249] A unit for identifying a source file or document of content rendered within a zoom window;

[0250] means for allowing access to a source file or document during an interactive communication session; and

[0251] A unit used to identify additional source files or documents for new content rendered in a representation.

[0252] The system of sample clauses QQ and PP also includes:

[0253] means for receiving input data indicative of a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window within the representation;

[0254] means for repositioning the zoom window to a new location on the UI in response to the third gesture, wherein a size of the zoom window is maintained during the repositioning; and

[0255] Means for rendering within the zoom window an updated magnified view of an updated portion of the representation proximate a new position of the zoom window.

[0256] Example clause RR, the system of any one of example clauses PP to QQ, further comprising means for modifying a border of the zoom window to indicate a changed zoom ratio.

[0257] Example clause SS, the system of any of example clauses PP to RR, wherein the portion of the representation proximate to the location of the zoom window includes the participant, the system further comprising:

[0258] A unit for automatically repositioning a zoom window as a participant moves to remain close to the participant's new position.

[0259] Example clause TT, the system of any of example clauses PP to SS, further comprising:

[0260] means for receiving input data indicating a change in perspective of a representation; and

[0261] Means for automatically repositioning a zoom window to maintain a view of a portion of a three-dimensional representation in response to a change in viewing angle.

[0262] The disclosure presented herein covers the subject matter set forth in the following example clauses.

[0263] Sample Clause AAA, a system comprising:

[0264] one or more data processing units; and

[0265] A computer-readable medium having encoded thereon computer-executable instructions for causing one or more data processing units to:

[0266] rendering a contemporaneous representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0267] receiving first input data indicating a location within the representation where a zoom window is to be placed;

[0268] responsive to the first input data, rendering a zoom window on the UI at a location within the representation;

[0269] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window;

[0270] receiving second input data indicative of an interaction with the zoom window;

[0271] In response to the interaction, rendering an editing pane on the UI, wherein the editing pane includes:

[0272] a representation of the contents of the zoom window; and

[0273] one or more selectable options for actions to be applied to the content; and

[0274] receiving third input data indicating a selection of one of the selectable options; and

[0275] In response to the selection, performing an editing action on the content;

[0276] Therein, the editing pane is configured to send data indicative of the action, which can be used to render a shared and contemporaneously updated view of the action to the interactive communication session.

[0277] Example clause BBB, the system of example clause AAA, wherein the selectable options are determined based on the context of the content of the zoom window.

[0278] Example clause CCC, the system of any of example clauses AAA through BBB, wherein the selectable option includes storing the contents of the zoom window.

[0279] Example clause DDD, the system of any one of example clauses AAA to CCC, wherein the instructions further cause the one or more data processing units to:

[0280] Identify the source file of the content, where optional options include accessing the source file.

[0281] Example clause EEE, the system of any one of example clauses AAA to DDD, wherein the instructions further cause the one or more data processing units to perform the following operations:

[0282] A rendering device is identified for the content, wherein the selectable options include inputting one or more commands to the rendering device to update the rendered content.

[0283] Example clause FFF, the system of any of example clauses AAA to EEE, wherein the instructions further cause the one or more data processing units to:

[0284] identifying available image capture devices for the real-world environment; and

[0285] Based on the content of the zoom window, a current image capture device used to provide an image of the three-dimensional representation is changed to improve the image quality of the rendered content.

[0286] Example clause GGG, the system of any of example clauses AAA through FFF, wherein the selectable option includes sending the content or source file to a selected recipient.

[0287] Example clause HHH, the system of any of example clauses AAA through GGG, wherein the selectable option includes sending the content or source file to a selected recipient.

[0288] Example clause III, the system of any of example clauses AAA through HHH, wherein the editing pane includes a timeline navigable to points in time of captured events of the environment.

[0289] Example clause JJJ, the system of any of example clauses AAA through III, wherein the selectable option includes sending an object representing the real-world environment to a selected user, wherein the object can be used to join a current collaboration session in the environment.

[0290] Example Clause KKK, a method for interacting with a rendered environment, the method comprising:

[0291] rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users;

[0292] receiving first input data indicating a location within the representation where a zoom window is to be placed;

[0293] responsive to the first input data, rendering a zoom window on the UI at a location within the representation;

[0294] rendering within the zoom window a magnified view of a portion of the representation proximate a location of the zoom window;

[0295] receiving second input data indicative of an interaction with the zoom window;

[0296] In response to the interaction, rendering an editing pane on the UI, wherein the editing pane includes:

[0297] a representation of the contents of the zoom window; and

[0298] one or more selectable options for actions to be applied to the content;

[0299] receiving third input data indicating a selection of one of the selectable options; and

[0300] In response to the selection, performing an editing action on the content;

[0301] Therein, the editing pane is configured to send data indicative of the action, which can be used to render a shared and contemporaneously updated view of the action to users of the interactive communication session.

[0302] Example clause LLL, the method of example clause KKK, wherein the selectable options are determined based on the context of the content of the zoom window.

[0303] Example clause MMM, the method of any one of example clauses KKK to LLL, wherein the selectable option includes storing the contents of the zoom window.

[0304] Example clause NNN, the method of any of example clauses KKK to MMM, further comprising:

[0305] Identify the source file of the content, where optional options include accessing the source file.

[0306] Example clause OOO, the method of any of example clauses KKK through NNN, further comprising:

[0307] A rendering device is identified for the content, wherein the selectable options include inputting one or more commands to the rendering device to update the rendered content.

[0308] Sample Provisions PPP, a system comprising:

[0309] a unit for rendering a representation of a real-world environment on a user interface (UI);

[0310] means for receiving first input data indicating a location within the three-dimensional representation of the real-world environment where a zoom window is to be placed;

[0311] means for rendering, on the UI, a zoom window at a location within the three-dimensional representation of the real-world environment in response to the first input data;

[0312] means for rendering within the zoom window an enlarged view of a portion of the representation proximate a location of the zoom window;

[0313] means for receiving second input data indicative of an interaction with the zoom window;

[0314] Means for rendering an editing pane on a UI in response to an interaction, wherein the editing pane comprises:

[0315] a representation of the contents of the zoom window; and

[0316] one or more selectable options for actions to be applied to the content; and

[0317] receiving third input data indicating a selection of one of the selectable options; and

[0318] In response to the selection, performing an editing action on the content; and

[0319] A unit for sending data indicative of an action that can be used to render a shared and contemporaneously updated view of the action.

[0320] Example Clause QQQ, a system of Example Clause PPP, wherein the content of an editing pane and permissions for allowed actions are determined based on a role of a user of the editing pane.

[0321] A system of any of Example Clauses RRR, Example Clauses PPP through QQQ, further comprising:

[0322] means for identifying available image capture devices for a real-world environment; and

[0323] Means for changing a current image capture device used to provide an image of the three-dimensional representation based on the content of the zoom window to improve the image quality of the rendered content.

[0324] Example Clause SSS, a system of any of Example Clauses PPP to RRR, wherein the selectable options include sending the content or source file to a selected recipient.

[0325] Example clause TTT, the system of any of example clauses PPP to SSS, wherein the content of the editing pane and permissions for allowed actions are determined based on a role of a user of the editing pane.

[0326] Among many other technical benefits, the techniques herein enable more efficient use of computing resources, such as processor cycles, memory, network bandwidth, and power, compared to previous solutions that rely on inefficient manual placement of virtual objects in a 3D environment. Other technical benefits not specifically mentioned herein may also be achieved through implementations of the disclosed subject matter.

[0327] Although the technology has been described in language specific to structural features and / or methodological acts, it should be understood that the appended claims are not necessarily limited to the described features or acts. Rather, the features and acts are described as example implementations of the technology.

Claims

1. A system, include: one or more data processing units; as well as A computer-readable medium having encoded thereon computer-executable instructions for causing the one or more data processing units to: rendering a representation of an environment on a user interface (UI), the environment being indicative of an interactive communication session between a plurality of users; receiving input data indicating a location within the representation where a zoom window is to be placed; rendering the zoom window at the location within the representation on the UI in response to the input data; rendering within the zoom window a magnified view of a portion of the representation proximate the location of the zoom window; receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating an adjustment of a size of the zoom window; In response to the first gesture, adjusting a size of the zoom window according to the first gesture on the UI, wherein a proportion of the magnified view within the zoom window is maintained when the size of the zoom window is adjusted; receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window; in response to the second gesture, updating, on the UI, the magnified view of the portion of the representation proximate the location of the zoom window according to the second gesture, wherein a size of the zoom window is maintained while updating the magnified view; identifying a source file or document of content rendered within the zoom window; allowing access to the source file or document during the interactive communications session; and Identifies additional source files or documents for new content rendered in the representation.

2. The system according to claim 1, in, The instructions further cause the one or more data processing units to perform the following operations: receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window within the representation; In response to the third gesture, repositioning the zoom window to the new location on the UI, wherein the size of the zoom window is maintained during the repositioning; as well as An updated magnified view of an updated portion of the representation proximate the new position of the zoom window is rendered within the zoom window.

3. The system according to claim 1, in, The first gesture is a resize gesture input with two fingers applied to a touch-sensitive surface at an edge of the zoom window.

4. The system according to claim 1, in, The second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

5. The system according to claim 1, in, The instructions further cause the one or more data processing units to modify a boundary of the zoom window to indicate the changed zoom ratio.

6. The system according to claim 1, in, The instructions further cause the one or more data processing units to perform the following operations: receiving input data indicative of a third gesture applied to the zoom window, the third gesture indicative of scrolling the portion of the representation proximate the location of the zoom window; as well as In response to the third gesture, rendered content within the zoom window is updated based on the scrolling.

7. The system according to claim 1, in, When the zoom window approaches a participant in the environment, the instructions further cause the one or more data processing units to perform the following operations: As the participant moves, the zoom window is automatically repositioned to remain close to the participant's new location.

8. The system according to claim 1, in, The instructions further cause the one or more data processing units to perform the following operations: receiving input data indicating a change in perspective of the representation; and In response to the change in viewing angle, the zoom window is automatically repositioned to maintain a view of the portion of the representation.

9. A method for interacting with a rendered environment, the method include: rendering a representation on a user interface (UI), the representation indicating an interactive communication session between a plurality of users; receiving input data indicating a location within the representation where a zoom window is to be placed; Rendering the zoom window at the location within the representation on the UI in response to the input data, the zoom window having a size determined based on one or more criteria; rendering within the zoom window a magnified view of a portion of the representation proximate the location of the zoom window; receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating an adjustment of a size of the zoom window; In response to the first gesture, adjusting a size of the zoom window according to the first gesture on the UI, wherein a proportion of the magnified view within the zoom window is maintained when the size of the zoom window is adjusted; receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window; in response to the second gesture, updating, on the UI, the magnified view of the portion of the representation proximate the location of the zoom window according to the second gesture, wherein the size of the zoom window is maintained while updating the magnified view; identifying a source file or document of content rendered within the zoom window; allowing access to the source file or document during the interactive communications session; and Identifies additional source files or documents for new content rendered in the representation.

10. The method according to claim 9, further comprising: include: receiving input data indicating a third gesture applied to the zoom window, the third gesture indicating a new position of the zoom window within the representation; In response to the third gesture, repositioning the zoom window to the new location on the UI, wherein the size of the zoom window is maintained during the repositioning; as well as An updated magnified view of an updated portion of the representation proximate the new position of the zoom window is rendered within the zoom window.

11. The method according to claim 9, in, The first gesture is a resize gesture input with two fingers applied to a touch-sensitive surface at an edge of the zoom window.

12. The method according to claim 9, in, The second gesture is a pinch gesture applied to the touch-sensitive surface within the zoom window.

13. The method according to claim 9, in, The representation is a video feed of the collaborative work environment.

14. The method according to claim 9, further comprising: include: receiving input data indicative of a third gesture applied to the zoom window, the third gesture indicative of scrolling the portion of the representation proximate the location of the zoom window; as well as In response to the third gesture, rendered content within the zoom window is updated based on the scrolling.

15. A system, include: means for rendering a representation of a real-world environment on a user interface (UI), the representation being indicative of an interactive communication session between a plurality of users; means for receiving input data indicating a position within the representation where a zoom window is to be placed; means for rendering the zoom window at the location within the representation on the UI in response to the input data, the zoom window having a size determined based on one or more criteria; means for rendering within the zoom window a magnified view of a portion of the representation proximate the location of the zoom window; means for receiving input data indicating a first gesture applied to the zoom window, the first gesture indicating an adjustment of a size of the zoom window; means for adjusting a size of the zoom window according to the first gesture on the UI in response to the first gesture, wherein a proportion of the magnified view within the zoom window is maintained when the zoom window is adjusted in size; means for receiving input data indicating a second gesture applied to the zoom window, the second gesture indicating a change in a zoom scale of content within the zoom window; means for updating, on the UI in response to the second gesture, the magnified view of the portion of the representation proximate the location of the zoom window according to the second gesture, wherein the size of the zoom window is maintained while updating the magnified view; means for identifying a source file or document of content rendered within the zoom window; means for allowing access to said source file or document during said interactive communication session; and An element used to identify additional source files or documents for new content rendered in the representation.