Dynamic configuration of a communication video stream arrangement based on aspect ratio of available display area

Abstract

The technology provided improves user engagement and more efficient use of computing resources by dynamically configuring video stream arrangements based on the aspect ratio of the available display area (401) and a target aspect ratio (410). Such technology provides a dynamically optimized user interface arrangement that accommodates any given shape of available display area. The technology can also help viewers maintain focus on the salient content of a multi-stream display as users resize display windows, move windows into random shapes, rotate devices, join communication sessions, leave communication sessions, present content streams, remove content streams, etc. The target aspect ratio can be leveraged in the process to recursively divide the display area horizontally or vertically to accommodate any number of stream renderings. Adjustments to the target aspect ratio can allow for optimization of display of any number of video stream renderings for any shape of display screen.

Description

BACKGROUND

[0001] There are many different communication systems that allow users to collaborate. For example, some systems allow people to collaborate by using live video streams, live audio streams, and other forms of text-based or image-based media. Participants of a communication session can share video streams that show a person or a group of people and a display of shared content. Such systems can provide participants of a communication session with an experience that simulates an in-person meeting.

[0002] Despite the many different types of systems that allow users to collaborate, such systems still have many drawbacks. For example, most existing systems have a user interface layout that adds or removes participant images in a fixed order when the number of people participating in an online meeting changes. When a participant joins a meeting, each participant's image is added to a fixed location or a randomly selected location. Such activity can distract from the displayed content. Worse, some video stream renderings can be rearranged or moved as new streams are added to or removed from the user interface. Such movement can make it difficult for viewers to follow the activity of the meeting. Such user interface arrangements can not optimally facilitate user engagement because participants can not clearly see important gestures performed by everyone, especially if a particular person's image is repositioned as other users join or leave the session. Such problems can hinder user engagement and reduce the efficiency of a communication session.

[0003] Software applications that do not facilitate user engagement can result in lost productivity and inefficiency with respect to computing resources. For example, participants of a communication session, such as an online meeting, can need to reference a recording or other resource when they miss or overlook content. When users miss a point during a live meeting, content can need to be resent or replayed. Such activity can result in inefficient use of network, processor, memory, or other computing resources. Likewise, when a participant's level of engagement is negatively affected by a poor user interface arrangement, such lost productivity can result in the need for extended meetings or follow-up meetings, which in turn consume additional computing resources. Such lost productivity and inefficiency with respect to computing resources can be even more severe when a system is used to provide a collaboration environment for a large number of participants.

[0004] In addition to the loss of user engagement, when a communication system is unable to effectively display live video of people, many other inefficiencies can result. Participants can miss important social cues, such as when a person raises their hand, begins to speak, looks in a particular direction, and so on. Such shortcomings sometimes require users to manually interact with many different systems. For example, some users will still send text messages or emails, and so on, to other participants in a teleconference if a cue is missed. Such manual steps disrupt people's workflows and are very inefficient in helping people establish a collaborative protocol with a group of people. Such shortcomings of existing systems can result in productivity losses and inefficient use of computing resources. SUMMARY

[0005] The technology disclosed herein improves user engagement and more efficient use of computing resources by providing a system that dynamically configures a communication video stream layout based on an aspect ratio of an available display area and a target aspect ratio. Such technology provides a dynamically optimized user interface "UI" layout to accommodate a wide range of dimensions of available display areas. The technology also helps viewers maintain focus on salient content of a multi-stream display as users resize display windows, reshape display windows to irregular sets of dimensions, rotate devices, join communication sessions, leave communication sessions, present content streams, remove content streams, and so on. The target aspect ratio can be utilized in a process to recursively divide a display area horizontally or vertically to accommodate any number of stream renderings. The system can use the target aspect ratio to determine the position, size, and aspect ratio for each stream. The overall look and feel of the user interface can be controlled by the target aspect ratio. Adjustments to the target aspect ratio can allow the system to automatically lay out the user interface to optimally display any number of video stream renderings. One of the benefits of the target aspect ratio is the movement of any stream rendering, for example, a new person added to the UI or removed from the UI, away from a rendering of any stream with salient content. The use of the target aspect ratio also minimizes changes to the size, shape, and position of salient streams.

[0006] The examples described herein are provided in the context of a collaborative environment involving a communication session, such as a private chat session, a multi-user editing session, a group meeting, a live broadcast, and so on. For illustrative purposes, it can be appreciated that a computer that manages a collaborative environment involves any type of computer that manages a communication session in which two or more computers share video data: both recorded video streams and live video streams. In addition, it can be appreciated that the technology disclosed herein can be applied to any user interface layout used to display content. The scope of the disclosure is not limited to embodiments associated with a collaborative environment.

[0007] The technology disclosed herein provides improvements to a number of features of existing computers. For example, the use of computing resources such as processor cycles, memory, network bandwidth, and power is more efficient because the system is able to dynamically control the size, position, and shape of video streams that depict a threshold number of people. By providing a dynamic controlled user interface that provides more visual detail for objects of interest, the technology disclosed herein is able to provide more efficient use of computing resources. The system is able to improve user interaction with a computing device by mitigating the need for additional communication systems because the disclosed system is able to mitigate or eliminate the need for requests to resend, repeat content, and the like. Improvements to user interaction with a device can also reduce unnecessary or redundant inputs, which can reduce inadvertent inputs, corrected inputs, and other types of user interaction that utilize computing resources. Other technical benefits not specifically mentioned herein can also be realized by implementing the disclosed subject matter.

[0008] Those skilled in the art will also appreciate that aspects of the subject matter described herein can be implemented on and / or in connection with other computer systems configurations, including multiprocessor systems, microprocessor-based or programmable consumer electronics, augmented reality or virtual reality devices, video game devices, palmtop computers, smart phones, smart televisions, self-driving cars, smart watches, e-readers, tablet computing devices, special purpose hardware devices, networked devices, and the like.

[0009] Features and technical benefits other than those specifically described herein, as prior described, will become apparent throughout this specification and appended drawings. This summary is provided to introduce a selection of concepts, which are further described below in the detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended for use in determining the scope of the claimed subject matter. For example, the term "technology" can refer to system(s), method(s), computer readable instructions, module(s), algorithm(s), hardware logic, and / or operation(s) as permitted by the context and throughout the document. BRIEF DESCRIPTION OF DRAWINGS

[0010] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of the reference numbers identify the figure in which the reference number first appears. Like or similar references in different figures can indicate similar or like items. References to individual items in a set of items can use reference numbers with letters in the letter sequence to refer to each individual item. General references to items can use specific reference numbers without the letter sequence.

[0011] FIG. 1AAspects of a communication system are illustrated in a first state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0012] FIG. 1B Aspects of a communication system are illustrated in a second state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0013] FIG. 1C Aspects of a communication system are illustrated in a third state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0014] FIG. 1D Aspects of a communication system are illustrated in a fourth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0015] FIG. 1E Aspects of a communication system are illustrated in a fifth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0016] FIG. 1F Aspects of a communication system are illustrated in a sixth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0017] FIG. 1G Aspects of a communication system are illustrated in a seventh state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0018] FIG. 1H Aspects of a communication system are illustrated in an eighth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0019] FIG. 1I Aspects of a communication system are illustrated in a ninth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area.

[0020] FIG. 2A Aspects of a communication system are illustrated in a first state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0021] FIG. 2B Aspects of a communication system are illustrated in a second state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0022] FIG. 2C Aspects of the communication system are illustrated in a third state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0023] FIG. 2D Aspects of the communication system are illustrated in a fourth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0024] FIG. 2E Aspects of the communication system are illustrated in a fifth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0025] FIG. 2F Aspects of the communication system are illustrated in a sixth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0026] FIG. 3 Aspects of the communication system are illustrated in a sixth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area modified by user input.

[0027] FIG. 4A Aspects of a first state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0028] FIG. 4B Aspects of a second state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0029] FIG. 4C Aspects of a third state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated

[0030] FIG. 4D Aspects of a fourth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0031] FIG. 4E Aspects of a fifth state of a process of configuring an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0032] FIG. 4FAspects of a sixth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0033] FIG. 4G Aspects of a seventh state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0034] FIG. 4H Aspects of an eighth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0035] FIG. 4I Aspects of a ninth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0036] FIG. 4J Aspects of a tenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0037] FIG. 4K Aspects of an eleventh state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0038] FIG. 4L Aspects of a twelfth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0039] FIG. 4M Aspects of a thirteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0040] FIG. 4N Aspects of a fourteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0041] FIG. 4O Aspects of a fifteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0042] FIG. 4P Aspects of a sixteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0043] FIG. 4QAspects of a seventeenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0044] FIG. 4R Aspects of an eighteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0045] FIG. 4S Aspects of a nineteenth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated.

[0046] FIG. 5A Aspects of a first state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0047] FIG. 5B Aspects of a second state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0048] FIG. 5C Aspects of a third state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0049] FIG. 5D Aspects of a fourth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0050] FIG. 5E Aspects of a fifth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0051] FIG. 5F Aspects of a sixth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0052] FIG. 5G Aspects of a seventh state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0053] FIG. 5H Aspects of an eighth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0054] FIG. 5I Aspects of a ninth state of a process to configure an arrangement of video stream rendering within a user interface based on an aspect ratio of an available display area and an adjusted target aspect ratio are illustrated.

[0055] FIG. 6 Aspects of a number of different user interface arrangements configured based on an aspect ratio of an available display area and various target aspect ratios are illustrated.

[0056] FIG. 7 is a flowchart illustrating aspects of a routine to generate a user interface arrangement in an efficient manner based on an aspect ratio of an available display area and a predetermined target aspect ratio.

[0057] FIG. 8 is a computing system diagram showing aspects of an illustrative operating environment for the technology disclosed herein.

[0058] FIG. 9 is a computing architecture diagram showing aspects of a configuration and operation of a computing device capable of implementing aspects of the technology disclosed herein. DETAILED DESCRIPTION

[0059] The technology disclosed herein improves user engagement and more efficient use of computing resources by providing a system to dynamically configure a communication video stream arrangement based on an aspect ratio of an available display area. Such technology provides a dynamically optimized user interface "UI" arrangement that accommodates any given shape of available display area. FIGS. 1A-1I Aspects of a process to configure a video stream arrangement of a user interface based on an aspect ratio of an available display area and a target aspect ratio are illustrated. In particular, FIG. 1A Aspects of a system 100 including a server 110 and at least one client device 101 are illustrated. The server 110 can manage a plurality of data streams 111 (individually referred to as 111A, 111B, etc.), each having a video component and / or an audio component that allows the client device 101 to communicate with a plurality of other remote client devices. Generally, the server 110 can generate session data 113 that can control a user interface layout displayed on the client device 101. Additionally, the session data 113 can include one or more data streams that include an audio component and / or a video component. Additional aspects of the server 110 that manages a communication session between client computers are described in greater detail below with reference to the system 602 shown in FIG. 8 Additional aspects of the client device 101 are described in greater detail below with reference to the device 606 shown in FIG. 8

[0060] ​In some configurations, the user interface 120 can include an available display area 401 and a secondary display area 402. Each display area can be used to display different types of content. For example, the available display area 401 can be used to display a live or recorded video stream of a communication session, while the secondary display area 402 can be used to display graphical elements representing individual participants in the communication session. These graphical elements can be used to represent participants who are only transmitting audio streams. The graphical elements can also be utilized when the number of video streams displayed is controlled by one or more predetermined configuration settings. Such a configuration allows some participants to be represented in the secondary display area 402 instead of in the available display area 401, which can be reserved for selected video streams with a threshold priority level.

[0061] The available display area 401 can be a specified display area within one or more display screens. For example, the available display area 401 can be a specified display area spanning multiple display devices or a specified display area within a single display device. In an illustrative example, a specific available display area 401 can be selected based on the boundaries of a particular display window and one or more reserved areas (such as secondary display area 402). The available display area 401 can be defined by one or more parameters, such as aspect ratio, one or more coordinates of the corners defining the available display area 401, vector data, and / or other data defining the viewable area of ​​the user interface 120.

[0062] exist FIG. 1A In the example, the available display area 401 is a portion of the user interface 120 reserved for displaying the rendering of the selected stream 111 transmitted from server 110. The user interface 120 can be controlled at least in part by display data 104 that can be generated by client device 101. As shown, a first state of system 100 indicates that server 110 is transmitting a first stream 111A and session data 113 to client device 101, and causes client device 101 to generate display data 104 that defines the arrangement of user interface 120. As a result, user interface 120 is able to display a first rendering 111A' of the first stream 111A.

[0063] As in FIGS. 1A-1IThe series of images shown illustrate how, as the server 110 adds additional streams 111B-111N to the session data 113, the client device 101 changes the arrangement of the stream renderings 111A'-111N' shown within the available display area 401 of the user interface 120. The series of images are provided to illustrate one example of a user interface arrangement that can result from the techniques disclosed herein. In one aspect, the example illustrates how the techniques disclosed herein can minimize movement of selected renderings as streams are added to a communication session. Additionally, the example illustrates, in conjunction with other examples, how a computing system can use a target aspect ratio to change characteristics of a user interface layout to maximize use of a display area.

[0064] FIG. 1B A second state of the process for configuring a user interface displaying a communication video stream arrangement is illustrated. In this state, the addition of the second stream 111B causes the client device to display a rendering 111B' of the second stream 111B in accordance with the techniques disclosed herein. As will be described in greater detail below, the aspect ratio of the available display area is compared to a target aspect ratio to determine the position and size of the rendering 111B' of the second stream 111B. Additionally, the client device 101 adjusts the rendering of the first rendering 111A' of the first stream 111A, allocating space for the added rendering.

[0065] FIG. 1C A third state of the process for configuring a user interface displaying a communication video stream arrangement is illustrated. In this state, the addition of the third stream 111C causes the client device to display a rendering 111C' of the third stream 111C in accordance with the techniques disclosed herein. As will be described in greater detail below, one or more portions of the available display area 401 are analyzed to determine the position and size of the rendering 111C' of the third stream 111C. Additionally, the client device 101 adjusts the existing renderings 111A'-111B', allocating space for the added rendering.

[0066] FIG. 1D A fourth state of the process for configuring a user interface displaying a communication video stream arrangement is illustrated. In this state, the addition of the fourth stream 111D causes the client device 101 to display a rendering 111D' of the fourth stream 111D in accordance with the techniques disclosed herein. As will be described in greater detail below, one or more portions of the available display area 401 are analyzed to determine the position and size of the rendering 111D' of the fourth stream 111D. Additionally, the client device 101 adjusts the existing renderings 111A'-111C', allocating space for the added rendering.

[0067] FIGS. 1E-1IFIGURE 13 illustrates a series of images showing an example process for configuring an arrangement of video stream renderings within a user interface. As shown, as each stream (e.g., fifth stream 111E through ninth stream 1111) is added, the client device 101 displays a rendering 111E' - 1111' of the fifth stream 111E through the ninth stream 1111 in accordance with the techniques disclosed herein. As will be described in greater detail below, as each stream 111 is added to the communication session, the client device 101 analyzes the aspect ratio of individual portions of the available display area 401 to determine the position and size of each rendering 111E' - 1111' of the newly added streams 111E - 1111. In addition, the client device 101 adjusts the existing renderings 111A' - 111D' to allocate space for the added renderings.

[0068] Reference is now made to FIGS. 2A-2F Another example process for configuring a user interface including an arrangement of video stream renderings is shown and described below. In this example, the arrangement of video stream renderings within the user interface 120 is configured when the user interface 120 is resized and / or reshaped. In some configurations, the techniques disclosed herein can be performed each time an input to resize or reshape the user interface 120 is received, including the routine shown in FIG. 7 FIGURE 14.

[0069] As shown in the series of images in FIGS. 1A-1I FIGURE 13, as the server 110 transmits five streams 111A - 11 IE to the client device 101. As the shape and size of the user interface changes, the client device recalculates the size, shape, and position of each rendering 111A' - 111E' of the streams 111A - 111E. This series of images is provided to illustrate one example of a user interface arrangement that can result from the techniques disclosed herein. In one aspect, this example illustrates how the techniques disclosed herein can minimize movement of selected renderings while moving other streams or resizing them. In addition, this example, in conjunction with other examples, illustrates how the computing system can use a target aspect ratio to change features of the user interface layout to maximize the use of the display area.

[0070] In FIG. 2B and FIG. 2CThe transition between the user interfaces shown in FIG. 2B and FIG. 2C the user interface transitions to an arrangement that includes rendering of the first and second streams in the second row and rendering of the third, fourth, and fifth streams. In the transition between the user interfaces shown in FIG. 2C and FIG. 2D the user interface transitions to an arrangement that includes rendering of the first stream in the first column, rendering of the second and fifth streams in the second column, and rendering of the third and fourth streams in the third column. In the transition between the user interfaces shown in FIG. 2D and FIG. 2E the user interface transitions to an arrangement that includes rendering of the first stream in the first column, rendering of the second stream in the second column, rendering of the third stream in the third column, and rendering of the fourth and fifth streams in the fourth column. In the transition between the user interfaces shown in FIG. 2E and FIG. 2F the user interface transitions to an arrangement that includes rendering of the first stream in the first column, rendering of the second stream in the second column, rendering of the third stream in the third column, rendering of the fourth stream in the fourth column, and rendering of the fifth stream in the fifth column. As will be described below, the techniques disclosed herein, including the routines described in connection with FIG. 7 can be used to determine a layout of stream renderings based on a number of streams, an aspect ratio of an available display area, and a target aspect ratio when the user interface 120 is resized or reshaped.

[0071] Turning now toFIG. 3 The following illustrates and describes another exemplary process for configuring a user interface that includes the arrangement of video stream rendering. In this example, the arrangement of video stream rendering within the user interface is configured when the user interface 120 is reshaped due to a change in the orientation of the display screen. In this example, the aspect ratio of the available display area (e.g., the visible portion of the display screen) is modified when the device rotates from portrait to landscape. Based on the aspect ratio of the available display area, a given target aspect ratio, and the number of selected streams, the system is able to arrange the rendering of each selected stream to optimize the display for each stream. In this example, in portrait mode, the system is able to create individual rows for the first three streams 111A-111C and position the fourth and fifth streams side-by-side in the fourth row. In landscape mode, the system is able to create individual columns for the first stream 111A, second columns for the second and fourth streams, and third columns for the third and fifth streams. As described in more detail below, these layouts can be further customized to optimize the use of the available display area by adjusting the target aspect ratio. Such an arrangement can be achieved through a recursive partitioning process (e.g., FIG. 7 The routine 500 is generated each time the screen is redirected, each time the window is reshaped or resized, or each time an available display area is allocated.

[0072] Now for reference FIGS. 4A-4S The following describes the process for configuring a user interface that includes the arrangement of video stream renderings. In this example, the available display area 401 will be used to display nine renderings of nine individual video streams received by the client device. The rendering of the nine individual video streams is arranged based on the order in which the streams are added to the communication session. The order and position in which the renderings are added to the user interface minimizes the movement of renderings of prominent streams and the movement of renderings of streams located near prominent streams. Stream renderings can also be removed in the same order to minimize the movement of renderings of prominent streams and the movement of renderings of streams located near prominent streams.

[0073] Typically, the process for determining the arrangement of video stream rendering within the user interface is recursive and divides the available display area into blocks (also referred to herein as “divisions”) with units for displaying each video stream rendering. These blocks can be horizontal blocks (“block rows”) or vertical blocks (“block columns”), depending on the aspect ratio of the available display area 401 or the aspect ratio of the units within the block. The process optimizes the position and shape of each unit by using a specified target aspect ratio. The target aspect ratio can be any small value, such as 4:3 (1.33), 1:2 (0.5), 1:1 (1), 7:9 (.77), etc. By optimizing the user interface based on the target aspect ratio, the division of the available display area is configured to mitigate the movement of rendering within the user interface, thus preventing user distraction from the movement of added or removed rendering. The units are labeled alphabetically to illustrate how each unit maintains its relative position when other units are added to the user interface arrangement. Units can be added or removed according to the label sequence shown in these examples. For illustrative purposes, a "grid" is an array of "tiles". A "grid" also has a target aspect ratio (TAR) and attributes that define whether its tiles are vertical tiles ("columns") or horizontal tiles ("rows"). A "tile" is an array of "cells". A "cell" is a subdivision of a tile and has an "order" attribute used to maintain tracking of the order in which they were added to the grid. Each cell is reserved for a single rendering of the video stream. The aspect ratio of each cell is based on the aspect ratio of the available display area, the aspect ratio of each tile, and the number of cells in each tile.

[0074] The process can begin by analyzing the number of selected streams provided to the client to determine if the number of units in the available display area equals the number of selected streams. If the number of units in the available display area equals the number of selected streams, the system can then display the rendering. For example, if the client device will only display one selected stream and the available display area contains one unit, the available display area is not divided, and the client device will render the selected stream to fill the entire available display area.

[0075] Return to the current location FIG. 4A The example shown assumes that the client device will display nine (9) selected video streams. To begin the process, the available display area begins with a single block 411, which is an allocated area starting with a unit "Unit A 412A". Because the number of selected streams is greater than the number of units, the process proceeds to the operation of dividing the available display area 401.

[0076] In one illustrative example, the available display area 401 can be divided vertically when the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition when the aspect ratio of the available display area 401 is greater than the target aspect ratio. In some embodiments, the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition when the aspect ratio of the available display area 401 is greater than or equal to the target aspect ratio. In some embodiments, the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition if the aspect ratio of the available display area 401 is not less than the target aspect ratio.

[0077] The available display area 401 can be divided horizontally when the aspect ratio of the available display area 401 satisfies the aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 satisfies the target aspect ratio condition when the aspect ratio of the available display area 401 is less than the target aspect ratio. These examples are provided for illustrative purposes only and should not be construed as limiting. It can be appreciated that any condition can be utilized by the techniques disclosed herein. In general, a cell or display area can be divided horizontally if its width and height are greater than target values, respectively; and the cell or display area can be divided vertically if it is narrower than the target values. The techniques disclosed herein can utilize a general definition of aspect ratio that defines width and height values. However, the techniques disclosed herein can also utilize other definitions that define the shape of a viewing cell or display area, e.g., width / height or height / width, etc. In such embodiments, the target aspect ratio condition can utilize the target values "inverse" ratio or "greater" or "less" accordingly if the opposite or different definition (e.g., width / height or height / width) is used to set the threshold.

[0078] In the example of FIG. 4A, the available display area 401 is divided into two display areas 402 and 403. In this example, the available display area 401 is divided vertically because the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition. In this example, the target aspect ratio is 7:9 and the aspect ratio of the available display area 401 is 1:1. In this example, the available display area 401 is divided into two display areas 402 and 403, each having an aspect ratio of 1:1. FIG. 4A In the example of FIG. 4A, the available display area 401 is divided into two display areas 402 and 403. In this example, the available display area 401 is divided vertically because the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition. In this example, the target aspect ratio is 7:9 and the aspect ratio of the available display area 401 is 1:1. In this example, the available display area 401 is divided into two display areas 402 and 403, each having an aspect ratio of 1:1. FIG. 4BAs shown in FIG. 4, the available display area is divided vertically. In particular, the target aspect ratio is less than the aspect ratio of the available display area. Thus, the available display area is divided vertically and data defining the vertical tiles is generated. When the available display area is divided vertically, each portion resulting from the division is populated with a vertical tile 411, each tile containing one cell. In this example, each vertical tile 411 is populated with a first cell (cell A 412A) and a second cell (cell B 412B), respectively. Each cell can be populated with a video stream rendering based on a priority of each stream. In some configurations, the stream rendering can be prioritized based on an order in which participants joined the communication session. In some configurations, the streams can be prioritized based on an activity level associated with each stream, such as a level of movement of a participant depicted in the stream, a volume level of a participant depicted in the stream, and the like. Each stream can be positioned within each cell based on the priority, e.g., cell A can be used to display streams having a first priority and cell B can be used to display streams having a second priority. The priority of a stream can be based on an order in which the stream joined the communication session, a type of content depicted in the stream (e.g., a word document versus a data spreadsheet), or a given priority based on a number of people depicted in a video component of the stream.

[0079] Next, as shown in FIG. 4B, the tiles are analyzed to determine whether the tiles are equally populated with cells. In this example, each tile 411 contains only one cell, e.g., the cells are equally populated. Thus, the process continues to the operation of analyzing at least one cell. FIG. 4C

[0080] FIG. 4D FIG. 4C illustrates aspects of an operation for analyzing a cell of a user interface. For this operation, any cell can be selected for analysis. In some configurations, the bottom cell, the cell to the right of the tiles, or the most recently added cell can be selected. In this example, cell B 412B is selected. In this operation, if the aspect ratio of the selected cell is less than the target aspect ratio (CAR < TAR) and if the selected cell is in a vertical tile, a new cell is added to the last tile, e.g., the last tile can be divided horizontally. In this particular example, since the selected cell is in a vertical tile and since the aspect ratio of the selected cell, 1 :2, is less than the target aspect ratio, 7:9, a new cell (cell C 412C) is added to the last tile, as shown in FIG. 4D. FIG. 4E

[0081] ​​Each time a new cell is added, the system checks whether the number of cells is equal to the number of selected streams. Once the number of cells is equal to the number of selected streams, the routine can terminate. For example, if the number of selected streams is three, the recursive process of further dividing the display area will terminate in the configuration shown in FIG. 4E However, returning to the current example, assume that the number of selected streams is nine, then the routine continues with the operations shown in FIG. 4F

[0082] Once a cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are equally filled. As shown in FIG. 4F Assume that the vertical tiles 411 do not have the same number of cells, then a new cell is added to the tile with the least number of cells. As shown in FIG. 4G A new cell (cell D 412D) is added to the first tile.

[0083] As described above, each time a cell is added, and if the number of cells is less than the number of selected streams, the process starts with the operation where the system determines whether the cells are equally filled. In the state shown in FIG. 4H If it is determined that the tiles are equally filled, then the process proceeds to the operation of selecting one of the cells for analysis. Any of the cells can be selected for analysis. In some configurations, the bottom right cell of the display area can be selected for analysis. In this example, cell C 412C is selected for analysis. The analysis can compare the aspect ratio of the selected cell to the target aspect ratio.

[0084] If the cell aspect ratio is greater than the target aspect ratio and if the selected cell is in a vertical tile, then the system rotates the tile, e.g., changes the vertical tile to a horizontal tile, and adds a new cell to the last tile. As shown in FIG. 4I The cell aspect ratio 1 : 1 is greater than the target aspect ratio 7:9. As a result, as shown in FIG. 4J The vertical tiles are converted to horizontal tiles 413, where the individual horizontal tiles 413 are aligned with individual rows of cells. After rotating the tiles, a new cell, such as cell E 412E, is added to the last tile. The last tile can include the rightmost tile and / or the bottom tile.

[0085] Once a cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are equally filled. In FIG. 4K ​In the state of the display area shown in FIG. 4L a new cell is added to the first horizontal chunk 413, as shown in

[0086] Likewise, once the cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are equally filled. In the state of the display area shown in FIG. 4M it is determined that the chunks are equally filled. If it is determined that the chunks are equally filled, the process proceeds to the operation of selecting one of the cells for analysis. Any of the cells can be selected for analysis. In some configurations, the bottom right cell of the display area can be selected for analysis. In this example, as shown in FIG. 4N cell E 412E is selected for analysis. The analysis can compare the aspect ratio of the selected cell to a target aspect ratio.

[0087] If the cell aspect ratio is less than the target aspect ratio and if the selected cell is in a horizontal chunk, the system rotates the chunk, e.g., changes the horizontal chunk to a vertical chunk, and adds a new cell to the last chunk. As shown in FIG. 4N the cell aspect ratio 2:3 is less than the target aspect ratio 7:9. As a result, as shown in FIG. 4O the horizontal chunk is converted to a vertical chunk 411, where the vertical chunk 411 is aligned with each existing column of cells. After generating the vertical chunk 411, a cell, such as cell G 412G, is added to the last chunk. The last chunk can include the right and / or bottom chunk.

[0088] Once the cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are equally filled. In the state of the display area shown in FIG. 4P it is determined that the chunks are not equally filled. The rightmost chunk has three cells, while the other chunks have two cells. As a result, assuming the chunks do not have the same number of cells, a new cell is added to one of the chunks with the least number of cells. As shown in FIG. 4Q a new cell (cell H 412H) is added to the middle chunk 411. In such embodiments with two chunks having the least number of cells, the right and / or bottom chunk can be selected to insert the new cell.

[0089] Once a cell is added, and if the number of cells is less than the number of selected streams, the process re-starts at the operation where the system determines whether the cells are equally filled. In FIG. 4R the display area shown in FIG. 4S , it is determined that the tiles are not equally filled. The middle and rightmost tiles have three cells, while the left tile has one cell. As a result, given that the tiles do not have the same number of cells, a new cell is added to one of the tiles with the fewest number of cells. As shown in FIG. 4S , a new cell (cell I 412I) is added to the left tile 411. Since the number of cells is equal to the number of selected streams, the recursive partitioning process can terminate.

[0090] Reference is now made to FIGS. 5A-5I , another example process for configuring a user interface including an arrangement of video stream renderings is shown and described below. This example follows the regression process described above, also using an initial available display area 401 having a 1 : 1 aspect ratio. However, in this example, the target aspect ratio is 4:3. This example illustrates how the user interface arrangement can be modified by changing only the target aspect ratio. In this example, the available display area 401 will be used to display five renderings of five individual video streams that are selected for display. The renderings of the individual video streams are arranged based on the order in which the streams were added to the communication session. The order and position in which the renderings are added to the user interface can minimize movement of the rendering of a dominant stream and minimize movement of stream renderings located near the rendering of a dominant stream.

[0091] The process can begin by analyzing the plurality of selected streams to determine whether the available display area has a number of cells equal to the number of selected streams. If the number of cells of the available display area is equal to the number of selected streams, the system can display the renderings. For example, if the client device displays only one selected stream and the available display area contains one cell, the available display area is not partitioned, and the client device fills the entire available display area with a rendering of the selected stream.

[0092] In the current example of FIG. 5A , assume that the client device will display five (5) selected video streams. To begin the process, the available display area begins with a single tile 411, which is an allocated area that begins with one cell, "cell A 412A". Since the number of selected streams is greater than the number of cells, the process proceeds to the operation of partitioning the available display area 401. In any embodiment, when the available display area 401 contains only one tile, the tile can be a horizontal tile or a vertical tile.

[0093] In one illustrative example, the available display area 401 can be divided vertically when the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition when the aspect ratio of the available display area 401 is greater than the target aspect ratio. In some embodiments, the aspect ratio of the available display area 401 does not satisfy the target aspect ratio condition if the aspect ratio of the available display area 401 is not less than the target aspect ratio. The available display area 401 can be divided horizontally when the aspect ratio of the available display area 401 satisfies the aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 satisfies the target aspect ratio condition when the aspect ratio of the available display area 401 is less than the target aspect ratio.

[0094] In FIG. 5A the example, assume for illustrative purposes that the target aspect ratio is 4:3 and the aspect ratio of the available display area for one is 1 : 1. When such parameters are processed using the aspect ratio condition defined above, as shown in FIG. 5B , the available display area is divided horizontally. In particular, the target aspect ratio is greater than the aspect ratio of the available display area. Accordingly, the available display area is divided horizontally and data defining horizontal tiles is generated. When the available display area is divided horizontally, each portion resulting from the division is filled with a horizontal tile 413, each tile containing one cell. In this example, each horizontal tile 413 is filled with a first cell (cell A 412A) and a second cell (cell B 412B), respectively. Each cell can be filled with a video stream rendering based on a priority of each stream. In some configurations, the stream rendering can be prioritized based on an order in which participants join the communication session. In some configurations, the streams can be prioritized based on an activity level associated with each stream, such as a level of movement of a participant depicted in the stream, a level of volume of a participant depicted in the stream, and the like. Each stream rendering can be positioned within each cell based on a priority of each stream, for example, cell A can be used to display a stream having a first priority and cell B can be used to display a stream having a second priority.

[0095] Next, as shown in FIG. 5C , the tiles are analyzed to determine whether the tiles are equally filled with cells. In this example, each tile 413 contains only one cell, for example, the cells are equally filled. Accordingly, the process continues to the operation of analyzing at least one cell.

[0096] FIG. 5DAspects of an operation for analyzing selected cells of a user interface are shown. For this operation, any cell can be selected for analysis. In some configurations, the bottom cell can be selected, or the most recently added cell can be selected. In this example, cell B 412B is selected. In this operation, if the aspect ratio of the selected cell is greater than the target aspect ratio (CAR > TAR) and if the selected cell is in a horizontal partition, a new cell is added to the last partition, e.g., the last partition can be divided vertically. In this particular example, since the selected cell is in a horizontal partition, and since the aspect ratio of the selected cell, 2: 1, is greater than the target aspect ratio, 4:3, a new cell (cell C 412C) is added to the last partition, as shown in FIG. 5E .

[0097] Each time a new cell is added, the system checks to see if the number of cells is equal to the number of selected streams. Once the number of cells is equal to the number of selected streams, the routine can terminate. For example, if the number of selected streams is three, the recursive process of further dividing the display area will terminate in the configuration shown in FIG. 5E . However, returning to the current example, assume the number of selected streams is five and the number of cells is three, the routine continues with the operation shown in FIG. 5F .

[0098] Once a cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are evenly filled. As shown in FIG. 5F , assume that the partitions 413 do not have the same number of cells, a new cell is added to the partition with the fewest number of cells. As shown in FIG. 5G , a new cell (cell D 412D) is added to the first partition.

[0099] Likewise, as part of the recursive process, once a cell is added, and if the number of cells is less than the number of selected streams, the process restarts at the operation where the system determines whether the cells are evenly filled. In the state shown in FIG. 5G , it is determined that the partitions are evenly filled. If it is determined that the partitions are evenly filled, the process proceeds to the operation of selecting one of the cells for analysis. Any of the cells can be selected for analysis. In some configurations, the bottom right cell of the display area can be selected for analysis. In this example, cell C 412C is selected for analysis. The analysis can compare the aspect ratio of the selected cell to the target aspect ratio.

[0100] As shown in FIG. 5HAs shown in FIG. 4B, if the cell aspect ratio is less than the target aspect ratio (CAR < TAR) and if the selected cell is in a horizontal tile, the system rotates the tile, e.g., changes a horizontal tile to a vertical tile, and adds a new cell to the last tile. In the current example, the cell aspect ratio 1 : 1 is less than the target aspect ratio 4:3, and the selected cell is in a horizontal tile. As a result, as shown in FIG. 4C, the horizontal tile 413 is converted to a vertical tile 411, and the individual vertical tiles 411 are aligned with individual cell rows. After rotating the tile, a new cell, such as cell E 412E, is added to the last tile. The last tile can include the rightmost tile and / or the bottom tile. At this point, the process stops the recursive partitioning process because the number of selected streams is equal to the number of cells in the available display area. FIG. 5I As shown in FIG. 4B, if the cell aspect ratio is less than the target aspect ratio (CAR < TAR) and if the selected cell is in a horizontal tile, the system rotates the tile, e.g., changes a horizontal tile to a vertical tile, and adds a new cell to the last tile. In the current example, the cell aspect ratio 1 : 1 is less than the target aspect ratio 4:3, and the selected cell is in a horizontal tile. As a result, as shown in FIG. 4C, the horizontal tile 413 is converted to a vertical tile 411, and the individual vertical tiles 411 are aligned with individual cell rows. After rotating the tile, a new cell, such as cell E 412E, is added to the last tile. The last tile can include the rightmost tile and / or the bottom tile. At this point, the process stops the recursive partitioning process because the number of selected streams is equal to the number of cells in the available display area.

[0101] Referring now to FIGS. 5A-5D, FIG. 6 a plurality of different user interface arrangements based on the aspect ratio of the available display area and various target aspect ratio configurations are shown and described below. These examples illustrate the versatility of the technology disclosed herein. A single target aspect ratio can be used to configure a user interface or available display area having any aspect ratio. Similar to the examples described above, the order in which different streams are displayed within a cell can be based on a designated letter, e.g., A, B, C, etc.

[0102] These examples are provided for illustrative purposes and should not be construed as limiting. It can be appreciated that a user interface of an available display area having any dimensions can be configured using any target aspect ratio. At the same time, it can be appreciated that different target aspect ratios can be used to produce different arrangements for any given user interface dimension to optimize the screen space of a device while providing a variety of different arrangement styles to suit different needs.

[0103] FIG. 7 is a diagram illustrating aspects of a routine 500 to improve user engagement and more efficient use of computing resources by providing a system to dynamically configure a communication video stream arrangement based on the aspect ratio of the available display area and a target aspect ratio. One of ordinary skill in the art will appreciate that the operations of the methods disclosed herein do not have to be performed in any particular order and that some or all of the operations can be performed in alternative orders or in parallel (that is, concurrently, or with partial concurrence). Operational flow is illustrated for simplicity and ease of understanding. Operations can occur in an order different than that described in this figure "A" and "B" are merely used to represent two individual operations that are performed sequentially or with partial concurrence. While the method is shown as a particular sequence of operations, it is understood that some steps can be skipped, reversed or performed concurrently with others, and / or at least some steps performed concurrently with other steps, at least in part, without departing from the teachings of the present disclosure.

[0104] It should also be understood that the methods exemplarily illustrated can end at any time and need not complete all of the steps shown. Some or all operations of these methods can be performed by a computer readable instruction executable by a computer on a computer storage medium, and / or substantially equivalent operations, as defined herein. The term "computer readable instructions" and variations thereof as used in the description and claims herein as used herein generally refer to machine language instructions, statements, code, or similar expressions. Computer readable instructions can be implemented in various ways, such as in instructions executable by a processor. Additionally, it is common practice to use various techniques to implement the same function by a different set of instructions in similar or different programming languages. Alterations and permutations of the illustrated methods, other than those explicitly described, are therefore to be considered within the scope of the present disclosure. The terms "instructions" and "code" should be interpreted broadly to include any set of instructions, statements, code, or similar expressions, whether in a language that is interpreted or compiled, and whether expressed in a text file, source file, binary file, executable file, or other file format. Also, the terms "first," "second," or the like do not imply that the components so described must be in a given order. Rather, these terms are generally used to distinguish one component from another. The terms "program," "application," "module," "engine," "system," and the like are used to refer to an ordered listing of executable instructions, routines, or code that are designed to implement particular processes, algorithms, or methods.

[0105] Accordingly, it will be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system such as those described herein and / or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations can be implemented with software, firmware, special purpose digital logic, and any combination thereof.

[0106] Additionally, the operations illustrated in the other figures can be implemented in association with the exemplary rendering UI described above. For example, the various devices and / or modules described herein can generate, transmit, receive, and / or display data associated with content of a communication session (e.g., live content, a broadcast event, recorded content, etc.) and / or a rendered presence UI that includes one or more participants of a remote computing device associated with the communication session. FIG. 7

[0107] The routine 500 begins at operation 502 where one or more computing modules receive a plurality of streams. As described in greater detail below, a system can manage a plurality of streams received from a plurality of different client devices. The streams can be bundled and transmitted to individual computing devices that can be used to display different arrangements of each stream. Each stream can include an audio component and a video component.

[0108] ​Next, at operation 504, the one or more computing devices analyze the received streams to determine whether the number of cells of the available display area is equal to the number of streams selected for display. If the number of cells of the available display area is equal to the number of selected streams, the routine can terminate or return to operation 502, where the one or more computing devices can receive the streams and display the rendering of the streams. If the one or more devices display only one selected stream, and the available display area contains one cell, the available display area is not divided, and the one or more devices render the selected stream. However, at operation 504, if the number of selected streams is greater than the number of cells of the user interface, the routine 500 proceeds to operation 506, where the divisions of the user interface are analyzed to determine whether the divisions are equally filled with cells.

[0109] At operation 506, if the divisions of the user interface are not equally filled with cells, the routine 500 proceeds to operation 508, where a cell is added to the division with the least number of cells. However, if the divisions of the user interface are equally filled with cells, the routine 500 proceeds to operation 510, where the selected cells of at least one division are analyzed to determine whether the aspect ratio of the cells meets one or more criteria.

[0110] In one illustrative example, at operation 510, when there is only one division and one cell, the available display area 401 can be divided vertically when the aspect ratio of the available display area 401 does not meet a target aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 does not meet the target aspect ratio condition when the aspect ratio of the available display area 401 is greater than a target aspect ratio. In some embodiments, the aspect ratio of the available display area 401 does not meet the target aspect ratio condition if the aspect ratio of the available display area 401 is not less than the target aspect ratio. The available display area 401 can be divided horizontally when the aspect ratio of the available display area 401 meets the aspect ratio condition. In some embodiments, the aspect ratio of the available display area 401 meets the target aspect ratio condition when the aspect ratio of the available display area 401 is less than the target aspect ratio.

[0111] In some embodiments, at operation 510, if the aspect ratio of the selected cell is less than the target aspect ratio (CAR < TAR) and if the selected cell is in a vertical division, the routine 500 proceeds to operation 512, where a new cell is added to the last division, e.g., the last division can be divided horizontally. An example of this operation is illustrated in FIG. 5B. FIG. 4EAs shown in FIG. 4B, the vertical tiles are converted to horizontal tiles 413, where the individual horizontal tiles 413 are aligned with the individual rows of cells. After rotating the tiles, new cells, such as cell E 412E, are added to the last tile. The last tile can include the rightmost tile and / or the bottom tile. After operation 514, the routine 500 returns to operation 504, where the one or more devices determine whether the number of streams is equal to the number of cells in the user interface.

[0112] In some embodiments, at operation 510, if the cell aspect ratio is greater than the target aspect ratio (CAR > TAR) and if the selected cell is in a vertical tile, the routine 500 proceeds to operation 514, where the system rotates the tile, e.g., changes the vertical tile to a horizontal tile, and adds a new cell to the last tile. This operation is illustrated in FIG. 4I In this example, the cell aspect ratio 1 : 1 is greater than the target aspect ratio 7:9. As a result, as shown in FIG. 4B, the vertical tiles are converted to horizontal tiles 413, where the individual horizontal tiles 413 are aligned with the individual rows of cells. After rotating the tiles, new cells, such as cell E 412E, are added to the last tile. The last tile can include the rightmost tile and / or the bottom tile. After operation 514, the routine 500 returns to operation 504, where the one or more devices determine whether the number of streams is equal to the number of cells in the user interface. FIG. 4J In this example, the cell aspect ratio 1 : 1 is greater than the target aspect ratio 7:9. As a result, as shown in FIG. 4B, the vertical tiles are converted to horizontal tiles 413, where the individual horizontal tiles 413 are aligned with the individual rows of cells. After rotating the tiles, new cells, such as cell E 412E, are added to the last tile. The last tile can include the rightmost tile and / or the bottom tile. After operation 514, the routine 500 returns to operation 504, where the one or more devices determine whether the number of streams is equal to the number of cells in the user interface.

[0113] In some embodiments, at operation 510, if the cell aspect ratio is less than the target aspect ratio (CAR < TAR) and if the selected cell is in a horizontal tile, the routine 500 proceeds to operation 514, where the tile is rotated, e.g., changed from a horizontal tile to a vertical tile, and a new cell is added to the last tile. An example of this operation is illustrated in FIG. 4N In this example, the cell aspect ratio 2:3 is less than the target aspect ratio 7:9. As a result, as shown in FIG. 4A, the horizontal tiles are converted to vertical tiles 411, where the vertical tiles 411 are aligned with each existing column of cells. After generating the vertical tiles 411, a new cell, such as cell G 412G, is added to the last tile. The last tile can include the right and / or bottom tile. After operation 514, the routine 500 returns to operation 504, where the one or more devices determine whether the number of streams is equal to the number of cells in the user interface. FIG. 4O In this example, the cell aspect ratio 2:3 is less than the target aspect ratio 7:9. As a result, as shown in FIG. 4A, the horizontal tiles are converted to vertical tiles 411, where the vertical tiles 411 are aligned with each existing column of cells. After generating the vertical tiles 411, a new cell, such as cell G 412G, is added to the last tile. The last tile can include the right and / or bottom tile. After operation 514, the routine 500 returns to operation 504, where the one or more devices determine whether the number of streams is equal to the number of cells in the user interface.

[0114] In some embodiments, at operation 510, if the aspect ratio of the selected cell is greater than the target aspect ratio (CAR > TAR) and if the selected cell is in a horizontal tile, the routine 500 proceeds to operation 512, in which a new cell is added to the last tile, e.g., the last tile can be divided vertically. An example of this operation is shown in FIG. 4B. In this particular example, since the selected cell is in a horizontal tile and since the aspect ratio of the selected cell, 2: 1, is greater than the target aspect ratio, 4:3, the routine 500 proceeds to operation 512, in which a new cell (cell C 412C) is added to the last tile, as shown in FIG. 4B. After operation 512, the routine 500 returns to operation 504, in which the one or more devices determine whether the number of streams is equal to the number of cells in the user interface. FIG. 5D FIG. 5E

[0115] It should be appreciated that the above-described subject matter can be implemented as a computer-controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. The operations of the example methods are illustrated in individual blocks and summarized with reference to those blocks. The methods are illustrated as logical flow of blocks, each block of which can represent one or more operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, enable the one or more processors to perform the recited operations.

[0116] Generally, computer-executable instructions include routines, programs, objects, modules, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be executed in any order, combined in any order, subdivided into multiple operations, and / or executed in parallel to implement the described processes. The described processes can be performed by resources associated with one or more devices, such as one or more internal or external CPUs or GPUs, and / or one or more hardware logic, such as field-programmable gate arrays ("FPGAs"), digital signal processors ("DSPs"), or other types of accelerators.

[0117] All of the methods and processes described above can be embodied in, and fully automated via, software code modules executed by one or more general purpose computers or processors. The code modules can be stored in any type of computer-readable storage medium or other computer storage device, such as those described below. Some or all of the methods can alternatively be embodied in specialized computer hardware, such as that described below.

[0118] ​​Any routine descriptions, elements or blocks in the flow diagrams described herein and / or depicted in the attached figures should be understood as potentially representing modules, segments, or codes of the software by one or more executable instructions for implementing specific logic functions or elements in the processes. Alternatives in

[0119] FIG. 8 FIG. 1 is a diagram illustrating an example environment 100 in which techniques disclosed herein can be implemented. In some implementations, the environment 100 can be used to collect, analyze, and share data defining one or more objects displayed to users of a communication session 104.

[0120] As illustrated, the communication session 104 can be implemented between a plurality of client computing devices 106(1) through 106(N) (where N is a number two or greater) associated with or as part of a system 102. The client computing devices 106(1) through 106(N) enable users (also referred to as individuals) to participate in the communication session 104.

[0121] In this example, the communication session 104 is hosted by the system 102 over one or more networks 108. That is, the system 102 can provide a service enabling users of the client computing devices 106(1) through 106(N) to participate in the communication session 104 (e.g., via live viewing and / or recorded viewing). Thus, "participants" of the communication session 104 can include users and / or client computing devices (e.g., multiple users can be in a room participating in the communication session via use of a single client computing device), each participant can communicate with other participants. As an alternative, the communication session 104 can be hosted by one of the client computing devices 106(1) through 106(N) utilizing peer-to-peer technology. The system 102 can also host chat conversations and other team collaboration functionality (e.g., as part of an application suite).

[0122] In some implementations, such chat conversations and other team collaboration functionality are considered to be external communication sessions that are distinct from the communication session 603. The computing system 602 that collects participant data in the communication session 603 can be able to link to such external communication sessions. Thus, the system can receive information that enables connection to such external communication sessions, such as date, time, session details, etc. In one example, chat conversations can be conducted in accordance with the communication session 603. Additionally, the system 602 can host the communication session 603 that includes at least multiple participants that are co-located in a meeting place, such as a conference room or auditorium, or that are located at different locations.

[0123] In examples described herein, the client computing devices 606(1) through 606(N) participating in the communication session 603 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live content and / or recorded content. The collection of various instances or streams of live content and / or recorded content can be provided by one or more cameras, such as video cameras. For example, an individual stream of live or recorded content can include media data associated with a video feed provided by a video camera (e.g., audio and visual data that captures the appearance and speech of a user participating in the communication session). In some implementations, the video feed can include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images can also include one or more avatars.

[0124] Another example of an individual stream of live or recorded content can include media data that includes an avatar of a user participating in the communication session and audio data that captures the speech of the user. Yet another example of an individual stream of live or recorded content can include media data that includes a file being displayed on a display screen and audio data that captures the speech of the user. Thus, the various streams of live or recorded content within the communication data enable facilitating remote meetings among a group of people and sharing content within the group of people. In some implementations, the various streams of live or recorded content within the communication data can originate from multiple co-located video cameras located in a space, such as a room, to live record or stream the presentation of one or more individuals presenting presented content and one or more individuals consuming the presented content.

[0125] Participants or attendees can view the content of the communication session 603 in real-time as the activity occurs, or alternatively, at a later time after the activity occurs via a recording. In examples described herein, the client computing devices 606(1) through 606(N) participating in the communication session 603 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live content and / or recorded content. For example, an individual stream of content can include media data associated with a video feed (e.g., audio and visual data capturing the appearance and speech of a user participating in the communication session). Another example of an individual stream of content can include media data including an avatar of a user participating in the meeting session and audio data capturing the speech of the user. Yet another example of an individual stream of content can include media data including a content item displayed on a display screen and / or audio data capturing the speech of the user. Thus, the various streams of content within the communication data enable facilitating a meeting or a broadcasted presentation among a group of people dispersed at remote locations.

[0126] A participant or attendee of a communication session is a person within range of a camera or other image and / or audio capturing device such that the actions and / or sounds of the person as the person views and / or listens to content shared via the communication session can be captured (e.g., recorded). For example, a participant can be sitting in an audience viewing shared content at a broadcast location where a stage presentation is occurring. Or, a participant can be sitting in an office conference room viewing shared content of a communication session with other colleagues via a display screen. Still further, a participant can be sitting or standing in front of a personal device (e.g., a tablet, a smartphone, a computer, etc.) viewing shared content of a communication session alone in their office or home.

[0127] FIG. 8 The system 602 of FIG. 6 includes a device 610. The device 610 and / or other components of the system 602 can include distributed computing resources that communicate with each other and / or with the client computing devices 606(1) through 606(N) via one or more networks 608. In some examples, the system 602 can be a standalone system tasked with managing aspects of one or more communication sessions (e.g., the communication session 603). As an example, the system 602 can be managed by an entity such as SLACK, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0128] Network 608 can include, for example, a public network such as the Internet, a private network such as an institutional and / or personal intranet, or some combination of private and public networks. Network 608 can also include any type of wired and / or wireless network, including, but not limited to, a local area network (“LAN”), a wide area network (“WAN”), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communications network (e.g., a 3G, 4G, etc. network), or any combination thereof. Network 608 can utilize communication protocols including packet-based protocols and / or datagram-based protocols, such as Internet Protocol (“IP”), Transmission Control Protocol (“TCP”), User Datagram Protocol (“UDP”), or other types of protocols. Moreover, network 608 can also include a number of devices that facilitate network communications and / or form a hardware basis for a network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, and the like.

[0129] In some examples, network 608 can also include devices capable of connecting to wireless networks, such as wireless access points (“WAPs”). Example support connection through WAPs that transmit and receive data over various electromagnetic frequencies (e.g., radio frequencies), including WAPs that support Institute of Electrical and Electronics Engineers (“IEEE”) 802.11 standards (e.g., 802.11g, 802.11n, 802.11ac, etc.) and other standards.

[0130] In various examples, devices 610 can include one or more computing devices that operate in a cluster or other grouped configuration to share resources, balance load, improve performance, provide failover support or redundancy, or for other purposes. For example, devices 610 can belong to various categories of devices, such as traditional server-type devices, desktop computer-type devices, and / or mobile-type devices. Thus, although illustrated as a single type of device or server-type devices, devices 610 can include multiple types of devices and are not limited to a particular type of device. Devices 610 can represent, but are not limited to, server computers, desktop computers, network server computers, personal computers, mobile computers, laptop computers, tablet computers, or any other type of computing device.

[0131] A client computing device (e.g., one of the client computing devices 606(1) through 606(N), each of which is also referred to herein as a “data processing system”) can belong to a variety of classes of devices, which can be the same as or different from the device 610, such as a traditional client-type device, a desktop computer-type device, a mobile-type device, a special-purpose-type device, an embedded-type device, and / or a wearable-type device. Accordingly, a client computing device can include, but is not limited to, a desktop computer, a game console and / or a gaming device, a tablet computer, a personal data assistant (“PDA”), a mobile phone / tablet hybrid, a laptop computer, a telecommunication device, a computer navigation-type client computing device (e.g., a satellite-based navigation system, including a global positioning system (“GPS”) device), a wearable device, a virtual reality (“VR”) device, an augmented reality (“AR”) device, an implantable computing device, an automobile computer, a network-enabled television, a thin client, a terminal, an Internet of Things (“IoT”) device, a workstation, a media player, a personal video recorder (“PVR”), a set-top box, a camera, an integrated component contained in a computing device (e.g., a peripheral device), an appliance, or any other type of computing device. Moreover, a client computing device can include a combination of examples of the above-listed client computing devices, such as a desktop computer-type device or a mobile-type device in combination with a wearable device, etc.

[0132] The client computing devices 606(1) through 606(N) of various classes and device types can represent any type of computing device having one or more data processing units 692 operatively connected to a computer-readable medium 694, in some cases, the bus 616 can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of a variety of local, peripheral, and / or independent buses.

[0133] The executable instructions stored on the computer-readable medium 694 can include, for example, an operating system 619, a client module 620, a profile module 622, and other modules, programs, or applications that are loadable and executable by the data processing units 692.

[0134] Client computing devices 606(1) to 606(N) may also include one or more interfaces 624 to enable communication between client computing devices 606(1) to 606(N) and other networked devices (e.g., device 610) via network 608. Such network interfaces 624 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. In addition, client computing devices 606(1) to 606(N) may include input / output (“I / O”) interfaces (devices) 626 that support communication with input / output devices, such as user input devices including peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices such as microphones, cameras for acquiring and providing video feeds and / or still images, touch input devices, gesture input devices, etc.) and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, haptic output devices, etc.). FIG. 8 The illustration shows a client computing device 606(1) connected in some way to a display device (e.g., display screen 629(N)) that can display a UI according to the techniques described herein.

[0135] exist FIG. 8 In the example environment 600, client computing devices 606(1) to 606(N) can use their respective client modules 620 to interconnect and / or connect to other external devices to participate in a communication session 603 or to contribute activities to a collaborative environment. For example, a first user can use client computing device 606(1) to communicate with a second user on another client computing device 606(2). When client module 620 is executed, users can share data, which allows client computing device 606(1) to connect to system 602 and / or other client computing devices 606(2) to 606(N) via network 608.

[0136] Client computing devices 606(1) to 606(N) can use their respective profile modules 622 to generate participant profiles (in FIG. 8 (Not shown in the diagram), and provides participant profiles to other client computing devices and / or to one or more devices 610 of system 602. Participant profiles may include one or more of the following: the identity of a user or group of users (e.g., name, unique identifier (“ID”), etc.), user data (such as personal data), machine data (such as location (e.g., IP address, room in a building, etc.)) and technical capabilities, etc. Participant profiles can be used to register participants for communication sessions.

[0137] As in FIG. 8As shown in the middle, the device(s) 610 of the system 602 include a server module 630 and an output module 632. In this example, the server module 630 is configured to receive media streams 634(1) through 634(N) from individual client computing devices, such as client computing devices 606(1) through 606(N). As described above, the media streams can include video feeds (e.g., audio and video data associated with a user), audio data to be output with a presentation of an avatar of the user (e.g., a pure audio experience where video data of the user is not transmitted), text data (e.g., text messages), file data, and / or screen share data (e.g., a document, slides, images, videos, etc. displayed on a display screen), etc. Thus, the server module 630 is configured to receive a collection of various media streams 634(1) through 634(N) (which is referred to herein as "media data 634") during a live viewing of the communication session 603. In some scenarios, not all client computing devices participating in the communication session 603 provide a media stream. For example, a client computing device can be a consumption or "listening" only device, such that it only receives content associated with the communication session 603, but does not provide any content to the communication session 603.

[0138] In various examples, the server module 630 can select aspects of the media streams 634 to share with individual ones of the participating client computing devices 606(1) through 606(N). Thus, the server module 630 can be configured to generate session data 636 based on the streams 634 and / or pass the session data 636 to the output module 632. The output module 632 can then communicate communication data 639 to client computing devices (e.g., client computing devices 606(1) through 606(3) participating in a live viewing of the communication session). The communication data 639 can include video, audio, and / or other content data provided by the output module 632 based on content 650 associated with the output module 632 and based on the received session data 636. The content 650 can include the streams 634 or other shared data, such as image files, spreadsheet files, slides, documents, etc. The streams 634 can include video components that depict images captured by the I / O devices 626 on each client computing machine.

[0139] As shown, the output module 632 transmits communication data 639(1) to the client computing device 606(1), and transmits communication data 639(2) to the client computing device 606(2), and transmits communication data 639(3) to the client computing device 606(3), etc. The communication data 639 transmitted to the client computing devices can be the same or different (e.g., the positioning of the content streams within the user interface can differ from one device to the next).

[0140] In various implementations, the device(s) 610 and / or the client module 620 can include a GUI presentation module 640. The GUI presentation module 640 can be configured to analyze communication data 639 for delivery to one or more client computing devices 606. In particular, the UI presentation module 640 at the device(s) 610 and / or the client computing devices 606 can analyze the communication data 639 to determine an appropriate way to display video, images, and / or content on a display screen of the associated client computing device 606. In some implementations, the GUI presentation module 640 can provide video, images, and / or content to a presentation GUI 646 rendered on a display screen 629 of the associated client computing device 606. The GUI presentation module 640 can cause the presentation GUI 646 to be rendered on the display screen 629. The presentation GUI 646 can include video, images, and / or content analyzed by the GUI presentation module 640.

[0141] In some implementations, the presentation GUI 646 can include multiple portions or grids that can render or include video, images, and / or content for display on the display screen 629. For example, a first portion of the presentation GUI 646 can include a video feed of a presenter or individual, and a second portion of the presentation GUI 646 can include a video feed of an individual consuming meeting information provided by the presenter or individual. The GUI presentation module 640 can populate the first and second portions of the presentation GUI 646 in a manner that appropriately mimics the environmental experience that the presenter and individual can be sharing.

[0142] In some implementations, the GUI presentation module 640 can zoom in or provide a zoomed view of an individual represented by a video feed in order to highlight a reaction of the individual to the presenter, such as facial features. In some implementations, the presentation GUI 646 can include video feeds of multiple participants associated with a meeting, such as a general communication session. In other implementations, the presentation GUI 646 can be associated with a channel, such as a chat channel, an enterprise team channel, or the like. Thus, the presentation GUI 646 can be associated with an external communication session that is different from a general communication session.

[0143] FIG. 9 FIG. 7 illustrates a diagram showing example components of an example device 700 (also referred to herein as a “computing device”) configured to generate data for some of the user interfaces disclosed herein. The device 700 can generate data including one or more portions that can render or include video, images, virtual objects, and / or content for display on a display screen 629. The device 700 can be representative of one of the devices described herein. Additionally or alternatively, the device 700 can be representative of one of the client computing devices 606.

[0144] As illustrated, the device 700 includes one or more data processing units 702, computer-readable media 704, and a communication interface 706. The components of the device 700 are operatively connected, e.g., via a bus 709, which can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any number of local, peripheral, and / or independent buses.

[0145] As used herein, a data processing unit (e.g., data processing unit 702 and / or data processing unit 692) can represent, e.g., a CPU-type data processing unit, a GPU-type data processing unit, a field-programmable gate array (“FPGA”), another class of DSP, or other hardware logic components that in some cases can be driven by a CPU. For example, and without limitation, illustrative types of hardware logic components that can be utilized include application-specific integrated circuits (“ASICs”), application-specific standard products (“ASSPs”), system-on-a-chip systems (“SOCs”), complex programmable logic devices (“CPLDs”), and so forth.

[0146] As used herein, a computer-readable medium (e.g., computer-readable medium 704 and computer-readable medium 694) can store instructions executable by a data processing unit. The computer-readable medium can also store instructions executable by an external data processing unit (e.g., by an external CPU, an external GPU) and / or by an external accelerator (e.g., an FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated in a computing device, while in some examples one or more of the CPUs, GPUs, and / or accelerators is external to the computing device.

[0147] Computer-readable media (which can also be referred to herein as computer- readable media) can include computer-storage media and / or communication media. Computer-storage media can include one or more of volatile memory, non-volatile memory, and / or other persistent and / or auxiliary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Thus, computer-storage media includes tangible and / or physical forms of media included in a device and / or hardware component that is part of the device or external to the device, including but not limited to random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), phase change memory (“PCM”), read-only memory (“ROM”), erasable programmable read-only memory (“EPROM”), electrically erasable programmable read-only memory (“EEPROM”), flash memory, compact disc read-only memory (“CD-ROM”), digital versatile discs (“DVDs”), optical storage, magnetic storage, fixed or removable media, solid-state storage, storage arrays, network-attached storage, storage area networks, hosted computer storage devices, or any other storage memory, storage devices, and / or storage media that can be used for storage and maintenance of information for computer devices.

[0148] Communication media can embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism. As defined herein, computer-storage media does not include communication media. That is, computer-storage media does not include communications media solely in the form of modulated data signals, carrier waves, or propagated signals per se.

[0149] The communication interface 706 can represent, for example, a network interface controller (“NIC”) or other type of transceiver device to transmit and receive communications over a network. Additionally, the communication interface 706 can include one or more video cameras and / or audio devices 722 to enable generation of video feeds and / or still images, among others.

[0150] In the illustrated example, computer-readable media 704 includes data storage 708. In some examples, data storage 708 includes a data storage device, such as a database, data warehouse, or other type of structured or unstructured data storage device. In some examples, data storage 708 includes a corpus and / or a relational database with one or more tables, indexes, stored procedures, and the like to enable data access, including, for example, one or more HyperText Markup Language (“HTML”) tables, Resource Description Framework (“RDF”) tables, Web Ontology Language (“OWL”) tables, and / or Extensible Markup Language (“XML”) tables.

[0151] Data storage 708 can store data for processes, applications, components, and / or modules stored in computer-readable media 704 and / or executed by data processing units 702 and / or accelerators. For example, in some examples, data storage 708 can store session data 710 (e.g., as illustrated session data 636), profile data 712 (e.g., associated with participant profiles), and / or other data. Session data 710 can include a total number of participants (e.g., users and / or client computing devices) in a communication session, activities that occur in a communication session, an invitee list for a communication session, and / or other data related to when and how a communication session is conducted or hosted. Data storage 708 can also include content data 714, such as content including video, audio, or other content for rendering and display on one or more display screens 629. FIG. 8

[0152] Alternatively, some or all of the above-described data can be stored on separate memory 716 on one or more data processing units 702, such as memory on CPU-type processors, GPU-type processors, FPGA-type accelerators, DSP-type accelerators, and / or other accelerators. In this example, computer-readable media 704 also includes an operating system 718 and an application programming interface 710 (API) configured to expose functionality and data of device 700 to other devices. Further, computer-readable media 704 includes one or more modules, such as server module 730, output module 732, and GUI presentation module 740, although the number of illustrated modules is an example only, and the number can become higher or lower. That is, the functionality described herein in connection with the illustrated modules can be performed by a fewer number of modules on one device or a greater number of modules spread across multiple devices.

[0153] The disclosure presented herein also encompasses the subject matter set forth in the following clauses.

[0154] ​Clause A: A method performed by a computing system 110, the method comprising: receiving a plurality of streams 111, individual streams of the plurality of streams including video components; analyzing the plurality of streams 111 to identify a selected stream 111A having at least one video component depicting at least a threshold 118 number of individuals 119A; selecting a first aspect ratio 131 from a plurality of preset aspect ratios, reserving the first aspect ratio for the selected stream 111A having one or more video components depicting at least the threshold 118 number of individuals 119A; selecting a second aspect ratio 132 from the plurality of preset aspect ratios, reserving the second aspect ratio for other streams having one or more video components depicting less than a threshold 118 number of individuals 119B, wherein the first aspect ratio 131 is greater than the second aspect ratio 132; causing communication session data 113 and the plurality of streams 111 to be transmitted to a plurality of client computing devices 101, the communication session data 113 causing the plurality of client computing devices 101 to display a user interface 120 including individual renderings 111A' of a first set of the selected stream 111A having video components depicting at least the threshold 118 number of individuals 119A, the individual renderings of the first set displayed using the first aspect ratio selected with the plurality of depicted individuals, and individual renderings 111B'- 111E' of a second set of the other streams 111B- 111E having the one or more video components depicting less than the threshold 118 number of individuals 119B- 119E, the individual renderings of the second set having a second aspect ratio 132 that is less than the first aspect ratio 131.

[0155] Clause B: The method of clause A, further comprising: analyzing the other streams 111B- 111E having the one or more video components depicting less than the threshold 118 number of individuals 119B- 119E to determine whether one or more video components of the other streams 111B- 111E begin to depict the threshold number of individuals; and responsive to determining whether one or more video components of the other streams 111B- 111E begin to depict the threshold number of individuals, modifying the communication session data 113 to cause the user interface 120 to transition display of at least one of the other streams that begins to depict the threshold number of individuals from the second aspect ratio to the first aspect ratio.

[0156] Clause C: The method of clauses A and B, further comprising: analyzing the selected stream 111E having video components depicting at least the threshold 118 number of individuals 119E to determine whether one or more video components of the selected stream 111E begin to depict fewer than the threshold number of individuals; and responsive to determining whether one or more video components of the selected stream 111E begin to depict fewer than the threshold number of individuals, modifying the communication session data 113 to cause the user interface 120 to transition display of the selected stream 111E that begins to depict fewer than the threshold number of individuals from the first aspect ratio to the second aspect ratio.

[0157] Clause D: The method of clauses A-C, further comprising: analyzing the plurality of streams to determine a number of people depicted in the video components of each stream; and configuring the user interface to order individual renderings of each stream based on the number of people depicted in the video components of each stream.

[0158] Clause E: The method of clauses A-D, wherein a target aspect ratio is selected based on one or more dimensions of a display device in communication with the computing system, wherein the first aspect ratio is greater than the target aspect ratio and the second aspect ratio is less than the target aspect ratio.

[0159] Clause F: The method of clauses A-E, wherein the first aspect ratio is greater than a first target aspect ratio and the second aspect ratio is less than a second target aspect ratio.

[0160] Clause G: The method of clauses A-F, wherein the first target aspect ratio and the second target aspect ratio are selected based on one or more dimensions of a display device in communication with the computing system.

[0161] Clause H: The method of clauses A-G, further comprising: receiving user input to adjust the first aspect ratio; generating usage data defining an adjusted target aspect ratio based on the input used to adjust the first aspect ratio, the adjusted target aspect ratio being greater than the target aspect ratio if the user input increased the first aspect ratio; and storing the adjusted target aspect ratio such that subsequent executions of the method set the first aspect ratio to a value that is greater than or equal to the adjusted target aspect ratio.

[0162] Clause I: The method of clauses A-H, further comprising: receiving user input to adjust the second aspect ratio; generating usage data defining an adjusted target aspect ratio based on the input to adjust the second aspect ratio, the adjusted target aspect ratio being less than the target aspect ratio if the user input decreases the second aspect ratio; and storing the adjusted target aspect ratio such that subsequent execution of the method sets the second aspect ratio to a value less than or equal to the adjusted target aspect ratio.

[0163] Clause J: A system 101 comprising: one or more processing units 692; and a computer- readable medium 694 having encoded thereon computer-executable instructions for causing the one or more processing units 692 to: analyze a plurality of streams 111 to identify a selected stream 111 A having at least one video component depicting at least a threshold 118 number of individuals 119A; select a first aspect ratio 131 from a plurality of preset aspect ratios, the first aspect ratio being reserved for the selected stream 111 A having one or more video components depicting at least the threshold 118 number of individuals 119A; select a second aspect ratio 132 from the plurality of preset aspect ratios, the second aspect ratio being reserved for other streams having one or more video components depicting less than the threshold 118 number of individuals 119B, wherein the first aspect ratio 131 is greater than the second aspect ratio 132; cause display of a user interface 120, the user interface 120 including a first set of individual renderings 111 A' of the selected stream 111 A having video components depicting at least the threshold 118 number of individuals 119A, the first set of individual renderings being displayed using the first aspect ratio selected with the plurality of depicted individuals, and a second set of individual renderings 111B'- 111E' of the other streams 111B- 111E having the one or more video components depicting less than the threshold 118 number of individuals 119B- 119E, the second set of individual renderings having the second aspect ratio 132 less than the first aspect ratio 131.

[0164] Clause K: The system of clause J, wherein the instructions further cause the one or more processing units to: analyze the other streams 111B- 111E having the one or more video components depicting less than the threshold 118 number of individuals 119B- 119E to determine whether one or more video components of the other streams 111B- 111E begin to depict the threshold number of individuals; and responsive to determining whether one or more video components of the other streams 111B- 111E begin to depict the threshold number of individuals, modify the user interface 120 to display at least one of the other streams that begins to depict the threshold number of individuals using the first aspect ratio.

[0165] Clause L: The system of clauses J and K, wherein the instructions further cause the one or more processing units to: analyze the selected stream 111 E having video components depicting at least the threshold 118 number of individuals 119E to determine whether one or more video components of the selected stream 111 E begin to depict less than the threshold number of individuals; and in response to determining whether one or more video components of the selected stream 111 E begin to depict less than the threshold number of individuals, modify the communication session data 113 to cause the user interface 120 to display the selected stream 111 E that begins to depict less than the threshold number of individuals using the second aspect ratio.

[0166] Clause M: The system of clauses J through L, wherein the instructions further cause the one or more processing units to: analyze the plurality of streams to determine a number of people depicted in the video components of each stream; and configure the user interface to order individual renderings of each stream based on the number of people depicted in the video components of each stream.

[0167] Clause N: The system of clauses J through M, wherein a target aspect ratio is selected based on one or more dimensions of a display device in communication with the computing system, wherein the first aspect ratio is greater than the target aspect ratio and the second aspect ratio is less than the target aspect ratio.

[0168] Clause O: A system 110 comprising: means for receiving a plurality of streams 111, individual streams of the plurality of streams including video components; means for analyzing the plurality of streams 111 to identify a selected stream 111 A having at least one video component that depicts at least a threshold 118 number of individuals 119A; means for selecting a first aspect ratio 131 from a plurality of preset aspect ratios, the first aspect ratio being reserved for the selected stream 111 A having one or more video components that depict at least the threshold 118 number of individuals 119A; means for selecting a second aspect ratio 132 from the plurality of preset aspect ratios, the second aspect ratio being reserved for other streams having one or more video components that depict less than the threshold 118 number of individuals 119B, wherein the first aspect ratio 131 is greater than the second aspect ratio 132; means for causing a display of a user interface 120, the user interface 120 having individual renderings 111 A' of a first set of the selected stream 111 A displayed using the first aspect ratio and individual renderings 111 B' - 111 E' of a second set of the other streams 111 B - 111 E displayed using a second aspect ratio 132 that is less than the first aspect ratio 131.

[0169] Clause P: The method of clause O, further comprising: means for analyzing the other streams 111B-111E having the one or more video components depicting fewer than a threshold 118 number of individuals 119B-119E to determine whether one or more video components of the other streams 111B-111E begin to depict the threshold number of individuals; and means for modifying the communication session data 113 to cause the user interface 120 to transition the display of at least one of the other streams from the second aspect ratio to the first aspect ratio, the transition responsive to determining whether one or more video components of the other streams 111B-111E begin to depict the threshold number of individuals.

[0170] Clause Q: The system of clauses O and P, further comprising: means for analyzing the selected stream 111E having video components depicting at least the threshold 118 number of individuals 119E to determine whether one or more video components of the selected stream 111E begin to depict fewer than the threshold number of individuals; and means for modifying the communication session data 113 to cause the user interface 120 to transition display of the selected stream 111E from the first aspect ratio to use the second aspect ratio, wherein the transition occurs responsive to determining whether one or more video components of the selected stream 111E begin to depict fewer than the threshold number of individuals

[0171] Clause R: The system of clauses O through Q, further comprising: means for analyzing the plurality of streams to determine a number of people depicted in the video components of each stream; and means for configuring the user interface to order individual renderings of each stream based on the number of people depicted in the video components of each stream.

[0172] Clause S: The system of clauses O through R, wherein a target aspect ratio is selected based on one or more dimensions of a display device in communication with the computing system, wherein the first aspect ratio is greater than the target aspect ratio and the second aspect ratio is less than the target aspect ratio.

[0173] Clause T: The system of clauses O through S, wherein the first aspect ratio is greater than a first target aspect ratio and the second aspect ratio is less than a second target aspect ratio.

[0174] It should also be appreciated that numerous modifications and changes can be made to the examples described above, and that elements of one or more examples can be understood to be in other acceptable examples. All such modifications and changes are intended to be included within the scope of this disclosure and are to be construed as falling within the ambit of the following claims.

[0175] Finally, while various configurations have been described with specific reference to structural features and / or methodological acts, it will be understood that the subject matter defined in the appended claims is not necessarily limited to just those described specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method performed by a computing system, the method comprising: Receive multiple streams, wherein each of the multiple streams includes video components; Receive configuration data that defines a target aspect ratio condition based on a target aspect ratio, wherein if the aspect ratio of the available display area is less than the target aspect ratio, then the aspect ratio of the available display area satisfies the target aspect ratio condition, and the aspect ratio is a fraction of width / height; Configure the user interface data that defines the user interface using the following steps: If the aspect ratio of the available display area does not meet the target aspect ratio condition, the available display area is vertically divided to create a first vertical block and a second vertical block, or If the aspect ratio of the available display area meets the target aspect ratio condition, the available display area is horizontally divided to create a first horizontal block and a second horizontal block; The vertical or horizontal blocks are recursively divided to insert a number of units equal to the number of flows, wherein individual units are arranged within the vertical or horizontal blocks based on the target aspect ratio condition and at least one of the aspect ratio of the individual units within the vertical block or the aspect ratio of the individual units within the horizontal block; and This enables the display of the user interface, which has a plurality of units arranged according to at least one of the target aspect ratio condition and the aspect ratio of individual units within the vertical block or the aspect ratio of individual units within the horizontal block, wherein each individual unit in the plurality of units includes the rendering of an individual stream in the plurality of streams, wherein the aspect ratio of each unit is based on the number of divisions of the horizontal block or the vertical block.

2. The method according to claim 1, wherein, If the aspect ratio of the available display area is greater than the target aspect ratio, then the aspect ratio of the available display area does not meet the target aspect ratio condition.

3. The method according to claim 1, wherein, If the aspect ratio of the available display area is greater than or equal to the target aspect ratio, then the aspect ratio of the available display area does not meet the target aspect ratio condition.

4. The method according to claim 1, wherein, Recursively dividing the vertical blocks or the horizontal blocks includes: It is determined that the vertical blocks are evenly filled with cells; and In response to determining that the vertical block is equally filled with cells, and if the aspect ratio of the individual cells present in the selected vertical block is less than the target aspect ratio, a new cell is added to the selected vertical block so that the selected vertical block is horizontally divided.

5. The method according to claim 4, further comprising: Determine whether the first vertical block and the second vertical block are equally filled with cells; as well as In response to determining that the first vertical block and the second vertical block are not equally filled with cells, a new cell is added to the vertical block with the fewest cells.

6. The method according to claim 5, further comprising: It is determined that the vertical blocks are evenly filled with cells; as well as In response to determining that the vertical block is equally filled with cells, and if the aspect ratio of the individual cells present in the selected vertical block is greater than the target aspect ratio, data defining the transformation from the vertical block to a new horizontal block is generated, and the new cells are added to one of the new horizontal blocks.

7. The method according to claim 1, wherein, Recursively dividing the vertical blocks or the horizontal blocks includes: It is determined that the first horizontal block and the second horizontal block are equally filled with cells; and In response to determining that the first horizontal block and the second horizontal block are equally filled with cells, and if the aspect ratio of the individual cells present in the selected horizontal block is greater than the target aspect ratio, a new cell is added to the selected horizontal block so that the selected horizontal block is vertically divided.

8. The method according to claim 7, further comprising: Determine whether the first horizontal block and the second horizontal block are equally filled with cells; as well as In response to determining that the first horizontal block and the second horizontal block are not equally filled with cells, a new cell is added to the horizontal block with the minimum number of cells.

9. The method according to claim 8, further comprising: Determine whether the first horizontal block and the second horizontal block are equally filled with cells; as well as In response to determining that the first horizontal block and the second horizontal block are equally filled with cells, and if the aspect ratio of the individual cells present in the selected horizontal block is less than the target aspect ratio, data defining the transformation from the horizontal block to a new vertical block is generated, and new cells are added to one of the new vertical blocks.

10. A computer-readable storage medium including instructions configured to cause the computing system to perform a method when run on a computing system, the method comprising: Receive multiple streams, wherein each of the multiple streams includes video components; Receive configuration data that defines a target aspect ratio condition based on a target aspect ratio, wherein if the aspect ratio of the available display area is less than the target aspect ratio, then the aspect ratio of the available display area satisfies the target aspect ratio condition, and the aspect ratio is a fraction of width / height; Configure the user interface data that defines the user interface using the following steps: If the aspect ratio of the available display area is greater than the target aspect ratio, then the available display area is vertically divided to generate a first vertical block and a second vertical block within the available display area, or If the aspect ratio of the display area is less than the target aspect ratio, the available display area is horizontally divided to generate a first horizontal block and a second horizontal block within the available display area; Utilizing individual units to configure the vertical or horizontal blocks; and The user interface is displayed based on a comparison between the target aspect ratio and at least one of the aspect ratios of individual units within the vertical block or the individual units within the horizontal block. The user interface has the units arranged within the vertical block or the horizontal block, wherein each individual unit includes rendering of an individual stream among the plurality of streams.

11. The computer-readable storage medium according to claim 10, wherein, The method, in which the instructions are configured to cause the computing system to perform, further includes: It is determined that the first vertical block and the second vertical block are equally filled with cells; and The first vertical block and the second vertical block are determined to be equally filled with cells, and if the aspect ratio of an individual cell existing in the selected vertical block is less than the target aspect ratio, a new cell is added to the selected vertical block so that the selected vertical block is horizontally divided.

12. The computer-readable storage medium according to claim 11, wherein, The method, in which the instructions are configured to cause the computing system to perform, further includes: Determine whether the first vertical block and the second vertical block are equally filled with cells; and In response to determining that the first vertical block and the second vertical block are not equally filled with cells, a new cell is added to the vertical block with the fewest cells.

13. The computer-readable storage medium according to claim 10, wherein, The method, in which the instructions are configured to cause the computing system to perform, further includes: Determine whether the first vertical block and the second vertical block are equally filled with cells; and In response to determining that the first vertical block and the second vertical block are equally filled with cells, and if the aspect ratio of the individual cells present in the selected vertical block is greater than the target aspect ratio, data defining the transformation from the vertical block to a new horizontal block is generated, and new cells are added to one of the new horizontal blocks to vertically divide the selected horizontal block in the new horizontal block.

14. The computer-readable storage medium according to claim 10, wherein, Recursively dividing the vertical blocks or the horizontal blocks includes: It is determined that the first horizontal block and the second horizontal block are equally filled with cells; and In response to determining that the first horizontal block and the second horizontal block are equally filled with cells, and if the aspect ratio of the individual cells present in the selected horizontal block is greater than the target aspect ratio, a new cell is added to the selected horizontal block so that the selected horizontal block is vertically divided.

Images