Automation of drill sessions of communication sessions

Abstract

The techniques disclosed herein provide a set of tools that help event organizers better prepare important events by facilitating characteristics of drilling sessions. In some configurations, the system can create a drill session during establishment of a conference. Thus, if a conference organizer is establishing a company-full conference, the tool provides options that allow an event organizer to establish multiple drill sessions effortlessly during an event scheduling flow. The tool enables a system to perform a test that simulates an event attendee. This allows an organizer to test specific user activities, including the experience of each user when joining a conference or perform common attendee behaviors, such as virtual hand raising or reactions. The system also provides an intelligent technical check that automatically tests the device settings of each presenter and displays real-time telemetry data to efficiently communicate any performance issues to a conference organizer.

Description

Background Art

[0001] There are many different types of collaboration systems that allow users to communicate. For example, some systems allow people to collaborate by sharing content such as video and audio streams, shared files, chat messages, etc. Some systems provide user interface formats that allow users to share content with an audience. Such systems can provide a specific set of permissions that allow users to assume specific roles (such as presenters, audience members, etc.).

[0002] Although some collaboration systems can provide a platform for multiple users to use a specific set of permissions to assume specific roles and share live video and audio streams, such systems still have multiple deficiencies. For example, when a communication session involves a large number of people in different parts of the world, a large number of technical settings are required to optimize the meeting. These settings can include, but are not limited to: network settings, audio settings, video settings, user access to user permissions, etc. The process of optimizing these technical settings can be a complex and cumbersome process.

[0003] To test the settings of a meeting, some existing systems require the meeting organizer to spend time manually setting up another meeting with the same attendees. This requires many people to be available to test network connections, audio settings, etc. For example, if an upcoming meeting involves people in three countries, the meeting organizer will have to set up a meeting including people from each country to properly test the connection. If connectivity issues occur, the team must go through a complex task flow to identify the problem and ultimately develop solutions that may be required to solve the identified problem. Existing systems do not provide telemetry data, and therefore may require special tools and professional technicians to thoroughly test the system.

[0004] Even if solutions can be developed based on manually configured test meetings, the meeting organizer still needs to record how the problem was identified and solved, and manually apply any learned solutions to the original meeting. Given the need for a large amount of manual input, this manual process can lead to many errors, which can result in unintentional input and other problems. If permissions are not configured correctly, these errors can also lead to security issues. Summary of the Invention

[0005] The technology disclosed herein provides a set of tools that help event organizers better prepare for important events by facilitating the features of a rehearsal session. In some configurations, the system is able to create a rehearsal session during the process of establishing a meeting. Thus, if a meeting organizer is establishing an all-company meeting, the tools provide options that allow the event organizer to effortlessly establish multiple rehearsal sessions in the event scheduling process. The tools enable the system to perform tests that simulate event attendees. This allows organizers to test specific user activities, including the experience of each user when joining the meeting or performing common attendee behaviors such as virtual raising of hands or reactions. Attendee simulation can also test other attendee experiences related to the delivery of messages, audio streams, and video streams. The system also provides intelligent technical checks that automatically test the device settings of each presenter. The system also generates real-time telemetry data to effectively convey any performance issues to the meeting organizer. This enables the system and the meeting organizer to ensure that each device is properly configured. If a problem is detected, the system also provides automatic alerts and actionable suggestions.

[0006] The system disclosed herein is also able to generate summary data to effectively record all changes made to the system configuration settings during a rehearsal session. This allows the meeting organizer to store the changes made to the system configuration settings and easily reapply them to the actual event. The system disclosed herein also uses visualizations located in a display with predefined metrics and thresholds to provide real-time telemetry data, making it easy for organizers to identify problems. When the telemetry data indicates one or more problems, the system is also able to generate suggestions on steps that end users can take to resolve these problems. These suggestions can be based on an artificial intelligence (AI) model from past observations. The settings can also be automatically integrated back into the settings of the original meeting.

[0007] By reading the following detailed description and review of the associated drawings, features and technical benefits other than those explicitly described above will be apparent. The present invention content is provided to introduce a selected concept in a simplified form, which will be further described in the following detailed implementation. The present invention content is not intended to identify the key or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The term "technology" can refer, for example, to one or more systems, one or more methods, computer-readable instructions, one or more modules, algorithms, hardware logic, and / or one or more operations permitted by the context described above and throughout the document. Brief Description of the Drawings

[0008] The specific embodiments are described with reference to the accompanying drawings. In the drawings, the leftmost digit of the reference numeral identifies the drawing in which the reference numeral first appears. The same reference numerals in different drawings indicate similar or identical items, and references to individual items among multiple items may use reference numerals with a sequence of letters to refer to each individual item. General references to an item may use a specific reference numeral without a sequence of letters.

[0009] Figure 1 A user interface configured to establish settings for an original meeting is shown.

[0010] Figure 2 A user interface configured to establish settings for a rehearsal meeting (e.g., a rehearsal session or a test session) is shown.

[0011] Figure 3 A user interface configured to establish the time for a rehearsal meeting is shown.

[0012] Figure 4 A user interface configured to add an additional rehearsal meeting is shown.

[0013] Figure 5 A user interface of a meeting with collaborators shown to those in a rehearsal meeting is shown.

[0014] Figure 6 A user interface with a text indicator indicating that the meeting is a rehearsal meeting is shown.

[0015] Figure 7 A user interface shown during a rehearsal meeting, in which a technical check is performed for the organizer, is shown.

[0016] Figure 8 A user interface shown during a rehearsal meeting, in which the organizer's settings can be accessed, is shown.

[0017] Figure 9 A user interface shown during a rehearsal meeting, in which the organizer's settings are displayed, is shown.

[0018] Figure 10 A user interface shown during a rehearsal meeting, in which the organizer can move to a stage for testing the presenter's settings and hardware, is shown.

[0019] Figure 11 A user interface shown during a rehearsal meeting, in which the presenter's telemetry data can be accessed, is shown.

[0020] Figure 12 A user interface shown during a rehearsal meeting, in which the presenter's telemetry data is displayed, is shown.

[0021] Figure 13 Shows the user interface displayed during a rehearsal session, where the presenter's telemetry data is displayed and buttons are shown for accessing the solution.

[0022] Figure 14 Shows the user interface displayed during a rehearsal session, where the solution is displayed.

[0023] Figure 15 Shows the user interface displayed during a rehearsal session, where the presenter's telemetry data is displayed and buttons are shown for accessing the meeting details.

[0024] Figure 16 Shows the user interface displayed during a rehearsal session, where the meeting details are displayed.

[0025] Figure 17 Shows the user interface displayed during a rehearsal session, where the telemetry data for the rehearsal stage room phase is displayed.

[0026] Figure 18 Shows the user interface displayed during a rehearsal session, where buttons are shown for starting a live rehearsal.

[0027] Figure 19 Shows the user interface displayed during a rehearsal session, where the device is capable of receiving an input indicating the number of robots for simulating attendees.

[0028] Figure 20 Shows a map of a geographical location, where computing resources are allocated for testing the system using robots with simulated attendees in the geographical location.

[0029] Figure 21 Shows the user interface displayed during a rehearsal, where the event assistant is shown.

[0030] Figure 22 Shows the user interface displayed during a rehearsal, where the display shows buttons for activating a live rehearsal.

[0031] Figure 23 Shows the user interface displayed during a live rehearsal, where the display shows buttons for activating the attendee view.

[0032] Figure 24 Shows the user interface displayed during a live rehearsal, where the display shows the attendee view.

[0033] Figure 25 Shows the user interface displayed during a live rehearsal, where the display shows the event assistant used for simulating attendee activities.

[0034] Figure 26 Shows a user interface displayed during a live rehearsal, where the display shows the results of inputs to an event assistant for simulating attendee activities.

[0035] Figure 27 Shows a user interface displayed during a live rehearsal, where the display shows a button for stopping the simulation.

[0036] Figure 28 Shows a user interface displayed after a live rehearsal, where the display includes a summary of the rehearsal.

[0037] Figure 29 Shows a user interface displayed after a live rehearsal, where the display includes a report of the rehearsal.

[0038] Figure 30 Is a flowchart illustrating aspects of a sample routine for implementing the techniques disclosed herein.

[0039] Figure 31 Is a computing system diagram showing aspects of an illustrative operating environment for the techniques disclosed herein.

[0040] Figure 32 Is a computing architecture diagram showing aspects of the configuration and operation of a computing device capable of implementing aspects of the techniques disclosed herein. Detailed Description

[0041] The techniques disclosed herein provide a set of tools that help event organizers better prepare for important events by facilitating features of rehearsal sessions. If a meeting organizer wants to prepare for a meeting (which can include large conferences), the system provides features that help the organizers and presenters of the meeting rehearse or pre - rehearse. The system introduces features that automate many technical checks and measurements to help organizers and presenters identify problems during the rehearsal session. The tests can include, but are not limited to: tests on audio signals, video signals, network settings, etc. The tests can also provide a preview of the user interface layout from the perspective of audience members. Then, the system can generate specific settings or recommendations for a specific setup based on the technical checks and measurements. The system can also manage the generated settings and recommendations for other setups so that meeting organizers and presenters can use those generated settings and recommendations as a reference. The system can also automatically integrate the generated settings and recommendations from the rehearsal session into one or more meetings.

[0042] A rehearsal session can be generated based on the data object that defines the original meeting, or a rehearsal session can be generated based on the process flow used to configure the original meeting. The tool for establishing and managing the rehearsal session enables the organizers and presenters of the meeting to conduct a "rehearsal" of the meeting. The rehearsal session can include a subset of the people invited to the original meeting. The subset of people can be based on the role of each invitee. For example, if the original meeting includes an organizer, a presenter, and audience members, the system can generate a rehearsal session that includes only the organizer and the presenter. The system can also provide hardware and software that simulate audience members, so that the organizers and presenters can obtain a preview of what the attendees will see and hear during the meeting. As described in more detail below, the rehearsal session can conduct multiple tests on the subset of people by using different test measurement tools, which eliminates the need for a wide audience and their corresponding devices involved in the test session.

[0043] Reference Figures 1-5 , aspects of the process flow for establishing a rehearsal session are shown and described. In this example, a rehearsal session is generated based on the process flow used to configure the original meeting. When the organizer schedules the original meeting, the system prompts the organizer (e.g., the first user 10A) to see if the organizer also wants to schedule the rehearsal session. Figure 1 Shows a first user interface 101A configured to be displayed to the organizer (e.g., user 10A). This first user interface 101A allows the organizer to list multiple individuals who will be invited to the meeting. The user interface also allows the organizer to assign a role to each invitee among the invitees. For example, Bryan is assigned the co-organizer role; Charlotte, Krystal, and Ray are assigned the presenter role, and others under the company alias are assigned the audience role. The user interface is also configured to receive the date and time of the original meeting. The user interface is also configured to receive a set of options that the organizer can view by selecting the "Options" menu.

[0044] Figure 2 Shows a second user interface 101B configured to display different menu items to allow the organizer to configure meeting options. In this example, the second user interface 101B allows the organizer to select a menu item for "Event Collaboration". The menu allows the organizer to establish a new rehearsal session. As in Figure 3As shown in the UI format 101C, in response to the selection of the new rehearsal session button, the system can provide multiple options for the time and date of the rehearsal session. Some default dates can be automatically displayed, which will cause the rehearsal session to start before the original meeting date. The system can enable the user to set more than one rehearsal session. The user can select a menu for adding as many rehearsal sessions as needed. As in Figure 4 As shown in the UI format 101D, the system can allow the user to set the time for each rehearsal session in the rehearsal session. Once those times and dates are selected for the rehearsal session, for example, by selecting the "Apply" button, the system can generate a rehearsal data object that stores various aspects of the rehearsal session.

[0045] The rehearsal session can include a subset of the users included in the original meeting. In some configurations, people with a predetermined role can be automatically included in the rehearsal session. For example, the system can import the identifiers of the organizer and the presenter from the data defining the original meeting into the rehearsal session. The system can also grant management permissions to the organizer and the presenter to allow the organizer and the presenter to perform tests during the rehearsal session. Additionally, the system can also grant management permissions to the organizer and the presenter to write or approve the meeting settings for the rehearsal session and for the original meeting. This includes meeting settings that control the tests that can be performed, the invocation of robots at a specified location, and the control permissions for other invitees. These are permissions that were not granted to the organizer and the presenter before the generation of the rehearsal session. These management permissions can also be configured to be automatically revoked after the rehearsal session or after the original meeting.

[0046] The system can also copy other options from the data object defining the original meeting to the rehearsal session. For example, if the audience members have permission to receive audio and video streams and have permission to chat, then those meeting options will be copied from the data object defining the original meeting to the rehearsal session. For the rehearsal session, the permissions for the audience members can be applied to the robots used to test the network configuration.

[0047] Other options can also be copied from the data object defining the original meeting to the rehearsal session, and the other options are, for example, permissions and settings such as audio settings and video settings. If the organizer has established specific options or permissions for specific roles in the original meeting, then those permissions are copied to the rehearsal session. For example, if the organizer sets all audience members to mute, but has exceptions for three specific members, then such permissions can be copied from the original meeting settings to the data object defining the rehearsal session.

[0048] It is also possible to copy other meeting options for controlling visualization, such as managed mode options, from the original meeting settings to the data object that defines the rehearsal session. For example, if the organizer sets options that only allow audience members to see a specific video stream or a specific view of a view of the content, those options can also be copied from the original meeting settings to the data object that defines the rehearsal session. Other options for other functions can also be copied from the original meeting settings to the data object that defines the rehearsal session, which can include data objects or plugins for question and answer (Q&A) sessions, polls, etc. If such functions are included in the original meeting, based on the data that defines the original meeting, the function is copied in or linked to the rehearsal session. This automatic copying of parameters from the original meeting to the rehearsal session also includes the copying of shared files. For example, if the presenter has a specific set of slides in a specific format, that set of slides is copied from the original meeting to the data object that defines the rehearsal session. This allows the rehearsal session to test the actual files that will be shared in the original meeting. This allows the system to identify issues with the files, especially if it presents formatting issues or if it has other problems, such as corrupted or incompatible data. This automatic copying of parameters and files from the original meeting to the rehearsal session eliminates the need for the user to manually set up a new meeting to test the parameters of the original meeting.

[0049] Once the parameters of the rehearsal session are generated, the rehearsal object is sent to the attendees of the rehearsal session. In this example, the rehearsal session is sent to the calendars of the designated invitees of the rehearsal session, and in this example, the designated invitees are the organizer and the presenter. Figure 5 An example of a rehearsal session populated on the calendar of a second user (e.g., co-organizer, Bryan Wright) is shown.

[0050] Figures 6-18 Examples of various tests and measurements that can be performed during the test phase, which is referred to herein as the "rehearsal staging room" of the rehearsal session, are shown. In these examples, the visualization of the presentation, the user interface format, the audio signal, the video signal, and other parameters can be tested for a subset of users, such as the organizer and the presenter. This allows the presenter and the organizer to test all parameters to ensure that all settings work properly before the live rehearsal.

[0051] Figure 6An example of a pre-join screen is shown. In this example, the user interface 101F is shown for the second user 10B, co-organizer, Bryan Wright. In the pre-join screen, the organizer sees the rehearsal label, so they understand that the meeting is invisible and does not affect any attendees. This allows the user to select the "Join Now" button to start the "Rehearsal Stage Room". In this example, when the organizer is mentioned as joining the meeting, it means that the system is running in the stage of the "Rehearsal Stage Room" which is referred to as a rehearsal session in this document.

[0052] Once the organizer enters the meeting, as shown in Figure 7 , a technical check is automatically completed. The technical check includes communication tests for the organizer's video data and audio data. If there are any errors regarding their audio and video, the organizer will see an alert and actions to fix those issues will be invoked. For example, the UI shows that the organizer's audio has not been input, but there is no problem with the video. This can be done by running tests to see if the hardware devices are connected. If a hardware device (such as a headset with a microphone) is not connected, the system can generate an alert indicating the situation. The alert can be provided with guidance on how to solve the detected problem. This allows the user to solve the problem by ensuring the audio device is connected and checking the device settings.

[0053] Figure 8 Another example of a detected problem is shown. In this example, even though the organizer's audio and video are both of good quality, the system detects that the audio is coming from the monitor rather than the headset. In the detection of such a setting, the system recommends to the organizer to switch the device for optimal results. This is typically something that a meeting producer would tell each presenter and organizer during a manual technical check, but the system automatically provides this feature to save time for all users. Hardware optimization can be achieved by building a database of different types of ranked devices (such as headsets and different types of speakers). These devices can be ranked, and when a device is detected as connected, the system can pick the next (e.g., higher-ranked) device on the list to make a recommendation for optimization. As shown, the user can select the "Open Device Settings" user interface element to adjust the device settings.

[0054] Figure 9Shows a device settings user interface. The device settings can be shown in a side rail that provides options for the user to select new hardware devices such as headphones. The user can also adjust the volume level for each device. Other features can include activation of noise suppression features and adjustment of video camera parameters. As shown, the user interface can also show real-time telemetry adjusted in response to the settings change. Thus, when a person inserts headphones and the headphones meet one or more criteria or thresholds, the system displays a checkbox in the audio indicator. As shown, the user can save these changes and close the device settings window.

[0055] Figure 10 Shows the user interface after the device settings window is closed. Once the alerts for the organizer are processed, the system can provide features for checking the audio and video of each presenter. The menu can provide user interface elements such as buttons to allow the organizer to move the test phase to the presenters.

[0056] Figure 11 Shows the user interface during the test phase for the presenters. During this test phase, the system displays an overview of the telemetry data, errors, and alerts for each presenter. The system runs technical checks on each of the audio and video signals for the presenters. When problems are detected, such as missing hardware devices, the system displays an indication of those problems. When the system's technical checks determine that the audio and video signals meet the thresholds, the system provides a positive indication such as a checkmark. The user interface also provides elements for activating the telemetry view, such as a telemetry button.

[0057] Figure 12 Shows the user interface in which the presenter's telemetry data is shown. This shows all the raw data from the audio and video checks and their history. The user can also hover over the chart and cause the system to show specific data, such as detected jitter and bitrate. By providing the organizer with a raw telemetry view in the form of a chart, the system provides access to the presenter's telemetry data and enables them to identify specific times at which problems occurred. Additionally, the system shows the audio and video quality that can be output. This enables more detailed analysis from IT administrators or support teams. As shown in Figure 13 the user interface can also provide user interface elements to allow the user to find solutions for each detected problem. In this example, the second presenter has a problem with their audio, which is below the baseline, and the organizer selects a solution link.

[0058] Figure 14An array of solutions that the system can provide based on the detected problems is shown. The system can make many recommendations. For example, the system can recommend reconnecting hardware, such as cables and hardware. If the system detects that other applications are open, the system can recommend closing other applications. The system can also advise the user to turn off other electronic devices that may interfere with the connection. Once the solutions are displayed, the user can select the back button and return to the telemetry data.

[0059] Figure 15 A user interface that displays telemetry data in response to a user selection of the back button on the solution user interface is shown. As an organizer, the system provides access to the telemetry data of any presenter. This can show real-time audio and video quality data. Once the telemetry data is displayed, the user can select the back button and cause a display of an overview of the rehearsal stage room.

[0060] As shown in Figure 16 The system displays a summary of the detected problems, as well as the solutions provided by the system. The system also aggregates a graphical chart of the number of participants and telemetry data for the rehearsal stage room session. This data can also be exported for analysis. The user interface also displays the specific problems identified for each presenter and optimization suggestions such as using a headset device instead of a speaker. The user interface also provides graphical elements that allow the organizer to view the detailed telemetry data for each presenter.

[0061] Figure 17 Detailed telemetry data for each presenter is shown. This example shows a historical graph of the overall network connection indicator and network performance readings for the rehearsal session. The user interface also provides audio and video telemetry data for the organizer and each presenter. If a particular presenter does not have an audio device, such as the audio data chart for the first presenter, the system will provide an indication of that status.

[0062] Figure 18 Allows the organizer to start a live rehearsal session that involves simulating other attendees (such as audience members). The user interface provides a start button that enables the organizer to control when the actual live rehearsal starts. This gives each of the presenters and the organizer time to discuss any open issues. Once the user selects the start rehearsal button, the system can transition from a first operating state involving the rehearsal stage room to a second operating state involving the live rehearsal. As described below, "live rehearsal" includes using robots and other systems to simulate other attendees, such as audience members.

[0063] Figure 19Disclosed is a configuration user interface for a live drill. This user interface can be displayed in response to a user selecting the "Start Drill" button in a previous user interface. The configuration user interface for the live drill can include a text input field that allows an organizer to enter the number of projected attendees. This enables the system to receive data indicating the size of an expected meeting. This allows the system to generate multiple robots that simulate the behavior of attendees. This allows the system to test the overall load of the system based on the size of the actual event. The system can also automatically analyze the original meeting data, determine the number of attendees, and automatically enter this information. In response to receiving or determining this information, the system can start a simulation, which is also referred to herein as a live drill.

[0064] In some configurations, the system can determine the locations of the people invited to the original meeting. Based on this location information for each attendee, the system can configure robots in various geographical locations to test network connection firewalls and other devices involved in the original meeting. In one example, the system can analyze organizational data to determine the locations of the invitees for a meeting. When the system determines that the locations of each invitee are in different geographical regions, the system can configure computing devices in each region for the drill session. For example, as shown in Figure 20 if the meeting is stored in a specific meeting object and the meeting involves invitees located in Bangalore, Beijing, Bellevue, Redmond, and Rio, the system will allocate test resources in those locations to test the system configuration during the live drill.

[0065] As shown in Figure 21 the system can display a user interface that shows aspects of what each presenter will see during the original meeting. Additionally, this user interface shows status indicators for audio, video, and network measurements. A large user interface area on the right side of the screen shows what the presenter will see during the original meeting. A smaller user interface area on the left side of the screen shows the video stream rendering of the presenter and the organizer(s). This user interface also shows an event assistant. The event assistant provides controls that allow the organizer to control the activities of the robots of the simulated attendees and a control button for stopping the simulation. This user interface also allows the organizer to minimize the event assistant, as shown by the user interface control icon under the pointer.

[0066] Figure 22Shows the rehearsal stage room user interface after the organizer has minimized the event assistant. At this stage of the process, the organizer can select the Go Live button. The Go Live button activates the robot and starts the live rehearsal. Once the robot is activated, as shown in Figure 23 a status indicator is displayed on the user interface to indicate that the live rehearsal has started. The Go Live button causes the system to transition from a first operating mode involving the rehearsal stage room to a second operating mode involving the live rehearsal. The live rehearsal operating mode involves simulating the activities of the attendees by the robot.

[0067] Figure 23 Shows the live rehearsal session user interface after the organizer has activated the robot. As shown, this user interface provides a notification that the operating mode of the system has changed to a live rehearsal involving the active robot. When in the live rehearsal, the system allows the user to select the "Open Attendee View" button to allow the user to see what the attendees will see, such as what the attendees would see during the original meeting.

[0068] Figure 24 Shows the attendee view user interface after the organizer has activated the "Open Attendee View" button. This user interface allows the presenter to view what the audience members will see during the actual meeting. They can see the orientation of the shared content (left side), such as a slide deck or video, which shows the presenter's layout (upper right, middle right, and also the video stream of the attendees (lower right)).

[0069] Figure 25 Shows an example of how the user interacts with the event assistant to simulate the activities of the audience members. And this is an example where the organizer selects the "Raise Hand" button. This causes one or more robots that simulate the activities of the audience members, for example, to generate a signal indicating that the audience members have raised their hands and wish to share information.

[0070] Figure 26 Shows an example of the raise hand indicator 401 displayed in response to the simulation of raising a hand (e.g., in response to the organizer tapping the "Raise Hand" button). This allows the organizer to show the presenter during the meeting where the raise hand indicator 401 will be displayed. This can be helpful in demonstrating the features of the communication program, especially for users who are not familiar with the program. Other features can cause the robot to generate emoji responses and text messages to allow the presenter to see how each of these features can be displayed.

[0071] As shown in Figure 27 the user selects the "Stop Simulation" button to stop the live rehearsal. When the simulation is terminated, as shown in Figure 28As shown, the system generates a summary of the live rehearsal period, and the report can provide an overall summary of the success or failure of the simulation. The summary also indicates the problems detected during the rehearsal stage room phase and the live rehearsal phase. The summary also indicates that the installation of a specific application (such as a polling application), as described herein, can install software features and configure them for rehearsal in response to detecting the application selected for the original meeting object. Thus, if the original meeting is established with a polling feature, such as a vote collection feature, the rehearsal meeting object will also include the same feature.

[0072] Figure 29 Shows a complete report generated in response to data collected during the rehearsal stage room and the live rehearsal. The report can show meeting details, such as the number of real participants, simulated participants, and total participants, as well as the meeting duration and the number of identified problems. Other graphical indicators can be provided to show audio and video readings during the meeting. This can be summarized by providing simplified indicators such as good, OK, or failed. The report can also show the participation timeline of real participants and simulated participants. This shows when each person or robot joined the session, when they left the session, and when there were failure points (the shaded part of the activity bar for Ray Tanaka). The report can also include feedback from each participant.

[0073] Figure 30 Is a diagram illustrating aspects of routine 900 for computationally efficient management of an automated rehearsal session. Those of ordinary skill in the art should understand that the operations of the methods disclosed herein are not necessarily presented in any particular order, and some or all operations may be performed and contemplated in alternative orders. For ease of description and illustration, the operations are presented in a demonstrative order. Operations can be added, omitted, performed, and / or performed simultaneously without departing from the scope of the appended claims.

[0074] It should also be understood that the illustrated methods can end at any time and need not be executed in their entirety. Some or all of the operations of the method and / or substantially equivalent operations can be performed by executing computer-readable instructions included on a computer storage medium, as defined herein. As used in the specification and claims, the term "computer-readable instructions" and its variants are used herein broadly to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, etc. Computer-readable instructions can be implemented on a variety of system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, etc. Although the example routines described below operate on a system (e.g., one or more computing devices), it should be appreciated that the routines can be executed on any computing system, which can include any number of computers working together to perform the operations disclosed herein.

[0075] Accordingly, it should be appreciated that the logical operations described herein are implemented as: (1) a sequence of computer-implemented acts or program modules running on a computing system, such as those described herein, and / or (2) interconnected machine logic circuits or circuit modules within the computing system. This implementation is a matter of choice depending on the performance and other requirements of the computing system. Accordingly, the logical operations can be implemented in software, firmware, dedicated digital logic, and any combination thereof.

[0076] In addition, the operations illustrated in Figure 30 and other figures can be implemented in association with the (one or more) example presentation user interfaces (UIs) described above. For example, the various devices and / or (one or more) modules described herein can generate, send, receive, and / or display data associated with the content of a communication session (e.g., live content, broadcast events, recorded content, etc.) and / or present a UI that includes one or more participants, avatars, channels, chat sessions, video streams, images, virtual objects, and / or applications associated with the communication session of a remote computing device.

[0077] Routine 900 begins at operation 902, where the system generates a conference object that defines a communication session for a plurality of users, where the conference object defines a conference start time and the role of an individual user among the plurality of users. The conference object can be generated by a conference wizard that prompts the user for the parameters of the original conference. For illustrative purposes, the original conference (also referred to herein as a "communication session") is referred to herein as a conference that will be established by an organizer or presenter. The rehearsal session is another conference that includes a subset of the users invited to the original conference.

[0078] At operation 904, using the conference object, the system generates a test session object that defines a test session for a subset of the users among the multiple users, the subset of users being selected based on a predetermined role assigned for the subset of users in the conference object. The test session object defines a test start time before the conference start time. The test session is also referred to herein as a rehearsal session. The system copies parameters and other software objects from the conference object to populate the parameters and software objects in the test session object, e.g., the rehearsal session object that defines the rehearsal session.

[0079] At operation 906, the system measures performance parameters for at least one of network bandwidth, audio volume, and image quality during the test session. The system is capable of recording this information and displaying it in real time to the users invited to the test session. This is referred to herein as telemetry data.

[0080] At operation 908, the system determines that one or more performance parameters do not meet one or more threshold requirements. This includes a network bandwidth measurement that does not meet the threshold, an audio quality metric that does not meet the audio threshold, or a video quality metric that does not meet the video threshold. The parameters can also include optimization measurements, such as a device type that does not meet the quality threshold. For example, if the device detects computer speakers and the graph indicates higher quality for headphone speakers, the system can determine that the computer speakers do not meet the threshold requirements and the system will recommend headphones.

[0081] At operation 910, in response to determining that the one or more performance parameters do not meet the one or more threshold requirements, the system generates data that defines an adjustment to one or more settings that control the performance parameters of the test session. This can be in the form of a notification or comment within a summary that indicates that an adjustment to the conference options is needed. This can include a notification that the user should change the hardware, change the settings, close an application, etc. The system is also capable of automatically changing the settings or options in the test session. For example, if the microphone does not produce an appropriate level of signal, the system can increase the volume of the microphone in response to determining that the measured performance parameter does not meet the threshold.

[0082] Any recommended or any adjustments made to the test session can also be applied to the communication session. Thus, the system can measure activities during the test session, determine if the settings are incorrect for the test session, and then adjust the test session parameters and / or the original meeting parameters. This part of the process can include audio measurement results, video measurement results, and network performance measurement results. The thresholds can include volume, sound quality, video quality, brightness level, jitter threshold, bandwidth threshold for an individual, or bandwidth threshold for a group of individuals. For example, the system can measure performance parameters during the test session, where the performance parameters include at least one of network bandwidth parameters, audio parameters, or video parameters. If the system detects a packet loss rate exceeding the threshold, the system can determine that the test has failed and the system can generate one or more solutions to fix the problem. In another example, if the system detects that the frame rate for video is below the threshold, the system can determine that the test has failed and the system can generate an appropriate solution. The solution can include an adjustment to a setting or option, for example, the selection of a particular network driver or the selection for a particular device, those adjustments can be made to the settings for the rehearsal session, and at the same time, those adjustments can be made in the original meeting. Thus, when the original meeting starts, those settings will be automatically presented for the live meeting. In another example, if the volume setting is changed during the rehearsal, the system will also copy that change in the original meeting object.

[0083] The system can also copy software plugins from the test session object to the original meeting object. For example, if the organizer determines during the test session that an answer object for a function to facilitate question and answer sessions is needed, the system can also install that object into the meeting object for the original meeting.

[0084] The system can also change the permissions for an individual during the test session. For example, if the organizer determines that an attendee has the permission to share content, but the presenter determines that the preference is to have a restriction that blocks audience members from sharing content, the system can change the permissions of the robot simulating the audience member in the test session. Additionally, the system can update the permissions for the audience members in the original meeting object. This provides increased security by helping the meeting organizer control permissions at a macro level without the user having to change the individual settings for each person or each role in the meeting.

[0085] The routine is also capable of storing any measurements collected during a rehearsal session and applying them to a machine learning model. The model is capable of storing adjustments made to the rehearsal session and adjustments made to the original session. Then, the system can determine whether those adjustments are used in the original session or in any subsequent rehearsal session. If those adjustments are used or if feedback indicates that the adjustments are successful, the system can save those adjustments for future iterations of the routine. This allows preferred or successful adjustments to take precedence over future iterations of the routine. In this way, recommendations or adjustments to parameters that are considered successful or frequently used can typically be recommended more frequently in future meetings and rehearsal sessions. These adjustments to the machine learning model can be made in operation 910. Then, the routine can return to operation 902 in future iterations of the routine, regardless of whether the routine can utilize the adjusted model.

[0086] Figure 31 is a diagram of an exemplary environment 1100 in which a system 1102 (which can be the Figure 1 system 100) can implement the techniques disclosed herein. In some embodiments, the system 1102 can be used to collect, analyze, and share data defining one or more objects presented to a user of a communication session 1104.

[0087] As illustrated, the communication session 1104 can be implemented between a plurality of client computing devices 1106(1) through 1106(N) (where N is a number having a value of two or greater), which are associated with or are part of the system 1102. The client computing devices 1106(1) through 1106(N) enable users, also referred to as individuals, to participate in the communication session 1104.

[0088] In this example, the communication session 1104 is hosted by the system 1102 on one or more networks 1108. That is, the system 1102 can provide services that enable users of the client computing devices 1106(1) through 1106(N) to participate in the communication session 1104 (e.g., via live viewing and / or recorded viewing). Thus, the "participants" in the communication session 1104 can include users and / or client computing devices (e.g., multiple users can participate in the communication session in a room via the use of a single client computing device), each of which can communicate with other participants. As an alternative, the communication session 1104 can be hosted by one of the client computing devices 1106(1) through 1106(N) using peer-to-peer technology. The system 1102 can also host chat conversations and other team collaboration functions (e.g., as part of an application suite).

[0089] In some embodiments, such chat conversations and other team collaboration functions are considered external communication sessions distinct from the communication session 1104. A computerized agent configured to collect participant data in the communication session 1104 may be able to link to such an external communication session. Thus, the computerized agent may receive information enabling connection to such an external communication session, such as date, time, session details, etc. In one example, a chat conversation can be conducted based on the communication session 1104. Additionally, the system 1102 may host the communication session 1104, which includes at least multiple participants co-located at a meeting location (such as a conference room or auditorium) or located at different locations.

[0090] In the example described herein, the client computing devices 1106(1) to 1106(N) participating in the communication session 1104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live content and / or recorded content. The collection of various instances or streams of live content and / or recorded content can be provided by one or more cameras (such as video cameras). For example, an individual stream of live or recorded content can include media data associated with a video feed provided by a camera (e.g., audio and visual data capturing the appearance and speech of users participating in the communication session). In some embodiments, the video feed can include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images can also include one or more avatars.

[0091] Another example of an individual stream of live and / or recorded content can include media data that includes an avatar of a user participating in the communication session and audio data capturing the user's speech. Yet another example of an individual stream of live or recorded content can include media data that includes a file displayed on a display screen and audio data capturing the user's speech. Thus, the various streams of live and / or recorded content within the communication data enable a remote conference to be facilitated among a group of people and content to be shared within the group of people. In some embodiments, the various streams of live and / or recorded content within the communication data can originate from multiple co-located video cameras positioned in a space (such as a room) to record or stream a presentation including one or more individuals giving the presentation and one or more individuals consuming the presented content.

[0092] Participants or attendees can view the content of communication session 1104 as it occurs, or alternatively, view the recording at a later time after the event has occurred. In the examples described herein, client computing devices 1106(1) through 1106(N) participating in communication session 1104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live and / or recorded content. For example, an individual stream of content can include media data associated with a video feed (e.g., audio and video data that captures the appearance and voice of a user participating in the communication session). Another example of an individual stream of content can include media data that includes an avatar of a user participating in a meeting session and audio data that captures the user's voice. Yet another example of an individual stream of content can include media data that includes content items displayed on a display screen and / or audio data that captures the user's voice. Thus, the various streams of content within the communication data enable a meeting or broadcast presentation to be facilitated among a group of people dispersed at remote locations.

[0093] Participants or attendees of the communication session are people within the range of a camera or other image and / or audio capture device such that the actions and / or sounds of the people as they are watching and / or listening to content shared via the communication session can be captured (e.g., recorded). For example, a participant can be sitting in a crowd that is live watching shared content at a broadcast location where a stage presentation is occurring. Or, a participant can be sitting in an office conference room and viewing shared content of a communication session with other colleagues via a display screen. Further still, a participant can be sitting or standing in front of a personal device (e.g., a tablet, a smart phone, a computer, etc.) and viewing the shared content of the communication session individually in their office or at home.

[0094] System 1102 includes device(s) 1110. Device(s) 1110 of system 1102 and / or other components can include distributed computing resources that communicate with each other and / or with client computing devices 1106(1) through 1106(N) via one or more networks 1108. In some examples, system 1102 can be an independent system responsible for managing aspects of one or more communication sessions such as communication session 1104. As an example, system 1102 can be managed by entities such as SLACK, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0095] One or more networks 1108 may include, for example, a public network such as the Internet, a private network such as an institutional and / or personal intranet, or some combination of private and public networks. One or more networks 1108 may also include any type of wired and / or wireless network, including but not limited to: local area networks (“LANs”), wide area networks (“WANs”), satellite networks, cable networks, Wi-Fi networks, WiMax networks, mobile communication networks (e.g., 3G, 4G, etc.), or any combination thereof. One or more networks 1108 may utilize communication protocols, including: packet- and / or datagram-based protocols such as Internet Protocol (“IP”), Transmission Control Protocol (“TCP”), User Datagram Protocol (“UDP”), or other types of protocols. Additionally, one or more networks 1108 may also include a plurality of devices that facilitate network communication and / or form the hardware foundation of the network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, etc.

[0096] In some examples, one or more networks 1108 may further include a device that enables connection to a wireless network, such as a wireless access point (“WAP”). Example supports connections via a WAP, which sends and receives data over various electromagnetic frequencies (e.g., radio frequency), including WAPs that support Institute of Electrical and Electronics Engineers (“IEEE”) 802.11 standards (e.g., 802.11g, 802.11n, 802.11ac, etc.) and other standards.

[0097] In various examples, one or more devices 1110 may include one or more computing devices that operate in a cluster or other grouping configuration to share resources, balance load, increase performance, provide failover support or redundancy, or for other purposes. For example, one or more devices 1110 may belong to various types of devices, such as conventional server-type devices, desktop computer-type devices, and / or mobile-type devices. Thus, although illustrated as a single type of device or a server-type device, one or more devices 1110 may include a wide variety of device types and are not limited to a particular type of device. One or more devices 1110 may represent, but are not limited to: server computers, desktop computers, web server computers, personal computers, mobile computers, laptop computers, tablet computers, or any other type of computing device.

[0098] A client computing device (e.g., one of client computing devices 1106(1) to 1106(N)) can belong to various types of devices, which can be the same as or different from the device(s) 1110, such as conventional client type devices, desktop computer type devices, mobile type devices, special purpose type devices, embedded devices, and / or wearable type devices. Thus, the client computing device can include, but is not limited to: desktop computers, game consoles and / or game devices, tablet computers, personal data assistants ("PDAs"), mobile phone / tablet hybrid devices, laptop computers, telecommunications devices, computer navigation type client computing devices such as satellite-based navigation systems (including Global Positioning System ("GPS") devices), wearable devices, virtual reality ("VR") devices, augmented reality ("AR") devices, implanted computing devices, automotive computers, network-enabled televisions, thin clients, terminals, Internet of Things ("IoT") devices, workstations, media players, personal video recorders ("PVRs"), set-top boxes, cameras, integrated components for inclusion in a computing device (e.g., peripherals), appliances, or any other type of computing device. Additionally, the client computing device can include a combination of the previously listed examples of client computing devices, such as, for example, a desktop computer type device or a mobile type device combined with a wearable device, etc.

[0099] Client computing devices 1106(1) to 1106(N) of various categories and device types can represent any type of computing device having one or more data processing units 1192, the one or more data processing units 1192 being operably connected to a computer-readable medium 1194, such as via a bus 1116, which in some cases can include a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and one or more of any various local, peripheral, and / or independent buses.

[0100] The executable instructions stored on the computer-readable medium 1194 can include, for example, an operating system 1119, a client module 1120, a profile module 1122, and other modules, programs, or applications that can be loaded and executed by the data processing unit(s) 1192. The client computing devices 1106(1) to 1106(N) can also include one or more interfaces 1124 to enable communication between the client computing devices 1106(1) to 1106(N) and other networked devices (such as the device(s) 1110) via the network(s) 1108. Such network interface(s) 1124 can include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. Additionally, the client computing devices 1106(1) to 1106(N) can include input / output (“I / O”) interface(s) (devices) 1126 that enable communication with user input devices such as including peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices such as microphones, video cameras for obtaining and providing video feeds and / or still images, touch input devices, gesture input devices, etc.) and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, haptic output devices, etc.). Figure 31 Illustrated is that the client computing device 1106(1) is connected to a display device (e.g., the display screen 1129(1)) in a manner that can display a UI according to the techniques described herein.

[0101] In Figure 31 In the example environment 1100, the client computing devices 1106(1) to 1106(N) can use their respective client modules 1120 to connect to each other and / or to one or more other external devices to participate in a communication session 1104 or to contribute activities to the collaborative environment. For example, a first user can communicate with a second user of another client computing device 1106(2) using the client computing device 1106(1). When the client module 1120 is executed, users can share data, which can cause the client computing device 1106(1) to connect to the system 1102 and / or other client computing devices 1106(2) to 1106(N) via the network(s) 1108.

[0102] The client computing devices 1106(1) to 1106(N) can use their respective profile modules 1122 to generate participant profiles (in Figure 31(not shown in the figure), and provide the participant profile to other client computing devices and / or one or more devices 1110 of the system 1102. The participant profile may include one or more of the following: the identity of the user or group of users (e.g., name, unique identifier ("ID"), etc.), user data such as personal data, user data such as location (e.g., IP address, room in a building, etc.) and technical capabilities. The participant profile can be used to register participants for a communication session.

[0103] As shown in Figure 31 the one or more devices 1110 of the system 1102 include a server module 1130 and an output module 1132. In this example, the server module 1130 is configured to receive media streams 1134(1) to 1134(H) from individual client computing devices such as client computing devices 1106(1) to 1106(N). As described above, the media stream can include a video feed (e.g., audio and visual data associated with the user), audio data to be output together with the presentation of the user's avatar (e.g., an audio-only experience without sending the user's video data), text data (e.g., text messages), file data, and / or screen sharing data (e.g., documents, slide sets, images, videos displayed on a display screen, etc.). Thus, the server module 1130 is configured to receive a collection of various media streams 1134(1) to 1134(N) (the collection is referred to herein as "media data 1134") during the live viewing of the communication session 1104. In some scenarios, not all client computing devices participating in the communication session 1104 provide media streams. For example, a client computing device can be a consumption or "listening" device only, such that it only receives content associated with the communication session 1104, but does not provide any content to the communication session 1104. The communication session 1104 can have a start time and an end time, or the communication session 1104 can be in progress. The communication session 1104 can also be classified as an event and have phases, where each phase causes the computer to change the role of an individual user as the event transitions through each phase.

[0104] In various examples, the server module 1130 can select aspects of the media stream 1134 to share with individual client computing devices among the participating client computing devices 1106(1) through 1106(N). Thus, the server module 1130 can be configured to generate session data 1136 based on the stream 1134 and / or to pass the session data 1136 to the output module 1132. The output module 1132 can then transmit communication data 1139 to the client computing devices (e.g., the client computing devices 1106(1) through 1106(N) participating in the live viewing of the communication session). The communication data 1139 can include video, audio, and / or other content data provided by the output module 1132 based on the content 1150 associated with the output module 1132 and based on the received session data 1136. The device(s) 1110 of the system 1102 can also access the queue data 101 described above in connection with Figure 1 the standard data 1191 for defining the standards and / or thresholds described herein. The standard data 1191 can also include machine learning data accessible by a machine learning service or machine learning module, which can be part of the server module 1130 or part of a remote machine learning service, such as those accessible by a public API at a site run by IBM, Google, or Microsoft.

[0105] As shown, the output module 1132 transmits the communication data 1139(1) to the client computing device 1106(1), and transmits the communication data 1139(2) to the client computing device 1106(2), and transmits the communication data 1139(3) to the client computing device 1106(3), and so on. The communication data 1139 transmitted to the client computing devices can be the same or can be different (e.g., the positioning of the content stream within the user interface can vary from one device to the next).

[0106] In various embodiments, the device(s) 1110 and / or the client module 1120 of the system 1102 can include a GUI rendering module 1140. The GUI rendering module 1140 can be configured to analyze communication data 1139 for delivery to one or more of the client computing devices 1106. Specifically, the UI rendering module 1140 at the device(s) 1110 and / or the client computing device 1106 can analyze the communication data 1139 to determine an appropriate way to display video, images, and / or content on the display screen 1129 of the associated client computing device 1106. In some embodiments, the GUI rendering module 1140 can provide video, images, and / or content to a rendered presentation GUI 1146 on the display screen 1129 of the associated client computing device 1106. The presentation GUI 1146 can be rendered by the GUI rendering module 1140 on the display screen 1129. The presentation GUI 1146 can include video, images, and / or content analyzed by the GUI rendering module 1140.

[0107] In some embodiments, the presentation GUI 1146 can include multiple sections or grids that can render or include video, images, and / or content for display on the display screen 1129. For example, a first section of the presentation GUI 1146 can include a video feed of a presenter or an individual, and a second section of the presentation GUI 1146 can include a video feed of personal consumption meeting information provided by the presenter or the individual. The GUI rendering module 1140 can populate the first and second sections of the presentation GUI 1146 in a manner that appropriately mimics the environmental experience that the presenter and the individual can share.

[0108] In some embodiments, the GUI rendering module 1140 can zoom in or provide a zoomed view of an individual represented by a video feed to highlight the individual's reaction to the presenter, such as facial features. In some embodiments, the presentation GUI 1146 can include video feeds of multiple participants associated with a meeting, such as a general communication session. In other embodiments, the presentation GUI 1146 can be associated with a channel such as a chat channel, an enterprise team channel, etc. Thus, the presentation GUI 1146 can be associated with an external communication session different from a general communication session.

[0109] Figure 32FIG. illustrates a diagram showing example components of an example device 1200 (also referred to herein as a “computing device”), the example device 1200 being configured to generate and process data for some user interfaces in the user interfaces disclosed herein. The device 1200 can generate data that can include one or more portions of video, images, and / or content that can be rendered or included for display on a display screen 1129. The device 1200 can represent one of the (one or more) devices described herein. Additionally or alternatively, the device 1200 can represent one of the client computing devices 1106.

[0110] As illustrated, the device 1200 includes one or more data processing units 1202, a computer-readable medium 1204 (also referred to herein as computer storage medium 1204), and (one or more) communication interfaces 1206. The components of the device 1200 are operatively connected, for example, via a bus 1209, which can include a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and one or more of any of various local, peripheral, and / or independent buses.

[0111] As used herein, (one or more) data processing units (such as (one or more) data processing units 1202 and / or (one or more) data processing units l192) can represent, for example, a CPU-type data processing unit, a GPU-type data processing unit, a field-programmable gate array (“FPGA”), another type of digital signal processor (“DSP”), or other hardware logic components that can be driven by a CPU in some cases. For example, but not limited to: illustrative types of hardware logic components that can be used include application-specific integrated circuits (“ASICs”), application-specific standard products (“ASSPs”), systems-on-a-chip (“SOCs”), complex programmable logic devices (“CPLDs”), etc.

[0112] As used herein, computer-readable media (such as computer-readable medium 1204 and computer-readable medium 1194) can store instructions executable by (one or more) data processing units. The computer-readable media can also store instructions executable by an external data processing unit (such as an external CPU, an external GPU) and / or executable by an external accelerator (such as an FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated in the computing device, while in some examples, one or more of the CPU, GPU, and / or accelerator are external to the computing device.

[0113] A computer-readable medium (which may also be referred to herein as a computer-readable medium) can include computer storage media and / or communication media. "Computer storage media", "non-transitory computer storage media", or "non-transitory computer-readable media" can include volatile memory, non-volatile memory, and / or other persistent and / or auxiliary computer storage media, removable and non-removable computer storage media, implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Thus, the computer storage media or variants of the term described above include the tangible and / or physical form of the media included in devices and / or hardware components that are part of a device or hardware components external to the device, including but not limited to: random access memory ("RAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), phase change memory ("PCM"), read-only memory ("ROM"), erasable programmable read-only memory ("EPROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory, compact disc read-only memory ("CD-ROM"), digital versatile disc ("DVD"), optical card, or other optical storage media, magnetic tape cartridge, tape, disk storage, magnetic card, or other magnetic storage device or media, solid-state memory device, storage array, network-attached storage device, storage area network, hosted computer storage device, or any other storage memory, storage device, and / or any storage media that can be used for local storage and maintenance of information for access at a computing device.

[0114] Compared with computer storage media, communication media can embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transmission mechanism. As defined herein, computer storage media does not include communication media. That is, computer storage media itself does not include communication media consisting solely of modulated data, signals, carrier waves, or propagated signals.

[0115] (One or more) communication interfaces 1206 can represent, for example, a network interface controller ("NIC") or other types of transceiver devices for sending and receiving communications over a network. In addition, (one or more) communication interfaces 1206 can include one or more cameras and / or audio cues 1222 to enable the generation of video feeds and / or still images, etc.

[0116] In the illustrated example, the computer-readable medium 1204 includes a data repository 1208. In some examples, the data repository 1208 includes a data storage device, such as a database, a data warehouse, or other types of structured or unstructured data storage devices. In some examples, the data repository 1208 includes a corpus and / or a relational database having one or more tables, indexes, stored procedures, etc. to enable data access, including for example one or more of Hypertext Markup Language ("HTML") tables, Resource Description Framework ("RDF") tables, Web Ontology Language ("OWL") tables, and / or Extensible Markup Language ("XML") tables.

[0117] The data repository 1208 can store data for operations of processes, applications, components, and / or modules stored in the computer-readable medium 1204 and executed by the (one or more) data processing units 1202 and / or the (one or more) accelerators. For example, in some examples, the data repository 1208 can store meeting objects 1210, test session objects 1212, and / or other data. The meeting object 1210 can include the total number of participants (e.g., users and / or client computing devices) in a communication session, the activities that occur in the communication session, a list of invitees to the communication session, and / or other data related to when and how the communication session is conducted or hosted. The test session object 1212 stores aspects of a rehearsal session, including identifiers of a subset of the attendees of the original meeting object, as well as other settings and options for controlling the rehearsal session.

[0118] The data repository 1208 can also include context data 1214, which can include any information that defines the activities of the users, criteria, or thresholds disclosed herein.

[0119] Alternatively, some or all of the data described above can be stored on a separate memory 1216 on the (one or more) data processing units 1202, such as on-board memory, CPU-type processors, GPU-type processors, FPGA-type accelerators, DSP-type accelerators, and / or another accelerator. In this example, the computer-readable medium 1204 also includes an operating system 1218 and an application programming interface 1210 (API) configured to expose the functions and data of the device 1200 to other devices. Additionally, the computer-readable medium 1204 includes one or more modules, such as a server module 1230, an output module 1232, and a GUI rendering module 1240, although the number of illustrated modules is merely an example and the number can vary higher or lower. That is, the functionality described in connection with the illustrated modules herein can be performed by a smaller number of modules or a larger number of modules on one device or spread across multiple devices.

[0120] It should be appreciated that, unless otherwise expressly stated, conditional language such as "can", "could", "might", or "may", used herein, is understood in context to present particular examples including, and other examples excluding, a particular feature, element, and / or step. Thus, such conditional language is generally not intended to imply that a particular feature, element, and / or step is required in any way for one or more examples or that one or more examples necessarily include logic for deciding, with or without user input or prompting, whether to include or to perform a certain feature, element, and / or step in any particular example. Unless otherwise expressly stated, phrase language such as "at least one of X, Y, or Z" should be understood to present that items, terms, etc. may be X, Y, or Z, or combinations thereof. Additionally, the words "the" and "if" may be used interchangeably. Thus, a phrase such as "determine that the criteria are met" may also be interpreted as "determine that the criteria are satisfied", and vice versa.

[0121] It should also be appreciated that numerous variations and modifications can be made to the examples described above, and such variations and modifications should be understood to be within other acceptable examples. All such modifications and variations are intended to be included within the scope of the present disclosure and be protected by the appended claims.

[0122] In summary, although various configurations have been described in language specific to structural features and / or method acts, it should be understood that the subject matter defined in the appended claims need not be limited to the particular features or acts described. Rather, the particular features and acts are disclosed as example forms of implementing the claimed subject matter.

Claims

1. A method, run by a computing system, for performing a test session for a communication session for multiple users, comprising: Generating a conference object that defines the communication session for the multiple users, the conference object defining a conference start time and roles for individual users among the multiple users; Selecting a subset of users from the multiple users, wherein individual users in the subset of users are selected based on the individual users having a predetermined role assigned in the conference object; Using the conference object to generate a test session object that defines the test session for the subset of users from the multiple users, the subset of users being selected based on the predetermined role assigned for the subset of users in the conference object, the test session object defining a test session start time before the conference start time; Measuring one or more performance parameters during the test session, wherein the one or more performance parameters include at least one of the following: a network bandwidth parameter, an audio parameter, or a video parameter; Determining that the one or more performance parameters do not meet one or more threshold requirements; and Generating data in response to determining that the one or more performance parameters do not meet the one or more threshold requirements, the data defining an adjustment to one or more settings that control the performance parameters of the test session.

2. The method according to claim 1 further comprises: Causing a display of the one or more performance parameters measured during the test session, wherein the display includes a graphical indicator of at least one of the following: the network bandwidth parameter, the audio parameter, or the video parameter.

3. The method according to claim 1 further comprises: Storing the adjustment to the one or more settings into the test session object that defines the test session.

4. The method according to claim 1 further comprises: Causing an adjustment to one or more settings of the communication session based on the adjustment to the one or more settings during the test session.

5. The method according to claim 1, further comprising: Analyzing organizational data to determine the locations of users having a second predetermined role; Activating remote computing devices in a geographic region based on the locations of the users having the second predetermined role, wherein the remote computing devices simulate the activities of conference attendees from the geographic region for measuring the one or more performance parameters via a network connecting the system and the remote computing devices; and Receiving live telemetry data from the remote computing devices based on the one or more performance parameters measured at the remote computing devices located in the geographic region, wherein determining that the one or more performance parameters do not meet one or more threshold requirements includes using the live telemetry data from the remote computing devices.

6. The method according to claim 1, wherein, The test session includes: a first preliminary stage for measuring the one or more performance parameters for a first group of users having a first role, a second preliminary stage for measuring the one or more performance parameters for a second group of users having a second role, and a live session for measuring the one or more performance parameters for the second group of users having the second role and a remote computing device configured to simulate the activities of other invitees of the communication session.

7. The method according to claim 1, further comprising: Receiving communication data from a remote computing device simulating the activities of a conference attendee; And Causing a video stream depicting at least one user to be displayed on a device associated with at least one user in a subset of the users, wherein a format of a user interface including rendering of the video stream depicting at least one user is based on settings of a user having an attendee role.

8. The method according to claim 1, further comprising: Receiving communication data from a remote computing device simulating the activities of a conference attendee; And Causing a display of a visual element indicating the activities of the conference attendee, wherein a position and a format of the visual element simulate a user interface format displayed in response to live activities of a user having an attendee role during the communication session.

9. A system for performing a test session of a communication session for multiple users, the computing device comprising: One or more processing units; And A computer-readable storage medium encoded with computer-executable instructions that cause the one or more processing units to: Generate a conference object that defines the communication session for the multiple users, the conference object defining a conference start time and roles for individual users among the multiple users; Select a subset of users from the multiple users, wherein an individual user in the subset of users is selected based on the individual user having a predetermined role assigned in the conference object; Use the conference object to generate a test session object that defines the test session for the subset of users from the multiple users, the subset of users being selected based on the predetermined role assigned for the subset of users in the conference object, the test session object defining a test session start time before the conference start time; Measure one or more performance parameters during the test session, wherein the one or more performance parameters include at least one of the following: a network bandwidth parameter, an audio parameter, or a video parameter; Determine that the one or more performance parameters do not meet one or more threshold requirements; and Generate data in response to determining that the one or more performance parameters do not meet the one or more threshold requirements, the data defining an adjustment to one or more settings controlling the performance parameters of the test session.

10. The system according to claim 9, wherein, The instruction further causes the one or more data processing units to: cause a display of the one or more performance parameters measured during the test session, wherein the display includes a graphical indicator of at least one of: the network bandwidth parameter, the audio parameter, or the video parameter.

11. The system according to claim 9, wherein, The instruction further causes the one or more data processing units to: store the adjustment to the one or more settings to the test session object that defines the test session.

12. The system according to claim 9, wherein The instruction further causes the one or more data processing units to: cause an adjustment to one or more settings of the communication session based on the adjustment to the one or more settings during the test session.

13. The system according to claim 9, wherein The instruction further causes the one or more data processing units to: Analyze organizational data to determine the location of a user having a second predetermined role; Activate a remote computing device in a geographical area based on the location of the user having the second predetermined role, wherein the remote computing device simulates the activities of a meeting attendee from the geographical area for measuring the one or more performance parameters via a network connecting the system and the remote computing device; and Receive live telemetry data from the remote computing device based on the one or more performance parameters measured at the remote computing device located in the geographical area, wherein determining that the one or more performance parameters do not meet one or more threshold requirements includes using the live telemetry data from the remote computing device.

14. The system according to claim 9, wherein, The test session includes: a first preliminary stage for measuring the one or more performance parameters for a first group of users having a first role, a second preliminary stage for measuring the one or more performance parameters for a second group of users having a second role, and a live session for measuring the one or more performance parameters for the second group of users having the second role and a remote computing device configured to simulate the activities of other invitees of the communication session.

15. The system according to claim 9, wherein The instruction further causes the one or more data processing units to: Receive communication data from a remote computing device that simulates the activities of a meeting attendee; and Cause a video stream depicting at least one user of the subset of users to be displayed on a device associated with at least one user of the subset of users, wherein the format of the user interface including the rendering of the video stream depicting at least one user is based on the settings of a user having an attendee role.

16. A computer-readable storage medium encoded with computer-executable instructions for performing a test session of a communication session for a plurality of users, the computer-executable instructions for causing the one or more processing units of a computing device to: Generate a meeting object that defines the communication session for the plurality of users, the meeting object defining a meeting start time and the role of an individual user among the plurality of users; Select a subset of users from the plurality of users, wherein, Individual users in the subset of the users are selected based on the individual users having a predetermined role assigned in the conference object; Use the conference object to generate a test session object that defines the test session for the subset of the users among the multiple users, the subset of the users being selected based on the predetermined role assigned for the subset of the users in the conference object, the test session object defining a test session start time before the conference start time; Measure one or more performance parameters during the test session, where the one or more performance parameters include at least one of the following: a network bandwidth parameter, an audio parameter, or a video parameter; Determine that the one or more performance parameters do not meet one or more threshold requirements; and Generate data in response to determining that the one or more performance parameters do not meet the one or more threshold requirements, the data defining an adjustment to one or more settings that control the performance parameters of the test session.

17. The computer-readable storage medium according to claim 16, wherein, The instructions further cause the one or more data processing units to: cause a display of the one or more performance parameters measured during the test session, where the display includes a graphical indicator of at least one of the following: the network bandwidth parameter, the audio parameter, or the video parameter.

18. The computer-readable storage medium according to claim 16, wherein, The instructions further cause the one or more data processing units to: store the adjustment to the one or more settings into the test session object that defines the test session.

19. The computer-readable storage medium according to claim 16, wherein, The instructions further cause the one or more data processing units to: cause an adjustment to one or more settings of the communication session based on the adjustment to the one or more settings during the test session.

20. The computer-readable storage medium according to claim 16, wherein, The instructions further cause the one or more data processing units to: Analyze organizational data to determine the location of users having a second predetermined role; Activate a remote computing device in a geographic region based on the location of the users having the second predetermined role, where The remote computing device simulates the activities of conference attendees from the geographic region for measuring the one or more performance parameters via a network connecting the system and the remote computing device; and Receive live telemetry data from the remote computing device based on the one or more performance parameters measured at the remote computing device located in the geographic region, where determining that the one or more performance parameters do not meet one or more threshold requirements includes using the live telemetry data from the remote computing device.

Images