Automation of license control for group messages

Abstract

A system provides permission control for group messages. The system can support user accessibility by using permissions indicating that the user has accessibility requirements with respect to the voice input. When a user is known to have an accessibility issue to provide voice input, a system data structure, such as a Teams directory, can indicate when a particular user has a special demand. The system can grant permission to them to prevent others from providing messages to message threads. This allows people to become the demonstrator of the conference while giving their presentation using message threads without disruption from others. The system may unlock the thread and allow others to provide messages when the user completes its presentation. By controlling the message thread in this way, the system may allow users with special requirements to participate as conference presenter.

Description

Background Art

[0001] Nearly half a billion people are deaf or hard of hearing. They are excluded from the hearing world because they cannot distinguish or identify the sounds in their environment. As a result, they miss out on experiencing the world in the same way as hearing people.

[0002] When it comes to communication programs such as Zoom, Slack, Teams, Google Hangouts, or Facebook Messenger, some systems may not provide a complete user experience for people with even a small degree of hearing loss. For example, some programs can process audio and video streams to allow meeting participants to talk and see each other. In addition, such systems can allow users to share text messages, which can be displayed in conjunction with the rendering of the audio and video streams. Although text messages and audio and video streams can be transmitted in real time, such technology may not provide the same experience for every user, especially when some users cannot speak or are deaf or hard of hearing.

[0003] As an illustrative example, consider a scenario in which a group of users are participating in a conference in which they share video streams, audio streams, and text messages. When one of the users requests the floor, for example, to enter presentation mode, some systems can modify permissions to allow that user to speak. The system can also mute the other users to allow that user to give a presentation without interruption from others. This feature set may generally be helpful for most users, but for users who cannot speak or are deaf or hard of hearing, existing systems for controlling audio and video streams may not provide an effective feature set that allows them to give a presentation. In some cases, users who cannot speak or are deaf or hard of hearing may rely on a messaging user interface to provide their presentation. In this case, when the user who cannot speak wants to give a presentation, the system may allow others to flood the messaging user interface with messages. Such systems fail to provide an effective communication forum for users who cannot speak.

[0004] These shortcomings can lead to ineffective interactions between computing devices and users. In addition, particularly in communication systems, the above-mentioned shortcomings of existing systems can lead to inefficient communication of information and loss of user participation. Computing devices that do not promote user participation or, worse, promote loss of user participation can lead to production losses and inefficiencies relative to multiple computing resources. For example, when a user becomes tired or idle, or when a user does not receive information in an effective manner, when shared content is missed or ignored, the user may need to refer to other resources such as documents, or use other forms of communication. When a viewer misses a significant point or clue during a live conference, the missing content may need to be resent. Such activities can lead to inefficient or reused use of networks, processors, memory or other computing resources. Therefore, there is a need to develop improvements to help make the user experience of communication systems more attractive and more like face-to-face meetings. In addition, there is still a need to develop improvements to help improve the accessibility features of communication systems. Summary of the Invention

[0005] The technology disclosed herein provides accessibility controls for group messages. To address some of the above issues and others, the system can support user accessibility by using a data structure that indicates that a user has accessibility requirements for providing voice input. In some configurations, system data structures such as the Teams directory can indicate when a specific user has a special requirement. For example, the Teams directory can indicate that a specific user has limited ability to provide voice input, limited ability to listen to audio signals, etc. The system can also receive input controls and access permissions that allow a meeting coordinator or administrator to designate a user as someone with accessibility requirements. The designation can be propagated to the Teams directory as a temporary designation for a predetermined time, such as the duration of a conference call. Using the Teams directory, the system can allow a user to control the access rights that other users have to provide messages to a message thread. When a user with the appropriate designation participates in a meeting with a specific role (e.g., a meeting presenter), the user can provide another input to lock the message thread so that only the user with the appropriate designation can provide messages to the thread and all other users are blocked from providing messages to the message thread. The user can invoke the presenter role by providing an input such as a virtual raise of the hand. When the user's presentation is complete, the system can unlock the thread for other users. The user can also provide subsequent input to unlock the thread for other users, for example by lowering their virtual hand. By controlling message threads in this way, the system can allow users with accessibility needs to participate in specific roles, such as conference presenters, using existing message threads or new message threads that exclusively allow the user to contribute messages.

[0006] The thread locking mechanism can be implemented in a variety of ways. In some embodiments, when an individual user is designated as a presenter, the system can prevent a group of users from providing messages to a particular message thread. In some configurations, when the system designates an individual user as requiring accessibility facilities, the system can prevent messages from being sent from the group of users. The system can determine that a user requires accessibility facilities for a data structure such as a Teams directory. When a user with that designation indicates that they are a presenter, the system can prevent others from providing messages to a particular thread. In some configurations, the system can prevent a group of users from providing messages by disabling text input functionality at the user interface for each user in the group of users.

[0007] In another example, the system can prevent a group of users from posting messages to a specific thread by receiving messages from the user and sending those messages to a supplementary thread. This allows presenters who require accessibility features to post messages to the main presentation thread while still allowing others to communicate in the supplementary thread. When a user exits presentation mode, for example, by lowering their virtual hand, the system can merge the messages in the supplementary thread with the main presentation thread. The supplementary thread can be in a separate user interface, or at least graphically distinguished from the main presentation thread. These examples are provided for illustrative purposes and should not be construed as limiting. Other mechanisms for restricting the communication of messages from attendees based on the accessibility status of presenter Kevin are contemplated and can be implemented in other ways. It is also contemplated that these techniques can be applied to group chats even without video, audio, or conferencing capabilities. Restricting the communication of messages also includes prohibiting the entry of messages or preventing the communication of messages by disabling the send button. Restricting the communication of messages also includes prohibiting the display of the message on the receiving computer, even if the receiving computer receives the message.

[0008] The present technology disclosure provides many technical benefits. For example, by providing a system that automates visual indicators to show active speakers, particularly in communication systems, users can exchange information more efficiently and gain a better understanding of the broader context of the meeting. Similarly, by viewing the actual video stream of the active speaker, rather than relying on a displayed avatar with a reduced size, users can easily benefit from viewing subtle gestures and body language. This helps to mitigate the occurrence of missing or ignoring shared content. This can reduce the occurrence of users needing to resend information. More efficient communication of shared content can also help avoid the need for external systems (such as mobile phones for text messaging and other messaging platforms). This can help reduce the reuse of network, processor, memory or other computing resources, especially when extended meetings or additional meetings can be avoided.

[0009] Features and technical benefits in addition to those explicitly described above will be apparent upon reading the following detailed description and reviewing the associated drawings. This summary is provided to introduce a selection of concepts in simplified form, which will be further described in the detailed description below. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to serve as an aid in determining the scope of the claimed subject matter. For example, the term "technology" may refer to systems, methods, computer-readable instructions, modules, algorithms, hardware logic, and / or operations permitted by the above context and the entire document. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The same reference numbers in different drawings represent similar or identical items. References to individual items in a plurality of items may use reference numerals with letters of the alphabetical sequence to refer to each individual item. Generic references to items may use specific reference numerals without the alphabetical sequence.

[0011] Figure 1A A first user interface arrangement showing multiple users contributing to a message thread is illustrated.

[0012] Figure 1B A first user interface arrangement is illustrated showing multiple users contributing to a message thread and one user providing input to invoke a state change of the system.

[0013] Figure 1C A second user interface arrangement is illustrated showing a feature for preventing a group of users from contributing to a message thread while allowing a presenter to contribute to the message thread.

[0014] Figure 2A Illustrated is a set of permissions that allow a first user to control message thread access for a group of users.

[0015] Figure 2B Illustrated is a set of permissions that allows a first user to control message thread access for a group of users and a second set of permissions that prevents the group of users from contributing messages to the message thread.

[0016] Figure 2C Illustrated is a set of permissions that allows a first user to control access to a message thread for a group of users and a second set of permissions that allows the group of users to contribute messages to the message thread.

[0017] Figure 3A A first user interface arrangement showing multiple users contributing to a message thread is illustrated.

[0018] Figure 3BA first user interface arrangement is illustrated showing multiple users contributing to a message thread and one user providing input to invoke a state change of the system.

[0019] Figure 3C A second user interface arrangement is illustrated showing features for preventing a group of users from contributing to a first message thread while allowing the group of users to contribute to a secondary thread and allowing the presenter to contribute to the first message.

[0020] Figure 3D Another user interface arrangement is illustrated showing features for merging messages of a first thread and a secondary thread when a user's presentation is complete.

[0021] Figure 4A Illustrated is a set of permissions that allow a first user to control message thread access for a group of users.

[0022] Figure 4B Illustrated is a first set of permissions allowing a first user to control message thread access for a group of users, a second set of permissions preventing the group of users from contributing messages to the first message thread, and a third set of permissions allowing the group of users to contribute to a second message thread.

[0023] Figure 4C Illustrated is a set of permissions that allow the system to merge messages from two separate message threads into a single updated thread when a presentation from a user is complete.

[0024] Figure 5A Illustrated is a first user interface arrangement showing multiple users contributing to a message thread, which is a first stage in an embodiment where a new thread is created for the first user to exclusively provide messages while other threads are able to write to the original thread.

[0025] Figure 5B A first user interface arrangement is illustrated showing multiple users contributing to a message thread and one user providing input to invoke a state change of the system.

[0026] Figure 5C A second user interface arrangement is illustrated showing features for preventing a group of users from contributing to a new message thread while allowing the group of users to contribute to the original thread and allowing the presenter to contribute to the new message thread.

[0027] Figure 5D Another user interface arrangement showing features for merging a new thread and messages of the original thread upon completion of a user presentation is illustrated.

[0028] Figure 6A Illustrated is a set of permissions that allow a first user to control message thread access for a group of users.

[0029] Figure 6B Illustrated are first permissions allowing a first user to control message thread access for a group of users, a second permission preventing the group of users from contributing messages to new message threads, and a third set of permissions allowing the group of users to contribute to the original message thread.

[0030] Figure 6C Illustrated is a set of permissions that allow the system to merge messages from two separate message threads into a single updated thread when a presentation from a user is complete.

[0031] Figure 7A A first user interface arrangement showing a plurality of users participating in a communication session is illustrated.

[0032] Figure 7B A first user interface arrangement is illustrated showing a plurality of users participating in a communication session and a first user providing input to invoke a state change of the system.

[0033] Figure 7C Illustrated is a user interface arrangement showing a message thread superimposed on an image of a user who is allowed to exclusively provide messages to the message thread, wherein the user interface arrangement shows images of other users.

[0034] Figure 7D A user interface arrangement is illustrated showing a message thread replacing an image of a user who is allowed to exclusively provide messages to the message thread.

[0035] Figure 7E A user interface arrangement is illustrated showing a message thread superimposed on an image of a user who is allowed to exclusively provide messages to the message thread.

[0036] Figure 7F A user interface arrangement is illustrated showing a message thread superimposed on an image of content presented together with an image of a user permitted to exclusively provide messages to the message thread.

[0037] Figure 7G Illustrated is a user interface arrangement displayed from the perspective of a user permitted to contribute exclusively to a message thread during a second operating mode that restricts other users from contributing to the message thread.

[0038] Figure 7H The illustration shows a user interface arrangement displayed from the perspective of a user, wherein the user is allowed to contribute exclusively to a first message thread during a second operating mode, wherein the second operating mode restricts other users from contributing to the first message thread, and the display of a second message thread, which accepts and displays messages from other users.

[0039] Figure 8Illustrated are aspects of a process for displaying messages within a communication session user interface.

[0040] Figure 9 is a flow diagram illustrating aspects of a routine for controlling permissions for a message thread based on the operating state of the system.

[0041] Figure 10 is a computer architecture diagram illustrating an illustrative computer hardware and software architecture for a computing system capable of implementing various aspects of the techniques and technologies presented herein.

[0042] Figure 11 is a computer architecture diagram illustrating a computing device architecture of a computing device capable of implementing various aspects of the techniques and technologies presented herein. DETAILED DESCRIPTION

[0043] Figures 1A-1C The diagram illustrates an example of a scenario in which the system allows a user to control permissions for other users' message threads. In this example, multiple people are engaged in a communication session involving the communication of text messages via message threads as well as audio and video streams. The communication session can be managed by a system 100 comprising multiple computers 11, each corresponding to multiple users 10. The technology disclosed herein can operate on a single device or on a computing platform utilizing a combination of computers and servers as described herein.

[0044] In this example, a first user 10A, Serena Davis, is associated with a first computer 11A, a second user 10B, Miguel Silva, is associated with a second computer 11B, a third user 10C, Krystal McKinney, is associated with a third computer 11C, a fourth user 10D, Jazine Simmons, is associated with a fourth computer 11D, a fifth user 10E, James Smith, is associated with a fifth computer 11E, a sixth user 10F, Traci Isaac, is associated with a sixth computer 11F, a seventh user 10G, MJ Price, is associated with a seventh computer 11G, an eighth user 10H, Jessica Kline, is associated with an eighth computer 11H, a ninth user 10I, Kat Larsson, is associated with a ninth computer 11I, a tenth user 10J, Stella Jones, is associated with a tenth computer 11J, an eleventh user 10K, Tim Johnson, and a twelfth user 10L, Brett Michael, are associated with a twelfth computer 11L. The users can also be referred to as "User A," "User B," "User C," etc., respectively. Each user can be displayed as a two-dimensional (2D) image 151 in the user interface. The 2D image 151 can be a static image or a video stream generated by a camera pointed at the user. The users can also communicate using real-time audio streaming. The computer can be in the form of a desktop computer, a head-mounted display unit, a tablet computer, a mobile phone, etc.

[0045] The system can generate a user interface that shows aspects of the communication session with each user participating in the communication session. In this example, the user interface arrangement 101 is initially displayed with a rendering of a user with a message thread 111 having individual messages 112. The message thread 111 in the message area 109 can be displayed simultaneously with the video area 110 having the user's image.

[0046] In a first mode of operation, each of the computers for each user can contribute messages to the message thread 111. In a second mode of operation, select computers (e.g., users with appropriate permissions) can contribute messages to the message thread 111, while also having controls to restrict other computing devices from contributing messages to the message thread. In some configurations, the system can operate using the first mode of operation. This mode of operation can also cause the system to be used as described in the following example: Figure 1A The first user interface arrangement shown in . In this particular example, the example user interface can be displayed on the devices of a group of users, eg, user B through user L, using devices 11B-11L.

[0047] During an event such as an online meeting, a user such as the first user 10A can provide input to change the state of the meeting or change the operating state of the system. In one illustrative example, the input can be in the form of keyboard input, voice command, gesture captured by a camera, or any other suitable input. In a specific example, the input can be a gesture performed by the user, such as raising a hand. When the system detects that a person has raised their hand a predetermined distance, the system can determine it as a valid input for invoking a state change in the system or an interrupt in the invocation event. In some combination, the input of raising a hand can invoke an accessibility feature. Invoking an accessibility feature can involve the system converting a first type of operation using a first set of permissions to a second type of operation using a second set of permissions. In some configurations, the input can be referred to as a qualified input in this article. A qualified input can be any input including a predetermined gesture (such as raising a hand), or an input involving a predetermined key input, etc.

[0048] exist Figure 1B In an example, in response to input, such as a hand-raising gesture performed by a user, the system can generate a graphical element 152 acknowledging receipt of the input. This graphical element can notify the other users 10B-10L that the first user 10A wants to share information, for example, to give a presentation. In one illustrative example, the system can receive input associated with the user 10A, also referred to herein as a qualifying input. Input is received to initiate a change in the operating state of the system 100. The system can be configured to cause the system 100 to transition from a first type of operation 788 to a second type of operation 789, wherein the first type of operation 788 allows multiple computing devices 11A-10L associated with the multiple users 10A-10L to contribute messages 112 to the message thread 111. The second type of operation 789 allows the computing device 11A of the user 10A to contribute messages 112 to the message thread 111, while restricting the other computing devices 11B-11L of the other users 10B-10L from contributing messages 112 to the message thread 111.

[0049] In response to input for initiating an operating state change, the system is able to analyze permission data to determine that user 10A associated with the input is designated to invoke an operating state change of the system 100 from a first class of operation to a second class of operation, which allows computing device 11A of user 10A to contribute messages 112 to message thread 111 while restricting other computing devices 11B-11L of other users 10B-10L from contributing messages 112 to message thread 111.

[0050] In response to determining that the permission data indicates that the user 10A associated with the input is designated to invoke a change in the operating state of the system 100 from utilizing a first class of operations 788 to utilizing a second class of operations, the system is able to invoke a change in the operating state of the system 100 from utilizing a first class of operations to utilizing a second class of operations, wherein the first class of operations 788 allow multiple computing devices 11A-10L associated with multiple users 10A-10L to contribute messages 112 to the message thread 111, and wherein the second class of operations 789 allow the computing device 11A of the user 10A to contribute messages 112 to the message thread 111 while restricting other computing devices 11B-11L of other users 10B-10L from contributing messages 112 to the message thread 111.

[0051] As in Figure 1C As shown in , one way to restrict other computing devices 11B-11L of other users 10B-10L from contributing messages 112 to message thread 111 is to prevent computing devices 11B-11L from sending messages. This can be done internally, where permission is sent to the client. Additionally or alternatively, one or more computers can disable at least one of a text input field 112 or a send button 113 of a user interface displayed on the other computing devices 11B-11L. The text input field or send button of the user interface can remain disabled until a second input is received from user 10A, where the second input can be configured to initiate a second operational state change from utilizing the second type of operation to utilizing the first type of operation.

[0052] Now refer to Figures 2A-2C , showing a set of permissions 152. Figure 2A In the example of FIG. 1 , each of up to N users has a first set of accessibility permissions 250 and a second set of thread permissions 251. The first set of accessibility permissions allows a user to control the thread permissions of other users. For example, a first user has accessibility permissions that allow first user 10A to control thread permissions 251 for other users. Meanwhile, other users 10B-10N do not have the ability to control the thread permissions of other users. In this first state of permissions, each user has the ability to read and write to the main thread 111. Also in this example, any portion of the permissions 152 can be transferred to individual client computing devices 11 to allow each of the computers to comply with the permissions they have modified throughout the process.

[0053] Thus, in response to the qualifying input from the first user, the system can analyze the permissions 152 and determine that the first user is designated as having the ability to control the operating state of the system, which also means that they have the ability to change the thread permissions of other users. In this case, the example shows that the first user has accessibility with respect to audio, for example, they may have an impairment with respect to the ability to speak or an impairment with respect to the ability to hear. When such a need exists, even temporarily, the system can analyze these permissions to determine whether there is an impairment and react accordingly. In this example, the system can determine that the user is unable to speak and, in such a case, is unable to provide an audio stream for the presentation. In this case, the system can allow the user to provide the presentation using a chat thread that is not interrupted by messages sent from other users.

[0054] Figure 2B The diagram illustrates a change in permissions in response to a first user providing input indicating they would like to give a presentation. Because that user has permissions to control the operating state of the system, e.g., accessibility permissions indicating the first user has "allowed" control permissions, the system responds to the input by changing the thread permissions for the other users, User B through User N, to restrict those other users from writing to the main thread 111. In this example, those other users have the ability to read messages from the main thread. In this example, the first user also retains the ability to read and write to the main thread.

[0055] As in Figure 2C , once the first user 10A is finished with their presentation or they provide a second input indicating completion of the presentation, the system is able to change the permissions back to the original state of allowing other users to contribute messages to the main thread 111. This transition back to the original state can occur in response to a number of different events. For example, the first user's presentation mode may only last for a predetermined period of time, and at the end of that period of time, the system can restore the permissions back to the original state. The system is also able to determine when a user has completed a presentation. For example, if the first user stops providing text messages for a predetermined period of time, the system can determine that the user has completed their presentation and can restore the permissions back to the original state, as in Figure 2C As shown in .

[0056] Figures 3A-3D Another embodiment is illustrated in which a first user can restrict other users from contributing to the main thread, and also allow those other users to contribute to the auxiliary thread. Figure 3A As shown in , the system provides a user interface that illustrates rendering of multiple users, message threads 111, input text fields 112, and text send buttons 113. Figure 3B As shown in , a first user provides qualifying input, and the system generates a graphical element 152 indicating receipt of the input. Figure 3C In response to the qualified input, the system can generate a second message thread 115, which is reserved for messages from other devices 11B-11L of other users 10B-10L. The second message thread 115 can also have a corresponding input text box 112' and a send button 113'. The second message thread 115 can also be displayed in the second message area 115.

[0057] In a first operating state of the system, such as Figure 3A As shown in , each of the users is able to provide messages to the main thread 111. However, since the first user qualifies as a user who can change the operating state of the system, the system can change to the second operating state in response to a predetermined input by the first user (e.g., raising a hand). Once in the second operating state, the other devices 11B-11L of the other users 10B-10L are restricted from contributing messages to the main thread 111, while the first device 11A of the first user 10A is allowed to provide messages to the main thread 111. In the second operating state, the other devices 11B-11L of the other users 10B-10L are allowed to provide messages to the second thread 115.

[0058] Once user 10A has completed his presentation, as in Figure 3D As shown in , the system can be restored back to the first operating state and allow other devices 11B-11L of other users 10B-10L to contribute to the main thread 111. In addition, the system can be merged in Figure 3D Messages for both the first and second threads are shown in .

[0059] Figures 4A-4C As shown above about Figures 3A-3D The entire process described is subject to change. In this case, as in Figure 4A As shown in , the first user has the ability to call a change in the operating state of the system. Other users do not have the appropriate accessibility permissions to change the operating state of the system. In this state of permission, each user has the ability to read and write to the main thread 111. As shown in Figure 4B As shown in , in response to a first user providing input that changes the operating state of the system, the system can modify the thread permissions of the other users 10B-10N and restrict those users from contributing to the main thread, but at the same time give those users permission to contribute messages to the auxiliary thread 115. In this state of permissions, the first user is the only user who can write messages to the main thread. Once the users have completed their presentations, as in Figure 4C As shown in , the system can change the state of the thread permissions back to their original state and allow all users to contribute to the main thread. In addition, the system can merge the main thread and the auxiliary thread into a new thread 111 that can also be displayed to all users.

[0060] Although the examples described above illustrate examples in which a thread is used by all users of a communication session and is then modified during a presentation to allow only one user to contribute messages to the thread, the system is capable of using other UI arrangements to allow a presenter to use a message thread for a presentation. For example, during a first operating mode, the system may begin a communication session, such as an online meeting, using a first thread. In this first operating mode, the system may allow all users of the session to contribute messages to the first thread. Then, during a presentation provided by the first user, for example, during a second operating mode, the system may allow other users to continue using the main thread, but then create a second thread that can be used exclusively by the first user with appropriate accessibility permissions to create a new thread. This allows the first user to provide a presentation using a message thread that is not interrupted by messages from other users. Accessibility permissions (e.g., users with permission control) only allow specific users to create separate threads for presentations. This prevents a situation where the system is overwhelmed by a large number of threads, which can lead to confusion and inefficiency with respect to computing resources. Examples of this embodiment are described in

[00145] Figures 5A-5D and Figures 6A-6B Shown in.

[0061] Figures 5A-5D Another embodiment is illustrated in which a first user is able to restrict other users from contributing to a new thread 115 (also referred to herein as a secondary thread 115) and also allow those other users to contribute to the original thread 111 (also referred to herein as the main thread 111). Figure 5A As shown in , the system provides a user interface that illustrates rendering of multiple users, an original message thread 111, an input text field 112, and a text send button 113. Figure 5B As shown in , a first user provides qualifying input, and the system generates a graphical element 152 indicating receipt of the input. Figure 5C In response to the qualified input, the system can generate a new message thread 115 of the message reserved for the device 11A of the first user 10A, while the original thread is reserved for the other devices 11B-11L of the other users 10B-10L. The new message thread 115 can also have a corresponding input text box 112' and a send button 113'.

[0062] In a first operating state of the system, such as Figure 5A, each of the users is able to provide messages to the main thread 111. However, since the first user qualifies as a user who can change the operating state of the system, the system can change to the second operating state in response to a predetermined input by the first user (e.g., raising a hand). Once in the second operating state, the other devices 11B-11L of the other users 10B-10L are restricted from contributing messages to the new thread 115, while the first device 11A of the first user 10A is allowed to exclusively provide messages to the new thread 115. In the second operating state, the other devices 11B-11L of the other users 10B-10L are allowed to provide messages to the original thread 111.

[0063] Once user 10A has completed his presentation, as in Figure 5D As shown in , the system can be restored back to the first operating state, wherein the system shows only the original thread 111. In addition, the system can be merged in Figure 5D Messages for both the new thread and the original thread are shown in .

[0064] Figures 6A-6C Shown in the above Figures 5A-5D The entire process described here is subject to change. In this case, as in Figure 6A As shown in , the first user has the ability to call a change in the operating state of the system. Other users do not have the appropriate accessibility permissions to change the operating state of the system. In this state of permission, each user has the ability to read and write to the original thread 111 (e.g., the main thread). As shown in Figure 6B As shown in , in response to a first user providing input to change the operating state of the system, the system can modify the thread permissions of the other users 10B-10N and restrict those users from contributing to the new thread 115, but at the same time give those users permission to contribute messages to the original thread 111. In this state of permissions, the first user is the only user who can write messages to the original thread. Once the user has completed his presentation, as shown in Figure 6C As shown in , the system is able to change the state of the thread permissions back to their original state and only show the original thread and allow all users to contribute to the original thread.

[0065] Now go to Figures 7A-7H , other user interface arrangements are shown and described below. For example, Figure 7A A user interface showing a rendering of individual users participating in a communication session is shown. This particular user interface initially does not show message threads. Then, as shown in Figure 7B As shown in , when the first user 10A provides input indicating that the user is interested in sharing information, for example, by showing a hand-raising gesture, the system generates a graphical element 152 to notify other users that the system is about to change its operating state. Figure 7CAs shown in , when the system transitions from a first operating state to a second operating state, the system begins generating messages of a thread that is at least partially superimposed on the image of the user providing input. For example, the message thread can be an existing message thread 111 that initially allows all users to provide messages when the system is operating in the first operating state, but then only allows the first user to provide messages when the system is operating in the second operating state. In another embodiment, the message thread superimposed on the user can also be a new message thread 117 that only accepts messages from the user providing input (e.g., the first user). In this case, if the system creates a new message thread 117 that only allows the first user to provide messages, other users 10B-10E can provide messages to the existing thread (not shown). The new message thread 117 can be in the new display area 117. Figure 7D Another example of a user interface arrangement is illustrated in which a message thread that accepts only messages from a first user is rendered in place of the first user's rendering. Figure 7E A further example of a user interface arrangement is illustrated showing a zoomed-in area within the video display area 110 reserved for a first user's image. In this embodiment, a message thread is superimposed on the zoomed-in image of the user. Figure 7F Another example of a user interface arrangement is illustrated showing a zoomed-in area within the video display area 110 reserved for content shared by a first user. In this embodiment, a message thread is superimposed on an image of the shared content.

[0066] Figure 7G and 7H The user interface is illustrated from the perspective of a first user. Figure 7G In the example of , the message thread is not superimposed on the content, and the user interface also provides an input text field 112 and an input element 113 configured for typing a message in the text field to send to the thread. Figure 7H In the example of FIG, the user interface shows a first message thread 111 configured to only allow messages from a first user, and a second message thread 115 configured to accept messages from all users. This allows the first user to provide a coherent presentation during the second operating mode without allowing other users to interrupt the message thread. At the same time, the user interface arrangement allows other users to continue sharing messages on a separate thread without interrupting the first user.

[0067] For the purpose of illustration, consider a conference scenario using Teams, Slack or Google Hangout, where user A is unable to provide voice input. User A, the first user 10A, raises his hand, and when user A has the floor, the system allows the user to rely on using the IM chat window to convey his ideas in real time using text. Alternatively, the system can generate a separate workspace and allow the user to input text and other content into the whiteboard area of ​​Teams. For conference calls with many people (e.g., 10 or more people), there is a potential interruption from other messages received by other users that may flood the chat window during user A's presentation. User A is typing and using several chats, but is interrupted by someone else's IM. Therefore, the system only allows user A to provide messages to the IM chat or the new whiteboard area.

[0068] To support accessibility in the above scenario, user A must be aware of the accessibility issues with voice. That is, a designation in the Teams directory shows that user A has special needs. Alternatively, the coordinator / administrator can designate user A as a person with special needs, and the designation is propagated to the Teams directory as a temporary designation for the duration of the conference call. When user A has the floor, text entered in the chat window or in other workspaces on the Teams client (e.g., a whiteboard) is shown using a variety of different configurations. For example, as in Figure 7D As shown in , the text streaming video can replace the video of user A. Figure 7C and Figure 7F As shown in , the text streaming video can be rendered adjacent to or alongside the video of user A. Figure 7E As shown in , the system can increase the display area reserved for user A to obtain a better user experience with video and text side by side or partially overlapping using semi-transparent text video streams. Figure 7F As shown in , the system can also allow users to display shared content as part of a presentation as well as display a message thread dedicated to only allowing user A to provide messages.

[0069] The technology behind the embodiments described above that utilize text streams is based on a system configured to convert a text stream into a video stream. This conversion can occur locally on the client computer. Alternatively, as in Figure 8As shown in , the server 602 can handle the conversion. In a server embodiment, the system is configured to transmit text messages from each client computer to the server. The server can then convert the text message and render it into a video stream. And in some configurations, the system can utilize a video stream codec such as H.264. The video stream is then overlaid or combined with the video of the user or shared content and transmitted back to the client to be displayed to the user.

[0070] Figure 9 is a diagram illustrating aspects of a routine 700 for providing automatic control of permissions for a message thread. Those skilled in the art will appreciate that the operations of the methods disclosed herein are not necessarily presented in any particular order, and that it is contemplated that some or all operations may be performed in an alternative order. For ease of description and illustration, the operations are presented in the order presented. Operations may be added, omitted, performed, and / or performed simultaneously without departing from the scope of the appended claims.

[0071] It should also be understood that the illustrated method can be started or ended at any time and does not require its entirety to be performed. Some or all of the operations of the method and / or substantially equivalent operations can be performed by executing computer-readable instructions contained on a computer storage medium, as defined herein. As used in the specification and claims, the term "computer-readable instructions" and its variants are broadly used herein to include routines, applications, application modules, program modules, programs, components, data structures, algorithms, and the like. Computer-readable instructions can be implemented on various system configurations, including single-processor or multi-processor systems, minicomputers, mainframe computers, personal computers, handheld computing devices, microprocessor-based programmable consumer electronics, combinations thereof, and the like. Although the example routines described below operate on a system (e.g., one or more computing devices), it can be appreciated that the routines can be executed on any computing system, which may include any number of computers working together to perform the operations disclosed herein.

[0072] Therefore, it should be appreciated that the logical operations described herein are implemented as a sequence of computer-implemented actions or program modules running on a computing system such as that described herein and / or as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice depending on the performance and other requirements of the computing system. Therefore, the logical operations may be implemented in software, firmware, dedicated digital logic, and any combination thereof.

[0073] In addition, Figure 9The operations illustrated in the figures and other figures can be implemented in association with the example user interfaces and systems described herein. For example, the various devices and / or modules described herein can generate, send, receive, and / or display data associated with the content of a communication session (e.g., live content, broadcasted events, recorded content, etc.) and / or a rendering UI including one or more participants of a remote computing device, avatars, channels, chat sessions, video streams, images, virtual objects, and / or applications associated with the communication session.

[0074] The routine 700 includes operation 703, in which the system causes display of a first user interface arrangement including an image rendering of a user and a message thread. Although the examples disclosed herein provide a video rendering of a user, it will be appreciated that the techniques disclosed herein can be applied to a user interface displaying a message thread configured to control which messages can be posted based on a user identity and associated permissions.

[0075] At operation 705, the system receives input associated with user 10A. The user input can include the user's identity. The user input can indicate that the user wants to share information or provide a presentation. The input can invoke a state change within the device or cause an analysis of permissions to determine whether the permissions should be changed in response to the user input.

[0076] In some configurations, an input is received for initiating a change in the operational state of the system 100. The input causes the system 100 to transition from a first type of operation 788 to a second type of operation 789. The first type of operation 788 allows a plurality of computing devices 11A-10L associated with a plurality of users 10A-10L to contribute messages 112 to the message thread 111. The second type of operation 789 allows the computing device 11A of the user 10A to contribute messages 112 to the message thread 111 while restricting the other computing devices 11B-11L of the other users 10B-10L from contributing messages 112 to the message thread 111.

[0077] At operation 707, wherein the system can analyze the accessibility permissions to determine whether the user providing the input can modify the thread permissions of other users and prevent other users from contributing messages to the thread. As shown in the examples provided herein, a user, such as a first user, can have accessibility permissions to modify the thread permissions of other users. If the user does not have the correct accessibility permissions, the system can deny the user permission to change the thread permissions of other users. However, if the user does have the appropriate accessibility permissions, the process can continue with the user or the user being able to change the thread permissions of other users. In some cases, a user can provide input to prevent other users from contributing messages to a particular message thread. In Figure 2BIn the example of , the first user can provide input that changes the thread permissions of other users. In this example, since the second user does not have control permissions, the second user cannot provide input that changes the thread permissions of other users.

[0078] At operation 709, the system can lock other users from contributing messages to the thread while allowing the user associated with the input to contribute messages to the message thread. The system only allows the user to block others from contributing messages to the thread if the user has appropriate accessibility permissions. In some configurations, the thread can be locked by modifying user interface features to prevent users from entering text messages into the computing device. In other embodiments, the system can prevent the client computer from sharing text messages with a server or other client computers.

[0079] At operation 711, the system may unlock the thread and allow other users to contribute messages to the thread in response to one or more events. In some configurations, when the presenter has completed the presentation, the system may unlock the thread and allow other users besides the presenter to contribute to the thread. In other embodiments, the thread may be unlocked after other events, such as receiving an auxiliary input indicating that the user does not intend to share additional information. The auxiliary input may also be configured to change the operating state of the system, for example, the closet computing device switches from utilizing the second type of operation to utilizing the first type of operation. Operation 711 may also include other activities that cause the system to revert to the original permissions. For example, regarding Figure 2A For example, the system may start with an original set of thread permissions and allow everyone to contribute to a particular thread. After a user provides a first input indicating an intent to share information or give a presentation, the permissions are transferred to a second set of permissions that restrict other users from contributing to the message thread. However, after a predetermined event, the system can revert back to the original message thread permissions. Pretty Chair events can also include an intent to stop sharing. For example, if a user stops typing for a predetermined period of time, the system can determine that the user has completed their presentation. Additionally, if the user types a specific keyword or combination of keywords, the system can determine that the user intends to complete the presentation. Upon completion of the added routine, the system can also revert the user's accessibility permissions back to their original settings. For example, if a user had a control feature in their accessibility permissions for the duration of a meeting, the system can change the accessibility permissions for that user to a setting, such as a "no control" setting, after the meeting has ended. This means that the user only has the ability to change other users' thread permissions during the meeting and not after an event, such as the meeting.

[0080] Routines can include operations combining features described herein. Figures 1A-1C, 2A-2C, 3A-3D, and 4A-4C: illustrate operations for receiving input for a hand raise accessibility feature for dynamically modifying a conference chat UI when a speech impaired user has the right to speak. The system is capable of executing a method for controlling permissions to a message thread (111) for a virtual conference session (604) between a plurality of users (10A-10L), the method being executed on a data processing system (100) configured to operate in a first class of operations (788) and a second class of operations (789). As in Figure 1B 、 3B As shown in , wherein the system receives a "raise hand" input for invoking an accessibility feature, referred to herein as a "second type of operation". A method can include: receiving a text message input by a plurality of computing devices (11A-10L) associated with users (10A-10L) and directed to a message thread; providing a first user interface in the first type of operation, wherein the first user interface includes an attendee display area (110) displaying a representation or video of at least one of the users and a text message UI area (109) displaying the received text message; receiving an input associated with a user (10A) of a virtual conference session, the input including a command for initiating an operational state change of the system (100) to transition from the first type of operation (788) to the second type of operation (789). As shown in Figure 2A and 4A As shown in , the system is capable of analyzing the Teams directory ("permissions") to determine whether a user is eligible to access accessibility features. For example, in response to receiving an input for initiating an operational state change, analyzing the permission data (152) to determine whether the user privileges associated with the user (10A) associated with the input satisfy the permission requirements for invoking the operational state change of the system (100) from a first type of operation (788) to a second type of operation (789); and as shown in Figure 1C and Figure 2CAs shown in: based on accessibility permissions, in response to determining that the permission data (152) indicates that the user privileges of the user (10A) associated with the received input meet the permission requirements, while allowing the primary user to contribute messages, locking other users from contributing messages to the message thread, causing the operating state of the system (100) to transition from a first type of operation (788) to a second type of operation (789) by: changing from providing a first user interface to providing a second user interface in the second type of operation, wherein the second user interface handles displaying the text message received from the user (11A) differently from the text messages from the other computing devices (11B-11L) by: Displaying text messages added by a user (10A) to a message thread (111) in a message UI area (109) while restricting at least one computing device among other computing devices (11B-11L) of other users (10B-10L) from contributing text messages (112) to the message thread (111); or providing in a second user interface (a) a first text message display area (111 / 115 / 117) for displaying text messages received from the user (10A), and (b) a second text message display area different from the first text message display area for displaying text messages added to the message thread (111) by other users (10B-10L).

[0081] Figures 1A-1C 2A-2C show an embodiment of a UI for blocking other people during a user presentation. The blocking remains until the first user exits presentation mode. The system is capable of limiting other computing devices of other users from contributing messages to the message thread by disabling at least one of a text input field or a send button of a user interface displayed on the other computing device, wherein the text input field or the send button of the user interface remains disabled until a second input is received for initiating a change in a second operating state from utilizing a second type of operation to utilizing a first type of operation, for providing in a second user interface (a) a first text message display area (111 / 115 / 117) for displaying text messages received from the user (10A), and (b) a second text message display area different from the first text message display area for displaying text messages added to the message thread (111) by other users (10B-10L).

[0082] Figures 3A-3C 4A-4C show an embodiment of the second queue. The method herein also includes a first text message display area (111 / 115 / 117) (10A) for displaying text messages received from a user blocking the display of messages received from other users.

[0083] Figure 3DAn embodiment is shown in which messages from the auxiliary thread are merged after presentation. The method herein also includes: responsive to receiving a second input for initiating a second operational state change from utilizing the second type of operation to utilizing the first type of operation, merging the message from the auxiliary thread with the message of the main thread.

[0084] Figure 5C An embodiment is shown in which messages are displayed in association with an image of a user in a grid mode.The method herein also includes causing messages received from a user's computing device to be displayed as an overlay to a video rendering of the user when the system is utilizing the second type of operation.

[0085] Figure 5D An embodiment is shown in which messages replace images of users in a grid pattern. In the method, when the system is utilizing a second type of operation, messages received from a user's computing device are caused to be displayed in a user interface arrangement including multiple renderings of multiple users' images, wherein the messages received from the user's computing device are displayed as replacements for the users' renderings.

[0086] The system also manages permissions that prevent a user from blocking other users if the other users do not have appropriate accessibility permissions.The method may also include preventing the system from invoking an operational state change of the system from a first class of operation to a second class of operation in response to determining that the permissions data does not specify that the user has permission to invoke the operational state change.

[0087] In other embodiments, Figures 1A-1C , 2A-2C, 3A-3D, and 4A-4C illustrate other aspects of a system capable of receiving input for a hand raise accessibility feature for dynamically modifying a conference chat UI when a speech-impaired user has the floor. A method for controlling permissions for a message thread (111) for a communication session (604) can include an operation for receiving input associated with a user (10A), the input being received for initiating an operational state change of the system (100), wherein the input causes the system (100) to transition from a first type of operation (788) that allows a plurality of computing devices (11A-10L) associated with a plurality of users (10A-10L) to contribute messages (112) to the message thread (111) to a second type of operation (789), the first type of operation (788) allowing a computing device (11A) of the user (10A) to contribute messages (112) to the message thread (111) while restricting other computing devices (11B-11L) of other users (10B-10L) from contributing messages (112) to the message thread (111). The operation can be Figure 1B and 3B, wherein the system receives a "raise hand" input for invoking an accessibility feature, referred to herein as a "second class operation." Next, in response to the input for initiating an operational state change, the permission data (152) is analyzed to determine that the user (10A) associated with the input is designated to invoke an operational state change of the system (100) from a first class operation (788) to a second class operation (789), which allows the computing device (11A) of the user (10A) to contribute messages (112) to the message thread (111) while restricting other computing devices (11B-11L) of other users (10B-10L) from contributing messages (112) to the message thread (111). This can be Figure 2A and 4A , where the system is able to analyze the Teams directory ("permissions") to determine whether a user is eligible to access accessibility features. Then, in response to determining that the permission data (152) indicates that the user (10A) associated with the input is designated to invoke a change in the operating state of the system (100) from utilizing a first class of operations (788) to utilizing a second class of operations (789), the system (100) invokes a change in the operating state of the system from utilizing the first class of operations (788) to utilizing the second class of operations (789), wherein the first class of operations (788) allows a plurality of computing devices (11A-10L) associated with the plurality of users (10A-10L) to contribute messages (112) to the message thread (111), and wherein the second class of operations (789) allows the computing device (11A) of the user (10A) to contribute messages (112) to the message thread (111) while restricting other computing devices (11B-11L) of other users (10B-10L) from contributing messages (112) to the message thread (111). As in Figure 1C and Figure 2C As shown in , based on accessibility permissions, while the main user is allowed to contribute messages, other users are locked out from contributing messages to the message thread.

[0088] As in Figures 1A-1C 2A-2C, specifically an embodiment of blocking the UI of other persons during a user's presentation, wherein the blocking remains until the first user exits presentation mode. Here, the method can include limiting other computing devices of other users from contributing messages to the message thread including: disabling at least one of a text input field or a send button of a user interface displayed on the other computing device, wherein the text input field or the send button of the user interface remains disabled until a second input is received for initiating a change in a second operating state from utilizing the second type of operation to utilizing the first type of operation.

[0089] As in Figures 3A-3Cand 4A-4C, a specific embodiment of adding a second queue, wherein the message thread is a main thread, wherein restricting other computing devices of other users from contributing messages to the message thread includes: restricting other computing devices of other users from sending messages to the main thread; generating an auxiliary thread reserved for receiving messages from other computing devices of other users; and causing messages received from other computing devices of other users within the auxiliary thread to be displayed, wherein the auxiliary thread is displayed on the user interface simultaneously with the main thread.

[0090] As in Figure 3D As shown in , messages from the auxiliary thread can be merged after the demonstration. The method can include merging the messages from the auxiliary thread with the messages of the main thread in response to receiving a second input for initiating a second operating state change from utilizing the second type of operation to utilizing the first type of operation.

[0091] As in Figure 5C , displaying the messages in association with the user's image in a grid pattern as shown in . The method can include: when the system is utilizing the second type of operation, causing messages received from the user's computing device to be displayed as an overlay on the user's video rendering.

[0092] As in Figure 5D As shown in , the message can replace the image of the user in a grid pattern. The method can include utilizing a second type of operation such that the message received from the user's computing device is displayed in a user interface arrangement including multiple image renderings of multiple users, wherein the message received from the user's computing device is displayed as a replacement for the user's rendering.

[0093] The permissions can be used to prevent a user from blocking other users if the other users do not have appropriate accessibility permissions. For example, the method can prevent the system from invoking an operational state change of the system from a first class of operation to a second class of operation in response to determining that the permissions data does not specify that the user has permission to invoke the operational state change.

[0094] Figure 10 6 is a diagram illustrating an example environment 600 in which a system 602 can implement the techniques disclosed herein. It should be appreciated that the subject matter described above can be implemented as a computer-controlled device, a computer process, a computing system, or an article of manufacture such as a computer-readable storage medium. The operations of the example methods are shown in individual blocks and are summarized with reference to these blocks. The methods are illustrated as a logical flow of blocks, each of which can represent one or more operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, enable the one or more processors to perform the recited operations.

[0095] Generally, computer-executable instructions include routines, programs, objects, modules, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as limiting, and any number of the described operations can be performed in any order, combined in any order, subdivided into multiple sub-operations, and / or performed in parallel to implement the described processes. The described processes can be performed by resources associated with one or more devices, such as one or more internal or external CPUs or GPUs, and / or one or more hardware logic, such as a field programmable gate array ("FPGA"), a digital signal processor ("DSP"), or other types of accelerators.

[0096] All of the methods and processes described above can be embodied in software code modules executed by one or more general-purpose computers or processors and fully automated. The code modules can be stored in any type of computer-readable storage medium or other computer storage device, such as those described below. Some or all of the methods described can alternatively be embodied in dedicated computer hardware, such as the computer hardware described below.

[0097] Any routine description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code, which includes one or more executable instructions for implementing the specific logical functions or elements in the routine. Alternative implementations are included within the scope of the examples described herein, where elements or functions can be deleted or performed from the order shown or discussed, including substantially simultaneously or in reverse order, depending on the functionality as will be understood by those skilled in the art.

[0098] In some implementations, system 602 can be used to collect, analyze, and share data displayed to users of a communication session 604. As illustrated, a communication session 603 can be implemented between a plurality of client computing devices 606(1) through 606(N) (where N is a number having two or greater values) that are associated with or are part of system 602. Client computing devices 606(1) through 606(N) enable users, also referred to as individuals, to participate in communication session 603.

[0099] In this example, a communication session 603 is hosted by a system 602 over one or more networks 608. That is, the system 602 can provide services that enable users of client computing devices 606(1) to 606(N) to participate in the communication session 603 (e.g., via live viewing and / or recorded viewing). Thus, "participants" of the communication session 603 can include users and / or client computing devices (e.g., multiple users can be in a room participating in a communication session using a single client computing device), each of which can communicate with the other participants. Alternatively, the communication session 603 can be hosted by one of the client computing devices 606(1) to 606(N) using peer-to-peer technology. The system 602 can also host chat conversations and other team collaboration functionality (e.g., as part of an application suite).

[0100] In some embodiments, such chat conversations and other team collaboration functions are considered external communication sessions distinct from communication session 603. The computing system 602 that collects participant data in communication session 603 may be able to link to such external communication sessions. Thus, the system can receive information that enables connection to such external communication sessions, such as date, time, session details, etc. In one example, a chat conversation can be conducted based on communication session 603. Additionally, system 602 can host communication session 603 that includes at least a plurality of participants co-located at a meeting location (such as a conference room or auditorium) or located at different locations.

[0101] In the examples described herein, client computing devices 606(1) to 606(N) participating in a communication session 603 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live content and / or recorded content. The collection of various instances or streams of live content and / or recorded content can be provided by one or more cameras (such as video cameras). For example, individual streams of the live or recorded content streams can include media data associated with a video feed provided by a video camera (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). In some embodiments, the video feed can include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images can also include one or more avatars.

[0102] Another example of an individual stream of live or recorded content can include media data including avatars of users participating in the communication session and audio data capturing the users' voices. Yet another example of an individual stream of live or recorded content can include media data including a file displayed on a display screen and audio data capturing the users' voices. Thus, the various streams of live or recorded content within the communication data enable teleconferencing between a group of people and sharing content within a group of people. In some embodiments, the various streams of live or recorded content within the communication data can originate from multiple co-located video cameras positioned in a space, such as a room, to record or stream the presentation of one or more individuals including presenting and consuming the presented content.

[0103] Participants or attendees are able to view the content of the communication session 603 as the activity occurs or, alternatively, via a recording at a later time after the activity occurs. In the examples described herein, client computing devices 606(1) to 606(N) participating in the communication session 603 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live and / or recorded content. For example, an individual stream of content can include media data associated with a video feed (e.g., audio and visual data capturing the appearance and voice of users participating in the communication session). Another example of an individual stream of content can include media data including avatars of users participating in the conference session and audio data capturing the voice of the users. Yet another example of an individual stream of content can include media data including content items displayed on a display screen and / or audio data capturing the voice of the users. Thus, the various streams of content within the communication data enable facilitating a conference or broadcast presentation among a group of people dispersed across remote locations.

[0104] Participants or attendees of a communication session are people who are within range of a camera or other image and / or audio capture device, so that the person's movements and / or sounds generated when the person is viewing and / or listening to content shared via the communication session can be captured (e.g., recorded). For example, a participant may be sitting in a crowd viewing shared content live at a broadcast location where a stage presentation is taking place. Alternatively, a participant may be sitting in an office conference room viewing shared content of a communication session with other colleagues via a display screen. Further, a participant may be sitting or standing in front of a personal device (e.g., tablet, smartphone, computer, etc.) viewing the shared content of a communication session individually in their office or home.

[0105] Figure 10System 602 includes device(s) 610. Device(s) 610 and / or other components of system 602 can include distributed computing resources that communicate with each other and / or with client computing devices 606(1) through 606(N) via one or more networks 608. In some examples, system 602 can be a standalone system responsible for managing various aspects of one or more communication sessions, such as communication session 603. As an example, system 602 can be managed by an entity such as Slack, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0106] The network(s) 608 may include, for example, a public network such as the Internet, a private network such as an institutional and / or personal intranet, or some combination of private and public networks. The network(s) 608 may also include any type of wired and / or wireless network, including but not limited to a local area network ("LAN"), a wide area network ("WAN"), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communication network (e.g., 3G, 4G, etc.), or any combination thereof. The network(s) 608 may utilize communication protocols, including packet-based and / or datagram-based protocols, such as the Internet Protocol ("IP"), the Transmission Control Protocol ("TCP"), the User Datagram Protocol ("UDP"), or other types of protocols. In addition, the network(s) 608 may also include a plurality of devices that facilitate network communications and / or form the hardware foundation for the network, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone equipment, and the like.

[0107] In some examples, network(s) 608 may also include devices that enable connection to wireless networks, such as wireless access points (“WAPs”). Examples support connections through WAPs that send and receive data over various electromagnetic frequencies (e.g., radio frequencies), including WAPs that support the Institute of Electrical and Electronics Engineers (“IEEE”) 802.11 standards (e.g., 802.11g, 802.11n, 802.11ac, etc.), as well as other standards.

[0108] In various examples, (one or more) devices 610 may include one or more computing devices that operate in a cluster or other grouping configuration to share resources, balance loads, increase performance, provide failover support or redundancy, or for other purposes. For example, (one or more) devices 610 may belong to various types of devices, such as conventional server-type devices, desktop-type devices, and / or mobile devices. Therefore, although illustrated as a single type of device or server-type device, (one or more) devices 610 may include a variety of device types and are not limited to a particular type of device. (One or more) devices 610 may represent, but are not limited to, a server computer, a desktop computer, a web server computer, a personal computer, a mobile computer, a laptop computer, a tablet computer, or any other type of computing device.

[0109] The client computing device (e.g., one of client computing devices 606(1) to 606(N)) (each of which is also referred to herein as a "data processing system") can be of various types of devices, which can be the same as or different from device(s) 610, such as a conventional client-type device, a desktop-type device, a mobile-type device, a dedicated-type device, an embedded device, and / or a wearable-type device. Thus, client computing devices can include, but are not limited to, desktop computers, game consoles and / or gaming devices, tablet computers, personal data assistants ("PDAs"), mobile phone / tablet hybrids, laptop computers, telecommunication devices, computer navigation-type client computing devices such as satellite-based navigation systems including global positioning system ("GPS") devices, wearable devices, virtual reality ("VR") devices, augmented reality ("AR") devices, implantable computing devices, automotive computers, web-enabled televisions, thin clients, terminals, Internet of Things ("IoT") devices, workstations, media players, personal video recorders ("PVRs"), set-top boxes, cameras, integrated components (e.g., peripherals) for inclusion in computing devices, appliances, or any other type of computing device. Furthermore, client computing devices can include combinations of the previously listed examples of client computing devices, such as, for example, desktop computer-type devices or mobile-type devices combined with wearable devices, etc.

[0110] The various classes and device types of client computing devices 606(1) to 606(N) can represent any type of computing device having one or more data processing units 692 that are operably connected to a computer-readable medium 694, such as via a bus 616, which in some cases can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of a variety of local, peripheral, and / or independent buses.

[0111] The executable instructions stored on the computer-readable medium 694 may include, for example, the operating system 619 , the client module 620 , the profile module 622 , and other modules, programs, or applications that can be loaded and executed by the data processing unit(s) 692 .

[0112] The client computing device(s) 606(1) to 606(N) may also include one or more interfaces 624 to enable communication between the client computing devices 606(1) to 606(N) and other networked devices (such as device(s) 610) over the network(s) 608. Such network interface(s) 624 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. Additionally, the client computing device(s) 606(1) to 606(N) may include input / output ("I / O") interfaces (devices) 626 that enable communication with user input devices, such as peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices such as microphones, video cameras for obtaining and providing video feeds and / or still images, touch input devices, gesture input devices, etc.) and / or output devices, such as peripheral output devices (e.g., displays, printers, audio speakers, tactile output devices, etc.). Figure 10 The client computing device 606 ( 1 ) is illustrated as being connected in some manner to a display device (eg, display screen 629 (N)) capable of displaying a UI according to the techniques described herein.

[0113] exist Figure 10In the example environment 600 of FIG. 6 , client computing devices 606 ( 1 ) through 606 (N) can use their respective client modules 620 to connect to each other and / or to other external devices (one or more) to participate in a communication session 603 or to contribute activities to a collaborative environment. For example, a first user can utilize client computing device 606 ( 1 ) to communicate with a second user of another client computing device 606 ( 2 ). When executing client module 620 , the users can share data, which can cause client computing device 606 ( 1 ) to connect to system 602 and / or other client computing devices 606 ( 2 ) through 606 (N) via network (s) 608 .

[0114] The client computing device(s) 606(1) to 606(N) may use their respective profile modules 622 to generate participant profiles (in Figure 10 6) and provides the participant profile to other client computing devices and / or device(s) 610 of system 602. The participant profile may include one or more of an identity of a user or a group of users (e.g., name, unique identifier ("ID"), etc.), user data such as personal data, user data such as location (e.g., IP address, room in a building, etc.) and technical capabilities. The participant profile may be used to register participants for a communication session.

[0115] As in Figure 10 As shown in , the device(s) 610 of the system 602 include a server module 630 and an output module 632. In this example, the server module 630 is configured to receive media streams 634(1) to 634(N) from individual client computing devices, such as client computing devices 606(1) to 606(N). As described above, the media streams can include a video feed (e.g., audio and visual data associated with a user), audio data to be output along with the presentation of the user's avatar (e.g., an audio-only experience without sending the user's video data), text data (e.g., a text message), file data and / or screen sharing data (e.g., a document, a slide deck, an image, a video displayed on a display screen, etc.), etc. Thus, the server module 630 is configured to receive a collection of various media streams 634(1) to 634(N) (the collection being referred to herein as "media data 634") during viewing of the communication session 603. In some scenarios, not all client computing devices participating in the communication session 603 provide media streams. For example, the client computing device may be a consuming device or a “listening” device only, such that it only receives content associated with the communication session 603 , but does not provide any content to the communication session 603 .

[0116] In various examples, the server module 630 can select aspects of the media stream 634 to be shared with individual client computing devices of the participating client computing devices 606(1) to 606(N). Thus, the server module 630 can be configured to generate session data 636 based on the stream 634 and / or pass the session data 636 to the output module 632. The output module 632 can then transmit communication data 639 to the client computing devices (e.g., the client computing devices 606(1) to 606(3) participating in the live viewing of the communication session). The communication data 639 can include video, audio, and / or other content data provided by the output module 632 based on content 650 associated with the output module 632 and based on the received session data 636. The content 650 can include the stream 634 or other shared data, such as an image file, a spreadsheet file, a slide deck, a document, etc. The stream 634 can include a video component depicting an image captured by the I / O device 626 on each client computer. The content 650 also includes input data from each user, which can be used to control the direction and position of the presentation. The content can also include instructions for sharing data and identifiers of recipients for the shared data. Therefore, the content 650 is also referred to herein as input data 650 or input 650.

[0117] As shown, output module 632 transmits communication data 639(1) to client computing device 606(1), and transmits communication data 639(2) to client computing device 606(2), and transmits communication data 639(3) to client computing device 606(3), etc. The communication data 639 transmitted to the client computing devices can be the same or can be different (e.g., the positioning of the content stream within the user interface can vary from one device to the next).

[0118] In various embodiments, the device(s) 610 and / or the client module 620 can include a GUI presentation module 640. The GUI presentation module 640 can be configured to analyze communication data 639 for delivery to one or more of the client computing devices 606. Specifically, at the device(s) 610 and / or the client computing device 606, the GUI presentation module 640 can analyze the communication data 639 to determine an appropriate manner for displaying video, images, and / or content on the display screen 629 of the associated client computing device 606. In some embodiments, the GUI presentation module 640 can provide the video, images, and / or content to a presentation GUI 646 that is rendered on the display screen 629 of the associated client computing device 606. The presentation GUI 646 can be rendered on the display screen 629 by the GUI presentation module 640. The presentation GUI 646 can include the video, images, and / or content analyzed by the GUI presentation module 640.

[0119] In some embodiments, the presentation GUI 646 may include multiple sections or grids that may render or include videos, images, and / or content for display on the display screen 629. For example, a first section of the presentation GUI 646 may include a video feed of a presenter or individual, and a second section of the presentation GUI 646 may include a video feed of the individual consuming conference information provided by the presenter or individual. The GUI presentation module 640 may populate the first and second sections of the presentation GUI 646 in a manner that appropriately mimics an environment experience that the presenter and individual may share.

[0120] In some embodiments, the GUI presentation module 640 can zoom in or provide a zoomed view of an individual represented by a video feed to highlight the individual's reactions to the presenter, such as facial features. In some embodiments, the presentation GUI 646 can include video feeds of multiple participants associated with a conference, such as a general communication session. In other embodiments, the presentation GUI 646 can be associated with a channel, such as a chat channel, an enterprise team channel, etc. Thus, the presentation GUI 646 can be associated with an external communication session that is different from a general communication session.

[0121] Figure 11A diagram illustrating example components of an example device 700 (also referred to herein as a "computing device") configured to generate data for some of the user interfaces disclosed herein is illustrated. Device 700 can generate data that can include one or more portions of video, images, virtual objects, and / or content that can be rendered or included for display on display screen 629. Device 700 can represent one of the device(s) described herein. Additionally or alternatively, device 700 can represent one of client computing devices 606.

[0122] As illustrated, device 700 includes one or more data processing units 702, computer-readable media 704, and communication interface(s) 706. The components of device 700 are operatively connected, for example, via bus 709, which may include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of various local, peripheral, and / or independent buses.

[0123] As used herein, data processing unit(s) (such as data processing unit(s) 702 and / or data processing unit(s) 692) may represent, for example, a CPU-type data processing unit, a GPU-type data processing unit, a field programmable gate array ("FPGA"), another type of DSP, or other hardware logic components that may be driven by a CPU in some cases. By way of example and not limitation, illustrative types of hardware logic components that may be utilized include application specific integrated circuits ("ASICs"), application specific standard products ("ASSPs"), systems on chips ("SOCs"), complex programmable logic devices ("CPLDs"), and the like.

[0124] As used herein, computer-readable media (such as computer-readable media 704 and computer-readable media 694) can store instructions that can be executed by (one or more) data processing units. The computer-readable medium can also store instructions that can be executed by an external data processing unit, such as by an external CPU, an external GPU, and / or can be executed by an external accelerator, such as an FPGA type accelerator, a DSP type accelerator, or any other internal or external accelerator. In various examples, at least one CPU, GPU, and / or accelerator is incorporated into the computing device, while in some examples, one or more of the CPU, GPU, and / or accelerator is external to the computing device.

[0125] Computer-readable media (also referred to herein as computer-readable media) may include computer storage media and / or communication media. Computer storage media may include one or more of volatile memory, non-volatile memory, and / or other persistent and / or secondary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Thus, computer storage media includes media in a tangible and / or physical form that is included in a device and / or hardware component that is part of a device or a hardware component external to a device, including, but not limited to: random access memory ("RAM"), static random access memory ("SRAM"), dynamic random access memory ("DRAM"), phase change memory ("PCM"), read-only memory ("ROM"), erasable programmable read-only memory ("EPROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory, compact disk read-only memory ("CD-ROM"), digital versatile disk ("DVD"), optical cards or other optical storage media, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid-state memory devices, storage arrays, network attached storage devices, storage area networks, hosted computer storage devices, or any other storage memory, storage device, and / or storage medium that can be used to store and maintain information for access by a computing device. Computer storage media may also be referred to herein as computer-readable storage media, non-transitory computer-readable storage media, non-transitory computer-readable media, or computer storage media.

[0126] In contrast to computer storage media, communication media can embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. As defined herein, computer storage media does not include communication media. That is, computer storage media itself does not include communication media consisting solely of a modulated data signal, carrier wave, or propagated signal.

[0127] The communication interface 706 may represent, for example, a network interface controller ("NIC") or other type of transceiver device for sending and receiving communications over a network. Additionally, the communication interface 706 may include one or more video cameras and / or audio devices 722 to enable the generation of video feeds and / or still images, etc.

[0128] In the example shown, computer-readable medium 704 includes a data store 708. In some examples, data store 708 includes a data store, such as a database, a data warehouse, or other type of structured or unstructured data store. In some examples, data store 708 includes a corpus and / or a relational database having one or more tables, indexes, stored procedures, etc., to enable data access, including, for example, one or more of a Hypertext Markup Language ("HTML") table, a Resource Description Framework ("RDF") table, a Web Ontology Language ("OWL") table, and / or an Extensible Markup Language ("XML") table.

[0129] The data store 708 can store data used for the operation of processes, applications, components, and / or modules stored in the computer-readable medium 704 and / or executed by the data processing unit(s) 702 and / or the accelerator(s). For example, in some examples, the data store 708 can store session data 710 (e.g., session data 636 as described herein). The session data 710 can include the total number of participants (e.g., users and / or client computing devices) in a communication session, the activities occurring in the communication session, a list of invitees to the communication session, and / or other data related to when and how the communication session was conducted or hosted.

[0130] The session data 710 can define any type of activity or state related to the individual users 10A-10L, each associated with a respective one of the plurality of video streams 634. For example, the contextual data can define the level of a person in the organization, how each person's level relates to the levels of other people, the performance level of a person, or any other activity or state information that can be used to determine the position of a person's rendering within the virtual environment. This data 710 can also be fed into any model to help emphasize keywords spoken by a person of a certain level, highlight the UI when background voices of a person of a certain level are detected, or change the emotional display in a specific way when a person of a certain level is detected to have a certain emotion.

[0131] The data store 708 may also include other data structures, including first-category operations 788 and second-category operations 789. The first-category operations may include operations for analyzing permissions to determine whether a user is allowed to restrict others from accessing or contributing to a thread. The first-category operations may also include operations that allow a user to exclusively contribute messages to a specific thread. The second-category operations may include operations that allow specific users to contribute messages to a thread while also restricting others from contributing messages to the thread. The system described herein may operate in a first operating mode utilizing the first-category operations. The system described herein may also operate in a second operating mode utilizing the second-category operations. A user may invoke a state change, also referred to as an operating mode change, by using input, which may include using a camera to capture a user gesture. For example, if a user raises their hand, the camera may capture an image of the movement. In response to determining that the user has made such a movement, or any other similar gesture, the system may provide input to invoke an operating state change from the first-category operation to the second-category operation. The input for invoking the operating state change may include a keyboard entry, a touchpad input gesture, a voice command, or any other input that may indicate a user's intent to create an event or conference interruption, which allows for the transmission of additional information via a file, stream, text input, voice input, or the like. For example, a user can raise their hand in a meeting to invoke a state change in the system that allows them to switch from an attendee role to a presenter role. The first operating state (also referred to herein as an operating state utilizing the first type of operation) is also referred to herein as a normal mode. The second operating state (also referred to herein as an operating state utilizing the second type of operation) is also referred to herein as a presentation mode.

[0132] Alternatively, some or all of the above data may be stored on a separate memory 716 on one or more data processing units 702, such as memory on a single board, a CPU-type processor, a GPU-type processor, an FPGA-type accelerator, a DSP-type accelerator, and / or another accelerator. In this example, the computer-readable medium 704 also includes an operating system 718 and an application programming interface 710 (API) configured to expose the functions and data of the device 700 to other devices. In addition, the computer-readable medium 704 includes one or more modules, such as a server module 730, an output module 732, and a GUI presentation module 740, although the number of modules shown is merely an example and the number may vary. That is, the functionality described herein in conjunction with the illustrated modules may be performed by a smaller number of modules or a larger number of modules on one device or distributed across multiple devices.

[0133] In summary, although various configurations have been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the accompanying representations is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed subject matter.

Claims

1. A method for controlling permissions for a message thread for a virtual conference session between a plurality of users, the method being executed on a data processing system configured to operate in a first type of operation and a second type of operation, the method comprising: receiving text messages entered by a plurality of computing devices associated with the user and directed to the message thread; providing a first user interface in the first category of operations, wherein the first user interface includes an attendee display area that displays a representation or video of at least one of the users and a text message UI area that displays received text messages; receiving input associated with a user of the virtual conference session, the input comprising a command for initiating a change in an operational state of the system from the first type of operation to the second type of operation; In response to the received input for initiating the change in operating state, analyzing permission data to determine whether user privileges associated with the user associated with the input satisfy permission requirements to invoke the change in operating state of the system from the first class of operation to the second class of operation; and In response to determining that the permission data indicates that the user privileges of the user associated with the received input satisfy the permission requirements, causing the operating state of the system to transition from the first category of operation to the second category of operation by: Changing from providing the first user interface to providing a second user interface in the second type of operation, wherein the second user interface is processed by the following operations to display text messages received from the user differently from text messages from other computing devices: displaying, in the text message UI area, text messages added by the user to the message thread while restricting at least one of other computing devices of other users from contributing text messages to the message thread; or Provided in the second user interface are: (a) a first text message display area for displaying text messages received from the user, and (b) a second text message display area different from the first text message display area, for displaying text messages added to the message thread by other users.

2. The method according to claim 1, wherein: Restricting other computing devices of other users from contributing messages to the message thread includes: disabling at least one of a text input field or a send button of a user interface displayed on the other computing device, wherein the text input field or the send button of the user interface remains disabled until a second input is received for initiating a change in a second operating state from utilizing the second type of operation to utilizing the first type of operation, for providing in the second user interface: (a) a first text message display area for displaying text messages received from the user, and (b) a second text message display area, different from the first text message display area, for displaying text messages added to the message thread by other users.

3. The method according to claim 1, wherein: The first text message display area for displaying text messages received from the user blocks display of messages received from the other users.

4. The method according to claim 3, wherein: The method further includes merging the message from the auxiliary thread with the message of the main thread in response to receiving a second input for initiating a second operational state change from utilizing the second type of operation to utilizing the first type of operation.

5. The method of claim 1, causing messages received from the computing device of the user to be displayed as an overlay on a video rendering of the user while the system is utilizing the second type of operation.

6. The method of claim 1, causing messages received from the computing devices of the users to be displayed in a plurality of rendered user interface arrangements including images of the plurality of users while the system is utilizing the second type of operation, wherein: Messages received from the computing device of the user are displayed as a replacement for the user's rendering.

7. The method according to claim 1, wherein: In response to determining that the permissions data does not specify that the user has permission to invoke the operating state change, the system prevents the system from invoking the operating state change of the system from the first class of operation to the second class of operation.

8. A system for controlling permissions for a message thread of a communication session, the system comprising: one or more processing units; as well as A computer-readable storage medium having computer-executable instructions encoded thereon to cause the one or more processing units to: receiving text messages entered by a plurality of computing devices associated with the user and directed to the message thread; providing a first user interface in the first category of operations, wherein the first user interface includes an attendee display area that displays a representation or video of at least one of the users and a text message UI area that displays received text messages; receiving input associated with a user of the virtual conference session, the input comprising a command for initiating a change in an operational state of the system from the first type of operation to the second type of operation; In response to the received input for initiating the change in operating state, analyzing permission data to determine whether user privileges associated with the user associated with the input satisfy permission requirements to invoke the change in operating state of the system from the first type of operation to the second type of operation; and In response to determining that the permission data indicates that the user privileges of the user associated with the received input satisfy the permission requirements, causing the operating state of the system to transition from the first category of operation to the second category of operation by: Changing from providing the first user interface to providing a second user interface in the second type of operation, wherein the second user interface is processed by the following operations to display text messages received from the user differently from text messages from other computing devices: displaying, in the text message UI area, text messages added by the user to the message thread while restricting at least one of other computing devices of other users from contributing text messages to the message thread; or Provided in the second user interface are: (a) a first text message display area for displaying text messages received from the user, and (b) a second text message display area different from the first text message display area, for displaying text messages added to the message thread by other users.

9. The system according to claim 8, wherein: Limiting other users' other computing devices from contributing messages to the message thread includes: disabling at least one of a text input field or a send button of a user interface displayed on the other computing device, wherein the text input field or the send button of the user interface remains disabled until a second input is received for initiating a change in a second operating state from utilizing the second type of operation to utilizing the first type of operation.

10. The system according to claim 8, wherein: The message thread is a main thread, wherein limiting other computing devices of other users from contributing messages to the message thread includes: Restricting the other computing devices of the other users from sending messages to the main thread; generating an auxiliary thread, the auxiliary thread being reserved for messages received from the other computing devices of the other users; and The message received from the other computing device of the other user is caused to be displayed in the auxiliary thread, wherein the auxiliary thread and the main thread are displayed on the user interface simultaneously.

11. The system according to claim 10, wherein: The instructions further cause the one or more processing units to merge the message from the auxiliary thread with the message of the main thread in response to receiving a second input for initiating a second operating state change from utilizing the second type of operation to utilizing the first type of operation.

12. The system according to claim 8, wherein: When the system is utilizing the second type of operation, messages received from the computing device of the user are caused to be displayed as an overlay on a video rendering of the user.

13. The system according to claim 8, wherein: When the system is utilizing the second type of operation, messages received from the computing devices of the users are displayed in a user interface arrangement that includes multiple renderings of images of the multiple users, wherein the messages received from the computing devices of the users are displayed as replacements for the renderings of the users.

14. The system according to claim 8, wherein: In response to determining that the permissions data does not specify that the user has permission to invoke the operating state change, the system prevents the system from invoking the operating state change of the system from the first class of operation to the second class of operation.

Images