Generating summary data from audio data or video data in group-based communication system

Abstract

Techniques for transcribing and / or summarizing multimedia collaboration sessions are discussed herein. For example, a user may communicate within a teleconference associated with a channel. In some examples, a first machine learning model may be configured to receive audiovisual data and user interaction data (e.g., selected emoticons, detected gestures, messages or text entered by a user, threads of messages, etc.) and output a teleconference summary. In some examples, the teleconference summary may include one or more highlight points, action items, and / or artificial intelligence (AI) annotations generated using a machine learning model.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 385,219, filed October 30, 2023, entitled "GENERATING SUMMARY DATA FROM AUDIO DATA OR VIDEO DATA IN A GROUP-BASED COMMUNICATION SYSTEM", which claims priority and the benefit of U.S. Provisional Application No. 63 / 429,082, filed November 30, 2022. All of the above applications are hereby incorporated by reference in their entirety. Technical Field

[0003] Communication platforms are becoming increasingly popular for facilitating work-related communication, such as for project collaboration within a single organization and across two or more organizations. To facilitate project collaboration, users typically share information or participate in ad-hoc discussions within the communication platform. However, reviewing conversations within the communication platform requires a significant amount of time to extract the important parts of the conversation. Existing systems may allow users to search conversations based on key terms, search functions, etc. to access relevant parts of the conversation, or may provide transcripts of ad-hoc discussions.

[0004] In speech-to-text transcription, a neural network classifies the sounds of the utterance as containing or not containing phonemes (units of speech), and matches the phonemes to known words, phrases, and sentences based on a model previously trained on a set of audio clips of the speech and the text transcription of the speech. However, existing speech-to-text transcription methods may not be able to robustly contextualize audio (and / or video) data that has accompanying non-verbal signals. Brief Description of the Drawings

[0005] The detailed description is described with reference to the drawings. In the drawings, the leftmost digit of the reference number identifies the drawing in which the reference number first appears. The same reference numbers are used in different drawings to indicate similar or identical components or features. The drawings are not drawn to scale.

[0006] Figure 1 An exemplary system for performing the techniques described herein is shown.

[0007] Figure 2A A user interface for a group-based communication system for a specific example is shown.

[0008] Figure 2B A user interface for a multimedia collaboration session within a group-based communication system for a specific example is shown.

[0009] Figure 2CShows a user interface for inter-organization collaboration within a group-based communication system for a specific example.

[0010] Figure 2D Shows a user interface for a collaborative document within a group-based communication system for a specific example.

[0011] Figure 3A Depicts a user interface for a workflow within a group-based communication system.

[0012] Figure 3B Depicts a block diagram for implementing a specific example as discussed herein.

[0013] Figure 4 Depicts a schematic diagram of a machine learning model for a specific example that can be trained to receive an input and use the input to output an overview document.

[0014] Figure 5 Shows an exemplary process associated with generating an overview document for a synchronous multimedia collaboration session within a virtual space.

[0015] Figure 6 Shows an exemplary user interface for requesting and generating an overview of a synchronous multimedia collaboration session and for generating annotations during the synchronous multimedia collaboration session.

[0016] Figure 7 Shows an exemplary user interface for requesting and generating an overview of a message thread as discussed herein.

[0017] Figure 8 Shows an exemplary process associated with generating an overview of a virtual space as discussed herein. Detailed Description

[0018] The following detailed description addresses the above deficiencies by more robustly transcribing and / or summarizing synchronous or asynchronous multimedia collaboration sessions in a group-based communication platform by using a combination of signals (including non-verbal signals that contextualize audio-visual data).

[0019] The following detailed description of the examples refers to the accompanying drawings that show specific examples in which the techniques may be practiced. These examples are intended to describe aspects of the systems and methods in sufficient detail so that those skilled in the art can practice the techniques discussed herein. Other examples may be utilized and changes may be made without departing from the scope of the disclosure. Accordingly, the following detailed description should not be construed as limiting. The scope of the disclosure is defined only by the appended claims and the full scope of equivalents to such claims.

[0020] Group-based communication system

[0021] Figure 1FIG. 100 illustrates an exemplary environment for performing the techniques described herein. In at least one example, the exemplary environment 100 may be associated with a communication platform that may utilize a network-based computing system to enable users of the communication platform to exchange data. In at least one example, the communication platform may be “group-based” such that the platform and associated systems, communication channels, messages, collaborative documents, canvases, audio / video conversations, and / or other virtual spaces have security (which may be defined by permissions) to limit access to a defined group of users. In some examples, such groups of users may be defined by group identifiers which, as described above, may be associated with common access credentials, domains, etc. In some examples, the communication platform may be a hub that provides a secure and private virtual space for users to chat, meet, call, collaborate, transfer files or other data, or otherwise communicate with one another or among themselves. As described above, each group may be associated with a workspace such that users associated with the group can chat, meet, call, collaborate, transfer files or other data, or otherwise communicate with one another or among themselves in a secure and private virtual space. In some examples, the members of the group and thus the workspace may be associated with the same organization. In some examples, the members of the group and thus the workspace may be associated with different organizations (e.g., entities having different organization identifiers).

[0022] In at least one example, the exemplary environment 100 may include one or more server computing devices (or “servers”) 102. In at least one example, the server 102 may include one or more servers or other types of computing devices that may be embodied in any number of ways. For example, in the case of a server, the functional components and data may be implemented on a single server, server cluster, server farm or data center, cloud-hosted computing services, cloud-hosted storage services, etc., but other computer architectures may be additionally or alternatively used.

[0023] In at least one example, server 102 can communicate with user computing device 104 via one or more networks 106. That is, server 102 and user computing device 104 can use network 106 to transmit, receive, and / or store data (e.g., content, information, etc.) as described herein. User computing device 104 can be any suitable type of computing device, e.g., portable, semi-portable, semi-fixed, or fixed. Some examples of user computing device 104 can include tablet computing devices, smart phones, mobile communication devices, laptop computers, netbooks, desktop computing devices, terminal computing devices, wearable computing devices, augmented reality devices, Internet of Things (IoT) devices, or any other computing device capable of sending communications and performing functions according to the techniques described herein. Although a single user computing device 104 is shown, in practice, exemplary environment 100 can include multiple (e.g., dozens, hundreds, thousands, millions) user computing devices. In at least one example, among other things, a user computing device (such as user computing device 104) can be operated by a user to access a communication service via a communication platform. The user can be an individual, a group of individuals, an employer, a business, an organization, etc.

[0024] Network 106 can include, but is not limited to, any type of network known in the art, such as a local area network or wide area network, the Internet, a wireless network, a cellular network, a local wireless network, Wi-Fi, and / or near field communication, Bluetooth Low Energy (BLE), Near Field Communication (NFC), a wired network, or any such network or any combination thereof. The components for such communication can depend at least in part on the type of network, the selected environment, or both. The protocols for communicating over such network 106 are well known and will not be discussed in detail herein.

[0025] In at least one example, server 102 can include one or more processors 108, computer-readable medium 110, one or more communication interfaces 112, and / or input / output devices 114.

[0026] In at least one example, each processor in processor 108 can be a single processing unit or multiple processing units and can include single or multiple computing units or multiple processing cores. Processor 108 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units (CPUs), graphics processing units (GPUs), state machines, logic circuits, and / or any device that manipulates signals based on operational instructions. For example, processor 108 can be one or more hardware processors and / or any suitable type of logic circuit that is specifically programmed or configured to execute the algorithms and processes described herein. Processor 108 can be configured to fetch and execute computer-readable instructions stored in a computer-readable medium, which can program the processor to perform the functions described herein.

[0027] Computer-readable medium 110 can include volatile and non-volatile memory and / or removable and non-removable media implemented in any type of technology for storing data such as computer-readable instructions, data structures, program modules, or other data. Such computer-readable medium 110 can include, but is not limited to, RAM, ROM, EEPROM, flash memory, or other memory technologies, optical storage, solid-state storage, magnetic tape, disk storage, RAID storage systems, storage arrays, network-attached storage, storage area networks, cloud storage, or any other medium that can be used to store the desired data and can be accessed by a computing device. Depending on the configuration of server 102, computer-readable medium 110 can be a type of computer-readable storage medium and / or can be a tangible non-transitory medium, to the extent that when mentioned, non-transitory computer-readable media exclude media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0028] Computer-readable medium 110 can be used to store any number of functional components that can be executed by processor 108. In many implementations, these functional components include instructions or programs that can be executed by processor 108 and that, when executed, specifically configure processor 108 to perform the actions attributed to server 102 above. The functional components stored in the computer-readable medium can optionally include messaging component 116, audio / video component 118, overview engine 120, operating system 122, and data repository 124.

[0029] In at least one example, the messaging component 116 may handle messages between users. That is, in at least one example, the messaging component 116 may receive an outgoing message from a user computing device 104 and may send the message as an incoming message to a second user computing device. Messages may include direct messages sent from an originating user to one or more designated users and / or communication channel messages sent from an originating user to one or more users associated with the communication channel via the communication channel. Additionally, messages may be transmitted in association with a collaborative document, canvas, or other collaborative space. In at least one example, the canvas may include a flexible canvas for curating, organizing, and sharing collections of information among users. In at least one example, a collaborative document may be associated with a document identifier (e.g., a virtual space identifier, a communication channel identifier, etc.), the document identifier being configured to enable messaging functionality attributable to a virtual space (e.g., a communication channel) within the collaborative document. That is, the collaborative document may be regarded as a virtual space (such as a communication channel) and include functionality associated with the virtual space (such as a communication channel). The virtual space or communication channel may be a data route for exchanging data between and among systems and devices associated with the communication platform.

[0030] In at least one example, the messaging component 116 may establish communication routes between and among various user computing devices, allowing the user computing devices to communicate and share data with one another. In at least one example, the messaging component 116 may manage such communication and / or data sharing. In some examples, data associated with a virtual space (such as a collaborative document) may be presented via a user interface. Additionally, metadata associated with each message transmitted via the virtual space may be stored in association with the virtual space, such as a timestamp associated with the message, a sender user identifier, a recipient user identifier, a conversation identifier, and / or a root object identifier (e.g., a conversation associated with a thread and / or a root object), etc.

[0031] In various examples, the messaging component 116 can receive messages transmitted in association with a virtual space (e.g., a direct message instance, a communication channel, a canvas, a collaborative document, etc.). In various examples, the messaging component 116 can identify one or more users associated with the virtual space and can cause the message to be presented in association with an instance of the virtual space on the respective user computing device 104. In various examples, the messaging component 116 can identify the message as an update to the virtual space and, based on the identified update, can cause a notification associated with the update to be presented in association with a sidebar of the user interface, the user interface being associated with one or more of the users associated with the virtual space. For example, the messaging component 116 can receive a message transmitted in association with a virtual space from a first user account. In response to receiving the message (e.g., interaction data associated with the first user's interaction with the virtual space), the messaging component 116 can identify a second user associated with the virtual space (e.g., another user who is a member of the virtual space). In some examples, the messaging component 116 can cause a notification of an update to the virtual space to be presented via a sidebar of the user interface associated with the second user's second user account. In some examples, the messaging component 116 can cause the notification to be presented in response to determining that the sidebar of the user interface associated with the second user account includes an affordance representation associated with the virtual space. In such examples, the notification can be presented in association with the affordance representation associated with the virtual space.

[0032] In various examples, the messaging component 116 can be configured to identify mentions or tags associated with messages transmitted in association with a virtual space. In at least one example, a mention or tag can include an @ mention (or other special character) of a user identifier associated with a communication platform. The user identifier can include a username, a real name, or other unique identifier associated with a particular user. In response to identifying a mention or tag of a user identifier, the messaging component 116 can cause a notification to be presented on the user interface associated with the user identifier, such as in a sidebar of the user interface associated with a particular user and / or in the virtual space associated with mentions and reactions in association with an affordance representation associated with the virtual space. That is, the messaging component 116 can be configured to alert a particular user that they have been mentioned in the virtual space.

[0033] In at least one example, the audio / video component 118 can be configured to manage audio and / or video communications among and between users. In some examples, the audio and / or video communications can be associated with an audio and / or video conversation. In at least one example, the audio and / or video conversation can include a discrete identifier configured to uniquely identify the audio and / or video conversation. In some examples, the audio / video component 118 can store user identifiers associated with user accounts of members of a particular audio and / or video conversation, such as to identify users having appropriate permission to access the particular audio and / or video conversation.

[0034] In some examples, communications associated with an audio and / or video conversation ("conversation") can be synchronous and / or asynchronous. That is, the conversation can include a real-time audio and / or video conversation between a first user and a second user over a period of time, and after a first period of time, a third user associated with the conversation (e.g., as a member of the conversation) can contribute to the conversation. The audio / video component 118 can be configured to store audio and / or video data associated with the conversation, such as to enable users having appropriate permission to listen to and / or view the audio and / or video data.

[0035] In some examples, the audio / video component 118 can be configured to generate a transcription of the conversation and, further, can store the transcription in association with the audio and / or video data. The transcription can include a text representation of the audio and / or video data. In at least one example, the audio / video component 118 can use known speech recognition techniques to generate the transcription. In some examples, the audio / video component 118 can generate the transcription simultaneously with or substantially simultaneously with the conversation. That is, in some examples, the audio / video component 118 can be configured to generate a text representation of the conversation while the conversation is in progress. In some examples, the audio / video component 118 can generate the transcription after receiving an indication that the conversation has been completed. The indication that the conversation has been completed can include an indication that the moderator or administrator associated therewith has stopped the conversation, an indication that a threshold number of meeting attendees have closed the associated interface, etc. That is, the audio / video component 118 can identify the completion of the conversation and then generate the transcription associated therewith based on the completion.

[0036] In at least one example, the audio / video component 118 can be configured to present a transcription in association with a virtual space associated with an audio and / or video conversation. For example, a first user can initiate an audio and / or video conversation associated with a communication channel. The audio / video component 118 can process the audio and / or video data between the participants of the audio and / or video conversation and, further, can generate a transcription of the audio and / or video data. In response to generating the transcription, the audio / video component 118 can cause the transcription to be published or otherwise presented via the communication channel. In at least one example, the audio / video component 118 can render one or more segments of the transcription as selectable for commentary, such as to enable members of the communication channel to comment on or further contribute to the conversation. In some examples, the audio / video component 118 can update the transcription based on the commentary.

[0037] In at least one example, the audio / video component 118 can manage one or more audio and / or video conversations in association with a virtual space associated with a group (e.g., an organization, a team, etc.) administrator or a command center. The group administrator or the command center can be referred to herein as the virtual (and / or digital) headquarters associated with the group. In at least one example, the audio / video component 118 can be configured to coordinate with the messaging component 116 and / or other components of the server 102 to transmit communications in association with other virtual spaces associated with the virtual headquarters. That is, the messaging component 116 can transmit data (e.g., messages, images, drawings, files, etc.) associated with one or more communication channels, direct messaging instances, collaborative documents, canvases, etc. associated with the virtual headquarters. In some examples, the communication channels, direct messaging instances, collaborative documents, canvases, etc. can have one or more audio and / or video conversations associated therewith that are managed by the audio / video component 118. That is, the audio and / or video conversations associated with the virtual headquarters can be further associated with or independent of one or more other virtual spaces of the virtual headquarters.

[0038] In at least one example, the summarization engine 120 may manage the transcription and / or summarization of synchronous and / or asynchronous multimedia collaboration sessions. The task of the summarization engine 120 may be to transcribe and / or summarize a multimedia collaboration session and generate one or more summary documents representing actionable tasks and / or deadlines characterizing the multimedia collaboration session. The summarization engine 120 may utilize a machine learning (ML) model 142 (or MLM) that accepts input and uses the input to output such summary documents. In various examples, a virtual space (e.g., group DM, direct DM, communication channel, etc.) may be created or otherwise accessed to discuss tasks and deadlines related to various goals, and within the virtual space, users may collaborate with each other via a synchronous multimedia collaboration session to achieve these goals. This may involve users posting messages to the virtual space, posting content to the virtual space (e.g., audio, video, links, files, etc.), and so on. Thus, in at least some examples, the input to the ML model 142 may include environmental data associated with a synchronous multimedia collaboration session, the synchronous multimedia collaboration session including real-time communication occurring between other users within the virtual space. However, the ML model 142 may access various other data for generating one or more summary documents, such as relationship data between participants in the multimedia session, contextual data of the virtual space (e.g., virtual space data, channel data, message data, thread data, emoji data, reaction emoji data, etc.), organizational data, and so on.

[0039] The ML model 142 may be trained to transcribe and / or summarize communications within the virtual space. For example, the ML model 142 may extract important parts of a conversation within the virtual space based on contextual cues, the communication frequency of users, past feedback, communication (or other interactions of users) above a threshold level, communications of users marked as favorites, task assignments to users, ratings of users, expertise of users, user preferences implied by their respective roles, user-specified parameters, user-specified permissions, heuristics from user activities, emojis, reaction emojis, and / or other interactions of users with previously generated summary documents. Additionally, the ML model 142 may be trained to identify one or more users to be associated with a task or assignment including a summary document, such as those users mentioned in the virtual space, those users posting to the virtual space, or those users who are members of the virtual space. The ML model 142 may also be trained to identify corresponding messages, contributions, posts, etc. for users within the virtual space. Thus, the ML model 142 may infer actionable context from the input it ingests and output a summary document representing such actionable context. For example, the ML model 142 may be trained to infer that a first user and a second user represent an assigner and an assignee of a task from the communication frequency between the two users exceeding a threshold level and other contextual cues.

[0040] In some examples, a communication platform may manage communication channels. In some examples, the communication platform may be a channel-based messaging platform. In some examples, the platform may be accessible by a group of users. Users of the communication platform may communicate with other users via the communication channels. The communication channels or virtual spaces may be data routes for exchanging data between and among systems and devices associated with the communication platform. In some examples, a channel may be a virtual space in which people can post messages, documents, and / or files. In some examples, access to a channel may be controlled via permissions. In some examples, a channel may be restricted to a single organization, shared between different organizations, may be public, private, or a special channel (e.g., a hosted channel with guest accounts where guests can post but are blocked from performing certain actions such as inviting other users to the channel). In some examples, some users may be invited to a channel via email, channel invitation, direct message, text message, etc. Examples of channels and associated functionality are discussed throughout the disclosure text.

[0041] In at least one example, the operating system 122 may manage the processor 108, computer-readable medium 110, hardware, software, etc. of the server 102.

[0042] In at least one example, the data repository 124 may be configured to store data that can be accessed, managed, and updated. In some examples, the data repository 124 may be integrated with the server 102, as Figure 1 shown. In other examples, the data repository 124 may be located remotely from the server 102 and may be accessible by the server 102 and / or user devices (such as the user computing device 104). The data repository 124 may include multiple databases, and the multiple databases may at least include first training data 126 and second training data 128. Additional or alternative data may be stored in the data repository 124 and / or stored in one or more other data repositories. For example, the data repository 124 may alternatively or additionally store one or more overview documents (not shown) therein.

[0043] In at least one example, the user / organization data 129 may include data associated with a user of the communication platform. In at least one example, the user / organization data 129 may store data in a user profile (which may also be referred to as a "user account"), and the user profile may store data associated with the user, including but not limited to: one or more user identifiers associated with multiple different organizations or entities with which the user is associated, one or more communication channel identifiers associated with communication channels to which the user has been authorized to access, one or more group identifiers of groups (or organizations, teams, entities, etc.) with which the user is associated, an indication of whether the user is an owner or manager of any communication channels, an indication of whether the user has any communication channel constraints, multiple messages, multiple emojis, multiple conversations, multiple conversation topics, an avatar, an email address, a real name (e.g., John Doe), a username (e.g., j doe), a password, a time zone, a status, a token, etc.

[0044] In at least one example, the user / organization data 129 may include permission data associated with permissions of an individual user of the communication platform. In some examples, the permissions may be set automatically or by an administrator of the communication platform, an employer, a business, an organization, or other entity using the communication platform, a team leader, a group leader, or other entity using the communication platform to communicate with team members, group members, etc., an individual user, etc. Permissions associated with an individual user may be mapped to an account or profile within the user / organization data 129, or otherwise associated with the account or profile. In some examples, the permissions may indicate which users may communicate directly with other users, which channels a user is permitted to access, constraints on individual channels, which workspaces a user is permitted to access, constraints on individual workspaces, etc. In at least one example, the permissions may support the communication platform by maintaining security for limiting access to a defined group of users. In some examples, such users may be defined by a common access credential, group identifier, etc., as described above.

[0045] In at least one example, the user / organization data 129 may include data associated with one or more organizations of the communication platform. In at least one example, the user / organization data 129 may store data in an organization profile, and the organization profile may store data associated with the organization, including but not limited to: one or more user identifiers associated with the organization, one or more virtual space identifiers associated with the organization (e.g., workspace identifier, communication channel identifier, direct message instance identifier, collaboration document identifier, canvas identifier, audio / video conversation identifier, etc.), an organization identifier associated with the organization, one or more organization identifiers associated with other organizations authorized to communicate with the organization, etc.

[0046] In at least one example, virtual space data can include data associated with one or more virtual spaces associated with a communication platform. Virtual space data can include text data, audio data, video data, images, files, and / or any other type of data configured to be transmitted in association with a virtual space. Non-limiting examples of virtual spaces include workspaces, communication channels, direct messaging instances, collaborative documents, canvases, and audio and / or video conversations. In at least one example, virtual space data can store data associated with individual virtual spaces separately, such as based on discrete identifiers associated with each virtual space. In some examples, a first virtual space can be associated with a second virtual space. In such examples, first virtual space data associated with the first virtual space can be stored in association with the second virtual space. For example, data associated with a collaborative document generated in association with a communication channel can be stored in association with the communication channel. As another example, data associated with an audio and / or video conversation conducted in association with a communication channel can be stored in association with the communication channel.

[0047] As described above, discrete identifiers that uniquely identify virtual spaces can be assigned to each virtual space of a communication platform. In some examples, a virtual space identifier associated with a virtual space can include a physical address where data related to the virtual space is stored in the virtual space data. A virtual space can be "public", which can allow any user within an organization (e.g., associated with an organization identifier) to join and participate in data sharing through the virtual space, or a virtual space can be "private", which can restrict data communication in the virtual space to specific users or users with appropriate viewing permissions. In some examples, a virtual space can be "shared", which can allow users associated with different organizations (e.g., entities associated with different organization identifiers) to join and participate in data sharing through the virtual space. A shared virtual space (e.g., a shared channel) can be public, such that it can be accessed by any user of any organization, or a shared virtual space (e.g., a shared channel) can be private, such that it is restricted to be accessed by specific users (e.g., users with appropriate permissions) of two organizations.

[0048] In some examples, the data repository 124 can be divided into discrete data items (e.g., data shards) that can be accessed and managed separately. Data shards can simplify many technical tasks, such as data retention, expansion (e.g., detecting that a message content includes a link, crawling metadata of the link, and determining a unified overview of the metadata), and integration settings. In some examples, data shards can be associated with an organization, a group (e.g., a workspace), a communication channel, a user, etc.

[0049] In some examples, an individual organization can be associated with a database shard that stores data related to a specific organization identifier within the data repository 124. For example, the database shard can store electronic communication data associated with members of a specific organization, which enables members of a specific organization to communicate and exchange data with other members of the same organization in real-time or near real-time. In an example, the organization itself can be the owner of the database shard and have control over where and how the related data is stored. In some examples, the database shard can store data related to two or more organizations (e.g., as in a shared virtual space).

[0050] In some examples, an individual group can be associated with a database shard that stores data related to a specific group identifier (e.g., a workspace) within the data repository 124. For example, the database shard can store electronic communication data associated with members of a specific group, which enables members of a specific group to communicate and exchange data with other members of the same group in real-time or near real-time. In an example, the group itself can be the owner of the database shard and have control over where and how the related data is stored.

[0051] In some examples, a virtual space can be associated with a database shard that stores data related to a specific virtual space identifier within the data repository 124. For example, the database shard can store electronic communication data associated with the virtual space, which enables members of a specific virtual space to communicate and exchange data with other members of the same virtual space in real-time or near real-time. As described above, communication via the virtual space can be synchronous and / or asynchronous. In at least one example, a group or an organization can be the owner of the database shard and can control where and how the related data is stored.

[0052] In some examples, an individual user can be associated with a database shard that stores data related to a specific user account within the data repository 124. For example, the database shard can store electronic communication data associated with an individual user, which enables the user to communicate and exchange data with other users of the communication platform in real-time or near real-time. In some examples, the user itself can be the owner of the database shard and have control over where and how the related data is stored.

[0053] In some examples, such as when a channel is shared between two organizations, each organization can be associated with its own encryption key. When a user associated with an organization posts a message or file to the shared channel, the message or file can be encrypted in the data repository 124 with the organization-specific encryption key, and the other organization can decrypt the message or file before accessing it. Additionally, in examples where the organizations are located in different geographical regions, data associated with a particular organization can be stored in a location corresponding to the organization and can be temporarily cached closer to the client (e.g., associated with another organization) when such a message or file is to be accessed. Data can be maintained, stored, and / or deleted in the data repository 124 according to the data management policies associated with each particular organization.

[0054] The communication interface 112 can include one or more interfaces and hardware components for facilitating communication with various other devices (e.g., the user computing device 104), such as via the network 106 or directly. In some examples, the communication interface 112 can facilitate communication via WebSocket, application programming interface (API) (e.g., using API calls), hypertext transfer protocol (HTTP), etc.

[0055] The server 102 can also be equipped with various input / output devices 114 (e.g., I / O devices). Such I / O devices 114 can include a display, various user interface controls (e.g., buttons, joysticks, keyboards, mice, touchscreens, etc.), audio speakers, connection ports, etc.

[0056] In at least one example, the user computing device 104 can include one or more processors 130, a computer-readable medium 132, one or more communication interfaces 134, and input / output devices 136.

[0057] In at least one example, each of the processors 130 can be a single processing unit or multiple processing units, and further, can include a single or multiple computing units or multiple processing cores. The processors 130 can include any type of processor described above with reference to the processor 108 and can be the same as or different from the processor 108.

[0058] The computer-readable medium 132 can include any type of computer-readable medium 132 described above with reference to the computer-readable medium 110 and can be the same as or different from the computer-readable medium 110. The functional components stored in the computer-readable medium can optionally include at least one application 138 and an operating system 140.

[0059] In at least one example, the application 138 can be a mobile application, a web application, or a desktop application, which can be provided by a communication platform or which can be another dedicated application. In some examples, separate user computing devices associated with the environment 100 can have instances or versioned instances of the application 138, which can be downloaded from an app store, accessible via the Internet, or otherwise executable by the processor 130 to perform the operations described herein. That is, the application 138 can be an access point such that the user computing device 104 can interact with the server 102 to access and / or use communication services available via the communication platform. In at least one example, the application 138 can facilitate data exchange among and between various other user computing devices, e.g., via the server 102. In at least one example, the application 138 can present a user interface as described herein. In at least one example, a user can interact with the user interface via touch input, keyboard input, mouse input, voice input, or any other type of input.

[0060] Figure 1 A non-limiting example of the user interface 143 is shown. As Figure 1 shown, the user interface 143 can present data associated with one or more virtual spaces, which can include one or more workspaces. That is, in some examples, the user interface 143 can integrate data from multiple workspaces into a single user interface such that a user (e.g., of the user computing device 104) can access data associated with, interact with, and / or otherwise communicate with other users associated with multiple workspaces to which the user is associated. In some examples, the user interface 143 can include a first region 144 or pane that includes indicators (e.g., user interface elements or objects) associated with workspaces associated with the user (e.g., the user's account). In some examples, the user interface 143 can include a second region 146 or pane that includes indicators (e.g., user interface elements, affordances, objects, etc.) representing data associated with workspaces associated with the user (e.g., the user's account). In at least one example, the second region 146 can represent a sidebar of the user interface 143.

[0061] In at least one example, the user interface 143 may include a third region 148 or pane that may be associated with a data feed (or “feed”) of messages indicating posts to one or more communication channels and / or other virtual spaces for facilitating communication (e.g., a virtual space associated with direct message communication, a virtual space associated with an event and / or action, etc.) and / or actions taken with respect to one or more communication channels and / or other virtual spaces, as described herein. In at least one example, the data associated with the third region 148 may be associated with the same or different workspaces. That is, in some examples, the third region 148 may present data associated with the same or different workspaces via an integrated feed. In some examples, the data may be organized and / or sorted by workspace, time (e.g., the time at which the associated data was posted or otherwise the associated operation was performed), type of action, communication channel, user, etc. In some examples, such data may be associated with an indication of which user (e.g., a member of the communication channel) posted the message and / or performed the action. In examples where the third region 148 presents data associated with multiple workspaces, at least some of the data may be associated with an indication of which workspace the data is associated with. In some examples, the third region 148 may be resized or popped up as a separate window.

[0062] In at least one example, the operating system 140 may manage the processor 130, computer-readable medium 132, hardware, software, etc. of the server 102.

[0063] The communication interface 134 may include one or more interfaces and hardware components for enabling communication with various other devices (e.g., the user computing device 104), such as via the network 106 or directly. In some examples, the communication interface 134 may facilitate communication via WebSocket, API (e.g., using API calls), HTTP, etc.

[0064] The user computing device 104 may also be equipped with various input / output devices 136 (e.g., I / O devices). Such I / O devices 136 may include a display, various user interface controls (e.g., buttons, joysticks, keyboards, mice, touchscreens, etc.), audio speakers, connection ports, etc.

[0065] Although the techniques described herein are described as being performed by the messaging component 116, audio / video component 118, overview engine 120, and application 138, the techniques described herein may be performed by any other component or combination of components that may be associated with the server 102, the user computing device 104, or a combination thereof.

[0066] User interface for a group-based communication system

[0067] Figure 2A A user interface 200 of a group-based communication system is shown, which will be used to illustrate the operations of various examples discussed herein. The group-based communication system may include communication data such as messages, queries, files, mentions, users or user profiles, interactions, work orders, channels, applications integrated into one or more channels, conversations, workspaces, or other data generated by or shared among users of the group-based communication system. In some cases, the communication data may include data associated with a user, such as a user identifier, channels that the user has been authorized to access, groups to which the user is associated, permissions, and other user-specific information.

[0068] The user interface 200 includes a plurality of objects, such as panes, text input fields, buttons, messages, or other user interface components viewable by a user of the group-based communication system. As shown, the user interface 200 includes a title bar 202, a workspace pane 204, a navigation pane 206, a channel 208, a document 210 (e.g., a collaborative document), a direct message 212, an application 214, a synchronized multimedia collaboration session pane 216, and a channel pane 218.

[0069] By way of example and not limitation, when the user opens the user interface 200, the user can select a workspace via the workspace pane 204. A particular workspace can be associated with workspace-specific data and can be accessible via a permission associated with the workspace. Different sections of the navigation pane 206 can present different data and / or options to the user. Different graphical indicators can be associated with virtual spaces (e.g., channels) to outline the properties of the channels (e.g., the channel is public, private, shared between organizations, locked, etc.). When the user selects a channel, the channel pane 218 can be presented. In some examples, the channel pane 218 can include a header, pinned items (e.g., documents or other virtual spaces), an "about" document providing an overview of the channel, etc. In some cases, members of the channel can search within the channel, access content associated with the channel, add other members, post content, etc. In some examples, depending on the permission associated with the channel, users who are not members of the channel can have limited ability to interact with the channel (or even view or otherwise access the channel). When users navigate within the channel, they can view messages 222 and can react to the messages (e.g., by selecting a reaction button 224), reply in a thread, start a thread, etc. Additionally, the channel pane 218 can include a compose pane 228 for composing messages and / or other data to be associated with the channel. In some examples, the user interface 200 can include a thread pane 230 that provides additional levels of detail for the messages 222. In some examples, the sizes of different panes can be adjusted, the panes can be popped out into separate windows, and / or separate windows can be merged into multiple panes of the user interface 200. In some examples, the user can communicate with other users via the multimedia collaboration session pane 216, which can provide synchronous or asynchronous voice and / or video capabilities for communication. Of course, these are illustrative examples, and additional examples of the foregoing features are provided throughout the disclosure text.

[0070] In some examples, the title bar 202 includes a search bar 220. The search bar 220 can allow the user to search for content located in the current workspace of the group-based communication system, such as files, messages, channels, members, commands, functions, etc. The user can refine their search by attributes such as content type, content author, and by users associated with the content. The user can optionally search within a particular workspace, channel, direct message conversation, or document. In some examples, the title bar 202 includes navigation commands that allow the user to move forward and backward between different panes and view the history of the accessed content. In some examples, the title bar 202 can include additional resources, such as links to help documents and user configuration settings.

[0071] In some examples, a group-based communication system can include multiple different workspaces, where each workspace is associated with a different user group and channel group. Each workspace can be associated with a group identifier, and one or more user identifiers can be mapped to, or otherwise associated with, the group identifier. Users corresponding to such user identifiers can be referred to as members of the group. In some examples, the user interface 200 includes a workspace pane 204 for navigating between, adding, or deleting various workspaces in the group-based communication system. For example, a user can be part of a workspace for Acme, where the user is an employee of Acme or otherwise affiliated with Acme. The user can also be a member of a local volunteer organization that also uses the group-based communication system for collaboration. To navigate between these two groups, the user can use the workspace pane 204 to change from the Acme workspace to the volunteer organization workspace. A workspace can include one or more channels specific to the workspace and / or one or more channels shared between one or more workspaces. For example, Acme Corporation can have a workspace for Acme projects (such as Project Zen), a workspace for social discussions, and additional workspaces for general corporate matters. In some examples, an organization (such as a particular company) can have multiple workspaces, and a user can be associated with one or more workspaces belonging to the organization. In other examples, a particular workspace can be associated with one or more organizations or other entities associated with the group-based communication system.

[0072] In some examples, a navigation pane 206 allows a user to navigate within virtual spaces such as pages, channels 208, documents 210 (such as in Figure 2Dnavigate between the applications 214 and the direct messages 212 (discussed elsewhere). For example, the navigation pane 206 can include indicators of virtual spaces that can aggregate data associated with multiple virtual spaces of which the user is a member. In at least one example, each virtual space can be associated with an indicator in the navigation pane 206. In some examples, the indicator can be associated with an actuation mechanism (e.g., an affordance, also known as a graphical element) such that when actuated, it can cause the user interface 200 to present data associated with the corresponding virtual space. In at least one example, the virtual space can be associated with all unread data associated with each of the workspaces associated with the user. That is, in some examples, if the user requests access to the virtual space associated with "unread", all data that the user has not yet read (e.g., viewed) can be presented, e.g., presented in a feed. In such examples, different types of events and / or actions that can be associated with different virtual spaces can be presented via the same feed. In some examples, such data can be organized and / or sorted according to the associated virtual space (e.g., the virtual space via which the communication was transmitted), time, type of action, user, etc. In some examples, such data can be associated with an indication of which user (e.g., a member of the associated virtual space) posted the message and / or performed the action.

[0073] In some examples, the virtual space can be associated with the same type of events and / or actions. For example, "threads" can be associated with messages, files, etc. posted in a thread to messages posted in a virtual space, and "mentions and reactions" can be associated with messages or threads in which the user has been mentioned (e.g., via tagging) or another user has reacted to a message or thread posted by the user (e.g., via an emoji, reaction, etc.). That is, in some examples, the same type of events and / or actions that can be associated with different virtual spaces can be presented via the same feed. Similar to the "unread" virtual space, the data associated with such virtual spaces can be organized and / or sorted according to the virtual space, time, type of action, user, etc.

[0074] In some examples, the virtual space can be associated with facilitating communication between the user and other users of the communication platform. For example, "contacts" can be associated with enabling the user to generate an invitation to communicate with one or more other users. In at least one example, in response to receiving an indication of a selection of the "contacts" indicator, the communication platform can cause a connection interface to be presented.

[0075] In some examples, a virtual space can be associated with one or more boards or collaborative documents associated with a user. In at least one example, a document can include a collaborative document configured to be accessed and / or edited by two or more users with appropriate permissions (e.g., view permission, edit permission, etc.). In at least one example, if a user requests access to a virtual space associated with one or more documents associated with the user, one or more documents can be presented via user interface 200. In at least one example, as described herein, a document can be associated with an individual (e.g., a user's private document), with a group of users (e.g., a collaborative document), and / or with one or more communication channels (e.g., members of the communication channel are given access permissions to the document), such that users of a communication platform can create data associated with such a document, interact with such a document, and / or view data associated with such a document. In some examples, a collaborative document can be a virtual space, board, canvas, page, etc. for collaborative communication and / or data organization within a communication platform. In at least one example, a collaborative document can support editable text and / or objects that can be sorted, added, deleted, modified, etc. In some examples, a collaborative document can be associated with permissions that define which users of the communication platform can view and / or edit the document. In some examples, a collaborative document can be associated with a communication channel, and members of the communication channel can view and / or edit the document. In some examples, a collaborative document can be shareable such that data associated with the document is accessible and / or interactive for members of multiple communication channels, workspaces, organizations, etc.

[0076] In some examples, a virtual space can be associated with a group (e.g., an organization, a team, etc.) headquarters (e.g., an administrator or a command center). In at least one example, a group headquarters can include a virtual or digital headquarters for an administrator or command function associated with a group of users. For example, "HQ" can be associated with an interface that includes a list of indicators associated with a virtual space configured such that associated members can communicate. In at least one example, a user can associate one or more virtual spaces with the "HQ" virtual space, such as via a drag-and-drop operation. That is, a user can determine relevant virtual spaces to be associated with the virtual or digital headquarters, such as to associate virtual spaces important to the user therewith.

[0077] In some examples, a virtual space can be associated with one or more boards or collaborative documents associated with a user. In at least one example, a document can include a collaborative document configured to be accessed and / or edited by two or more users with appropriate permissions (e.g., view permission, edit permission, etc.). In at least one example, if a user requests access to a virtual space associated with one or more documents associated with the user, one or more documents can be presented via the user interface 200. In at least one example, as described herein, a document can be associated with an individual (e.g., a user's private document), with a group of users (e.g., a collaborative document), and / or with one or more communication channels (e.g., members of the communication channel are given access permissions to the document), such that users of the communication platform can create data associated with such a document, interact with such a document, and / or view data associated with such a document. In some examples, a collaborative document can be a virtual space, board, canvas, page, etc. for collaborative communication and / or data organization within a communication platform. In at least one example, a collaborative document can support editable text and / or objects that can be sorted, added, deleted, modified, etc. In some examples, a collaborative document can be associated with permissions that define which users of the communication platform can view and / or edit the document. In some examples, a collaborative document can be associated with a communication channel, and members of the communication channel can view and / or edit the document. In some examples, a collaborative document can be shareable such that data associated with the document is accessible and / or interactive for members of multiple communication channels, workspaces, organizations, etc.

[0078] Additionally or alternatively, in some examples, a virtual space can be associated with one or more canvases associated with a user. In at least one example, a canvas can include a flexible canvas for curating, organizing, and sharing collections of information among users. That is, the canvas can be configured to be accessed and / or modified by two or more users with appropriate permissions. In at least one example, the canvas can be configured to enable sharing of text, images, videos, GIFs, drawings (e.g., user-generated drawings via the canvas interface), game content (e.g., users manipulating game controls synchronously or asynchronously), etc. In at least one example, modifications to the canvas can include adding, deleting, and / or modifying previously shared (e.g., transmitted, presented) data. In some examples, content associated with the canvas can be shareable via another virtual space such that data associated with the canvas can be accessed by members of the virtual space and / or rendered to be interactive by members of the virtual space.

[0079] The navigation pane 206 may also include an indicator representing a communication channel (e.g., channel 208). In some examples, the communication channel may include a public channel, a private channel, a shared channel (e.g., between groups or organizations), a single workspace channel, a cross-workspace channel, a combination of the foregoing channels, etc. In some examples, the represented communication channel may be associated with a single workspace. In some examples, the represented communication channel may be associated with different workspaces (e.g., cross-workspace). In at least one example, if the communication channel is cross-workspace (e.g., associated with different workspaces), the user may be associated with both workspaces, or alternatively, may be associated with only one of these workspaces. In some examples, the represented communication channel may be associated with a combination of the communication channels associated with a single workspace and the communication channels associated with different workspaces.

[0080] In some examples, the navigation pane 206 may depict some or all of the communication channels to which the user has access permissions (e.g., as determined by permission data). In such examples, the communication channels may be arranged alphabetically, based on recent interactions, based on the frequency of interactions, based on the communication channel type (e.g., public, private, shared, cross-workspace, etc.), based on workspaces, in user-specified segments, etc. In some examples, the navigation pane 206 may depict some or all of the communication channels of which the user is a member, and the user may interact with the user interface 200 to browse or view other communication channels of which the user is not a member but are not currently displayed in the navigation pane 206. In some examples, different types of communication channels (e.g., public, private, shared, cross-workspace, etc.) may be in different segments of the navigation pane 206, or may have their own sub-regions or sub-panes in the user interface 200. In some examples, the communication channels associated with different workspaces may be in different segments of the navigation pane 206, or the communication channels may have their own regions or panes in the user interface 200.

[0081] In some examples, the indicator may be associated with a graphical element that visually differentiates the type of communication channel. For example, item_zen is associated with a lock graphical element. As a non-limiting example and for the purposes of this discussion, the lock graphical element may indicate that the associated communication channel (item_zen) is private and access to it is restricted, while another communication channel (general) is public and access to it is available to any member of the organization with which the user is associated. In some examples, additional or alternative graphical elements may be used to differentiate between shared communication channels, communication channels associated with different workspaces, communication channels of which the user is or is not a current member, etc.

[0082] In at least one example, the navigation pane 206 can include indicators representing communications with individual users or multiple specified users (e.g., rather than all members of an organization or a subset thereof). Such communications can be referred to as "direct messages". The navigation pane 206 can include indicators representing virtual spaces associated with private messages between one or more users.

[0083] A direct message 212 can be a communication between a first user and a second user, or they can be a group direct message between a first user and two or more second users. The navigation pane 206 can be sorted and organized hierarchically or segmented according to user preferences. In some examples, all channels that a user has been authorized to access can appear in the navigation pane 206. In other examples, the user can choose to hide specific channels or collapse segments containing specific channels. Items in the navigation pane 206 can indicate when a new message or update has been received or that the new message or update is currently unread, such as by bolding text associated with the channel in which the unread message is located or adding an icon or badge to the channel name (e.g., with a count of unread messages). In some examples, a group-based communication system can additionally or alternatively store permission data associated with the permissions of individual users of the group-based communication system, indicating which channels a user can view or join. Permissions can, for example, indicate which users can communicate directly with other users, which channels a user is permitted access to, constraints on individual channels, which workspaces a user is permitted access to, and constraints on individual workspaces.

[0084] Additionally or alternatively, the navigation pane 206 can include sub-segments, which are personalized sub-segments associated with teams of which the user is a member. That is, the "team" sub-segment can include affordance representations of one or more virtual spaces associated with the team (such as communication channels, collaborative documents, direct messaging instances, audio or video synchronous or asynchronous meetings, etc.). In at least one example, the user can associate a selected virtual space with the team sub-segment, such as by drag-and-drop, setting a pin, or otherwise associating the selected virtual space with the team sub-segment.

[0085] Channels within a group-based communication system

[0086] In some examples, the group-based communication system is a channel-based messaging platform, as Figure 2A shown. Within a group-based communication system, communications can be organized into channels, each dedicated to a specific topic and set of users. A channel is typically a virtual space related to a specific topic, including messages and files posted by the members of the channel.

[0087] For the purposes of this discussion, a "message" can refer to any electronically generated digital object provided by a user using a user computing device 104 and configured to be displayed within a communication channel as described herein and / or within other virtual spaces for facilitating communication (e.g., a virtual space associated with direct messaging, etc.). A message can include any text, image, video, audio, or combination thereof provided by a user (using the user computing device). For example, a user can provide a message that includes text within the message as well as an image and a video as message content. In such examples, the text, image, and video will comprise the message. Each message sent or posted to a communication channel of a communication platform can include metadata that includes a sending user identifier, a message identifier, message content, a group identifier, a communication channel identifier, etc. In at least one example, each of the foregoing identifiers can include American Standard Code for Information Interchange (ASCII) text, a pointer, a memory address, etc.

[0088] Channel discussions can be persisted for days, months, or years and provide a historical log of user activity. Members of a particular channel can post messages within the channel that are visible to other members of the channel along with the other messages in the channel. A user can select channels to view to only see those messages relevant to the topic of the channel and not see messages posted in other channels about different topics. For example, a software development company can have different channels for each software product being developed, where developers working on each particular project can have a conversation about a generally single topic (e.g., the project) without the noise from unrelated topics. Because channels are typically persistent and relate to a particular topic or group, users can quickly and easily refer back to previous communications for reference. In some examples, the channel pane 218 can display information related to the channel that the user has selected in the navigation pane 206. For example, a user can select the project_zen channel to discuss the software development work being done on Project Zen. In some examples, the channel pane 218 can include a header that includes information about the channel, such as the channel name, a list of users in the channel, and other channel controls. A user can be able to pin items to the header for later access and be able to add bookmarks to the header. In some examples, a link to a collaborative document can be included in the header. In other examples, each channel can have a corresponding virtual space that includes channel-related information, such as a channel overview, tasks, bookmarks, pinned documents, and other channel-related links that can be editable by members of the channel.

[0089] A communication channel or other virtual space can be associated with data and / or content different from a message or data and / or content associated with the message. Non-limiting examples of additional data that can be presented via the channel pane 218 of the user interface 200 include collaborative documents (e.g., documents that can be collaboratively edited in real time or near real time, etc.), audio and / or video data associated with a conversation, members added to and / or removed from the communication channel, files (e.g., file attachments uploaded and / or removed from the communication channel), applications added to and / or removed from the communication channel, posts added to and / or removed from the communication channel (data that can be collaboratively and near real-time edited by one or more members of the communication channel), descriptions added to, modified, and / or removed from the communication channel, modifications to the nature of the communication channel, etc.

[0090] The channel pane 218 can include messages such as message 222, which is the content posted by a user to the channel. A user can post text, an image, a video, audio, or any other file as message 222. In some examples, a specific identifier (in a message or elsewhere) can be represented by prefixing it with a predetermined character. For example, a channel can be prefixed with the "#" character (such as in #project_zen), and a username can be prefixed with the "@" character (such as in @J_Smith or @user_A). A message (such as message 222) can include an indication of which user posted the message and the time the message was posted. In some examples, a user can react to a message by selecting a reaction button 224. The reaction button 224 allows the user to select an icon (sometimes called a reaction emoji in context) to associate with the message, such as a like. A user can respond to another user's message (such as message 222) with a new message. In some examples, such conversations in the channel can be further broken down into threads. Threads can be used to aggregate messages related to a specific conversation to make it easier to follow and reply to the conversation without cluttering the main channel with the discussion. Under the message where the thread starts, a thread reply preview 226 appears. The thread reply preview 226 can show information related to the thread, such as the number of replies and the members who have replied. Thread replies can appear in a thread pane 230 that can be separate from the channel pane 218 and can be viewed by other members of the channel by selecting the thread reply preview 226 in the channel pane 218.

[0091] In some examples, one or both of the channel pane 218 and the thread pane 230 can include a compose pane 228. In some examples, the compose pane 228 allows a user to compose a message 222 and transmit it to members of the channel or those members of the channel who are following the thread (when sending a message in a thread). The compose pane 228 can have text editing features such as bold, strikethrough, and italic, and / or can allow the user to format their message or attach files, such as a collaborative document, an image, a video, or any other file to be shared with other members of the channel. In some examples, the compose pane 228 can implement additional formatting options, such as numbered or bulleted lists, via a user interface or an API. The compose pane 228 can also be used as a workflow trigger to initiate a workflow related to the channel or the message. In other examples, a link or a document sent via the compose pane 228 can include expansion instructions related to how the content should be displayed.

[0092] Transcribing and / or summarizing a multimedia collaboration session

[0093] Conventional transcription algorithms merely transform audio signal data and apply a linguistic model in order to output written language from spoken language. However, conventional transcription models cannot take into account the rich contextual data inherent in group-based communication channels. As a result, conventional transcription models typically provide only a text record, lacking any contextual actionable information. In contrast, an ML model (such as ML model 142) as disclosed in the examples herein can be trained to at least ingest data including a multimedia collaboration session and output a summary document characterized by the contextually most relevant and / or actionable information propagated during the multimedia collaboration session. That is, the ML models of the various examples can be trained to utilize the channel context associated with the virtual space within which the multimedia collaboration session is initiated. Such non-verbal context includes, but is not limited to, the identities of the users of the group-based communication platform and the relationships between the users, references to users (i.e., @-mentions), submissions of documents or other file types in the context of a discussion, etc. The channel context provides powerful data for training the ML models of the various examples to infer meeting objectives, purposes, intents, targets, etc. Accordingly, the ML models of the various examples can be advantageously trained to at least learn one or more relationships between instances of environmental data including a multimedia collaboration session in the various examples disclosed herein and a transcription representing a synchronous multimedia collaboration session. In at least some examples, a machine learning model can be trained to learn one or more relationships between instances of environmental data and a transcription defining tasks and deadlines, or otherwise correlate actors with actions to be taken and time frames for taking the actions.

[0094] Figure 2BShows a multimedia collaboration session triggered from a channel (e.g., a synchronous multimedia collaboration session, also referred to herein as an audio and / or video conference), as shown in the synchronous multimedia collaboration session pane 216. The synchronous multimedia collaboration session can provide ambient ad-hoc multimedia collaboration in a group-based communication system. Users of the group-based communication system can quickly and easily join and leave these synchronous multimedia collaboration sessions at any time without causing other users to interrupt the synchronous multimedia collaboration session. In some examples, the synchronous multimedia collaboration session can be based on a specific topic, a specific channel, a specific direct message or a group direct message, or a set of users to facilitate the completion of a task according to a deadline.

[0095] The synchronous multimedia collaboration session pane 216 can be associated with a synchronous multimedia collaboration session for multiple users in a channel, users in a group direct message conversation, or users in a direct message conversation. Thus, a synchronous multimedia collaboration session for a specific channel or conversation can be initiated by one or more members of a specific channel, a group direct message conversation, or a direct message conversation. A user can start a synchronous multimedia collaboration session in a channel as a means of communicating with other currently online members of the channel. For example, a user may have an urgent decision and wish to get immediate verbal feedback from other members of the channel. Also, a synchronous multimedia collaboration session with one or more other users of the group-based communication system can be initiated via direct messaging. In some examples, the audience of the synchronous multimedia collaboration session can be determined based on the context in which the synchronous multimedia collaboration session is initiated. For example, starting a synchronous multimedia collaboration session in a channel can automatically invite the entire channel to attend. Also, starting a synchronous multimedia collaboration session allows a user to start an immediate audio and / or video conversation with other members of the channel without having to schedule or initiate a communication session through a third-party interface. In some examples, users can be directly invited to attend a synchronous multimedia collaboration session via a message or notification.

[0096] In some examples, the content of the synchronous multimedia collaboration session can be automatically persisted in the channel associated with the synchronous multimedia collaboration session. Members of a specific synchronous multimedia collaboration session can post messages within the thread associated with the synchronous multimedia collaboration session, and these messages are visible to other members of the synchronous multimedia collaboration session along with other messages in the thread.

[0097] The multimedia in a synchronous multimedia collaboration session can include collaboration tools such as any or all of audio, video, screen sharing, collaborative document editing, whiteboards, pair programming, or any other form of media. The synchronous multimedia collaboration session can also allow a user to share the user's screen with other members of the synchronous multimedia collaboration session. In some examples, members of the synchronous multimedia collaboration session can mark, comment on, draw on, or otherwise annotate the shared screen. In other examples, such annotations can be saved and persisted after the synchronous multimedia collaboration session has ended. A canvas can be created directly from the synchronous multimedia collaboration session to further enhance collaboration among users.

[0098] In some examples, a user can start a synchronous multimedia collaboration session via Figure 2B the back-and-forth switching keys in the synchronous multimedia collaboration session pane 216 shown. Once the synchronous multimedia collaboration session has started, the synchronous multimedia collaboration session pane 216 can expand to provide information about the synchronous multimedia collaboration session (such as how many members there are, which user is currently speaking, which user is sharing the user's screen) and / or a screen sharing preview 232. In some examples, icons indicating that a user is participating in the synchronous multimedia collaboration session can be used to display the users in the synchronous multimedia collaboration session. In other examples, an expanded view of the participants can show which users are active and which users are not active in the synchronous multimedia collaboration session. The screen sharing preview 232 can depict the desktop view of the user sharing the user's screen, or a particular application or presentation. Changes to the user's screen, such as the user advancing to the next slide in a presentation, will be automatically depicted in the screen sharing preview 232. In some examples, the screen sharing preview 232 can be actuated such that the screen sharing preview 232 is magnified and displayed as its own pane within the group-based communication system. In some examples, the screen sharing preview 232 can be actuated such that the screen sharing preview 232 pops up into a new window or application separate and distinct from the group-based communication system. In some examples, the synchronous multimedia collaboration session pane 216 can include tools for the synchronous multimedia collaboration session that allow a user to mute the user's microphone or invite other users. In some examples, the synchronous multimedia collaboration session pane 216 can include a screen sharing button 234 that can allow a user to share the user's screen with other members of the synchronous multimedia collaboration session pane 216. In some examples, the screen sharing button 234 can provide additional controls to the user during screen sharing. For example, additional screen sharing controls can be provided to the user sharing the user's screen to specify which screen to share, annotate the shared screen, or save the shared screen.

[0099] In some cases, the Synchronous Multimedia Collaboration Session pane 216 can be persistently saved in the Navigation pane 206 regardless of the state of the group-based communication system. In some examples, the Synchronous Multimedia Collaboration Session pane 216 can be hidden or removed from being presented via the user interface 200 when there is no active Synchronous Multimedia Collaboration Session and / or depending on which item is selected from the Navigation pane 206. In some cases, when the Multimedia Collaboration Session pane 216 is active, the Multimedia Collaboration Session pane 216 can be associated with the currently selected channel, direct message, or group direct message such that a Synchronous Multimedia Collaboration Session can be initiated and associated with the currently selected channel, direct message, or group direct message.

[0100] The list of Synchronous Multimedia Collaboration Sessions can include one or more active Synchronous Multimedia Collaboration Sessions selected for recommendation. For example, a Synchronous Multimedia Collaboration Session can be selected from among multiple currently active Synchronous Multimedia Collaboration Sessions. Additionally, a Synchronous Multimedia Collaboration Session can be selected based in part on user interaction with the session or some association of the instant user with the session or the users involved in the session. For example, a recommended Synchronous Multimedia Collaboration Session can be displayed based in part on the instant user having been invited to the corresponding Synchronous Multimedia Collaboration Session or having previously collaborated with the users in the recommended Synchronous Multimedia Collaboration Session. In some examples, the list of Synchronous Multimedia Collaboration Sessions also includes additional information for each corresponding Synchronous Multimedia Collaboration Session, such as an indication of the participating users or the number of participating users, the topic of the Synchronous Multimedia Collaboration Session, and / or an indication of the associated group-based communication channel, group direct message conversation, or direct message conversation.

[0101] In some examples, the list of recommended active users can include multiple group-based communication system users recommended based on at least one of user activity, user interactions, or other user information. For example, the list of recommended active users can be selected based on: the active state of users within the group-based communication system; historical, recent, or frequent user interactions with the instant user (such as communicating within a group-based communication channel); or similarity between the recommended user and the instant user (such as determining a shared membership in channels between the recommended user and the instant user). In some examples, machine learning techniques (such as clustering analysis) can be used to determine the recommended users. The list of recommended active users can include status user information for each recommended user, such as whether the recommended user is active, in a meeting, idle, in a synchronous multimedia collaboration session, or offline. In some examples, the list of recommended active users further includes a plurality of actuatable buttons corresponding to some or all of the recommended users (e.g., those recommended users having a status indicating availability), the plurality of actuatable buttons being configurable to initiate at least one of a text-based communication session (such as a direct message conversation) or a synchronous multimedia collaboration session when selected.

[0102] In some examples, one or more recommended asynchronous multimedia collaboration sessions or meetings can be displayed during an asynchronous meeting segment. Compared to synchronous multimedia collaboration sessions (described above), asynchronous multimedia collaboration sessions allow each participant to collaborate at their convenience. The collaboration participation is then recorded for consumption by other participants at a later time, who can generate additional multimedia responses. In some examples, the responses are aggregated in a multimedia thread (e.g., a video thread) corresponding to the asynchronous multimedia collaboration session. For example, an asynchronous multimedia collaboration session can be used for an asynchronous meeting where the topic is posted in a message at the start of the meeting thread, and participants in the meeting can respond by posting messages or video responses. The resulting thread then includes any documents, videos, or other files related to the asynchronous meeting. In some examples, a preview of a subset of video responses can be shown within the asynchronous collaboration session or thread. This can allow, for example, a user to jump to a relevant segment of the asynchronous multimedia collaboration session or resume from where they previously left off.

[0103] In other examples, the multimedia collaboration session can be a short-lived session that does not persist any data therein. In some other examples, the multimedia collaboration session can be recorded, transcribed, and / or summarized for later review and interaction.

[0104] Combined Figure 4 and Figure 5 Additional discussion of determining summary data is provided throughout the disclosure.

[0105] Contacting within a group-based communication system

[0106] Figure 2C The user interface 200 showing the contact pane 252 is shown. The contact pane 252 can provide users with tools and resources for contacting across different organizations, where each organization can have its own (usually private) instance of a group-based communication system or may not yet belong to a group-based communication system. For example, a first software company may form a joint venture with a second software company and they wish to collaborate with the latter in jointly developing a new software application. The contact pane 252 can enable users to determine which other users and organizations are already within the group-based communication system and invite those users and organizations currently outside the group-based communication system to join.

[0107] The contact pane 252 can include a contact search bar 254, recent contacts 256, connections 258, a create channel button 260, and / or a start direct message button 262. In some examples, the contact search bar 254 can allow users to search for users within the group-based communication system. In some examples, only users from organizations that have contact with the user's organization will be shown in the search results. In other examples, users from any organization using the group-based communication system can be shown. In other examples, users from organizations that have not yet used group-based communication can also be shown, thus allowing the search user to invite them to join the group-based communication system. In some examples, users can be searched via the user's group-based communication system username or their email address. In some examples, email addresses can be suggested or auto-completed based on external data sources such as an email directory or the contact list of the search user.

[0108] In some examples, external organizations as well as individual users can be shown in response to a user search. External organizations can be matched based on the organization name or internet domain, as the search results can include organizations that have not yet joined the group-based communication system (similar to the search and matching for specific users discussed above). External organizations can be ranked in part based on how many users from the user's organization have contact with the users of the external organization. In response to selecting an external organization in the search results, the search user can be able to invite the external organization to contact via the group-based communication system.

[0109] In some examples, the recent contacts 256 can display the users with whom the instant user has most recently interacted. The recent contacts 256 can display the names, companies, and / or status indicators of the users. The recent contacts 256 can be sorted based on which contacts the instant user interacts with most frequently or based on which contacts the instant user has most recently interacted with. In some examples, each recent contact in the recent contacts 256 can be an actuatable control that allows the instant user to quickly initiate a direct message conversation with the recent contact, invite them to a channel, or take any other appropriate user action with respect to the recent contact.

[0110] In some examples, the connections 258 can display a list of companies (e.g., organizations) with which the user has interacted. For each company, the name of the company can be displayed along with the company's identifier and an indication of how many times the user has interacted with the company (e.g., the number of conversations). In some examples, each connection in the connections 258 can be an actuatable control that allows the instant user to quickly invite an external organization to a shared channel, display the most recent connection with the external organization, or take any other appropriate organizational action with respect to the connection.

[0111] In some examples, the create channel button 260 allows the user to create a new shared channel between two different organizations. Selecting the create channel button 260 can also allow the user to name the new contact channel and enter a description for the contact channel. In some examples, the user can select one or more external organizations or one or more external users to add to the shared channel. In other examples, the user can add an external organization or external user to the shared channel after creating the shared channel. In some examples, the user can select whether to make the contact channel private (i.e., accessible only by invitation from the current members of the private channel).

[0112] In some examples, the start direct message button 262 allows a user to quickly initiate a direct message (or group direct message) with an external user at an external organization. In some examples, the external user identifier at the external organization can be provided by the instant user as the group-based communication system username of the external user or as the email address of the external user. In some examples, the analysis of the email domain of the external user's email address can affect the message between the user and the external user. For example, the identifier of the external user can indicate (e.g., based on the email address domain) that the user's organization and the external user's organization have communicated. In some such examples, the email address can be converted to a group-based communication system username. Alternatively, the identifier of the external user can indicate that the external user's organization belongs to the group-based communication system but has not communicated with the instant user's organization. In some such examples, an invitation to communicate with the instant user's organization can be generated as a response. As another alternative, the external user can not be a member of the group-based communication system, and an invitation to join the group-based communication system as a guest or member can be generated as a response.

[0113] Collaboration document

[0114] Figure 2D A user interface 200 showing a collaborative document pane 264 is shown. The collaborative document can be of any file type, such as PDF, video, audio, word processing document, etc., and is not limited to word processing documents or spreadsheets. The collaborative document can be modified and edited by two or more users. The collaborative document can also be associated with different user permissions, such that based on the user's permission for the document (or a section of the document, as described below), users can be selectively allowed to view, edit, or comment on the collaborative document (or a section of the collaborative document). Thus, users within the set of users who can access the document can have permission to view, edit, comment on, or otherwise hand off changes to the collaborative document. In some examples, the permissions can be automatically determined and / or assigned based on the way the document is created and / or shared. In some examples, the permissions can be determined manually. The collaborative document can allow users to create and modify the document simultaneously or asynchronously. The collaborative document can be integrated with a group-based communication system and can both initiate a workflow and be used to store the results of the workflow, which will be further discussed below with respect to Figure 3A and Figure 3B be discussed further.

[0115] In some examples, the user interface 200 may include one or more collaborative documents (or one or more links to such collaborative documents). A collaborative document (also referred to as a document or a canvas) may include a flexible workspace for planning, organizing, and sharing a collection of information among users. Such a document may be associated with a synchronous multimedia collaboration session, an asynchronous multimedia collaboration session, a channel, a multi-person direct message conversation, and / or a direct message conversation. The shared canvas may be configured to be accessed and / or modified by two or more users with appropriate permissions. Alternatively or additionally, a user may have one or more private documents that are not associated with any other users.

[0116] In addition, such a document may be @-mentioned so that a specific document can be referenced within a channel (or other virtual space or document) and / or other users can be mentioned within such a document. For example, @-mentioning a user within a document may provide an indication to the user and / or may provide access to the document to the user. In some examples, tasks may be assigned to a user via @-mention, and such tasks may be populated in a pane or sidebar associated with the user.

[0117] In some examples, a channel and a collaborative document 268 may be associated such that when a comment is posted in the channel, the comment may be populated into the document 268 and vice versa.

[0118] In some examples, when a first user interacts with a collaborative document, the communication platform may identify a second user account associated with the collaborative document and present an affordance (e.g., a graphical element) indicating the interaction in a sidebar (e.g., the navigation pane 206). Additionally, the second user may select the affordance or notification associated with or representing the interaction with accessing the collaborative document to efficiently access the document and view updates thereto.

[0119] In some examples, when one or more users interact with a collaborative document, an indication (e.g., an icon or other user interface element) may be presented via the user interface having the collaborative document to represent such interaction. For example, if a first instance of a document is currently open on a first user's first user computing device and a second instance of the document is currently open on a second user's second user computing device, one or more presence indicators may be presented on the respective user interfaces to show the various interactions with the document and the users performing the interactions. In some examples, a presence indicator may have an attribute (e.g., an appearance attribute) indicating information about the respective user, such as but not limited to a permission level (e.g., an edit permission, read-only access, etc.), virtual space membership (e.g., whether the member belongs to the virtual space associated with the document), and the way the user interacts with the document (e.g., currently editing, viewing, open but inactive, etc.).

[0120] In some examples, a preview of a collaborative document can be provided. In some examples, the preview can include an overview of the collaborative document and / or a dynamic preview that shows various content (e.g., as changing text, images, etc.) to allow a user to quickly understand the context of the document. In some examples, the preview can be based on user profile data associated with the user viewing the preview (e.g., permissions associated with the user, content the user has viewed, edited, created, etc.).

[0121] In some examples, a collaborative document can be created independently of or in combination with a virtual space and / or a channel. The collaborative document can be published in a channel and edited or interacted with as described herein, with various affordances or notifications indicating the presence of users and / or various interactions associated with the document.

[0122] In some examples, a machine learning model can be used to determine an overview of the content of a channel, and a collaborative document including the overview can be created for publication in the channel. In some examples, a communication platform can identify users within a virtual space, actions associated with the users, and other contributions to a conversation to generate an overview document. Thus, the communication platform can enable users to create a document (e.g., a collaborative document) for summarizing content and events occurring within the virtual space.

[0123] In some examples, a document can be configured to enable sharing of content, including (but not limited to) text, images, videos, GIFs, drawings (e.g., user-generated drawings via a drawing interface), or game content. In some examples, a user accessing a canvas can add new content or delete (or modify) previously added content. In some examples, a user may need appropriate permissions to add content or to delete or modify content added by a different user. Thus, for example, some users may only be able to access some or all of the document in a read-only mode, while other users may be able to access some or all of the document in an edit mode that allows those users to add or modify its content. In some examples, the document can be shared via a message in a channel, a group direct message, or a direct message, such that data associated with the document can be accessed by members of the channel or recipients of the group direct message or direct message and / or rendered for interaction by members of the channel or recipients of the group direct message or direct message.

[0124] In some examples, the collaborative document pane 264 can include a collaborative document toolbar 266 and a collaborative document 268. In some examples, the collaborative document toolbar 266 can provide the ability to edit or format the publication, as discussed herein.

[0125] In some examples, a collaborative document can include free-form unstructured segments and workflow-related structured segments. In some examples, the unstructured segments can include areas of the document where users can freely modify the collaborative document without any constraints. For example, a user can be able to freely type text to explain the purpose of the document. In some examples, a user can add a workflow or a structured workflow segment by typing the name of the workflow (or otherwise referring to the workflow). In other examples, typing an "at" sign (@) or a previously selected symbol or a predetermined special character or symbol can provide the user with a list of workflows that the user can select to add to the document. For example, a user can indicate that a marketing team member needs to sign a proposal to initiate a workflow that ends when the members of the marketing team approve the proposal by typing "! Marketing Approval". Placing an exclamation point before the group name of "Marketing Approval" initiates a request for a canonical action, which in this case is routing the proposal for approval. In some examples, the structured segments can include text inputs, selection menus, tables, checkboxes, tasks, calendar events, or any other document segments. In other examples, the structured segments can include text input spaces that are part of a workflow. For example, a user can input text detailing the reason for seeking approval into a text input space and then select a submit button that will advance the workflow to the next step of the workflow. In some examples, a user can be able to add, edit, or remove the structured segments that make up the workflow components of the document.

[0126] In an example, segments of a collaborative document can have separate permissions associated with them. For example, a collaborative document with segments having separate permissions can provide a first user with permission to view, edit, or comment on a first segment, while a second user does not have permission to view, edit, or comment on the first segment. Alternatively, the first user can have permission to view the first segment of the collaborative document, while the second user has permission to view and edit the first segment of the collaborative document. Permissions associated with a specific segment of a document can be assigned by the first user via various methods, including manually selecting a specific segment of the document by the first user or another user having the permission to assign permissions, typing or selecting an "assign" indicator (such as the "@" symbol), or selecting the segment by the name of the segment. In other examples, permissions can be assigned to multiple collaborative documents at a single instance via these methods. For example, multiple collaborative documents each have a segment titled "Group Information", where a first user having the permission to assign permissions desires that the entire user group can access the information in the "Group Information" segments of the multiple collaborative documents. In an example, the first user can select the multiple collaborative documents and the "Group Information" segment to effect permission for the entire user group to access (or view, edit, etc.) the "Group Information" segment of each of the multiple collaborative documents.

[0127] Automation in a group-based communication system

[0128] Figure 3A Shown is a user interface 300 for automation in a group-based communication system. Automation, also known as workflow, allows a user to automate functionality within the group-based communication system. A workflow builder 302 is depicted that allows a user to create new workflows, modify existing workflows, and review workflow activities. The workflow builder 302 can include a workflow tab 304, an activity tab 306, and / or a settings tab 308. In some examples, the workflow builder can include a publish button 314 that allows a user to publish a new or modified workflow.

[0129] The workflow tab 304 can be selected to enable a user to create a new workflow or modify an existing workflow. For example, a user may wish to create a workflow to automatically welcome new users joining a channel. The workflow can include workflow steps 310. The workflow steps 310 can include at least one trigger that initiates the workflow and at least one function that takes an action once the workflow is triggered. For example, when a user joins a channel, the workflow can be triggered, and the function of the workflow can be to post a welcome message for the new user in the channel. In some examples, a workflow can be triggered from a user action (such as a user reacting to a message, joining a channel, or collaborating in a co-authored document), from a scheduled date and time, or from a web request from a third-party application or service. In other examples, workflow functionality can include sending a message or form to a user, channel, or any other virtual space, modifying a co-authored document, or interfacing with an application. The workflow functionality can include workflow variables 312. For example, a welcome message can include the name of the user via a variable to enable a customized message. A user can edit existing workflow steps or add new workflow steps according to the desired workflow functionality. Once the workflow is complete, the user can use the publish button 314 to publish the workflow. The published workflow will wait until it is triggered, at which time the function will be executed.

[0130] The activity tab 306 can display information related to the activities of a workflow. In some examples, the activity tab 306 can show how many times a workflow has been executed. In other examples, the activity tab 306 can include information related to each workflow execution, including status, most recent activity date, execution time, the user who initiated the workflow, and other relevant information. The activity tab 306 can allow a user to sort and filter workflow activities to find useful information.

[0131] The Settings tab 308 can allow a user to modify the settings of a workflow. In some examples, the user can change the title or icon associated with the workflow. The user can also manage the collaborators associated with the workflow. For example, the user can add additional users as collaborators to the workflow, such that the additional users can modify the workflow. In some examples, the Settings tab 308 can also allow the user to delete the workflow.

[0132] Figure 3B FIG. depicts elements related to a workflow in a group-based communication system and is generally referred to by reference numeral 316. In various examples, the trigger 318 can be configured to invoke the execution of a function 336 in response to a user instruction. The trigger initiates the function execution and can take the form of one or more schedules 320, webhooks 322, shortcuts 324, and / or slash commands 326. In some examples, the schedule 320 operates like a timer, such that the trigger can be scheduled to fire periodically or once at a predetermined time in the future. In some examples, an end user of an event-based application sets any schedule for the trigger to fire, such as once an hour or at 9:15 am every day.

[0133] Additionally, the trigger 318 can take the form of a webhook 322. The webhook 322 can be a software component that listens at a webhook URL and port. In some examples, the trigger fires when an appropriate HTTP request is received at the webhook URL and port. In some examples, the webhook 322 requires appropriate authentication, such as via a bearer token. In other examples, the trigger will depend on the payload content.

[0134] Another source of one of the triggers 318 is the shortcut in the shortcut 324. In some examples, the shortcut 324 can be global to the group-based communication system, rather than specific to a group-based communication system channel or workspace. A global shortcut can trigger a function that can be executed without the context of a specific group-based communication system message or group-based communication channel. In contrast, a message- or channel-based shortcut is specific to a group-based communication system message or channel and operates in the context of a group-based communication system message or group-based communication channel.

[0135] Another source of one of the triggers in trigger 318 can be provided by slash command 326. In some examples, slash command 326 can be used as an entry point for group-based communication system functionality, integration with external services, or group-based communication system message responses. In some examples, a user of the group-based communication system can input slash command 326 to trigger the execution of application functionality. After the slash command can be slash command line parameters, which can be passed to any group-based communication system functionality that is called in conjunction with the triggering of a group-based communication system functionality such as one of the functions in function 336.

[0136] An additional way to call a function is when an event, such as one of the events in event 328, matches one or more conditions predetermined in a subscription, such as subscription 334. Event 328 can be subscribed to by any number of subscriptions 334, and each subscription can specify different conditions and trigger different functions. In some examples, events are implemented as group-based communication system messages received in one or more group-based communication system channels. For example, all events can be published as non-user visible messages in the associated channels monitored by subscription 334. Application event 330 can be a group-based communication system message with associated metadata, which is created by the application in a group-based communication system channel. Event 328 can also be a direct message received by one or more group-based communication system users, which can be an actual user or a technical user, such as a bot. A bot is a technical user of the group-based communication system for automating tasks. A bot can be programmatically controlled to perform various functions. A bot can monitor and assist in processing group-based communication system channel activities, as well as post messages in a group-based communication system channel and react to in-channel activities of members. A bot can be able to post messages and upload files, as well as be invited to or removed from public and private channels in the group-based communication system.

[0137] Event 328 can also be any event associated with the group-based communication system. Such group-based communication system (GBCS) events 332 include events related to the creation, modification, or deletion of user accounts in the group-based communication system, or events related to messages in a group-based communication system channel, such as creating a message, editing or deleting a message, or reacting to a message. Event 328 can also involve the creation, modification, or deletion of a group-based communication system channel or channel membership. Event 328 can also involve user profile modification or group creation, member maintenance, or group deletion.

[0138] As described above, subscription 334 indicates one or more conditions that trigger a function when an event matches. In some examples, a set of event subscriptions is maintained in conjunction with a group-based communication system such that when an event occurs, information about the event is matched against the set of subscriptions to determine which, if any, of the functions in function 336 should be invoked. In some examples, an authorization framework manages the events that a particular application can subscribe to. In some cases, the types of events that are matched against subscriptions are managed by OAuth permission scopes, which can be maintained by an administrator of a particular group-based communication system.

[0139] In some examples, function 336 can be triggered by triggers 318 and events 328 that the function subscribes to. Function 336 takes zero or more inputs, performs processing (potentially including accessing external resources), and returns zero or more results. Function 336 can be implemented in various forms. First, there is a group-based communication system built-in 338, which is associated with the core functionality of a particular group-based communication system. Some examples include creating group-based communication system users or channels. Second, there is a no-code builder function 340, which can be developed by a user of a group-based communication system in conjunction with an automated user interface (such as a workflow builder user interface). Third, there is a managed code function 342, which is implemented by developing a group-based communication system application as software code in conjunction with a software development environment.

[0140] These various types of function 336 can then be integrated with API 344. In some examples, API 344 is associated with third-party services, and function 336 employs these third-party services to provide a custom integration between a particular third-party service and the group-based communication system. Examples of third-party service integrations include video conferencing, sales, marketing, customer service, project management, and engineering application integrations. In such examples, one of the triggers 318 will be a slash command 326, which is used to trigger the managed code function 342, and the managed code function makes an API call to a third-party video conferencing provider via one of the APIs in API 344. As Figure 3B shown, API 344 itself can also be the source of any number of triggers 318 or events 328. Continuing the above example, the successful completion of a video conference will trigger one of the functions in function 336, which sends a message to the third-party video conferencing provider to initiate a further API call to download and archive the recording of the video conference and store it in a group-based communication system channel.

[0141] In addition to integrating with the API 344, the functionality 336 can persistently store and access data in the table 346. In some examples, the table 346 is implemented in conjunction with a database environment associated with a serverless execution environment in which a particular event-based application is executing. In some cases, the table 346 can be provided in conjunction with a relational database environment. In other examples, the table 346 is provided in conjunction with a database mechanism that does not employ relational database technology. As Figure 3B shown, in some examples, reading from or writing to one or more tables 346 with specific data or data in the table matching a predefined condition is itself a source of a number of triggers 318 or events 328. For example, if the table 346 is used to maintain billing data in an incident management system, a count of outstanding work orders exceeding a predetermined threshold can trigger the posting of a message in an incident management channel in a group-based communication system.

[0142] Determining summary data

[0143] Figure 4 A schematic diagram depicting a specific example of an ML model capable of being trained to receive an input and use the input to output an overview document.

[0144] The server 102 (not shown) can be configured to implement the ML model 142 that ingests the input data 405. The input data 405 can be environmental data including a synchronous multimedia collaboration session, and the environmental data includes raw audiovisual data 405A, user response data 405B, context data, etc. As used herein, the phrase "raw audiovisual data" refers to audiovisual data when directly received or ingested from a provided device. Thus, the raw audiovisual data can include, for example, audiovisual data presented in a device-specific format or for which processing steps have been performed within or prior to reaching the provided device. However, such raw audiovisual data does not exclude environmental audiovisual data that is processed, filtered, or otherwise altered during or substantially contemporaneously with a synchronous multimedia collaboration session including the audiovisual data. Examples of the raw audiovisual data 405A include, but are not limited to, video streams, augmented / virtual reality streams, audio streams including uncompressed audio, lossless audio, lossy compressed audio, and video or audio stream data that has been modified or enhanced, including, for example, video or images that have been modified by filters, etc. Additionally, in some cases, the server 102 is configured to extract audio data from video data and separately parse and analyze the audio and video data. Examples of the user response data 405B include, but are not limited to, emojis, perceived physical manifestations, text (and / or keystrokes), messages, etc.

[0145] As described above with respect to Figure 2AAs discussed, a communication channel or other virtual space can be associated with data and / or content. For example, the user interface 200 described above can include a composition pane through which a user can transmit a message that includes text (i.e., ASCII characters). Thus, the ML model 142 of various examples can be trained to receive and / or obtain input data 405 for transcribing and / or summarizing a synchronous multimedia collaboration session. That is, the ML model 142 of various examples can be trained to receive input data 405 and parse the raw audiovisual data and user response data to infer the context associated with the multimedia collaboration session and / or obtain previous context data associated with the virtual space. For example, the ML model 142 can be trained to receive and parse messages or other data sent or posted to the virtual space to identify metadata associated with the message, the metadata including one or more of a sending user identifier, a message identifier, a group identifier, a communication channel identifier, etc.

[0146] The ML model can also be configured to identify specific users within the virtual space. For example, a user within the virtual space can have a specific role (e.g., CEO, CTO, etc.). The ML model 142 can also be trained to learn the relationships between users based on these roles. For example, the ML model 142 can be trained to learn the relationship between author metadata and edit metadata associated with messages and various file or document types to infer a supervisor - supervisee relationship. The ML model 142 can be trained to apply specific pre - determined weights to the communications of users based on their roles / relationships. For example, greater weight can be given to the communications of the channel owner compared to other users in the channel. In some examples, the ML model 142 can be trained to identify or receive an input regarding the weights to be applied to the communications of these individuals, and can give precedence to the communications of these individuals when generating a summary document. In at least some examples, the ML model 142 can be trained to identify communications as constituting one or more of a task assignment to a user and a deadline for completing the task, and can accordingly assign greater weight to such communications. However, the ML model 142 can additionally or alternatively give precedence to the communications of a specific user based on the recency of the user's communication, the frequency of the user's communication, communications (or other interactions of the user) above a threshold level, the user's communications marked as favorites, the user's rating, the user's expertise, user preferences implied by their respective roles, user - specified parameters, user - specified permissions, heuristics from the user's activities, and / or other interactions of the user with previously generated summary documents.

[0147] Now return Figure 4, in some cases, the machine learning model 142 can include a single ML model, which can be trained to fuse various data types including the input data 405. For example, the machine learning model 142 can include a deep neural network (DNN) capable of multimodal fusion of audio, video, and text data. Thus, some examples of the machine learning model 142 can include a DNN architecture that includes separate fully connected layers for each data modality. That is, the ML model 142 can be trained to learn the relationships between instances of the same type of input data (such as by implementing fully connected layer pairs for each data modality). Then, the outputs of each fully connected layer can be combined (such as by concatenation). Alternatively, the ML model 142 can include a first model and a second model (not shown), each of which can be trained to learn the relationships between different types of input data and the transcription. That is, the first model and the second model (including some examples of the ML model 142) can implement an ensemble learning algorithm. In some examples, the ML model 142 can include a Generative Pretrained Transformer 3 (GPT-3) model, a neural model for summarization (such as an abstractive or generative summarization model), a machine learning summarization model, natural language processing, machine learning, and / or other techniques for identifying meaning and / or sentiment in messages within a virtual space and / or a communication session.

[0148] The output data 415 can include a summary document 415A that summarizes the content published and the communications that occurred within the virtual space during the synchronous multimedia collaboration session. In this way, the communication platform can treat the summary document 415A as a summary or outline of the synchronous multimedia collaboration session and the tasks, deadlines, etc. being discussed. Thus, the functionality of the virtual space can be embedded in the summary document. The summary document 415A can accordingly identify the actions, uploads, conversations (such as conversation threads), etc. that occurred in the virtual space. Specifically, the summary document 415A can include data that characterizes one or more goals of the multimedia collaboration session according to tasks and / or deadlines. The data can include text, user objects, documents and / or references or links to files, calendar objects, relevance data, etc. The text can be organized in any suitable format, including but not limited to bullet format, paragraph format, table format, etc. For example, the text can identify one or more users and their corresponding actions to be taken according to the username or other ID. In some cases, the summary document 415A can support editable text. In at least some examples, the ML model 142 can be trained to rank the identified conversation topics according to one or more relevance schemes. The ML model 142 can implement any suitable ranking metric, including average precision, discounted cumulative gain, etc. Thus, the summary document 415A can classify text or other content according to a relevance score or other overall importance metric.

[0149] The summary document can also identify actions, uploads, conversations, etc. that occur in the virtual space. For example, the summary document can include message timing documentation (e.g., timestamps) associated with messages, audio and / or video communications, file uploads, notifications, newly posted messages, mentions or tags, mention or tag functionality, event conveyance, workflow management and tracking, access permissions, and / or other functionality that occurs within or is enabled based on the type of virtual space with the user.

[0150] In addition, the summary document 415A can include various objects that can be sorted, added, deleted, modified, etc. Such objects can include, but are not limited to, files (e.g., text, messages previously transmitted via the communication platform, audio, video, applications, local objects, remote objects (e.g., third - party objects), etc.) or links to files, and calendar objects indicating tasks to be completed or other events (e.g., pre - scheduled events, calendar invitations, etc.). Thus, the tasks of the ML model 142 in various examples can be to use the input data 405 to learn to automatically schedule meetings of the participants in a synchronous multimedia collaboration session, automatically assign tasks from a task list input via the user interface (i.e., typed into the chat interface within the virtual space), automatically send links to documents referenced in the synchronous multimedia collaboration session, etc. Thus, some examples of the ML model 142 can be trained to use the input data 405 to define a series of executable steps necessary to complete the tasks discussed during the synchronous multimedia collaboration session. In some examples, the executable steps can be defined by one or more summary objects (i.e., object code, etc.). In examples where the summary document 415A is configured to be displayed within the user interface, the execution of the series of steps can be initiated by user interaction. The ML model 142 can be trained to generate one or more summary interface elements associated with the summary objects to receive user interaction and thereby initiate the completion of the task.

[0151] For example, the ML model 142 can receive audio / video data, messages associated with the audio / video data, emojis (e.g., text / emoji-based messages during a meeting by participants in the meeting), context data (e.g., channel data, virtual space data, organizational data, relationship data, etc.), and can output one or more summary documents that summarize the content of the meeting and / or can provide any tasks, deadlines, and / or action items from the meeting. The ML model 142 can classify text to determine relevance data based on comparing an embedding associated with text from the meeting and an embedding associated with the channel (e.g., intermediate output data of the ML model 142, such as a vector encoding a portion of the data). For example, the distance between the embeddings can indicate that the discussion in the meeting is more similar to the discussion in the channel and, thus, can be more relevant to determining action items, tasks, deadlines, etc. If the distance between the embeddings meets or exceeds a threshold, the text from the meeting can be downweighted when determining the summary in the summary document.

[0152] Figure 5 An exemplary process 500 associated with generating a summary document representative of a synchronous multimedia collaboration session within a virtual space is shown.

[0153] In some examples, process 500 can begin at operation 502, which can include receiving teleconference data associated with a channel of a group-based communication platform. In some examples, the teleconference data can be context data associated with a synchronous multimedia collaboration session occurring within a channel of a group-based communication platform. Thus, in some examples, the teleconference data can include raw audiovisual data and user response data. Additionally, user response data in some examples includes one or more of the following: emojis selected by a user, a user's perceived physical manifestations (e.g., poses detected from video data using machine vision techniques), messages or text input by a user (e.g., during a meeting), or a thread of messages input by multiple users (e.g., associated with a channel or virtual space). In some examples, the user response data includes messages or text input near a user interface for video data substantially simultaneously during the generation of the audiovisual data.

[0154] In at least some examples, receiving the raw audiovisual data includes capturing and / or converting an audiovisual signal recorded by an analog or digital device. In some examples, receiving the user response data includes at least capturing keystroke or mouse click data recorded by an analog or digital device. In other words, the server 102 of various examples can be configured to interpret the selection of a user interface object representing a reaction (i.e., an emoji) or a keystroke combination (e.g., a sequence of capital letters, a sequence of repeated characters, etc.) as representing a user response. In various other examples, receiving the user response data includes at least capturing audiovisual data including a user response (e.g., a change in voice tone, a gaze pattern, etc.).

[0155] In some examples, process 500 can include, at operation 504, performing natural language processing (NLP) on the raw audiovisual data using a first trained ML model (such as ML model 142 described above) to generate transcription data associated with the teleconference. In some cases, performing NLP can include converting and / or interpreting keystroke data recorded by an analog or digital device, and performing one or more of keystroke identification analysis, keystroke verification analysis, keystroke logging, etc. That is, ML model 142 can be trained to interpret keystroke data in order to identify participants in a synchronous multimedia collaboration session, verify that a participant provided a particular input to the synchronous multimedia collaboration session, mark syntactic patterns, etc. In some examples, the ML model can further be trained to analyze and interpret keystroke data to infer that the corpus of received keystroke data includes computer code. For example, ML model 142 can be trained to recognize a predefined keystroke combination as constituting the start or end of an encoding syntax. In at least some cases, the ML model can be trained to rank the contextual relevance of individual instances of teleconference data. Thus, in the above example, ML model 142 can be trained to rank the encoding syntax as having a higher contextual relevance within a communication channel dedicated to releasing a new computer software product than the audio data of a conversation determined to be related to "today's lunch".

[0156] In some examples, ML model 142 can include a single ML model, or in some examples, can include multiple ML models with discrete tasks (e.g., a first ML model can convert speech to text, a second ML model can determine an overview, a third ML model can determine action items / related details, etc. based on contextual data from the channel).

[0157] Under certain circumstances, generating transcription data can occur in real time. Accordingly, some example ML models can be trained to transform and / or interpret received audiovisual signals and perform one or more of speech recognition, voice recognition, environmental sound recognition, or other such audio analysis. In some examples, performing NLP includes statistically correlating one or more interpreted emotions and / or gaze patterns with text data to infer context. In at least one non-limiting example, performing NLP can include determining a semantic distance associated with an identified term uttered or otherwise input by a participant within a synchronous multimedia collaboration session. Thus, if, for example, a synchronous multimedia collaboration session is initiated within the project_zen channel (which is dedicated to discussing ongoing software development work for Project Zen), the ML model 142 can be trained to recognize any one of the frequently occurring terms "development", "testing", "script", etc. as being particularly contextually relevant. In an example, the embeddings associated with these terms indicate that the discussion within the channel is more relevant when generating action items.

[0158] In some examples, process 500 can include, at operation 506, inputting the raw audiovisual data and user response data into a second trained ML model (which can be similar to the ML model 142 described above).

[0159] In some examples, process 500 can include, at operation 508, receiving a conference call summary associated with the conference call data as an output from the second trained ML model. In at least some cases, the conference call summary can be configured for display within an interface of a display of a user device. Additionally, the summaries of the various examples can thus include one or more summary objects configured to be executed by the user device. As described above, the ML model can be configured to output a context-actionable record or "summary document" representing a synchronous multimedia collaboration session. That is, the ML model 142 of the various examples described herein can be configured to create one or more data structures that can perform tasks within a group-based communication platform. Thus, in at least some cases, the summary document can include one or more collaborative documents, as described in further detail with reference to Figure 2D Further described in detail. In at least some cases, the ML model 142 can be further configured to associate one or more summary interface elements with one or more corresponding summary objects. Each summary interface element can be configured to receive user interaction and thereby initiate the execution of a task associated with the summary object. Briefly returning to our Project Zen example, as Figure 2D shown, one or more summary objects can include a task list associated with the dates of June 13th and 14th. The corresponding summary interface element includes a checkbox configured to receive user interaction to indicate the completion of a particular task.

[0160] In some examples, process 500 may terminate at operation 510, which may include causing an overview of a conference call to be displayed. In at least some examples, causing the overview of the conference call to be displayed may be initiated by a user request.

[0161] Figure 6 An exemplary user interface 600 associated with a communication platform is shown for allowing a user to request and generate an overview of a conference call. Figure 6 Also shown is the generation of artificial intelligence (AI) annotations or notes during a conference call.

[0162] Exemplary user interface 600 shows an exemplary multimedia collaboration session (also referred to herein as a conference call). Users of a group-based communication system can join and leave a synchronous multimedia collaboration session at any time without causing other users to interrupt the session. As discussed above with respect to Figure 2A a synchronous multimedia collaboration session can be based on a particular topic, a particular channel, a particular direct message, or a multi-person direct message, etc. to facilitate task completion according to a deadline. In a particular example, the multimedia collaboration session is based on a channel (e.g., the #Team-native-ai channel). In some examples, a user can start a synchronous multimedia collaboration session in a channel as a means of communicating with other members currently online in the channel.

[0163] In at least one example, user interface 600 may include multiple segments, sub-segments, and / or panes that present various data or content. For example, a first segment 602 may include an agenda sub-segment 604, an action item sub-segment 606, a file sub-segment 608, an artificial intelligence (AI) annotation 610 sub-segment, etc. In some examples, user interface 600 may include live video 620 of one or more users, a transcription 622 of the conference call (generated in real-time or near real-time), threads 626, and / or tools 628 that enable a user to control various features of the collaboration session. Although user interface 600 is shown as including specific fields or text, other content is also contemplated.

[0164] In some examples, the content presented in the first segment 602 may have been added or uploaded from another virtual space associated with the communication platform, such as a canvas, a communication channel, a workspace, a collaborative document, etc. Content uploaded or pulled from a virtual space may include text data, audio data, video data, images, files, and / or any other type of data that can be configured to be presented in the first segment 602 of the synchronous multimedia collaboration session. In some examples, content items presented in the first segment 602 may be input and / or edited by one or more users during the conference call. Alternatively or additionally, an ML model may be used to generate and / or update one or more content items in the first segment 602.

[0165] In some examples, one or more ML models can generate and output information that can be presented via user interface 600. For example, a first ML model can convert speech to text (e.g., transcribe 622) as a phone call progresses, a second ML model can further process or filter the text to remove redundant or irrelevant data, a third ML model can generate (i.e., output) an overview of the phone call from the filtered data input to the third ML model, a fourth ML model can determine action items / related details based on context data from the channel, a fifth ML model can output AI annotations near real-time during the phone call, and / or a sixth ML model can be configured to predict the likelihood of receiving a request to generate an overview. In some examples, information generated by the ML models (e.g., transcript, overview of the phone call, AI annotations, etc.) can be downloaded and / or uploaded to different parts of the communication platform (e.g., channel or workspace).

[0166] In some examples, an ML model can be trained, at least in part, by analyzing and learning the relationship between (i) a first data including prior audiovisual data and (ii) a second data including prior overview data (e.g., as learned relationship data). In some examples, an ML model can be configured to analyze context data from the channel based at least in part on channel data associated with the channel from which the phone call is initiated (e.g., as shown, the meeting is initiated from the #Team-native-ai channel). In some examples, the context data from the channel can be input to an ML model configured to output an overview of the phone call. Figure 6 The agenda subsection 604 can represent a list of items that the user plans to discuss during the phone call. The action item subsection 606 can include action items that have or have not been completed from prior phone call action items (which are copied or uploaded from a virtual space (e.g., channel, thread, canvas, etc.)) and / or action items manually entered by the user, rather than action items generated by an ML model. The file subsection 608 can include documents and / or links to documents, web pages, channels, etc. that can be relevant to the phone call. The information presented in the agenda subsection 604, the action item subsection 606, and / or the file subsection 608 can be input to one or more ML models configured to output a transcript of the phone call, an overview of the phone call, AI annotations during the phone call, and / or action items presented in the AI annotations. In some examples, additional information can provide context data and help the ML models determine the relevance of the information.

[0167]

[0168] ​Transcription 622 can be generated by an ML model using known speech recognition techniques. The transcription 622 can include a text representation of audio and / or video data, user reactions (e.g., emojis), messages typed in thread 626, etc. In some examples, the audio / video component can generate the transcription concurrently or substantially concurrently with a conference call. In some examples, the audio / video component can generate the transcription when receiving an indication of a selection of a "generate transcription" indicator (not shown), when receiving an indication of a selection of a record indicator, or automatically at the initiation of a conference call (e.g., based on permissions and / or settings associated with a user account, channel, workspace, communication platform, etc.). In at least one example, the audio / video component can be configured to present the transcription 622 in association with a virtual space associated with the audio and / or video conversation. For example, a first user can initiate an audio and / or video conversation associated with a conference or communication channel.

[0169] In some examples, the audio / video component can process the audio and / or video data among the attendees of a conference call and generate a transcription of the audio, video data, and / or message data. For example, the transcription can include timestamped text typed into thread 626 (i.e., the transcription can include that User 3 typed "Be right back" into thread 626 at 9:30 a.m.). In some examples, the transcription can include reactions and / or emojis selected by one or more users during the conference call in response to what a user said, the user's editing of the agenda, the user's editing of an action item, the user's editing of an AI annotation, the user's thread message or reply (e.g., the timestamp of a like for User 3's thread message at 9:30 a.m.). In at least one example, the audio / video component can render one or more segments of the transcription as selectable for commentary, such that members of the communication channel can comment on or further contribute to the conversation. In some examples, the audio / video component can update the transcription based on the commentary.

[0170] In some examples, the audio / video component can stop generating the transcription when receiving an indication of the completion of the conversation (e.g., an indication that the associated moderator or administrator has ended a collaboration session, a threshold number of attendees have closed the associated interface (e.g., more than 50% of the attendees, more than 75% of the attendees, etc.), the conference call is no longer being recorded, etc.). In some examples, the audio / video component can identify the completion of the conversation and then generate the associated transcription based on the completion.

[0171] In some examples, the ML model can be configured to receive as input raw audiovisual data, user response data (e.g., emojis selected by the user), detected poses in the video (e.g., the user shakes their head "no", nods "yes", waves goodbye, etc.), messages or text input by the user (e.g., the user edits the AI annotation 610), a thread of messages input by multiple users (e.g., in thread 626), and / or other data, and filter the text to remove redundant and / or irrelevant data for use. Removing redundant and / or irrelevant data or information reduces the amount of data that the ML model configured to output an overview and / or AI annotation has to process. The output from a first ML model configured to filter audiovisual data and / or user interaction data can be input into another ML model (e.g., an overview model such as a large language model (LLM)), and the other ML model outputs an overview and / or AI annotation of the conference call during the conference call. Using an additional ML model can reduce latency issues and / or error rates that may be associated with using a single ML model or an ML model that would otherwise process unfiltered data. In some examples, determining what data is irrelevant is at least partially based on determining and comparing the context of all data associated with the conference call.

[0172] The user interface 600 can also include an "Overview" affordance representation 624 (e.g., a graphical indicator) that initiates an overview of a conference call or a portion of the conference call. For example, to initiate the generation of an overview of a conference call (e.g., an audio and / or video conference in channel #Team-native-ai), the user can select the Overview affordance representation 624. In response, the user interface can display a dialogue window that allows the user to input details or criteria for the overview to be generated. For example, the user can request what a particular user or group of users said, the topic of discussion, an overview of the discussion around a keyword (e.g., emergency, deadline, promotion, etc.), an overview of one or more specific times of the meeting (e.g., the first 5 minutes and / or the last 5 minutes), an overview of action items, an overview of the high-level theme, or any combination thereof. In such examples, the requested overview can be presented in a second window within a section or pane of the user interface 600, prompting the user to download the overview to the user's computing device, etc.

[0173] In some examples, a conference call summary can be automatically generated at the end of a meeting. In some examples, a conference call summary can be automatically generated for meetings that last a threshold amount of time (e.g., at least 10 minutes, 15 minutes, etc.). In some examples, an ML model can be configured to predict the likelihood that a user will request a conference call summary at the end of a meeting. Predicting when a conference call summary can be requested enables the ML model to pre-compute at least a portion of the conference call summary before the conference call ends. For example, transcription, AI annotations, audiovisual data, and / or user response data can be input into an ML model configured to predict the likelihood or estimate that a user will request a summary of the meeting during the meeting. For example, determining that a conference call summary may be requested can be based at least in part on transcription, AI annotations, audiovisual data, etc. (indicating that a user must leave the conference call early, that a threshold number of action items (e.g., 3 or more) were generated or discussed during the meeting), based on the status or identity of one or more users in the meeting (e.g., the likelihood of requesting a conference call summary is higher when a manager is present in the meeting, and / or a new employee), the duration of the conference call (e.g., the likelihood of requesting a summary is higher when the conference call lasts a threshold amount of time), based on the number of users present in the meeting (e.g., the likelihood of a user requesting a summary is higher when a threshold number of users are present and / or participating), based on the topics discussed, based on the number of questions and / or answers discussed, etc.

[0174] In some examples, an ML model can be used to generate AI annotations 610 as the conference call progresses (i.e., in real-time or near real-time). In some examples, the ML model is configured to generate AI annotations by processing data as it becomes available or based on a predetermined data set or time frame (e.g., data received every minute, two minutes, etc.). In some examples, the ML model can be prompted to generate a data set when the transcription reaches a conclusion (e.g., when a change in topic is determined, when a user joining / leaving the meeting is detected, when the number of responses exceeds a threshold or average, etc.). That is, the ML model can be configured to measure the similarity between texts and predict whether the conversation has changed in a significant way. In at least one example, a dedicated ML model can be used to logically partition the transcription into data sets that can be input into a large language model (LLM).

[0175] The generated AI annotation 610 may include information related to a conference call. For example, sub-segments of the AI annotation 610 may summarize the user's interpretations (e.g., "User 1 summarized the previous conversion of using markdown as the output from the model"), responses, suggestions (e.g., "User 1 believes we should now continue using markdown because it easily captures bullet points", or "User 4 suggests converting the markdown from the model into formatted headings and text styles", etc.), questions (e.g., "User 2 asked how to solve the tagging problem"), annotations when the user exits the meeting (e.g., "User 3 left the meeting at 9:30 am", as determined based on transcription, thread messages, replies, camera shut-off, the user walking out of the camera's field of view, etc.), and so on.

[0176] In some examples, the AI annotation may include generated action items, key points, highlights, etc. of the conference call. In the examples, the action items, short summaries, key points, and / or highlights may be presented in bullet point format, paragraph format, table format, etc. Separate action items, short summaries, key points, and / or highlights may be associated with links that, when selected, direct the user to specific portions of the transcription and / or video associated with the specific action item, short summary, key point, or highlight. For example, selecting a link 616 associated with a bullet point that states "User 5 suggests giving the model semantically meaningful tags" may direct the user to a portion of the transcription 622 where User 5 discussed the topic and / or highlights multiple portions of the transcription associated with the topic.

[0177] The action item may represent a recommended task that the user can perform at some time after the conference call ends. For example, as Figure 6 shown, the AI annotation may include a first action item 618(1) associated with User 6 and a second action item 618(2) associated with User 7. In some examples, the action item may be associated with a due date. The action item may be associated with a single user, a group of users, or involve a single user, a group of users, or users associated with a communication channel or other virtual space (e.g., the task may state "Upload Document A to Channel A"). In some examples, the action item may be associated with an urgency level. For example, the first action item 618(1) may be associated with a first urgency level represented by a first symbol (e.g., a single exclamation point, a yellow symbol), and the second action item 618(2) may be associated with a second urgency level represented by a second symbol (e.g., a third exclamation point, a red symbol, etc.). The urgency level may be at least partially based on the due date associated with the action item, the user associated with the action item, the status or role of the user suggesting the action item (e.g., a manager suggesting an employee complete the action item), the context or relevance of the action item, and so on.

[0178] In some examples, one or more users can collaborate on AI annotations during a phone conference (e.g., comment on and / or edit the AI annotation 610 when it is generated). For example, a user can select the affordance representation 612 to add or delete text, edit the urgency of an action item, edit the user responsible for an action item, etc. In some examples, one or more users can edit the text of an AI annotation without first selecting the affordance representation for editing. In some examples, before and / or during an audio and / or video conference is initiated, a user can indicate the information to be included in an AI annotation. For example, a user can indicate that the AI annotation provides a list of questions and answers, action items, suggestions provided by a particular user, etc. discussed during the conference.

[0179] In some examples, an overview and / or an AI annotation generated during a phone conference can be automatically uploaded to another virtual space (e.g., a canvas, a channel, etc.). In some examples, a user can select the export affordance representation 614 associated with the AI annotation 610. Selecting the export affordance representation 614 can present the user with one or more options for selecting a location to export the AI annotation (e.g., via a pop-up window or a sidebar). In some examples, an AI annotation can be automatically exported to a specified location in a communication platform at some time after the phone conference ends (e.g., exported to the channel from which the audio and / or video conference was initiated or a canvas associated with the channel).

[0180] In some examples, a user interface can present a tool 628. The tool 628 enables a user to perform actions of and / or control features of a phone conference. For example, selectable elements can enable a user to mute the user's microphone, turn on / off the user's camera, share the user's screen, send reactions (e.g., emojis), update user settings, leave the phone conference, and / or invite other users. A screen sharing button can allow a user to share the user's screen with other attendees of the phone conference and provide additional controls to the user during screen sharing. For example, additional screen sharing controls can be provided to a user sharing the user's screen to specify which screen to share, annotate the shared screen, or save the shared screen.

[0181] Figure 7 An exemplary user interface 700 for requesting and generating an overview of a message thread in a virtual space (e.g., channel #Team-native-ai) is shown. The exemplary user interface 700 can include a navigation pane 702. As described above with respect to Figure 2AAs described above, the navigation pane 702 can include different graphical indicators that can be associated with various virtual spaces, such as channels 704. The exemplary user interface 700 can include any number of channels 704 that can be used to organize conversations among and between users according to topics. In some examples, the exemplary user interface 700 can include channels 704, such as general channels, social channels, Team-native-ai channels 706, and / or any other channels. When a user selects a channel 704, a channel pane 708 can be presented. In some examples, in addition to enabling the user to add other members, post content, etc., the channel pane 708 can include access to content associated with the channel. The channel pane 708 can include a user interface object (not shown) that, when actuated (e.g., selected), can display a canvas (e.g., virtual space) associated with the selected channel.

[0182] In some examples, the exemplary user interface 700 can include a pane 708 to view virtual space data that has been posted and / or input by users of the virtual space. As described above, some or all channels can include an associated (or corresponding) canvas (e.g., virtual space). That is, the Team-native-ai channel 706 can include an associated canvas that is configured to curate, organize, and / or share a collection of information among users of the Team-native-ai channel 706. The exemplary user interface 700 can include a canvas pane 710. In an example, the canvas pane 710 can indicate that the user is viewing a portion of the Team-native-ai channel canvas.

[0183] In some examples, the channel pane 708 can include channel-specific data (e.g., text). While in a channel, a user can comment on messages in a "thread". In some examples, a thread can be a message associated with another message that is not posted to the channel but is instead maintained within an object associated with the original message. Messages and / or threads can be associated with files, emojis, apps, links, images, videos, audio, and / or other attachments. The exemplary user interface 700 shows a thread 722 that includes multiple responses posted by user 3.

[0184] The user interface 700 may include a messaging box 726 that allows users within a channel to post to a virtual space. In some examples, a user may request to generate a thread overview. For example, a user may select an affordance (not shown) associated with generating a thread overview. In another example, using an ML model, an overview engine may generate a thread overview in response to receiving a command within the messaging box 726. The command may be used to overview all or a portion of data having a specific thread (e.g., a reply, an emoji, a thread-specific attachment, a link, etc.). Here, a user may input (e.g., type) various commands into the messaging box 726 to invoke a bot, an API, or other program of the communication platform to generate a thread overview. In some examples, a command may be input as if it were a reply to the thread 722. Exemplary commands include: "@bot / overview this thread" to overview data within a specific thread, " / overview data in the thread posted by <user1>" to generate a thread overview of content specifically posted by user1, or " / overview data in the tagged thread" or other topics or keywords. Although specific commands 724 are shown in Figure 7 , other commands are contemplated.

[0185] In response to inputting and submitting a command 724 (e.g., pressing submit, send, enter, etc.), a thread overview 728 may be presented. In some examples, the thread overview 728 may be presented within the user interface 700 as a reply to the thread, within a pop-up window, a sidebar, etc. The thread overview may be presented as a public post or reply to the overviewed thread, such that other users having access (e.g., permission) to the channel can view the overview. Alternatively, the thread overview may be private and visible only to the user who requested the thread overview.

[0186] In some examples, the thread overview may be updated in the event that additional communication occurs within the thread. For example, additional replies may be posted to the thread 722 after the thread overview 728 has been generated, and in response, the overview engine may utilize the ML model to generate a second thread overview or update the original thread overview. In some examples, the user who requested the generation of the thread overview may be notified that a new thread overview is available for review (e.g., receive an automated message, an email, etc.). In some examples, prior to generating a new thread overview, the communication platform may send a notification to the user confirming the thread overview the user wishes to update.

[0187] The thread overview 728 may include specific fields, text, and / or enabling representations. For example, the thread overview 728 may include an overview analysis field, the topics and / or keywords discussed in the thread, the highlights of the thread, action items for the user to perform, etc. In some examples, the thread overview 728 may present an option for the user to "send an updated overview" in the event that additional replies, reactions, etc. are received for the thread. In an example, the communication platform may send an updated thread overview to the user based in part on using an ML model to determine that an additional reply substantially changes the thread overview (e.g., changes to due dates, action items, topics discussed, etc.). However, if the additional reply contains information that would be redundant or unrelated to the original overview thread (e.g., a reply stating "Okay. Have a great weekend!" or a like emoji), the ML model may determine that the new reply, reaction, etc. is not substantially related to the original thread overview and does not automatically generate an updated thread overview.

[0188] In some examples, the thread overview 728 may include a customization enabling representation 730. Upon receiving an indication of a selection of the customization enabling representation 730, the communication platform may render (e.g., to pane 708) a window or list (not shown) containing various operations, where the communication platform may modify, update, and / or replace the data presented in the thread overview analysis. That is, the customization enabling representation 730 may enable the user to customize the information presented in the thread overview 728. For example, the user may update the output location of the thread overview, update the permissions associated with the thread overview (public versus private, identify specific users who can view and / or not view the thread overview), change the level of detail of the overview (e.g., the degree of detail), identify the type of data in the thread that the user wants to be included in the overview (e.g., include an overview of the documents rather than links within the thread), etc.

[0189] In some examples, the thread overview 728 may include an add document (doc.) enabling representation 732. In response to receiving an indication of a selection of the add doc. enabling representation 732, the communication platform may cause a dialogue window (not shown) to be presented within the virtual space. The dialogue window may include a number of fields, information inputs, etc. associated with providing the user with options for including additional documents, threads, virtual spaces, etc. for use in creating the thread overview.

[0190] In some examples, the thread overview 728 may include a feedback enabling representation 734. To improve the performance of the ML model, the user may provide feedback regarding the accuracy of the overview analysis for the thread. For example, the user may edit the thread overview, and the edited version may be used to retrain the ML model. In some examples, the user may rate the accuracy of the thread overview analysis (e.g., on a scale of 1 to 5, etc.).

[0191] In some examples, the canvas pane 710 can present an overview of a conference call (such as the conference call described with respect to Figure 6 . Of course, this is just an example, and the canvas can relate to one or more alternative topics. In some examples, the canvas pane 710 can include a conference call overview that can be generated to keep a user up to date with the conversation that takes place during the conference call. The conference call overview can include a title 712 and indicate the date and / or time at which the overview was generated.

[0192] The conference call overview can include a short overview 714 that represents a brief description or summary of the conference call and is intended to provide the user with an insight into what was discussed during the conference call. In some examples, the conference call overview can include highlights 716 of the conference call. For example, the highlights of the conference can include the more important or significant moments, points, or outcomes of the conference. In some examples, the highlights can include key points or notable aspects that efficiently summarize what happened during the conference. The highlights of the conference can depend on the purpose and nature of the conference, but can include, for example, key discussion points (the main topics or issues discussed), decisions made during the conference that are associated with or resolve action items. In some examples, the highlights can include achievements or milestones. For example, if the conference is centered around a project or initiative, the achievements or milestones reached can be identified as highlights (e.g., progress made on the project, goals achieved, obstacles overcome, etc.). In some examples, the highlights can include key new information or insights that may impact the organization or project. In some examples, the highlights can include challenges, difficulties, or problems discussed during the conference that are critical to the success of the project, as well as the solutions or plans proposed to address them. In some examples, the highlights can include future plans or goals that define strategies or objectives. In some examples, the highlights can include key speakers or attendees and the points and insights provided by the key speakers. In some examples, the highlights can include any important questions or concerns raised during the conference and / or any formal agreements or resolutions reached during the conference (e.g., strategy changes, etc.).

[0193] In some examples, the conference call overview can include action items 718 or tasks identified during the conference call. For example, an ML model can be configured to detect keywords or phrases indicating action items (e.g., "to do", "action", "task", "follow up", "assign", etc.). In some examples, the action items can be organized based on urgency level, importance, due date, department, user role or identity, etc. In some examples, the action items can be associated with a specific user, user group, virtual space (e.g., channel).

[0194] In some examples, the user interface 700 can include a messaging box 720 that allows users within a channel to post to the canvas. For example, a user can type within the messaging box 720, upload images, documents, suggest edits to an overview, react to an overview, etc. In response to the message, the messaging component can cause the virtual space to update and display the message. In some examples, the messaging component can identify mentions or tags (e.g., @ mentions or other special characters) of usernames, documents, channels, etc. associated with the communication platform. The mention or tag can be associated with a user identifier. In response to identifying the mention or tag, the messaging component can cause a notification to be presented on the user interface associated with the user identifier.

[0195] In some examples, the conference overview can include additional or supplementary information about the conference call. For example, the conference overview can include questions and answers discussed during the meeting, a list of users who joined the meeting and / or the level of individual user participation in the meeting, key topics, phrases, or keywords discussed during the meeting, an overview or recap of previous meetings related to the topic, the date of a follow-up meeting (e.g., "The follow-up meeting is scheduled for next week, September 25, 2023"), etc. In some examples, the conference overview can include AI annotations (e.g., AI annotation 610) generated during the meeting, a portion of the AI annotation, or an enabling representation that, when selected, prompts the user device to download the AI annotation.

[0196] Figure 8 An exemplary process 800 associated with generating an overview of a virtual space (e.g., an overview of a conference call and / or an overview of a message thread) is shown. As described above, the virtual space can include means for transmitting written communications among users (e.g., direct message instances, communication channels, canvases, collaborative documents, canvases, audio calls, video calls, etc.).

[0197] In some examples, at operation 802, process 800 can include receiving one or more of message data or conference call data. That is, the process can include obtaining data associated with generating an overview of a conference call, an AI annotation, a thread overview, etc. The message data and conference call data can be associated with the virtual space of the communication platform and can represent exemplary virtual space data. In some examples, the virtual space data can include raw audio-visual data of the conference call, user reaction data, emojis, perceived user performance (e.g., poses detected from video data using machine vision techniques), messages or text entered by users during the meeting, a thread of messages entered by multiple users (e.g., associated with a channel or virtual space), attachments (e.g., pdf, text, video, photo, etc.), links uploaded to the virtual space, etc.

[0198] In some examples, process 800 may include, at operation 804, performing natural language processing (NLP) on the raw audiovisual data using a first trained ML model (such as ML model 142 described above) to generate transcription data associated with the conference call. In certain cases, generating the transcription data may occur in real time. Thus, the ML models of some examples may be trained to transform and / or interpret the received audiovisual signals and perform one or more of speech recognition, voice recognition, ambient sound recognition, or other such audio analysis. In some examples, performing NLP includes statistically correlating one or more interpreted emotions and / or gaze patterns with the text data to infer context. In at least one non-limiting example, performing NLP may include determining a semantic distance associated with the recognized terms uttered or otherwise input by participants within a synchronous multimedia collaboration session. Thus, if, for example, a synchronous multimedia collaboration session is initiated within a Team-native-ai channel (which is dedicated to discussing ongoing machine learning model development), ML model 142 may be trained to recognize any one of the frequently occurring terms "development", "model", "artificial intelligence", etc. as being particularly contextually relevant. In an example, the embeddings associated with these terms indicate that the discussion within the channel is more relevant when generating action items.

[0199] In some examples, ML model 142 may include a single ML model, or in some examples, may include multiple ML models with discrete tasks (e.g., a first ML model may convert speech to text, a second ML model may process or filter the text to remove irrelevant text, a third ML model may determine an overview from the filtered text, a fourth ML model may determine action items / relevant details based on context data from the channel, a fifth ML model may output annotations near real-time during a synchronous multimedia collaboration session, a sixth ML model may be configured to predict when a user will likely request an overview before the user requests to generate an overview, etc.). In at least one example, the ML model may be configured to output different types of outputs based on receiving different cues (e.g., AI annotations and overviews of a synchronous multimedia collaboration session).

[0200] In some examples, process 800 may include, at operation 806, inputting one or more of the transcription, raw text data, audiovisual data, user response data, and / or other virtual space data generated at operation 804 into a second trained ML model. The second ML model may be configured to filter out irrelevant text and improve the efficiency and performance of the ML model used to generate the summary and / or AI annotation. By removing redundant or irrelevant information, the size of the data set can be reduced, and in turn, the computational complexity of the ML model algorithm can be reduced. In some examples, the second ML model may use embedding techniques (e.g., word embeddings, user and item embeddings, etc.) that incorporate feature selection into the training process of ML algorithms (e.g., Lasso and Ridge regression, decision trees, neural networks, etc.), use filter-based techniques (e.g., variance threshold, correlation-based feature selection, chi-square test, etc.) that involve selecting features based on statistical or mathematical criteria, use wrapper methods (e.g., forward selection, backward elimination, recursive feature elimination, etc.) that involve selecting features by training the ML model multiple times on different subsets of features, etc., to filter the virtual space data.

[0201] In some examples, process 800 may include, at operation 808, receiving the filtered text data, audiovisual data, user response data, and / or other virtual space data as an output from the second trained ML model.

[0202] In some examples, process 800 may include, at operation 810, inputting the filtered text data, audiovisual data, user response data, and / or other virtual space data into a third ML model. In some examples, the third ML model may be configured to output one or more of a conference call summary, AI annotation, thread summary, action items, etc. In at least some cases, the conference call summary, AI annotation, thread summary, and action items may be configured to be displayed within an interface of a display of a user device. Additionally, the conference call summary, AI annotation, thread summary, and / or action items may include one or more objects configured to be executed by the user device. As described above, the ML model may be configured to output a contextually actionable record or “summary document” representing a synchronous multimedia collaboration session or thread. That is, the ML model 142 of the various examples described herein may be configured to create one or more data structures that can perform tasks within a group-based communication platform. Thus, in at least some cases, the generated summary may include one or more collaboration documents, as described in further detail with reference to Figure 2D As described in further detail below. In at least some cases, the ML model 142 may be further configured to associate one or more summary interface elements (e.g., check boxes) with one or more corresponding summary objects. Each summary interface element may be configured to receive user interaction and thereby initiate the execution of a task associated with the summary object.

[0203] In some examples, process 800 may include, at operation 812, receiving a request to generate an overview of a conference call. Alternatively, process 800 may proceed to operation 814, as described below. In some cases, the overview component may receive a request to generate an overview from a user. In some examples, the request may be received in association with a virtual space. In some examples, the request may specify what content to use when generating the conference call overview or context. For example, the user interface may display a dialogue window that allows the user to enter details or criteria for the overview to be generated via the dialogue window (e.g., generate an overview of the following: what a particular user said, a group of users, the topic of discussion, mentions of keywords or phrases, overview information received during one or more specific times of the meeting, etc.).

[0204] In some examples, process 800 may include, at operation 814, generating an overview of a conference call using an ML model (e.g., a third ML model trained to output a conference call overview). For example, the overview engine 120 may utilize one or more ML models to summarize the content discussed, presented, and / or associated with the conference call. In some examples, the conference call overview may be configured to be displayed within the interface of a display of the user interface. In some examples, the conference call overview may include one or more highlights, action items, a list of participants, AI annotations, keywords, a list of questions and answers raised during the conference call, etc.

[0205] In some examples, process 800 may include, at operation 816, receiving a request to generate an overview of a thread. For example, a user may select an affordance associated with generating a thread overview. In another example, using an ML model, the overview engine may generate a thread overview in response to receiving a command within a messaging box. The command may request an overview of all or a portion of the data associated with a particular thread (e.g., a reply, an emoji, a thread-specific attachment, a link, etc.). For example, a user may enter various commands (e.g., type) into the messaging box to invoke a bot, API, or other program of the communication platform to generate a thread overview using an ML model.

[0206] In some examples, process 800 may include, at operation 818, generating an overview of a thread. That is, a thread overview may be received as output from an ML model trained to output a thread overview. In some cases, the thread overview may be configured to be displayed within the interface of a display of the user device.

[0207] Exemplary clause

[0208] A. A computer-implemented method for summarizing a teleconference of a group-based communication platform, the computer-implemented method comprising: receiving teleconference data associated with a channel of the group-based communication platform, the teleconference data including: audiovisual data; and user response data, the user response data including one or more of the following: an emoji selected by the user; a detected gesture associated with the user; a message or text input by the user; and a thread of messages input by multiple users; performing natural language processing (NLP) on the audiovisual data using a first trained machine learning model (MLM) to generate transcription data associated with the teleconference; inputting the audiovisual data and the user response data into a second trained MLM; receiving a teleconference summary associated with the teleconference data as an output from the second trained MLM, wherein the teleconference summary is configured to be displayed within an interface of a display of a user device, and further wherein the teleconference summary includes one or more summary objects configured to be executed by the user device; and causing the teleconference summary to be displayed.

[0209] B. The computer-implemented method according to paragraph A, wherein receiving the teleconference data includes receiving one or more of the audiovisual data and the user response data in real time.

[0210] C. The computer-implemented method according to paragraph A or B, wherein causing the teleconference summary to be displayed includes at least causing one or more summary interface elements representing one or more summary objects to be displayed.

[0211] D. The computer-implemented method according to paragraph C, further comprising: in response to detecting a user interaction with one or more summary interface elements representing one or more summary objects, identifying one or more executable steps associated with the one or more summary objects; and causing the one or more executable steps associated with the one or more summary objects to be executed.

[0212] E. The computer-implemented method according to any one of paragraphs A to D, wherein the second trained MLM is trained based on: (i) first data including previous audiovisual data; and (ii) second data including previous summary data, to determine relationship data between the first data and the second data, such that the second trained MLM is configured to learn to use the relationship data to generate a teleconference summary when inputting the audiovisual data.

[0213] F. The computer-implemented method according to any one of paragraphs A to E, further comprising: determining context data based at least in part on channel data associated with a channel of the group-based communication platform; and inputting the context data into the second trained MLM, wherein the output from the second trained MLM is further based on the context data.

[0214] G. A computer-implemented method according to any one of paragraphs A to F, wherein the context data includes one or more messages generated asynchronously with respect to the audiovisual data.

[0215] H. A computer-implemented method according to any one of paragraphs A to G, further comprising: generating, by a second trained MLM, action items associated with a conference call summary; generating user identifiers based on the action items; and sending the action items to a computing device associated with the user identifier for presentation in a display associated with an instance of a group-based communication platform, examples being associated with the user identifier.

[0216] I. A computer-implemented method according to any one of paragraphs A to H, wherein generating the action items includes: classifying text associated with the conference call; and determining a relevance weight associated with the text based on the classification of the text.

[0217] J. A computer-implemented method according to any one of paragraphs A to I, wherein a message or text input by a user is input near a user interface of video data substantially simultaneously during generation of the audiovisual data.

[0218] K. A system comprising: one or more processors; and a non-transitory memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations including: receiving conference call data associated with a conference call of a group-based communication platform, the conference call data including audiovisual data and user response data; performing natural language processing (NLP) on the audiovisual data using a first trained machine learning model (MLM) to generate transcription data associated with the conference call; inputting the audiovisual data and the user response data into a second trained MLM; receiving a conference call summary associated with the conference call data as an output from the second trained MLM, wherein the conference call summary is configured to be displayed within an interface of a display of a user device, and further wherein the conference call summary includes one or more summary objects configured to be executed by the user device; and causing the conference call summary to be displayed.

[0219] L. The system according to paragraph K, wherein the second trained MLM is trained based on: (i) first data including previous audiovisual data; and (ii) second data including previous summary data, to learn a relationship between the first data and the second data such that the second trained MLM is configured to learn to use the learned relationship to generate a conference call summary when inputting audiovisual data.

[0220] System according to paragraph K or L, in which the teleconference data is associated with a channel of a group-based communication platform, the operation further comprising: determining context data based at least in part on channel data associated with the channel of the group-based communication platform; and inputting the context data into a second trained MLM, wherein the output from the second trained MLM is further based on the context data.

[0221] System according to any one of paragraphs K to M, wherein the context data includes one or more messages generated asynchronously with respect to the audiovisual data.

[0222] System according to any one of paragraphs K to N, the operation further comprising: generating, by a second MLM, action items associated with the teleconference summary; generating user identifiers based on the action items; and sending the action items to a computing device associated with the user identifier for presentation in a display associated with an instance of the group-based communication platform, e.g., associated with the user identifier.

[0223] System according to any one of paragraphs K to O, wherein receiving the teleconference data includes receiving one or more of real-time audiovisual data and user response data.

[0224] One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving teleconference data associated with a teleconference of a group-based communication platform, the teleconference data including audiovisual data and user response data; performing natural language processing (NLP) on the audiovisual data using a first trained machine learning model (MLM) to generate transcription data associated with the teleconference; inputting the audiovisual data and the user response data into a second trained MLM; receiving a teleconference summary associated with the teleconference data as an output from the second trained MLM, wherein the teleconference summary is configured to be displayed within an interface of a display of a user device, and further wherein the teleconference summary includes one or more summary objects configured to be executed by the user device; and causing the teleconference summary to be displayed.

[0225] One or more non-transitory computer-readable media according to paragraph Q, the operation further comprising: generating, by a second MLM, action items associated with the teleconference summary; generating user identifiers based on the action items; and sending the action items to a computing device associated with the user identifier for presentation in a display associated with an instance of the group-based communication platform, e.g., associated with the user identifier.

[0226] S. One or more non-transitory computer-readable media according to paragraph Q or R, wherein the user response data includes messages or text input to the user interface proximate to the video data substantially contemporaneously during the generation of the audiovisual data.

[0227] T. One or more non-transitory computer-readable media according to any one of paragraphs Q to S, wherein the teleconference data is associated with a channel of a group-based communication platform, and the operations further include: determining context data based in part on channel data associated with the channel of the group-based communication platform; and inputting the context data into a second trained MLM, wherein the output from the second trained MLM is further based on the context data.

[0228] While the exemplary clauses described above are described with respect to a particular implementation, it should be understood that in the context of this document, the content of the exemplary clauses may also be implemented via a method, apparatus, system, computer-readable medium, and / or another implementation. Additionally, any one of Examples A to T may be implemented alone or in combination with any other one or more of Examples A to T.

[0229] Conclusion

[0230] While one or more examples of the techniques described herein have been described, various changes, additions, permutations, and equivalents thereof are included within the scope of the techniques described herein.

[0231] In the description of the examples, reference is made to the drawings, which form a part of the present invention, and which illustrate specific examples of the claimed subject matter by way of illustration. It should be understood that other examples may be used and changes or alterations may be made, such as structural changes. Such examples, changes, or alterations do not necessarily depart from the scope of the claimed subject matter as contemplated. While the steps herein may be presented in a particular order, in some cases the ordering may be changed so that specific inputs are provided at different times or in a different order without changing the functionality of the described systems and methods. The disclosed programs may also be executed in a different order. Additionally, the various computations herein need not be performed in the order disclosed, and other examples using alternative computational orderings can be readily implemented. In addition to being reordered, computations can also be broken down into sub-computations that achieve the same result.

Claims

1. A computer-implemented method for summarizing a telephone conference of a group-based communication platform, the computer-implemented method comprising: Receiving telephone conference data associated with a channel of the group-based communication platform, the telephone conference data including: Audio-visual data; and User response data, the user response data including one or more of the following: Emojis selected by the user, Detected postures associated with the user; Messages or texts input by the user; and Threads of messages input by multiple users; Performing natural language processing (NLP) on the audio-visual data using a first trained machine learning model (MLM) to generate transcription data associated with the telephone conference; Inputting the audio-visual data and the user response data into a second trained MLM; Receiving a telephone conference summary associated with the telephone conference data as an output from the second trained MLM, wherein the telephone conference summary is configured to be displayed within an interface of a display of a user device, and further, wherein the telephone conference summary includes one or more summary objects configured to be executed by the user device; and Causing the telephone conference summary to be displayed.

2. The computer-implemented method according to claim 1, wherein receiving the teleconference data comprises: Receiving one or more of the audio-visual data and the user response data in real time.

3. The computer-implemented method according to claim 1, wherein causing the display of the conference call overview comprises: At least causing to display one or more summary interface elements representing the one or more summary objects.

4. The computer-implemented method according to claim 3, further comprising: In response to detecting a user interaction with the one or more summary interface elements representing the one or more summary objects, identifying one or more executable steps associated with the one or more summary objects; And Causing the one or more executable steps associated with the one or more summary objects to be executed.

5. The computer-implemented method according to claim 1, wherein the second trained MLM is trained based on: (i) first data including previous audio-visual data; and (ii) second data including previous summary data, to determine relationship data between the first data and the second data, such that the second trained MLM is configured to learn to use the relationship data to generate a telephone conference summary when inputting the audio-visual data.

6. The computer-implemented method according to claim 1, further comprising: Determining context data based at least in part on channel data associated with the channel of the group-based communication platform; And Inputting the context data into the second trained MLM, wherein the output from the second trained MLM is further based on the context data.

7. The computer-implemented method according to claim 6, wherein the context data includes one or more messages generated asynchronously with the audio-visual data.

8. The computer-implemented method according to claim 1, further comprising: Generating action items associated with the telephone conference summary by the second trained MLM; Generating user identifiers based on the action items; And Send the action item to a computing device associated with the user identifier for presentation in a display associated with an instance of the group-based communication platform, the example being associated with the user identifier.

9. The computer-implemented method of claim 8, wherein generating the action item comprises: Classifying text associated with the teleconference; And Based on classifying the text, determining a relevance weight associated with the text.

10. The computer-implemented method of claim 1, wherein the message or text input by the user is input near a user interface of video data substantially simultaneously during generation of the audiovisual data.

11. A system comprising: One or more processors; And A non-transitory memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform operations including: Receiving teleconference data associated with a teleconference of a group-based communication platform, the teleconference data including audiovisual data and user response data; Performing natural language processing (NLP) on the audiovisual data using a first trained machine learning model (MLM) to generate transcription data associated with the teleconference; Inputting the audiovisual data and the user response data into a second trained MLM; Receiving a teleconference summary associated with the teleconference data as an output from the second trained MLM, Wherein the teleconference summary is configured to be displayed within an interface of a display of a user device, and further wherein the teleconference summary includes one or more summary objects configured to be executed by the user device; and Causing the teleconference summary to be displayed.

12. The system of claim 11, wherein the second trained MLM is trained based on: (i) a first data including previous audiovisual data; and (ii) a second data including previous summary data, to learn a relationship between the first data and the second data such that the second trained MLM is configured to learn to use the learned relationship to generate a teleconference summary when inputting the audiovisual data.

13. The system of claim 11, wherein the teleconference data is associated with a channel of the group-based communication platform, and the operations further include: Determining context data based in part on channel data associated with the channel of the group-based communication platform; And Inputting the context data into the second trained MLM, wherein the output from the second trained MLM is further based on the context data.

14. The system of claim 11, the operations further include: Generating an action item associated with the teleconference summary by the second MLM; Generating a user identifier based on the action item; And Send the action item to a computing device associated with the user identifier for presentation in a display associated with an instance of the group-based communication platform, the example being associated with the user identifier.

15. One or more non-transitory computer-readable media storing processor-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the operations according to any one of claims 1 to 14.

Images