Intelligent summarization based on automatic learning and user input contextual analysis

Abstract

The technology provided herein improves existing systems by automatically generating summaries of documents in response to user input defining selected segments of the document. The document can include any type of content, such as but not limited to channel conversations, chat threads, transcripts, word processing documents, spreadsheets, etc. When a user indicates a selection of a segment, the system can dynamically update the summary of the segment to inform the user of important information shared in the selected segment. The summary can include a textual description of information having a threshold priority level. The system can analyze documents referenced within the selected segment and provide a summary of the documents. The technology disclosed herein also provides multiple graphical elements that convey additional context for each portion of the summary.

Description

BACKGROUND

[0001] There are many different systems and applications that allow users to collaborate. For example, some systems provide a collaboration environment that allows participants to exchange live video, live audio, and other forms of data within a communication session. In other examples, some systems allow users to post messages to channels that have access restricted to a selected group of individuals for the purpose of facilitating team-focused or topic-focused conversations.

[0002] Despite the many different types of systems and applications that allow users to collaborate, users can not always benefit from a particular exchange of information or meeting using these systems. For example, if a person is on leave, the user can miss many events. The user can need some time to catch up on the details of each event. In one particular example, when it comes to tracking messages within a channel that has a large number of entries, a user can have difficulty keeping up with the conversation. Worse, if a person is away from the office for a long time, such as on vacation, there can be hundreds or even thousands of messages within a particular channel. Given the large amount of information that can be shared, it can be difficult for anyone to catch up on the events of a channel, or even worse, the events of multiple channels.

[0003] These shortcomings of existing systems can result in a loss of productivity and inefficient use of computing resources. When a person is needed to review a large amount of data, it can not be possible to optimize the use of a large amount of computing resources, such as network resources and processing resources. SUMMARY

[0004] The technology disclosed herein improves existing systems by automatically generating a summary of a document in response to user input that defines a selected segment of the document. The document can include any type of content, such as but not limited to a channel conversation, a chat thread, a transcript, a word processing document, a spreadsheet, a presentation file, etc. When the user indicates a selection of a segment, the system can dynamically update a summary of the segment to inform the user of important information that was shared within a particular time period. The summary can include a textual description of the important information. The textual description can include a computer-generated sentence or a sentence that is extracted from the selected segment. In addition, the system can analyze documents that are referenced within the selected segment and provide a summary of the content of the documents. The summary can be dynamically adjusted based on user input. Thus, when the user adjusts the selection of the segment, the summary can be updated in response to each adjustment of the input. For purposes of illustration, the summary that is generated in response to user input that defines a selection of a segment of a document is referred to herein as an "instant summary."

[0005] In some embodiments, the summary can include computer-generated portions and other portions that are direct quotes of selected segments. The summary can graphically distinguish the computer-generated portions from the other portions that are direct quotes of selected content. For example, if the summary includes two computer-generated sentences that describe selected segments of a channel and three sentences that are direct quotes of posts of the selected segments, the two computer-generated portions of the summary can be a first color and the other sentences can be a second color. By distinguishing the quoted portions from the computer-generated portions, the system can easily communicate the reliability of the content.

[0006] In some embodiments, different portions of the summary can include links to resources of particular content. For example, if a particular portion of the summary is generated to summarize channel posts of a particular user, the identity of the user can be displayed in association with that particular portion of the summary. In some configurations, the display of the user identity can be in response to a particular user input, such as a hover or another input that indicates a selection of that portion of the summary.

[0007] A number of different inputs can be utilized to select segments of a document. For example, the inputs can include voice commands or other gestures that indicate a selection of a segment. In one illustrative example, a user can provide an input stating "I want a summary of chat threads from the shipping team channel from the beginning of the year to now." In another example, a user can state "please show me a summary of the shipping team channel when I am not in the office." In such embodiments, the system can then access scheduling information from an external resource, such as a calendar database. The scheduling information can then provide parameters for defining the selected segments.

[0008] Among other benefits, the system can also identify permissions for certain portions of the summary and take action on those summaries based on the permissions. For example, if someone attaches a file to a thread of a channel and the file has protected portions, the summary can redact the protected portions of the summary.

[0009] The efficiencies resulting from the above-described techniques can result in more efficient use of computing systems. In particular, by automating a number of different processes for generating and identifying summaries, user interactions with computing devices can be improved. Reducing manual data entry and improving user interactions between people and computers can result in a number of other benefits. For example, by reducing the need for manual entry, unintentional input and human error can be reduced. This can ultimately result in more efficient use of computing resources, such as memory usage, network usage, processing resources, etc. The techniques disclosed herein can result in more efficient use of computing resources by eliminating the need for a person to retrieve, display, and view large amounts of data.

[0010] Additional features and technical advantages will be readily apparent to one skilled in the art from the following detailed description, simply from the attached claims as well as from the drawings. The summary is provided to introduce a selection of concepts discussed in the detailed description below to provide a context for the disclosed subject matter. The summary does not necessarily describe the key or essential features or functions of the claimed subject matter, and is not intended to be used in determining the scope of the claimed subject matter. For example, the term "technology" can refer to systems, methods, computer-readable instructions, modules, algorithms, hardware logic, and / or operations as permitted throughout the context of the present document. BRIEF DESCRIPTION OF DRAWINGS

[0011] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of each reference number identifies the figure in which that reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items. References to individual items in a plurality of items can use a specific reference number without a letter sequence to refer to each individual item. General references to items can use a specific reference number without a letter sequence.

[0012] Figure 1 An example user interface that can be generated from an application configured to display a document having multiple segments is shown.

[0013] Figure 2 A first step of a selection process for a subset of segments is shown.

[0014] Figure 3 A second step of a selection process for a subset of segments is shown.

[0015] Figure 4 An example summary that is dynamically updated based on adjustments to user input is shown.

[0016] Figure 5 An example of a user interface that distinguishes between computer-generated sentences and content extracted from a document is shown.

[0017] Figure 6 An example of a user interface that provides graphical elements that reveal information sources is shown.

[0018] Figure 7A A first step of a transition of a user interface when a user selects a portion of a summary is shown.

[0019] Figure 7B A second step of a transition of a user interface when a user selects a portion of a summary is shown.

[0020] Figure 8A A user interface that displays multiple summaries generated from selected segments of a document is shown.

[0021] Figure 8B The example shown is a user interface displaying updated graphical elements, representing the history of channel items with highlighted topics.

[0022] Figure 9 An example data flow diagram is shown, illustrating how a system for generating one or more summaries can gather information from various resources.

[0023] Figure 10 This is a flowchart illustrating aspects of routines used to efficiently generate and manage tasks.

[0024] Figure 11 This is a computing system diagram illustrating various aspects of the illustrative operating environment used in the techniques disclosed herein.

[0025] Figure 12 This is a computing architecture diagram illustrating the configuration and operation of a computing device that can implement the various aspects of the technologies disclosed herein. Detailed Implementation

[0026] Figure 1 An example user interface 10 generated from an application configured to display a document is shown. In this example, the document has the form of channels comprising multiple channel entries provided by multiple users. For illustrative purposes, each entry in the document is referred to herein as a fragment. The document may also be referred to herein as a “thread” or “original document.” Although this example illustrates a document in the form of a channel dialogue, it will be understood that any type of data structure with multiple fragments can be used with the aspects disclosed herein. For example, the document may be in the form of a word processing document, and each fragment may consist of any type of formatting characters, such as paragraph breaks, section breaks, page breaks, etc. Similarly, for spreadsheets and other documents, fragments may include any type of data portion, such as a single cell or a group of cells. Other data formats may include comma-separated text documents, images with fragments separated by graphic features, and so on.

[0027] exist Figure 1 In the example shown, the application that generates user interface 10 can display a history graphical element 11 (i.e., an overview of the comprehensive history) representing the history of a thread with multiple segments. User interface 10 may also include another graphical element, namely a visual item window 12, for indicating the segments displayed in a window of viewable items. As shown, the entry displayed in the center of user interface 10 is represented in a highlighted portion of the history graphical element 11.

[0028] In some configurations, the system executing the application for generating the user interface 10 can receive input indicating a selection of a subset of the clips. As described below, the input can define a timeline for generating a summary of the subset of clips within the timeline.

[0029] Figure 2 A first step of the selection process is shown. When the user hovers a pointing device or interacts with a touchscreen at a particular location within the history graphical element 11, the system can generate a start time graphical element 18 that can display a time and / or date associated with that selected location. As shown, the graphical element can direct the user to find a desired start time for the timeline.

[0030] Figure 3 A second step of the selection process is shown. After selecting a start time, the user can move a pointing device or interact with a touchscreen to select a second location within the history graphical element. In this example, the user has moved the pointer down to a second location, selecting a subset of clips, such as selecting the channel entry. As the user positions the pointer, the system can generate an end time graphical element 19 that can display a time and / or date associated with that selected location. In this example, the user input indicates a selection of clips from January 1, 2020 to February 27, 2020.

[0031] Once the start time and end time of the timeline are established, the system analyzes the subset of clips, such as the selected clips, to generate a plurality of sentences describing content of at least a portion of the subset of clips. Additionally, or alternatively, the system can select quotes from the content of the selected clips.

[0032] The system can generate a plurality of sentences summarizing the content of the selected snippets. In some configurations, the sentences can be generated from a subset of snippets that meet a threshold requirement or a priority threshold. In one illustrative example of a threshold requirement, a plurality of sentences summarizing a plurality of selected snippets can be generated based on a priority of a particular theme. For example, if there are several different entries (e.g., snippets) stating "transportation is a problem," "transportation will be delayed," "we can be delayed due to packaging," and "transportation will not be on time, will be late," the priority of a particular keyword can be determined using the number of occurrences of the particular keyword, and the priority can be compared to a threshold (e.g., a priority threshold). If the number of occurrences of a particular keyword exceeds the threshold, the system can determine that the particular keyword is the theme. The system can then generate a number of sentences around the theme. In the current example, the word "transportation" occurs a threshold number of times. In response to the determination, the word "transportation" is given priority, causing the system to generate a plurality of sentences around the selected word. Other words around the selected word, also referred to herein as supporting words, can be used to generate sentences, such as "delay," "late," and the like. Common phrases can be used, such as "_ will be _." The system can fill in the fields with the selected word and the supporting words to produce a sentence, such as "transportation will be late." This example is provided for illustrative purposes and should not be construed as limiting. It can be appreciated that the system can generate a plurality of sentences from different snippets based on any type of threshold requirement, and the threshold requirement is not limited to embodiments involving priority. For example, the threshold requirement can define any criteria that can be used to identify a subset of snippets or a relationship between portions of a subset of snippets. For example, a subset of snippets can meet the threshold requirement based on a plurality of phrases or keywords that align with a template or other keywords defined in one or more preferences. Any suitable machine learning techniques for identifying similarities between keywords and phrases can also be utilized to select a subset of snippets of a document.

[0033] It can be appreciated that priority can be based on a number of other factors. For example, in addition or alternatively, priority can be based on explanatory language from one or more snippets. For example, the word "important" or "urgent" can increase the priority of a particular set of content within a thread. Other predetermined adjectives or adverbs can be used in conjunction with a word count of a particular keyword to increase or decrease priority. These examples are provided for illustrative purposes and should not be construed as limiting. It can be appreciated that any suitable explanation or analysis of a thread or document can be used to associate priority with particular snippets of the thread or document.

[0034] In some configurations, the system can only compute keywords for certain categories, such as nouns or verbs. Some words can be ignored, such as “the,” “and,” “she,” “he,” etc. Additionally, the system can also select multiple sentences from the selected snippet and directly quote the sentences. A general algorithm for checking the grammar of each sentence can be used. If a sentence in the selected snippet meets one or more criteria, such as it is a complete sentence, it has proper grammar, and it contains a subject, then the system can quote the sentence in the summary.

[0035] Figure 3 An example of a summary 20 generated by the system is also shown. In this example, the summary is based on the content of the selected snippet. In addition to generating sentences that describe the content of the selected snippet, the system can also identify the username and display the username 61 in one or more graphical elements. The summary 20 also includes a section 62 that contains the generated and quoted sentences.

[0036] In some configurations, the summary can be dynamically updated. Thus, when the user adjusts the selection of the snippet, the summary can be updated in response to each adjustment of the input.

[0037] Figure 4 An example of this feature is shown. As shown, the user has expanded the selected region of the history, and the user input now indicates a start date of January 1, 2020 and an end date of March 30, 2020. Based on this indication, the summary is dynamically updated to include new sections and new content based on this adjustment to the input.

[0038] In Figure 4 In the example shown, the summary 20 includes an action item 63 and a related file 64. The action item can be generated by detecting a request within the selected snippet. For example, if a channel entry directs a task to a particular user, the task will be noted and entered into the summary as an action item. The action item can also be displayed with a radial checkbox that can be initiated by the recipient of the action item when the task is complete. Additionally, the system can identify files that are attached to or linked to the selected snippet. Such files can be identified and provided within the summary.

[0039] Also as Figure 4As shown, additional content derived from the file can also be summarized. In this example, the system analyzed the guideline.pptx file and determined that the document includes content that references the vendor Acme, Inc. Such content can be included in the summary and can be generated using the techniques disclosed herein or any other techniques for summarizing documents. In one illustrative example, the document can be analyzed for content that has a threshold level of relevance to the selected segment. For example, if the selected portion indicates a particular topic that has a threshold priority, then any document content having keywords related to that topic can be extracted from the document and provided in the summary.

[0040] Any suitable technique for identifying a threshold level of relevance can be utilized. One or more machine learning algorithms can be utilized to identify similarities and differences between a file and one or more selected segments of a thread or document and score them. The threshold level of relevance can be based on a number of keywords or phrases shared between the file and the one or more selected segments of the thread or document. Alternatively or additionally, one or more machine learning mechanisms can be utilized. For example, a classification mechanism can be utilized to determine whether a file has a threshold level of relevance to one or more selected segments of a thread or document. The classification mechanism can classify portions of the file and segments of the file into different categories that provide an indication of whether a similarity or difference exists. For example, portions of the file can be classified into a first category (e.g., unlikely to be relevant) and a second category (e.g., likely to be relevant). In some configurations, more classification categories can be used. In other examples, a statistical mechanism can be utilized to determine whether a file has a threshold level of relevance to one or more selected segments of a thread or document. For example, a linear regression mechanism can be used to generate a score that indicates a likelihood that a file has a threshold level of relevance to one or more selected segments of a thread or document. Linear regression can refer to a process of modeling the relationship between one variable and one or more other variables. Different linear regression models can be used to compute the score. For example, least squares can be used, maximum likelihood estimation can be used, or another approach can be used. These machine learning algorithms can also be used for other aspects of the present disclosure.

[0041] In some configurations, the system can receive an adjustment to the input and the adjustment can increase or decrease the number of segments in the subset of segments. The system can then add content to the summary in response to the adjustment to the input that increases the number of segments in the subset of segments. Alternatively, the system can remove content from the summary in response to the adjustment to the input that decreases the number of segments in the subset of segments.

[0042] In some embodiments, the summary can include computer-generated portions and other portions that are direct quotes of the selected content. The user interface can graphically distinguish the computer-generated portions from the other portions that are direct quotes of the selected snippet. For example, if the summary includes two computer-generated sentences that describe the selected snippet of a channel and three sentences that are direct quotes of a post of the selected snippet, the two computer-generated portions of the summary can be a first color and the other sentences can be a second color. By distinguishing the quoted portions from the computer-generated portions, the system can easily convey the reliability of the content.

[0043] Figure 5 An example of a user interface 20 is shown that includes a first graphical element 22 that indicates the computer-generated portions of the summary. The example also includes a second graphical element 23 that indicates the portions of the summary that are direct quotes of the selected snippet. This example is provided for illustrative purposes and should not be construed as limiting. It can be appreciated that other graphical elements can be used to distinguish the computer-generated portions from the quoted portions. Different colors, shapes, and / or textual descriptions can be used to distinguish the portions.

[0044] In some configurations, the user interface 20 of the summary can also include a plurality of graphical elements that indicate the sources of information included in the summary. These graphical elements can identify the user that provided the information or the system that provided the information. Figure 6 An example of a user interface 20 that provides graphical elements that reveal the sources of information is shown. In this example, when the user selects a portion of the summary, the user interface 20 transitions from a first state (left UI) to a second state (middle UI). In this example, the selected portion describes the “beta ship schedule.” In response to the selection, the system causes the user interface 20 to display a graphical element 24A that indicates the identity of the user that contributed to the content of the selected portion.

[0045] When the user selects another portion of the summary, the user interface 20 transitions from the second state (middle UI) to a third state (right UI). In this example, the newly selected portion, which describes “project 08423,” is highlighted. In addition, the system causes the user interface 20 to display another graphical element 24B that indicates the identity of another user that contributed to the content of the newly selected portion.

[0046] Figure 7A And Figure 7B An example of the transition of the user interface 10 when the user selects a portion of the summary is shown. Specifically, Figure 7A A first step of the transition of the user interface when the user selects a portion of the summary is shown. Figure 7BA second step showing the transition of the user interface when the user selects a portion of the summary is shown. In this example, the system can cause the user interface 10 to navigate to a particular portion of the document related to the selected portion of the summary 20 in response to the user's selection of a particular portion of the summary 20. Figure 7A The user interface 10 shown in FIG. 1 is configured to receive user input within certain portions of the summary. For example, the user can select a particular topic, action item, or file. In some embodiments, the selection can be made through a second level interaction, such as a hover and "tap" (actuation of a mouse button) of a pointing device or double tap of a touch screen, or a voice command. Based on the second level interaction, the user input indicates a selection of the "Beta Ship Schedule" portion, and in response to the selection, as shown in FIG. 2, the main display area 39 of the user interface 10 can automatically navigate, e.g., scroll, to the document fragment related to the selected portion of the summary. Figure 7B

[0047] In some configurations, the system can generate multiple different summaries based on a set of selected fragments. For example, as shown in FIG. 3, if the system identifies multiple topics such as "Transportation," "Safety," and "Design," the system can generate a summary around each topic. The user can then select one of the summaries for viewing. The selection of a particular summary can be used as input to indicate that the particular summary has a higher priority level than the other summaries. Such data can be communicated back to the system 100 for the purpose of updating the machine learning data. In this way, the system can generate summaries with a higher priority level for the topics selected by the user in the future. If the user selects multiple different summaries, the order in which the summaries are selected can indicate the priority. For example, the first selected summary can have a higher priority level than the second selected summary. Figure 8A

[0048] Figure 8A An example of a user interface 30 showing multiple summaries 31 based on different topics is shown. In this example, the first summary 31 A is about "Transportation," the second summary 31 B is about "Safety," and the third summary 31 C is about "Design." The user can select each summary in response to a selection, and a summary such as the one described above can be displayed to the user. In addition, the topic of the summary and other supporting keywords can be sent back to the machine learning service to update the machine learning data in response to the selection of the summary. The machine learning service can then increase the priority or relevance level with respect to the selected topic and supporting keywords for the purpose of improving the generation of summaries in the future.

[0049] ​​Machine learning data collected from the techniques disclosed herein can be used for many different purposes. For example, as a person interacts with an abstract, a machine learning service can interpret the interaction to curate, order, or arrange the sentences of the abstract. The user interaction can be based on any type of detectable activity. For example, the system can determine whether a user read the abstract. In another example, the system can determine whether a person had a particular interaction with the user interface displaying the abstract, e.g., they selected a task within the abstract, opened a file within the abstract, etc. If a particular arrangement of sentences proves useful to many users, the arrangement of sentences can be communicated to other users to optimize the effectiveness of board case abstracts.

[0050] Also as shown in FIG. 3B, the user interface 30 displays a plurality of selectable interface elements that display other topics. These topics can come from keywords found in the selected snippet, but the number of occurrences of these keywords did not reach a threshold. Such keywords can be provided as particular topics for the user to select. Figure 8A

[0051] In response to a user selection of a selectable interface element, such as the "Advertising" button 33A or the "development" button 33B, the system 100 can use keywords or sentences found near that topic to generate an abstract about those topics. For example, if multiple entries of the channel contain the word "Advertising," the keywords in the same sentence as the word "Advertising" can be used to generate an abstract. In addition, the full sentence can be quoted from a particular channel entry and used for at least a portion of the generated abstract.

[0052] In response to a selection of a topic, the system can send data defining the topic to a machine learning service to update machine learning data. The machine learning service can then increase a priority or a relevance level for the selected topic and supporting keywords for the purpose of improving the generation of future abstracts.

[0053] In general, the techniques disclosed herein (some of which are described in Figure 8A ​The user interface 10 can also allow the user to refine the parameters used to generate the summary. Some embodiments enable the system 100 to identify more than one topic to generate a summary. For example, a summary can include two topics that both involve multiple usernames. If the summary appears too broad, the user reviewing the summary can narrow the summary to a single topic or to a particular person. For example, by using a voice command or any other suitable type of input 90, the user can cause the system 100 to generate an updated summary 91 by adding parameters to refine the summary to a preferred topic, a particular person, or a particular group of people. This can allow the user to further control the level of granularity of the summary. This can be helpful for very large threads that can have multiple topics. In addition, this type of input can be provided to the machine learning service to improve the generation of other summaries. For example, if a particular person or topic is selected in the input 90 a threshold number of times, the priority of that particular topic or person can be increased, which can make that person or topic more prevalent in other summaries.

[0054] In addition to updating the summary based on user interactions for selecting a topic for a person, the system 100 can also update the history graphical element 11 that represents the history of the thread or document. Figure 1 Figure 8B An example is shown in which the user interface 10 displays an example of an updated history graphical element 11 that represents the history of the channel item with a highlighted topic. In this example, the user interface 10 includes a history graphical element 11 that is modified based on user interactions with the summary or summary list, such as a selection of a topic. For example, if a person interacts with the user interface of the summary 31 and selects a particular summary that focuses on one topic or selects a topic user interface element, the data defining that selection can be stored and used to provide an indication of the priority of that person with respect to that topic. Thus, through analysis of such activity data, the system can display a highlight or other graphical feature that draws the user’s attention to a particular segment of the thread or document that is related to the selected topic or selected person. Figure 8A

[0055] Consider the following scenario: a person interacts with the user interface of the summary 31 and selects a particular summary that focuses on one topic or selects a topic user interface element, the data defining that selection can be stored and used to provide an indication of the priority of that person with respect to that topic. Thus, through analysis of such activity data, the system can display a highlight or other graphical feature that draws the user’s attention to a particular segment of the thread or document that is related to the selected topic or selected person. Figure 8A ​​The user interface 30 of the system 100 can be configured to provide a user with the ability to select one or more topics of interest. For example, the user interface 30 can include a transport and advertisement interface element that a user can interact with and select to express interest in these respective topics. In response to such an indication, the system can cause display of a history with a first graphical element 84A configured to draw the user's attention to a location within the thread, e.g., one or more document segments, that are relevant to one of the selected topics, e.g., the "transport" topic. Also shown, the history image element 11 also includes a second graphical element 84B configured to draw the user's attention to a location within the thread, e.g., one or more document segments, that are relevant to the other selected topic, e.g., the "advertisement" topic. By providing these updates based on user interaction with the summary or the user interface that displays the summary, the user can easily view a portion of the document, e.g., a segment that can be relevant to a topic of interest. This helps the user to navigate through a large thread and, in some cases, to direct the user directly to an area of interest. By providing graphical elements that direct the user to important information, the system can save a significant amount of computing resources by avoiding the need to typically use manual user input to find areas of interest.

[0056] Figure 9 The system is shown how it interacts with many different resources to generate a summary. In some embodiments, the system 100 can send a query 77 to an external file source 74 to obtain a document 80 that is referenced in the selected segment. The query can be based on information received from the original document, e.g., a channel. In addition, the system 100 can send another query 77 to a calendar database 75 to receive calendar data 82. Such information can be used to identify dates and other scheduling information that can be used to generate a summary. For example, if a particular deadline is stored in the calendar database 75, a query can be constructed from the content of the one or more selected segments, and the calendar database 75 can send calendar data 82 to confirm one or more dates. As also described herein, the system 100 can send usage data 78 to one or more machine learning services 76. In response, the machine learning services 76 can return machine learning data 83 to help the system 100 generate a summary. For example, a priority can be communicated back to the system 100 regarding certain keywords to help the system generate a relevant summary with the most relevant topics to the conversation or the selected set of segments. The system 100 can also access other resources, e.g., a social network 77. For example, if the content of the selected segment indicates a first name and last name of a person, additional information about the person can be retrieved, e.g., credentials or accomplishments, to integrate and generate a relevant summary.

[0057] In some configurations, the technology disclosed herein can access permissions regarding various aspects of the summary and control the content of the summary based on those permissions. For example, the system 100 can determine whether permissions regarding any retrieved content of a file or original document are restricted, such as a portion of the file or entries of a channel being encrypted. If it is determined that permissions regarding the file or any retrieved content are restricted, the system can limit the amount of disclosure of the summary based on the file or retrieved content. Figure 9 One example of such a summary is shown. Instead of listing the name of the vendor (e.g. Figure 4 the example shown in FIG. 1 1, Figure 9 The summary of the example of FIG. 1 1 provides a summary of the file without providing details regarding any parties, names, or identities.

[0058] The detected permissions can also alter the content of the summary on a per-user basis. For example, if a first user has full access permissions to a file and a second user has partial access permissions to the same file, the summary displayed to the first user can include a full set of complete sentences generated for the summary. On the other hand, if the user's permissions are restricted in any way, the system can edit the summary displayed to the second user and only display a subset of the sentences or a subset of the content.

[0059] Figure 10 is a diagram illustrating aspects of a routine 1000 for computing efficient generation and management of summaries. It should be understood by those of ordinary skill in the art that the operations of the methods disclosed herein are not necessarily presented in any particular order and that performance of some or all of the operations in an alternative order is possible and is contemplated. The operations have been presented in the demonstrated order for ease of description and illustration. Additions, omissions, combinations, and / or

[0060] It should also be understood that the illustrated methods can end at any time and need not wait for other events to occur. The methods may, optionally, be performed alone, in sequence, or in combination with other operations relating to example aspects of the subject matter described herein. The operations of the methods are not necessarily performed in the order indicated and are not necessarily performed by the same entity. For example, one or more operations can be performed in parallel, in an alternative order, or omitted entirely.

[0061] Thus, it should be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system such as those described herein and / or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations can be implemented in software, in firmware, in special purpose digital logic, and any combination thereof.

[0062] Additionally, the operations illustrated in the above examples can be implemented in association with presenting a UI Figure 10 and other figures. For example, the various devices and / or modules described herein can generate, transmit, receive, and / or display data associated with content of a communication session (e.g., live content, a broadcast event, recorded content, etc.) and / or present a UI that includes a rendering of one or more participants of the communication session, a remote computing device, an avatar, a channel, a chat session, a video stream, an image, a virtual object, and / or an application associated with the communication session.

[0063] The routine 1000 begins at operation 1002, where the system 100 can display a graphical element representing a history of a thread, such as a snippet of a document. The graphical element can include a plurality of different portions, each portion representing a snippet of the document. The examples provided herein are for illustrative purposes and should not be construed as limiting. It can be appreciated that any type of graphical element can be used to implement selection of a particular snippet of a document. It can also be appreciated that the operation can involve a computer generated communication of sound or speech summarizing the history of a plurality of snippets of a thread or document.

[0064] Next, at operation 1004, the system 100 can receive an input indicating a selection of a snippet. The input can be based on user input using a pointing device or touch screen. In addition, the input can involve a voice command indicating a start time and an end time.

[0065] Next, at operation 1006, the system 100 can analyze the selected snippet. As described herein, the content of the selected snippet can be interpreted using one or more techniques for purposes of generating an abstract. Priorities can be based on a word count of certain keywords, and based on the priorities, the keywords can be considered as topics. Once a keyword is considered a topic, other keywords in the same sentence can be selected to generate a sentence describing the topic.

[0066] Next, at operation 1008, the system 100 can access a file associated with the selected snippet. For example, if the selected snippet includes a link to a file, the system 100 can analyze the file for purposes of generating summary content. If the content has a threshold level of relevance to the selected snippet or the selected topic, the system 100 can summarize the content of the file by using computer-generated sentences or by extracting sentences from the file itself. If the file is a video file, images can be presented to enable the system 100 to interpret text that can be displayed within the file. If the file includes an audio component, one or more techniques can be used to transcribe any speech within the audio component.

[0067] Next, at operation 1010, the system can display a summary of the selected snippet and / or the content of the file. In some configurations, the system can utilize direct quotes from the selected snippet and the file. Alternatively, a combination of computer-generated sentences and direct quotes from the selected snippet and the file can be used.

[0068] Next, at operation 1012, the system receives input indicating a selection of a portion of the summary. In some embodiments, the selection in operation 1012 can involve a first level of interaction, such as a hover or a click on a touch device. The first level of interaction can be used to display a graphical element, such as a display of the user identity or the source of the selected content of the summary. A second level of interaction, such as a double-click on a touch device or an actual input from a pointing device (such as a mouse), can be used for other types of actions (such as causing a user interface of the document to scroll to a relevant portion, etc.), which are provided for illustrative purposes and should not be construed as limiting. It can be appreciated that any level of interaction can be used to invoke the different operations disclosed herein.

[0069] Next, at operation 1014, the system 100 can display a graphical element showing the source of the selected portion of the summary. As described herein, a user input can select a portion of the summary and, in response to the input, the system can display the name of the individual who contributed to the portion of the summary. Operation 1014 can also involve different types of actions, such as, but not limited to, causing a user interface displaying the document to scroll to a particular location of the document.

[0070] It should be appreciated that the above-described subject matter can be implemented as a computer-controlled apparatus, a computer process, a computing system, or as an article of manufacture such as a computer-readable storage medium. The operations of the example methods are illustrated in individual blocks and summarized with reference to those blocks. The methods are illustrated as logical flow of blocks, each block of which can represent one or more operations that can be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, enable the one or more processors to perform the recited operations.

[0071] Generally, computer-executable instructions include routines, programs, objects, modules, components, data structures, and the like that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be executed in any order, combined in any order, subdivided into multiple operations, and / or executed in parallel to implement the described processes. The described processes can be performed by resources associated with one or more devices, such as one or more internal or external CPUs or GPUs and / or one or more pieces of hardware logic, such as field-programmable gate arrays (“FPGAs”), digital signal processors (“DSPs”), or other types of accelerators.

[0072] All of the methods and processes described above can be embodied in, and fully automated via, software code modules executed by one or more general purpose computers or processors. The code modules can be stored in any type of computer-readable storage medium or other computer storage device, such as those described below. Alternatively, some or all of the methods can be embodied in specialized computer hardware, as described below.

[0073] Any conventional descriptions, elements or blocks in flow charts described herein and / or depicted in the attached drawings should be understood as potentially representing modules, segments or portions of code which include one or more executable instructions for implementing specific logical functions or elements in the routines. Alternative implementations are within the scope of the examples described herein in which elements or functions can be deleted, or executed out of order from that shown or discussed, including substantially synchronously or in reverse order, depending on the functionality involved, as would be understood by those skilled in the art.

[0074] Figure 11 FIG. 1 100 is a diagram illustrating an example environment 1 100 in which systems 1 102 can implement the techniques disclosed herein. In some implementations, the systems 1 102 can be used to collect, analyze, and share data defining one or more objects displayed to users of a communication session 1 104.

[0075] As illustrated, a communication session 1104 can be implemented between a plurality of client computing devices 1106(1) through 1106(N) associated with or part of the system 1102 (where N is a number two or greater). The client computing devices 1106(1) through 1106(N) enable users (also referred to as individuals) to participate in the communication session 1104.

[0076] In this example, the communication session 1104 is hosted by the system 1102 on one or more networks 1108. That is, the system 1102 can provide a service that enables users of the client computing devices 1106(1) through 1106(N) to participate in the communication session 1104 (e.g., via live viewing and / or recorded viewing). Thus, the “participants” of the communication session 1104 can include users and / or client computing devices (e.g., multiple users can be in a room participating in the communication session via use of a single client computing device), each of which can communicate with the other participants. Alternatively, the communication session 1104 can be hosted by one of the client computing devices 1106(1) through 1106(N) utilizing peer-to-peer technology. The system 1102 can also host chat conversations and other team collaboration functionality (e.g., as part of an application suite).

[0077] In some implementations, such chat conversations and other team collaboration functionality are considered to be external communication sessions that are distinct from the communication session 1104. The computerized agent for collecting participant data in the communication session 1104 can be able to link to such external communication sessions. Thus, the computerized agent can receive information such as date, time, session-specific information, and the like that enables connection to such external communication sessions. In one example, a chat conversation can be conducted in accordance with the communication session 1104. Additionally, the system 1102 can host the communication session 1104 that includes at least multiple participants co-located in a meeting location (e.g., a conference room or auditorium) or located at different locations.

[0078] In examples described herein, the client computing devices 1106(1) through 1106(N) participating in the communication session 1104 are configured to receive and render communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live and / or recorded content. The collection of various instances or streams of live and / or recorded content can be provided by one or more cameras (e.g., video cameras). For example, a single stream of live or recorded content can include media data (e.g., audio and visual data capturing the appearance and speech of users participating in the communication session) associated with a video feed provided by a video camera. In some embodiments, the video feed can include such audio and visual data, one or more still images, and / or one or more avatars. The one or more still images can also include the one or more avatars.

[0079] Another example of various streams of live or recorded content can include media data including an avatar of a user participating in the communication session and audio data capturing the speech of the user. Another example of various streams of live or recorded content can include media data including a file displayed on a display screen along with audio data capturing the speech of the user. Thus, the various streams of live or recorded content in the communication data enable facilitating remote conferencing among a group of people and sharing content within the group of people. In some implementations, the various streams of live and / or recorded content within the communication data can originate from a plurality of co-located video cameras positioned in a space (e.g., a room) for recording or live streaming a presentation including one or more individual presentations and one or more individuals using the presented content.

[0080] Participants or attendees can watch the content of the communication session 1104 live as the activity occurs, or at a later time after the activity occurs by way of a recording. In the examples described herein, the client computing devices 1106(1) through 1106(N) participating in the communication session 1104 are configured to receive and render the communication data for display on a user interface of a display screen. The communication data can include a collection of various instances or streams of live and / or recorded content. For example, an individual stream of content can include media data associated with a video feed (e.g., audio and visual data capturing the appearance and speech of a user participating in the communication session). Another example of an individual stream of content can include media data including an avatar of a user participating in the meeting session along with audio data capturing the speech of the user. Yet another example of an individual stream of content can include media data including a content item displayed on a display screen and / or audio data capturing the speech of a user. Thus, the various streams of content within the communication data enable a conference or broadcast presentation to be facilitated among a group of people dispersed at remote locations. Each stream can also include text, audio, and video data, such as data communicated in a channel, chat board, or private messaging service.

[0081] A participant or attendee of a communication session is a person within range of a camera or other image and / or audio capturing device such that the person's actions and / or sounds produced when the person is watching and / or listening to content shared through the communication session can be captured (e.g., recorded). For example, a participant can be sitting in a crowd watching a live broadcast of shared content at a broadcast location where a stage presentation is occurring. Or, a participant can be sitting in an office conference room watching shared content of a communication session with other colleagues through a display screen. Even further, a participant can be sitting or standing in front of a personal device (e.g., tablet, smartphone, computer, etc.) watching shared content of a communication session alone in their office or at home.

[0082] The system 1102 includes a device 1110. The device 1110 and / or other components of the system 1102 can include distributed computing resources that communicate with each other and / or the client computing devices 1106(1) through 1106(N) via one or more networks 1108. In some examples, the system 1102 can be a standalone system responsible for managing the tasks of various aspects of one or more communication sessions (e.g., the communication session 1104). As an example, the system 1102 can be managed by an entity such as SLACK, WEBEX, GOTOMEETING, GOOGLE HANGOUTS, etc.

[0083] The network 1108 can include, for example, a public network such as the Internet, a private network such as an institutional and / or personal intranet, or some combination of private and public networks. The network 1108 can also include any type of wired and / or wireless network, including, but not limited to, a local area network (“LAN”), a wide area network (“WAN”), a satellite network, a cable network, a Wi-Fi network, a WiMax network, a mobile communications network (e.g., a 3G, 4G, etc. network), or any combination of these or other types of networks. The network 1108 can utilize communication protocols including packet-based and / or datagram-based protocols, such as Internet Protocol (“IP”), Transmission Control Protocol (“TCP”), User Datagram Protocol (“UDP”), or other types of protocols. Moreover, the network 1108 can also include a number of devices that facilitate network communication and / or form a hardware basis for the networks, such as switches, routers, gateways, access points, firewalls, base stations, repeaters, backbone devices, and the like.

[0084] In some examples, the network 1108 can also include devices that enable connection to a wireless network, such as a wireless access point (“WAP”). Example support connection through WAPs that send and receive data over various electromagnetic frequencies (e.g., radio frequencies), including WAPs that support Institute of Electrical and Electronics Engineers (“IEEE”) 802.11 standards (e.g., 802.11g, 802.11h, 802.11ac, etc.) and other standards.

[0085] In various examples, the devices 1110 can include one or more computing devices operating in a cluster or other grouped configuration to share resources, balance load, improve performance, provide failover support or redundancy, or for other purposes. For example, the devices 1110 can be of various types of devices, such as traditional server-type devices, desktop-type computers, and / or mobile-type devices. Thus, although illustrated as a single type of device or server-type devices, the devices 1110 can include a variety of device types and are not limited to a particular type of device. The devices 1110 can represent, without limitation, server computers, desktop computers, network server computers, personal computers, mobile computers, laptop computers, tablet computers, or any other kind of computing devices.

[0086] A client computing device (e.g., one of the client computing devices 1106(1) through 1106(N)) can belong to various categories of devices, which can be the same as or different from the device 1110, such as traditional client-type devices, desktop-type computing devices, mobile-type devices, special-purpose-type devices, embedded-type devices, and / or wearable-type devices. Thus, a client computing device can include, but is not limited to, a desktop computer, a game console and / or a gaming device, a tablet computer, a personal data assistant (“PDA”), a mobile phone / tablet hybrid device, a laptop computer, a telecommunications device, a computer navigation-type client computing device (e.g., a satellite-based navigation system including a global positioning system (“GPS”) device), a wearable device, a virtual reality (“VR”) device, an augmented reality (“AR”) device, an implantable computing device, an automobile computer, a television with network capabilities, a thin client, a terminal, an Internet of Things (“IoT”) device, a workstation, a media player, a personal video recorder (“PVR”), a set-top box, a video camera, an integrated component (e.g., a peripheral device) for inclusion in a computing device, an appliance device, or any other category of computing device. Moreover, a client computing device can include a combination of the earlier-listed examples of client computing devices, such as a combination of a desktop-type computing device or a mobile-type device with a wearable device, and so on.

[0087] The various categories and device types of client computing devices 1106(1) through 1106(N) can represent any type of computing device having one or more data processing units 1192 operatively connected to a computer-readable medium 1194 (e.g., via a bus 1116), which in some instances can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any of various local, peripheral, and / or independent buses.

[0088] The executable instructions stored on the computer-readable medium 1194 can include, for example, an operating system 1119, a client module 1120, a profile module 1122, and other modules, programs, or applications that can be loaded and executed by the data processing units 1192.

[0089] Client computing devices 1106(1) to 1106(N) may also include one or more interfaces 1124 to enable communication between client computing devices 1106(1) to 1106(N) and other networked devices (e.g., device 1110) on network 1108. Such network interfaces 1124 may include one or more network interface controllers (NICs) or other types of transceiver devices to send and receive communications and / or data over the network. In addition, client computing devices 1106(1) to 1106(N) may include input / output (“I / O”) interfaces (devices) 1126 capable of communicating with input / output devices, such as user input devices including peripheral input devices (e.g., game controllers, keyboards, mice, pens, voice input devices such as microphones, cameras for obtaining and providing video subscription sources and / or still images, touch input devices, gesture input devices, etc.) and / or output devices including peripheral output devices (e.g., displays, printers, audio speakers, haptic output devices, etc.). Figure 11 A client computing device 1106(1) is shown connected in some way to a display device (e.g., display screen 1129(1)) which can display a UI according to the techniques described herein.

[0090] exist Figure 11 In the example environment 1100, client computing devices 1106(1) to 1106(N) can use their respective client modules 1120 to connect to each other and / or other external devices to participate in communication sessions 1104 or to contribute activities to a collaborative environment. For example, a first user can use client computing device 1106(1) to communicate with a second user on another client computing device 1106(2). When client module 1120 is executed, the user can share data, which can cause client computing device 1106(1) to connect to system 1102 and / or other client computing devices 1106(2) to 1106(N) via network 1108.

[0091] Client computing devices 1106(1) to 1106(N) (each of which is also referred to herein as a “data processing system”) can use their respective profile modules 1122 to generate participant profiles. Figure 11 (Not shown in the diagram), and provides participant profiles to other client computing devices and / or systems 1102 via device 1110. Participant profiles may include one or more of the following: the identity of the user or group of users (e.g., name, unique identifier (“ID”), etc.), user data (e.g., personal data), machine data (e.g., location) (e.g., IP address, room in a building, etc.), and technical capabilities, etc. Participant profiles can be used to register participants in a communication session.

[0092] like Figure 11 As shown, device 1110 of system 1102 includes server module 1130 and output module 1132. In this example, server module 1130 is configured to receive media streams 1134(1) to 1134(N) from various client computing devices (e.g., client computing devices 1106(1) to 1106(N)). As described above, media streams may include video subscription sources (e.g., audio and visual data associated with a user), audio data that will be output along with a presentation of the user's avatar (e.g., a pure audio experience without sending user video data), text data (e.g., text messages), file data and / or screen sharing data (e.g., documents, slides, images, videos, etc. displayed on a display screen), and so on. Therefore, server module 1130 is configured to receive a set of various media streams 1134(1) to 1134(N) (this set is referred to herein as "media stream 1134") during a live viewing communication session 1104. In some cases, not all client computing devices participating in communication session 1104 provide media streams. For example, the client computing device may be a consumer device or a “listening” device, such that it only receives content associated with communication session 1104 without providing any content to communication session 1104.

[0093] In various examples, server module 1130 can select aspects of media stream 1134 to be shared with each of the participating client computing devices 1106(1) to 1106(N). Therefore, server module 1130 can be configured to generate session data 1136 based on media stream 1134 and / or transmit session data 1136 to output module 1132. Output module 1132 can then transmit communication data 1139 to the client computing devices (e.g., client computing devices 1106(1) to 1106(3) participating in a live-streamed communication session). Communication data 1139 may include video, audio, and / or other content data, provided by output module 1132 based on content 1150 associated with output module 1132 and based on the received session data 1136.

[0094] As shown, output module 1132 sends communication data 1139(1) to client computing device 1106(1), and sends communication data 1139(2) to client computing device 1106(2), and sends communication data 1139(3) to client computing device 1106(3), and so on. The communication data 1139 sent to the client computing device may be the same or may be different (for example, the location of the content stream within the user interface may vary between devices).

[0095] In various implementations, the device 1110 and / or the client module 1120 can include a GUI presentation module 1140. The GUI presentation module 1140 can be configured to analyze the communication data 1139 for delivery to one or more client computing devices 1106. In particular, the GUI presentation module 1140 at the device 1110 and / or the client computing devices 1106 can analyze the communication data 1139 to determine an appropriate manner for displaying video, images, and / or content on the display screen 1129 of the associated client computing device 1106. In some implementations, the GUI presentation module 1140 can provide the video, images, and / or content to a presentation GUI 1146 rendered on the display screen 1129 of the associated client computing device 1106. The GUI presentation module 1140 can cause the presentation GUI 1146 to be rendered on the display screen 1129. The presentation GUI 1146 can include the video, images, and / or content analyzed by the GUI presentation module 1140.

[0096] In some implementations, the presentation GUI 1146 can include multiple sections or grids that can render or include video, images, and / or content for display on the display screen 1129. For example, a first section of the presentation GUI 1146 can include a video feed of a presenter or individual, and a second section of the presentation GUI 1146 can include a video feed of an individual consuming conference information provided by the presenter or individual. The GUI presentation module 1140 can populate the first and second sections of the presentation GUI 1146 in a manner that appropriately mimics the environmental experience that the presenter and the individual can share.

[0097] In some implementations, the GUI presentation module 1140 can zoom in or provide a zoomed view of an individual represented by a video feed in order to highlight the individual’s reaction to the presenter, such as facial features. In some implementations, the presentation GUI 1146 can include video feeds of multiple participants associated with a conference (e.g., a general communication session). In other implementations, the presentation GUI 1146 can be associated with a channel, such as a chat channel, an enterprise team channel, and the like. Thus, the presentation GUI 1146 can be associated with an external communication session that is different from a general communication session.

[0098] Figure 12A diagram illustrating example components of an example device 1200 (also referred to herein as a “computing device”) configured to generate data for some of the user interfaces disclosed herein is described. The device 1200 can generate data that can include one or more portions that can render or include video, images, virtual objects, and / or content for display on a display screen 1129. The device 1200 can represent one of the devices described herein. Additionally or alternatively, the device 1200 can represent one of the client computing devices 1106.

[0099] As described, the device 1200 includes one or more data processing units 1202, computer-readable media 1204, and a communication interface 1206. The components of the device 1200 are operatively connected, e.g., via a bus 1209, which can include one or more of a system bus, a data bus, an address bus, a PCI bus, a Mini-PCI bus, and any kind of local, peripheral, and / or independent bus.

[0100] As utilized herein, a data processing unit (e.g., data processing unit 1202 and / or data processing unit 1192) can represent, e.g., a CPU-type data processing unit, a GPU-type data processing unit, a field-programmable gate array (“FPGA”), another type of DSP, or in some cases, other hardware logic components that can be driven by a CPU. By way of example and not limitation, illustrative types of hardware logic components that can be utilized include application-specific integrated circuits (“ASICs”), application-specific standard products (“ASSPs”), system-on-a-chip (“SOCs”), complex programmable logic devices (“CPLDs”), etc.

[0101] As utilized herein, a computer-readable medium (e.g., computer-readable medium 1204 and computer-readable medium 1194) can store instructions that are executable by a data processing unit. The computer-readable medium can also store instructions that are executable by an external data processing unit (e.g., an external CPU, an external GPU) and / or by an external accelerator (e.g., a FPGA-type accelerator, a DSP-type accelerator, or any other internal or external accelerator). In various examples, at least one CPU, GPU, and / or accelerator is incorporated in the computing device, while in some examples one or more of them is external to the computing device.

[0102] Computer-readable media (which can also be referred to herein as computer- readable media) can include computer-storage media and / or communication media. Computer-storage media can include one or more of volatile memory, nonvolatile memory, and / or other persistent and / or auxiliary computer storage media, removable and non-removable computer storage media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Thus, computer-storage media includes tangible and / or physical forms of media included in a device and / or hardware component that is part of the device or external to the device, including but not limited to random access memory (“RAM”), static random access memory (“SRAM”), dynamic random access memory (“DRAM”), phase change memory (“PCM”), read only memory (“ROM”), erasable programmable read only memory (“EPROM”), electrically erasable programmable read only memory (“EEPROM”), flash memory, compact disc read only memory (“CD-ROM”), digital versatile disc (“DVD”), optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage, magnetic cards or other magnetic storage devices or media, solid-state memory devices, storage arrays, network attached storage, storage area networks, hosted computer storage, or any other storage medium, storage device, and / or storage media that can be used for storage and / or access by a computing device.

[0103] In contrast to computer-storage media, communication media can embody computer- readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism. As defined herein, computer-storage media does not include communication media. That is, computer-storage media does not include communications media that solely functions to

[0104] The communication interface 1206 can represent, for example, a network interface controller (“NIC”) or other type of transceiver device to transmit and receive communications over a network. Additionally, the communication interface 1206 can include one or more video cameras and / or audio devices 1222 to enable generation of video feeds and / or still images, among other things.

[0105] In the illustrated example, computer-readable media 1204 includes a data storage unit 1208. In some examples, data storage unit 1208 includes a data storage unit such as a database, a data warehouse, or other type of structured or unstructured data storage unit. In some examples, data storage unit 1208 includes a corpus and / or a relational database having one or more tables, indexes, stored procedures, and / or the like to enable data access including one or more of HyperText Markup Language ("HTML") tables, Resource Description Framework ("RDF") tables, Web Ontology Language ("OWL") tables, and / or Extensible Markup Language ("XML") tables.

[0106] Data storage unit 1208 can store data for operations of processes, applications, components, and / or modules stored in computer-readable media 1204 and / or executed by data processing unit 1202 and / or accelerators. For example, in some examples, data storage unit 1208 can store session data 1210 (e.g., session data 1136), profile data 1212 (e.g., associated with a participant profile), and / or other data. Session data 1210 can include a total number of participants (e.g., users and / or client computing devices) in a communication session, activities that occurred in the communication session, an invitee list of the communication session, and / or other data related to when and how the communication session was conducted or hosted. Data storage unit 1208 can also include content data 1214, such as content including video, audio, or other content for rendering and display on one or more display screens 1129.

[0107] Optionally, some or all of the above-referenced data can be stored on separate memory 1216 of the on-board one or more data processing units 1202, such as memory of on-board CPU-type processors, GPU-type processors, FPGA-type accelerators, DSP-type accelerators, and / or other accelerators. In this example, computer-readable media 1204 also includes an operating system 1218 and an application programming interface 1211 (API) configured to expose functionality and data of device 1200 to other devices. Additionally, computer-readable media 1204 includes one or more modules, such as a server module 1230, an output module 1232, and a GUI presentation module 1240, although the number of modules shown is merely an example, which number can become higher or lower. That is, the functionality described herein as associated with the illustrated modules can be performed by a fewer number of modules or a greater number of modules on one device or distributed across multiple devices.

[0108] Unless otherwise specifically stated, it is to be understood that the conditional language used herein, such as, among others, "can," "could," "might," or "may," and other similar expressions, refers to a possibility that a certain example includes, but does not require, the

[0109] It is also to be understood that the above description is one possible example, and that many variations and modifications can be made to the described example, and that the elements, functions, and / or steps can be understood in other acceptable examples. All such modifications and variations are intended to be included herein within the scope of the present disclosure, and protected by the following claims.

[0110] Finally, while a particular order of actions is described in connection with various configurations, it will be appreciated that the subject matter defined herein is not necessarily limited to the particular order presented. Rather, the various actions described in connection with the subject matter defined herein can be performed in any order, and the subject matter defined herein can be implemented in any order.

Claims

1. A method performed by a data processing system, the method comprising: displaying, at the data processing system, a user interface comprising a main display area for displaying a message thread and a graphical element having a plurality of segments representing the message thread, wherein the user interface is configured to receive input defining a range of messages in the thread based on a number of selected segments representing individual messages in the thread; receiving input from the graphical element indicating a selection of a subset of the segments, the input defining a range of messages in the thread to generate a summary from messages within the range; analyzing messages associated with the subset of the segments to generate a plurality of sentences describing content of at least a portion of the messages associated with the subset of the segments, the subset of the segments being selected according to the input defining the range based on the number of selected segments representing individual messages; and displaying a summary comprising the plurality of sentences describing the content of the at least a portion of the messages associated with the subset of the segments, the subset of the segments being selected according to the input defining the range based on the number of selected segments representing individual messages.

2. The method of claim 1, further comprising: receiving an adjustment to the input, the adjustment increasing or decreasing a number of segments in the subset of the segments; adding content to the summary in response to the adjustment to the input, the adjustment increasing the number of segments in the subset of the segments; and removing content from the summary in response to the adjustment to the input, the adjustment decreasing the number of segments in the subset of the segments.

3. The method of claim 1, further comprising: analyzing content of the subset of the segments to identify a task and an associated user; and simultaneously displaying the identified task and the plurality of sentences and the content of at least one file having a threshold relevance to the subset of the segments. generating a graphical element associated with a first portion of the summary, the graphical element indicating a source of the content of the first portion.

4. The method of claim 1, further comprising: generating a graphical element distinguishing computer-generated sentences from sentences extracted from the subset of the segments.

5. The method of claim 1, further comprising:

6. The method of claim 1, further comprising: determining one or more permissions for at least one file having a threshold relevance to the subset of the segments, the permissions being associated with at least one user identity; editing at least a portion of the plurality of sentences based on one or more permissions for a summary to be displayed to a user identity other than the at least one user identity.

7. The method of claim 1, further comprising: determining one or more permissions for at least one file having a threshold relevance to the subset of the segments; and editing at least a portion of the plurality of sentences based on the one or more permissions. ​ ​ 8. The method of claim 1, further comprising: generating a graphical element associated with a portion of the summary, the graphical element indicating that the content of the portion of the summary is a direct quote from at least one item in the subset of the snippets.

9. The method of claim 1, further comprising: receiving input identifying a portion of the summary; and navigating, in response to the input identifying a portion of the summary, a user interface display of the subset of the snippets to the snippet having content used as a source to generate the portion of the summary.

10. The method of claim 1, further comprising: causing display of the summary contemporaneously with an additional summary generated from the plurality of snippets, wherein the summary and the additional summary are each associated with a separate topic; receiving a selection of the summary or the additional summary and selecting a respective topic based on the selection; and communicating the topic to at least one machine learning resource for updating machine learning data to increase a priority of the topic for generating other summaries based on the plurality of snippets.

11. The method of claim 1, further comprising: causing display of the summary contemporaneously with an additional summary generated from the plurality of snippets, wherein the summary and the additional summary are each associated with a separate topic; receiving a selection of the summary or the additional summary and selecting a respective topic based on the selection; and communicating the topic to at least one machine learning resource for updating machine learning data to increase a relevance level of the topic for generating other summaries based on the plurality of snippets.

12. The method of claim 1, further comprising: receiving input indicating a selected topic; generating a set of new sentences using the plurality of sentences describing the content of the subset of the snippets, wherein the new sentences are generated by filtering content that is not relevant to the selected topic; and generating a display of an updated summary using the new sentences.

13. A system for automatically generating summaries of documents, comprising: one or more data processing units; and a computer-readable medium having encoded thereon computer-executable instructions to cause the one or more data processing units to: display a user interface, the user interface including a main display area for displaying a message thread and having graphical elements representing a plurality of snippets of the message thread, wherein the user interface is configured to receive input defining a range of messages in the thread based on a number of selected snippets representing individual messages in the thread; receive input from the graphical elements indicating a selection of a subset of the snippets, the input defining a range of messages in the thread to generate a summary from messages within the range; analyze messages associated with the subset of the snippets to generate a plurality of sentences describing content of at least a portion of the messages associated with the subset of the snippets, the subset of the snippets being selected from the input defining the range based on the number of selected snippets representing individual messages; ​ displaying a summary comprising a plurality of sentences describing content of the at least a portion of the message associated with the subset of the segments, the subset of the segments being selected according to the input defining the range based on the number of the selected segments representing individual messages.

14. The system of claim 13, wherein, in response to determining that a number of occurrences of a keyword in the subset of the segments exceeds a threshold, the content exceeding a priority threshold, wherein the keyword is selected as a subject of at least one of the plurality of sentences.

15. The system of claim 13, wherein, the instructions further cause the one or more data processing units to: receive an adjustment to the input, the adjustment increasing or decreasing the number of the segments in the subset of the segments; in response to the adjustment to the input to add content to the summary, the adjustment increasing the number of the segments in the subset of the segments; and in response to the adjustment to the input to remove content from the summary, the adjustment decreasing the number of the segments in the subset of the segments.

Images