Method and apparatus for determining the non-textual content of a response for inclusion in a reply to an electronic communication

Abstract

A computer-implemented method comprising the following: The identification of an electronic communication sent to a user, performed by one or more computer devices; the determination of a message characteristic of the electronic communication, carried out by one or more of the computer devices; the determination of an additional message feature of the electronic communication, carried out by one or more of the computer devices; Providing at least the additional message feature as input to a trained machine learning system; obtaining a single document feature as output from the trained machine learning system; initiating a search of one or more corpora of electronic documents by one or more of the computer devices using a search parameter based on the message characteristic of electronic communication; in response to the initiation of the search, the receipt by one or more computer devices of a reference to a subset of one or more electronic documents of the one or more corpora that refer to the search; the selection of at least one selected electronic document from the electronic documents of the subgroup, carried out by one or more computer devices and based on the receipt of the information, wherein the selection of the at least one selected electronic document is further based on at least one document attribute; and The provision, by one or more computer devices, of at least one section of a selected electronic document for inclusion in a response to electronic communication, which is a response from the user to electronic communication.

Description

background

[0001] Users are often inundated with electronic communications, such as emails, text messages, and social media. Much of the electronic communication explicitly sent to a user requests a non-text response or otherwise includes information to which the user wishes to respond with a non-text response. For example, an email containing the question, "Can you send me a copy of your presentation slides from yesterday's session?" may explicitly request a response that includes non-text content (i.e., an electronic document representing the presentation slides).An email containing the question "How is the home renovation coming along?" does not explicitly request a non-text response, but a user receiving the email might still want to include non-text content as part of their reply (e.g., a picture showing the current status of the home renovation). To include non-text content as part of a reply, users must recognize that non-text content is suitable for the response, manually search for non-text content on their computer device, and select the appropriate non-text content through the user interface of the computer device's input device so that this content is included in the reply.

[0002] US 6782393 B1 describes how, when a message is composed on a display, an initial set of documents, including the current context of a user relating to the message, is identified. Furthermore, a second set of documents, linked to the first set, is automatically provided. This allows any documents from the first set to be inserted into the composed message.

[0003] US 2012 / 245925 A1 describes a method for analyzing email messages or texts, online posts, online websites, social media sites and online news sites to detect predefined and actionable events and intentions.

[0004] US 2015 / 213372 A1 describes techniques for predicting a user response to email content by accessing email interaction data associated with a particular member and email content data that describes a particular email content element.

[0005] US 2013 / 275429 A1 describes a system for enabling context-related recommendations and recommendations for collaboration based on a user's current work.

[0006] US 2006 / 149710 A1 describes how functions that can be used to represent relevance information, such as properties, characteristics, etc., of an entity such as a document or concept, can be linked to the document by accepting an identifier that identifies a document; using the document identifier, search query information and / or other provisioning parameter information relating to the document can be retrieved. Summary

[0007] The invention is defined by the features specified in the attached claims.

[0008] Some implementations of this specification are generally directed toward methods and devices that involve identifying the non-textual response content for a reply to an electronic communication and providing the non-textual response content for inclusion in the reply (e.g., automatically providing it in the reply or suggesting it for inclusion). Based on an electronic communication transmitted to a user, some of these implementations are directed toward identifying one or more electronic documents that refer to the electronic communication and providing one or more electronic documents for inclusion in a user's reply to the electronic communication.For example, electronic documents can be automatically attached to the response and / or the link(s) to the electronic documents can be automatically provided in the response. Furthermore, for example, one or more user interface (e.g., graphical, audio) cues for the electronic documents can be presented, and if a user generating the response selects one or more options via a user interface input, the corresponding electronic documents can be attached to the response and / or the link(s) to the documents can be automatically provided in the response.In some implementations, the electronic document(s) may be made available for inclusion in the response before the user has provided any textual content for the response and / or before the user has provided any other type of content for the response. In some implementations, the electronic document (or documents) may be made available in response to the user's selection of an interface element regarding the attachment document or in response to any other user interface input indicating a desire to include an attachment in the response.

[0009] The electronic document(s) selected for inclusion in a response to electronic communication to a user may be identified from one or more of the different corpora, such as the one or more cloud-based corpora associated with the user, using the local storage device(s) of a computer device to generate the response, and the computer device and / or the user, etc., being able to access the local network storage device(s).In some implementations, the electronic document (or documents) can be ordered by performing a search of one or more corpora of documents, where the search includes one or more search parameters derived from the electronic communication. For example, if a message contains the question, "Can you send me photos of the trip to Chicago you took last week?", a search can be initiated using an image search parameter derived from an n-gram of the message (e.g., "photos"), a data search parameter derived from an n-gram of the message (e.g., "last week"), and / or a location parameter derived from an n-gram of the message (e.g., "Chicago") to identify the photos that meet the search criteria.In some implementations, the corpus (or corpora) searched to identify documents to be included in a response can be restricted based on one or more features derived from the electronic communication. For example, the corpora searched in the previous example can be restricted to a single corpus containing only images and optionally other media based on an n-gram of the message (e.g., "photos").

[0010] In some implementations, the non-textual response content provided for inclusion in an electronic communication response can be determined based on the output provided by a trained machine learning system in response to the provision of one or more message features of the electronic communication. For example, the trained machine learning system can provide one or more features of the non-textual response content (e.g., document type(s)) that can be used as a search parameter in a search initiated to identify the non-textual response content and / or that can be used to limit one or more corpora of a search initiated to identify the non-textual response content.

[0011] In some implementations, whether and / or how the non-textual response content is provided for inclusion in a response to the electronic communication can be determined based on the output generated by a trained machine learning system in response to one or more message features of the electronic communication provided to the trained machine learning system. For example, the trained machine learning system can provide a probability that a response to the electronic communication will include non-textual response content, and this probability can be used to determine whether and / or how the non-textual response content is provided to a user for inclusion in a response to the electronic communication.

[0012] Some implementations are generally aimed at analyzing a corpus of electronic communications to determine relationships between the one or more "original" message features of the original messages of the electronic communication and the non-textual response content included in the "response messages" of that electronic communication. For example, the corpus can be analyzed to determine relationships between message features of the original messages and the probability that responses to the original message, which has those message features, will include a document or a link to a document. For example, the corpus can also be analyzed to determine relationships between message features of the original message and the document type(s) (e.g.,to identify images, videos, media, PDS, slides) of documents that are part of the responses to the original message, that have such message characteristics, or that are linked to in such messages.

[0013] As one example, the corpus can be analyzed to determine that original messages containing the n-gram "send me" are highly likely to also include a document and / or a link to a document in replies to those original messages. As another example, the corpus can be analyzed to determine that original messages containing the n-gram "presentation" often include files that have ".ppt", ".cvs", or ".pdf" extensions in replies to those original messages.

[0014] These relationships, identified through analysis of the corpus of past electronic communications, can be used, for example, to determine one or more parameters for the initiated search discussed above, to narrow the corpus (corpora) of the initiated search discussed above, and / or to determine whether and / or how electronic documents are provided for inclusion in a response to the message (e.g., a low probability that a response will include an electronic document may result in no documents being provided, or in the document (documents) being 'suggested' in a less conspicuous manner).

[0015] In some implementations, determining relationships between one or more original message features of the original messages of the electronic communication and the non-textual response content contained in the response messages of these electronic communications can be achieved by generating appropriate training examples based on a corpus of electronic communications and training a machine learning system based on these training examples. The machine learning system can be trained to receive one or more message features of an "original message" as input and to provide at least one feature related to the non-textual response content, such as one of the features described above, as output.

[0016] For example, some implementations can generate training examples, each with an output parameter indicating a document type (or types) contained in a reply message to a corresponding electronic communication, and one or more input parameters based on the original message of that electronic communication. For instance, an initial training example might include, as an output parameter, the document type of an "image," and as an input parameter, all subsections of the text of the original message (and optionally, annotations associated with the text) of an initial electronic communication containing an image in a reply message.A second training example can include, as an output parameter, a document type of the "image," and as an input parameter, all or sections of the text of the original message (and optionally annotations associated with the text) of a second electronic communication containing an image in a reply message. Additional training examples can be generated similarly, including additional examples with an output parameter (or parameters) based on a different document type (or document types) of a reply message from a corresponding electronic communication, and with input parameters based on the original message of the corresponding electronic communication. The machine learning system can then be trained based on these training examples.

[0017] In some implementations, a procedure may be provided, performed by at least one computer device, which involves identifying an electronic communication sent to a user, determining a message characteristic of the electronic communication, and initiating a search of one or more corpora of electronic documents using a search parameter based on the message characteristic of the electronic communication. In response to the initiation of the search, the procedure further involves receiving information on a subset of the one or more electronic documents of the one or more corpora that are referenced by the search.Based on the receipt of the notification, the procedure further includes selecting at least one selected electronic document from the electronic documents of the subgroup and providing at least one section of the at least one selected electronic document for inclusion in a response to the electronic communication, which is a response to the electronic communication written by the user.

[0018] This method and other implementations of the technology disclosed herein may optionally include one or more of the following features.

[0019] In some implementations, the search is initiated independently of any text-related input provided by the user's computer device when generating the response to the electronic communication. In some of these implementations, at least one section of the at least one selected electronic document is provided for inclusion in the response to the electronic communication, independently of any text-related input provided by the computer device when generating the response to the electronic communication.

[0020] In some implementations, at least one corpus of the one or more corpora that are not publicly accessible is accessible to the user. In other implementations, at least one or more corpora are accessible only to the user and one or more additional users or systems authorized by the user.

[0021] In some implementations, initiating a search of one or more corpora involves initiating a search of one or more indexes that index the electronic documents of one or more corpora.

[0022] In some implementations, the subset comprises a large number of electronic documents, and in response to initiating the search, the procedure further includes obtaining a search ranking for the subset of electronic documents from the one or more corpora referenced by the search. The selection of the at least one selected electronic document from the subset can also be based on the search rankings for the subset of electronic documents.In some implementations, the at least one selected electronic document includes a first document and a second document, and providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication involves: determining a prominence for providing the first document in the second document based on search rankings and providing both the first and second documents for inclusion in the response to the electronic communication along with a specification of their respective prominences.

[0023] In some implementations, the procedure also includes identifying an additional message feature of the electronic communication and restricting one or more corpora of the search based on the additional message feature of the electronic communication.

[0024] In some implementations, providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication involves attaching the at least one section of the at least one selected electronic document with respect to the response, without requiring the user to confirm this via a user-initiated user interface input.

[0025] In some implementations, providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication involves: providing a graphical representation of the at least one section of the at least one selected electronic document; receiving a selection of the graphical representation via a user interface input device; and, in response to receiving the selection, attaching at least one selected electronic document to the response.

[0026] In some implementations, providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication involves providing a link in the response, the link being to at least one section of the at least one selected electronic document.

[0027] In some implementations, the at least one selected electronic document comprises a first document and a second document, and providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication includes: providing a first graphic representation of the first document and a second graphic representation of the second document; receiving a selection of the first graphic representation and the second graphic representation via a user interface input device; and, in response to receiving the selection, adding a corresponding selection of the first document and the second document to the response.

[0028] In some implementations, the procedure further includes: determining an additional message attribute of the electronic communication; providing the at least one additional message attribute as input to a trained machine learning system; receiving the one document attribute as output from the trained machine learning system; and using an additional parameter for the search based on at least one document attribute. In some of these implementations, the at least one document attribute includes a document type attribute that specifies a closed class of one or more filename suffixes.

[0029] In some implementations, the procedure further includes: determining an additional message feature of the electronic communication; providing the at least one additional message feature as input to a trained machine learning system; and receiving the at least one document feature as output from the trained machine learning system. In some of these implementations, the selection of the at least one selected electronic document is also based on at least one document feature.

[0030] In some implementations, the procedure further includes: determining an additional message feature of the electronic communication; providing the at least one additional message feature as input to a trained machine learning system; receiving the at least one document feature as output from the trained machine learning system; and restricting the one or more corpora of the search based on the at least one document feature.

[0031] In some implementations, the message feature is embedded in the vector of one or more electronic communication features.

[0032] In some implementations, the message feature is based on an N-gram in a main part of the electronic communication, and the determination of the message feature based on the N-gram is based on the proximity of the N-gram to the requesting verb N-gram in the main part of the electronic communication.

[0033] In some implementations, the procedure further involves marking each set of N-grams of electronic communication with at least one corresponding grammatical marker. In some of these implementations, determining the message feature involves selecting one N-gram from the N-grams based on the corresponding grammatical marker of that N-gram and then determining the message feature based on that N-gram.

[0034] Other implementations may include a non-volatile, computer-readable storage medium that stores instructions executable by a processor to perform a procedure, such as one or more of the procedures described above. A still further implementation may include a system, including memory, and one or more processors that can be operated to execute instructions stored in the memory to perform a procedure, such as one or more of the procedures described above.

[0035] It is understood that all combinations of the foregoing concepts and the additional concepts described herein are provided for as part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are provided for as part of the subject matter disclosed herein. Brief description of the drawings Fig. Figure 1 illustrates an exemplary environment in which the non-textual response content intended as part of a reply to electronic communication is determined based on one or more message features of the electronic publication. Fig. Figure 2 illustrates an example of how a non-textual response content, which is intended to be part of a response to electronic communication, can be determined based on one or more message features of the electronic communication. Fig. Figure 3 is a flowchart illustrating an exemplary procedure for determining non-textual response content intended as part of a response to electronic communication based on one or more message features of the electronic communication. Fig. 4A-4E illustrate exemplary graphical user interfaces for providing non-textual response content for inclusion in a response to electronic communication. Fig. Figure 5 illustrates an exemplary environment in which electronic communications can be analyzed to generate training examples for training a machine learning system, to identify one or more non-textual response contents, and to determine in which of these the machine learning system should be trained based on the training examples. Fig. Figure 6 illustrates an example of how training examples based on electronic communications can be generated and used to train a machine learning system and to identify one or more non-textual response content features. Fig. Figure 7 is a flowchart illustrating an exemplary procedure for generating training examples and using the training examples to train a machine learning system to identify one or more non-textual response content features. Fig. Figure 8 illustrates an exemplary architecture of a computer device. Detailed description

[0036] Fig. Figure 1 illustrates an exemplary environment in which the non-textual response content, intended as part of a reply to electronic communication, is determined based on one or more message features of the electronic publication. The exemplary environment includes a communication network 101 that enables communication between the various components in the environment. In some implementations, the communication network 101 may include the Internet, one or more intranets, and / or one or more bus subsystems. The communication network 101 may optionally use one or more standard communication technologies, protocols, and / or interprocess communication techniques.The exemplary environment also includes a client device 106, an electronic communications system 110, a non-textual response content system 120, at least one trained machine learning system 135, electronic document corpora 154A-N, and at least one electronic communications database 152.

[0037] The electronic communication system 110, the non-textual response content system 120, and the trained machine learning system 135 can each be implemented in one or more computer devices that communicate, for example, via a network (e.g., network 101 and / or other networks). The electronic communication system 110, the non-textual response content system 120, and the trained machine learning system 135 are exemplary systems in which the systems, components, and techniques described herein can be implemented and / or with which the systems, components, and techniques described herein can be connected via an interface.The electronic communication system 110, the non-textual response content system 120, and the trained machine learning system 135 each include one or more memories for storing data and software applications, one or more processors for accessing data and executing applications, and other components that enable communication over a network. In some implementations, the electronic communication system 110, the non-textual response content system 120, and the trained machine learning system 135 may include one or more components of the exemplary computer device. Fig. 8 include. The operations performed by the electronic communication system 110, the non-textual response content system 120, and the trained machine learning system 135 can be distributed across multiple computer systems. In some implementations, one or more aspects of the electronic communication system 110, the non-textual response content system 120, and / or the trained machine learning system 135 can be combined into a single system, and / or one or more aspects can be implemented by the client device 106.

[0038] In general, in some implementations, the non-textual response content system 120 identifies and provides the non-textual response content to be included as part of a response to the electronic communication, such as an electronic communication provided by the client device 106 and / or an electronic communication from the electronic communications database 152 to which a user still needs to respond. The non-textual response content system 120 can identify the non-textual response content based on one or more message features of the electronic communication.In some implementations, the non-textual response content system 120 can provide the determined non-textual response content for inclusion in a response to a communication that is provided by the user when generating the response to the communication independently of any textual input and / or independent of any other content provided by the user when generating the response to the communication.

[0039] In some implementations, the non-textual response content retrieved and provided by the Non-Textual Response Content System 120 includes all or portions of the one or more electronic documents relating to the electronic communication, such as the one or more electronic images, videos, word processing documents, spreadsheet documents, presentation slides, structured data containing the user's travel itineraries, other electronic communications, etc. The electronic document(s) retrieved and provided for an electronic communication is a document that exists not only separately from and in addition to the electronic communication itself, but also separately from and in addition to the response to the electronic communication.The electronic documents can be selected from one or more corpora of electronic documents 154 AN, which are provided on one or more storage media. In some implementations, the one or more corpora of electronic documents 154 AN from which an electronic document is selected to be included in a user's response may include or be limited to one or more corpora that, although not publicly accessible, are accessible to the user. For example, one or more of the corpora of electronic documents 154A-N may be accessible only to the user and one or more systems and / or other users authorized by the user. For example, the one or more corpora of electronic documents 154A-N may be cloud-based corpora to which the user accesses the local storage devices of a computer device (e.g., a hard drive, a hard drive, etc.).Client device 106) of the user, which is used to generate the response, and local network storage devices to which the computer device and / or the user can access, etc. As described herein, the non-textual response content system 120 can select the electronic document(s) for inclusion in a response in some implementations from a set of electronic documents received in response to initiating a search of one or more corpora of electronic documents 154A-N, the search including one or more search parameters derived from an electronic communication to which the response refers.

[0040] The electronic communications database 152 comprises one or more storage media containing all or portions of the electronic communications of a large number of users. In some implementations, the electronic communications database 152 is maintained by the electronic communications system 110. For example, the electronic communications system 110 may include one or more email systems, and the electronic communications database 152 may contain a large number of emails that have been sent and / or received through the email systems. As another example, the electronic communications system 110 may include one or more social networking systems, and the electronic communications database 152 may contain a large number of messages, posts, or other communications that have been sent and / or received through the social networking systems.

[0041] As used herein, the term “electronic communication” or “communication” refers to an email, a text message (e.g., SMS, MMS), an instant message, a transcribed voicemail message, or any other electronic form of communication sent by an initial user to a restricted group consisting of one or more additional users. In different implementations, electronic communication may include various metadata, and the metadata may optionally be used in one or more of the techniques described herein. For example, a form of electronic communication, such as an email, may include an electronic communication address, such as one or more sender identifiers (e.g., sender email addresses), one or more recipient identifiers (e.g.,Recipient email addresses (including CC and BCC recipients), a sending date, one or more attachments, a subject, a type of device that sent and / or received the form of electronic communication, etc.

[0042] As used herein, the terms "electronic communication" and "communication" are used contextually to refer both to electronic communication that contains only original messages and to electronic communication that contains one or more original messages and, in addition, one or more reply messages. Electronic communication may be a single document, such as an email, containing both an original message and a reply message, and it may be possible to process and distinguish between the original message and the reply message.Processing an electronic communication to distinguish between an original message and a reply message can involve separating the message based on the presence of metadata, message pauses, header information contained within the message, quotes provided around the public message, and so on. An electronic communication can also consist of multiple documents that, based on at least several documents referencing one another, lead to a different communication. For example, an electronic communication might include an initial email, which is the original message received by a user, and a second email, which is sent by the user in reply to that initial message, or an initial SMS and a reply SMS to that email.Merging multiple documents into one can be accomplished, for example, using the electronic communication system 110. Based on a user selecting a response user interface element while viewing the first email and then composing the second email based on that selection, the electronic communication system 110 can, for example, merge a first email into a second email.

[0043] As used herein, a referencing reply message follows an original message, although the original message is not necessarily the first message in an electronic communication. For example, an original message may be the first message within an electronic communication, and a reply message that refers to the original message may occur at a later point (e.g., next). For example, an original message may additionally and / or alternatively be a second, third, or fourth message in an electronic communication, and a reply message that refers to the original message may be a later message in the electronic communication. Both the original message and the reply message may contain one or more related texts, metadata, and / or other content (e.g.,attached documents, links to the documents) are included.

[0044] In different implementations, the non-textual response content system 120 may include a message feature engine 122, a search parameter engine 124, a search engine 126, and / or a presentation engine 128. In some implementations, aspects of one or more of the engines 122, 124, 126, and / or 128 may be omitted. In some implementations, all or aspects of the engines 122, 124, 126, and / or 128 may be combined. In some implementations, all or aspects of the engines 122, 124, 126, and / or 128 may be implemented in a component separate from the non-textual response content system 120, such as the client device 106 and / or the electronic communications system 110.

[0045] Descriptions of machines 122, 124, 126, and 128 are provided below with reference to a user's electronic communication, such as an electronic communication provided by the client device 106 and / or an electronic communication from the electronic communication database 152, to which the user still needs to respond. Although the examples refer to a single electronic communication for the sake of clarity, it is understood that the non-text response content system 120 can determine the response content for additional electronic communications from the user and / or additional users.

[0046] The message feature engine 122 determines one or more original message features based on an electronic communication sent to a user (i.e., at least partially based on the content of the electronic communication while that content was being sent to the user). Various original message features can be used. For example, the message feature engine 122 can determine one or more N-grams in the electronic communication as message features. For example, one or more of the N-grams can be determined based on the N-gram's transit-time frequency in the electronic communication (i.e., how often the N-gram occurs in the electronic communication) and / or inverse document frequencies of the N-gram in a document collection (i.e., how often the N-gram occurs in a document collection, such as a collection of electronic communication forms).Furthermore, one or more n-grams can be identified based on their positional distance from other n-grams, such as a requesting verb n-gram (e.g., "send," "provide," "attach"). In a further step, one or more n-grams can be identified based on a part of a linguistic unit of the n-gram (e.g., "noun") or based on whether the n-gram is included in a list of potentially relevant n-grams and / or not in a list of irrelevant n-grams (e.g., stop words such as "ein," "eine," "einer," "der," "die," and "das").

[0047] Message feature machine 122 can, for example, also determine the joint occurrence of two or more N-grams within the context of electronic communication as a message feature, such as their joint occurrence in a specific sequence (e.g., a first N-gram before a second N-gram) in a specific positional relationship (e.g., within n propagation times or characters of each other), etc. Message feature machine 122 can, for example, also identify one or more message features based on one or more natural language processing tags or other identifiers that refer to the text of the electronic communication (e.g.,linguistic aspects, named entities, entity types, tone) are applied; features based on text, particularly in the subject line, opening sentences, closing sentences, or any other section of the electronic communication; features based on metadata of the electronic communication, such as the time the electronic communication was sent, the day of the week the electronic communication was sent, the number of recipients, the type of device that sent the electronic communication, etc.

[0048] Message Feature Engine 122 can, for example, also identify an embedding vector of one or more features from the entire electronic communication or a subset thereof (e.g., one or more paragraphs, one or more sentences, one or more words). The features of the embedding vector include one or more n-grams, labels applied to the one or more n-grams, syntactic features, semantic features, metadata-related features, and / or other features.

[0049] In a specific example, let's assume that the electronic communication has a subject line with the text "spreadsheet" and a body with the text "Please send me yesterday's sales-related spreadsheet. Thank you." The message feature engine 122 can determine message features that include the n-grams "send", "yesterday", "sales-related", and "spreadsheet", while excluding other n-grams, such as "please" and "thank you".The message feature engine 122 can identify "send" as a message feature, for example, because it is an imperative verb n-gram; it can identify "yesterday's" as a message feature because it is a time-related indicator and / or because it is a short positional distance from "send"; it can identify "sales-related" as an n-gram because it is an adjective in the text and / or because it is a short positional distance from "send"; and it can identify "spreadsheet" as an n-gram because it is a noun, its short positional distance from "send," and / or because it is in a list of potentially relevant terms.

[0050] The message feature engine 122 provides the search parameter engine 124 and, optionally, the trained machine learning system 135 with the determined message features. The message feature engine 122 can provide the same message features to the search parameter engine 124 and the trained machine learning system 135, or the provided message features can differ. The search parameter engine 124 generates one or more search parameters based on the one or more message features provided by the message feature engine 122.

[0051] In an example of generating search parameters based on one or more message attributes provided by the message attribute engine 122, let's assume that the message attribute engine 122 provides a group of message attributes that includes the n-grams "send," "yesterday," "sales-related," and "spreadsheet." The search parameter engine 124 can generate a date search parameter based on "yesterday." This date search parameter can, for example, specify a creation or modification date for a document, and this date can be yesterday's date or one or more specific dates specified based on the term "yesterday." The search parameter engine 124 can also generate additional parameters for the terms "sales-related" and "spreadsheet." The search parameter for "sales-related" can be a general text-related parameter.The search parameter for "spreadsheet" can be a general text-related parameter and / or a document-type-related parameter that influences this type(s) of document(s) in a search, restricts a search to one or more types of documents (e.g., those with a ".pdf", ".xls", ".ods", ".cvs", and / or ".tsv" filename suffix), and / or restricts a search to one or more corpora that contain (and are optionally limited to) documents of the respective document type.

[0052] In another example of generating search parameters based on one or more message features provided by the message feature engine 122, let us assume that the message feature engine 122 provides an embedding vector of one or more features of the entire electronic communication or a subset of the electronic communication. The search parameter engine 124 can generate search parameters based on a decoded version of the embedding vector.

[0053] The search parameter engine 124 can optionally generate one or more parameters based on the output provided to the search parameter engine 124 by at least one trained machine learning system 135. As described herein (see, e.g., Fig. 5-7) The trained machine learning system 135 can, for example, be trained to receive one or more message features as input and to provide one or more features of the non-textual response content as output, such as the one or more document types of the non-textual response content. The search parameter engine 124 can use the one or more features of the output to generate a search parameter.In cases where the trained machine learning system 135, for example, provides the output of the document type (or document types) of the non-textual response content based on an input of the message functions for electronic communication, the document type (document types) can be used as a search parameter that directs this document type (these document types), restricts the search to that document type (document types), and / or restricts the search to the one or more corpora that contain (and is optionally limited to) documents of this type (these types).

[0054] In some implementations, the search parameter engine 124 can determine whether it should generate search parameters for electronic communication based on the output provided to the parameter engine 124 by at least one trained machine learning system 135. As described, for example, herein (see e.g. Fig. 5-7) The at least one trained machine learning system 135 can be trained to receive one or more message features as input and to provide as output a probability according to which a response to the electronic communication will contain non-textual response content. The search parameter machine 124 can use the probability of determining whether search parameters should be generated. For example, the parameter machine 124 can generate search parameters only if the probability matches a certain threshold.

[0055] Search Engine 126 searches for one or more of the corpora of electronic documents 154A-N based on the search parameters determined by Search Parameter Engine 124. In some implementations, searching for one or more corpora of electronic documents 154AN may involve searching for one or more indexes that index the electronic documents of the one or more document corpora. Search Engine 126 may initiate one or more searches based on the search parameters, each combining the search parameters in one or more ways. For example, Search Engine 126 may initiate one or more searches to identify documents that match one or more (e.g., all) search parameters (e.g., a search that combines multiple search parameters with "AND") belonging to a group.For example, the search engine 126 can also initiate one or more searches to identify documents that only need to match some of the search parameters (e.g., a search that combines multiple parameters with "OR"). In some implementations, the search engine 126 can identify a section of a document as a response to a search. For example, the search engine 126 can identify one or more paragraphs of a document consisting of multiple paragraphs, one or more slides of a slide deck, and one or more sentences of a document consisting of multiple sentences, and so on.

[0056] In some implementations, the search engine can identify multiple documents in response to a search based on one or more search parameters derived from electronic communication. In some implementations, the multiple documents can be ranked based on various criteria, such as a query-based ranking (e.g., based on how closely a document matches the query) and / or a document-based ranking. A query-based ranking of a document is based on a relationship between the query and the document, such as how closely the document matches the query. For example, a first document containing every word in a query might have a higher query-based ranking than a second document containing only some of the words in the query.For example, a first document that includes all the words in the query in a heading and / or other prominent position may have a "higher" query-based ranking than a second document that only includes words in the last paragraph of a main body. A document-based ranking is based on one or more properties of the document that are independent of the query. For example, a document's ranking might be based on the document's popularity with the user (e.g., the frequency of access), on a user-specific date, and / or on when the user last accessed the document, or on the document's creation date.

[0057] The search engine 126 provides the presentation engine 128 with information (e.g., document identifier) ​​of one or more electronic documents, relating to the search and optionally to the ranking of the electronic documents. The presentation engine 128 selects one or more of the electronic documents and makes the selected electronic documents available for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication.

[0058] Providing an electronic document for inclusion in a response can involve, for example, providing the actual document (e.g., embedding or otherwise attaching the document), providing a link to the document, providing a user interface reference to the document that is associated with the document, and so on. For example, the Presentation Engine 128 can automatically attach a selected electronic document to the response and / or automatically integrate links to the electronic documents into the response. The Presentation Engine 128 can also, for example, include one or more user interface references (e.g.,The electronic documents are presented graphically and audibly, and when a user generating a response selects one of the options, the corresponding electronic document(s) can be attached to the response and / or a link to the document(s) can be automatically provided in the response. In some implementations, the electronic document(s) can be included in a response in response to the user's selection of an interface element regarding the attachment document, or in response to any other user interface input indicating a desire to include an attachment in the response. This achieves the following technical effects and benefits.Automatically selecting and attaching non-textual response content, such as a document or link, to a response message saves the user from having to manually search for and select the non-textual response content, which simplifies the user interface and not only reduces the number of user dialog steps but also eliminates the corresponding input that is normally required to create a response message including the non-textual response content.

[0059] In some implementations, the Presentation Engine 128 selects and / or makes available the electronic documents for generating a response to the electronic communication, independently of any text-related input and / or other content provided via a user's computer device. In some of these implementations, the Presentation Engine 128 even selects the electronic documents before the user has viewed and otherwise consumed the communication.For example, the non-textual response content system 120 can process a communication before the user has seen it; the presentation engine 128 can select one or more electronic documents to include in a response to the communication and can even attach the selected electronic documents or otherwise interact with the electronic communication before the user has seen them. Thus, if a user sees or otherwise consumes the electronic communication, a response to the electronic communication, which already includes attached documents, links, or corresponding selection options as described above, can be provided quickly and without delay.

[0060] In some implementations, the presentation engine 128 selects and presents the electronic elements for inclusion in the response based on the optional ranking provided by the search engine 126. For example, in some implementations, the presentation engine 128 may select an electronic document only if its assigned ranking matches a corresponding threshold (e.g., if it is one of the highest-ranked electronic documents on X). The prominence with which a selected electronic document is presented and / or how the selected electronic document is presented may also be based on the selected electronic document's ranking.

[0061] In some implementations, the presentation engine 128 can select and provide multiple electronic documents for inclusion in a response. In some of these implementations, the multiple electronic documents can be provided based on an optional ranking of the electronic documents. For example, a presentation prominence can be determined for each of the multiple electronic documents based on the ranking of the multiple documents, which, along with an indication of presentation prominence, is provided for potential inclusion in the response to the electronic communication.

[0062] In some implementations where the client device 106 provides the electronic communication to the non-textual response content system 120, the presentation machine 128 can provide the client device 106 with the selected electronic documents for presentation to the user as an option for inclusion in a response. In some implementations where the electronic communication is provided to the non-textual response content system 120 by the electronic communication system 110 via the electronic communication database 152, the presentation machine 128 can store a link to the selected electronic documents for electronic communication in the electronic communication database 152 and / or another database. In some implementations, one or more (e.g.,all) aspects of the presentation machine 128 are implemented by the client device 106 and / or the electronic communication system 110.

[0063] The presentation machine 128 can also provide prominence information to be displayed with the selected electronic document, indicating the prominence (e.g., position, size, color) with which the selected electronic documents should be presented. Generally, the higher the ranking for a particular selected electronic document, the greater its prominence. As an example, when multiple electronic documents are selected by the presentation machine 128, the presentation machine 128 can provide information regarding the ranking of the multiple electronic documents, which is used to determine the order in which the multiple candidates should be presented to a user via a user interface output device of the client device 106.

[0064] In some implementations, the presentation machine 128 provides only document identifiers of the selected documents and potential prominence information, and the client device 106 and / or the electronic communication system 110 can generate a display of the selected documents based on the provided data. In such implementations, the presentation machine 128 can additionally provide some or all of the data required to generate the display. In some of these implementations, any of the provided prominence information can be included in the data specifying how the display should be presented.

[0065] In some implementations, the presentation machine 128 can determine whether and / or how to include electronic documents in a response based on an output provided by at least one trained machine learning system 135 in response to the original message features provided by the message feature machine 122 to the trained machine learning system 135. As described, for example, herein, at least one trained machine learning system 135 can be trained to receive one or more message features as input and to provide as output a probability that a response to the electronic communication will include non-textual response content. In some implementations, the presentation machine 128 can use the probability of whether to include electronic documents in a response.For example, the presentation machine 128 can provide the electronic document for inclusion in a response only if the probability matches a corresponding threshold. In some implementations, the presentation machine 128 can use probability in determining how to provide electronic documents for inclusion in a response. For example, the presentation machine 128 can automatically include the electronic documents in a response if the probability matches a corresponding threshold, but can require user interface input before including them in the response if the probability does not match the corresponding threshold. For example, the presentation machine 128 is also capable of doing the following: if the probability is greater than or equal to X (e.g.,X=0.8) is to provide electronic documents for inclusion in a response before any user interface input of any kind has been provided indicating a request for a response (e.g. . Fig. 4A); if the probability is less than X and greater than or equal to Y (e.g., Y=0.5), request a user interface prompt indicating a wish to respond before providing the electronic document (e.g., Fig. 4B); and if the probability is less than Y, request a user interface prompt indicating a wish to attach a document (e.g. Fig. 4D1 and Fig. 4D2) before the electronic document is made available.

[0066] Fig. Figure 2 illustrates an example of how non-textual response content, intended to be part of a reply to electronic communication, can be determined based on one or more message features of the communication. Message feature machine 122 determines one or more message features of the electronic communication 152A sent to a user. The electronic communication 152A can originate, for example, from client device 106 or electronic communication system 110. Fig. 1 will be provided.

[0067] The message feature machine 122 provides one or more of the determined message features to the search parameter machine 124 and provides one or more of the determined message features to the at least one trained machine learning system 135. The message features provided by the message feature machine 122 to the search parameter machine 124 and the machine learning system 135 can be the same or different features.

[0068] The trained machine learning system 135 provides one or more document attributes to the search parameter engine 124 based on the received message attributes. These one or more document attributes could, for example, be one or more document types within the non-textual response content. The search parameter engine 124 can use one or more document attributes to generate a search parameter.In cases where one of the trained machine learning systems 135, for example, provides the output of the document type(s) of the non-textual response content based on an input of the message functions for electronic communication, the document type(s) can be used as a search parameter that directs this document type(s), restricts the search to that document type(s), and / or restricts the search to the one or more corpora that contain (and is optionally limited to) documents of this type(s). This search parameter engine 124 also generates one or more search parameters based on the message features obtained from the message feature engine 122.

[0069] Search parameter engine 124 provides search parameters 126 to search engine 126. Search engine 126 searches for one or more of the corpora of electronic documents 154A-N based on the search parameters determined by search parameter engine 124. In some implementations, searching for one or more corpora of electronic documents 154A-N may involve searching for one or more indexes that index the electronic documents of the one or more document corpora. Search engine 126 may initiate one or more searches based on the search parameters, each combining the search parameters in one or more ways. In some implementations, search engine 126 may identify multiple documents in response to a search based on one or more search parameters that can be derived from an electronic communication.In some of these implementations, the multiple documents can each be classified based on different criteria, such as a query-based and / or a document-based evaluation result.

[0070] The search engine 126 provides the presentation engine 128 with document identifiers of the one or more electronic documents that refer to the search and provides the optional ranking of the electronic documents.

[0071] The presentation machine 128 selects one or more of the electronic documents and makes the selected electronic documents 159 available for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication. Making an electronic document available for inclusion in a response may, for example, involve providing the actual document, providing a link to the document, providing a user interface reference to the document that is associated with the document, etc.

[0072] Fig. Figure 3 is a flowchart illustrating an exemplary procedure for determining non-textual response content intended as part of a response to electronic communication based on one or more message features of the communication. For ease of reference, the operations of the flowchart are described with reference to a system performing the operations. This system may include various components of different computer systems. For example, some operations may be performed by one or more components of the non-textual response content system 120, such as machines 122, 124, 126, and / or 128. Furthermore, operations of the procedure from Fig. The fact that the three operations are shown in a specific order is not to be understood as restrictive. One or more operations can be rearranged, omitted, or added.

[0073] Block 300 identifies an electronic communication being sent to a user.

[0074] In block 302, the system identifies one or more message features of the electronic communication. Various features of the original message, such as text-related, semantic, and / or syntactic features, can be used. For example, the system can identify message features based on multiple n-grams in the electronic communication, the simultaneous occurrence of two or more n-grams in the electronic communication, features based on the text, particularly in subject lines, opening sentences, closing sentences, or other sections of the electronic communication, features based on metadata of the electronic communication, and so on.

[0075] At block 304, the system initiates a search of one or more corpora of electronic documents using search parameters based on one or more of the message features from block 302. For example, the system can use message features as search parameters and / or derive the search parameters from the message features. In some implementations, the system can provide one or more of the message features as input to a trained machine learning system and use the output of the trained machine learning system as one or more search parameters and / or to derive one or more of the search parameters.

[0076] Block 306 provides the system with information about a subset of electronic documents. This subset is a subset of corpora representing electronic documents and references the search from Block 304. In some implementations, the system also receives a ranking of the subset.

[0077] For block 308, the system selects at least one electronic document from the subset. For example, the system can select at least the one electronic document based on the optional ranking from block 306.

[0078] In Block 310, the system provides at least one electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication. Providing an electronic document for inclusion in a response may involve, for example, providing the actual document (e.g., embedding or otherwise attaching the document), providing a link to the document, providing a user interface reference to the document that is associated with the document, etc.Accordingly, a user is not required to manually browse for or select non-textual response content, which simplifies the user interface and not only reduces the number of user dialog steps but also eliminates the corresponding input that is normally required to create a response message including the non-textual response content.

[0079] Fig. Sections 4A-4E illustrate exemplary graphical user interfaces for providing non-textual response content for inclusion in a reply to electronic communication. The graphical user interfaces from Fig. 4A-4E can be presented on the client device 106 based on the non-textual response content, which is determined and provided by the non-textual response content system 120. In some implementations, one or more aspects of the non-textual response content system 120 (e.g., all or some aspects of the presentation engine 128) can be integrated into the client device 106, either wholly or partially.

[0080] In Fig. 4A: An original email 475A sent to a user is viewed by the user before the user provides any user interface input to indicate a wish to reply to the original email 475A. The candidate's electronic documents, represented by the graphical user interface elements 481A and 482A, are determined based on the original email 475A and presented for inclusion in a reply to the email. For example, a user selection of the graphical user interface element 481A can automatically present an editable reply email to the user, which includes the electronic document "Patent Presentation.pdf" attached to the reply, or contains a link to this electronic document integrated into the reply (e.g., a hyperlink embedded in the body of the reply).Similarly, a user selection of graphical user interface element 482A can automatically present the user with an editable reply email containing the electronic document "Budget Presentation.pdf" attached to the reply, or containing a link to this electronic document integrated into the reply (e.g., a hyperlink embedded in the reply body). In some implementations, both graphical user interface elements 481A and 482A can be selected to integrate both associated electronic documents into the reply.

[0081] Both graphical user interface elements 481A and 482A include a title of the associated electronic document (presented in bold and underlined) with related additional information to enable the user to re-capture the associated electronic document. Specifically, both graphical user interface elements 481A and 482A include additional information that provides a temporal indication of when the user last accessed the document, a temporal indication of when the document was last modified, a corpus ("cloud") and corpus folder location of the document ("Presentations"), and an author indicating a user who created the document. Additional and / or alternative additional information can be provided, such as a small portion of the text from the document (e.g.,(the first X words of the document), an image of the entire document or sections thereof, etc.

[0082] In some implementations, the presentation position of graphical user interface (GUI) elements 481A and 482A may be based on the determined display prominences derived from document rankings associated with these GUI elements 481A and 482A. For example, GUI element 481A may be presented higher in position than GUI element 482A because the document associated with GUI element 481A is ranked higher than the document associated with GUI element 482A. As described herein, the document rankings may be those resulting from a search initiated with parameters based on one or more message characteristics of the original email 475A.Additional and / or alternative presentations based on the classifications of the documents associated with graphical user interface elements 481A and 482A can be provided. For example, graphical user interface element 481A can be highlighted and / or presented with more additional information compared to graphical user interface element 482A. Fig. 4A also includes a selectable graphical user interface element 483A, which, when selected by a user, displays additional documents. These additional documents may be identified based on the original email (e.g., based on initiating a search with search parameters based on one or more message attributes of the original email), but they may have a lower priority than the documents associated with graphical user interface elements 481A and 482A.

[0083] In Fig. 4B A user has provided a user interface input (e.g., selecting a "Reply" element in the graphical user interface) to initiate a reply to the original email and receives a reply section 480B containing the phantom text "Composing email," informing the user that they can compose a reply in reply section 480B. The original email 475B, to which the reply refers, is also in Fig. 4B illustrates the user's reference when writing the answer.

[0084] The graphical user interface elements 481B and 482B are described in Fig. 4B presents the candidate's assigned electronic documents for inclusion in the answer, which are determined based on the original email 475B. Graphical user interface elements 481B and 482B are presented before the user has provided any text-related input or other content for the answer. The user's selection of graphical user interface element 481B allows the electronic document "Patent Presentation.pdf" to be attached to the answer or a link to this electronic document to be included in the answer (e.g., a hyperlink included in the answer section 480B). Similarly, a user's selection of graphical user interface element 482B allows the electronic document "Budget Presentation.pdf" to be attached to the answer or a link to this electronic document to be included in the answer.In some implementations, both graphical user interface elements 481B and 482B can be selected to integrate both associated electronic documents into the response. For example, an attachment can be integrated into the response via a single mouse click, a single tap, or another single user interface input.

[0085] Both graphical user interface elements 481B and 482B include a title of the associated electronic document (presented in bold and underlined) with related additional information to facilitate re-capture of the associated electronic document by the user. In some implementations, the presentation position of graphical user interface elements 481B and 482B may be based on the determined display prominences, which are based on the ranking of documents associated with these graphical user interface elements 481B and 482B.For example, graphical user interface element 481B can be presented to the left of graphical user interface element 482B, based on the document associated with graphical user interface element 482B that is ranked higher than the document associated with graphical user interface element 481B. As described herein, the document rankings can be the rankings resulting from a search initiated with parameters based on one or more message attributes of the original email.

[0086] In Fig. In response to email 4C, a user has provided a user interface input (e.g., selecting a "Reply" item in the graphical user interface) to initiate a reply to the original email and receives a response section 480C containing the phantom text "Composing email," informing the user that they can compose a reply in response section 480C. The original email 475C, to which the reply refers, is also included in response section 480C. Fig. 4C is illustrated as a reference for the user when writing the answer.

[0087] The graphical user interface element 481C is presented to and associated with the candidate's multiple electronic documents, which are identified based on the original email 475C. For example, the graphical user interface element 481C can be associated with all electronic presentations according to which the user composes the answer. The electronic presentations can be identified by initiating a search of the user's documents using a presentation search parameter based on the original email 475C.The user selection of graphical user interface element 481C can present the user with the graphical details of the user's electronic presentations, whereby one or more of them can be selected to attach the selected electronic presentation(s) to the answer or to integrate a link (links) to the electronic presentation(s) into the answer (e.g., a hyperlink integrated into the answer section 480C).

[0088] Graphical user interface element 482C is associated with all of the user's electronic documents. The user's selection of graphical user interface element 482C can present the user with graphical representations of all of the user's electronic documents. The user can browse all of the electronic documents and select one or more to attach to the answer or to include a link (links) to the electronic document(s) in the answer (e.g., a hyperlink included in an answer section 480C). Graphical user interface elements 481C and 482C are described in Fig. 4C presents the response before the user has provided any text-related input or other content.

[0089] In Fig. In 4D1, a user has provided a user interface input (e.g., selecting a "Reply" item in the graphical user interface) to initiate a reply to the original email and receives a reply section 480D containing the phantom text "Composing email," informing the user that they can compose a reply in reply section 480D. The original email 475D, to which the reply refers, is also included in Fig. 4D1 is illustrated as a reference for the user when writing the answer. Fig. 4D1 also presents an appended element of the graphical user interface 477D. In response to the user interface input that selects the appended elements of the graphical user interface 477D, the appended graphical user interfaces are selected from Fig. 4D2 presents.

[0090] The graphical user interfaces used for attaching from Fig. 4D2 includes the graphical user interface elements 481D and 482D for inclusion in the answer of the candidate's related electronic documents, which are determined based on the original email 475D. The graphical user interface elements 481D and 482D are presented in response to the selection of the attachment elements of the graphical user interface 477D and are presented before the user has provided any text-related input or other content for the answer. The user selection of graphical user interface element 481D can attach the electronic document "Patent Presentation.pdf" to the answer or include a link to this electronic document in the answer (e.g., a hyperlink included in the answer section 480D). Similarly, a user selection of graphical user interface element 482D can attach the electronic document "Budget Presentation.pdf" to the answer.Attach a PDF to the response or include a link to this electronic document in the response. In some implementations, both elements of the graphical user interface 481D and 482D can be selected to include both associated electronic documents in the response. For example, an attachment can be included in the response via a single mouse click or other single user interface input.

[0091] Both graphical user interface elements 481D and 482D include a title of the associated electronic document (presented in bold and underlined) with related additional information to facilitate re-capture of the associated electronic document by the user. In some implementations, the presentation position of graphical user interface elements 481D and 482D may be based on the determined display prominences, which are based on the ranking of documents associated with these graphical user interface elements 481D and 482D.For example, graphical user interface element 481D can be presented to the left of graphical user interface element 482D, based on the document associated with graphical user interface element 481D that ranks higher than the document associated with graphical user interface element 482D. As described herein, the document rankings can be the rankings resulting from a search initiated with parameters based on one or more message attributes of the original email.

[0092] The graphical user interfaces used for attaching from Fig. 4D2 also includes the graphical user interface element 483D, which, when provided via a user-selected user interface input, displays additional documents determined based on the original email 475D (e.g., based on initiating a search with search parameters based on one or more message features of the original email), but which have a lower priority than the documents associated with the graphical user interface elements 481A and 482A. The graphical user interface used for attaching documents consists of Fig. 4D2 also includes a graphical user interface element 484D which, when selected via the user interface input provided by a user, enables the user to browse all of the user's electronic documents and select one or more to attach to the response or to include a link (links) to the electronic document (to the electronic documents) in the response.

[0093] In Fig. 4E A user has provided a user interface input (e.g., selecting a "Reply" item in the graphical user interface) to initiate a reply to the original email and receives a reply section 480E containing the phantom text "Reply," informing the user that they can compose a reply in reply section 480E. The original text message 475E, to which the reply refers, is also included in Fig. 4E illustrates the user's reference when writing the answer.

[0094] Graphical User Interface (GUI) element 481E is presented and associated with an initial set of the candidate's multiple electronic documents, identified based on the original email 475E. Specifically, GUI element 481E is associated with all images taken by the responding user "last week" in "Chicago," which can be identified by initiating a search of the user's documents using the parameters "images," "Chicago," and "last week," based on the original text message 475E. The user of GUI element 481E can attach these images to the response or include a link(s) to these images in the response (e.g., a hyperlink included in the response section 480E).

[0095] Graphical User Interface (GUI) element 482E is presented and associated with a second set of the candidate's multiple electronic documents, which are determined based on the original text message 475E. Specifically, GUI element 482E is associated with all of the answering user's images taken "last week" in "Chicago" that include "Tom" in the image and that can be identified by initiating a search of the user's documents using the search parameters "images," "Chicago," "last week," and "Tom" (e.g., a user identifier associated with Tom), based on the original text message 475E. The user selection of GUI element 482E can attach these images to the answer or include a link(s) to these images in the answer (e.g.,a hyperlink that is integrated into answer section 480E).

[0096] Graphical user interface element 483E is also presented and associated with the first set of images as a graphical user interface element 481E. However, the user selection of graphical user interface element 483E can be presented to the user with graphical representations of the images, allowing the user to select one or more of them to attach to the response or to include a link (or links) to the image(s) in the response (e.g., a hyperlink integrated into the response section 480E). According to the implementations and examples described, this simplifies user dialogue regarding the inclusion of non-textual response content in a response message. For example, it is no longer necessary for a user to browse and select non-textual response content.Furthermore, the methods described above can also simplify the user interface, as the appending of non-textual response content occurs automatically, thus automatically avoiding the need for a browsing and selection interface or requiring only a simple selection option. Such a simplification of the user interface is particularly useful and advantageous, for example, on mobile communication devices, which typically have limited display size and / or user dialog capabilities.

[0097] With reference to the following Fig. 5-7, an additional description of the training of the at least one trained machine learning system 135 from the Fig. 1 and Fig. 2 provided, for example in different devices and procedures that relate to the Fig. 1-4 can be described and used.

[0098] Fig. Figure 5 illustrates an exemplary environment in which electronic communications can be analyzed to generate training examples for training a machine learning system, to identify one or more non-textual response contents, and to determine in which of these the machine learning system should be trained based on the training examples.

[0099] The exemplary environment from Fig. 5 includes the electronic communication system 110, the electronic communication database 152, a machine learning system 135A, which includes at least one trained machine learning system 135 from Fig. 1 represents his untrained state. The exemplary environment from Fig. 5 also includes a selection engine 130 and a training example system 140.

[0100] Selection Machine 130, Training Example System 140, and Machine Learning System 135A can each be implemented in one or more computer devices that communicate, for example, via a network. Selection Machine 130, Training Example System 140, and Machine Learning System 135A are exemplary systems in which the systems, components, and techniques described herein can be implemented and / or with which the systems, components, and techniques described herein can be connected via an interface. Selection Machine 130, Training Example System 140, and Machine Learning System 135A each include one or more memories for storing data and software applications, one or more processors for accessing data and executing applications, and other components that enable communication via a network.In some implementations, the selection engine 130 and the training example system 140 may consist of one or more components of the exemplary computer device. Fig. 8. The operations performed by the selection machine 130 and the training example system 140 and / or the machine learning system 135A can be distributed across multiple computer systems. In some implementations, one or more aspects of the selection machine 130, the training example system 140 and / or the machine learning system 135A can be combined into a single system.

[0101] In general, in some implementations without direct human access, the training example system 140 uses past electronic communications from the electronic communications database 152 to generate training examples for training the machine learning system 135A. These training examples can be generated to train the machine learning system 135A to learn the relationships between one or more message features of the original messages and one or more features related to the attachments in the responses to the original messages.For example, in some implementations, the 135A machine learning system can be trained to determine relationships between message features of the original message and the probability that replies to the electronic communication containing these message features will include a document or a link to that document. Furthermore, in other implementations, the 135A machine learning system can be trained, for example, to determine relationships between message features of the original message and the document type(s) (e.g., images, videos, media, PDFs, slides) of documents that are included, either directly or via a link, in replies to the electronic communication containing such message features.

[0102] In some implementations, the selection engine 130 can select communications used by the training example system 140 based on one or more criteria to generate training examples for training each of the one or more machine learning systems 135. For example, the selection engine 130 can flag or otherwise identify certain communications from the electronic communications database 152 that are suitable for use by the training example system 140. In some implementations, the selection engine 130 can select the electronic communications to be used based on these electronic communications, including the original message and a reply message that refers to the original message.As described herein, the electronic communication, which includes an original message and a reply message, can be a single document and / or multiple related documents. In some implementations, the selection engine 130 can select the electronic communication used for the training examples for the machine learning system 135A based on this electronic communication, including a reply with non-textual content, such as an attached document and / or a link to a document.

[0103] In some implementations, the selection engine 130 can employ one or more methods to reduce the occurrence of certain communication types used in generating the training examples. For example, in cases where the database 152 contains emails, the selection engine 130 can use techniques to filter out emails that are highly likely to originate from businesses. For instance, it can filter out emails from specific email addresses, emails from email addresses with specific domain names, emails from email addresses with specific prefixes, emails with specific n-grams in a subject line, and so on. Furthermore, it can also filter out emails that conform to specific business templates. Additionally, it can filter out emails that are highly likely to be spam.Reducing the occurrence of certain communication types has the technical effect and advantage that processing and training for the machine learning process can be carried out more efficiently, leading to higher precision in the output provided by the machine learning system. In some implementations, the selection machine chooses 130 electronic communications based on other attributes associated with the communication and / or the sender and / or the receiver. For example, it is desirable to identify relationships between features of the original message and the response N-gram for a specific geographic region and / or domain, enabling the selection of communications associated with that geographic region and / or domain.

[0104] In various implementations, the training example system 140 includes a feature detection engine for the original message 142 and a feature detection engine for non-textual response content 144. In some implementations, all or aspects of the machines 142 and / or 144 may be omitted, combined, and / or implemented in a component separate from the training example system 140.

[0105] In general, the original message feature detection engine 142 and the non-textual response content feature detection engine 144 work together to determine a variety of training examples, each based on a corresponding event from a variety of events in an electronic communication, including an original message and a response message.For a given electronic communication that has an original message and a reply message, the original message feature determination engine 142 determines a vector of original message features based on the original message of the given electronic communication, and the non-textual reply content feature engine 144 determines a vector of one or more non-textual reply content features based on the reply message of the given electronic communication.

[0106] The vector of features of the original message and the vector of one or more features of the non-textual response content included in the training examples depend on the desired input and output parameters of the machine learning system 135A implementation. For example, suppose that the machine learning system 135A has been trained to provide as output a probability that a response to an electronic communication contains a document or a link to a document.In such a situation, the feature engine of non-textual response content can generate 144 training examples, each containing a vector of one or more features of the non-textual response content, where the feature is a single feature that is either "correct" (includes a document and / or a link to a document) or "incorrect" (does not include a document and / or a link to a document).In another example, it is assumed that the machine learning system 135A has been trained to provide the following output: 1) a probability that a response to electronic communication includes a word processing document and / or a link to a word processing document; 2) a probability that a response to electronic communication includes a presentation and / or a link to a presentation; 3) a probability that a response to electronic communication includes an image and / or a link to an image; 4) a probability that a response to electronic communication includes a video and / or a link to a video; and 5) optionally, the probability(s) for additional and / or alternative document types.In such a situation, the feature engine of the non-textual response content can generate 144 training examples, each containing a vector of a multitude of the content features of the response that is either "correct" (includes a document of this type and / or a link to a document of this type) or "incorrect" (does not include a document of this type and / or a link to a document of this type).

[0107] Various features of the original message can be determined by the Original Message Feature Determination Engine 142, including syntactic, semantic, n-gram, and / or metadata-based features. For example, one or more original message features can indicate whether a particular n-gram is present in one or more locations within the original message, and whether any n-gram belonging to a specific class of n-grams is present in one or more locations within the original message. A class of n-grams might, for example, be a group of n-grams that share a similar semantic meaning, such as a group of "requiring verbs," like "provide," "send," "include," "can," "attach," and so on. As another example, an original message feature might indicate the number of recipients of the original email, such as "one," "two to five," or "five or more."

[0108] As yet another example, a feature of an original message can specify a semantic feature of one or more sections of the original message, such as a semantic feature of the subject line of the original message, the entire body, or sections of the body of the original message, etc. In some implementations, the original message feature discovery engine can determine one or more semantic features of an original message based on the grouping of electronic communication events into a multitude of clusters, and determine the semantic features of the original message based on its cluster.In some implementations, the original message feature detection engine groups the electronic communication events into a multitude of clusters based on similarities between the text of the original messages in the corpus. These similarities include, but are not limited to, semantic, syntactic, and / or textual similarities. Generally, the original messages grouped into a given cluster exhibit greater similarities to one another (based on the similarities used in that grouping) than the original messages grouped into other clusters. Each of the identified clusters corresponds to a different semantic category of original message content.Based on similarity measures between the original messages, the original message feature detection engine 142, in some implementations, can use one or more clustering techniques to group electronic communication events into a variety of clusters. For example, in some implementations, X-means clustering can be used, where the distance between original messages is based on similarity measures between them. In general, X-means clustering is an unsupervised procedure for determining the ideal k for use in the K-means clustering procedure. In general, the K-means clustering procedure aims to divide the observations into a variety of groups, with each observation being included in the group to which it has the greatest relationship. Additional and / or alternative clustering procedures can be used optionally.

[0109] The training examples generated by training example system 140 for machine learning system 135A are provided to machine learning system 135A for training purposes. During training, machine learning system 135A iteratively learns a hierarchy of feature representations based on the training examples generated by training example system 140.

[0110] With reference to the following Fig. Section 6 illustrates how training examples based on electronic communications can be generated and used to train a machine learning system and to identify one or more non-textual response content features. The selection engine 130 can select communication events from the electronic communication database 152 based on one or more criteria. These events must be used to generate training examples for training the machine learning system 135A. For example, the selection engine 130 can mark or otherwise identify specific communications from the electronic communication database 152 that are suitable for use in generating training examples.

[0111] For a variety of electronic communications, the feature-matching engine of the original message 142 determines a vector of the features of the original message, based on the original message of the given electronic communication, and includes the features of the original message as input parameters of a corresponding training example 145. The feature-matching engine of the non-textual response content 144 determines a vector of one or more features of the non-textual response content based on the response message of the given electronic communication and includes the features of the non-textual response content as output parameters of the corresponding training example 145. The training example 145 is used to train the machine learning system 135A. Although only a single training example 145 in Fig. As illustrated in Figure 6, it is self-evident that machines 142 and 144 will generate a multitude of training examples (each based on a corresponding electronic communication) and the multitude of training examples will be used to train the machine learning system 135A.

[0112] As a specific implementation from Fig. 6. Training example 145 and all additional training examples each have an output parameter that indicates a document type (or types) contained in a response message to a corresponding electronic communication and one or more input parameters based on the original message of the corresponding electronic communication. For example, the feature engine for non-textual response content 144 can, based on a response message containing an image as an attachment, generate the features of the non-textual response content for an output of an initial training example that includes a specification of the document type of the image. The feature engine for the original message 142 can, based on a corresponding original message, generate features of an original message as an input to the first training example.These features of the original message can include, for example, one or more syntactic, semantic, and / or N-gram-based features of the original message. Furthermore, the feature engine for non-textual response content 144 can, for example, generate the features of the non-textual response content for an output of a second training example that includes a document type specification for the PDF file, based on a response message containing a "PDF" file as an attachment. The feature engine for the original message 142 can generate features of an original message as an input to the second training example, based on a corresponding original message. These features of the original message can include, for example, one or more syntactic, semantic, and / or N-gram-based features of the original message.Additional training examples can be generated similarly, including additional examples, each with output features based on other document types of a corresponding reply message of the additional electronic communication, and input features based on a corresponding original message of the electronic communication. The machine learning system can be trained based on these training examples. The trained machine learning system can receive as input one or more message features of an original message and can provide as output one or more document types and, optionally, the associated probability of being included in a reply to the original message.The trained machine learning system can be used, for example, to determine one or more search parameters for future electronic communications based on the document types provided as output by the trained machine learning system, and / or to restrict the corpora of the initiated search based on the document types provided as output by the trained machine learning system.

[0113] As another specific implementation from Fig. 6. Training example 145 and all additional training examples each have an output parameter that indicates the probability of a document being contained in a reply message to a corresponding electronic communication and include one or more input parameters based on the original message of the corresponding electronic communication. For example, the feature engine of non-textual response content 144 can, based on a reply message containing an attached document and / or a link to a document, generate features of non-textual response content for an output of an initial positive training example that includes the statement "Document contained in reply".The Original Message Feature Recognition Engine 142 can generate features of an original message as input for the first training example, based on a corresponding original message. These original message features can include, for example, one or more syntactic, semantic, and / or n-gram-based features of the original message. Furthermore, the Non-Textual Response Content Feature Engine 144 can, for example, generate features of the non-textual response content for an output of a second negative training example containing the statement "Document not included in response," based on a response message that does not contain a document and / or a link to a document.The feature detection engine of the original message 142 can generate features of an original message as input for the second training example, based on a corresponding original message. These features of the original message can include, for example, one or more syntactic, semantic, and / or N-gram-based features of the original message. Additional training examples can be generated in a similar manner. The trained machine learning system can receive one or more features of an original message as input and can provide as output the probability that a document will be included in a response to the original message.The trained machine learning system can be used, for example, to determine for future electronic communications whether and / or how attachments are provided for inclusion in a response to that electronic communication (e.g., a low probability that a response will include an attachment may result in no attachments being provided, or the attachments being "suggested" in a less conspicuous way).

[0114] Fig. Figure 7 is a flowchart illustrating an exemplary procedure for generating training examples and using these examples to train a machine learning system to identify one or more non-textual response content features. For ease of reference, the operations of the flowchart are described with reference to a system performing the operations. This system may include various components of different computer systems. For example, some operations may be performed by one or more components of the training example system 140 and / or the machine learning system 135A. Fig. 5 are performed. Furthermore, operations of the procedure are also carried out. Fig. The fact that the 7 operations are shown in a specific order is not to be understood as restrictive. One or more operations can be rearranged, omitted, or added.

[0115] At block 700, the system identifies an electronic communication. Every electronic communication includes an original message and a reply message.

[0116] In block 705, the system generates input features for a training example based on the original message of a single event in the electronic communication. Various features of the original message can be determined by the system as syntactic, semantic, n-gram, and / or metadata-based features.

[0117] In Block 710, the system generates output attributes of the training example based on the non-textual response content associated with the response message of the electronic communication. For example, in one implementation, the output attribute(s) can be either "true" or "false," and are true if the response message contains a document or links to a document. Otherwise, they are false. Furthermore, in another implementation, the output attribute can include, for example, multiple attributes, each either "true" or "false," indicating whether the response message contains a document of a corresponding file type that specifies a closed class of one or more filename suffixes. For example, a first attribute might specify "images," indicating a closed class of images (e.g., jpg, .png, etc.).gif), and a second feature can specify "presentations", which indicates a closed class of presentations, etc. (e.g., pdf, .ppt), etc.

[0118] In block 715, the system trains a machine learning system based on the training example.

[0119] Although the procedure from Fig. Since section 7 is described with reference to a single training example, it is self-evident that one or more of the steps (e.g., blocks 705, 710 and 715) are performed iteratively in order to identify multiple training examples and to train the machine learning system based on the multiple training examples.

[0120] In situations where systems described herein collect or utilize personal information about users, users may be provided with a means of controlling whether programs or features should collect user information (e.g., information about a user's social network, social actions or activities, occupation, user preferences, or current geographic location). Alternatively, users may be provided with a means of controlling whether and / or how content that may be of greater relevance to the user is retrieved from the content server. Furthermore, certain data may be processed in one or more ways before being stored or used, such that information that could identify individuals is removed.For example, a user's identity can be handled in such a way that no personally identifiable information can be determined for the user, or a user's geographic location can be generalized by extracting geographic location information (such as a city, postal code, or state level) so that a specific geographic location of the user cannot be determined. Therefore, the user can have control over how information about the user is collected or used.

[0121] Fig. Figure 8 is a block diagram of an exemplary computer system 810. Computer device 810 typically includes at least one processor 814, which communicates with a number of peripheral devices via the bus subsystem 812. These peripheral devices may include a memory subsystem 824, including, for example, a memory subsystem 825 and a file storage subsystem 826, user interface output devices 820, user interface input devices 822, and a network interface subsystem 816. The input and output devices enable user interaction with the computer device 810. The network interface subsystem 816 provides an interface to external networks and is coupled to corresponding interface devices in other computer devices.

[0122] User interface input devices 822 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, and / or graphics tablet, a scanner, a touchscreen integrated into a display, audio input devices such as voice recognition systems, microphones, and / or other types of input devices. In general, the use of the term "input device" is intended to include all possible types of devices and methods of inputting information into a computer device 810 or into a communications network.

[0123] User interface devices 820 can include a display subsystem, a printer, a fax machine, or a non-visual display, such as audio output devices. The display subsystem can include a cathode ray tube (CRT), a flat-panel display device, such as a liquid crystal display (LCD), a projector, or any other mechanism for producing a visible image. The display subsystem can also provide a non-visual display, such as through audio output devices. In general, the use of the term "output device" is intended to include all possible types of devices and methods of outputting information from a computer device 810 or any other machine or computer device.

[0124] Memory subsystem 824 stores program and data constructs that provide the functionality of some or all of the modules described herein. For example, memory subsystem 824 may contain the logic to execute selected aspects of the procedures from Fig. 3 and / or Fig. 7 to be carried out.

[0125] These software modules are generally executed by the 817 processor alone or in combination with other processors. The 825 memory, used in the memory subsystem, can include several memories, including a main access memory (RAM) 830 for storing instructions and data during program execution, and a read-only memory (ROM) 832 in which fixed instructions are stored. A file storage subsystem 826 can provide persistent storage for program and data files and can include a hard disk drive, a floppy disk drive along with associated removable media, a CD-ROM drive, an optical drive, or removable media.The modules which implement the functionality of certain implementations can be stored from the file storage subsystem 826 in the memory subsystem 827 or in other machines accessible to the processor 817.

[0126] The 812 bus subsystem provides a mechanism that allows different components and subsystems of the 810 computer to communicate with each other as intended. Although the 812 bus subsystem is schematically shown as a single bus, alternative implementations of the bus subsystem can use multiple buses.

[0127] Computer Device 810 can be of various types, including a workstation, server, computer cluster, blade server, server farm, and other data processing systems or computer equipment. Because of the constantly changing nature of computers and networks, the description of Computer Device 810, which is set out in Fig. Figure 8 is shown only as a specific example to illustrate some implementations. Many other configurations of the Computer Device 810 may have more or fewer components than the Computer Device shown in Figure 810. Fig. 8 is shown.

[0128] Further implementations are summarized in the following examples: Example 1: A computer-implemented method comprising the following: the identification, by one or more computer devices, of an electronic communication sent to a user; the determination, by one or more of the computer devices, of a message feature of the electronic communication; the initiation, by one or more of the computer devices, of a search of one or more corpora of electronic documents using a search parameter based on the message feature of the electronic communication; and, in response to the initiation of the search, the receipt, by one or more computer devices, of a reference to a subset of the one or more electronic documents of the one or more corpora that are relevant to the search.The selection, carried out by one or more computer devices and based on the receipt of the information, of at least one electronic document selected from the subgroup of electronic devices; the provision, carried out by one or more computer devices, of at least one section of the at least one electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication. Example 2: Computer-implemented method according to Example 1, wherein initiating the search is independent of any text-related input provided via a user's computer device when generating the response to the electronic communication, wherein the user's computer device is one of the computer devices or is in addition to the computer devices. Example 3: Computer-implemented method according to Example 1 or 2, wherein the provision of at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication is independent of any text-related input provided via the computer device when generating the response to the electronic communication. Example 4: Computer-implemented method according to one of Examples 1 to 3, wherein at least one corpus of the one or more corpora that are not publicly accessible is accessible to the user. Example 5: Computer-implemented method according to one of Examples 1 to 4, wherein at least one or more corpora are accessible only to the user and one or more additional users or systems authorized by the user. Example 6: Computer-implemented method according to one of Examples 1 to 5, wherein initiating the search of the one or more corpora includes initiating a search of the one or more indexes that index the electronic documents of the one or more corpora. Example 7: A computer-implemented method of one of Examples 1 to 6, wherein the subset comprises a plurality of electronic documents and further comprises the following: in response to the initiation of the search, the obtaining by one or more computer devices of a search ranking for the subset of electronic documents of the one or more corpora referring to the search; and wherein the selection of the at least one selected electronic document of the electronic documents of the subset is further based on the search rankings for the subset of electronic documents. Example 8: Computer-implemented method according to Example 7, wherein the at least one selected electronic document comprises a first document and a second document, and the provision of the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises: determining a prominence for the provision of the first document in the second document based on the search rankings and providing both the first and the second document for inclusion in the response to the electronic communication along with a specification of the respective prominences. Example 9: Computer-implemented method according to any one of Examples 1 to 8, further comprising: the determination of an additional message feature of the electronic communication by one or more of the computer devices; and the determination of an additional message feature of the electronic communication by one or more of the computer devices. Example 10: Computer-implemented method according to any of Examples 1 to 9, wherein providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication includes attaching the at least one section of the at least one selected electronic document with respect to the response, without requiring the user to confirm it via a user-initiated user interface input. Example 11: Computer-implemented method according to any of Examples 1 to 9, wherein providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises: providing a graphic representation of the at least one section of the at least one selected electronic document; receiving a selection of the graphic representation via a user interface input device; and, in response to receiving the selection, attaching at least one selected electronic document to the response. Example 12: Computer-implemented method according to any of Examples 1 to 11, wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication includes providing a link in the response, the link being to at least one section of the at least one selected electronic document. Example 13: Computer-implemented method according to Examples 1 to 9, wherein the at least one selected electronic document comprises a first document and a second document, and wherein providing the at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: providing a first graphic representation of the first document and a second graphic representation of the second document; receiving a selection of the first graphic representation and the second graphic representation via a user interface input device; and, in response to receiving the selection, adding a corresponding selection of the first document and the second document to the response. Example 14: Computer-implemented method according to any of Examples 1 to 13 further comprising: the determination of an additional message feature of the electronic communication by one or more of the computer devices; the provision of the at least additional message feature as input to a trained machine learning system; the receipt of the one document feature as output from the trained machine learning system; and the use of an additional parameter for the search based on at least one document feature. Example 15: Computer-implemented method according to Example 14, wherein the at least one document attribute includes a document type attribute that specifies a closed class of one or more file name suffixes. Example 16: Computer-implemented method according to any of Examples 1 to 13 further comprising: the determination of an additional message feature of the electronic communication by one or more of the computer devices; the provision of the at least additional message feature as input to a trained machine learning system; the receipt of the one document feature as output from the trained machine learning system; and wherein the selection of the at least one selected electronic document is further based on at least one document feature. Example 17: Computer-implemented method according to any of Examples 1 to 13 further comprising: the determination of an additional message feature of the electronic communication by one or more of the computer devices; the provision of the at least additional message feature as input to a trained machine learning system; the receipt of the one document feature as output from the trained machine learning system; and the restriction of the one or more corpora of the search based on the at least one document feature. Example 18: Computer-implemented method according to one of Examples 1 to 17, wherein the message feature is embedded in the vector of one or more features of the electronic communication. Example 19: Computer-implemented method of one of Examples 1 to 18, wherein the message feature is based on an N-gram in a main part of the electronic communication and wherein determining the message function based on the N-gram is based on the proximity of the N-gram to the requesting verb N-gram in the main part of the electronic communication. Example 20: Computer-implemented method according to any of Examples 1 to 19 further comprising: marking each plurality of N-grams of electronic communication with at least one corresponding grammatical annotation; wherein determining the message feature comprises selecting an N-gram of the N-grams based on the corresponding grammatical annotation of the N-gram and determining the message feature based on the N-gram. Example 21: A system comprising: an electronic communication stored on one or more non-transitory computer-readable media, wherein the electronic communication is sent to a user; at least one processor; memory coupled to the processor, wherein the memory stores instructions to be executed by the processor to perform steps that include: identifying a message feature of the electronic communication; initiating a search of one or more corpora of electronic documents using a search parameter based on the message feature of the electronic communication; in response to initiating the search, receiving a reference to a subset of the one or more electronic documents of the one or more corpora that refer to the search;Based on receiving the information, selecting at least one electronic document from the subset of electronic devices; providing at least one section of the at least one electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication. Example 22: A non-transitory, computer-readable storage medium that stores at least one program configured to be executed by at least one processor of a computer system, wherein the at least one program comprises instructions to: identify an electronic communication sent to a user; determine a message feature of the electronic communication; initiate a search of one or more corpora of electronic documents using a search parameter based on the message feature of the electronic communication; receive, in response to initiating the search, a specification of a subset of the one or more electronic documents of the one or more corpora to be referenced by the search; and, based on receiving the specification, select at least one from the subset of electronic devices of the selected electronic document.to provide at least one section of the at least one electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication.

[0129] Although various implementations are described and illustrated herein, a variety of other mechanisms and / or structures can be used to perform the function and / or to obtain the results and / or to achieve one or more of the advantages described herein, and it is intended that any such variations and / or modifications are included within the scope of the implementations as described herein. More precisely, any parameters, dimensions, materials, and configurations as described herein are to be considered exemplary, and the actual parameters, dimensions, materials, and / or configurations are to depend on the specific application or applications in which the teaching of the invention is applied.The person skilled in the art will recognize, or be able to ensure, that many equivalents of the specific implementations described herein can be obtained using a merely routine procedure. It should therefore be understood that the aforementioned embodiments are merely illustrated by way of example, and that, within the scope of the attached claims and their equivalents, embodiments according to the invention may be implemented in practice in ways other than those specifically described and claimed. Implementations of the present description are directed to each individual feature, system, article, material, set, and / or method as described herein.Additionally, any combination of two or more such features, systems, articles, materials, sentences and / or processes, provided that such features, systems, articles, materials, sentences and / or processes are not mutually inconsistent, is included within the scope of this disclosure.

Claims

[1] Computer-implemented method comprising the following: The identification of an electronic communication sent to a user, performed by one or more computer devices; the determination of a message characteristic of the electronic communication, carried out by one or more of the computer devices; the determination of an additional message feature of the electronic communication, carried out by one or more of the computer devices; Providing at least the additional message feature as input to a trained machine learning system; obtaining a single document feature as output from the trained machine learning system; initiating a search of one or more corpora of electronic documents by one or more of the computer devices using a search parameter based on the message characteristic of electronic communication; in response to the initiation of the search, the receipt by one or more computer devices of a reference to a subset of one or more electronic documents of the one or more corpora that refer to the search; the selection of at least one selected electronic document from the electronic documents of the subgroup, carried out by one or more computer devices and based on the receipt of the information, wherein the selection of the at least one selected electronic document is further based on at least one document attribute; and The provision, by one or more computer devices, of at least one section of a selected electronic document for inclusion in a response to electronic communication, which is a response from the user to electronic communication. [2] Computer-implemented method according to claim 1, wherein initiating the search is independent of any text-related input provided via a user's computer device when generating the response to the electronic communication, wherein the user's computer device is one of the computer devices or is in addition to the computer devices. [3] Computer-implemented method according to claim 1 or 2, wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication, independent of any text-related input provided via the computer device when generating the response to the electronic communication. [4] Computer-implemented method according to one of claims 1 to 3, wherein at least one corpus of the one or more corpora that are not publicly accessible is accessible to the user. [5] Computer-implemented method according to any one of claims 1 to 4, wherein at least one or the several corpora are accessible only to the user and one or more additional users or systems that have been authorized by the user. [6] Computer-implemented method according to any one of claims 1 to 5, wherein initiating the search of one or more corpora comprises initiating a search for one or more indexes that index the electronic documents of one or more corpora. [7] Computer-implemented method according to any one of claims 1 to 6, wherein the subgroup includes a plurality of electronic documents and further comprises the following: in response to the initiation of the search, the obtaining of a search ranking by one or more computer devices for the subset of electronic documents of the one or more corpora that refer to the search; and where the selection of at least one selected electronic document from the electronic documents of the subgroup is further based on the search rankings for the subgroup of electronic documents. [8] Computer-implemented method according to claim 7, wherein the at least one selected electronic document comprises a first document and a second document; and wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: Identifying a celebrity for providing the first document in the second document based on search rankings and Providing both the first and second documents for inclusion in the response to electronic communication, along with a description of the respective celebrities. [9] Computer-implemented method according to any one of claims 1 to 8, wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: the attachment of at least one section of the at least one selected electronic document regarding the answer, without requiring the user to confirm it via a user-initiated user interface input. [10] Computer-implemented method according to any one of claims 1 to 8, wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: Providing a graphic representation of at least one section of at least one selected electronic document; Receiving a selection of the graphical representation via a user interface input device; and In response to receiving the selection, attach at least one selected electronic document to the reply. [11] Computer-implemented method according to any one of claims 1 to 10, wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: Providing a link in the response, where the link leads to at least one section of the at least one selected electronic document. [12] Computer-implemented method according to any one of claims 1 to 8, wherein the at least one selected electronic document comprises a first document and a second document, and wherein providing at least one section of the at least one selected electronic document for inclusion in the response to the electronic communication comprises the following: Providing a first graphic representation of the first document and a second graphic representation of the second document; Receiving a selection of the first graphical representation and the second graphical representation via a user interface input device; and In response to receiving the selection, adding a corresponding selection of the first document and the second document to the response. [13] Computer-implemented method according to any one of claims 1 to 12, wherein the message feature is embedded in the vector of one or more features of the electronic communication. [14] Computer-implemented method according to any one of claims 1 to 13, wherein the message feature is based on an N-gram in a main part of the electronic communication and wherein determining the message function based on the N-gram is based on the proximity of the N-gram to the requesting verb N-gram in the main part of the electronic communication. [15] Computer-implemented method according to any one of claims 1 to 14, further comprising: the marking of each multitude of N-grams of electronic communication with at least one corresponding grammatical notation; where determining the message feature involves selecting an N-gram from the N-grams based on the corresponding grammatical note of the N-gram and determining the message feature based on the N-gram. [16] System comprising the following: an electronic communication that is stored in one or more non-transitory computer-readable media, wherein the electronic communication is sent to a user; at least one processor; Memory that is coupled to the processor, wherein the memory stores instructions to be executed by the processor to perform steps that include: Determining a message characteristic of electronic communication; Determining an additional message feature of electronic communication; Providing at least the additional message feature as input to a trained machine learning system; the receipt of at least one document attribute as output from the trained machine learning system; initiating a search of one or more corpora of electronic documents using a search parameter based on the message characteristic of electronic communication, and using an additional search parameter based on the at least one document characteristic; in response to initiating the search, receiving information on a subset of one or more electronic documents of the one or more corpora that refer to the search; based on receiving the information, selecting at least one selected electronic document from the electronic documents of the subgroup; and Providing at least one section of the at least one selected electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication. [17] A non-transitory computer-readable storage medium that stores at least one program configured to be executed by at least one processor of a computer system, wherein the at least one program comprises instructions to: to identify an electronic communication sent to a user; to determine a message characteristic of electronic communication; to determine an additional message feature of electronic communication; at least the additional message feature to be provided as input to a trained machine learning system; to obtain a document feature as output from the trained machine learning system; and to restrict one or more corpora of electronic documents in a search based on at least one document attribute; to initiate the search of one or more corpora of electronic documents using a search parameter based on the message characteristic of electronic communication; in response to initiating the search, receiving information on a subset of one or more electronic documents of the one or more corpora that refer to the search; based on receiving the information to select at least one selected electronic document from the electronic documents of the subgroup; and Providing at least one section of the at least one selected electronic document for inclusion in a response to the electronic communication, which is a response from the user to the electronic communication.

Images