Construct a fictional discourse tree to improve the ability to answer convergent questions

Abstract

Systems and methods for improving question-and-answer recall for complex, multi-sentence, convergent problems. More specifically, an autonomous agent accesses an initial answer that partially answers a question received from a user device. The agent represents the question and the initial answer as a discourse tree. The agent identifies entities in the question that are not resolved by the answer from the discourse tree. The agent forms additional discourse trees from additional resources such as a text corpus. The additional discourse trees rhetorically connect the unresolved entities to the answer. The agent designates the discourse tree as a fictional discourse tree. The fictional discourse tree, when combined with the initial answer discourse tree, is used to generate an answer that is improved relative to existing solutions.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 729,335, filed Sep. 10, 2018, and U.S. Provisional Application No. 62 / 668,963, filed May 9, 2018, the entire contents of which are incorporated herein by reference. Background Art

[0003] Linguistics is the scientific study of language. One aspect of linguistics is the application of computer science to human natural languages such as English. Due to the great increase in the speed of processors and the capacity of memory, the computer applications of linguistics are increasing. For example, computer - enabled analysis of language utterances has facilitated many applications that can answer questions from users, such as automated agents. Using autonomous agents or “chatbots” to answer questions, facilitate discussions, manage conversations, and provide social promotion is becoming increasingly popular.

[0004] Autonomous agents can serve queries received from user devices by generating answers based on information found using specific resources such as databases or by querying search engines. However, sometimes, a single resource or search - engine results cannot fully resolve complex user queries or convergent problems. A convergent problem is a problem that requires an answer with a high degree of accuracy.

[0005] Accordingly, when providing answers in multiple resources, existing systems are unable to form a complete and accurate answer to the question. Thus, new solutions are needed. Summary of the Invention

[0006] Aspects described herein use fictional discourse trees to improve question - and - answer recall for complex, multi - sentence, convergent problems. More specifically, an improved autonomous agent accesses an initial answer to a question received from a user device. The initial answer only partially resolves the question. The agent represents the question and the initial answer as a discourse tree and identifies entities in the question that are not resolved in the answer. The agent accesses additional resources such as a text corpus and determines answers that rhetorically connect the missing entities to another entity in the answer. The agent selects this additional resource, thereby forming a fictional discourse tree that, when combined with the discourse tree of the answer, can be used to generate an answer that is improved relative to existing solutions.

[0007] In one aspect, a method includes using a computing device and constructing a question discourse tree including question entities according to a question. The question discourse tree represents the rhetorical interrelationships between the basic discourse units of the question. The method further includes using the computing device and accessing an initial answer from a text corpus. The method further includes using the computing device to construct an answer discourse tree including answer entities from the initial answer. The answer discourse tree represents the rhetorical interrelationships between the basic discourse units of the initial answer. The method further includes using the computing device to determine that a score indicating the relevance of the answer entity to the question entity is lower than a threshold. The method further includes generating a fictional discourse tree. Generating the fictional discourse tree includes creating additional discourse trees from the text corpus. Generating the fictional discourse tree includes determining that the additional discourse trees include a rhetorical relationship connecting the question entity and the answer entity. Generating the fictional discourse tree includes extracting a subtree of the additional discourse tree including the question entity, the answer entity, and the rhetorical relationship, thereby generating the fictional discourse tree. Generating the fictional discourse tree includes outputting an answer represented by a combination of the answer discourse tree and the fictional discourse tree.

[0008] In an example, accessing the initial answer includes determining an answer relevance score for a portion of the text and, in response to determining that the answer relevance score is greater than the threshold, selecting the portion of the text as the initial answer.

[0009] In an example, the fictional discourse tree includes nodes representing rhetorical relationships. The method further includes integrating the fictional discourse tree into the answer discourse tree by connecting the nodes to the answer entity.

[0010] In an example, creating the additional discourse trees includes calculating a score indicating a number of question entities for each additional discourse tree, the number of question entities including a mapping to one or more answer entities in the corresponding additional discourse tree. Creating the additional discourse trees includes selecting the additional discourse tree having the highest score from the additional discourse trees.

[0011] In an example, creating the additional discourse trees includes calculating a score for each additional discourse tree by applying a trained classification model to (a) the question discourse tree and (b) one or more of the corresponding additional answer discourse trees; and selecting the additional discourse tree having the highest score from the additional discourse trees.

[0012] In an example, the question includes keywords, and accessing the initial answer includes obtaining an answer based on a search query including the keywords by performing a search of an electronic document. Accessing the initial answer includes determining an answer score indicating the degree of match between the question and the corresponding answer for each answer. Accessing the initial answer includes selecting the answer having the highest score from the answers as the initial answer.

[0013] In an example, calculating a score includes applying a trained classification model to one or more of (a) a question discourse tree and (b) an answer discourse tree; and receiving a score from the classification model.

[0014] In an example, constructing a discourse tree includes accessing a sentence that includes segments. At least one segment includes a verb and words, and each word includes the role of the word within the segment. Each segment is a basic discourse unit. Constructing the discourse tree further includes generating a discourse tree that represents the rhetorical interrelationships between the segments. The discourse tree includes nodes, and each non-terminal node represents a rhetorical interrelationship between two segments, and each terminal node among the nodes of the discourse tree is associated with one of the segments.

[0015] In an example, the method further includes determining, from the question discourse tree, a question communication discourse tree that includes a question root node. A communication discourse tree is a discourse tree that includes communication actions. The generating further includes determining, from a fictional discourse tree, an answer communication discourse tree. The answer communication discourse tree includes an answer root node. The generating includes merging the communication discourse trees by identifying that the question root node and the answer root node are the same. The generating includes calculating a degree of complementarity between the question communication discourse tree and the answer communication discourse tree by applying a prediction model to the merged communication discourse tree. The generating includes outputting a final answer corresponding to the fictional discourse tree in response to determining that the degree of complementarity is higher than a threshold.

[0016] In an example, a discourse tree represents rhetorical interrelationships between text segments. The discourse tree includes nodes. Each non-terminal node that represents a rhetorical interrelationship between two segments among the segments and each terminal node of the discourse tree is associated with one of the segments. Constructing a communication discourse tree includes matching each segment that has a verb with a verb signature. The matching includes accessing the verb signature. The verb signature includes the verb of the segment and a sequence of thematic roles. The thematic role describes the relationship between the verb and the associated words. The matching further includes, for each verb signature among the verb signatures, determining the thematic role of the corresponding signature that matches the role of the word within the segment. The matching further includes selecting the particular verb signature from the verb signatures based on the particular verb signature including the largest number of matches. The matching further includes associating the particular verb signature with the segment.

[0017] In an example, a method includes constructing a question discourse tree for a question that includes question entities. The method further includes constructing an answer discourse tree for an initial answer that includes answer entities. The method further includes establishing a mapping between a first question entity of the question entities and an answer entity of the answer entities, the mapping establishing a correlation between the answer entity and the first question entity. The method further includes, in response to determining that a second question entity among the question entities is not resolved by any of the answer entities, generating a fictional discourse tree by combining an additional discourse tree corresponding to an additional answer with the answer discourse tree. The method further includes determining a question communication discourse tree from the question discourse tree. The method further includes determining an answer communication discourse tree from the fictional discourse tree. The method further includes calculating a degree of complementarity between the question communication discourse tree and the answer communication discourse tree by applying a prediction model, the question communication discourse tree, and the answer communication discourse tree. The method further includes, in response to determining that the degree of complementarity is higher than a threshold, outputting a final answer corresponding to the fictional discourse tree.

[0018] The above method can be implemented as a system that includes one or more processing devices and / or a non-transitory computer-readable medium on which program instructions can be stored, the program instructions being executable by one or more processors to perform the operations described with respect to the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Exemplary aspects of the present invention are described in detail below with reference to the following drawings.

[0020] Figure 1 An exemplary rhetorical classification environment according to one aspect is shown.

[0021] Figure 2 An example of a discourse tree according to one aspect is depicted.

[0022] Figure 3 A further example of a discourse tree according to one aspect is depicted.

[0023] Figure 4 An illustrative pattern according to one aspect is depicted.

[0024] Figure 5 A node-link representation of a hierarchical binary tree according to one aspect is depicted.

[0025] Figure 6 Depicts according to one aspect Figure 5 Exemplary indented text encoding of the representation in

[0026] Figure 7 An exemplary DT for an example request regarding property tax according to one aspect is depicted.

[0027] Figure 8 Depicts an exemplary response to the question represented in Figure 7 ​

[0028] Figure 9 Illustrates a discourse tree for an official answer according to one aspect.

[0029] Figure 10 Illustrates a discourse tree for an original answer according to one aspect.

[0030] Figure 11 Illustrates a communication discourse tree for a claim of a first agent according to one aspect.

[0031] Figure 12 Illustrates a communication discourse tree for a claim of a second agent.

[0032] Figure 13 Illustrates a communication discourse tree for a claim of a third agent according to one aspect.

[0033] Figure 14 Illustrates parsing thickets according to one aspect.

[0034] Figure 15 Illustrates an exemplary process for constructing a communication discourse tree according to one aspect.

[0035] Figure 16 Illustrates an exemplary process for constructing a fictional discourse tree according to one aspect.

[0036] Figure 17 Depicts an example discourse tree of questions, answers, and two fictional discourse trees according to one aspect.

[0037] Figure 18 Depicts a simplified diagram of a distributed system for implementing one of these aspects.

[0038] Figure 19 Is a simplified block diagram of components of a system environment according to one aspect, through which services provided by components of a system according to one aspect can be supplied as cloud services.

[0039] Figure 20 Illustrates an exemplary computer system in which various aspects of the present invention can be implemented. Detailed Description

[0040] As discussed above, existing systems for autonomous agents have deficiencies. For example, such systems cannot answer complex, multi-sentence, or convergent queries. These systems may also rely on ontologies that are difficult and expensive to build or on the interrelationships between different concepts in a domain. Additionally, some existing solutions employ knowledge graph-based methods, which may limit expressiveness and coverage.

[0041] In contrast, aspects described herein can answer complex user queries by employing domain - independent discourse analysis. Aspects described herein use fictional discourse trees to validate and in some cases complete the rhetorical links between questions and answers, thereby improving question - answering recall for complex, multi - sentence, convergent questions. A fictional discourse tree is a discourse tree that represents a combination of an initial answer to a question and additional supplementary answers. In this way, the fictional discourse tree represents a complete answer to a user query.

[0042] For an answer to be relevant to a given question, the entities in the answer should cover the entities in the question. An "entity" has an independent and distinct existence. Examples include objects, locations, and people. Entities can also be subjects or topics, such as "electric vehicle", "brake", or "France".

[0043] However, in some cases, one or more entities in the question are not addressed in the initial answer. For example, a user may ask about the "engine" of his car, but the initial answer does not discuss the "engine" at all. In other cases, alternatively, more specific entities may appear in the answer, but their connection to the question entities is not immediately obvious. Continuing with the previous example, the initial answer may contain the entities "spark plug" or "transmission" without connecting these entities to the "engine". In other cases, some answer entities may not be explicitly mentioned in the question but are assumed in the question. To provide a complete answer, the missing entities should be considered and explained where appropriate.

[0044] To fill this gap, certain aspects of the present disclosure use fictional discourse trees. For example, an autonomous agent application can identify answers that provide links between entities that exist in the question but are missing in the answer. The application combines relevant segments of the fictional answer discourse tree into a fictional discourse tree that represents a complete answer to the question. Thus, the fictional discourse tree provides rhetorical links that can be used to validate the answer and in some cases augment the answer. The application can then present the complete answer to the user device.

[0045] By creating fictional discourse trees augmented with tree segments obtained from documents mined on - demand from different sources, the disclosed solution eliminates the need for an ontology. For example, in the automotive domain, an ontology might represent the inter - relationships between brakes and wheels or transmissions and engines. In contrast, aspects obtain a canonical discourse representation of an answer that is independent of the thought structure of a given author.

[0046] In another example, the disclosed solution can use a communicative discourse tree to verify the rhetorical consistency between two parts of text. For example, a rhetorical consistency application working with an autonomous agent application can verify the rhetorical consistency between a question and an answer, between a question and a part of an answer, or between an initial answer and an answer represented by a fictional discourse tree, e.g., style.

[0047] A "communicative discourse tree" or "CDT" includes a discourse tree supplemented with communicative actions. Communicative actions are cooperative actions taken by individuals based on mutual negotiation and argumentation, or actions in other ways as known in the art. By incorporating tags that identify communicative actions, learning of communicative discourse trees can be performed on a feature set that is richer grammatically than just rhetorical relations and elementary discourse units (EDUs). Using such a feature set, additional techniques such as classification can be used to determine the level of rhetorical consistency between a question and an answer or a request-response pair, thereby enabling an improved automated agent. In doing so, a computing system implements an autonomous agent capable of intelligently answering questions and other messages.

[0048] Certain definitions

[0049] As used herein, "rhetorical structure theory" is a field of study and learning that provides a theoretical basis for analyzing the coherence of discourse.

[0050] As used herein, a "discourse tree" or "DT" refers to a structure that represents the rhetorical relations of sentences for a part of a sentence.

[0051] As used herein, "rhetorical relation", "rhetorical interrelation", or "coherence relation" or "discourse relation" refers to how two segments of discourse are logically connected to each other. Examples of rhetorical relations include elaboration, contrast, and attribution.

[0052] As used herein, a "sentence fragment" or "fragment" is a part of a sentence that can be separated from the rest of the sentence. Fragments are the basic units of discourse. For example, for the sentence "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down the plane", the two fragments are "Dutch accident investigators say that evidence points to pro-Russian rebels" and "as being responsible for shooting down the plane". Fragments can, but need not, contain a verb.

[0053] As used herein, a "signature" or "frame" refers to the properties of the verb in a fragment. Each signature can include one or more thematic roles. For example, for the fragment "Dutch accident investigators say that evidence points to pro-Russian rebels", the verb is "say" and the signature for this particular use of the verb "say" can be "agent verb topic", where "investigators" is the agent and "evidence" is the topic.

[0054] As used herein, a "thematic role" is a component of a signature that is used to describe the role of one or more words. Continuing the previous example, "agent" and "topic" are thematic roles.

[0055] As used herein, "nuclearity" refers to which text segment, fragment, or span is more crucial for the author's purpose. The nucleus is the more crucial span, while the satellite is the less crucial span.

[0056] As used herein, "coherency" refers to linking two rhetorical relations together.

[0057] As used herein, a "communicative verb" is a verb that indicates communication. For example, the verb "deny" is a communicative verb.

[0058] As used herein, "communicative action" describes an action performed by one or more agents and the principal of the agent(s).

[0059] Figure 1 An exemplary rhetorical classification environment is shown in accordance with one aspect. Figure 1 Autonomous agent 101, input question 130, output answer 150, data network 104, and server 160 are depicted. Autonomous agent 101 may include one or more of the following: autonomous agent application 102, database 115, rhetorical consistency application 112, rhetorical consistency classifier 120, or training data 125. Server 160 may be a public or private Internet server, such as a public database of user questions and answers. Examples of functions provided by server 160 include a search engine and a database. Data network 104 may be any public or private network, wired or wireless network, wide area network, local area network, or the Internet. Autonomous agent application 102 and rhetorical consistency application 112 may be executed within distributed system 1800, as further discussed with respect to Figure 18 discussed further below.

[0060] Autonomous agent application 102 may access input question 130 and generate output answer 150. Input question 130 may be a single question or a stream of questions such as in a chat. As shown, input question 130 is "What is an advantage of an electric car?". Autonomous agent application 102 may receive input question 130 and analyze input question 130, for example, by creating one or more fictional discourse trees 110.

[0061] For example, autonomous agent application 102 creates a question discourse tree based on input question 130 and creates an answer discourse tree based on an initial answer obtained from a resource such as a database. In some cases, autonomous agent application 102 may obtain a set of candidate answers and select the best match as the initial answer. By analyzing and comparing the entities in the question discourse tree and the entities in the answer discourse tree, autonomous agent application 102 determines one or more entities in the question that are not addressed in the answer.

[0062] Continuing with this example, the autonomous agent application 102 accesses additional resources such as a text corpus and determines one or more additional answer discourse trees from the text. The autonomous agent application 102 determines that one of the additional answer discourse trees establishes a rhetorical link between the missing question entity and another entity in the question or answer, thereby designating a particular additional answer discourse tree as a fictional discourse tree. Then, the autonomous agent application 102 can form a complete answer from the text represented by the initial answer discourse tree and the fictional discourse tree. The autonomous agent application 102 provides the answer as the output answer 150. For example, the autonomous agent application 102 outputs the text “There is no need for gasoline”.

[0063] In some cases, the rhetorical consistency application 112 can work in conjunction with the autonomous agent application 102 to promote improved rhetorical consistency between the input question 130 and the output answer 150. For example, the rhetorical consistency application 112 can verify the degree of rhetorical consistency between one or more discourse trees or portions thereof by using one or more communication discourse trees 114. By using communication discourse trees, the rhetorical consistency and communication actions between the question and the answer can be accurately modeled. For example, the rhetorical consistency application 112 can verify that the input question and the output answer maintain rhetorical consistency, thereby ensuring not only a complete responsive answer but also an answer that is consistent in style with the question.

[0064] For example, the rhetorical consistency application 112 can create a question communication discourse tree representing the input question 130 and additional communication discourse trees for each candidate answer. The rhetorical consistency application 112 determines the most suitable answer from the candidate answers. Different methods can be used. In one aspect, the rhetorical consistency application 112 can create candidate answer communication discourse trees for each candidate answer and compare the question communication discourse tree with each candidate discourse tree. The rhetorical consistency application 112 can then identify the best match between the question communication discourse tree and the candidate answer communication discourse tree.

[0065] In another example, the rhetorical consistency application 112 creates question-answer pairs for each candidate answer that include the question 130 and the candidate answer. The rhetorical consistency application 112 provides the question-answer pairs to a prediction model such as the rhetorical consistency classifier 120. The rhetorical consistency application 112 uses the trained rhetorical consistency classifier 120 to determine whether the question-answer pair has a matching degree above a threshold, e.g., indicating whether the answer addresses the question. If not, then the rhetorical consistency application 112 continues to analyze additional pairs that include the question and different answers until a suitable answer is found. In some cases, the rhetorical consistency application 112 can train the rhetorical consistency classifier 120 with the training data 125.

[0066] Rhetorical Structure Theory and Discourse Trees

[0067] Linguistics is the scientific study of language. For example, linguistics can include the structure of sentences (syntax), such as subject - verb - object; the meaning of sentences (semantics), such as "The dog bites the man" and "The man bites the dog"; and what the speaker does in a conversation, that is, the language analysis outside the sentence or discourse analysis.

[0068] The theoretical basis for discourse - the Rhetorical Structure Theory (RST) - can be attributed to Mann, William and Thompson, Sandra, "Rhetorical structure theory: A Theory of Text organization", Text - Interdisciplinary Journal for the Study of Discourse, 8(3):243–281, 1988. Similar to how the syntax and semantics of programming language theory help implement modern software compilers, RST helps implement the analysis of discourse. More specifically, RST places structural blocks at at least two levels, such as a first level of nuclearity and rhetorical relations, and a second level of structures or patterns. A discourse parser or other computer software can parse text into a discourse tree.

[0069] Rhetorical Structure Theory (RST) models the logical organization of a text, which is the structure adopted by the author and depends on the relationships between parts of the text. RST simulates text coherence by forming a hierarchical connection structure of the text via a discourse tree. Rhetorical relations are divided into coordinate and subordinate categories; these relations hold across two or more text segments, thus achieving coherence. These text segments are called elementary discourse units (EDUs). Clauses in a sentence and sentences in a text are logically connected by the author. The meaning of a given sentence is related to the meanings of the sentences before and after it. This logical relationship between clauses is called the coherence structure of the text. RST is one of the most popular discourse theories, which is based on a tree-like discourse structure - the discourse tree (DT). The leaves of the DT correspond to EDUs, i.e., consecutive atomic text segments. Adjacent EDUs are connected by coherence relations (e.g., attribution, sequence), thus forming higher-level discourse units. These units are then also constrained by such relation links. EDUs linked by a relation are then distinguished based on their relative importance: the nucleus is the core part of the relation, while the satellite is the peripheral part. As discussed, both topic and rhetorical consistency are analyzed to determine accurate request-response pairs. When a speaker answers a question, such as a phrase or a sentence, the speaker's answer should address the topic of the question. In cases where a question is implicitly posed via the seed text of a message, an appropriate answer that not only maintains the topic but also matches the general cognitive state of the seed is expected.

[0070] Rhetorical relation

[0071] As discussed, the aspects described herein use fictional discourse trees. Rhetorical relations can be described in different ways. For example, Mann and Thompson described 23 possible relations. See William C. Mann, William & Thompson, Sandra (1987) (“Mann and Thompson”). Maite “Rhetorical structure theory: A Theory of Text organization,” Text - Interdisciplinary Journal for the Study of Discourse, 8(3):243–281, 1988. Other numbers of relations are also possible.

[0072]

[0073]

[0074] Some empirical studies assume that most texts are constructed using a core-satellite relationship. See Mann and Thompson 1988. However, other relationships do not carry an explicit choice of core. Examples of such relationships are shown below.

[0075] Relationship Name Section Other Section Comparison An Alternative Another Alternative Combination (Unconstrained) (Unconstrained) List Item Next Item Sequence Item Next Item

[0076] Figure 2 Depicts an example of a discourse tree according to one aspect. Figure 2 Includes discourse tree 200. The discourse tree includes text segments 201, text segment 202, text segment 203, relationship 210, and relationship 228. Figure 2 The numbers in correspond to three text segments. Figure 3 Corresponds to the following example text with three text segments numbered 1, 2, and 3:

[0077] 1. Honolulu, Hawaii will be site of the 2017 Conference on Hawaiian History

[0078] 2. It is expected that 200 historians from the U.S. and Asia will attend

[0079] 3. The conference will be concerned with how the Polynesians sailed to Hawaii

[0080] For example, relationship 210 or elaboration describes the interrelationship between text segment 201 and text segment 202. Relationship 228 depicts the interrelationship between text segment 203 and 204, namely elaboration. As depicted, text segments 202 and 203 further elaborate on text segment 201. In the example above, given to inform the reader of the goal of the conference, text segment 1 is the core. Text segments 2 and 3 provide more details about the conference. In Figure 2 , the horizontal numbers (e.g., 1 - 3, 1, 2, 3) cover segments of the text (which may consist of further segments); the vertical lines indicate one or more cores; and the curves represent rhetorical relationships (elaboration) and the direction of the arrows is from the satellite to the core. If a text segment is used only as a satellite and not as a core, deleting the satellite will still leave a coherent text. If from Figure 2If the core is removed, then text segments 2 and 3 will be difficult to understand.

[0081] Figure 2 Depicts another example of a discourse tree according to one aspect. Figure 3 Includes components 301 and 302, text segments 305 - 307, relationship 310 and relationship 328. Relationship 310 depicts the mutual relationship 310 - enabling between components 306 and 305 and 307 and 305. Figure 3 Relates to the following text segments:

[0082] 1. The new Tech Report abstracts are now in the journal area of the library near the abridged dictionary. (Now the new technical report abstracts are in the periodical area of the library near the abridged dictionary).

[0083] 2. Please sign your name by any means that you would be interested in seeing. (Please sign your name in any way that you would like to see).

[0084] 3. Last day for sign - ups is 31 May. (The last day for signing up is May 31st).

[0085] As can be seen, relationship 328 depicts the mutual relationship (which is enabling) between entities 307 and 306. Figure 3 Illustrates that although the core can be nested, there is only one most core text segment.

[0086] Constructing a discourse tree

[0087] Discourse trees can be generated using different methods. A simple example of the bottom - up method for constructing a DT is:

[0088] (1) Divide the discourse text into units in the following way:

[0089] (a) The unit size can vary depending on the goal of the analysis

[0090] (b) Typically, the units are clauses

[0091] (2) Examine each unit and its neighbors. Do they maintain a relationship?

[0092] (3) If so, mark the relationship.

[0093] (4) If not, the unit may be at the boundary of a higher-level relationship. Look at the relationships that hold between larger units (segments).

[0094] (5) Continue until all units in the text have been considered.

[0095] Mann and Thompson also describe a second level of building block structure, called pattern applications. In RST, rhetorical relations are not mapped directly onto text; they are assembled onto structures called pattern applications, and these structures are in turn assembled onto text. Pattern applications are derived from simpler structures called patterns (e.g. Figure 4 ). Each pattern indicates how to decompose a particular unit of text into other smaller units of text. A rhetorical structure tree, or DT, is a hierarchical system of pattern applications. Pattern applications link multiple consecutive text segments and create complex text segments, which in turn can be linked by higher-level pattern applications. RST asserts that the structure of each coherent discourse can be described by a single rhetorical structure tree whose top pattern creates a segment that covers the entire discourse.

[0096] Figure 4 An illustrative mode according to one aspect is depicted. Figure 4 The joint mode is shown to be a list of items consisting of cores and no satellites. Figure 4 Patterns 401-406 are depicted. Pattern 401 depicts a contextual relationship between text segments 410 and 428. Pattern 402 depicts a sequence relationship between text segments 420 and 421 and a sequence relationship between text segments 421 and 422. Pattern 403 depicts a contrast relationship between text segments 430 and 431. Pattern 404 depicts a joint interrelationship between text segments 440 and 441. Pattern 405 depicts a motivational interrelationship between 450 and 451 and an enabling interrelationship between 452 and 451. Pattern 406 depicts a joint interrelationship between text segments 460 and 462. Figure 4 An example of a joint pattern for the following three text segments is shown in FIG:

[0097] 1. Skies will be partly sunny in the New York metropolitan area today.

[0098] 2. It will be more humid, with temperatures in the middle 80's.

[0099] 3.Tonight will be mostly cloudy, with the low temperature between 65 and 70.

[0100] Although Figures 2 - 4 Some graphical representations of discourse trees are depicted, but other representations are possible.

[0101] Figure 5 Depicted is a node-link representation of a hierarchical binary tree according to one aspect. Figure 6 As can be seen in , the leaves of DT correspond to consecutive non-overlapping text segments called elementary discourse units (EDUs). Adjacent EDUs are connected by relations (e.g., elaboration, attribution, ...) and form larger discourse units, which are also connected by relations. "Discourse analysis in RST involves two subtasks: discourse segmentation is the task of identifying EDUs, and discourse parsing is the task of linking discourse units into labeled trees."

[0102] Figure 5 The text segments are depicted as leaves or terminal nodes on the tree, each numbered in the order in which they appear in the full text, e.g. Figure 6 shown. Figure 5 Tree 500 is included. Tree 500 includes, for example, nodes 501-507. Nodes indicate mutual relationships. Nodes are either non-terminal nodes (such as node 501), or terminal nodes (such as nodes 502-507). As can be seen, nodes 503 and 504 are related by joint mutual relationships. Nodes 502, 505, 506, and 508 are cores. Dashed lines indicate that branches or text segments are satellites. Relationships are nodes in gray boxes. Figure 6 Describes the Figure 5 Example indented text encoding for representation in . Figure 6 Includes text more suited to computer programming 600. "N" is the core, and "S" is the satellite.

[0103] Example of a discourse parser

[0104] Automatic discourse segmentation can be performed in different ways. For example, given a sentence, a segmentation model identifies the boundaries of composite elementary discourse units by predicting whether a boundary should be inserted before each specific symbol (token) in the sentence. For example, a framework considers each symbol in the sentence sequentially and independently. In this framework, the segmentation model scans the sentence symbol by symbol and uses a binary classifier (such as a support vector machine or logistic regression) to predict whether it is appropriate to insert a boundary before the symbol being examined. In another example, the task is a sequential tagging problem. Once the text is segmented into elementary discourse units, sentence-level discourse parsing can be performed to construct a discourse tree. Machine learning techniques can be used.

[0105] In one aspect of the present invention, two Rhetorical Structure Theory (RST) discourse parsers are used: CoreNLPProcessor which relies on constituency grammar and FastNLPProcessor which uses dependency grammar. They are described in Mihai Surdeanu, Thomas Hicks and Marco A. Valenzuela-Escarcega, "Two Practical Rhetorical Structure Theory Parsers", Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies: Software Demonstrations (NAACL HLT) 2015.

[0106] In addition, the above two discourse parsers, namely, CoreNLPProcessor and FastNLPProcessor, use natural language processing (NLP) for syntactic parsing. For example, Stanford CoreNLP gives the basic forms of words, their part-of-speech, whether they are names of companies, people, etc.; normalizes dates, times, and numerical quantities; marks the structure of sentences based on phrases and syntactic dependencies; and indicates which noun phrases refer to the same entity. In fact, RST is still a theory that may work in many discourse cases but may not work in some cases. There are many variables, including but not limited to, what EDUs are in the coherent text, i.e., what discourse segmenter is used, what list of relations is used, what relations are chosen for the EDUs, the corpus of documents used for training and testing, and even what parser is used. Thus, for example, in the "Two Practical Rhetorical Structure Theory Parsers" by Surdeanu et al. cited above, a dedicated metric must be used to test a specific corpus to determine which parser gives better performance. Therefore, unlike computer language parsers that give predictable results, discourse parsers (and segmenters) may give unpredictable results depending on the training and / or testing text corpus. Thus, discourse trees are a mix of a predictable domain (e.g., compilers) and an unpredictable domain (e.g., like chemistry where experiments need to be conducted to determine which combinations will give you the desired results).

[0107] To objectively determine the quality of discourse analysis, a series of metrics are being used, such as the Precision / Recall / F1 metric from Daniel Marcu, “The Theory and Practice of Discourse Parsing and Summarization”, MIT Press, November 2000, ISBN: 9780262123722. Precision or positive predictive value is the proportion of relevant instances among the retrieved instances, while recall (also known as sensitivity) is the proportion of relevant instances that have been retrieved within the total number of relevant instances. Thus, both precision and recall are based on the understanding and measurement of relevance. Suppose a computer program for identifying dogs in photos identifies 8 dogs in a picture that contains 12 dogs and some cats. Among the eight dogs identified, five are actually dogs (true positives), while the rest are cats (false positives). The precision of the program is 5 / 8, while its recall is 5 / 12. When a search engine returns 30 pages, and only 20 of them are relevant, and fails to return 40 additional relevant pages, its precision is 20 / 30 = 2 / 3, and its recall is 20 / 60 = 1 / 3. Thus, in this case, precision is “how useful the search results are”, and recall is “how complete the results are”. The F1 score (also known as F-score or F-measure) is a measure of the accuracy of a test. It takes into account both the precision and recall of the test to calculate the score: F1 = 2 × ((precision × recall) / (precision + recall)) and is the harmonic mean of precision and recall. The F1 score reaches its best value (perfect precision and recall) when it is 1, and its worst value when it is 0.

[0108] Autonomous agents or chatbots

[0109] A conversation between Human A and Human B is a form of discourse. For example, there are such as Messenger, For applications such as SMS, in addition to the more traditional email and voice conversations, conversations between A and B can typically also be via messages. A chatbot (which can also be called an intelligent robot or virtual assistant, etc.) is an "intelligent" machine that, for example, replaces human B and mimics the conversation between two humans to various degrees. The example ultimate goal is that human A cannot tell whether B is a human or a machine (the Turing test developed by Alan Turing in 1950). Discourse analysis, artificial intelligence including machine learning, and natural language processing have made great progress in the long-term goal of passing the Turing test. Of course, as computers become more and more capable of searching and processing large data repositories and performing complex analysis on data including predictive analysis, the long-term goal is to make chatbots human-like and combined with computers.

[0110] For example, users can interact with an intelligent robot platform through session interaction. This interaction, also known as a session user interface (UI), is a conversation between the end user and the chatbot, just like a conversation between two humans. It can be as simple as the end user saying "Hello" to the chatbot, and then the chatbot responding with "Hi" and asking the user how it can help, or it can be a transaction interaction in a bank chatbot, such as transferring funds from one account to another, or an information interaction in an HR chatbot, such as checking remaining vacation days, or an FAQ inquiry in a retail chatbot, such as how to handle returns. The combination of natural language processing (NLP) and machine learning (ML) algorithms with other methods can be used to classify the end user's intent. A high-level intent is the goal that the end user wants to achieve (for example, obtaining an account balance, making a purchase). Intent is essentially a mapping of what the customer inputs to the work unit that the backend should execute. Therefore, based on the phrases spoken by the user in the chatbot, map them to specific and discrete use cases or work units. For example, checking the balance, transferring funds, and tracking expenses are all "use cases" that the chatbot should support and be able to infer which work unit should be triggered from the free-text entries typed by the end user in natural language.

[0111] The basic principle for an AI chatbot to respond like a human is that the human brain can formulate and understand requests and then give a much better response to human requests than a machine. Therefore, if mimicking human B, the request / response of the chatbot should be significantly improved. So, the first part of the question is how the human brain formulates and understands requests? To mimic, models are used. RST and DT allow this to be done in a formal and repeatable way.

[0112] At a high level, there are generally two types of requests: (1) requests to perform an action; (2) requests for information, e.g., questions. The first type has a response in which a work unit is created. The second type has a response such as a good answer to the question. For example, in some aspects, the answer can take the form of the AI constructing the answer from its extensive (one or more) knowledge bases or by searching the Internet or intranet or other publicly / privately available data sources to match the best existing answer.

[0113] Analysis of request and response pairs

[0114] Figure 7 Depicts an exemplary DT for an example request regarding property tax according to one aspect. The node labels are relationships and the arrow lines point to satellites. The core is solid. Figure 7 Depicts the following text.

[0115] Request: “My husbands’ grandmother gave him his grandfather’s truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund. I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?”

[0116] Response: "The property tax is assessed on property that you own. Just because you chose to not register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax. If you apply late, there will be penalties on top of the normal taxes and fees. You don't need to register it at the same time, but you absolutely need to title it within the period of time stipulated in state law."

[0117] Analysis of the above text reveals that "My husbands’ grandmother gave him his grandfather’s truck" is elaborated upon by "She signed the title over but due to my husband", which is in turn elaborated upon by "having unpaid fines on his license, he was not able to get the truck put in his name", which is further elaborated upon by "I wanted to put in my name", "and paid the property tax", and "and got insurance for the truck".

[0118] “My husband's grandmother gave him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck.” It is elaborated as follows:

[0119] “I didn't have the money”, which is further elaborated by “to do so”. The latter is contrasted with “By the time”. “By the time” is elaborated by “it came to sending off the title” “and getting the tag”.

[0120] “My husband's grandmother gave him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so” is contrasted with the following:

[0121] “Now, due to circumstances”, which is elaborated by “I am not going to be able to afford the truck”. The latter is elaborated as follows:

[0122] “I went to the insurance place”

[0123] “and was refused a refund”.

[0124] “My husbands’ grandmother gave him his grandfather’s truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund” through the following elaboration:

[0125] “I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?”

[0126] “I am just wondering” is attributed to:

[0127] “that” and “is it possible to get the property tax refunded?” are the same unit, and the condition of the latter is “since I am not going to have a tag on this truck”.

[0128] As can be seen, the main subject of the topic is "Property tax on a car". The problem includes a contradiction: on the one hand, all property should be taxed, and on the other hand, the ownership is somewhat incomplete. A good response must address both the subject of the problem and clarify the inconsistency. To do this, the responder makes a stronger statement about the necessity of taxing any property one owns regardless of its registration status. This example is a member of the positive training set in our Yahoo! Answers evaluation domain. The main subject of the topic is "Property tax on a car". The problem includes a contradiction: on the one hand, all property should be taxed, and on the other hand, the ownership is somewhat incomplete. A good answer / response must address both the subject of the problem and clarify the inconsistency. The reader can observe that since the problem includes a contrasting rhetorical relationship, the answer must match it with a similar relationship to be convincing. Otherwise, the answer will seem incomplete even to those who are not domain experts.

[0129] Figure 8 depicts an exemplary response to the Figure 7 problem represented in. The central core is "the property tax is assessed on property", which is elaborated by "that you own". "The property tax is assessed on property that you own" is also the core, which is elaborated by "Just because you chose to not register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax."

[0130] Core: "The property tax is assessed on property that you own. Just because you chose to not register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax." is elaborated as follows: "there will be penalties on top of the normal taxes and fees", with the condition: "If you apply late", which is further elaborated through the comparison of "but you absolutely need to title it within the period of time stipulated in state law" and "You don't need to register it at the same time".

[0131] By comparing Figure 7 the DT of Figure 8 with the DT of Figure 8 , the matching degree of the response ( Figure 7 ) to the request ( Figure 7 ) can be determined. In some aspects of the present invention, the above framework is at least partially used to determine the DT for the request / response and the rhetorical consistency between the DTs.

[0132] Figure 9 Illustrates a discourse tree for an official answer according to one aspect. As Figure 9As depicted, the official answer or mission statement states: "The Investigative Committee of the Russian Federation is the main federal investigating authority which operates as Russia's Anti-corruption agency and has statutory responsibility for inspecting the police forces, combating police corruption and police misconduct, is responsible for conducting investigations into local authorities and federal governmental bodies".

[0133] Figure 10 Illustrated is a discourse tree for the original answer according to one aspect. As Figure 10 As depicted, another perhaps more honest answer states: "The Investigative Committee of the Russian Federation is supposed to fight corruption. However, top-rank officers of the Investigative Committee of the Russian Federation are charged with creation of a criminal community. Not only that, but their involvement in large bribes, money laundering, obstruction of justice, abuse of power, extortion, and racketeering has been reported. Due to the activities of these officers, dozens of high-profile cases including the ones against criminal lords had been ultimately ruined."

[0134] The choice of answer depends on the context. The rhetorical structure allows for a distinction between "official", "politically correct", template-based answers and "actual", "raw", "reports from the field" or "controversial" answers. (See Figure 9 and Figure 10 ). Sometimes, the question itself can give a hint about the category of the expected answer. If the question is phrased as a factual or definitional question without a second meaning, then the first category of answers is appropriate. Otherwise, if the question has the meaning of "tell me what is actually there", then the second category is appropriate. Generally speaking, after extracting the rhetorical structure from the question, it is easier to select a suitable answer with a similar, matching or complementary rhetorical structure.

[0135] The official answer is based on elaboration and conjunction, which is neutral with respect to the controversies that the text may contain. (See Figure 9 ). At the same time, the raw answer includes a contrast relationship. This relationship is extracted between the phrases of what is expected of the agent and what has been found that the agent has done.

[0136] Classification of request-response pairs

[0137] The rhetorical consistency application 112 can determine whether a given answer or response (such as an answer obtained from the answer database 105 or a public database) is responsive to a given question or request. More specifically, the rhetorical consistency application 112 analyzes whether a request-response pair is correct or incorrect by determining one or both of (i) relevance or (ii) rhetorical consistency between the request and the response. Rhetorical consistency can be analyzed without considering relevance, and relevance can be processed orthogonally.

[0138] The rhetorical consistency application 112 can use different methods to determine the similarity between question-answer pairs. For example, the rhetorical consistency application 112 can determine the degree of similarity between a single question and a single answer. Alternatively, the rhetorical consistency application 112 can determine a measure of the similarity between a first pair including a question and an answer and a second pair including a question and an answer.

[0139] For example, the rhetorical consistency application 112 uses a rhetorical consistency classifier 120 trained to predict a match or non-match answer. The rhetorical consistency application 112 can process two pairs at a time, e.g., <q1,a1> and <q2,a2>. The rhetorical consistency application 112 compares q1 with q2 and a1 with a2, resulting in a combined similarity score. Such a comparison allows determining whether an unknown question / answer pair contains the correct answer by evaluating the distance to another question / answer pair with a known label. In particular, an unlabeled pair <q2,a2> can be processed such that instead of "guessing" correctness based on words or structures shared by q2 and a2, q2 and a2 can both be compared to their corresponding components q1 and a2 of the labeled pair <q2,a2> based on such words or structures. Since the method is for domain-independent classification of answers, only the structural cohesion between the question and the answer can be utilized, and not the "meaning" of the answer.

[0140] In one aspect, the rhetorical consistency application 112 uses training data 125 to train the rhetorical consistency classifier 120. In this way, the rhetorical consistency classifier 120 is trained to determine the similarity between question and answer pairs. This is a classification problem. The training data 125 can include a positive training set and a negative training set. The training data 125 includes request-response pairs that match in the positive data set and request-response pairs that are arbitrary or of low relevance or appropriateness in the negative data set. For the positive data set, various domains with different acceptance criteria indicating whether the answer or response is suitable for the question are selected.

[0141] Each training data set includes a set of training pairs. Each training set includes a question discourse tree representing the question, an answer discourse tree representing the answer, and the expected degree of complementarity between the question and the answer. By using iterative processing, the rhetorical consistency application 112 provides training pairs to the rhetorical consistency classifier 120 and receives the degree of complementarity from the model. The rhetorical consistency application 112 calculates a loss function by determining the difference between the determined degree of complementarity for a specific training pair and the expected degree of complementarity. Based on the loss function, the rhetorical consistency application 112 adjusts the internal parameters of the classification model to minimize the loss function.

[0142] The acceptance criteria may vary depending on the application. For example, for community question answering, automated question answering, automated and manual customer support systems, social network communication, and writing by individuals such as consumers about their experiences with products (such as reviews and complaints), the acceptance criteria may be low. In scientific texts, professional news, health and legal documents in the form of FAQs, professional social networks (such as "stackoverflow"), the RR acceptance criteria may be high.

[0143] Communicative Discourse Tree (CDT)

[0144] The rhetorical consistency application 112 can create, analyze, and compare communicative discourse trees. The communicative discourse tree is designed to combine rhetorical information with the speech act structure. The CDT contains arcs that are labeled with expressions for communicative actions. By combining communicative actions, the CDT enables modeling of RST relations and communicative actions. The CDT is a simplification of the parsing jungle. See Galitsky, B, Ilvovsky, D., and Kuznetsov S.O. Rhetoric Map of an Answer to Compound Queries Knowledge Trail Inc. ACL 2015, 681–686. (“Galitsky 2015”). The parsing jungle is a combination of parse trees of sentences with discourse-level interrelations between the words and parts of those sentences in a graph. By merging the labels that identify speech acts, the learning of communicative discourse trees can occur on a richer feature set than just the rhetorical relations and grammar of elementary discourse units (EDUs).

[0145] In the example, a dispute among three parties regarding the cause of the crash of a commercial airliner (i.e., Malaysia Airlines Flight 17) is analyzed. An RST representation of the conveyed arguments is established. In this example, three conflicting agents, namely, the Dutch investigators, the Investigative Committee of the Russian Federation, and the self-proclaimed People's Republic of Donetsk, exchanged their opinions on the matter. This example illustrates a controversial conflict where each party tries its best to blame the other. To sound more persuasive, each party not only presents its own statements but also responds in a way that rejects the other party's statements. To achieve this goal, each party attempts to match the style and discourse of the other party's statements.

[0146] Figure 11 Illustrated is a communicative discourse tree for the statement of the first agent according to one aspect. Figure 11Depicts the AC discourse tree 100, which represents the following text: "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down plane. The report indicates where the missile was fired from and identifies who was in control of the territory and pins the downing of MH17 on the pro-Russian rebels".

[0147] As can be seen Figure 11 The non-terminal nodes of the CDT are rhetorical relations, and the terminal nodes are the basic discourse units (phrases, sentence fragments) that are the subjects of these relations. Some arcs of the CDT are labeled with expressions for communicative actions, including the actor agent and the subject of these actions (what is being communicated). For example, the core node for the elaboration relation (on the left) is labeled with say(Dutch,evidence), and the satellite is labeled with responsible(rebels,shooting down). These labels are not intended to express that the subjects of the EDUs are evidence and shooting down, but rather to enable this CDT to match other CDTs in order to find similarities between them. In this case, linking these communicative actions only through rhetorical relations without providing information about the communicative discourse would be too limited to represent the structure of what is being communicated and how it is being communicated. The RR requirement for having the same or coordinated rhetorical relations is too weak, so there is a need for consistency in the CDT labels of the arcs on top of the nodes to be matched.

[0148] The straight edges of the diagram are syntactic relations, and the curved arcs are discourse relations such as anaphora, same entity, sub-entity, rhetorical relation, and communicative act. The diagram contains much richer information than just the combination of parse trees for individual sentences. In addition to CDT, parse jungles can also be generalized at the levels of words, relations, phrases, and sentences. Speech acts are logical predicates that represent the agents involved in individual speech acts and their subjects. As proposed by frameworks such as VerbNet, the arguments of logical predicates are formed according to their respective semantic roles. See Karin Kipper, Anna Korhonen, Neville Ryant, Martha Palmer, A Large-scale Classification of English Verbs, Language Resources and Evaluation Journal, 42(1), pp. 21-40, Springer Netherland, 2008 and / or Karin Kipper Schuler, Anna Korhonen, Susan W. Brown, VerbNet overview, extensions, mappings and apps, Tutorial, NAACL-HLT 2009, Boulder, Colorado.

[0149] Figure 12 Illustrated is a communicative discourse tree for a statement for a second agent according to one aspect. Figure 12 Depicted is communicative discourse tree 1200, which represents the following text: "The Investigative Committee of the Russian Federation believes that the plane was hit by a missile, which was not produced in Russia. The committee cites an investigation that established the type of the missile".

[0150] Figure 13 Illustrated is a communicative discourse tree for a statement for a third agent according to one aspect. Figure 13Depicts the communicative discourse tree 1300, which represents the following text: "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired. It became possible only after three months after the tragedy to say if rebels controlled one or another town".

[0151] As can be seen from the communicative discourse trees 1100 - 1300, the responses are not arbitrary. The responses talk about the same entities as the original text. For example, the communicative discourse trees 1200 and 1300 are related to the communicative discourse tree 1100. The responses support the disagreement of the estimations and viewpoints regarding these entities and the actions on these entities.

[0152] More specifically, the replies of the involved agents need to reflect the communicative discourse of the first sub-message. As a simple observation, since the first agent uses attribution to convey its statement, other agents must either follow this set of statements, or provide their own attribution or attack the validity of the supporter's attribution, or both. In order to capture various features for how the communicative structure of the seed message needs to be preserved in consecutive messages, the pairs of the corresponding CDT can be learned.

[0153] To verify the consistency of the request - response, usually only the discourse relations or speech acts (communicative actions) are not enough. As can be seen from Figures 11 - 13 the examples depicted, the discourse structure and the types of interactions between agents are useful. However, the domain of the interactions (e.g., military conflict or politics) or the subjects of these interactions (i.e., entities) do not need to be analyzed.

[0154] Represents rhetorical relations and communicative actions

[0155] To calculate the similarity between abstract structures, two methods are frequently used: (1) represent these structures in a numerical space and express the similarity as a number, which is a statistical learning method, or (2) use a structural representation without a numerical space, such as trees and graphs, and express the similarity as the maximum common substructure. Expressing the similarity as the maximum common substructure is called generalization.

[0156] Learning and communication actions contribute to the expression and understanding of arguments. A computational verb lexicon helps support the entities that perform the actions and provides rule-based forms to express their meanings. Verbs express the semantics of the described event and the relationship information between the participants in that event, and project the syntactic structures that encode that information. Verbs, especially communication action verbs, can vary greatly and may exhibit rich semantic behavior. In response, verb classification helps learning systems cope with this complexity by organizing verbs into groups that share core semantic properties.

[0157] VerbNet is such a lexicon that identifies the semantic role and syntactic pattern properties of verbs in each class and makes explicit the connection between the syntactic patterns and the underlying semantic relations that can be inferred for all members of that class. See Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer, Language Resources and Evaluation, Vol. 42, No. 1 (March 2008), 21. Each syntactic frame or verb signature of a class has a corresponding semantic representation that details the semantic relations between the event participants during the course of the event.

[0158] For example, the verb "amuse" is part of a cluster of similar verbs that have a similar argument (semantic role) structure, such as "amaze", "anger", "arouse", "disturb", and "irritate" (surprise, anger, arouse, disturb, and irritate). The roles of the arguments for these communication actions are as follows: Experience (the experiencer, usually a living entity), Stimulus, and Result. Each verb can have meaning categories distinguished by syntactic features for how the verb appears in a sentence or frame. For example, the frame for "amuse" is as follows, using the following key noun phrases (NP), nouns (N), communication actions (V), verb phrases (VP), adverbs (ADV):

[0159] NP V NP. Example: "The teacher amused the children". Syntax: Stimulus V Experiencer. Clause: amuse(Stimulus, E, Emotion, Experiencer), cause(Stimulus, E), emotional_state(result(E), Emotion, Experiencer).

[0160] NP V ADV-Middle. Example: "Small children amuse quickly". Syntax: Experiencer V ADV. Clause: amuse(Experiencer,Prop):-,property(Experiencer,Prop),adv(Prop).

[0161] NP V NP-PRO-ARB. Example "The teacher amused". Syntax: Stimulus V.amuse(Stimulus,E,Emotion,Experiencer):.cause(Stimulus,E),emotional_state(result(E),Emotion,Experiencer).

[0162] NP.cause V NP. Example "The teacher's dolls amused the children". Syntax: Stimulus<+genitive>('s)V Experiencer.amuse(Stimulus,E,Emotion,Experiencer):.cause(Stimulus,E),emotional_state(during(E),Emotion,Experiencer).

[0163] NP V NP ADJ. Example "This performance bored me totally". Syntax: Stimulus VExperiencer Result.amuse(Stimulus,E,Emotion,Experiencer).cause(Stimulus,E),emotional_state(result(E),Emotion,Experiencer),Pred(result(E),Experiencer).

[0164] Communication actions can be characterized as multiple clusters, such as:

[0165] Verbs with a predicate complement (appoint, characterize, dub, declare, conjecture, masquerade, orphan, captain, consider, classify), perception verbs (see, sight, peer).

[0166] Mental state verbs (amuse, admire, marvel, appeal), desire verbs (want, long).

[0167] Judgment verbs (judgment), evaluation verbs (assess, estimate), search verbs (hunt, search, stalk, investigate, rummage, ferret), social interaction verbs (correspond, marry, meet, battle), communication verbs (transfer (message), inquire, interrogate, tell, manner (speaking), talk, chat, say, complain, advise, confess, lecture, overstate, promise). Avoidance verbs (avoid), measurement verbs (register, cost, fit, price, bill), aspect verbs (begin, complete, continue, stop, establish, sustain).

[0168] The aspects described herein have advantages over statistical learning models. Compared to statistical solutions, aspects using a classification system can provide a verb or verb-like structure that is determined to result in a target feature (such as rhetorical consistency). For example, statistical machine learning models express similarity as a number, which can make it difficult to interpret.

[0169] Represents a request-response pair

[0170] The request-response pair is based on the promotion of classification-based operations. In an example, the request-response pair can be represented as a parse jungle. A parse jungle is a representation of the parse trees of two or more sentences, with discourse-level relationships between parts of the sentences and words in one graph. See Galitsky (2015). The topic similarity between a question and an answer can be expressed as a common subgraph of the parse jungle. The more common graph nodes, the higher the similarity.

[0171] Figure 14 Illustrates a parse jungle according to one aspect. Figure 14 Depicts a parse jungle 1400, which includes a parse tree 1401 and a parse tree 1402 for the corresponding response.

[0172] The parse tree 1401 represents the problem: "I just had a baby and it looks more like the husband I had my baby with. However, it does not look like me at all and I am scared that he was cheating on me with another lady and I had her kid. This child is the best thing that has ever happened to me and I cannot imagine giving my baby to the real mom".

[0173] The response 1402 represents the response "Marital therapists advise on dealing with a child being born from an affair as follows. One option is for the husband to avoid contact but just have the basic legal and financial commitments. Another option is to have the wife fully involved and have the baby fully integrated into the family just like a child from a previous marriage".

[0174] Figure 14 Represents a greedy method for representing linguistic information about paragraphs of text. The straight edges of the graph are syntactic relations, and the curved arcs are discourse relations such as anaphora, same entity, sub-entity, rhetorical relations, and communication acts. Solid arcs are used for same entity / sub-entity / anaphora relations, and dashed arcs are used for rhetorical relations and communication acts. The oval labels in the straight edges represent syntactic relations. Lemmas are written in the boxes of the nodes, and lemma forms are written to the right of the nodes.

[0175] Parse Jungle 1400 includes information that is much richer than just the combination of parse trees for individual sentences. Navigating along the edges of syntactic relations and the arcs of discourse relations in the graph allows the given parse jungle to be transformed into a semantically equivalent form to match other parse jungles, thus performing the text similarity assessment task. To form a complete formal representation of a paragraph, as many links as possible are expressed. Each discourse arc gives rise to a pair of jungle phrases that may be potential matches.

[0176] The thematic similarity between a seed (request) and a response is expressed as a common subgraph of the parse jungle. They are visualized as connected clouds. The greater the number of common graph nodes, the higher the similarity. For rhetorical coherence, the common subgraph does not have to be as large as it is in the given text. However, the rhetorical relations and communicative actions between the seed and the response are relevant and require correspondence.

[0177] Generalization of Communicative Actions

[0178] The similarity between two communicative actions A1 and A2 is defined as an abstract verb that has features common between A1 and A2. Defining the similarity of two verbs as an abstract class verb structure supports inductive learning tasks such as rhetorical coherence assessment. In an example, the similarity between the following two common verbs (agree and disagree) can be generalized as follows: agree^disagree = verb(Interlocutor, Proposed_action, Speaker), where Interlocutor is the person who proposes Proposed_action to Speaker and to whom Speaker conveys Speaker's response. Proposed_action is the action that Speaker will perform if Speaker accepts or rejects a request or a suggestion, and Speaker is the person to whom a specific action has been proposed and who responds to the request or suggestion made.

[0179] In another example, the similarity between the verbs agree and explain is expressed as follows: agree^explain = verb(Interlocutor, *, Speaker). The agent of the communicative action is generalized in the context of the communicative action and is not generalized with respect to other "physical" actions. Thus, each aspect generalizes the individual occurrence of a communicative action together with the corresponding agent.

[0180] In addition, the sequence of communicative actions representing a dialogue can be compared with other such sequences similar to a dialogue. In this way, the meaning of individual communicative actions and the dynamic discourse structure of the dialogue (compared with its static structure reflected by rhetorical relations) are represented. Generalization occurs in the representation of composite structures at each level. The lemmas of communicative actions are generalized together with the lemmas, and their semantic roles are generalized together with the corresponding semantic roles.

[0181] The text author uses communicative actions to indicate the structure or conflict of a dialogue. See Searle, J.R. 1969, Speech acts: an essay in the philosophy of language. London: Cambridge University Press. Agents are generalized in the context of these actions and not relative to other "physical" actions. Thus, individual occurrences of communicative actions, together with their agents and their pairs, are generalized into discourse "steps".

[0182] The generalization of communicative actions can also be considered from the perspective of matching verb frames (such as VerbNet). Communicative links reflect the discourse structure associated with the participation (or mention) of more than one agent in the text. These links form a sequence connecting the words (multiple words or verbs implicitly indicating the communicative intention of a person) for communicative actions.

[0183] A communicative action includes an actor, one or more agents on whom the action is being taken, and a phrase describing the characteristics of the action. A communicative action can be described as a function of the following form: verb(agent, subject, cause) (where the verb represents a certain type of interaction between the involved agents (e.g., explain, confirm, remind, disagree, reject, etc.), the subject refers to the information being transmitted or the object being described, and the cause refers to the motivation or explanation for the subject).

[0184] A scenario (marked as a directed graph) is a subgraph of the parsing jungle G = (V, A), where V = {action1, action2…action n} is a finite set of vertices corresponding to communicative actions, and A is a finite set of labeled arcs (ordered pairs of vertices), which are classified as follows:

[0185] Each arc action i , action j ∈A sequence corresponds to two actions v j = s i ) or different agents referring to the same agent (e.g., s i , agi , s i , c i and v j , ag j , s j , c j The time priority of. Each arc action i , action j ∈ A cause corresponds to the attack interrelationship between the action action i and action j which indicates that the reason of action i conflicts with the subject or reason of action j .

[0186] The sub - graphs of the parsing jungle associated with the interaction scenarios with the agent have some obvious characteristics. For example, (1) all vertices are sorted by time such that all vertices (except the initial and terminating vertices) have an incoming arc and an outgoing arc, (2) for A sequence arcs, at most one incoming arc and only one outgoing arc are allowed, and (3) for A cause arcs, a given vertex can have many outgoing arcs and many incoming arcs. The vertices involved can be associated with different agents or the same agent (i.e., when he is self - contradictory). To calculate the similarity between the parsing jungle and its communication actions, the inductive sub - graphs, sub - graphs of the same configuration with similar arc labels, and the strict correspondence of vertices are analyzed.

[0187] By analyzing the arcs of the communication actions of the parsing jungle, the following similarities exist: (1) One communication action whose subject comes from T1 is compared with another communication action whose subject comes from T2 (without using the communication action arc), and (2) One pair of communication actions whose subjects come from T1 is compared with another pair of communication actions from T2 (using the communication action arc).

[0188] Generalize two different communication actions based on their attributes. See (Galitsky et al. 2013). As in regarding Figure 14As can be seen in the example under discussion, one communicative action cheating(husband,wife,another lady) from T1 can be compared with a second communicative action avoid(husband,contact(husband,anotherlady)) from T2. Generalization results in communicative_action(husband,*), which introduces a constraint on A in the form that if a given agent ( = husband) is mentioned as the agent of a CA in Q, then he (she) should also be the agent of a (possibly different) CA in A. Two communicative actions can always be generalized, but not so for their agents: if the result of their generalization is empty, then the generalization result of the communicative actions with those agents is also empty.

[0189] Generalization of RST relations

[0190] Some relations between discourse trees can be generalized. For example, arcs representing the same type of relations (presentational relations such as contrast; topical relations such as condition; and multi-nuclear relations such as list) can be generalized. The nucleus or the situation represented by the nucleus is indicated by "N". The satellite or the situation presented by the satellite is indicated by "S". "W" indicates the author. "R" indicates the reader (audience). The situations are propositions, completed actions or ongoing actions, as well as communicative actions and states (including beliefs, desires, approvals, interpretations, reconciliations and others). The generalization of two RST relations with the above parameters is expressed as: rst1(N1,S1,W1,R1)^rst2(N2,S2,W2,R2)=(rst1^rst2)(N1^N2,S1^S2,W1^W2,R1^R2).

[0191] The text in N1, S1, W1, R1 is generalized as a phrase. For example, rst1^rst2 can be generalized as follows: (1) If relation_type(rst1)!= relation_type(rst2), it is generalized to empty. (2) Otherwise, the signature of the rhetorical relation is generalized to the sentence: sentence(N1, S1, W1, R1)^sentence(N2, S2, W2, R2). See Iruskieta, Mikel, Iria da Cunha, and Maite Taboada. A qualitative comparison method for rhetorical structures: identifying different discourse structures in multilingual corpora. Lang Resources & Evaluation. June 2015, Vol. 49, No. 2.

[0192] For example, the meaning of rst-background^rst-enablement = (S increases R's ability to understand elements in N) ^ (R understands that S increases R's ability to perform actions in N) = increase-VB the-DT ability-NN of-IN R-NN to-IN.

[0193] Since the relation rst-background^rst-enablement is different, the RST relation part is empty. Then, the generalization is the expression that serves as the linguistic definition of the corresponding RST relation. For example, for each word or placeholder for words such as agents, if the word is the same in each input phrase, the word (along with its POS) is retained, and if the word is different between these phrases, the word is removed. The resulting expression can be interpreted as the common meaning between the definitions of two different RST relations formally obtained.

[0194] Figure 14The two arcs between the question and answer depicted show generalization instances based on the RST relation "RST-contrast". For example, "I just had a baby" is an RST contrast with "it does not look like me" and is related to "husband to avoid contact", which is an RST-contrast with "have the basic legal and financial commitments". As can be seen, the answer does not have to be similar to the verb phrase of the question, but the rhetorical structures of the question and answer are similar. Not all phrases in the answer have to match the phrases in the question. For example, the non-matching phrases have certain rhetorical relations with the phrases in the answer that are related to the phrases in the question.

[0195] Establishing a communication discourse tree

[0196] Figure 15 Illustrated is an exemplary process for constructing a communication discourse tree according to one aspect. The rhetorical consistency application 112 can implement the process 1500. As discussed, the communication discourse tree achieves improved search engine results.

[0197] At block 1501, the process 1500 involves accessing a sentence that includes segments. At least one segment includes a verb and words, and each word includes the role of the word within the segment, and each segment is a basic discourse unit. For example, the rhetorical consistency application 112 accesses a sentence such as Figure 13 the one described "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired".

[0198] Continuing with the example, the rhetorical consistency application 112 determines that the sentence includes a number of segments. For example, the first segment is "rebels..deny". The second segment is "that they controlled the territory". The third segment is "from which the missile was allegedly fired". Each segment contains a verb, for example, "deny" in the first segment and "controlled" in the second segment. However, a segment is not required to contain a verb.

[0199] At block 1502, process 1500 involves generating a discourse tree representing the rhetorical relationships between sentence fragments. The discourse tree includes nodes, where each non-terminal node represents a rhetorical relationship between two sentence fragments in the sentence fragment, and each terminal node in the nodes of the discourse tree is associated with one of the sentence fragments in the sentence fragment.

[0200] Continuing with the example, rhetorical consistency application 112 generates a discourse tree as Figure 13 shown. For example, the third fragment “from which the missile was allegedly fired” elaborates on “that they controlled the territory”. The second and third fragments together are related to the attribution of what happened, i.e., the attack could not have been by the insurgents because they did not control the territory.

[0201] At block 1503, process 1500 involves accessing multiple verb signatures. For example, rhetorical consistency application 112 accesses a list of verbs such as in VerbNet. Each verb matches or is related to a verb in the fragment. For example, for the first fragment, the verb is “deny”. Accordingly, rhetorical consistency application 112 accesses a list of verb signatures related to the verb deny.

[0202] As discussed, each verb signature includes the verb of the fragment and one or more thematic roles. For example, the signature includes one or more of a noun phrase (NP), noun (N), communicative action (V), verb phrase (VP), or adverb (ADV). The thematic role describes the interrelationship between the verb and the related words. For example, “the teacher amused the children” has a different signature from “small children amuse quickly”. For the first fragment, with the verb “deny”, rhetorical consistency application 112 accesses a list of verb signatures or frames of verbs that match “deny”. The list is “NP V NP to be NP”, “NP V that S”, and “NP V NP”.

[0203] Each verb signature contains thematic roles. The thematic role refers to the role of the verb in the sentence fragment. Rhetorical consistency application 112 determines the thematic roles in each verb signature. Examples of thematic roles include agent, actor, asset, attribute, beneficiary, cause, location, destination, source, destination, source, location, experiencer, scope, instrument, material and product, material, product, patient, predicate, recipient, stimulus, stem, time, or topic.

[0204] At block 1504, process 1500 involves determining, for each verb signature in the verb signature, the number of thematic roles of that corresponding signature that match the roles of words within the segment. For the first segment, rhetorical consistency application 112 determines that the verb "deny" has only three roles: "agent", "verb", and "stem".

[0205] At block 1505, process 1500 involves selecting a particular verb signature from the verb signatures based on the particular verb signature having the maximum number of matches. For example, again referring to Figure 13 , "deny" in the first segment "the rebels deny...that they control the territory" matches the verb signature deny "NP V NP", and "control" matches control(rebel,territory). The verb signatures are nested, resulting in the nested signature "deny(rebel,control(rebel,territory))".

[0206] Fictitious discourse tree

[0207] Certain aspects described herein use a fictitious discourse tree to improve question-and-answer (Q / A) recall for complex, multi-sentence, convergent problems. By augmenting the discourse tree of an answer with tree fragments obtained from an ontology, the aspects obtain a canonical discourse representation of the answer that is independent of the thought structure of a given author.

[0208] As discussed, a discourse tree (DT) can represent how text is organized (specifically using text paragraphs). Discourse-level analysis can be used in many natural language processing tasks where learning the language structure may be essential. The DT outlines the interrelationships between entities. There are different ways to introduce entities and associated attributes within the text. Not all of the rhetorical relationships that exist between these entities appear in the DT of a given paragraph. For example, in another text body, the rhetorical relationship between two entities may be outside the DT of a given paragraph.

[0209] Therefore, in order for questions and answers to be fully relevant to each other, a more complete discourse tree of the answer is desired. Such a discourse tree will include all the rhetorical relationships between the entities involved. To achieve this, at least a partially resolved answer's initial or best-match discourse tree for the question is augmented with certain rhetorical relationships identified as missing in the answer.

[0210] These missing rhetorical relations can be obtained from text corpora or from external databases such as the Internet. Thus, rather than relying on an ontology that may have definitions of entities missing in the candidate answers, rhetorical relations between these entities are mined online in various ways. This process avoids the need to construct an ontology and can be implemented on top of a regular search engine.

[0211] More specifically, the baseline requirement for an answer A to be relevant to a question Q is that the entities (En) of A cover the entities of Q: Naturally, some answer entities E-A are not explicitly mentioned in Q but are needed to provide a complete answer A. Thus, the logical flow of Q through A can be analyzed. However, since it may be challenging to establish relations between En, an approximate logical flow of Q and A can be modeled, which can be represented in domain-independent terms EnDT-Q~EnDT-A and later verified and / or augmented.

[0212] There may be different deficiencies in the initial answers used to answer a question. For example, in one case, some entities E are not explicitly mentioned in Q but are assumed. In another case, when some entities in A used to answer Q do not appear in A but more specific or more general entities appear in A. To determine that some more specific entities do address the question from Q, some external or additional sources (referred to herein as fictional EnDT-A) can be used to establish these interrelations. Such sources contain information about the internal interrelations between En that are omitted in Q and / or A but are assumed to be known to the peers. Thus, for a computer-implemented system, the following knowledge is expected at the discourse level:

[0213] EnDT-Q~EnDT-A + fictional EnDT-A.

[0214] For the purpose of discussion, the following example is introduced. The first example is the question “What is an advantage of electric car? (What are the advantages of an electric vehicle)” and the corresponding answer is “No need to for gas (No need to refuel)”. To establish that a specific answer fits a specific question, the general entity advantage and the regular noun entity car are used. More specifically, the explicit entities in A{need, gas} are linked. A fragment of a possible fictional EnDT-A is shown: […No need… - Elaborate - Advantage]…[gas – Enablement - engine]…[engine – Enablement - car]. Only evidence of these rhetorical links needs to exist.

[0215] In a second example, a fictional discourse tree is used to improve search. A user asks an autonomous agent for information about "a faulty brake switch can affect the automatic transmission Munro". Existing search engines may identify certain keywords that are not found in the given search results. However, it is possible to indicate how these keywords are related to the search results by finding documents in which these unrecognized keywords are rhetorically connected to the keywords that appear in the query. This feature naturally improves the relevance of the answer on the one hand and provides the user with interpretability regarding how their keywords are resolved in the answer.

[0216] The fictional discourse tree enables the search engine to interpret the missing keywords in the search results. In a default search, munro is missing. However, by attempting to rhetorically connect munro to the entities in the question, the autonomous agent application 102 learns that Munro is the inventor of the automatic transmission. Figure 16 An example process that can be used by the autonomous agent application 102 is depicted.

[0217] Figure 16 An exemplary process 1600 for constructing a fictional discourse tree according to one aspect is illustrated. The autonomous agent application 102 can implement the process 1600. The process 1600 explains how the discourse tree (DT) facilitates an improved match between the question and the answer. For example, to verify that an answer A (or an initial answer) is suitable for a given question Q, the autonomous agent application 102 verifies whether DT-A and DT-Q are consistent and then optionally augments DT-A with fragments of other DTs to ensure that all entities in Q are resolved in the augmented DT-A. For the purpose of discussion, regarding Figure 17 The process 1600 is discussed.

[0218] Figure 17 An example discourse tree for a question, an answer, and two fictional discourse trees according to one aspect is depicted. Figure 17 A question discourse tree 1710, an answer discourse tree 1720, an additional answer discourse tree 1730, an additional answer discourse tree 1740, links 1750 - 1754, and nodes 1170 - 1174 are depicted.

[0219] The problem discourse tree 1710 is formed from the following problem: "[When driving the cruise control][the engine will turn off][when I want to accelerate,][although the check engine light was off.][I have turned on the ignition][and listen for the engine pump running][to see][if it is building up vacuum.][Could there be a problem with the brake sensor under the dash?][Looks like there could be a little play in the plug.]". For the purposes of discussion, the square brackets [] indicate the basic discourse units.

[0220] Return Figure 16 , at block 1601, process 1600 involves constructing a problem discourse tree (DT-Q) from the problem. The problem can include segments (or basic discourse units). Each word in the words indicates the role of the individual words within the segment.

[0221] The autonomous agent application 102 generates the problem discourse tree. The problem discourse tree represents the rhetorical relationships between segments and contains nodes. Each non-terminal node represents the rhetorical interrelationship between two segments. Each terminal node in the nodes of the discourse tree is associated with one of the segments.

[0222] The problem discourse tree can include one or more problem entities. The autonomous agent application 102 can identify these entities. As Figure 17 shown, the problem contains entities such as "CRUISE CONTROL", "CHECK ENGINE LIGHT", "ENGINE PUMP", "VACUUM", "BRAKE SENSOR". As will be discussed later, the answer resolves some of these entities, while some entities are not resolved.

[0223] At block 1602, process 1600 accesses an initial answer from a text corpus. The initial answer can be obtained from existing documents such as files or databases. The initial answer can also be obtained via a search of a local or external system. For example, autonomous agent application 102 can obtain an additional set of answers by submitting a search query. The search query can be derived from the question, e.g., formulated by one or more keywords identified in the question. Autonomous agent application 102 can determine a relevance score for each of the multiple answers. The answer score or ranking indicates the degree of match between the question and the corresponding answer. Autonomous agent application 102 selects the answers with scores higher than a threshold as the initial answer.

[0224] Continuing with this example, autonomous agent application 102 accesses the following initial answer: "[A faulty brake switch can affect the cruise control.][If it is,][there should be a code][stored in the engine control module.][Since it is not an emissions fault,][the check engine light will not illuminate.][First of all, watch the tachometer][to see][if engine speed increases 200rpm][when this happens.][If it does,][the torque converter is unlocking transmission.]"

[0225] At block 1603, process 1600 involves constructing an answer discourse tree including answer entities from the initial answer. Autonomous agent application 102 forms a discourse tree for the initial answer. At block 1603, autonomous agent application 102 performs steps similar to those described with respect to block 1601.

[0226] Continuing with this example, autonomous agent application 102 generates answer discourse tree 1720, which represents the text that is the initial answer to the question. Answer discourse tree 1720 includes some of the question entities, specifically, "CRUISE CONTROL", "CHECK ENGINE LIGHT", and "TORQUE CONVERTER".

[0227] At block 1604, process 1600 involves using a computing device to determine that a score indicating the relevance of an answer entity to a question entity is below a threshold. Entities in question Q may or may not be resolved by one or more entities in the answer, i.e., no entity in answer A resolves a particular entity. The unresolved entities in the question are indicated by E0.

[0228] Different methods can be used to determine how relevant a particular answer entity is to a particular question entity. For example, in one aspect, the autonomous agent application 102 can determine whether an exact text match of the question entity appears in the answer entity. In other cases, the autonomous agent application 102 can determine a score indicating the proportion of keywords indicating the question entity that appear in the answer entity.

[0229] There can be different reasons why an entity is not resolved. As shown, E0-Q includes {ENGINE PUMP, BRAKESENSOR, and VACUUM}. For example, any answer A is not completely relevant to question Q because the answer omits some of the entities in E0. Alternatively, conversely, the answer uses different entities. E0-Q may be ignored in answer A. To verify the latter possibility, background knowledge is used to find an entity E that links to both E0-Q and E-A. img 。

[0230] For example, it may not be clear how E-A = TORQUE CONVERTER is connected to Q. To verify this connection, a text snippet about the torque converter is obtained from Wikipedia and DT-A is constructed. img1 (1730). Aspects can determine that the torque converter is connected to the engine via the stated rhetorical relationship.

[0231] Therefore, E-A = Torque Converter is indeed relevant to the question, as shown by the vertical blue arc. Such a determination can be made without constructing an offline ontology of linked entities and learning the relationships between entities. Alternatively, discourse-level context is used to confirm that A includes relevant entities.

[0232] Continuing with this example, the autonomous agent application 102 determines that other entities in the question discourse tree 1710, specifically those that reference "ENGINE PUMP", "VACUUM", "BRAKE SENSOR", and "TORQUE CONVERTER", are not resolved in the answer discourse tree 1720. Therefore, these correspondences between E-Q and E-A are illustrated by links 1750 and 1751, respectively.

[0233] At block 1605, process 1600 involves creating additional discourse trees based on a text corpus. The additional discourse trees are created from the text. In some cases, autonomous agent application 102 selects appropriate text from one or more texts according to a scoring mechanism.

[0234] For example, autonomous agent application 102 can access an additional answer set. Autonomous agent application 102 can generate a set of additional fictional discourse trees. Each additional discourse tree corresponds to a corresponding additional answer. Autonomous agent application 102 calculates a score for each additional discourse tree, which indicates the number of question entities including mappings of one or more answer entities into the corresponding additional discourse tree. Autonomous agent application 102 selects the additional discourse tree with the highest score from the set of additional discourse trees. Once DT-A is augmented with fictional DT-A img the search relevance is measured as the reciprocal of the unresolved E0-Q.

[0235] For example, autonomous agent application 102 obtains a set of candidates A s . Then, for each candidate A in A s autonomous agent application 102 performs the following operations: c

[0236] (a) Construct DT-A c ;

[0237] (b) Establish the mapping E-Q -> E-A c ;

[0238] (c) Identify E0-Q;

[0239] (d) Form a query from E0-Q and E0-A c (entities not in E0-Q);

[0240] (e) Obtain search results for query d) from B and construct fictional DTs-A c ; and

[0241] (f) Calculate the remaining score |E0|.

[0242] Autonomous agent application 102 can then select the A with the best score.

[0243] In some cases, machine learning methods can be used to classify <EDT-Q, EDT-A> pairs as correct or incorrect. An example training set includes good (affirmative) Q / A pairs and bad (negative) Q / A pairs. Therefore, the DT kernel learning method (SVM TK, Joty and Moschitti 2014, Galitsky 2017) is selected, which applies SVM learning to the set of all sub-DTs of the DT of the Q / A pair. The tree kernel family of methods is not very sensitive to errors (syntax and rhetoric) in parsing because the incorrect subtrees are mostly random and will not be likely to be common between different elements of the training set.

[0244] A DT can be represented by a vector V of integer counts of each subtree type (disregarding its ancestors):

[0245] V(T) = (number of subtrees of type 1,...).

[0246] Given two tree segments DT1 and DT2, the tree kernel function K(EDT1, EDT2) ≤ V(EDT1) and

[0247]

[0248] where n1 ∈ N1, n2 ∈ N2, where N1 and N2 are the sets of all nodes in DT1 and DT2 respectively.

[0249] I1(n) is an indicator function.

[0250] I1(n) = {1, a subtree of type i appears and has a root at the node; 0 otherwise}

[0251] Continuing with the example, it is not clear how the E-Q "ENGINE PUMP" in the question is resolved in the initial answer. Therefore, the autonomous agent application 102 can identify additional resources that can resolve the ENGINE PUMP entity.

[0252] At block 1606, process 1600 involves determining that the additional discourse tree includes a rhetorical relationship that connects the question entity to the answer entity. Continuing with the example, as Figure 17 shown, node 1770 identifies the entity "VACUUM (vacuum cleaner)", and node 1171 identifies the entity "ENGINE". Therefore, nodes 1172 and 1173, both representing the rhetorical relationship "elaboration", are related to nodes 1170 and 1171. In this way, nodes 1172 and 1773 provide the missing link between "ENGINE" in the question and "VACUUM" in the question that was previously unresolved.

[0253] The autonomous agent application 102 connects the additional discourse tree to the E-A. As shown, DT-Aimg2 It is described that VACUUM and ENGINE are connected. Thus, the combined DT-A includes the true DT-A plus DT-A img1 and DT-A img2 . By adopting background knowledge in a domain-independent manner, both the true and fictional DTs are necessary for justifying the answer.

[0254] At block 1607, process 1600 involves extracting a subtree of an additional discourse tree including a question entity, an answer entity, and a rhetorical relationship, thereby generating a fictional discourse tree. In some cases, the autonomous agent application 102 can extract a subtree or a part of an additional discourse tree that relates the question entity to the answer entity. The subtree includes at least one node. In some cases, the autonomous agent application 102 can integrate the subtree from the fictional discourse tree into the answer discourse tree by connecting the node to the answer entity or another entity.

[0255] At block 1608, process 1600 involves outputting an answer represented by a combination of the answer discourse tree and the fictional discourse tree. The autonomous agent application 102 can combine the subtree identified at block 1607 with the answer discourse tree. The autonomous agent application 102 can then output an answer corresponding to the combined tree.

[0256] Experimental results

[0257] Traditional Q / A datasets for factual and non-factual questions, as well as SemEval and neural Q / A evaluations, are not suitable because the questions are ranked and not complex enough to observe the potential contributions of discourse-level analysis. For evaluation, two convergent Q / A sets are formed:

[0258] 1. The Yahoo! Answer (Webscope 2017) set of question-answer pairs with a wide range of topics. 3300 user questions are selected from a set of 140k, which include three to five sentences. Most of the answers to the questions are quite detailed, so no filtering is applied to the answers based on sentence length.

[0259] 2. A Q / A pair including 9300 automotive problem descriptions and suggestions on how to correct them for automotive repair conversations.

[0260] For each of these sets, we form positive pairs from the actual Q / A pairs and form negative pairs from the Q / A similar-entities form negative pairs: E-A similar-entities has a strong overlap with E-A, but A similar-entities is not truly the correct, comprehensive, and exact answer. Thus, the Q / A is reduced to a classification task measured by precision and recall via associating Q / A pairs as correct pairs of classes.

[0261]

[0262] Evaluation of Q / A Accuracy

[0263] The first two rows in Table 1 show the baseline performance of Q / A and indicate that in complex domains, the transformation from keywords to matching entities brings a performance improvement of more than 12%. The bottom three rows show the Q / A accuracy when applying discourse analysis. Ensuring the rule-based correspondence between DT-A and DT-Q provides a 12% increase relative to the baseline, and using fictional DT further increases it by 10%. Finally, the Q / A correspondence from rule-based to machine learning (SVM TK) gives a performance gain of approximately 7%.

[0264] Supplementing the fictional discourse tree with a rhetorical consistency classifier

[0265] By using the communication discourse tree, the rhetorical consistency application 112 can determine the complementarity between two sentences. For example, the rhetorical consistency application 112 can determine the degree of complementarity between the discourse tree of a question and the discourse tree of an initial answer, between the discourse tree of a question and the discourse tree of an additional or candidate answer, or between the discourse tree of an answer and the discourse tree of an additional answer. In this way, the autonomous agent application 102 ensures that complex questions are solved with answers that are complete in terms of rhetorical consistency or style.

[0266] In the example, the rhetorical consistency application 112 constructs a question communication discourse tree from the question and an answer communication discourse tree from the initial answer. The rhetorical consistency application 112 determines the question communication discourse tree for the question statement. The question discourse tree can include a root node. For example, referring again to Figure 13 and Figure 15 , the example question statement is "are rebels responsible for the downing of the flight". The rhetorical classification application 102 can use the processing 1500 described regarding Figure 15 . The example question has a root node of "elaborate".

[0267] The rhetorical consistency application 112 determines a second communication discourse tree for the answer statement. The answer communication discourse tree can include a root node. Continuing with the above example, the rhetorical consistency application creates a communication discourse tree as Figure 13 shown, which also has a root node labeled "elaborate".

[0268] The rhetorical consistency application 112 associates the communication discourse trees by identifying that the question root node and the answer root node are the same. The rhetorical consistency application 112 determines that the question communication discourse tree and the answer communication discourse tree have the same root node. The resulting associated communication discourse tree is as Figure 17as shown, and can be labeled as "request-response pair".

[0269] The rhetorical consistency application 112 calculates the degree of complementarity between the question exchange discourse tree and the answer exchange discourse tree by applying a prediction model to the merged discourse tree. Different machine learning techniques can be used. In one aspect, the rhetorical consistency application 112 trains and uses a rhetorical consistency classifier 120. For example, the rhetorical consistency application 112 can define positive and negative categories for request-response pairs. The positive category includes rhetorically correct request-response pairs, and the negative category includes relevant but rhetorically external request-response pairs. For each request-response pair, the rhetorical consistency application 112 can construct a CDT by parsing each sentence and obtaining the verb signature of the sentence fragments. The rhetorical consistency application 112 provides the associated pair of exchange discourse trees to the rhetorical consistency classifier 120, which in turn outputs the degree of complementarity.

[0270] The rhetorical consistency application 112 determines that the degree of complementarity is higher than a threshold, and then identifies the question and answer statements as being complementary. The rhetorical consistency application 112 can use the complementarity degree threshold to determine whether the question-answer pair is sufficiently complementary. For example, if the classification score is greater than the threshold, then the rhetorical consistency application 112 can use the answer. Alternatively, the rhetorical consistency application 112 can discard the answer and access the answer database 105 or a common database for another candidate answer, and repeat as needed.

[0271] In another aspect, the rhetorical consistency application 112 applies jungle kernel learning to the representation. Jungle kernel learning can occur instead of the above-mentioned classification-based learning. The rhetorical consistency application 112 constructs a parse jungle pair for the parse tree of the request-response pair. The rhetorical consistency application 112 applies discourse parsing to obtain a pair of discourse trees for the request-response pair. The rhetorical consistency application 112 aligns the basic discourse units of the discourse tree request-response and the parse tree request-response. The rhetorical consistency application 112 merges the basic discourse units of the discourse tree request-response and the parse tree request-response.

[0272] Related Work

[0273] At any point in the discourse, some entities are considered more important than others (appearing in the core part of the DT), and thus are expected to exhibit different properties. In centering theory (Grosz et al., 1995; Poesio et al., 2004), entity importance determines how they are realized in speech, including the pronominal relationships between them. In other discourse theories, entity importance can be defined via topicality (Prince 1978) and cognitive accessibility (Gundel et al., 1993).

[0274] Barzilay and Lapata (2008) automatically abstract text into a set of entity transformation sequences and record distributional, syntactic, and referential information about discourse entities. The authors formulate coherence assessment as a learning task and show that their entity-based representation is well-suited for ranking-based generation and text classification tasks.

[0275] (Nguyen and Joty 2017) proposed a local coherence model based on convolutional neural networks that operate on the distributed representation of entity transitions in a grid representation of the text. The local coherence model can model entity transitions long enough and can incorporate entity-specific features without losing generalization ability. Kuyten et al. (2015) developed a search engine that exploits the discourse structure in documents to overcome the limitations associated with the bag-of-words document representation in information retrieval. The system does not address the rhetorical coordination problem between Q and A, but given Q, the search engine can retrieve relevant A and separate statements from A that describe some rhetorical relationship with the query.

[0276] Answering questions in this research area is a significantly more complex task than factoid QA such as the Stanford QA Database (Rajpurkar et al., 2016) where only one or two entities and their arguments need to be involved. To answer the question of "how to solve the problem", the logical flow connecting the entities in the question needs to be maintained. Since some entities in Q will inevitably be omitted, these entities may need to be recovered from some background knowledge text about these omitted entities and the entities presented in Q. In addition, the logical flow needs to complement the logical flow of Q.

[0277] Domain-specific ontologies, such as those related to mechanical problems of cars, are difficult and costly to build. In this work, an alternative approach via domain-independent discourse-level analysis is proposed. More specifically, the unresolved parts of DT-A are addressed, for example, by finding text fragments in a background knowledge corpus such as Wikipedia. Thus, there is no need to maintain an ontology that must preserve the relationships between the entities involved.

[0278] The fictional DT features of the proposed Q / A system provide a significant improvement in the accuracy of answering complex convergent questions. However, compared to the baseline that focuses on relevance, DT for answer style matching improves Q / A accuracy by more than 10%, and relying on fictional DT can further improve it by 10%.

[0279] Aspects described herein analyze the complementary relationship between DT-A and DT-Q, thereby significantly reducing the learning feature space, making it reasonable to learn from a finite-sized available dataset such as a vehicle repair list.

[0280] Figure 18 A simplified diagram of a distributed system 1800 for implementing one of these aspects is depicted. In the aspect shown, the distributed system 1800 includes one or more client computing devices 1802, 1804, 1806, and 1808 configured to execute and operate client applications such as web browsers, proprietary clients (e.g., Oracle Forms), etc. via one or more networks 1810. A server 1812 can be communicatively coupled to the remote client computing devices 1802, 1804, 1806, and 1808 via the network 1810.

[0281] In aspects, the server 811 can be adapted to run one or more services or software applications provided by one or more components of the system. The services or software applications can include non-virtual and virtual environments. The virtual environments can include environments for virtual events, exhibitions, simulators, classrooms, shopping venues, and enterprises, whether in two-dimensional or three-dimensional (3D) representations, page-based logical environments, or other forms. In some aspects, these services can be provided as web-based services or cloud services or under a software as a service (SaaS) model to users of the client computing devices 1802, 1804, 1806, and / or 1808. Users operating the client computing devices 1802, 1804, 1806, and / or 1808 can in turn utilize one or more client applications to interact with the server 1812 to utilize the services provided by these components.

[0282] In the configuration depicted in the figure, the software components 1818, 1820, and 1822 of the distributed system 1800 are shown as being implemented on the server 1812. In other aspects, one or more components of the distributed system 1800 and / or the services provided by these components can also be implemented by one or more of the client computing devices 1802, 1804, 1806, and / or 1808. Then, users operating the client computing devices can utilize one or more client applications to use the services provided by these components. These components can be implemented in hardware, firmware, software, or a combination thereof. It should be recognized that various different system configurations are possible, which may differ from the distributed system 1800. Thus, the aspect shown in the figure is an example of a distributed system for implementing an aspect system and is not intended to be limiting.

[0283] The client computing devices 1802, 1804, 1806, and / or 1808 can be portable handheld devices (e.g., A cellular phone, a computing tablet, a personal digital assistant (PDA)), or a wearable device (e.g., Google Head-Mounted Display), which runs software such as Microsoft Windows and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 18, Palm OS, etc.), and enables Internet, email, Short Message Service (SMS), or other communication protocols. The client computing device can be a general-purpose personal computer, which, by way of example, includes personal computers and / or laptop computers running various versions of Microsoft Apple and / or Linux operating systems. The client computing device can be a workstation computer running any of a variety of commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively or additionally, the client computing devices 1802, 1804, 1806, and 1808 can be any other electronic device capable of communicating via the network(s) 1810, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a gesture input device) and / or a personal messaging device.

[0284] Although the exemplary distributed system 1800 is shown as having four client computing devices, any number of client computing devices can be supported. Other devices (such as devices with sensors, etc.) can interact with the server 1812.

[0285] One or more networks 1810 in the distributed system 1800 can be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, where the protocols include but are not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internetwork Packet Exchange), AppleTalk, and so on. Merely by way of example, one or more networks 1810 can be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, etc. One or more networks 1810 can be a wide area network and the Internet. It can include virtual networks, including but not limited to virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (e.g., a network operating in accordance with any one of the Institute of Electrical and Electronics Engineers (IEEE) 802.18 protocol suite, and / or any other wireless protocol); and / or any combination of these and / or other networks.

[0286] The server 1812 can be composed of one or more general-purpose computers, dedicated server computers (by way of example, including PC (personal computer) servers, servers, midrange servers, mainframes, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. The server 1812 can include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. A flexible pool of one or more logical storage devices can be virtualized to maintain the virtual storage devices of the server. The server 1812 can use software-defined networking to control the virtual network. In various aspects, the server 1812 can be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 1812 can correspond to a server for performing the processing described above according to aspects of the present disclosure.

[0287] The server 1812 can run an operating system including any of the operating systems discussed above, as well as any commercially available server operating system. The server 1812 can also run any server application and / or middleware application among a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, servers, database servers, and so on. Exemplary database servers include but are not limited to those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

[0288] In some implementations, server 1812 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 802, 804, 806, and 808. As an example, data feeds and / or event updates may include, but are not limited to, feeds, updates, or real-time updates and continuous data streams received from one or more third-party information sources, which may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc. Server 1812 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 1802, 1804, 1806, and 1808.

[0289] Distributed system 1800 may also include one or more databases 1814 and 1816. Databases 1814 and 1816 may reside in various locations. As an example, one or more of databases 1814 and 1816 may reside on a non-transitory storage medium local to (and / or residing within) server 1812. Alternatively, databases 1814 and 1816 may be remote from server 1812 and communicate with server 1812 via a network-based connection or a dedicated connection. In a set of aspects, databases 1814 and 1816 may reside in a storage area network (SAN). Similarly, any necessary files for performing the functions of server 1812 may be stored locally on server 1812 and / or remotely as appropriate. In a set of aspects, databases 1814 and 1816 may include relational databases adapted to store, update, and retrieve data in response to SQL-formatted commands, such as databases provided by Oracle.

[0290] Figure 19 is a simplified block diagram of one or more components of a system environment 1900 according to aspects of the present disclosure, through which services provided by one or more components of an aspect system may be provided as cloud services. In the illustrated aspect, system environment 1900 includes one or more client computing devices 1904, 1906, and 1908 that may be used by a user to interact with a cloud infrastructure system 1902 that provides cloud services. The client computing devices may be configured to operate client applications, such as a web browser, a proprietary client application (e.g., Oracle Forms), or some other application, which may be used by a user of the client computing device to interact with the cloud infrastructure system 1902 to use the services provided by the cloud infrastructure system 1902.

[0291] It should be recognized that the cloud infrastructure system 1902 depicted in the figure may have other components in addition to the depicted components. Additionally, the aspects shown in the figure are only one example of a cloud infrastructure system that can incorporate aspects of the present invention. In some other aspects, the cloud infrastructure system 1902 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration or arrangement.

[0292] The client computing devices 1904, 1906, and 1908 may be devices similar to those described above for 1002, 1004, 1006, and 1008.

[0293] Although the exemplary system environment 1900 is shown having three client computing devices, any number of client computing devices may be supported. Other devices such as devices having sensors, etc., may interact with the cloud infrastructure system 1902.

[0294] (One or more) networks 1910 may facilitate data communication and exchange between the client computing devices 1904, 1906, and 1908 and the cloud infrastructure system 1902. Each network may be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, including those described above for (one or more) networks 1810.

[0295] The cloud infrastructure system 1002 may include one or more computers and / or servers, which may include those computers and / or servers described above for server 1812.

[0296] In certain aspects, the services provided by the cloud infrastructure system may include many services that are available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. The services provided by the cloud infrastructure system may be dynamically scaled to meet the needs of the users of the cloud infrastructure system. The specific instantiation of the services provided by the cloud infrastructure system is referred to herein as a "service instance". Generally, any service that is available to a user via a communication network (such as the Internet) from a cloud service provider's system is referred to as a "cloud service". Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own local servers and systems. For example, the cloud service provider's system may host applications, and users may order and use the applications on demand via a communication network such as the Internet.

[0297] In some examples, services in a computer network cloud infrastructure can include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided by a cloud provider to users, or as otherwise known in the art. For example, a service can include password-protected access over the Internet to a remote storage device on the cloud. As another example, a service can include a hosted relational database and a scripting language middleware engine based on web services for private use by networked developers. As another example, a service can include access to an email software application hosted on a cloud provider's website.

[0298] In certain aspects, the cloud infrastructure system 1902 can include a suite of application, middleware, and database service offerings that are delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud offered by the present assignee.

[0299] Large amounts of data (sometimes referred to as big data) can be hosted and / or manipulated by the infrastructure system at many levels and different scales. Such data may contain data sets that are so large and complex that they are difficult to process using typical database management tools or traditional data processing applications. For example, it may be difficult to store, retrieve, and process petabytes of data using a personal computer or its rack-based counterpart. Using the latest relational database management systems and desktop statistics and visualization packages, it may be difficult to process data of this size. They may require large-scale parallel processing software that runs thousands of server computers and is beyond the structure of common software tools to capture, collate, manage, and process the data within a tolerable elapsed time.

[0300] Analysts and researchers can store and manipulate very large data sets to visualize large amounts of data, detect trends, and / or otherwise interact with the data. Dozens, hundreds, or thousands of processors linked in parallel can operate on such data to present the data or simulate external forces on the data or the things it represents. These data sets may involve structured data (e.g., data organized in a database or otherwise organized according to a structured model) and / or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing). By leveraging the ability of one aspect to relatively quickly focus more (or less) computing resources on a target, the cloud infrastructure system can be better utilized to perform tasks on large data sets according to the needs of enterprises, government agencies, research organizations, private individuals, like-minded individuals or organizations, or other entities.

[0301] In various aspects, the cloud infrastructure system 1002 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by the cloud infrastructure system 1902. The cloud infrastructure system 1002 may provide cloud services via different deployment models. For example, services may be provided according to a public cloud model, in which the cloud infrastructure system 1002 is owned by an organization that sells cloud services (e.g., owned by Oracle), and the services are available to the general public or enterprises in different industries. As another example, services may be provided according to a private cloud model, in which the cloud infrastructure system 1002 operates only for a single organization and may provide services to one or more entities within that organization. Cloud services may also be provided according to a community cloud model, in which the cloud infrastructure system 1002 and the services provided by the cloud infrastructure system 1002 are shared by several organizations in a related community. Cloud services may also be provided according to a hybrid cloud model, which is a combination of two or more different models.

[0302] In some aspects, the services provided by the cloud infrastructure system 1002 may include one or more services provided under a software as a service (SaaS) category, a platform as a service (PaaS) category, an infrastructure as a service (IaaS) category, or other service categories including hybrid services. A customer may order one or more services provided by the cloud infrastructure system 1902 via a subscription order. The cloud infrastructure system 1002 then performs processing to provide the services in the customer's subscription order.

[0303] In some aspects, the services provided by the cloud infrastructure system 1002 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services falling into the SaaS category. For example, the SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing SaaS services. By leveraging the services provided by the SaaS platform, a customer may utilize applications executed on the cloud infrastructure system. The customer may obtain application services without the customer having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0304] In some aspects, the platform services can be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform can be configured to provide cloud services that fall into the PaaS category. Examples of platform services can include, but are not limited to, services that enable an organization (such as Oracle) to integrate existing applications on a shared common architecture and build new applications by taking full advantage of the shared services provided by the platform. The PaaS platform can manage and control the underlying software and infrastructure used to provide the PaaS services. Customers can obtain the PaaS services provided by the cloud infrastructure system without the customers having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0305] By leveraging the services provided by the PaaS platform, customers can adopt programming languages and tools supported by the cloud infrastructure system and also control the deployed services. In some aspects, the platform services provided by the cloud infrastructure system can include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one aspect, the database cloud services can support a shared service deployment model that enables organizations to pool database resources and supply database as a service to customers in the form of a database cloud. In the cloud infrastructure system, the middleware cloud services can provide a platform for customers to develop and deploy various business applications, and the Java cloud services can provide a platform for customers to deploy Java applications.

[0306] A variety of different infrastructure services can be provided by the IaaS platform in the cloud infrastructure system. The infrastructure services facilitate the management and control of the underlying computing resources (such as storage devices, networks, and other basic computing resources) for customers to utilize the services provided by the SaaS platform and the PaaS platform.

[0307] In certain aspects, the cloud infrastructure system 1002 can also include infrastructure resources 1930 for providing resources to customers of the cloud infrastructure system for providing various services. In one aspect, the infrastructure resources 1930 can include a combination of pre-integrated and optimized hardware (such as servers, storage devices, and networking resources) to execute the services provided by the PaaS platform and the SaaS platform.

[0308] In some aspects, the resources in the cloud infrastructure system 1002 can be shared by multiple users and dynamically reallocated as needed. Additionally, resources can be allocated to users in different time zones. For example, the cloud infrastructure system 1002 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours and then enable the same resources to be reallocated to another group of users located in a different time zone, thereby maximizing the utilization of the resources.

[0309] In certain aspects, multiple internal shared services 1932 can be provided that are shared by different components or modules of the cloud infrastructure system 1902 and by the services provided by the cloud infrastructure system 1902. These internal shared services can include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, etc.

[0310] In certain aspects, the cloud infrastructure system 1902 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one aspect, the cloud management function can include the ability to provision, manage, and track customer subscriptions received by the cloud infrastructure system 1902, etc.

[0311] In one aspect, as depicted in the figure, the cloud management function can be provided by one or more modules, such as an order management module 1920, an order orchestration module 1922, an order provisioning module 1924, an order management and monitoring module 1926, and an identity management module 1928. These modules can include one or more computers and / or servers or use one or more computers and / or servers to provide, and these computers and / or servers can be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0312] In an exemplary operation 1934, a customer using a client computing device (such as client computing devices 1904, 1906, or 1908) can interact with the cloud infrastructure system 1902 by requesting one or more services provided by the cloud infrastructure system 1902 and placing a subscription order for one or more services provisioned by the cloud infrastructure system 1902. In certain aspects, the customer can access a cloud user interface (UI) (cloud UI 1919, cloud UI 1914, and / or cloud UI 1916) and place a subscription order via these UIs. The order information received by the cloud infrastructure system 1902 in response to the customer placing an order can include information identifying the customer and the one or more services that the customer wants to subscribe to provided by the cloud infrastructure system 1902.

[0313] After the customer places an order, order information is received via the cloud UIs 1010, 1014, and / or 1014.

[0314] At operation 1936, the order is stored in the order database 1918. The order database 1918 can be one of several databases operated by the cloud infrastructure system 1902 and operating in conjunction with other system components.

[0315] At operation 1938, the order information is forwarded to the order management module 1920. In some cases, the order management module 1920 can be configured to perform billing and accounting functions related to the order, such as verifying the order and booking the order after verification.

[0316] At operation 1940, information about the order is sent to the order orchestration module 1922. The order orchestration module 1922 can utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some cases, the order orchestration module 1922 can use the services of the order provisioning module 1924 to orchestrate the provisioning of resources to support the subscribed services.

[0317] In some aspects, the order orchestration module 1922 enables the management of the business processes associated with each order and the application of business logic to determine whether the order should proceed to provisioning. At operation 1942, upon receipt of a newly subscribed order, the order orchestration module 1922 sends a request to the order provisioning module 1924 to allocate resources and configure those resources required to fulfill the subscribed order. The order provisioning module 1924 enables the allocation of resources for the services ordered by the customer. The order provisioning module 1924 provides an abstraction layer between the cloud services provided by the cloud infrastructure system 1902 and the physical implementation layer for the resources used to provide the requested services. Thus, the order orchestration module 1922 can be isolated from implementation details, such as whether the services and resources are actually provisioned immediately or pre-provisioned and only allocated / assigned upon request.

[0318] At operation 1944, once the services and resources are provisioned, a notification of the provided services can be sent to the customer on the client computing devices 1904, 1906, and / or 1908 via the order provisioning module 1924 of the cloud infrastructure system 1902.

[0319] At operation 1946, the order management and monitoring module 1926 can manage and track the customer's subscribed orders. In some cases, the order management and monitoring module 1926 can be configured to collect usage statistics for the services in the subscribed order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0320] In some aspects, the cloud infrastructure system 1902 can include an identity management module 1928. The identity management module 1928 can be configured to provide identity services, such as access management and authorization services in the cloud infrastructure system 1902. In some aspects, the identity management module 1928 can control information about customers who wish to utilize the services provided by the cloud infrastructure system 1902. Such information can include information that authenticates the identities of these customers and information that describes what actions these customers are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1928 can also include management of descriptive information about each customer and about how and by whom this descriptive information can be accessed and modified.

[0321] Figure 20 An exemplary computer system 2000 is shown in which various aspects of the present invention can be implemented. The system 2000 can be used to implement any of the above computer systems. As shown, the computer system 2000 includes a processing unit 2004 that communicates with a plurality of peripheral subsystems via a bus subsystem 2002. These peripheral subsystems can include a processing acceleration unit 2006, an I / O subsystem 2008, a storage subsystem 2018, and a communication subsystem 2024. The storage subsystem 2018 includes a computer-readable storage medium 2022 and a system memory 2010.

[0322] The bus subsystem 2002 provides a mechanism for enabling the various components and subsystems of the computer system 2000 to communicate with each other as intended. Although the bus subsystem 2002 is schematically shown as a single bus, alternative aspects of the bus subsystem can utilize multiple buses. The bus subsystem 2002 can be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus that use any of various bus architectures. For example, such architectures can include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which can be implemented as Mezzanine buses manufactured to the IEEE P2086.1 standard.

[0323] The processing unit 2004, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 2000. One or more processors may be included in the processing unit 2004. These processors may include single-core or multi-core processors. In some aspects, the processing unit 2004 can be implemented as one or more independent processing units 2032 and / or 2034, where each processing unit includes a single-core or multi-core processor. In other aspects, the processing unit 2004 can also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0324] In various aspects, the processing unit 2004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed may reside in the processing unit(s) 2004 and / or the storage subsystem 2018. Through appropriate programming, the processing unit(s) 2004 can provide the various functions described above. The computer system 2000 may additionally include a processing acceleration unit 2006, which may include a digital signal processor (DSP), a dedicated processor, and so on.

[0325] The I / O subsystem 2008 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, an audio input device having a voice command recognition system, a microphone, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect motion sensor, which enables a user to control and interact with an input device such as a Microsoft Xbox 360 game controller using gestures and voice commands through a natural user interface. User interface input devices may also include eye gesture recognition devices, such as a Google Eye Mobile device that detects eye activity from a user (e.g., "blinking" when taking a photo and / or making a menu selection) and converts the eye gesture into an input to an input device (e.g., Google ) Blink Detector. Additionally, user interface input devices may include voice recognition sensing devices that enable a user to interact with a voice recognition system (e.g., Navigator) through voice commands.

[0326] The user interface input device may also include, but is not limited to, a three-dimensional (3D) mouse, joystick or pointing stick, game panel and graphics tablet, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders and eye tracking devices. In addition, the user interface input device may include, for example, medical imaging input devices such as computed tomography, magnetic resonance imaging, positron emission tomography, medical ultrasound devices. The user interface input device may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, and the like.

[0327] The user interface output device may include a display subsystem, indicator lights, or non-visual displays such as audio output devices, and so on. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as one using a liquid crystal display (LCD) or plasma display, a projection device, a touch screen, and so on. Generally speaking, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2000 to a user or other computer. For example, the user interface output device may include, but is not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, automotive navigation systems, plotters, voice output devices, and modems.

[0328] The computer system 2000 may include a storage subsystem 2018 containing software elements, shown as currently residing in the system memory 2010. The system memory 2010 may store program instructions that are loadable and executable on the processing unit 2004, as well as data generated during the execution of these programs.

[0329] Depending on the configuration and type of the computer system 2000, the system memory 2010 can be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by the processing unit 2004 and / or are currently being operated on and executed by the processing unit 2004. In some implementations, the system memory 2010 can include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS) that contains basic routines that help transfer information between components within the computer system 2000, for example, during startup, can typically be stored in the ROM. By way of example, and not limitation, the system memory 2010 also shows an application program 2012, program data 2014, and an operating system 2016. The application program 2012 can include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc. By way of example, the operating system 2016 can include various versions of Microsoft Apple and / or Linux operating systems, various commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google operating systems, etc.) and / or mobile operating systems such as iOS, Phone, OS, 10OS and OS operating systems.

[0330] The storage subsystem 2018 can also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide some aspects of the functionality. Software (programs, code modules, instructions) that provides the above functionality when executed by a processor can be stored in the storage subsystem 2018. These software modules or instructions can be executed by the processing unit 2004. The storage subsystem 2018 can also provide a repository for storing data used in accordance with the present invention.

[0331] The storage subsystem 2018 can also include a computer-readable storage medium reader 2020 that can be further connected to a computer-readable storage medium 2023. Together with, and optionally in combination with, the system memory 2010, the computer-readable storage medium 2022 can comprehensively represent remote, local, fixed, and / or removable storage devices plus storage media for temporarily and / or more persistently containing, storing, sending, and retrieving computer-readable information.

[0332] A computer-readable storage medium 2022 that contains code or portions of code may also include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented using any method or technology for the storage and / or transmission of information. This may include tangible non-transitory computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or other tangible computer-readable media. When so specified, this may also include intangible transitory computer-readable media such as data signals, data transmissions, or any other medium that can be used to send the desired information and that can be accessed by computing system 2000.

[0333] As an example, computer-readable storage medium 2022 may include a hard disk drive that reads from or writes to a non-removable non-volatile magnetic medium, a disk drive that reads from or writes to a removable non-volatile disk, and an optical disk drive that reads from or writes to a removable non-volatile optical disk (such as a CD ROM, DVD, and disk or other optical medium). Computer-readable storage medium 2022 may include, but is not limited to, drives, flash cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tapes, and so on. Computer-readable storage medium 2022 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), SSDs based on volatile memory (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computing system 2000.

[0334] The communication subsystem 2024 provides an interface to other computer systems and networks. The communication subsystem 2024 serves as an interface for receiving data from other systems and sending data from the computer system 2000 to other systems. For example, the communication subsystem 2024 may enable the computer system 2000 to connect to one or more devices via the Internet. In some aspects, the communication subsystem 2024 may include radio frequency (RF) transceiver components, global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks (e.g., using cellular phone technology, such as advanced data network technologies like 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.10 series standards), or other mobile communication technologies, or any combination thereof). In some aspects, in addition to or as an alternative to the wireless interface, the communication subsystem 2024 may provide a wired network connection (e.g., Ethernet).

[0335] In some aspects, the communication subsystem 2024 may also receive input communications in the form of structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc., on behalf of one or more users who may use the computer system 2000.

[0336] As an example, the communication subsystem 2024 may be configured to receive unstructured data feeds 2026 from users of social media networks and / or other communication services in real time, such as feeds, updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0337] In addition, the communication subsystem 2024 may also be configured to receive data in the form of continuous data streams, which may include event streams 2028 and / or event updates 2030 of real-time events that may be continuous or unbounded in nature and have no explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc.

[0338] The communication subsystem 2024 may also be configured to output structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc., to one or more databases, which may communicate with one or more streaming data source computers coupled to the computer system 2000.

[0339] The computer system 2000 may be one of various types, including handheld portable devices (e.g., cellular phones, computing tablets, PDAs), wearable devices (e.g., Google head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing system.

[0340] Due to the ever-changing nature of computers and networks, the description of the computer system 2000 depicted in the figure is merely intended as a specific example. Many other configurations with more or fewer components than the system depicted in the figure are possible. For example, custom hardware can also be used and / or specific elements can be implemented in hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices such as network input / output devices can also be employed. Based on the disclosure and teachings provided herein, those of ordinary skill in the art will recognize other ways and / or methods of implementing the various aspects.

[0341] In the foregoing specification, aspects of the invention have been described with reference to specific aspects thereof, but those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects of the above-described invention can be used singly or in combination. Additionally, aspects can be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method, comprising: Using a computing device and constructing a question discourse tree including question entities from a question, wherein the question discourse tree represents rhetorical interrelationships between basic discourse units of the question; Using a computing device to access an initial answer from a text corpus; Using a computing device to construct an answer discourse tree including answer entities from the initial answer, wherein the answer discourse tree represents rhetorical interrelationships between basic discourse units of the initial answer; Using a computing device to determine that a score indicating the relevance of the answer entity to the question entity is lower than a threshold; Generating a fictional discourse tree by: Creating additional discourse trees according to the text corpus; Determining that the additional discourse trees include rhetorical relationships connecting the question entity and the answer entity; Extracting subtrees of the additional discourse trees including the question entity, the answer entity, and the rhetorical relationships, thereby generating a fictional discourse tree; And Outputting an answer represented by a combination of the answer discourse tree and the fictional discourse tree.

2. The method according to claim 1, wherein accessing the initial answer includes: Determining an answer relevance score for a portion of the text; And Selecting the portion of the text as the initial answer in response to determining that the answer relevance score is greater than the threshold.

3. The method according to claim 1, wherein the fictional discourse tree includes nodes representing rhetorical relationships, and the method further includes integrating the fictional discourse tree into the answer discourse tree by connecting the nodes to the answer entity.

4. The method according to claim 1, wherein creating the additional discourse trees includes: Calculating a score for each of a plurality of additional discourse trees, the score indicating the number of question entities including mappings to one or more answer entities in the corresponding additional discourse tree; and Selecting the additional discourse tree with the highest score from the plurality of additional discourse trees.

5. The method according to claim 1, wherein creating the additional discourse trees includes: Calculating a score for each of a plurality of additional discourse trees by applying a trained classification model to one or more of (a) the question discourse tree and (b) the corresponding additional answer discourse tree; And Selecting the additional discourse tree with the highest score from the plurality of additional discourse trees.

6. The method according to claim 1, wherein the question includes a plurality of keywords, and wherein accessing the initial answer includes: Obtaining a plurality of answers by performing a search of a plurality of electronic documents based on a search query including the keywords; For each of the plurality of answers, determining an answer score indicating the degree of match between the question and the corresponding answer; And Selecting the answer with the highest score from the plurality of answers as the initial answer.

7. The method according to claim 1, wherein calculating the score includes: Applying a trained classification model to one or more of (a) the question discourse tree and (b) the answer discourse tree; And Receiving a score from the classification model.

8. The method according to claim 1, wherein constructing the discourse tree includes: Accessing a sentence including a plurality of segments, wherein at least one segment includes a verb and a plurality of words, each word including a role of the word within the segment, and wherein each segment is a basic discourse unit; and Generate a discourse tree representing the rhetorical interrelationships between the multiple segments, where the discourse tree includes multiple nodes, each non-terminal node representing the rhetorical interrelationship between two of the multiple segments, and each terminal node among the nodes of the discourse tree being associated with one of the multiple segments.

9. The method according to claim 1, further comprising: Determining a problem communication discourse tree including a problem root node based on the problem discourse tree, where the communication discourse tree is a discourse tree including communication actions, and where the generating further includes: Determining an answer communication discourse tree based on a fictional discourse tree, where the answer communication discourse tree includes an answer root node; Merging the communication discourse trees by identifying that the problem root node and the answer root node are the same; Calculating a degree of complementarity between the problem communication discourse tree and the answer communication discourse tree by applying a prediction model to the merged communication discourse tree; and Outputting a final answer corresponding to the fictional discourse tree in response to determining that the degree of complementarity is higher than a threshold.

10. The method according to claim 1, where the discourse tree represents the rhetorical interrelationships between multiple segments of text, where the discourse tree includes multiple nodes, each non-terminal node representing the rhetorical interrelationship between two of the multiple segments, and each terminal node among the nodes of the discourse tree being associated with one of the multiple segments; and where constructing the communication discourse tree includes: Matching each segment having a verb with a verb signature by: Accessing multiple verb signatures, where each verb signature includes the verb of the segment and a sequence of thematic roles, where the thematic role describes the relationship between the verb and related words; For each verb signature among the multiple verb signatures, determining multiple thematic roles of the corresponding signature that match the roles of the words in the segment; Selecting the specific verb signature from the multiple verb signatures based on the specific verb signature including the largest number of matches; And Associating the specific verb signature with the segment.

11. A computer-implemented method, comprising: Constructing a problem discourse tree for a problem, including multiple problem entities; Constructing an answer discourse tree for an initial answer, including multiple answer entities; Establishing a mapping between a first problem entity among the multiple problem entities and an answer entity among the multiple answer entities, the mapping establishing the relevance of the answer entity to the first problem entity; In response to determining that a second problem entity among the multiple problem entities is not resolved by any answer entity among the multiple answer entities, generating a fictional discourse tree by combining an additional discourse tree corresponding to an additional answer with the answer discourse tree; Determining a problem communication discourse tree based on the problem discourse tree; Determining an answer communication discourse tree based on the fictional discourse tree; Calculating a degree of complementarity between the problem communication discourse tree and the answer communication discourse tree by applying a prediction model, the problem communication discourse tree, and the answer communication discourse tree; And Outputting a final answer corresponding to the fictional discourse tree in response to determining that the degree of complementarity is higher than a threshold.

12. A system, comprising: A computer-readable medium storing non-transitory computer-executable program instructions; And A processing device communicatively coupled to a computer-readable medium for executing non-transitory computer-executable program instructions, wherein executing the non-transitory computer-executable program instructions configures the processing device to perform operations, the operations including: Constructing, using a computing device, a question discourse tree including question entities from a question, wherein the question discourse tree represents rhetorical interrelationships between basic discourse units of the question; Accessing, using a computing device, an initial answer from a text corpus; Constructing, using a computing device, an answer discourse tree including answer entities from the initial answer, wherein the answer discourse tree represents rhetorical interrelationships between basic discourse units of the initial answer; Using a computing device to determine that a score indicating the relevance of the answer entity to the question entity is below a threshold; Generating a fictional discourse tree by: Creating additional discourse trees according to the text corpus; Determining that the additional discourse trees include rhetorical relationships connecting the question entity and the answer entity; Extracting subtrees of the additional discourse trees including the question entity, the answer entity, and the rhetorical relationship, thereby generating a fictional discourse tree; and Outputting an answer represented by a combination of the answer discourse tree and the fictional discourse tree.

13. The system of claim 12, wherein accessing the initial answer includes: Determining an answer relevance score for a portion of the text; And In response to determining that the answer relevance score is greater than a threshold, selecting the portion of the text as the initial answer.

14. The system according to claim 12, wherein the fictional discourse tree includes nodes representing rhetorical relationships, and the operation further includes: Integrating the fictional discourse tree into the answer discourse tree by connecting the node to the answer entity.

15. The system of claim 12, wherein creating the additional discourse trees includes: Calculating a score for each of the multiple additional discourse trees, the score indicating the number of question entities including mappings to one or more answer entities in the corresponding additional discourse tree; and Selecting the additional discourse tree with the highest score from the multiple additional discourse trees.

16. The system of claim 12, wherein creating the additional discourse trees includes: Calculating a score for each of the multiple additional discourse trees by applying a trained classification model to one or more of (a) the question discourse tree and (b) the corresponding additional answer discourse tree; And Selecting the additional discourse tree with the highest score from the multiple additional discourse trees.

17. The system of claim 12, wherein the question includes multiple keywords, and wherein accessing the initial answer includes: Obtaining multiple answers based on a search query including the keywords by performing a search of multiple electronic documents; For each of the multiple answers, determining an answer score indicating the degree of match between the question and the corresponding answer; And Selecting the answer with the highest score from the multiple answers as the initial answer.

18. The system of claim 12, wherein calculating the score includes: Applying a trained classification model to one or more of (a) the question discourse tree and (b) the answer discourse tree; And Receiving a score from the classification model.

19. The system of claim 12, wherein constructing the discourse tree includes: Access a sentence that includes multiple segments, where at least one segment includes a verb and multiple words, each word including the role of the word within the segment, and where each segment is a basic discourse unit; and Generate a discourse tree representing the rhetorical interrelationships between the multiple segments, where the discourse tree includes multiple nodes, each non-terminal node representing the rhetorical interrelationship between two of the multiple segments, and each terminal node in the nodes of the discourse tree being associated with one of the multiple segments.

20. The system of claim 12, the operation further comprising: Determine a question communication discourse tree including a question root node according to the question discourse tree, where the communication discourse tree is a discourse tree including communication actions, and where the generating further includes: Determine an answer communication discourse tree according to a fictional discourse tree, where the answer communication discourse tree includes an answer root node; Merge the communication discourse trees by identifying that the question root node and the answer root node are the same; Calculate a complementary degree between the question communication discourse tree and the answer communication discourse tree by applying a prediction model to the merged communication discourse tree; and In response to determining that the complementary degree is higher than a threshold, output a final answer corresponding to the fictional discourse tree.

21. A system, comprising: A computer-readable medium storing non-transitory computer-executable program instructions; And A processing device communicatively coupled to the computer-readable medium for executing the non-transitory computer-executable program instructions, where executing the non-transitory computer-executable program instructions configures the processing device to perform the computer-implemented method of claim 11.

22. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1-10.

23. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the computer-implemented method of claim 11.

24. A computer program product including instructions that, when executed by one or more processors of a computer, cause the computer to perform the method of any one of claims 1-10.

25. A computer program product including instructions that, when executed by one or more processors of a computer, cause the computer to perform the computer-implemented method of claim 11.

Images