Improved Discourse Parsing

Abstract

The systems, devices, and methods of the present invention include a discourse tree. In some embodiments, the system creates a discourse tree by identifying basic discourse units in a text. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two basic discourse units, and each terminal node is associated with a basic discourse unit. The system identifies rhetorical relationships of type elaboration or joint in reference sentences of the discourse tree. The system selects a candidate sentence with the highest syntactic generalization score from a set of syntactic generalization scores. The system identifies semantic relationships corresponding to the candidate sentences. The semantic relationships correspond to words in the candidate sentence and define roles in the candidate sentence. The system replaces the rhetorical relationships in the discourse tree with updated rhetorical relationships corresponding to the semantic relationships, thereby creating an updated discourse tree.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to linguistics, and more particularly to generating improved discourse trees. [Background technology]

[0002] background Linguistics is the scientific study of language. One aspect of linguistics is the application of computer science to natural human languages, such as English. Due to the significant increase in processor speed and memory capacity, computer applications of linguistics are increasing. For example, computer-enabled analysis of linguistic discourse facilitates numerous applications, such as automated agents that can answer questions from users. However, such applications are unable to leverage the rich discourse-related information to answer questions, perform dialog management, or provide recommendation systems. Summary of the Invention

[0003] overview Generally, the systems, devices, and methods of the present invention relate to improved discourse trees.

[0004] In some aspects, a method for improving accuracy of a discourse tree includes creating a discourse tree from text by identifying elementary discourse units in the text, the discourse tree including nodes, each non-terminal node of the nodes in the discourse tree representing a rhetorical relationship between two elementary discourse units, each terminal node of the nodes in the discourse tree being associated with a elementary discourse unit, the method further includes identifying rhetorical relationships of type elaboration or joint in the discourse tree, the rhetorical relationship relating a first elementary discourse unit and a second elementary discourse unit, the first elementary discourse unit and the second elementary discourse unit forming a reference sentence, the method further includes determining a syntactic generalization score for each candidate sentence in a set of candidate sentences, each candidate sentence representing a corresponding determining a semantic relationship between the candidate sentence and the reference sentence includes identifying one or more common entities between the candidate sentence and the reference sentence and calculating a syntactic generalization score equal to the number of the identified one or more common entities, the method further includes selecting the candidate sentence having the highest syntactic generalization score among the syntactic generalization scores and identifying a semantic relationship corresponding to the candidate sentence, the semantic relationship corresponding to a word in the candidate sentence and defining a role in the candidate sentence, and the method further includes creating an updated discourse tree by replacing the rhetorical relationship with an updated rhetorical relationship corresponding to the semantic relationship in the discourse tree.

[0005] In some aspects, creating a discourse tree from the text includes providing the text to a classification model and using the classification model to identify a first basic discourse unit, a second basic discourse unit, and a rhetorical relation.

[0006] In some aspects, the updated rhetorical relationship is one of an end, a means, a cause, or a temporal sequence.

[0007] In some aspects, each of the one or more common entities shares a common portion of speech between the candidate sentence and the reference sentence.

[0008] In some aspects, the method further includes forming a response from the updated discourse tree and outputting the response to an external device.

[0009] In some aspects, the method includes forming a first syntactic parse tree from each candidate sentence and forming a second syntactic parse tree from the reference sentence, and identifying one or more common entities between the candidate sentence and the reference sentence includes, for each common entity, identifying the common entity in the first syntactic parse tree and the second syntactic parse tree.

[0010] In some aspects, the method includes forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature, identifying that the text contains an argument by applying a classification model trained to detect arguments to the communicative discourse tree, forming a response from the text, and outputting the response to an external device.

[0011] In some aspects, the method includes forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature; identifying that the text includes an argument corresponding to a claim by applying a classification model trained to detect arguments to the communicative discourse tree; and evaluating the consistency of the argument with respect to itself and with respect to domain-defining clauses associated with a domain of the text by solving a logic system, the logic system including a fixed portion including claim terms and domain-defining clauses and a variable portion including a set of defeasible rules from the communicative discourse tree and facts from communicative actions of the communicative discourse tree; and the method further includes, in response to determining that the evaluated consistency is greater than a threshold, forming a text response from the text and outputting the text response to an external device.

[0012] The above methods may be implemented as a tangible computer-readable medium and / or operating within a computer processor and associated memory. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 illustrates an exemplary discourse tree environment according to one aspect. [Figure 2] FIG. 1 illustrates an example of a discourse tree according to one aspect. [Figure 3] FIG. 10 illustrates yet another example of a discourse tree according to an aspect. [Figure 4] FIG. 1 illustrates an exemplary schema according to one aspect. [Figure 5] FIG. 2 illustrates a node-link representation of a hierarchical binary tree according to one aspect. [Figure 6] FIG. 6 illustrates an exemplary indented text encoding for the representation in FIG. 5 according to one aspect. [Figure 7]FIG. 2 illustrates an exemplary discourse tree for a property tax request example, according to one aspect. [Figure 8] FIG. 8 illustrates exemplary responses to the questions presented in FIG. 7. [Figure 9] FIG. 1 illustrates a discourse tree for a formal response according to one aspect. [Figure 10] FIG. 1 illustrates a discourse tree for an unprocessed response according to one aspect. [Figure 11] FIG. 2 illustrates a communicative discourse tree for a first agent's assertion according to one aspect. [Figure 12] FIG. 10 illustrates a communicative discourse tree for a second agent's assertion according to one aspect. [Figure 13] FIG. 10 illustrates a communicative discourse tree for a third agent's assertion according to one aspect. [Figure 14] FIG. 10 is a diagram showing perspective interlacing according to one aspect. [Figure 15] FIG. 1 illustrates an exemplary process for building a communicative discourse tree according to one aspect. [Figure 16] FIG. 1 illustrates a discourse tree and a scenario graph according to one aspect. [Figure 17] FIG. 1 illustrates the formation of a request-response pair according to one aspect. [Figure 18] FIG. 1 illustrates a discourse tree for a maximum common sub-communication according to one aspect. [Figure 19] FIG. 1 illustrates a tree in kernel learning format for a communicative discourse tree according to one aspect. [Figure 20] FIG. 1 illustrates an exemplary process used to implement a rhetorical match classifier according to one aspect. [Figure 21] FIG. 1 illustrates a chatbot that comments on posts, according to one aspect. [Figure 22] FIG. 1 illustrates a chatbot that comments on posts, according to one aspect. [Figure 23] FIG. 2 illustrates a discourse tree for an algorithmic text according to one aspect. [Figure 24] FIG. 1 illustrates an annotated sentence according to one aspect. [Figure 25] FIG. 1 illustrates an annotated sentence according to one aspect. [Figure 26] FIG. 1 illustrates discourse actions of a dialogue according to one aspect. [Figure 27] FIG. 1 illustrates discourse actions of a dialogue according to one aspect. [Figure 28] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 29] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 30] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 31] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 32] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 33] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 34] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 35] 1 illustrates an exemplary communicative discourse tree according to one aspect. [Figure 36] 1 illustrates an exemplary process for determining arguments using machine learning, according to one aspect. [Figure 37] 1 is a fragment of a discourse tree according to one embodiment. [Figure 38] 1 illustrates a discourse tree for boundary review according to one aspect. [Figure 39] 1 illustrates a discourse tree of a sentence illustrating a mixed semantic approach to sentiment analysis, according to one aspect. [Figure 40] 1 illustrates an exemplary method for validating an argument, according to one aspect. [Figure 41] 1 illustrates an exemplary communicative discourse tree of an argument according to one aspect. [Figure 42] 1 illustrates an exemplary method for verifying arguments using reversible logic programming, according to one aspect. [Figure 43] 1 illustrates an exemplary dialectical tree according to one aspect. [Figure 44] 1 illustrates a discourse tree and a semantic tree, according to one embodiment. [Figure 45] 1 illustrates a discourse tree and a semantic tree, according to one embodiment. [Figure 46] 1 is a flowchart of an exemplary process for generating an improved discourse tree, according to one aspect. [Figure 47] 1 illustrates generalization of sentences and templates with known semantic relationships, according to one aspect. [Figure 48] 1 illustrates alignment between two sentences, according to one embodiment. [Figure 49] 1 shows a simplified diagram of a distributed system for implementing one of the aspects. [Figure 50] FIG. 1 is a simplified block diagram of components of a system environment in which services provided by components of an aspect system may be offered as cloud services, according to one aspect. [Figure 51] 1 illustrates an exemplary computer system in which various aspects of the present invention may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0014] Detailed Description Aspects disclosed herein provide technical improvements to the field of computer-implemented linguistics. More specifically, the disclosed solution generates an improved discourse tree by determining updated rhetorical relations of the discourse tree from semantic relations of semantic representations of text. The improved discourse tree can enable improved applications that use discourse trees, such as dialog management, inference, argument detection, search, and navigation. The improved discourse tree can be extended with communicative actions, thereby forming a communicative discourse tree ("CDT"). Communicative actions are collaborative actions performed by individuals based on their mutual deliberation and discussion.

[0015] Thus, technical advantages of some aspects include discourse trees that more accurately represent source texts and improved autonomous agents, such as autonomous agents that can use CDT to verify arguments within texts. For example, CDT can be used to determine agreement between sentences or to detect or verify arguments within texts. A valid argument is, for example, an argument that is logically consistent, and the text of the argument supports the premises of the argument.

[0016] More specifically, by incorporating labels that identify communicative acts, learning of communicative discourse trees can occur across a richer set of features than simply the rhetorical relations and syntax of elementary discourse units (EDUs). With such feature sets, additional techniques such as classification can be used to determine the level of rhetorical match between questions and answers or request-response pairs, detect arguments within texts, and validate arguments within texts, thereby enabling improved automated agents. In doing so, computing systems enable autonomous agents that can intelligently answer questions.

[0017] In another example, a rhetorical classification application executing on a computing device receives a question from a user. The rhetorical classification application generates a communicative discourse tree for the question. The communicative discourse tree is a discourse tree that includes communicative acts. The rhetorical classification application accesses a database of possible answers to the question. Using a predictive model, the rhetorical matching application determines a level of complementarity between the question and each possible answer. In response to determining that the level of complementarity is above a threshold, the rhetorical matching classifier provides an answer to the user, for example, via a display device.

[0018] In another example, a rhetorical classification application generates a communicative discourse tree (CDT) from input text and uses machine learning to validate arguments in a subset of the text. The rhetorical classification application creates a logic program by extracting facts and revocable rules from the communicative discourse tree and provides the facts and revocable rules to a logic system such as defeasible logic programming (DeLP). The logic system then accesses the fixed rules and domain-specific definition clauses and solves the logic program, thereby determining whether the argument is valid (e.g., the argument supports the claim) or invalid (e.g., the argument does not support the claim).

[0019] Specific Definitions "Rhetorical structure theory," as used herein, is a field of research and study that has provided a logical foundation that can enable the analysis of discourse coherence.

[0020] "Discourse tree" or "DT" as used herein refers to a structure that expresses rhetorical relations about sentences that are part of a sentence.

[0021] "Rhetorical relation," "rhetorical relation," "coherence relation," or "discourse relation," as used herein, describes how two segments of discourse are logically connected to each other. Examples of rhetorical relations include elaboration, contrast, and attribution.

[0022] A "sentence fragment" or "fragment," as used herein, is a part of a sentence that can be separated from the rest of the sentence. A fragment is a basic discourse unit. For example, in the sentence "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down the plane," the two fragments are "Dutch accident investigators say that evidence points to pro-Russian rebels" and "as being responsible for shooting down the plane." A fragment may, but does not necessarily, contain a verb.

[0023] "Signature" or "frame," as used herein, refers to the properties of verbs in a fragment. Each signature may include one or more thematic roles. For example, in the fragment "Dutch accident investigators say that evidence points to pro-Russian rebels," the verb is "say," and the signature for this particular use of the verb "say" may be an "agent-verb-topic." In this case, "investigators" is the agent and "evidence" is the topic.

[0024] "Thematic role," as used herein, refers to a component of a signature that is used to describe the role of one or more words. Continuing with the example above, "agent" and "topic" are thematic roles.

[0025] "Nuclearity," as used herein, refers to which text segment, fragment, or span is closer to the center of the writer's purpose. A nucleus is a more central span, and a satellite is a less central one.

[0026] "Coherency," as used herein, refers to that which links two rhetorical relationships together.

[0027] A "communicative verb," ​​as used herein, is a verb that indicates communication. For example, the verb "deny" is a communicative verb.

[0028] "Communicative behavior," as used herein, describes an action performed by one or more agents and the subject of the agents.

[0029] As used herein, a "claim" is a statement of truth of something. For example, a claim can be "I'm not responsible for paying rent this month" or "My rent is late."

[0030] As used herein, an "argument" is a reason or set of reasons stated to support a claim. An exemplary argument for the above claim is "the necessary repairs were not completed."

[0031] As used herein, "argumentative validity" or "validity" refers to whether the argument supporting a claim is internally consistent. Internal consistency refers to whether the argument is consistent with itself, for example, whether it does not contain two contradictory statements. External consistency refers to whether the argument is consistent with known facts and rules.

[0032] As used herein, a "logic system" or "logic program" is a set of instructions, rules, facts, and other information that can represent an argument for a particular claim. Solving a logic system results in a determination of whether the argument is valid.

[0033] As used herein, a "dialectical tree" is a tree that represents an individual argument. A dialectical tree is solved to determine the truth or falsity of the claims supported by the individual arguments. Evaluating a dialectical tree involves determining the validity of the individual arguments.

[0034] Figure 1 illustrates an exemplary discourse tree environment according to one aspect. Figure 1 illustrates a computing device 101, input text 130, and argument indicator 165. Computing device 101 includes one or more of application 102, discourse parser 104, answer database 105, rhetorical match classifier 120, and training data 125. Examples of computing devices include devices 4902, 4904, 4906, and 4908 shown in Figures 49 and 50, respectively, as well as cloud computing device 5002 and client devices 5004, 5006, and 5008.

[0035] In one example, the application 102 generates a discourse tree with higher quality and / or accuracy than for previous solutions. In one example, the discourse parser 104 generates a discourse tree from the input text 130. The application 102 analyzes the discourse tree and generates a semantic representation, such as an Abstract Semantic Representation (AMR) graph. AMR is a semantic representation language. The AMR graph is a rooted, labeled, directed, acyclic graph (DAG) that contains the entire sentence. From the AMR graph, using techniques disclosed herein, the application 102 generates an improved discourse tree, which can then be used to perform discourse analysis. An example of a process for creating an improved discourse tree is discussed with respect to FIG. 45.

[0036] In another example, the application 102 answers a question received via a chat session. The input text 130 can be a single question or a stream of questions. The application 102 creates a question communication discourse tree from the input text 130 and selects one or more candidate answers. The answers can be retrieved from an existing database, such as the answer database 105. The input text 130 can be generated by any mobile device, such as a mobile phone, smartphone, tablet, laptop, or smartwatch. The mobile device can communicate with the computing device 101 over a data network. In this manner, the mobile device can, for example, provide a question from a user to the computing device 101.

[0037] Continuing the example, from the candidate answers, the application 102 determines the most suitable answer. Different methods can be used. In one aspect, the application 102 may create a candidate answer communication discourse tree for each candidate answer and compare the question communication discourse tree with each candidate discourse tree. The application 102 identifies the best match between the question communication discourse tree and the candidate answer communication discourse tree. The application 102 then accesses or queries a database for text from the best communication discourse tree. The application 102 then sends the text associated with the second communication discourse tree to the mobile device.

[0038] In another example, the application 102 creates an answer communicative discourse tree for each answer candidate. Then, for each answer candidate, the application 102 creates a question-answer pair including the input text 130 and the answer candidate. The application 102 provides the question-answer pair to a predictive model, such as a rhetorical match classifier 120. Using the trained rhetorical match classifier 120, the application 102 determines whether the question-answer pair exceeds a threshold level of match, indicating, for example, whether the answer addresses the question. If not, the application 102 continues to analyze additional pairs including the question and different answers until a suitable answer is found. Using a communicative discourse tree, the rhetorical match and communication behavior between questions and answers can be accurately modeled.

[0039] In a further example, the application 102 uses the rhetorical match classifier 120 to determine whether an argument is present or absent in the input text 130. For example, the rhetorical classification application 102 accesses the following input text 130: "[t]he rent was properly refused.... The landlord contacted me, the tenant, and the rent was requested." However, I refused the rent since I demanded repair to be done. I reminded the landlord about necessary repairs, but the landlord issued the three-day notice confirming that the rent was overdue. Regretfully, the property still stayed unrepaired.") Thus, input text 130 includes the assertion, "The rent was properly refused," and the related argument, "The landlord contacted me, the tenant, and the rent was requested. However, I refused the rent since I demanded repair to be done. I reminded the landlord about necessary repairs, but the landlord issued the three-day notice confirming that the rent was overdue. Regretfully, the property still stayed unrepaired."

[0040] To detect an argument, the application 102 determines a communicative discourse tree from the input text 130 and provides the communicative discourse tree to a trained classifier, such as the rhetorical match classifier 120. The application 102 receives a prediction from the rhetorical match classifier 120 of whether an argument is present. The application 102 provides the prediction as an argument indicator 165. The rhetorical match classifier 120 compares the communicative discourse tree to those identified as positive (argument) or negative (no argument) in the training set. An exemplary process is discussed with respect to FIG. 36.

[0041] In yet another aspect, the application 102 can verify an argument present in the input text 130. An exemplary process is discussed with respect to FIG. 40. In one example, the application 102 determines the presence of an argument, for example, by using the rhetorical match classifier 120. The application 102 can then determine whether the detected argument is valid or invalid. Reversible logic programming can be used. An exemplary process is discussed with respect to FIG. 42. The application 102 can output an argument indicator 165 that can indicate whether an argument is detected, and if so, can indicate whether the argument is valid or invalid.

[0042] Rhetorical Structure Theory and Discourse Trees Linguistics is the scientific study of language. For example, linguistics can include the structure of a sentence (syntax), e.g., subject-verb-object, the meaning of a sentence (semantics), e.g., "dog bites man" vs. "man bites dog", and also what speakers do in conversation, i.e., discourse analysis or the analysis of language beyond the sentence.

[0043] The theoretical foundation of discourse (Rhetoric Structure Theory (RST)) can be traced to "Rhetorical structure theory: A Theory of Text organization" by Mann, William, and Thompson, Sandra (Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243-281, 1988). RST enabled discourse analysis in a similar way to how syntax and semantics from programming language theory enabled modern software compilers. More specifically, RST envisions building blocks at at least two levels. The two levels include a first level, such as kernels and rhetorical relations, and a second level, such as structure or schema. A discourse parser or other computer software can parse text into a discourse tree.

[0044] Rhetorical structure theory models the logical organization of a text (the structure used by the writer) by relying on the relationships between parts of the text. RST simulates text coherence by forming a hierarchical, connected structure of the text through a discourse tree. Rhetorical relationships are divided into equivalence classes and subclasses. These relationships maintain across two or more text spans, thereby achieving coherence. These text spans are called elementary discourse units (EDUs). Clauses within a sentence and sentences within a text are logically connected by the author. The meaning of a given sentence is related to the meaning of the previous and following sentences. This logical relationship between clauses is called the coherence structure of the text. RST is one of the most widespread theories of discourse, based on a tree-like discourse structure, the discourse tree (DT). The leaves of the DT correspond to EDUs (successive atomic text spans). Adjacent EDUs are connected by coherence relations (e.g., attributes, sequences) that form higher-level discourse units. These units are further subordinate to this relational link. EDUs linked by a relation are further differentiated based on their relative importance. The core is the core part of the relation, and the satellites are the periphery. As mentioned above, to determine the correct request-response pair, both topic and rhetorical match are analyzed. When a speaker answers a question, such as a phrase or sentence, their response must address the topic of this question. If the question is formulated implicitly, the seed text of the message is used to anticipate an appropriate response that not only stays on topic but also matches the generalized cognitive state about this seed.

[0045] Rhetorical relations As mentioned above, several aspects described in this specification use communicative discourse trees. Rhetorical relations can be described in various ways. For example, Mann and Thompson describe 23 possible relations in "Rhetorical Structure Theory: A Theory of Text Organization" by C. Mann, William & Thompson, Sandra (1987) ("Mann and Thompson"). Several other relations are also possible.

[0046] [Table 1]

[0047] Some empirical studies assume that the majority of texts are constructed using core-satellite relationships (see Mann and Thompson). However, other relationships do not involve a finite choice of cores. Examples of such relationships are given below.

[0048] [Table 2]

[0049] Figure 2 shows an example of a discourse tree according to one aspect. Figure 2 includes a discourse tree 200. The discourse tree includes text span 201, text span 202, text span 203, relation 210, and relation 228. The numbers in Figure 2 correspond to the three text spans. Figure 3 corresponds to the following text example with three text spans numbered 1, 2, and 3:

[0050] 1. Honolulu, Hawaii will be the site of the 2017 Conference on Hawaiian History.

[0051] 2. It is expected that 200 historians from the US and Asia will attend.

[0052] 3. The conference will be concerned with how the Polynesians sailed to Hawaii.

[0053] For example, relationship 210, or elaboration, describes the relationship between text span 201 and text span 202. Relationship 228 indicates the relationship (elaboration) between text span 203 and text span 204. As shown, text spans 202 and 203 further elaborate on text span 201. In the example above, assuming the purpose is to inform the reader of a conference, text span 1 is the core. Text spans 2 and 3 provide more details about the conference. In Figure 2, horizontal numbers (e.g., 1-3, 1, 2, 3) cover spans of text (possibly composed of further spans), and vertical lines indicate a core or multiple cores. The curved lines represent rhetorical relationships (elaborations), with arrows pointing from the satellites to the core. If only text spans functioned as satellites, rather than as cores, removing the satellites would still leave the text coherent. Removing the core from Figure 2 would make text spans 2 and 3 difficult to understand.

[0054] Figure 3 shows a further example of a discourse tree according to one aspect. Figure 3 includes constituents 301 and 302, text spans 305-307, relation 310, and relation 328. Relation 310 shows the (enabling) relationship between constituent 306 and constituent 305, and between constituent 307 and constituent 305. Figure 3 points to the following text spans: 1. The new Tech Report abstracts are now in the journal area of ​​the library near the abridged dictionary.

[0055] 2. Please sign your name by any means that you would be interested in seeing.

[0056] 3. The last day for sign-ups is May 31st. As can be seen, relationship 328 indicates the relationship, or enabling, between entity 307 and entity 306. Figure 3 illustrates that although multiple kernels can be nested, there is only one most kernel text span.

[0057] Discourse tree construction Discourse trees can be generated using a variety of methods. A simple example of a method for building a DT bottom up is as follows: (1) Divide the discourse text into multiple units using (a) and (b) below.

[0058] (a) The unit size may vary depending on the purpose of the analysis. (b) Typically the unit is a clause.

[0059] (2) Examine each unit and each adjacent unit. Is there a relationship between them? (3) If the relationship is maintained, mark the relationship.

[0060] (4) If a relationship does not hold, the unit may be at the boundary of a higher-level relationship. Focus on the relationships that hold between larger units (spans).

[0061] (5) Continue until all units in the text are understood. Mann and Thompson also describe a second level of building block structures called schema applications. In RST, rhetorical relations are not mapped directly onto the text; rather, they are fitted onto structures called schema applications, which are then further fitted to the text. Schema applications are derived from simpler structures called schemas (as shown in Figure 4). Each schema shows how a particular unit of text can be decomposed into smaller textual units. A rhetorical structure tree, or DT, is a hierarchical system of schema applications. Schema applications link several consecutive text spans to create complex text spans. Complex text spans can then be linked by higher-level schema applications. RST asserts that the structure of any coherent discourse can be described by a single rhetorical structure tree, the top-level schema of which creates a span that encompasses the entire discourse.

[0062] FIG. 4 illustrates an exemplary schema according to one aspect. FIG. 4 shows that a joint schema is a list of items consisting of a core but no satellites. FIG. 4 illustrates schemas 401-406. Schema 401 illustrates a situational relationship between text span 410 and text span 428. Schema 402 illustrates a sequence relationship between text span 420 and text span 421, and a sequence relationship between text span 421 and text span 422. Schema 403 illustrates a contrastive relationship between text span 430 and text span 431. Schema 404 illustrates a joint relationship between text span 440 and text span 441. Schema 405 illustrates a motivational relationship between 450 and 451, and an enabling relationship between 452 and 451. Schema 406 illustrates a joint relationship between text span 460 and text span 462. An example of a joint scheme is illustrated in FIG. 4 for the following three text spans:

[0063] 1. Skies will be partly sunny in the New York metropolitan area today.

[0064] 2. It will be more humid, with temperatures in the middle 80's.

[0065] 3. Tonight will be mostly cloudy, with the low temperature between 65 and 70 degrees Fahrenheit.

[0066] Although Figures 2-4 show discourse trees in several graphs, other representations are possible.

[0067] Figure 5 shows a node-link representation of a hierarchical binary tree according to one aspect. As can be seen from Figure 5, the leaves of the DT correspond to consecutive but non-overlapping spans of text called elementary discourse units (EDUs). Adjacent EDUs are connected by relations (e.g., elaboration, attribute…) to form larger discourse units connected by relations. “Discourse analysis in RST involves two subtasks: discourse segmentation is the task of identifying EDUs, and discourse parsing is the task of linking discourse units into a labeled tree.” See Joty, Shafiq R, Giuseppe Carenini, Raymond T Ng, and Yashar Mehdad (2013), “Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis,” ACL (1), pages 486-496.

[0068] Figure 5 shows text spans that are leaves or terminal nodes on a tree, numbered in the order in which they appear throughout the text shown in Figure 6. Figure 5 includes a tree 500. Tree 500 includes, for example, nodes 501-507. The nodes represent relationships. The nodes are either non-terminal nodes, such as node 501, or terminal nodes, such as nodes 502-507. As can be seen, nodes 503 and 504 are related by a joint relationship. Nodes 502, 505, 506, and 508 are nuclei. The dotted lines indicate that the branches or text spans are satellites. These relationships are the nodes in the gray boxes.

[0069] FIG. 6 shows an exemplary indented text encoding for the representation in FIG. 5, according to one aspect. FIG. 6 includes text 600 and text sequences 602-604. Text 600 is represented in a manner that is more amenable to computer programming. Text sequence 602 corresponds to node 502. Sequence 603 corresponds to node 503. Sequence 604 corresponds to node 504. In FIG. 6, "N" indicates a nucleus and "S" indicates a satellite.

[0070] Discourse Parser Example Automatic discourse segmentation can be performed in various ways. For example, given a sentence, a segmentation model identifies boundaries of composite basic discourse units by predicting whether a boundary should be inserted before each particular token in the sentence. For example, one framework considers each token in the sentence sequentially and independently. In this framework, the segmentation model scans the sentence token by token and uses binary classification, such as a support vector machine or logistic regression, to predict whether it is appropriate to insert a boundary before the token being examined. In another example, the task is a sequential labeling problem. Once the text is segmented into basic discourse units, a sentence-level discourse analysis can be performed to construct a discourse tree. Machine learning techniques can be used.

[0071] In one aspect of the present invention, two Rhetorical Structure Theory (RST) discourse parsers are used: CoreNLPProcessor, which relies on constituent syntax, and FastNLPProcessor, which uses dependency syntax. See "Two Practical Rhetorical Structure Theory Parsers" by Surdeanu, Mihai & Hicks, Thomas & Antonio Valenzuela-Escarcega, Marco (2015).

[0072] Additionally, the two discourse parsers mentioned above, CoreNLPProcessor and FastNLPProcessor, use natural language processing (NLP) for parsing. For example, Stanford CoreNLP presents basic shapes for multi-word parts of speech, whether they are names of companies, people, etc.; standardizes dates, times, and numerical quantities; marks sentence structure in terms of phrases and syntactic dependencies; and indicates which noun phrases refer to the same entity. In practice, RST remains a theory of discourse that may work in many cases, but may not work in other cases. Many variables exist that determine what EDUs are in a coherent text, including, but not limited to, what discourse segmenter is used, what relationship inventory is used, what relationships are selected for the EDUs, the corpus of documents used for training and testing, and even what parser is used. For example, in the aforementioned paper "Two Practical Rhetorical Structure Theory Parsers" by Surdeanu et al., tests must be run on specific corpora using specialized metrics to determine which parser provides better performance. Thus, unlike computer language parsers, which produce predictable results, discourse parsers (and segmenters) can produce unpredictable results, depending on the training and / or testing text corpora. Discourse trees are thus a mixture of predictable techniques (e.g., compilers) and unpredictable techniques (e.g., chemistry, where experimentation is required to determine which combinations will produce the desired results).

[0073] To objectively judge how good a discourse analysis is, a set of metrics is used, such as the Precision / Recall / F1 metric by Daniel Marcu in "The Theory and Practice of Discourse Parsing and Summarization" (MIT Press) (2000). Precision, or positive predictive value, is the fraction of relevant instances among the instances retrieved, while recall (also known as sensitivity) is the fraction of relevant instances retrieved over the total amount of relevant instances. Thus, both precision and recall are based on an understanding and measure of relevance. Suppose a computer program for recognizing dogs in photographs identifies eight dogs in a photograph containing twelve dogs and some cats. Of the eight dogs identified, five are indeed dogs (true positives) and the rest are cats (false positives). The program's precision is 5 / 8 and its recall is 5 / 12. If a search engine returns 30 pages, but only 20 of them are relevant, and does not return an additional 40 relevant pages, its precision is 20 / 30 = 2 / 3 and its recall is 20 / 60 = 1 / 3. So, in this case, precision is "how useful the search results are," and recall is "how complete the results are." The F1 score (F-score or F-criterion) is a measure of the accuracy of a test. It takes into account both the precision and recall of a test to calculate the score: F1 = 2 x (precision x recall) / (precision + recall)), which is the harmonic mean of precision and recall. The F1 score reaches its optimum at 1 (perfect precision and recall) and its worst at 0.

[0074] Autonomous Agents or Chatbots A conversation between person A and person B is a form of discourse. For example, applications such as Facebook® Messenger, WhatsApp®, Slack®, and SMS exist, and conversation between A and B may typically be via messages in addition to more traditional email and voice conversations. A chatbot (sometimes referred to as an intelligent bot or virtual assistant) is an “intelligent” machine that, for example, replaces person B and mimics, to varying degrees, the conversation between the two people. One example of the ultimate goal is for person A to be unable to distinguish whether person B is a human or a machine (the Turing test, developed by Alan Turing in 1950). Artificial intelligence and natural language processing, including discourse analysis and machine learning, have made great strides toward the long-term goal of passing the Turing test. Of course, computers are increasingly able to search and process vast repositories of data and perform complex analyses on the data, including predictive analytics, and the long-term goal is to combine human-like chatbots with computers.

[0075] For example, users can interact with an intelligent bot platform through conversational interactions. This interaction, called a conversational user interface (UI), is a dialogue between the end user and the chatbot, just like a conversation between two people. It can be as simple as the end user saying "Hello" to the chatbot, the chatbot responding "Hi," and the chatbot then asking the user what they need. It can also be a transactional interaction with a banking chatbot, such as transferring money from one account to another, or an informational interaction with an HR chatbot, such as checking a vacation balance, or a FAQ question with a retail chatbot, such as how to process a return. Natural language processing (NLP) and machine learning (ML) algorithms, combined with other approaches, can be used to classify end-user intent. A high-level intent is what the end user wants to accomplish (e.g., get a balance and make a purchase). An intent is essentially a mapping of customer input to units of work for the backend to perform. So based on the phrases uttered by the user in the chatbot, these are mapped to specific distinct use cases or units of work for, for example, balance enquiry, money transfer and tracking expenses - all "use cases" that the chatbot should be able to support and solve, which units of work should be triggered from the free text entry that the end user types in natural language.

[0076] The underlying principle of making an AI chatbot respond like a human is that the human brain can formulate and understand requests, and furthermore, can provide superior responses to human requests far better than a machine can. Therefore, if human B is to be imitated, the chatbot's requests / responses must be significantly improved. So the first part of the problem is how the human brain formulates and understands requests. A model is used for imitating. RST and DT allow a formal and repeatable way to do this.

[0077] At a high level, there are typically two types of requests: (1) a request to perform some action, and (2) a request for information (e.g., a question). The first type has a response for which a unit of work is created. The second type has a response to the question (i.e., e.g., a good answer). For example, in some aspects the answer may take the form of AI constructing an answer from an extensive knowledge base or from matching the best existing answer by searching the internet or intranet or other publicly or privately available data sources.

[0078] Communicative discourse trees and rhetorical classifiers Aspects of the present disclosure build a communicative discourse tree and use the communicative discourse tree to analyze whether the rhetorical structure of a request or question matches an answer. More specifically, aspects described herein create representations of request-response pairs, learn these representations, and associate pairs with classes of valid or invalid pairs. In this manner, an autonomous agent can receive a question from a user, process the question by, for example, searching multiple answers, determine a best answer from among the multiple answers, and provide the answer to the user.

[0079] More specifically, to represent the linguistic features of text, the aspects described herein use rhetorical relations and speech acts (or communicative actions). Rhetorical relations are relationships between parts of sentences, typically obtained from a discourse tree. Speech acts are obtained as verbs from a verb resource such as VerbNet. By using both rhetorical relations and communicative actions, the aspects described herein can accurately recognize valid request-answer pairs. To do so, the aspects correlate the syntactic structure of a question with the syntactic structure of an answer. Using this structure, a better answer can be determined.

[0080] For example, if an autonomous agent receives an indication from a person that the person wishes to sell an item with a particular characteristic, the autonomous agent should provide search results that not only include the characteristic but also indicate an intent to purchase. In this manner, the autonomous agent has determined the user's intent. Similarly, if an autonomous agent receives a request from a person to share information about a particular item, the search results should include an intent to receive a recommendation. If a person asks the autonomous agent for opinions on a topic, the autonomous agent shares its opinion on the subject rather than soliciting another opinion.

[0081] Analyzing Request and Response Pairs Figure 7 shows an example discourse tree for the property tax request example, according to one aspect. Node labels are relationships, and arrowhead lines point to satellites. The kernel is a solid line. Figure 7 shows the following text:

[0082] Request: "My husbands' grandmother him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund. I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?(My husband's grandmother gave him his grandfather's truck. She transferred the title, but my husband couldn't put the truck in his name because he had an outstanding fine on his license. I wanted it in my name, so I paid the property taxes and got insurance on the truck. When it came time to get the title and get the tag, I didn't have the money to do so. Now, depending on the situation, I won't be able to afford to keep the truck. I went to the insurance agent and they refused to refund me. I'm wondering if I can get the property taxes refunded if I can't get the tag on the truck.)

[0083] Response: "The property tax is assessed on property that you own. Just because you chose to not register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax. If you apply late, there will be penalties on top of the normal taxes and fees. You don't need to register it at the same time, but you absolutely need to title it within the period of time stipulated in state Property taxes are levied on property you own. Just because you choose not to register it doesn't mean you don't own it. Therefore, the tax cannot be refunded. Even if you haven't yet title to the vehicle, you still own it within the taxing district boundaries and must pay the tax. Keep in mind that every state has a limited time to transfer title and pay use taxes. If you file late, you may be assessed penalties in addition to the regular taxes and fees. You don't have to register the vehicle at the same time, but you must make sure you title it within the time frame required by state law. As can be seen from Figure 7, by analyzing the above text, we obtain the following results: "My husband's grandmother gave him his grandfather's truck" is elaborated by the expression "She signed the title over but due to my husband", which is elaborated by "having unpaid fines on his license, he was not able to get the truck put in his name", which is elaborated by the expressions "I wanted to put in my name", "and paid the property tax", and "and got insurance for the truck".

[0084] "My husband's grandmother gave him his grandfather's truck. She signed the title over, but due to my husband having unpaid fines on his license, he was unable to get the truck put in his name. I wanted to put it in my name and paid the property tax and got insurance for the truck." is elaborated by "I didn't have the money," which is elaborated by "to do so," which is contrasted with "By the time," which is elaborated by "it came to sending off the title."

[0085] "My husband's grandmother gave him his grandfather's truck. She signed the title over, but due to my husband having unpaid fines on his license, he was unable to get the truck put in his name. I wanted to put in my name, paid the property tax, and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so" contrasts with "Now, due to circumstances," which is detailed by "I am not going to be able to afford the truck," which is detailed by "I went to the insurance place" and "and was refused a refund."

[0086] "My husbands' grandmother gave him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund.'' is detailed in ``I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?''.

[0087] "I am just wondering" belongs to the same unit "that" as "is it possible to get the property tax refunded?", which has the condition "since I am not going to have a tag on this truck."

[0088] As can be seen, the main topic theme is "Property Tax on Cars." The question contains a contradiction: on the one hand, all property is taxable; on the other hand, ownership is somewhat incomplete. A good response must address the topic of the question and clarify the contradiction. To do this, the respondent makes a stronger case for the need to pay tax on everything owned, regardless of registration status. This example is an element of a positive training set from the Yahoo! Answers reputation domain. The main topic theme is "Property Tax on Cars." The question contains a contradiction: on the one hand, all property is taxable; on the other hand, ownership is somewhat incomplete. A good answer / response must address the topic of the question and clarify the contradiction. The reader may realize that the question contains a rhetorical relation of contrast, and therefore the answer must match the question in a similar relation to be convincing. In other cases, the answer may appear incomplete even to people who are not experts in the field.

[0089] 8 shows an example response to the question displayed in FIG. 7 in accordance with certain aspects of the present invention. The central core is "The property tax is assessed on property" detailed by "that you own." "The property tax is assessed on property that you own" is also a core detailed by "Just because you chose not to register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax."

[0090] The core statement, "The property tax is assessed on property that you own. Just because you chose not to register it does not mean that you don't own it, so the tax is not refundable. Even if you haven't titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax." is expanded by the condition, "If you apply late," that "there will be penalties on top of the normal taxes and fees." This is further expanded by the contrast, "but you absolutely need to title it within the period of time stipulated in state law" and "You don't need to register it at the same time."

[0091] By comparing the DT of Figure 7 with the DT of Figure 8, it is possible to determine how well the response (Figure 8) matches the request (Figure 7). In some aspects of the invention, the above framework is used, at least in part, to determine the DT for request / response and rhetorical fit between the DTs.

[0092] In another example, the question "What does The Investigative Committee of the Russian Federation do?" has at least two answers, for example, the official answer or the actual answer.

[0093] Figure 9 shows the discourse tree for the official response according to one aspect. As shown in Figure 9, the official response or statement states, "The Investigative Committee of the Russian Federation is the main federal investigating authority which operates as Russia's Anti-corruption agency and has statutory responsibility for inspecting the police forces, combating police corruption and police misconduct, is responsible for conducting investigations into local authorities and federal governmental bodies."

[0094] Figure 10 shows the discourse tree of the raw answers according to one aspect. As shown in Figure 10, another, and possibly more honest, answer is as follows: "The Investigative Committee of the Russian Federation is supposed to fight corruption. However, top-ranking officers of the Investigative Committee of the Russian Federation are charged with the creation of a criminal community. Not only that, but their involvement in large bribes, money laundering, obstruction of justice, abuse of power, extortion, and racketeering has been reported. Due to the activities of these officers, dozens of high-profile cases, including those against criminal lords, had been ultimately ruined." The choice of answer depends on the context. Rhetorical structure allows us to distinguish between "official," "politically correct," template-based answers and "actual," "raw," "reports from the field," or "controversial" answers. See Figures 9 and 10. Sometimes, the question itself can give a hint as to which category of answer is expected. If the question is formulated as a factual or definitional question without a second meaning, an answer in the first category is appropriate. In other cases, if the question has the meaning "tell me what it actually is," the second category is appropriate. In general, after extracting the rhetorical structure from a question, it is easier to select an appropriate answer that will have a similar, consistent, or complementary rhetorical structure.

[0095] The official answer is based on elaborations and joints that are neutral in terms of the controversies the text may contain (see figure). At the same time, the raw answer contains a contrastive relation: this relation between a phrase about what the agent is expected to do and a phrase about what this agent was found to have done is extracted.

[0096] Classification of request-response pairs The application 102 can determine whether a given answer or response, such as an answer from the answer database 105 or a public database, is responsive to a given question or request. More specifically, the application 102 analyzes whether a request-response pair is accurate or inaccurate by determining one or both of (i) relevance or (ii) rhetorical match between the request and the response. Rhetorical match can be analyzed without taking relevance into account, which can be treated orthogonally.

[0097] The application 102 can determine the similarity between question-answer pairs using a variety of methods. For example, the application 102 can determine a level of similarity between each question and each answer. Alternatively, the application 102 can determine a measure of similarity between a first pair including a question and an answer and a second pair including a question and an answer.

[0098] For example, the application 102 uses a rhetorical match classifier 120 that is trained to predict matching or non-matching answers. The application 102 may use two pairs at a time, e.g.<q1,a1> and<q2,a2> , the application 102 compares q1 with q2 and a1 with a1 to generate a combined similarity score. Such comparisons allow determining whether an unknown question / answer pair contains the correct answer by assessing its distance from another question / answer pair with a known label. In particular, the unlabeled pair<q2,a2> does not "guess" accuracy based on words or structures shared by q2 and a2, but rather, both q2 and a2 use these words or structures as a basis for determining the accuracy of the labeled pair<q2,a2> Since this method aims to classify answers independently of the domain, it can only exploit the structural connectivity between questions and answers, but cannot exploit the "meaning" of the answers.

[0099] In one aspect, the application 102 uses training data 125 to train the rhetorical match classifier 120. In this manner, the rhetorical match classifier 120 is trained to determine the similarity between question and answer pairs, which is a classification problem. The training data 125 may include a positive training set and a negative training set. The training data 125 includes matching request-response pairs in the positive data set and optional or less relevant or appropriate request-response pairs in the negative data set. For the positive data set, different domains are selected with distinct acceptance criteria indicating whether an answer or response is suitable for the question.

[0100] Each training data set includes a set of training pairs. Each training set includes a question-communicating discourse tree that represents a question and an answer-communicating discourse tree that represents an answer and represents an expected level of complementarity between the question and the answer. Using an iterative process, the application 102 provides the training pairs to the rhetorical match classifier 120 and receives a level of complementarity from the model. The application 102 calculates a loss function by determining the discrepancy between the determined level of complementarity and the expected level of complementarity for a particular training pair. Based on the loss function, the application 102 adjusts the internal parameters of the classification model to minimize the loss function.

[0101] Acceptance standards may vary depending on the application. For example, acceptance standards may be low for community question and answering, automated question and answering, automated customer support systems, manual customer support systems, social network communications, and individual postings, such as consumers' experiences with products, such as surveys and complaints. RR acceptance standards may be high for scientific and technical documents, specialized newspapers and magazines, FAQ-style health and legal documents, and specialized social networks such as "stackoverflow."

[0102] Communicative Discourse Tree (CDT) The application 102 can create, analyze, and compare communicative discourse trees. Communicative discourse trees are designed to combine rhetorical information with speech act structure. CDTs contain arcs labeled with expressions about communicative actions. By combining communicative actions, CDTs enable the modeling of RST relations and communicative actions. CDTs are an epitome of parsing chiasm. See Galitsky, B. Ilvovsky, and D. Kuznetsov, SO, "Rhetoric Map of an Answer to Compound Queries Knowledge Trail Inc. ACL 2015, 681-686" (Galitsky (2015)). Parsing chiasm combines a parse tree for a sentence with discourse-level relationships between the words and parts of the sentence in a single graph. Incorporating labels that identify speech acts enables the learning of communicative discourse trees across a richer set of features than the syntax and proper rhetorical relationships of elementary discourse units (EDUs).

[0103] In one example, a dispute between three parties regarding the causes of the downing of a civilian airliner, Malaysian Airlines Flight 17, is analyzed. An RST representation of the back-and-forth argument is constructed. In this example, three competing agents—Dutch investigators, the Investigative Committee of the Russian Federation, and the self-proclaimed Donetsk People's Republic—exchange their opinions on the issue. This example illustrates a contentious conflict in which each party threatens to accuse the other. To appear more persuasive, each party not only makes its own claims but also formulates responses to reject the other party's claims. To achieve this goal, each party attempts to match the other party's argumentative style and discourse.

[0104] Figure 11 shows a communicative discourse tree for the first agent's argument according to one aspect. Figure 11 shows a communicative discourse tree 100 representing the following text: "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down the plane. The report indicates where the missile was fired from and identifies who was in control of the territory and pins the downing of MH17 on the pro-Russian rebels." As can be seen in Figure 11, the non-terminal nodes of the CDT are rhetorical relations, and the terminal nodes are the basic discourse units (phrases, sentence fragments) that are the subjects of these relations. Some arcs in the CDT are labeled with expressions about communicative actions, including the agents who are the actors and the subjects of these actions (what is being exchanged). For example, the core node for the elaboration relation (on the left) is labeled say(Dutch, evidence), and the satellite is labeled responsible(rebels, shooting down). These labels are not intended to represent that the subjects of the EDUs are evidence and shooting down, but rather to match this CDT with others in order to find similarities between them. In this case, simply linking these communicative actions by rhetorical relations, rather than providing information about the communicative discourse, is too limited to represent the structure of what is being exchanged and how it is being exchanged. The requirement for RR pairs to have the same or coordinated rhetorical relations is too weak, so in addition to matching nodes, it becomes necessary to match CDT labels on arcs.

[0105] The straight edges of this graph are syntactic relations, and the curved arcs are discourse relations such as anaphora, same-entity, subentity, rhetorical relations, and communicative acts. This graph contains much richer information than simply combining parse trees for individual sentences. In addition to CDT, parse chiasm can be generalized at the word, relation, phrase, and sentence levels. Speech acts are logical predicates that represent the agents involved in each speech act and their subject. Arguments for logical predicates are formed according to their semantic roles, as proposed by frameworks such as VerbNet. See "A Large-scale Classification of English Verbs" by Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer (Language Resources and Evaluation Journal, 42(1), 21-40, Springer Netherlands, 2008). and / or "VerbNet overview, extensions, mappings and apps" by Karin Kipper Schuler, Anna Korhonen, and Susan W. Brown, Tutorial, NAACL-HLT: 2009, Boulder, Colorado.

[0106] Figure 12 shows a communicative discourse tree for the second agent's assertion according to one aspect. Figure 12 shows a communicative discourse tree 1200 representing the following text: "The Investigative Committee of the Russian Federation believes that the plane was hit by a missile, which was not produced in Russia. The committee cites an investigation that established the type of the missile." Figure 13 shows a communicative discourse tree for a third agent's claim according to one aspect. Figure 13 shows a communicative discourse tree 1300 representing the following text: "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired. It became possible only after three months after the tragedy to say if rebels controlled one or another town." As can be seen from the communicative discourse trees 1100-1300, the responses are not arbitrary. They refer to the same entities as the original text. For example, communicative discourse trees 1200 and 1300 relate to communicative discourse tree 1100. The responses support inconsistencies with inferences and sentiments about these entities and about their behavior.

[0107] More specifically, the replies of participating agents should reflect the communicative discourse of the first seed message. As a simple observation, as the first agent uses attributes to convey its argument, other agents will follow that set and either provide their own attributes or attack the validity of the advocate's attributes, or both. Each CDT pair can be trained to capture a wide variety of features regarding how the communicative structure of the seed message should be preserved in subsequent messages.

[0108] To verify the match of requests and responses, discourse relations or speech acts (communicative actions) alone are often insufficient. As can be seen from the examples shown in Figures 11-13, the discourse structure and dialogue type of the dialogue between agents are useful. However, it is not necessary to analyze the domain of the dialogue (e.g., military conflict or politics) or the subject matter (i.e., entities) of these dialogues.

[0109] Representation of rhetorical relations and communicative actions Two approaches are often used to compute similarity between abstract structures: (1) represent these structures in a numerical space and represent similarity as a number (a statistical learning approach); or (2) use structural representations such as trees and graphs rather than numerical spaces to represent similarity as a maximal common substructure. Representing similarity as a maximal common substructure is called generalization.

[0110] Learning communicative actions aids in the representation and comprehension of arguments. A computational verb lexicon supports the capture of action entities and helps provide a rule-based format for representing their meaning. Verbs represent the semantics of the described event as well as relational information between participants in that event, projecting a syntactic structure that encodes that information. Verbs, particularly communicative action verbs, are highly variable and can display a rich range of semantic behaviors. Accordingly, verb classification helps learning systems address this complexity by organizing verbs into groups that share core semantic properties.

[0111] VerbNet is one such lexicon that identifies the semantic roles and syntactic pattern characteristics of verbs within each class and articulates the connection between the syntactic patterns and the underlying semantic relationships that can be inferred for all members of the class. See "Language Resources and Evaluation" by Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer, Vol. 42, No. 1 (March 2008) 21. Each syntactic frame or verb signature for a class has a corresponding semantic representation that details the semantic relationships between event participants over the course of the event.

[0112] For example, the verb "amuse" is part of a cluster of similar verbs with similar structures of argument (semantic roles), such as amaze, anger, arouse some emotion, disturb, and irritate. The argumentative roles of these communicative actions are Experiencer (usually a living entity), Stimulus, and Result. Each verb can have a semantic class that is distinguished by syntactic features of how the verb appears within a sentence or frame. For example, the frame for "amuse" uses the following main noun phrase (NP), noun (N), communicative action (V), verb phrase (VP), and adverb (ADV):

[0113] NP V NP. Example: "The teacher amused the children." Syntax: Stimulus V Experiencer. Clauses: amuse(Stimulus, E, Emotion, Experiencer), cause(Stimulus, E), emotional_state(Result(E), Emotion, Experiencer).

[0114] NP V ADV-Middle: Example: "Small children amuse quickly." Syntax: Experiencer V ADV. Clause: amuse(Experiencer, Prop):-, property(Experiencer, Prop), adv(Prop).

[0115] NP V NP-PRO-ARB. Example: "The teacher amused." Syntax: Stimulus V. amuse(Stimulus, E, Emotion, Experiencer): cause(Stimulus, E), emotional_state(Result(E), Emotion, Experiencer).

[0116] NPcause V NP. Example: "The teacher's dolls amused the children." Syntax: Stimulus <+genitive>('s) V Experiencer. amuse(Stimulus, E, Emotion, Experiencer): cause(Stimulus, E), emotional_state(during(E), Emotion, Experiencer).

[0117] NP V NP ADJ. Example: "This performance bored me totally." Syntax: Stimulus V Experiencer Result. amuse(Stimulus, E, Emotion, Experiencer). cause(Stimulus, E), emotional_state(result(E), Emotion, Experiencer), Pred(result(E), Experiencer).

[0118] Communicative actions can be characterized into several clusters, such as verbs with predicate complements (appoint, characterize, dub, declare, conjecture, masquerade, orphan, captain, consider, classify); verbs of perception (see, sight, peer); verbs of mental states (amu se (to entertain, admire, marvel, appeal); verbs of desire (want, long); verbs of judgment (judgment); verbs of evaluation (assess, estimate); verbs of search (hunt, search, stalk, investigate, rummage, ferret); social Verbs of speech (correspond, marry, meet, battle); verbs of communication (transfer (message), inquire, interrogate, tell, manner (speaking), talk, chat, say, complain, advise, confess) These include: to do, lecture, overstate, promise; avoidance verbs (avoid); measurement verbs (register, cost, fit, price, bill); and aspect verbs (begin, complete, continue, stop, establish, sustain).

[0119] Aspects described herein have advantages over statistical learning models. In contrast to statistical solutions, aspects that use classification systems can provide verbs or verb-like structures that are determined to yield desired features (such as rhetorical matches). For example, statistical machine learning models represent similarities as numbers, which can be difficult to interpret.

[0120] Representing a request-response pair Representing request-response pairs facilitates classification-based operations based on a pair. In one example, a request-response pair can be represented as a parse chiasm. A parse chiasm represents the parse trees for two or more sentences using discourse-level relationships between the words and parts of the sentences in a graph. See Galitsky (2015). Topical similarity between a question and an answer can be represented as a common subgraph of the parse chiasm. The more common graph nodes there are, the higher the similarity.

[0121] 14 illustrates parse interweaving according to one aspect. FIG. 14 illustrates parse interweaving 1400, which includes a parse tree for a request 1401 and a parse tree for a corresponding response 1402.

[0122] Parsetree 1401 poses the following question: "I just had a baby and it looks more like the husband I had my baby with. However it does not look like me at all and I am scared that he was cheating on me with another lady and I had her kid. This child is the best thing that has ever happened to me and I cannot imagine giving my baby to the real mom." Response 1402 represents the following response: "Marital therapists advise on dealing with a child being born from an affair as follows. One option is for the husband to avoid contact but just have the basic legal and financial commitments. Another option is to have the wife fully involved and have the baby fully integrated into the family just like a child from a previous marriage."

[0123] Figure 14 shows a greedy approach to representing linguistic information about a paragraph of text. The straight edges of this graph are syntactic relations, and the curved arcs are discourse relations such as anaphora, same-entity, subentity, rhetorical relations, and communicative acts. The solid arcs are for same-entity / subentity / anaphora relations, and the dotted arcs are for rhetorical relations and communicative acts. The oval labels on the straight edges indicate syntactic relations. Lemmas are written in boxes for the nodes, and the lemma forms are written to the right of the nodes.

[0124] The parse chiasm 1400 contains much richer information than simply combining the parse trees for individual sentences. Navigating through this graph along edges for syntactic relations and arcs for discourse relations allows a given parse chiasm to be converted into a semantically equivalent form for matching with other parse chiasms and performing text similarity assessment tasks. As many links as possible are represented to form a complete formal representation of the paragraph. Each discourse arc generates a pair of parse phrases that can be potential matches.

[0125] The topical similarity between the seed (request) and the response is represented as a common subgraph of the parsing intersection, visualized as a connected cloud. The greater the number of common graph nodes, the greater the similarity. For rhetorical matches, the common subgraph need not be large in a given text. However, the rhetorical relations and communicative actions of the seed and response must be interrelated and correspond.

[0126] Generalizations about communication behavior The similarity between two communicative actions A1 and A2 is defined as an abstract verb that possesses features common to A1 and A2. Defining the similarity between two verbs as an abstract verb-like structure supports inductive learning tasks such as evaluating rhetorical agreement. In one example, the similarity between two common verbs, agree and disagree, can be generalized as follows: agree ^ disagree = verb(Interlocutor, Proposed_action, Speaker). In this case, the Interlocutor is the person who proposes the Proposed_action to the Speaker, to whom the Speaker conveys his or her response. Furthermore, the proposed_action is the action the Speaker would perform if accepting or rejecting the request or proposal, and the Speaker is the person to whom a particular action is proposed and who responds to the request or proposal made.

[0127] In yet another example, the similarity between the verbs agree and explain is as follows: agree ^ explain = verb(Interlocutor, *, Speaker). The subject of a communicative action is generalized in the context of the communicative action, but not in other "physical" actions. Thus, an aspect generalizes individual occurrences of a communicative action with their corresponding subject.

[0128] Additionally, a sequence of communicative acts that represents a dialogue can be compared with other such sequences of similar dialogues. In this manner, the dynamic discourse structure of the dialogue as well as the meaning of individual communicative acts (as opposed to its static structure reflected by rhetorical relations) are represented. Generalization is a complex structural representation that occurs at each level: the lemmas of communicative acts are generalized by lemmas, and their semantic roles are generalized by their respective semantic roles.

[0129] Communicative acts are used by text authors to indicate the structure of a dialogue or conflict. See Searle, JR (1969), "Speech Acts: An Essay in the Philosophy of Language" (Cambridge University Press). The subject is generalized in the context of these acts, not in other "physical" actions. Thus, individual occurrences of communicative acts are generalized in their subjects and in pairs of them as discourse "steps."

[0130] The generalization of communicative actions can also be considered in terms of matching verb frames such as VerbNet. Communicative links reflect discourse structures related by participation (or mention) rather than by a single agent in a text. Links form sequences that connect words about communicative actions (verbs or multiple words that implicitly indicate one's communicative intention).

[0131] A communicative act includes an actor, one or more agents acting on the actor, and a phrase that describes the characteristics of this act. A communicative act can be described as a function of form: verb (agent, subject, cause). In this case, the verb characterizes some type of interaction between the agents involved (e.g., explain, confirm, remind, disagree, deny, etc.). The subject refers to the information conveyed or the object stated. The cause refers to the motivation or explanation for the subject.

[0132] A scenario (a labeled directed graph) is a subgraph of the parsed intersection G = (V, A), where V = {action1, action2, ...action n} is a finite set of vertices corresponding to communication actions, and A is a finite set of labeled arcs (ordered pairs of vertices) classified as follows:

[0133] Each arc action i , action j ∈A sequence is a set of questions about the same subject (e.g., s j =s i or various subjects) refer to two actions v i , ag i , s i , c i and v j , ag j , s j , c j Each arc corresponds to an action i , action j ∈A cause is action i The cause of the action j action that shows conflict with the subject or cause of i and action j This corresponds to the attacking relationship with

[0134] The subgraph of a parse intersection associated with a scenario of interaction between agents has several notable characteristics, such as: (1) all vertices are time-ordered, and therefore every vertex (except the initial and terminal vertices) has one incoming arc and one outgoing arc; sequence For arcs, at most one incoming arc and one outgoing arc are allowed. (3)A cause In the case of arcs, there can be many incoming arcs as well as many outgoing arcs from a given vertex. The vertices involved may be associated with different agents or the same agent (i.e., if this agent contradicts itself). To compute the similarity between the parsing intersections and their communication behaviors, the induced subgraphs, the subgraphs with similar labels and the same configuration of arcs, and the strict correspondence of vertices are analyzed.

[0135] The following similarities exist by analyzing the arcs of communicative actions in Peircean crossovers: (1) one communicative action with a theme from T1 compared to another communicative action with a theme from T2 (arc of communicative actions is not used), and (2) one pair of communicative actions with a theme from T1 compared to another pair of communicative actions from T2 (arc of communicative actions is used).

[0136] The generalization of two different communicative actions is based on their attributes, see Galitsky et al. (2013). As can be seen from the example illustrated in Figure 14, one communicative action from T1, namely cheating(husband, wife, another lady), can be compared with a second communicative action from T2, namely avoid(husband, contact(husband, another lady). The generalization results in a communicative_action(husband,*), which introduces a constraint on A of the form: if a given agent (= husband) is mentioned as the subject of a CA in Q, then he / she should also be the subject of (possibly another) CA in A. Two communicative actions can always be generalized, but this is not the case with their subjects. If the result of their generalization is empty, then the result of the generalization of communicative actions with these subjects is also empty.

[0137] Generalization of the RST relation Some relations between discourse trees can be generalized, such as arcs that represent the same type of relation (denotation relations such as contrast, thematic relations such as conditional, and multi-nuclear relations such as list). A nucleus or a situation indicated by a nucleus is denoted by "N". A satellite or a situation indicated by a satellite is denoted by "S". "W" denotes the writer. "R" denotes the reader (listener). Situations are proposals, completed or ongoing actions, and communicative actions and states (including beliefs, desires, approve, explain, reconcile, etc.). The generalization of the two RST relations with the above parameters is expressed as follows: rst1(N1,S1,W1,R1) ^ rst2(N2,S2,W2,R2)= (rst1 ^ rst2)(N1 ^ N2,S1 ^ S2,W1 ^ W2,R1 ^ R2). The texts in N1, S1, W1, and R1 are generalized as phrases. For example, rst1 ^ rst2 can be generalized as follows: (1) If relation_type(rst1) != relation_type(rst2), the generalization is empty. (2) Otherwise, the signature of the rhetorical relation is generalized as a sentence.

[0138] sentence(N1,S1,W1,R1) ^ sentence(N2,S2,W2,R2) See Iruskieta, Mikel, Iria da Cunha, and Maite Taboada, "A qualitative comparative method for rhetorical structures: identifying different discourse structures in multilingual corpora" (Lang Resources & Evaluation. June 2015, Volume 49, Issue 2).

[0139] For example, the meaning of rst-background ^ rst-enablement= (S increases the ability of R to comprehend an element in N) ^ (R comprehending S increases the ability of R to perform the action in N) = increase-VB the-DT ability-NN of-IN R-NN to-IN.

[0140] Also, since the rst-background ^ rst-enablement relations vary, the RST relation part is empty. Then, expressions that are verbal definitions of each RST relation are generalized. For example, for each word, or for a placeholder for a word such as agent, this word (with its POS) is kept if it is the same in each input phrase, and excluded if it is different between these phrases. The resulting expressions can be interpreted as the common meaning between two different formally obtained definitions of the RST relation.

[0141] The two arcs between the question and answer shown in Figure 14 represent generalization instances based on the RST relation "RST-contrast." For example, "I just had a baby" is an RST-contrast with "it does not look like me," which in turn is related to "husband to avoid contact," which is an RST-contrast with "have the basic legal and financial commitments." As can be seen from the above, the answer does not necessarily have to be similar to the verb phrase in the question, but the rhetorical structures of the question and answer are similar. Not all phrases in the answer necessarily match phrases in the question. For example, mismatched phrases have a specific rhetorical relationship with phrases in the answer that are related to phrases in the question.

[0142] Building a discourse tree for communication 15 illustrates an exemplary process for building a communicative discourse tree according to one aspect. The application 102 can implement the process 1500. As discussed above, a communicative discourse tree enables improved search engine results.

[0143] In block 1501, process 1500 includes accessing sentences containing fragments. At least one fragment includes a verb and multiple words. Each word includes the roles of the multiple words within the fragment. Each fragment is a basic discourse unit. For example, application 102 accesses a sentence such as "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired," as described in connection with FIG. 13.

[0144] Continuing with the above example, application 102 determines that a sentence contains several fragments. For example, a first fragment is "Rebels, ..., deny." A second fragment is "that they controlled the territory." A third fragment is "from which the missile was allegedly fired." Each fragment contains a verb (e.g., "deny" for the first fragment and "controlled" for the second fragment). However, a fragment need not contain a verb.

[0145] At block 1502, process 1500 includes generating a discourse tree representing rhetorical relationships between the sentence fragments. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two of the sentence fragments, and each terminal node of the discourse tree is associated with one of the sentence fragments.

[0146] Continuing with the above example, application 102 generates a discourse tree such as that shown in Figure 13. For example, the third fragment, "from which the missile was allegedly fired," recounts "that they controlled the territory." Both the second and third fragments concern attribution of what happened (i.e., the attack could not have been insurgent because insurgents did not control the territory).

[0147] In block 1503, process 1500 includes accessing a number of verb signatures. For example, application 102 accesses a list of verbs, such as from VerbNet. Each verb matches or is related to the verb of the fragment. For example, for the first fragment, the verb is "deny." Therefore, application 102 accesses a list of verb signatures related to the verb "deny."

[0148] As described above, each verb signature includes one or more of the verb and thematic roles of the fragment. For example, a signature may include one or more of a noun phrase (NP), a noun (N), a communicative act (V), a verb phrase (VP), or an adverb (ADV). Thematic roles describe the relationship between a verb and associated words. For example, "The teacher amused the children" has a different signature from "small children amuse quickly." For the first fragment, the verb "deny," the application 102 accesses a list of frames or verb signatures for verbs that match "deny." The list is "NP V NP to be NP," "NP V that S," and "NP V NP."

[0149] Each verb signature includes a thematic role. The thematic role refers to the role of the verb in the sentence fragment. The application 102 determines the thematic role in each verb signature. Exemplary thematic roles include "actor," "agent," "asset," "attribute," "beneficiary," "cause," "location destination source," "destination," "source," "location," "experiencer," "extent," "instrument," "material and product," "material," "product," "patient," "predicate," "recipient," "stimulus," "theme," "time," or "topic."

[0150] At block 1504, the process 1500 includes determining, for each one of the verb signatures, the number of thematic roles of each signature that match the roles of the words in the fragment. For the first fragment, the rhetorical classification application 102 determines that the verb "deny" has only three roles: "agent," "verb," ​​and "theme."

[0151] At block 1505, process 1500 includes selecting a particular verb signature from the verb signatures based on the particular verb signature with the most matches. For example, referring again to Figure 13, in the first fragment, "the rebels deny that they controlled the territory," "deny" matches the verb signature deny "NP V NP," and "control" matches control(rebels, territory). Verb signatures are nested, resulting in the nested signature "deny(rebel, control(rebel, territory))."

[0152] Request-Response Representation Request-response pairs can be analyzed singly or as pairs. In one example, request-response pairs can be spliced ​​together. When splicing together, rhetorical agreement is expected to hold not only between consecutive members, but also between triplet members and 4-tuple members. Discourse trees can be constructed for texts that represent sequences of request-response pairs. For example, in the domain of customer complaints, the request and response are in the same text from the perspective of the complaining customer. A customer complaint text can be divided into request and response text portions, which can then form pairs of positive and negative data sets. In one example, all text about the advocate and all text about the opponent are combined. The first sentence of each of the following paragraphs would form the request portion (which would contain three sentences), and the second sentence of each paragraph would form the response portion (which in this example would contain three sentences).

[0153] Figure 16 shows a discourse tree and a scenario graph according to one aspect. Figure 16 shows a discourse tree 1601 and a scenario graph 1602. The discourse tree 1601 corresponds to the following three sentences:

[0154] (1) I explained that my check bounced (I wrote it after I made a deposit). A customer service representative accepted that it usually takes some time to process the deposit.

[0155] (2) I was reminded that I was unfairly charged an overdraft fee a month ago in a similar situation. They denied that it was unfair because the overdraft fee was disclosed in my account information.

[0156] (3) I disagreed with their fee and wanted this fee deposited back to my account. They explained that nothing can be done at this point and that I need to look into the account rules more closely.

[0157] As can be seen from the discourse tree in Figure 16, determining whether a text represents a dialogue or an explanation can be difficult. Therefore, by analyzing the arcs of communicative action in the Peircean intersection, implicit similarities between texts can be discovered. For example, in general terms, (1) One communicative action with a theme from the first tree against another communicative action with a theme from the second tree (arcs of communicative actions are not used).

[0158] (2) one pair of themed communicative actions from the first tree against another pair of communicative actions from the second tree (arcs of communicative actions are used);

[0159] For instance, in the example above, a generalization of cheating(husband, wife, another lady) ^ avoid(husband, contact(husband, another lady)) gives us communicative_action(husband, *), which introduces a constraint on A of the form: if a given agent (=husband) is mentioned as the subject of a CA in Q, then he / she should also be the subject of (possibly another) CA in A.

[0160] To handle the meaning of words that represent the subject of CA, a word can be applied to a vector model such as the "word2vector" model. More specifically, to compute the generalization between themes of communicative behavior, the following rule can be used: if subject1 = subject2, then subject1^subject2 =<subject1, POS(subject1), 1> In this case, the subject remains and the score is 1. Otherwise, if the subjects have the same part-of-speech (POS), then subject1^subject2 = <*, POS(subject1), word2vecDistance(subject1^subject2)>, where * indicates that the lemma is a placeholder and the score is the word2vec distance between these words. If the POS are different, the generalization is an empty tuple and there may be no further generalization.

[0161] Classification settings for request-response pairs In traditional search, as a baseline, agreement between request-response pairs can be measured in terms of keyword statistics such as TF*IDF, which stands for term frequency-inverse document frequency. To improve search relevance, this score is augmented by item popularity, item location, or taxonomy-based scores (Galitsky (2015)). Search can also be formulated as a path reranking problem in a machine learning framework. A feature space contains request-response pairs as elements, and a separating hyperplane divides this feature space into correct and inaccurate pairs. Therefore, the search problem can be formulated in a local way, as the similarity between Request and Response, or in a global way, as the similarity between request-response pairs.

[0162] Other methods for determining a match between a request and a response are possible. In a first example, the application 102 extracts features for Req and Resp, compares these features as counts, and introduces a scoring function so that the score may indicate a class (low score for an inexact pair, high score for an exact pair).

[0163] In a second example, the application 102 compares the expressions for Req and Resp with each other and assigns a score to the comparison. Similarly, the score may indicate a class.

[0164] In a third example, the application 102 generates a pair of Req and Resp<Req,Resp> The application 102 then constructs a representation for all such elements<Req,Resp> Learning is performed in the feature space of

[0165] 17 illustrates the formation of a request-response pair according to one aspect. FIG. 17 shows a request-response pair 1701, a request tree (or object) 1702, and a response tree 1703.<Req,Resp> To form the object of, application 102 combines the discourse tree for the request and the discourse tree for the response into a single tree with a root RR. Application 102 then categorizes the object into exact (high match) and inexact (low match) categories.

[0166] Nearest Neighbor Graph Based Classification Once a CDT is constructed, to identify arguments in the text, the application 102 computes its similarity compared to the CDT for the positive class and verifies that it is low enough to reach the set of CDTs for the negative class. The similarity between the CDTs is defined by the largest common sub-CDT.

[0167] In one example,

[0168]

number

[0169] An ordered set G of CDTs (V,E) with vertex and edge labels from G is constructed. A labeled CDT Γ from G is a pair of the multi-pair form ((V,l),(E,b)), where V is a set of vertices and E is a set of edges,

[0170]

number

[0171] is a function that assigns labels to vertices, and b:E→Λ E is a function that assigns labels to edges. Isomorphic trees with identical labels are not distinguished.

[0172] The ordering is defined as follows: for two CDTs, from G, Γ1:=((V1,l1),(E1,b1)) and Γ2:=((V2,l2),((E2,b2))), and further, Γ1 dominates Γ2 or Γ2≦Γ1 if there exists a one-to-one mapping φ:V2 → V1 (or Γ2 is a sub-CDT of Γ1). Thus, this means that

[0173]

number

[0174] Taking into consideration the above, (2)

[0175]

number

[0176] The following applies: This definition allows for the computation of the similarity ("weakening") of labels between matched vertices when going from a "larger" CDT G1 to a "smaller" CDT G2.

[0177] Here, the similarity CDT Z of a pair of CDTs X and Y, denoted by X^Y=Z, is the set of all containment-maximum common sub-CDTs of X and Y, each of which satisfies the following additional conditions to match: Specifically, (1) two vertices from CDT X and CDT Y must exhibit the same RST relation, and (2) each common sub-CDT from Z contains at least one communication action with the same VerbNet signature as in X and Y.

[0178] This definition is easily extended to find generalizations for several graphs: the subsumption order for a pair of graph sets X and Y is naturally defined as XμY:=X*Y=X.

[0179] Figure 18 shows the discourse tree for the maximal common subcommunication according to one aspect. Note that the tree has been inverted and the arc labels have been generalized. The communicative action site() generalizes with the communicative action say(). The first (agent) argument of the former CA Committee generalizes with the first argument of the latter CA Dutch. The same operation applies to the second argument of this pair of CAs: investigator ^ evidence.

[0180] Since CDT U belongs to the positive class, (1) U is a positive example R + (having a non-empty common sub-CDT) and (2) any negative example R - Even if U is

[0181]

number

[0182] If it is similar to U*R - μU*R + This becomes: This condition introduces a similarity criterion, stating that to be assigned to a class, the similarity between an unknown CDT U and the closest CDT from the positive class should be higher than the similarity between U and each negative example. Condition 2 also states that for a positive example R + exists, and therefore, - If there is no U*R + μR - This suggests that we have ∇ ...

[0183] Interlaced kernel learning for CDT Tree kernel learning for strings, parse trees, and parse intersection is a recently established research area. The parse tree kernel counts the number of common subtrees between two instances as a measure of discourse similarity. The tree kernel is defined for DT by Joty, Shafiq, and A. Moschitti in "Discriminative Reranking of Discourse Parses Using Tree Kernels" (Proceedings of EMNLP; 2014). See also Wang, W., Su, J., & Tan, C. L. (2010) in "Kernel Based Discourse Relation Recognition with Temporal Ordering Information" (In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics) (which uses a special form of tree kernel for discourse relation recognition). The intersection kernel is defined for CDT by augmenting the DT kernel with information about communicative behavior.

[0184] The CDT can be represented by a vector V of integer counts of each subtree type (without taking into account its ancestors): V(T) = (# of type 1 subtrees, …, # of type I subtrees, …, # of type n subtrees). This results in a very high dimensionality, as the number of distinct subtrees is exponential in its size. Hence, the feature vector

[0185]

number

[0186] It is computationally infeasible to use it directly. To solve the computational problem, a tree kernel function is introduced to efficiently compute the dot product between the above high-dimensional vectors. Assuming there are two tree segments CDT1 and CDT2, the tree kernel function is defined as follows: K(CDT1,CDT2)=<V(CDT1),V(CDT2)> =ΣiV(CDT1)[i],V(CDT2)[i]=Σn1Σn2 Σi Ii(n1) * Ii(n2) In this case, n1∈N1 and n2∈N2, where N1 and N2 are the sets of all nodes in CDT1 and CDT2, respectively.

[0187] Ii(n) is the indicator function. Ii(n) = {1 if a subtree of type i occurs at the root at node, 0 otherwise}. K(CDT1,CDT2) is an instance of the convolution kernel for tree structures (Collins and Duffy, 2002), which can be computed by the recursive definition.

[0188] Δ(n1,n2)=ΣI Ii(n1) * Ii(n2) Here, Δ(n1, ​​n2)=0 if n1 and n2 are assigned the same POS tag or their children are different subtrees.

[0189] Otherwise, if n1 and n2 are both POS tags (pre-terminal nodes), then Δ(n1, ​​n2)=1×λ. In other cases:

[0190]

number

[0191] In this case, ch(n,j) is the jth child of node n, nc(n1) is the number of children of n1, and λ (0<λ<1) is a decay factor that makes the kernel value less variable with subtree size. Additionally, the recursive rule (3) holds because, given two nodes with the same children, we can use these children to construct a common subtree as well as common subtrees of further descendants. The parsetree kernel counts the number of common subtrees as a syntactic similarity measure between two instances.

[0192] FIG. 19 illustrates a tree in kernel learning format for a communicative discourse tree according to one aspect.

[0193] Terms for communicative actions as labels are converted into a tree that is added to the respective nodes for RST relations. For texts for EDUs as labels for terminal nodes, only phrase structure is preserved. Terminal nodes are labeled with sequences of phrase types rather than parse tree fragments.

[0194] If there is a rhetorical relation arc from node X to terminal EDU node Y, labeled A(B,C(D)), then the subtree AB→(CD) is attached to X.

[0195] Implementing a Rhetorical Match Classifier The rhetorical match classifier 120 can determine the complementarity between two sentences, such as a question and an answer, by using a communicative discourse tree. Figure 20 illustrates an exemplary process used to implement the rhetorical match classifier, according to one aspect. Figure 20 illustrates a process 2000 that can be implemented by the application 102. As described above, the rhetorical match classifier 120 is trained with training data 125.

[0196] The rhetorical match classifier 120 determines communicative discourse trees for both questions and answers. For example, the rhetorical match classifier 120 builds a question communicative discourse tree from a question, such as question 171 or input text 130, and builds an answer communicative discourse tree from candidate answers.

[0197] In block 2001, the process 2000 includes determining a discourse tree for question communication that includes a question root node for a question sentence. The question sentence can be a clear question, a request, or a comment. The application 102 creates the discourse tree for question communication 110 from the input text 130. Using the example described in connection with FIGS. 13 and 15, an exemplary question sentence is "are rebels responsible for the downing of the flight." The application 102 can use the process 1500 described in connection with FIG. 15. The exemplary question has a root node of "elaborate."

[0198] At block 2002, the process 2000 includes determining a second communicative discourse tree for the answer sentence. In this case, the answer communicative discourse tree includes an answer root node. Continuing with the above example, the application 102 creates a communicative discourse tree 111, as shown in FIG. 13, that also has a root node "elaborate."

[0199] At block 2003, process 2000 includes associating the communicative discourse trees by identifying that the question root node and the answer root node are identical. The application 102 determines that the question communicative discourse tree and the answer communicative discourse tree have the same root node. The resulting associated communicative discourse tree is shown in FIG. 17 and may be labeled as a "request-response pair."

[0200] At block 2004, the process 2000 includes computing a level of complementarity between the question communicating discourse tree and the answer communicating discourse tree by applying a predictive model to the merged discourse trees.

[0201] The rhetorical match classifier uses machine learning techniques. In one aspect, the application 102 trains and uses the rhetorical match classifier 120. For example, the application 102 defines positive and negative classes of request-response pairs. The positive class includes rhetorically accurate request-response pairs, and the negative class includes related but rhetorically disparate request-response pairs.

[0202] For each request-response pair, the application 102 builds a CDT by parsing each sentence and obtaining the verb signature for the sentence fragment.

[0203] The application 102 provides the associated communicative discourse tree pairs to a rhetorical match classifier 120. The rhetorical match classifier 120 outputs a level of complementarity.

[0204] At block 2005, process 2000 includes identifying the question sentence and the answer sentence as complementary in response to determining that the level of complementarity is above a threshold. Application 102 may use the threshold level of complementarity to determine whether a question-answer pair is sufficiently complementary. For example, if the classification score is greater than a threshold, application 102 may output the answer as answer 172 or answer 150. Alternatively, application 102 may discard the answer and access answer database 105 or a public database for another candidate answer, repeating process 2000 as needed.

[0205] In one aspect, the application 102 obtains co-references. In a further aspect, the application 102 obtains entity and subentity or hyponym links. A hyponym is a word that has a more specific meaning than the general or superordinate term that can apply to a word. For example, "spoon" is a hyponym of "cutlery."

[0206] In another aspect, the application 102 applies interlaced kernel learning to the representation. Interlaced kernel learning can be performed, for example, in place of classification-based learning in block 2004. The application 102 constructs a parse interlaced pair for the parse tree of the request-response pair. The application 102 parses the discourse to obtain a discourse tree pair for the request-response pair. The application 102 aligns the discourse tree request-response and the parse tree request-response basic discourse units. The application 102 merges the discourse tree request-response basic discourse units with the parse tree request-response basic discourse units.

[0207] In one aspect, the application 102 improves text similarity assessment with a word2vector model.

[0208] In a further aspect, the application 102 sends sentences corresponding to the discourse tree for communicating a question or sentences corresponding to the discourse tree for communicating an answer to a device such as a mobile device 170. The output from the application 102 can be used as input to a search query, database lookup, or other system. In this manner, the application 102 can be integrated with a search engine system.

[0209] 21 illustrates a chatbot commenting on a post, according to one aspect. FIG. 21 illustrates a chat 2100, user messages 2101-2104, and an agent response 2105. The agent response 2105 can be implemented by the application 102. As shown, the agent response 2105 has identified an appropriate response to the thread of user messages 2101-2104.

[0210] FIG. 22 illustrates a chatbot commenting on a post, according to one aspect. FIG. 22 illustrates a chat 2200, user messages 2201-2205, and an agent response 2206. FIG. 22 illustrates three messages from User 1, specifically 2201, 2203, and 2205, and two messages from User 2, specifically 2202 and 2204. The agent response 2206 can be implemented by the application 102. As shown, the agent response 2206 identified an appropriate answer to the thread of messages 2201-2204.

[0211] The features shown in Figures 21 and 22 may be implemented by the computing device 101 or by an apparatus that provides input text 130 to the computing device 101 and receives answers 150 from the computing device 101.

[0212] Additional Rules for RR Conformity and RR Irrationality Below is an example of a structural rule that introduces constraints to enforce RR matching: 1. Both Req and Resp have the same sentiment polarity (if the request is positive, the response should be positive as well, and vice versa).

[0213] 2. Both Req and Resp have logical arguments. Under rational reasoning, requests and responses would be perfectly consistent. A rational agent would offer an answer that would be relevant and consistent with the rhetoric of the question. However, in the real world, not all responses are fully rational. The body of research on cognitive biases examines people's tendency to think in particular ways that may systematically deviate from standards of rationality or preferred judgment.

[0214] Correspondence bias is the tendency of people, when responding to questions, to overemphasize personality-based explanations for behaviors observed in others (see Baumeister, RF & Bushman, BJ, Social psychology and human nature (International Edition: 2010)). At the same time, people responding to queries underemphasize the role and power of situational influences on the same behavior.

[0215] Confirmation bias is the tendency to seek out or interpret information in a way that confirms the preconceived notions of people answering questions. They may distrust information that does not support their point of view. Confirmation bias is related to the concept of cognitive dissonance, whereby individuals may reduce discrepancies by seeking out information that reconfirms their point of view.

[0216] Anchoring involves relying too heavily on or "anchoring" one characteristic or piece of information when making a decision.

[0217] The availability heuristic causes us to overestimate the likelihood of an event occurring in memory with a greater "availability," which can be affected by how recent the memory is or how abnormally or emotionally altered it may be.

[0218] Following the bandwagon effect, people answer questions believing that many other people will do (or believe) the same thing.

[0219] Belief bias is the effect whereby someone's assessment of the logical strength of an argument is biased by the credibility of its conclusion.

[0220] A bias blind spot is the tendency to view oneself with less bias toward oneself than toward other people, or to be able to identify others with higher cognitive biases than oneself.

[0221] evaluation The first domain of test data comes from question-answer pairs from Yahoo! Answers, which pair questions and answer sets across a wide range of topics. 20,000 questions containing more than two sentences were selected from the question sets of 4.4 million users. Because the answers to the majority of questions are adequately detailed, no filtering was applied to the answers. There are multiple answers per question, and the best answer is marked. Question-best answer pairs are considered as elements of the positive training set, and question-other answers are considered as elements of the negative training set. To derive the negative set, answers to different but somewhat related questions were arbitrarily selected, or queries were formed from the questions and answers were obtained from web search results.

[0222] Our second dataset involves social media. We extracted request-response pairs primarily from posts on Facebook®. We also used a smaller portion of employment-related conversations on LinkedIn.com and vk.com. In the social domain, the standards for description are quite low. The cohesion of text is very limited, and logical structure and relevance are often lacking. The authors formed a training set over several years from their own accounts and public Facebook accounts available via API (at the time of writing, it is not possible to write to the Facebook API to retrieve messages). In addition, we used 860 email threads from the Enron dataset. Furthermore, we collected data on manual responses to posts from an agent that automatically generates posts on behalf of human users. See Galitsky B., Dmitri Ilvovsky, Nina Lebedeva, and Daniel Usikov, "Improving Trust in Automation of Social Promotion: AAAI Spring Symposium on The Intersection of Robust Intelligence and Trust in Autonomous Systems; Stanford, CA 2014" ("Galitsky 2014"). We formed 4,000 pairs from various social network sources.

[0223] The third domain is customer complaints. In a typical complaint, dissatisfied customers describe their problems with products and services, as well as the process by which they attempted to communicate these problems to the company and the company's response. Complaints are often written with bias, exaggerating product failures and portraying the other party's behavior as unjustified and inappropriate. At the same time, complainants attempt to write their complaints in a coherent, logically consistent manner (Galitsky 2014). Thus, complaints serve as a domain with a high match rate between requests and responses. To assess the match between users' complaints and company responses (according to how they are written), we collected 670 complaints from planetfeedback.com over a 10-year period.

[0224] The fourth domain is journalistic interviews. Interview accounts by professional journalists are usually written in such a way that the level of agreement between questions and answers is very high. We also collected 1200 contributions by professional and citizen journalists from sources such as datran.com, allvoices.com, huffingtonpost.com, etc.

[0225] To facilitate data collection, we designed a crawler that searches a specific set of sites, downloads web pages, extracts candidate text, and verifies that the candidate text conforms to the format of a response to a question or request. Each pair of texts is then formed. This search is realized by the Bing Azure Search Engine API in the Web and News domains.

[0226] Recognizing valid and invalid answers The accuracy of response classification is shown in Table 1. Each row represents a specific method. Each class of method is shown in the gray area.

[0227] [Table 3]

[0228] It can be seen that the highest accuracy is achieved in the journalism and community response domains, and the lowest accuracy is achieved in customer complaints and social networks. We can conclude that the higher the accuracy of the method, the higher the level of agreement between Req and Resp, and accordingly, the higher the aptitude of the respondent.

[0229] The deterministic family of approaches (the middle two rows, local RR similarity-based classification) performs about 9%, lower than SVM TK. This indicates that the similarity between Req, Res, and p is substantially less important than the specific structure of the RR pair that indicates RR match. This means that we cannot assess the match between Req and Resp on an individual basis. If we require DT(Req) to be very similar to DT(Resp), we will obtain reasonable precision but very low recall. Progressing from DT to CDT only helps by 1%-2%, because communicative behavior no longer plays a primary role in structuring requests or shaping responses.

[0230] For the statistical family of approaches (bottom five rows, tree kernels), the richest source of discourse data (SVM TK for RR-DT) gives the best classification accuracy, nearly identical to RR similarity-based classification. SVM TK for RST and CA (full parse trees) included more linguistic data, but some part of it (most likely syntactic) was redundant, leading to poorer results on the limited training set. Using additional features under TK, such as sentiment and argument, does not help. In most cases, it is likely that these features are derived from RR-CDT features and do not contribute to classification accuracy by themselves.

[0231] By employing the TK family of CDT-based approaches, we obtain accuracies comparable to those achieved in classifying DTs as accurate and incorrect. In rhetorical parsing tasks, state-of-the-art systems have been in fierce competition over the past few years, yielding accuracies of over 80%.

[0232] The performance of direct analytical approaches in the deterministic family is somewhat weaker. This means that a larger number of more complex feature structures are required, and simply counting and taking into account the types of rhetorical relations is insufficient to determine how RRs match each other. If two RR pairs have the same types and counts of rhetorical relations and even communicative actions, they can still belong to the opposite RR match class in the majority of cases.

[0233] Although nearest-neighbor pair training on the CDT yields lower accuracy than SVM TK on the CDT, the former provides interesting examples of subtrees that are typical of arguments and shared in similar fact data. The number of CDT subtrees in the former group is naturally much larger. Unfortunately, the SVM TK approach does not help explain how exactly the RR matching problem is solved, merely showing the final scoring and class labels. It is also possible to express logical arguments in responses without communicative actions, but this is rare (this observation is supported by our data).

[0234] Measuring RR agreement in the evaluation domain From the perspective of evaluating recognition accuracy, we obtained the optimal method in the previous subsection. Now, by finalizing this method, we will measure RR agreement in our evaluation domain. Furthermore, we will show how the general agreement total provided by the optimal method is correlated with individual agreement criteria such as sentiment, logical argument, topic, and keyword relevance. When we use our optimal approach (SVM TK for RR-CDT) to label the training set, its size can be dramatically increased, allowing us to explore interesting properties of RR agreement in various domains. We will discover the contribution of several intuitive features to RR agreement on a larger dataset than previous evaluations.

[0235] In this subsection, we intend to prove that the RR pair validity epistemic framework can serve as a criterion for matching between any request and response. Furthermore, this epistemic framework can assess how strongly various features are correlated with RR pair validity.

[0236] From the evaluation of recognition accuracy, we have obtained an optimal method for recognizing whether an RR pair is valid. Now, with this recognition method established, we will measure RR agreement in our evaluation domain and further estimate how the total general agreement provided by the optimal method correlates with individual agreement criteria, such as sentiment, logical argument, topic, and keyword relevance. When we use our optimal approach to label the training set (SVM TK for RR-CDT), its size can be dramatically increased, allowing us to explore interesting properties of RR agreement in various domains. We will discover the contribution of some intuitive features to RR agreement on a larger dataset than previous evaluations. We will measure this agreement, feature-by-feature, on only the positive training dataset from the above evaluation as recognition accuracy (%, Table 2). Note that recall and negative datasets are not required for agreement assessment.

[0237] [Table 4]

[0238] For example, we estimate the accuracy of observing that RR pairs determined by topic matching as computed by the bag-of-words approach are valid RR pairs in the domain of customer complaints according to SVM TK for RR-CDT classification to be 64.3%.

[0239] Matching by affect indicates the contribution of appropriate affective agreement in an RR pair. Affective rules include, among other things, that when the polarity of the RRs is the same, the response should confirm what the request states. Conversely, when the polarity is reversed, the response should attack what the request asks for. Matching by logical argument requires appropriate communicative discourse in which the response does not match the assertions in the request.

[0240] This data helps clarify the nature of the linguistic match between what the supporter is saying and how the other party is responding. Not all matching features need to be present for a valid dialogue discourse. However, if most of these features do not match, the given response should be considered invalid and inappropriate, and a different response should be selected. Table 2 shows us which features, and to what extent, should be used to support a dialogue in various domains. Therefore, the proposed technique can serve as an automated means of assessing writing quality and customer support quality.

[0241] Chatbot Applications A Conversational Agent for Social Promotion (CASP) is an agent that serves as a simulated personality acting on behalf of its human host to facilitate and manage communication about him or her. Galitsky B., Dmitri Ilvovsky, Nina Lebedeva, and Daniel Usikov, "Improving Trust in Automation of Social Promotion," AAAI Spring Symposium on The Intersection of Robust Intelligence and Trust in Autonomous Systems; Stanford, CA 2014. CASP relieves the human host from routine, non-critical activities on social networks, such as sharing news and commenting on other people's messages, blogs, forums, images, and videos. Conversational agents for social promotion are evolving, but they can also lose trust. The overall performance of CASP, focused on RR pair matching to filter responses mined from the web, is evaluated.

[0242] On average, people have 200 to 300 friends or contacts on social networking systems such as Facebook and LinkedIn. Maintaining active relationships with this large number of friends requires several hours per week to read their posts and comments. In reality, people maintain relationships with only 10 to 20 of their closest friends, family, and colleagues, and rarely communicate with the rest. These less-close friends may feel abandoned by their social network relationships. However, maintaining active relationships with all members of a social network is beneficial for many aspects of life, from work-related to personal matters. Social network users are expected to let their friends know they are interested in them and care about them, and therefore respond to events in their lives and to messages they post. Therefore, social network users need to devote a significant amount of time to maintaining relationships on the social network, which they often do not have the time for. For close friends and family, users will still socialize using manual input. For the rest of their network, they will use the proposed CASP for social promotion.

[0243] CASP tracks user chats, user posts on blogs and forums, and comments on shopping sites to suggest web documents and their snippets relevant to purchasing decisions. This involves quoting a section of text, generating a search engine query, running it against a search engine API such as Bing, and filtering out search results deemed irrelevant to the seed message. This last step is crucial for the robust functioning of CASP; poor relevance in a rhetorical space will lead to a loss of trust in that space. Therefore, accurate assessment of RR matches is crucial for the successful use of CASP.

[0244] CASP is presented as a simulated personality that acts on behalf of a host to facilitate and manage that person's communications (Figures 21-22). The agent is designed to relieve the host from routine, non-essential social networking activities, such as sharing news and commenting on other people's messages, blogs, forums, images, and videos. Unlike most application domains for simulated personalities, the social partners are not necessarily aware that they are exchanging news, opinions, and updates with an automated agent. We conducted experiments using CASP's rhetorical matching and inferences about the mental states of its peers within several Facebook accounts. We evaluate its performance and accuracy in inferring about the mental states related to the users with whom it communicates. For a conversational system, users need to feel that the system responds appropriately to their actions and that the replies it provides make sense. To achieve this in horizontal domains, linguistic information must be fully utilized to enable messages to be exchanged in a meaningful way.

[0245] CASP takes a seed (a post written by a person) as input and outputs a message that the seed generates by tailoring content found on the web to be relevant to the input post. This relevance is based on relevance in terms of content and relevance in terms of RR match or mental state match (e.g., it answers a question by asking a question, an answer to a recommended post asking for more questions, etc.).

[0246] 21 and 22 illustrate examples of a chatbot that comments on a post. We evaluate how much a human user loses trust in CASP and their host when both content and mental state are irrelevant. Instead of evaluating rhetorical relevance, which is an intermediate parameter in terms of system usefulness, we assess how much a user loses trust in CASP when they are harassed by rhetorically irrelevant and inappropriate posts.

[0247] [Table 5]

[0248] In Table 3, we present the results of users' tolerance for CASP failures. After several failures, friends lose trust, file complaints, remove themselves from their friends lists, share negative information about their loss of trust with others, and even encourage other friends to remove CASP-enabled friends from their friends lists. The values ​​in the cells indicate the average number of posts that undermined rhetorical relevance when each trust-loss event occurred. These undermined relevance posts occurred within one month of conducting this assessment, and we do not have values ​​for the relative frequency of their occurrence. Each user responded to an average of 100 posts (1–4 per seed post).

[0249] We can see that depending on different domains, there are different scenarios where users lose trust in CASP. For domains such as travel and shopping, where information is less critical, the tolerance for losing relevance is relatively high.

[0250] Conversely, in domains that are interpreted more seriously, such as work-related and personal preferences, users are more sensitive to CASP not working, and their various forms of trust are eroded more quickly.

[0251] For all domains, acceptability slowly decreases as the posting becomes more complex. Users' perceptions are more negative for longer texts compared to shorter, single sentence or phrase postings by CASP, regardless of content or their expectations.

[0252] Domain of natural language descriptions of algorithms Being able to map natural language to a formal query or command language is important for developing more user-friendly interfaces to many computing systems, such as databases. However, relatively little research has addressed the problem of learning such semantic parsers from corpora of sentences paired with their formal language equivalents. Kate, Rohit, Y. W. Wong, and R. Mooney, "Learning to transform natural to formal languages," (AAAI, 2005), is one example. Furthermore, to our knowledge, no such research has been conducted at the discourse level. Learning to transform natural language (NL) into a fully formal language can make it easier to develop NL interfaces to hybrid computing and AI systems.

[0253] More than 40 years ago, Dutch computer scientist Dijkstra, who coined the concept of "structured programming," said, "I wonder whether machines that must be programmed in our native language (whether Dutch, English, American English, French, German, or Swahili) will become as staggeringly complex as the machines they will use." This visionary was clearly correct: the technicality and precision of programming languages ​​is what has made enormous advances in computing and computers possible. Dijkstra compared the invention of programming languages ​​to the invention of mathematical symbolism. He also wrote, "Instead of considering the obligation to use formal codes as a burden, we should consider the convenience of using formal codes as a privilege. These codes enable children to learn early what only geniuses could achieve." Yet, 40-plus years later, we remain stuck with the volume of code that typical industrial applications occupy—tens or hundreds of millions of lines of code that are daunting to support and develop. The phrase "the code is the best writing" has become something of a bad joke.

[0254] Natural language descriptions of programs are a domain in which textual rhetoric is specialized and correspondence between descriptions is essential. We will focus on common rhetorical and domain-specific representations that map algorithmic descriptions to software code.

[0255] Figure 23 shows a discourse tree for an algorithmic text according to one aspect. We have the following text and its DT (Figure 23):

[0256] 1) Find an arbitrary pixel p1. 2) Find the convex area a_off to which this pixel p1 belongs such that all pixels are less than 128.

[0257] 3) Verify that the boundary of the selected area has all pixels greater than 128.

[0258] 4) If the above verification is successful, stop with a positive result. Otherwise, add all pixels that are less than 128 to a_off.

[0259] 5) Check if the size of a_off is less than the threshold. Then go to 2. Otherwise stop with a negative result.

[0260] We now show how to translate specific sentences into a logical form and then into a software code representation. Several rhetorical relations help to combine the descriptions resulting from the translation of the individual sentences.

[0261] Verify that the boundary of the selected area has more than 128 pixels in all.

[0262] Figure 24 shows an annotated sentence according to one aspect. Please refer to Figure 24 for an annotated deconstruction of the pseudocode of 1-1 to 1-3.

[0263] By converting all constants into variables, we try to minimize the number of free variables and at the same time not over-constrain the representation. Connected arrows (linked by edges) indicate that the same constant value (pixel) maps to an equal variable (pixel) according to the rules of logic programming. To achieve this, we add predicates (unary) that require us to constrain the free variables.

[0264] 1-4) Add a predicate that suppresses the free variable. epistemic_action(verify) & border(Area) & border(Pixel) & above(Pixel, 128) & area(Area) Here we need to construct an explicit expression to quantify everything, but in this particular case it will not be used since we use a loop structure.

[0265] Figure 25 shows an annotated sentence according to one aspect. See Figure 25 for an annotated deconstruction of the pseudocode in 1-5 through 2-3.

[0266] In the end, we have the following: 2-3) The resulting code fragment

[0267]

number

[0268] Related Work While discourse analysis has a limited number of applications in question answering and text summarization and generalization, we have not found any applications for automatically constructed discourse trees. We list research related to the application of discourse analysis to two areas: dialogue management and dialogue games. These areas may be applied to the same problems as the present proposal is intended for. Both of these proposals have a range of logic-based approaches as well as analytical- and machine learning-based approaches.

[0269] Managing dialogue and answering questions When questions and answers are logically related, the matching of their rhetorical structures becomes less important.

[0270] De Boni proposed a method for determining the appropriateness of an answer to a question by evidence of logical relevance rather than logical evidence of truth. See De Boni and Marco, "Using logical relevance for question answering" (Journal of Applied Logic, Volume 5, Issue 1, March 2007, Pages 92-103). We define logical relevance as the idea that answers should not be considered absolutely true or false with respect to a question, but should be considered true more flexibly along a sliding scale of relevance. Furthermore, it allows for rigorous reasoning about the appropriateness of an answer even when the answer's source is incomplete, inconsistent, or contains errors. The authors show how logical relevance can be realized in a relaxed form by using measured simplification to seek logical evidence that an answer is in fact the answer to a particular question.

[0271] Our model of CDT attempts to combine general rhetorical information and speech act information in a single structure. While speech acts provide a useful characterization of certain pragmatic forces, more recent work, particularly in building dialog systems, has significantly expanded this core concept to model many more kinds of conversational functions that can be performed by utterances. The resulting richness of actions is called dialog acts. See Jurafsky, Daniel, and Martin, James H. (2000), "Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition" (Upper Saddle River, NJ, Prentice Hall). In their multilevel approach to conversational acts, Traum and Hinkelman identify four levels of dialog acts necessary to ensure both the coherence and content of a conversation. See Traum, David R. and James F. Allen (1994), "Discourse Obligations in Dialogue Processing," in Proceedings of the 32nd annual meeting on Association for Computational Linguistics (ACL '94). Association for Computational Linguistics, Stroudsburg, PA, USA, 1-8. The four levels of conversational moves are redirection moves, reasoning moves, core speech moves, and argument moves.

[0272] Research into the logical and philosophical foundations of Q / A has been conducted for decades, but has focused on small, restricted domains and systems, proving limited use in industrial environments. The idea of ​​"being an answer to" logical evidence, developed in linguistics and mathematical logic, has been presented as having limited applicability in real systems. Most recent applied research aimed at generating general-purpose ("open-domain") working systems is based on relatively simple architectures that combine information extraction and retrieval, as demonstrated by systems presented in a standard evaluation framework provided by the Text Retrieval Conference (TREC) Q / A track.

[0273] Sperber and Wilson (1986) judged the relevance of an answer according to the amount of effort required to "prove" that a particular answer is relevant to the question. This rule can be formulated in rhetorical terms as a criterion for relevance. The less presumptive the rhetorical relations required to prove that the answer matches the question, the more relevant the answer. The effort required can be measured in terms of the amount of prior knowledge, inferences from the text, or assumptions required. To provide a more manageable criterion, we propose simplifying the problem by focusing on how constraints or rhetorical relations can be excluded from the way the question is formulated. In other words, we evaluate how simplified the question can be to prove the answer. The resulting rule is formulated as follows: The relevance of an answer is judged by how many rhetorical constraints must be excluded from the question for the answer to be proven. The fewer rhetorical constraints that must be excluded, the more relevant the answer.

[0274] The corpus of research into how discovering rhetorical relationships can be useful in Q / A is very limited. Kontos introduced a system that allowed for the exploitation of rhetorical relationships between "base" texts proposing a model of a biomedical system and abstracts of papers presenting experimental findings that support this model. See Kontos, John, Ioanna Malagardi, and John Peros (2016), "Question Answering and Rhetoric Analysis of Biomedical Texts in the AROMA System" (unpublished manuscript).

[0275] An adjacent pair is defined as a pair of adjacent utterances produced by different speakers and ordered as a first part and a second part. A first part of a particular type requires a second part of a particular type. Some of these constraints may be removed to cover more cases of dependencies between utterances. See Popescu-Belis, Andrei, "Dialogue Acts: One or More Dimensions?" (Tech Report ISSCO Working paper no. 62. 2005).

[0276] Adjacency pairs are inherently relational but can be decomposed into labels ("first part", "second part", "none"), possibly augmented with pointers to other members of the pair. Frequently encountered observed types of adjacency pairs include: request / offer / invite → accept / refuse; assess → agree / disagree; blame → denial / admission; question → answer; apology → downplay; thank → welcome; greeting → greeting. See Levinson, Stephen C. (2000), "Presumptive Meanings; The Theory of Generalized Conversational Implicature" (Cambridge, MA: The MIT Press).

[0277] Rhetorical relations are relational concepts that relate utterances to non-utterances in isolation, similar to adjacency pairs. However, if an utterance is a satellite to the nucleus in only one relation, it becomes possible to assign relational labels to utterances. This strongly necessitates a deep analysis of dialogical structure. The number of rhetorical relations in RST ranges from the "dominates" and "gratification-prioritizes" classes used by Grosz and Sidner (1986) to over 100 types. Coherence relations are an alternative way of representing rhetorical structures in a text. See "Categories of coherence relations in discourse annotation" by Scholman, Merel, Jacqueline Evers-Vermeul, and Ted Sanders, Dialogue & Discourse; Vol 7, No 2 (2016)).

[0278] There are many classes of NLP applications that are expected to exploit the information structure of text. DTs can be very useful text summaries. Information about the characteristics of text segments must be considered to form accurate and coherent summaries based on the core-satellite relationship proposed by Sparck-Jones in 1995 and the structure of relationships between segments. See Sparck-Jones, K., "Summarizing: analytic framework, key component, experimental method, in Summarizing Text for Intelligent Communication" (Eds. B. Endres-Niggemeyer, J. Hobbs and K. Sparck Jones; Dagstuhl Seminar Report 79 (1995)). The most informative summary can be generated by combining the most important segments in the elaboration relationships starting from the root node. DTs have been used for multi-document summaries. See Radev, Dragomir R., Hongyan Jing, and Malgorzata Budzikowska (2000), "Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies," in Proceedings of the 2000 NAACL-ANLP Workshop on Automatic Summarization - Volume 4.

[0279] In natural language generation problems where coherence is a major challenge, the information structure of the text can be relied upon to organize extracted fragments of text in a coherent way. Methods for measuring textual coherence can be used in the automatic evaluation of essays. Because DT can capture textual coherence, the flexible discourse structure of an essay can be used to assess the style and quality of the essay. Burstein described a semi-automatic method for essay evaluation that assessed textual coherence. See Burstein, Jill C., Lisa Braden-Harder, Martin S. Chodoro, Bruce A. Kaplan, Karen Kukich, Chi Lu, Donald A. Rock, and Susanne Wolff (2002).

[0280] The neural network language model proposed in engio in 2003 attempts to predict the next word using a concatenation of several previous word vectors to form the neural network's input. See "A neural probabilistic language model" by Bengio, Yoshua, Rejean Ducharme, Pascal Vincent, and Christian Janvin (2003), J. Mach. Learn. Res. 3 (March 2003), 1137-1155. The result is as follows: Specifically, after the model is trained, word vectors are mapped into a vector space such that semantically similar words in the distributed representations of sentences and documents have similar vector representations. This kind of model can potentially operate on discourse relations, but it is difficult to provide as rich linguistic information as we provide for tree kernel learning. There are corpora of studies that extend word2vec models beyond the word level to achieve phrase- or sentence-level representations. For example, a naive approach uses a weighted average of all words in a document (a weighted average of word vectors), which destroys word ordering similar to how bag-of-words approaches do. More sophisticated approaches use matrix-vector operations to combine word vectors in the order given by the sentence's parse tree. See R. Socher, CD Manning, and AY Ng (2010), "Learning continuous phrase representations and syntactic parsing with recursive neural networks," in Proceedings of the NIPS-2010 Deep Learning and Unsupervised Feature Learning Workshop. Combining word vectors using the parse tree was shown to work only for sentences, since it relies on syntactic analysis.

[0281] Many early approaches to policy learning for dialog systems used small state spaces and action sets and focused only on limited policy learning experiments (e.g., confirmation type or initiative type). The Communicator dataset (Walker et al., 2001) is the largest available corpus of human-machine dialogues, further annotated with dialogue context. While this corpus has been used extensively to train and test dialogue managers, it is limited to the information required for dialogues in the air travel domain, with respect to a limited number of attributes, such as the destination city. At the same time, in this work, we relied on a wide corpus of request-response pairs of various natures.

[0282] Reichman (1985) provides a formal description of traditional methods for recognizing speech acts in utterances and an Augmented Transition Network (ATN) model of conversational action. The author uses an analysis of linguistic markers similar to those currently used for rhetorical analysis, such as preverb "please," modal verbs, prosody, references, and cue phrases (e.g., "Yes, but..." (subargument concessions and counterarguments); "Yes, and..." (argument agreement and further support); "No" and "Yes" (non-agreement / agreement); "Because..." (support), and other intra-utterance markers). See Reichman R. (1985), "Getting Computers to Talk Like You and Me: Discourse Context, Focus, and Semantics (an ATN model)" (Cambridge, Mass.; London: MIT Press).

[0283] Given a DT for texts as candidate answers to a compound query, a rule system for the occurrence of validity and invalidity of query keywords in this DT has been proposed, see Galisky (2015). To be a valid answer to a query, its keywords must occur in a set of basic discourse units of the answer such that the set of basic discourse units can be sufficiently ordered and connected by nucleus-satellite relations. If the query keywords occur only in the satellite discourse units of the answer, the answer may be invalid.

[0284] Dialogue games In any conversation, a question is typically followed by an answer, or some clear statement of an inability or refusal to answer. We have the following model of the intentional space of a conversation: when a question is asked by agent B, agent A recognizes agent B's goal to find the answer and adopts the goal of communicating the answer to B in order to cooperate. A then plans to generate the answer by achieving the goal. This provides an elegant explanation in simple cases, but requires strong assumptions for cooperation: agent A must adopt agent B's goal as her own. As a result, this does not explain why A speaks up when she does not know the answer or when she is not ready to accept B's goal.

[0285] Litman and Allen introduced intentional analysis at the discourse level in addition to the domain level, and assumed a set of traditional multi-agent behaviors at the discourse level. See Litman, DL and Allen, JF (1987), "A plan recognition model for subdialogues in conversation," Cognitive Science, 11:163-2. Others have attempted to explain this kind of behavior using social intentional structures such as intentions of solidarity. See Cohen, PR and Levesque, HJ (1990), "Intention is choice with commitment," Artificial Intelligence, 42:213-261. See also Grosz, Barbara J. and Sidner, Candace L. (1986), "Attentions, Intentions and the Structure of Discourse," Computational Linguistics, 12(3), 175-204. While these accounts have helped to explain some discourse phenomena more fully, they still require strong coordination to explain dialogic coherence, but do not provide a simple explanation for why agents behave in ways that do not support high-level mutual goals.

[0286] Imagine a stranger approaches someone and asks, "Do you have spare coins?" Because they have never met before, there is unlikely to be any intention of solidarity or shared plans. From a purely strategic perspective, the agent could not be interested in whether the stranger's objectives are met. Yet, typically, the agent will respond in such a situation. Thus, an explanation of Q / A must go beyond the recognition of the speaker's intentions. Something more than adopting the interlocutor's objectives is involved in formulating a response to a question, as the question does more than provide evidence of the speaker's objectives.

[0287] Mann proposed a library of discourse-level actions, sometimes called dialog games, that encode common communicative interactions. See Mann, William, and Sandra Thompson (1988), "Rhetorical structure theory: Towards a functional theory of text organization," Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243-281. To be cooperative, agents must always be participating in one of these games. Thus, when a question is asked, only a certain number of actions (i.e., those introduced by the question) are cooperative responses. Games provide a good explanation of coherence, but they still require agents to recognize each other's intentions in order to execute a dialogue game. As a result, this work can be viewed as a special case of the intentionality view. Because of this separation, agents need not assume cooperation for the task each agent is performing, but must recognize intentions and cooperation at the conversational level. What goals motivate conversational cooperation remains unexplained.

[0288] Coulthard and Brazil suggested that multiple responses can play a dual role as both a response and a new initiation: Initiation^ (Re-Initiation)^ Response^ (Follow-up). See "Exchange structure" by Coulthard RM and Brazil D. (1979) (Discourse analysis monographs no. 5.: Birmingham: The University of Birmingham, English Language Research). An exchange can consist of two to four utterances. Furthermore, a follow-up can itself be followed up. An opening move often signals the beginning of an exchange, but this does not restrict the type of next move. Finally, a closing move can occur that does not require a follow-up. When these observations are added to the equation, we end up with the following: (Open) ^ Initiation ^ (Re-Initiation) ^ Response ^ (Feedback) ^(Follow-up) ^ (Close) This can deal with anything you get from exchanging 2 to 7 or more here.

[0289] Figure 26 shows the discourse actions of a dialogue according to one phase. Tsui (1994) characterizes discourse actions according to a three-part transaction. Her system for choices about initiation, response, and follow-up is shown in Figure 26, corresponding to the top, middle, and bottom parts.

[0290] FIG. 27 shows discourse actions of a dialogue according to one aspect. The classification problem of valid versus invalid RR pairs is also applicable to the task of complete dialog generation, going beyond question answering and automated dialogue support. Popescu presented the building blocks of a logic-based rhetorical structuring for natural language generators for human-computer dialogue. See Popescu, Vladimir, Jean Caelen, and Corneliu Burileanu, "Logic-Based Rhetorical Structuring for Natural Language Generation in Human-Computer Dialogue," Lecture Notes in Computer Science Volume 4629, pp. 309–317, 2007. Practical and contextual aspects are taken into account during interactions with the task controller, providing domain- and application-dependent information and structured in a fully formalized task ontology. To achieve the goals of computational feasibility and generality, a discourse ontology has been constructed, and several axioms introducing constraints on rhetorical relations have been proposed.

[0291] For example, the axioms specifying the semantics of topic (α) are given as follows:

[0292]

number

[0293] In this case, K(α) is a clause that logically expresses the semantics of utterance α. The notion of a topic of an utterance is here specified in terms of a set of objects in a domain ontology and referenced in a prescribed way within the utterance. Thus, topic relations between utterances are computed using the task / domain ontology addressed by the task controller.

[0294] As instances of such rules, the following can be taken into consideration:

[0295]

number

[0296] In this case, t+ is "future and 'new'." Rhetorical relations and arguments Often, the primary means of linking questions and answers is logical argument. There is a clear connection between the RST relations and argument relations that we attempted to study in this study. There are four types of relations: directed relations, support, attack, elaboration, and undirected sequence relations. The support and attack relations are argumentative relations and are known from related work. See Peldszus, A. and Stede, M. (2013), "From Argument Diagrams to Argumentation Mining in Texts," A Survey. Int. J of Cognitive Informatics and Natural Intelligence 7(1), 1-31. The latter two correspond to discourse relations used in RST. The argument-sequence relation corresponds to "sequence" in RST, and the argument-elaboration relation roughly corresponds to "background" and "elaboration."

[0297] The argument elaboration relationship is important because in scientific and technical publications, some background information (e.g., definitions of terms) is often important for understanding the overall argument. A support relationship between an argument construct Resp and another argument construct Req indicates that Resp supports (infers, proves) Req. Similarly, an attack relationship between Resp and Req is annotated if Resp attacks (restricts, refutes) Req. An elaboration relationship is used when Resp is an elaboration of Req and provides more information or defines what is stated in Req without argumentative inference. Finally, we link two argument constructs (in Req or Resp) with a sequence relationship when the constructs belong to each other and only make sense when combined, i.e., when the constructs form a multi-sentence argument construct.

[0298] We found that SVM TK can distinguish between a wide range of text styles (Galitsky, 2015), including non-argumentative and argumentative styles. Each text style and genre has its own rhetorical structure that is exploited and automatically learned. Because the correlation between text style and text vocabulary is somewhat low, traditional classification approaches that consider only keyword statistics can lack accuracy in complex cases. We also performed text classification into somewhat abstract classes, such as those belonging to linguistic objects and metalanguages ​​within the literature domain, as well as style-based document classification into proprietary design documents. See Galitsky, B., Ilvovsky, D., and Kuznetsov, S.O., "Rhetoric Map of an Answer to Compound Queries" (Knowledge Trail Inc. ACL 2015, 681-686). The evaluation of text integrity in the domain of valid versus invalid customer complaints (containing disjointed argument streams indicating the complainant's bad mood) shows a stronger contribution of rhetorical structure information compared to sentiment profile information. While the discourse structure obtained by the RST parser is sufficient to perform text integrity assessment, sentiment profile-based approaches show much weaker results and are not a robust complement to rhetorical structure.

[0299] While extensive research corpora have been devoted exclusively to RST parsers, investigations into how to utilize RST parsing results for real-world NLP problems have been limited to content generation, summarization, and retrieval (Jansen et al., 2014). The DTs obtained by these parsers cannot be directly used in rule-based text filtering or structuring. Therefore, learning is required to exploit the implicit properties of the DTs. To the best of our knowledge, this work is the first to employ discourse trees and their extensions for general open-domain question answering, chatbots, dialog management, and text structure.

[0300] Dialogue chatbot systems need to be able to understand and match users' communicative intentions, reason on these intentions, construct each user's own communicative intention, and populate these intentions with the actual language to be conveyed to the user. Discourse trees themselves do not provide a representation of these communicative intentions. In this work, we introduce communicative discourse trees, built on traditional discourse trees, which, on the one hand, can now be generated on a large scale, and, on the other hand, can constitute a descriptive utterance-level model of a dialog. Processing dialogs through machine learning of communicative discourse trees allows us to model a wide range of dialogue types and collaboration modes (planning, execution, and interleaved planning and execution).

[0301] Statistical computational learning approaches offer several key potential advantages over manual, rule-based, hand-coding approaches to dialog system development.

[0302] ·Data-driven development cycle; · The optimal course of action in some cases; · More accurate models for response selection; · generalizability to unseen conditions; Reduced industrial development and deployment costs.

[0303] Comparing the inductive learning results with kernel-based statistical learning shows that relying on the same information allows us to perform more concise feature engineering than either approach.

[0304] The extensive corpus of literature on RST parsers does not address the question of how the resulting DTs can be employed in practical NLP systems. RST parsers are usually evaluated on their match with human-annotated test sets rather than their expressiveness of features of interest. In this work, we focus on the interpretation of DTs and explore how to represent them in a form that indicates match or non-match rather than a neutral enumeration of facts.

[0305] To provide a match criterion for how a given message in a dialog is followed by the next message, we used the CDT, which here contains labels for communicative actions in the form of surrogate VerbNet frames. We investigated discourse features that indicate precise versus imprecise request-response and question-answer pairs. We used two learning frameworks to recognize precise pairs: deterministic nearest-neighbor learning of the CDT as a graph and tree kernel learning of the CDT. In this case, the feature space of all CDT subtrees is subjected to SVM learning.

[0306] The positive training set was constructed from accurate pairs obtained from Yahoo Answers, social networks, corporate conversations (including Enron emails), customer complaints, and journalistic interviews. The corresponding negative training set was created by adding responses to a variety of arbitrary requests and questions containing relevant keywords to ensure high relevance similarity between requests and responses. Evaluations show that the system can recognize valid pairs in 68%–79% of cases in the weak request-response matching domain and 80%–82% of cases in the strong matching domain. These accuracies are essential for supporting automated conversations. These accuracies are comparable to benchmark tasks such as classifying discourse trees themselves as valid or invalid, as well as to similar fact question-and-answer systems.

[0307] We believe this work is the first to leverage automatically constructed discourse trees for question answering support. Previous work has used specific customer discourse models and features that are difficult to systematically collect, explainably learn, reverse engineer, and compare with each other. We conclude that learning rhetorical structures in the form of CDTs is a primary data source for supporting complex question answering, chatbots, and dialog management.

[0308] Argument detection using communicative discourse trees Aspects described herein use communicative discourse trees to determine whether a text contains an argument. Such an approach may be useful, for example, for a chatbot that can determine whether a user is arguing. When a user attempts to provide an argument for something, several argument patterns may be adopted. The argument may be a key point of any communication, persuasive essay, or speech.

[0309] A communicative discourse tree for a given text reflects the arguments present in the text. For example, the basic points of an argument are reflected in the rhetorical structure of a text in which arguments are presented. Texts without arguments have a different rhetorical structure. See Marie-Francine, Erik Boiy, Raquel Mochales Palau, and Chris Reed. 2007. Automatic detection of arguments in legal texts. (Proceedings of the 11th International Conference on Artificial Intelligence and the Law, ICAIL 07, pp. 225–230, Stanford, California, USA). Additionally, arguments can differ across domains. For example, for product recommendations, texts with positive sentiment are used to encourage potential buyers to make a purchase. In the political domain, the logical structure of sentiment versus argument versus agency is much more complex.

[0310] Machine learning can be used in conjunction with a communicative discourse tree to determine an argument. Determining an argument can be approached as a binary classification task, in which a communicative discourse tree representing a particular block of text is provided to a classification model. The classification model returns a prediction of whether the communicative discourse tree is a positive class or a negative class. The positive class corresponds to text with an argument, and the negative class corresponds to text without an argument. The aspects described herein can perform classification based on different syntactic and discourse features associated with logical arguments. In one example, for a text to be classified as text containing an argument, the text resembles elements of a first class that are assigned to this class. Two types of learning can be used to evaluate the contribution of our information sources: nearest neighbor learning and statistical learning techniques.

[0311] Nearest Neighbor (kNN) learning uses explicit engineering of graph descriptions. The similarity measured is the overlap between the graph of a given text and the graph of a given element of the training set. In statistical learning, various aspects learn structures with implicit features.

[0312] Generally, machine learning techniques estimate the contribution of each feature type and the above learning methods to the problem of argument identification, including the presence of opposing arguments (Stab and Gurevych, 2016). More specifically, various aspects exploit rhetorical relations and how discourse and semantic relations work together in the argument detection task.

[0313] Sentiment analysis is necessary for a wide range of industrial applications, but its accuracy remains quite low. Recognizing the presence of arguments, if done reliably, could potentially replace some opinion mining tasks when trying to distinguish strongly opinionated content from neutral content. The argument recognition results can then serve as features for a sentiment analysis classifier, distinguishing cases with high sentiment polarity from neutral and low polarity cases.

[0314] An example of using a communicative discourse tree to analyze an argument The following examples are introduced to illustrate the value of using communicative discourse trees to determine the presence of arguments in text. The first example discusses Theranos, a healthcare company that hoped to revolutionize blood testing. Several sources, including the Wall Street Journal, alleged that the company's actions were fraudulent. These allegations were based on accusations by employees who had left Theranos. At one point, the FDA became involved. In 2016, some believed Theranos's view that the problem was started by a Theranos competitor jealous of the efficiency of Theranos's promised blood-testing technology. However, using discourse analysis, the embodiments described herein show that the Theranos discourse patterns mined on the Theranos website were incomplete. Indeed, instances of misconduct emerged, leading to numerous fraud reports. According to the Securities and Exchange Commission, Theranos CEO Elizabeth Holmes raised over $700 million from investors "through a meticulous, years-long misconduct" in which she exaggerated or misrepresented statements about the company's technology and finances.

[0315] Considering content about Theranos, if a user is biased toward Theranos and not its opponents, the argument detection system will attempt to provide answers favorable to Theranos's point of view. In this case, a good argument from the proponent or a bad argument from the opponent would also be useful. Table 4 shows flags for various combinations of agency, sentiment, and argument to tailor search results for a given user with a particular preference for entity A versus entity B. The right gray side of the column has opposing flags for the second and third rows. For the fourth row, only cases with generally accepted opinion-sharing merit are flagged for display.

[0316] A chatbot can use the information in Table 4 to personalize responses or align search results or opinionated data with a user's expectations. For example, a chatbot can consider political viewpoints when providing news to a user. Additionally, personalizing responses is useful for product recommendations. For example, a particular user may prefer skiing to snowboarding, as evidenced by a user sharing a story about people who don't like snowboarders. In this way, embodiments described herein enable a chatbot to behave like a companion by showing empathy and ensuring that a user is not irritated by a lack of common ground with the chatbot.

[0317] [Table 6]

[0318] Continuing with the Theranos example, a RST representation of the argument is constructed, and various aspects can be observed to determine whether a discourse tree can indicate whether a paragraph conveys both a claim and supporting arguments. Additional information is added to the discourse tree to determine whether it represents an argument pattern. According to the Wall Street Journal, this is what actually happened: "Since October

[2015] , The Wall Street Journal has published a series of anonymously sourced allegations that inaccurately portray Theranos. Now, in its latest article ("US Investigates Theranos Complaints," December 20), the Journal again relies on anonymous sources to report two undisclosed and unconfirmed complaints allegedly filed by the Centers for Medicare and Medicaid Services (CMS) and the US Food and Drug Administration (FDA)" (Carreyrou, 2016).

[0319] 28 illustrates an exemplary communicative discourse tree according to one embodiment. FIG. 28 illustrates discourse tree 2800, communication behavior 2801, and communication behavior 2802. More specifically, discourse tree 2800 represents the following paragraph: "But Theranos has struggled behind the scenes to turn the excitement over its technology into reality. At the end of 2014, the lab instrument developed as the linchpin of its strategy handled just a small fraction of the tests then sold to consumers, according to four former employees." As can be seen, when any communication behavior is attached to discourse tree 2800 as a label for a terminal arc, it becomes clear that the document author is trying to make their point known rather than simply sharing facts. As shown, communication behavior 2801 is "struggle" and communication behavior 2802 is "develop."

[0320] 29 illustrates an exemplary communicative discourse tree, according to one embodiment. Figure 29 shows a discourse tree 2900 representing the following text: "Theranos remains actively engaged with its regulators, including CMS and the FDA, and no one, including the Wall Street Journal, has provided Theranos a copy of the alleged complaints to those agencies. Because Theranos has not seen these alleged complaints, it has no basis on which to evaluate the purported complaints." However, as can be seen, from the discourse tree alone, along with the rhetorical relationships of multiple elaborations and the instance of a single context, it is unclear whether the document author is arguing with an opponent or listing several observations. The CDT uses the words "engaged" or "not" to denote the context of the discourse tree. Communicative actions such as "I don't see" can be relied upon to express the fact that the document creator is actually arguing with the opponent.

[0321] 30 illustrates an exemplary discourse tree for communication, according to one embodiment. FIG. 30 illustrates a discourse tree 3000 representing the following text, in which Theranos is attempting to save the day: "It is not unusual for disgruntled and terminated employees in the heavily regulated health care industry to file complaints in an effort to retaliate against employers for termination of employment. Regulatory agencies have a process for evaluating complaints, many of which are not substantiated. Theranos trusts its regulators to properly investigate any complaints." Thus, discourse relations are necessary but insufficient to indicate the structure of an argument, as are speech acts (communicative actions). For the paragraph associated with Figure 30, it is necessary to know the discourse structure of the interactions between agents and what kind of interactions they are. Specifically, it is necessary to distinguish between neutral elaborations (which do not include communicative actions) and elaboration relations that include communicative actions with sentiments such as "not provide" that correlate to arguments. Note that the domain of the interactions (e.g., healthcare) is not necessary, nor is it necessary what the subjects (company, magazine, institution) or entities of these interactions are. However, mental domain-independent relationships between these entities are useful.

[0322] 31 illustrates an exemplary discourse tree for communication, according to one embodiment. Figure 31 illustrates a discourse tree 3100 representing the following text about Theranos' argument that its opponents' arguments are incomplete: "By continually relying on mostly anonymous sources, while dismissing concrete facts, documents, and expert scientists and engineers in the field provided by Theranos, the Journal denies its readers the ability to scrutinize and weigh the sources' identities, motives, and the veracity of their statements." From the perspective of common sense reasoning, Theranos has two options to confirm its argument that its tests are valid: (1) conduct an independent study, compare those results with peers, and publish the data, confirming that its analysis is correct; or (2) refute its opponents' arguments that its test results are invalid and supporting its claim that the opponents are wrong. Clearly, the former argument is much stronger, and the latter is usually chosen when the agent believes that the former argument is too difficult to implement. On the one hand, readers may agree with Theranos that the WSJ should have provided more evidence for its accusations against Theranos. On the other hand, readers may likely dislike Theranos's choice of the latter argument type (2), thus significantly weakening Theranos's position. One reason Theranos's argument is weak is because Theranos seeks to refute its opponents' allegations regarding client complaints about Theranos's services. Theranos's demand for evidence by forcing the WSJ to disclose its sources and the nature of the complaints is weak. One argument is that a third party (an independent research agent) would be more rational and decisive. However, some readers may find Theranos's argument (burden of proof avoidance) logical and valid. Note that the argument evaluator cannot identify rhetorical relationships within a text by relying on the text alone. Rather, the context of the situation is useful for understanding the argumenter's intent.

[0323] In a second example, the document author's objective is to attack claims that the Syrian government used chemical weapons in the spring of 2018. Figure 32 illustrates an exemplary communicative discourse tree according to one aspect. Figure 32 illustrates a communicative discourse tree 3200 for this second example.

[0324] Considering this example, admissible evidence would be the sharing of specific observations that, from a peer's perspective, correlate with the absence of a chemical weapons attack. For example, if it could be demonstrated that the timing of the alleged chemical weapons attack coincided with a period of very heavy rain, that would be a convincing way to attack the claim. However, because no such observations were confirmed, the source, Russia Today, resorted to insinuating a complex mental state regarding how the claim was conveyed, in which case most statements about the mental state of those involved are difficult to verify. The following are basic discourse units that are broken down by discourse parsers: [Whatever the Douma residents ,][who had firsthand experience of the shooting of the water][dousing after chemical attack video ,][have to say ,][their words simply do not fit into the narrative][allowed in the West ,][analysts told RT .] [Footage of screaming bewildered civilians and children][being doused with water ,][presumably to decontaminate them ,][was a key part in convincing Western audiences][that a chemical attack happened in Douma .] [Russia brought the people][seen in the video][to Brussels ,][where they told anyone][interested in listening][that the scene was staged .] [Their testimonies, however, were swiftly branded as bizarre and underwhelming and even an obscene masquerade][staged by Russians .] [They refuse to see this as evidence ,][obviously pending][what the OPCW team is going to come up with in Douma ], [Middle East expert Ammar Waqqaf said in an interview with RT .] [The alleged chemical incident ,][without any investigation , has already become a solid fact in the West ,][which the US , Britain and France based their retaliatory strike on .] (Whatever Douma residents who witnessed firsthand the footage of the water cannons after the chemical attack say, their words simply do not fit the Western-acceptable scenario, analysts told RT. Footage of screaming, bewildered civilians and children being doused in water, presumably for decontamination, was a key part of convincing Western audiences that a chemical attack had taken place in Douma. Russia took those seen on the footage to Brussels, where they told anyone who would listen that the scenes were fabricated. But their testimony was swiftly branded a bizarre, overwhelming and even despicable fabrication orchestrated by the Russians. Middle East expert Ammar Waqqaf said in an interview with RT that they refuse to see this as evidence and that the OPCW team is clearly still deciding what to do in Douma. The alleged chemical attack, without any investigation, has already become solid fact in the West, on which the US, UK and France can base their retaliatory strikes.) .

[0325] Note that the above text does not find any evidence to refute the chemical attack claim it seeks to debunk. Instead, the text states that opponents have no interest in observing this evidence. The article's main statement is that certain agents "deny" certain types of evidence that attack the main claim, rather than providing and supporting this evidence. Instead of debunking the chemical attack claim, the article constructs a clash of complex mental states between residents, the Russian agents who brought them to Brussels, the West, and Middle East experts.

[0326] FIG. 33 illustrates an exemplary communicative discourse tree according to one embodiment. FIG. 33 illustrates a communicative discourse tree 3300 for another controversial article, the Trump-Russia link acquisition (BBC 2018). For a long time, the BBC has been unable to verify this claim, and therefore, this article has been repeated many times to maintain reader hope that it will one day be corroborated. With no confirmation or denial that a dossier exists, the author's goal is to make the audience believe that such a dossier exists without misrepresenting events. To achieve this goal, the author may subject several hypothetical descriptions of existing dossiers to various mental states in order to impress upon the reader the authenticity and validity of the topic.

[0327] As depicted in Figures 32 and 33, many rhetorical relations are associated with mental states. Mental states are sufficiently complex that it is difficult for humans to verify the accuracy of the main claims. A communicative discourse tree shows that authors attempt to substitute complex mental states for the logical chains that would support their claims. Simply looking at the CDTs shown in Figures 32 and 33, without reading the associated text, is enough to see that the line of argument is incomplete.

[0328] Handling Heated Arguments FIG. 34 illustrates an exemplary communicative discourse tree according to one embodiment. FIG. 34 illustrates a communicative discourse tree 3400 for an example heated argument. Specifically, the following text, represented by the communicative discourse tree 3400, illustrates an example of a CDT for a heated argument involving a customer who was mistreated by the credit card company American Express (Amex) in 2007. The communicative discourse tree 3400 illustrates an emotional profile. An emotional profile is an emotional value attached to the direction of the proposer (here, "I") and the opponent (here, "Amex"). As can be seen, the proposer is almost always positive, and the opponent is negative, confirming the argument flow of this complaint. The fluctuating emotional values ​​would indicate a problem with how the document author presents the argument.

[0329] The text is divided into logical chunks as follows: [I'm another one of the many][that has been carelessly mistreated by American Express.] [I have had my card since 2004 and never late.] [In 2008][they reduced my credit limit from $16,600 to $6,000][citing several false excuses.] [Only one of their excuses was true - other credit card balances.] [They also increased my interest rate by 3%][at the same time.] [I have never been so insulted by a credit card company.] [I used to have a credit score of 830, not anymore, thanks to their unfair credit practices.] [They screwed my credit score.] [In these bad economic times you'd think][they would appreciate consistent paying customers like us][but I guess][they are just so full of themselves.] [I just read today][that their CEO stated][that they will be hurt less than their competitors][because 80 percent of their revenues][are generated from fees. That][explains their callous, arrogant, unacceptable credit practices.]It seems they have to screw every cardholder before the new law becomes effective. Well America, let's learn from our appalling experience and stop using our American Express credit card so we can pay it off! I'm one of many who have been casually wronged by American Express. I've had my card since 2004 and have never been late on a payment. In 2008, they lowered my credit limit from $16,600 to $6,000, citing several false excuses. Only one of their excuses—balances on other credit cards—was true. At the same time, my interest rate also increased by 3%. I have never been so insulted by a credit card company. I used to have an 830 credit score, but thanks to their unfair credit practices, it's no longer the case. They ruined my credit score. In today's bad economy, you'd think they'd appreciate consistent paying customers like us, but I think they're only thinking about themselves. I just read today that their CEO said they're hurting less than their competitors because 80% of their revenue comes from fees. That explains their ruthless, arrogant, and unacceptable credit practices. It seems they need to squeeze every cardholder they can before the new law goes into effect. Come on, America, learn from our horrible experience, stop using our American Express credit cards, and get revenge!

[0330] FIG. 35 illustrates an exemplary communicative discourse tree according to one aspect. Figure 35 shows a communicative discourse tree 3500 representing text giving advice on how to behave in communicating an argument: "When a person is in the middle of an argument, it can be easy to get caught up in the heat of the moment and say something that makes the situation even worse. Nothing can make someone more frenzied and hysterical than telling them to calm down. It causes the other person to feel as if one is putting the blame for the elevation of the situation on them. Rather than actually helping them calm down, it comes off as patronizing and will most likely make them even angrier." Figure 35 is an example of a meta-argument. A meta-argument is a discussion about how to conduct a heated argument and can be expressed by the same rhetorical relations.

[0331] Using machine learning models to judge arguments As discussed, the application 102 can detect arguments within the text. Figure 36 illustrates an exemplary process for determining arguments using machine learning, according to one embodiment.

[0332] At block 3601, the process 3600 includes accessing text that includes a fragment. The application 102 can receive the text from different sources, such as input text 130 or internet-based sources such as chat, Twitter, etc. The text can consist of a fragment, a sentence, a paragraph, or a longer amount.

[0333] At block 3602, process 3600 includes creating a discourse tree from the text, the discourse tree including nodes, each non-terminal node representing a rhetorical relationship between two of the fragments, and each terminal node of the discourse tree being associated with one of the fragments. Application 102 creates the discourse in a manner substantially similar to that described at block 1502 of process 1500.

[0334] At block 3603, process 3600 involves matching each fragment having a verb with a verb signature, thereby creating a communicative discourse tree. Application 102 creates the discourse in a manner substantially similar to that described in steps 1503-1505 of process 1500.

[0335] At block 3604, the process 3600 includes determining whether the communicative discourse tree contains an argument by applying a classification model trained to detect arguments to the communicative discourse tree. The classification model can use different learning techniques. For example, the classification model can use a support vector machine with tree kernel learning. Additionally, the classification model can use maximum common subtree nearest neighbor learning.

[0336] As an example, the application 102 can use machine learning to determine the similarity between the communication discourse tree identified in block 3603 and one or more communication discourse trees from a training set of communication discourse trees. The application 102 can select an additional communication discourse tree from a training set including multiple communication discourse trees. The training can be based on the communication discourse tree having the greatest number of similarities with the additional communication discourse tree. The application 102 identifies whether the additional communication discourse tree is from a positive set or a negative set. The positive set is associated with text that includes an argument, and the negative set is associated with text that does not include an argument. Based on this identification, the application 102 determines whether the text includes an argument or does not include an argument.

[0337] Evaluating logical argument detection To evaluate argument detection, a positive dataset is created from several sources to ensure heterogeneity and to capture different styles, genres, and argument types. First, we use a subset of data where arguments are frequent, such as opinionated data from newspapers such as The New York Times (1,400 articles), The Boston Globe (1,150 articles), The Los Angeles Times (2,140), and others (1,200). We also use customer complaint texts. Furthermore, we use a text style and genre recognition dataset (Lee, 2001). This dataset has specific dimensions associated with arguments (see section [ted] "Emotional speech on a political topic with an attempt to sound convincing"). And finally, we add some texts from standard argument mining datasets where the presence of arguments is established by annotators: the "Fact and Feeling" dataset (Oraby et al., 2015), 680 articles, and the dataset "Argument annotated essays v.2" (Stab and Gurevych, 2016), 430 articles.

[0338] For negative datasets, we can use Wikipedia (3500 articles), factual news sources (Reuters supply with 3400 articles), and also datasets containing sections of corpora such as (Lee, 2001) [tells] (450 articles), "Instructions for how to use software" (320 articles); [tele]; "Instructions for how to use hardware" (175 articles); [news]; "A presentation of a news article in an objective, independent manner" (220 articles), and other mixed datasets without argumentation (735 articles).

[0339] Both the positive and negative datasets contained 8,800 texts. The average text size was 400 words (always above 200 and below 1,000). We used Amazon Mechanical Turk to confirm that the positive dataset contained arguments in common sense according to the recruited workers. The labeling task was assigned to 12 workers with a previous acceptance score above 85%. For manual verification of the presence or absence of arguments, we randomly selected representatives (approximately 10%) from each set and confirmed that they properly belonged to a class with a confidence level above 95%. We avoided sources with a confidence level below 95%. For the first portion of the texts that underwent manual labeling, we evaluated inter-annotator agreement and found it to be above 90%. Therefore, for the remaining annotations, we relied on one worker per text. For evaluation, we divided the dataset into training and testing parts in a 4:1 ratio.

[0340] Specific discussion pattern dataset The purpose of this argument dataset is to collect complaint texts that use various argumentative methods to prove that the document author is a victim of the business. Customer complaints are emotive texts that contain descriptions of problems customers have experienced with a particular business. Raw complaints are collected from PlanetFeedback.com for a large number of banks, filed between 2006 and 2010. The 400 complaints are manually tagged for the following parameters related to the argument: · The perceived validity of the complaint; · The validity of the argument; the existence of specific argument patterns, and Detectable misrepresentations.

[0341] Judging by the complaints, most complainants are genuinely distressed due to a strong deviation from what they expected from the service, what they received, and how it was communicated. Most complainants report incompetence, flawed policies, neglect, indifference to customer needs, and misrepresentation from customer service personnel.

[0342] Document writers often exhaust the means of communication available to them, become confused, seek recommendations from other users, and advise others to avoid certain financial services. The focus of a complaint is proof that the proponent is right and the opponent is wrong, a proposed resolution, and a desired outcome.

[0343] Several argument patterns are used in complaints: The most frequent is the deviation of what actually happened from what was expected, according to common sense. This pattern covers both valid and invalid arguments (valid pattern). The second popular argument pattern cites the difference between what was promised (advertised, communicated) and what was received or actually occurred. This pattern also mentions that the opponent does not play by the rules (effective). Numerous complaints explicitly state that bank representatives lied. Lying includes inconsistencies between information provided by different bank representatives, misrepresentations of facts, and careless promises (valid). Another reason for complaints is due to the rudeness of bank agents and customer service personnel. Customers cite rudeness in both cases, and then the objection points are valid or not (and the complaint and argument validity are tagged accordingly). Even if there is no financial loss or inconvenience, the complainant does not agree with everything a given bank does if they are treated rudely (invalid pattern). Complainants cite their needs as a reason why banks should behave in a certain way. A popular argument is that the administration bailed out the banks through taxpayers, so now the banks should give back to their customers (invalid).

[0344] This dataset contains more emotionally charged complaints than other argument mining datasets. For a given topic, such as insufficient funding fees, this dataset provides many different ways of arguing that the fees are unfair. Therefore, our dataset allows for a systematic search for clusters of argument patterns independent of the topic, and we observe the link between argument types and the overall validity of the complaint. Other argument datasets, including legal arguments, student papers (Stab and Gurevych 2017), the Internet Argument Corpus (Abbot et al. 2016), the Fact-Sense dataset (Oraby et al. 2016), and political debates, have strong topical variations, making it more difficult to track the spectrum of possible argument patterns per topic. Unlike technical writings in the legal and political domains, authentic writings by users who file complaints have simple motivational structures, transparency of their objectives, and occur within a fixed domain and context. In the dataset used in this study, argumentation plays an important role for the well-being of document authors who are subject to unjustified large monetary claims or eviction from their homes. Therefore, document authors try to provide as strong arguments as possible to support their claims and strengthen their cases.

[0345] If a complaint is untrue, it is usually invalid: the customer is either complaining about bad feelings or trying to get compensation. However, even if a complaint is true, it can easily be invalid, especially if the argumentation is flawed. If a false complaint has a valid argument pattern, it is difficult for annotators to properly assign it as valid or invalid. Three annotators worked on this dataset, and the agreement between the annotators is over 80%.

[0346] Evaluation Settings and Results For nearest neighbor classification, we used the maximum common subgraph for the DT method, along with the maximum common subgraph for the CA method, which is based on a scenario graph constructed on the CA extracted from the text (Table 5). For SVM TK classification, we employ the tree kernel learning of the parse interlacing method, where each paragraph is represented by a parse interlacing that includes comprehensive syntactic and discourse information. We also use SVM TK for DT, and CA information is not considered.

[0347] The pre-baseline methods in our family are based on keywords and keyword statistics. For the naive Bayesian method, we relied on the WEKA framework (Hall et al., 2009). Because most lexical and length-based features are reliable for detecting poorly supported arguments (Stab and Gurevych 2017), we used non-NER as a feature, along with the number of tokens in phrases potentially expressing arguments. We also assumed that NER counts correlate with the strength of arguments. Even if these features are strongly correlated with arguments, they do not help us understand the nature of how arguments are structured and communicated in language, as represented by CDT.

[0348] [Table 7]

[0349] [Table 8]

[0350] The naive method relies solely on keywords to capture the presence of an argument. Typically, several communication actions, at least one of which has a negative sentiment polarity (associated with the opponent), are sufficient to deduce the presence of a logical argument. This naive method performs 29% worse than the best-performing CDT method. The naive Bayes classifier provides only a 2% improvement.

[0351] We can observe that for nearest neighbor learning, DT and CA indeed complement each other, resulting in CDT accuracies of over 26% for the former and over 30% for the latter. Simply CA yields worse results than independent DT (Table 6). As can be seen, SVM TK with CDT outperforms SVM TK with RST+CA and full syntactic features (the SVM TK baseline) by 5%. This is due to feature engineering and relying on less data, but more relevant than the baseline.

[0352] [Table 9]

[0353] Nearest neighbor learning on CDT achieves slightly lower accuracy than SVM TK on CDT, but the former provides interesting examples of subtrees typical of arguments and shared among factual data. The number of subtrees in the former group of CDT subtrees is, not surprisingly, significantly larger. Unfortunately, the SVM TK approach does not help explain how exactly the argument identification problem is solved; it merely provides the final scoring and class labels. Representing logical arguments without CA is possible, but rare. This observation is reinforced by our data.

[0354] It is worth mentioning that our evaluation setup is close to the SVM-based ranking of RST parsing. The problem is formulated as classifying a DT into a set of correct and incorrect trees that are close to manually annotated trees. Our setup is slightly different because it is better tuned to smaller datasets. Note that the improvement in argument detection going from DT to CDT demonstrates the validity of our extension of RST with speech act-related information.

[0355] Table 7 shows the SVM TK argument detection results for each source. As the positive set, we take only individual sources here. The negative set is formed from the same sources, but its size is reduced to fit the smaller size of the positive set. The cross-validation setting is similar to our evaluation of the entire positive set.

[0356] We found no correlation between the specificity of a particular domain and the contribution of discourse-level information to argument detection accuracy. At the same time, all four domains show monotonic improvement when we progress from Keywords and Naive Bayes to SVM TK. Because all four sources show improvement in argument detection rate with CDT, we conclude that the same is likely true for other sources of argument-related information.

[0357] [Table 10]

[0358] The pattern-specific argument detection results are shown in Table 8. We calculate the classification accuracy as the specific pattern versus other patterns and the absence of arguments. The first and second types of arguments are more difficult to recognize (7-10% lower than general arguments), while the third and fourth types are easier to detect (3% above general argument accuracy).

[0359] These argument recognition accuracies are comparable to current state-of-the-art argument mining techniques. One study analyzed text containing 128 premise-conclusion pairs and obtained an F-measure of 63–67% to determine the directionality of the inferential connections in the argument. See Lawrence, John, and Chris Reed. Mining Argumentative Structure from Natural Language Text Using Automatically Generated Premise-Conclusion Topic Models. Proceedings of the 4th Workshop on Argument Mining, pages 39–48, 2017. Bar-Haim et al. show that automatic expansion of the initial vocabulary can significantly improve both the accuracy and coverage (what supports and what contradicts the claim) of argument stance recognition, up to an F-measure of 69%. See Bar-Haim, Roy Lilach Edelstein, Charles Jochim, and Noam Slonim. Improving Claim Stance Classification with Lexical Knowledge Expansion and Context Utilization. Proceedings of the 4th Workshop on Argument Mining, pages 32-38. 2017. Aker et al. provide a comparative analysis of the performance of different supervised machine learning methods and feature sets on argument mining tasks, achieving an F-measure of 81% for detecting argumentative sentences and 59% for the argument structure prediction task.See Aker, Ahmet, Alfred Sliwa, Yuan Ma, Ruishen Liu, Niravkumar Borad, Seyedeh Fatemeh Ziyaei, Mina Ghbadi. What works and what does not: Classifier and feature analysis for argument mining. Proceedings of the 4th Workshop on Argument Mining, pages 91-96. 2017. Regarding argument segmentation of argumentative texts into argumentative units and their non-argumentative counterparts, Ajjour et al. achieve 88% accuracy using Bi-LSTM for essays and 84% accuracy for compilations. Ajjour, Yamen, Wei-Fan Chen, Johannes Kiesel, Henning Wachsmuth and Benno Stein. Unit Segmentation of Argumentative Texts. Proceedings of the 4th Workshop on Argument Mining, pages 118-128, 2017. Considering the complexity of the argument mining task, these classification accuracies are comparable to the current study, but lack the exploration of arguments' causes via discourse-level analysis. Therefore, this study proposes a much more direct feature engineering of general arguments and their specific patterns.

[0360] CDT construction While the division into EDUs works reasonably well, the assignment of RST relations is noisy, and in some domains, its accuracy can be as low as 50%. However, if the RST relation labels are random, it does not significantly degrade the performance of our argument detection system because random discourse trees are not very similar to elements in the positive or negative training set and are most likely not involved in the positive or negative decision. To overcome the noisy input problem, a more extensive training dataset is required so that the number of reliable and plausible discourse trees is large enough to cover the cases to be classified. As long as this number is large enough, the contribution of noisy and improperly constructed discourse trees is low.

[0361] There are certain systematic deviations from the correct intuitive discourse tree obtained by discourse parsers. In this section, we evaluate whether there is a correlation between the CDT deviations and our training set. We allow for the possibility that the CDT deviations for texts with arguments are stronger than the CDT deviations for texts without arguments.

[0362] For each source, we calculated the number of significantly deviant CDTs. For the purposes of this evaluation, we considered a CDT to be deviant if more than 20% of its rhetorical relations were judged to be inappropriate. We did not distinguish between specific RST relations associated with arguments, such as attributions and contrasts. The distortion evaluation dataset was significantly smaller than the detection dataset because substantial manual effort was required and the task could not be submitted to Amazon Mechanical Turk workers.

[0363] [Table 11]

[0364] It can be observed that there is no obvious correlation between the recognition class and the CDT distortion rate (Table 9). Therefore, we conclude that the training set of noisy CDTs can be adequately evaluated for argument detection. As can be seen, there is a strong correlation between these noisy CDTs and the presence of logical arguments.

[0365] emotions Because reliable emotion detection in any domain is difficult, we focus on specific emotion-related features, such as logical arguments with specific polarities. Detecting logical arguments can help improve the performance of emotion detection. We formulate the emotion detection problem at the paragraph level, detecting only the sentiment polarity.

[0366] Classifying emotions based on individual words can be misleading because atomic emotion carriers can be modified (weakened, strengthened, or reversed) based on lexical, discourse, or contextual factors. Words interact with each other to produce expression-level polarity. For example, the meaning of a compound expression is a function of the meanings of its parts and the syntactic rules by which they are combined. Therefore, considering more linguistic structures than required by RST motivates our combination of these insights from various discourse analysis models. Our hypothesis is that it is possible to very accurately calculate the polarity values ​​of larger syntactic elements of a text as a function of the polarities of their subcomponents, in a manner similar to the "principle of compositionality" in formal semantics. In other words, if the meaning of a sentence is a function of the meanings of its parts, then the overall polarity of the sentence is a function of the polarities of its parts. For example, we can attribute negative properties to the verb "reduce," but we can also attribute positive polarity to "reduce risk," even though "risk" itself is negative (see the negative polarity in "reduce productivity"). This polarity reversal is only captured when extending the analysis beyond the sentence level to calculate the overall polarity of the text as a whole. Any polarity conflicts are therefore resolved as a function of the overall meaning of the text, based on textual and contextual factors. Polarity weights are not properties of individual elements of the text, but are a function of properties operating at the level of cohesion, and are potential coherence relations at the syntactic, discourse, and pragmatic levels of discourse analysis.

[0367] Many studies have shown that discourse-related information can successfully improve the performance of sentiment analysis; for example, the importance of EDUs can be reweighted based on their relation type or depth in the DT (Hogenboom et al., 2015a). Some methods prune discourse trees at a specific threshold, resulting in trees of fixed depth between two and four levels. Other techniques train machine learning classifiers based on relation types as input features (Hogenboom et al., 2015b). Most research on sentiment in RDSTs attempts to map the DT structure to a mathematically simpler representation, since it is virtually impossible to encode unstructured data of arbitrary complexity in fixed-length vectors (Markle-Hus et al., 2017).

[0368] Figure 37 is a fragment of a discourse tree according to one embodiment. Figure 37 shows a discourse tree 3700 representing the following text. To demonstrate that the core-satellite relationship is important in determining the sentiment of an entity, we use the following two sentences: [Although the camera worked well,] [I could not use it because of the viewfinder], which expresses negative sentiment about the camera; and [The camera worked well,] [although the viewfinder was inconvenient], which expresses positive sentiment about the camera.

[0369] To evaluate sentiment detection, we used a dataset of positive and negative, real and fake traveler reviews of Chicago-area hotels. See Ott, C. Cardie, and JT Hancock. 2013. Negative Deceptive Opinion Spam. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Authors compile datasets with the goal of distinguishing between real and fake reviews. We find that the falsity of a review does not strongly correlate with the presence of logical arguments. Fake reviews created by mechanical turners support the author's opinion in the same way as real travelers do. The test corpus contains four groups of 400 reviews, each one between one and three paragraphs long. 1) 400 true positive reviews from TripAdvisor; 2) 400 fake positive reviews from Mechanical Turk; 3) 400 true negative reviews from Expedia, Hotels.com, Orbitz, Priceline, TripAdvisor; and 4) 400 fake negative reviews from Mechanical Turk.

[0370] As a baseline method, we use Stanford NLP Sentiment. We take the sentence-level polarity and aggregate it to the paragraph level. Typically, if the opinion is positive, the author will simply list what they like. However, if the opinion is negative, the author will often try to push it back and provide a comparison, explanation, or argument for why they are right and why their assessment is appropriate.

[0371] Therefore, the rules for integrating the default emotion detector and the argument-based emotion detector are as follows (Table 10): This rule is oriented towards consumer review data and would require modification to better handle other text genres.

[0372] [Table 12]

[0373] The following case is a borderline positive review that could easily flip to a negative one: "Like all hotels in Chicago, this hotel caters to wealthy and / or business clients with a very high parking price. However, if you are aware of that prior to arrival, it's not a big deal. It makes sense to find a different place to park the car and bring your own snacks for the room. It would be nice though if hotels such as the Swissotel had a fridge in the room for guest use. The staff was very helpful. Overall, if I can get a good rate again, I'll stay at the Swissotel the next time I am in Chicago. (Like all hotels in Chicago, this hotel offers very high parking fees for wealthy and / or business clients. However, if you are aware of this before arriving, it is not a big problem. It is wise to find a different place to park and bring your own snacks to your room. However, if a hotel such as the Swissotel has refrigerators in the rooms, that would be nice. The staff was very helpful. Overall, if I can get a good rate again, I will stay at the Swissotel next time I'm in Chicago.)" This text appears to be an overall negative review from a DT perspective. Most reviews with similar DTs are negative.

[0374] Figure 38 illustrates a discourse tree for boundary review according to one embodiment. Figure 38 illustrates a discourse tree for boundary review 3800. Boundary reviews are negative from a discourse perspective and neutral from a reader's perspective.

[0375] Extending compositional semantics towards discourse Let's examine how the sentiment in the first sentence is evaluated by a semantic compositionality model. See R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. Conference on Empirical Methods in Natural Language Processing (EMNLP 2013). Here, it's hard to tell that "high price" has a negative sentiment value, judging from the individual words and their composition. In a training movie database, "high" is likely assigned a positive sentiment, and "high price" would likely not be tagged as negative. Even if "high price" were recognized as negative, it would be difficult to determine how the rest of the tree, such as the phrase "wealthy and / or business clients," influences it. Note that neither the movie domain nor the words in this phrase are assigned the correct emotion.

[0376] Determining the sentiment polarity of this sentence alone is quite difficult, given its words and phrasing. Instead, the discourse of successive sentences can be considered to determine the sentiment of the overall paragraph and the sentiment of a given sentence with greater accuracy.

[0377] Figure 39 shows a discourse tree of a sentence illustrating a compositional semantic approach to sentiment analysis according to one embodiment.

[0378] We argue that a sentiment analysis benefiting from "compositional semantics" insights would accurately assign polarity sentiment in the above example if it captured more than just the word "high" (which would be assigned a negative polarity), the phrase "high price" (with a negative polarity), or the sentence-level structure "Like all ....price" (where determining polarity is difficult because one needs to read the entire text for a global polarity attribute). Sentiment analysis is computed based on global polarity, not relying on individual elements of the sentence, but more interestingly, on discourse-level structures (macrostructures). For example, "high reliability" is neutral in the sense of "I want a car with high reliability" because it is a positive attribute but does not refer to a specific car.

[0379] result Since our evaluation of emotion detection is domain independent, the baseline system (Socher et al., 2013) is trained on a domain different from the test domain.

[0380] The results of the sentiment analysis achieved by the hybrid compositional semantics and discourse analysis are shown in Table 11. The first row shows the accuracy of the baseline system on our data. The second gray row shows the improvement achieved by the hybrid system. This improvement is achieved by discovering the overall negative sentiment at the paragraph level when the presence of an argument is recognized. In some of these cases, the negative sentiment is implicit and can only be detected indirectly from the discourse structure, where individual words do not indicate negative sentiment.

[0381] [Table 13]

[0382] We investigate a standalone SVM TK emotion recognition system with various representations (lines 3-5). The CDT representation outperforms the Parse-Interlaced and DT representations. Simpler representations that do not consider any discourse-level information result in significantly lower emotion recognition accuracy (not shown).

[0383] We also investigated whether fake opinion texts have a different rhetorical structure than authentic ones. See Jindal and Liu, Opinion Spam and Analysis, Department of Computer Science, University of Illinois at Chicago, 2008. Jindal and Liu addressed the problem of detecting troll opinion spam: obvious instances that are easily identified by human readers, such as advertisements, questions, and other irrelevant or opinion-free text. (Ott et al. investigated potentially more insidious types of opinion spam, such as troll opinion spam, which is deliberately written to sound authentic in order to deceive readers. See M. Ott, Y. Choi, C. Cardie, and J.T. Hancock. 2011. Finding Deceptive Opinion Spam by Any Stretch of the Imagination. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.) The fake reviews were written by workers on Amazon Mechanical Turk. Instructions asked the workers to assume they were hired by the hotel's marketing department and pretend they were being asked to write a fake review (as if they were a customer) to be posted on a travel review website; furthermore, the review needed to sound realistic and portray the hotel in a positive light. Requests for negative reviews were made as well.)

[0384] Although our SVM TK system did not achieve 90% performance, the task of detecting fake review text was performed by a general-purpose text classification system (with 76-77% accuracy, the two bottom gray rows), which extracts arguments and evaluates sentiment polarity.

[0385] Verifying the argument Aspects of the present disclosure validate arguments. To be persuasive, text or utterances contain valid arguments. The application 102 extracts argument structure from the body of text and represents the argument via a Communicative Discourse Tree (CDT). The application 102 can then verify that claims or target claims in the text are valid, i.e., cannot be logically attacked by other claims, and are consistent with external truths, i.e., rules. Domain knowledge can be used to verify the validity of claims. However, in some cases, domain knowledge may not be available, and other domain-independent information, such as writing style and writing logic, is used.

[0386] Some aspects enable applications such as customer relationship management (CRM), which addresses handling customer complaints (Galitsky and de la Rosa 2011). In customer complaints, the writer is upset about the product or service they received and how the problem was communicated by customer support. Complainants often write complaints in very strong emotional language, which can distort the logic of the argument and therefore make it difficult to judge the validity of the complaint. Both emotional and logical arguments are heavily used.

[0387] To facilitate improved autonomous agents, some embodiments use argument mining, which is linguistics-based, and logical verification of arguments, which is logic-based. The concept of automatically identifying argument schemes was first discussed in (Walton et al., 2008). Ghosh et al. (2014) investigated a specific type of communication—the argumentative discourse structure of online dialogue threads. Identifying arguments in text leads to the problem of identifying truth, misinformation, and disinformation on the web (Pendyala and Figueira, 2015, Galitsky 2015, Pisarevskaya et al. 2015). (Lawrence and Reed, 2015) combines three types of argument structure identification: linguistic features, topic shifts, and machine learning. As further described herein, some aspects employ Defeasible Logic Programming (DeLP) (Garcia and Simari, 2004; Alsinet et al., 2008) in conjunction with communicative discourse trees.

[0388] 40 illustrates an example process 4000 for verifying an argument according to one aspect. The application 102 may perform the process 4000.

[0389] At block 4001, process 4000 includes accessing text that includes a fragment. At block 4001, process 4000 performs steps substantially similar to those described at block 3601 of process 3600. The text may include input text 130, which may be a paragraph, a sentence, an utterance, or other text.

[0390] At block 4002, process 4000 includes identifying the presence of an argument in a subset of text by creating a communicative discourse tree from the text and applying a classification model trained to detect arguments to the communicative discourse tree. At block 4002, process 4000 performs steps substantially similar to those described in blocks 3602-3604 of process 3600. Other argument detection methods may be used.

[0391] At block 4003, process 4000 includes evaluating the argument by using a logic system. The application 102 can evaluate the argument using different types of logic systems. For example, Deterministic Logic Programming (DeLP) can be used. Figure 42 shows example operations that can implement block 4003. For illustrative purposes, process 4000 is described with reference to Figure 41.

[0392] Figure 41 illustrates an exemplary communicative discourse tree for an argument according to one embodiment. Figure 41 includes a communicative discourse tree 4101. The communicative discourse tree 4101 includes a node 4120 and other nodes, some of which are labeled with communicative actions 4110-4117.

[0393] In one example, a judge wants to hear the eviction case and determine whether rent has been provably paid (or deposited), as indicated by rent_receipt. The input is text in which the defendant expresses his or her point. CDT4101 represents the following text: "The landlord contacted me, the tenant, and the rent was requested. However, I refused the rent since I demanded repair to be done. I reminded the landlord about necessary repairs, but the landlord issued the three-day notice confirming that the rent was overdue. Regretfully, the property still stayed unrepaired." Figure 42 shows an exemplary method for verifying an argument using defeasible logic programming, according to one aspect. Defeasible logic programming (DeLP) is a set of a set of facts, a set of definite rules Π of the form (A:-B), and a set of defeasible rules Δ of the form A-<B, whose intended meaning is that "if B holds, then usually A also holds". Let P=(Π,Δ) be a DeLP program and L be a ground literal. Definite rules are based on opinions and cannot be changed. In contrast, defeasible rules can be false in some cases.

[0394] In the above example, the underlined words form clauses in DeLP, while other expressions can form facts. An example of a fact is "rent_refused," i.e., the landlord refused the rent. An example of an absolute rule is "The Earth is flat." An example of a revocable rule is "rent_receipt -< rent_deposit_transaction," which usually means "rent_receipt" (rent was received) if "rent_deposit_transaction" (rent deposit transaction). However, a revocable rule may not always be true if, for example, rent is deposited into the wrong account or there is an error in the bank.

[0395] The application 102 can use the results from the communicative discourse tree created in block 4002 as input to DeLP. The communicative discourse tree reveals valuable information such as how facts are interconnected by revocable rules. The basic discourse units of the CDT, which are of the rhetorical relation type "contrast," and the communicative actions, which are of the type "disagreement," reveal revocable rules.

[0396] At block 4201, the method 4200 includes creating a fixed portion of a logic system. The fixed portion of the logic system includes one or more assertion terms and one or more domain-defining clauses. A domain-defining clause is associated with a domain of text and can include legal, scientific terminology, and common sense knowledge in a particular domain. A scientific example is "If an object is moving with acceleration, it is subjected to a physical force." In the field of landlord-tenant law, an example of a standard definition is "If repairs are made -> the house is habitable and the appliances are working."

[0397] Continuing with the above example, the text includes the target assertion to be evaluated, "rent_receipt," i.e., "Was the rent received?". The application 102 also extracts the following clause, "repair_is_done -< rent_refused," from the text, "refused the rent since I demanded repair to be done."

[0398] At block 4202, the method 4200 includes creating a variable portion of the logic system by determining a set of revocable rules and a set of facts. The application 102 determines the set of revocable rules from the communicative discourse tree by extracting from the communicative discourse tree one or more of: (i) basic discourse units of the rhetorical relation type contrast; and (ii) communicative actions of the class type disagreement. The class disagreement includes actions such as "deny," "have a different opinion," "disbelieve," "refuse to believe," "contradict," "deviate," "deviate," "go against," "different," "significant," and "dissimilar." Other examples are possible.

[0399] The application 102 determines the following revocability rules:

[0400]

number

[0401] In addition, application 102 determines additional facts from communication actions of type "disagree". Continuing with this example, referring back to FIG. 41, application 102 determines the following facts from the subject of the communication action of CDT: contact_tenant (communication action 4111), rent_is_requested (communication action 4112), rent_refused (communication action 4113), stay_unrepaired (communication action 4114), remind_about_repair (communication action 4115), three_days_notice_is_issued (communication action 4116), and rent_is_overdue (communication action 4117).

[0402] In block 4203, method 4200 includes determining a defeasible derivation that includes a non - conflicting set of defeasible rules from a set of defeasible rules. A defeasible derivation of L from P consists of a finite sequence L1, L2,... L n = L, where each literal L i is within that sequence because (a) L i is a fact in Π or (b) there is a rule R i (absolute or defeasible) in P with head L i and body B1, B2,... B k such that all literals of the body are elements L j (j < i) of the sequence that appear before L j in the sequence. Let h be a literal and P = (Π, Δ) be a DeLP program. <A, h> is an argument for h, where A is a set of defeasible rules of Δ,<000158​​​​​​​​Therefore, the argument<A,h> is the minimal consistent set of retractable rules that can be obtained from a retractable derivation for a given literal h associated with a program P. As mentioned above, a minimal subset means that there is no subset that satisfies conditions 1 and 2.

[0404] At block 4204, the method 4200 includes creating one or more rebuttals from the set of facts. A rebuttal is an argument that can be in a sequence that can be attacked by other arguments, as in human dialogue. An argument line is a sequence of arguments, with each element in the sequence overturning its predecessor. In DeLP, there are several admissibility requirements for argument lines to avoid fallacies (such as circular reasoning by repeating the same argument twice).

[0405] A rebuttal can be formulated as follows:<A1,h1> but<A2,h2> Attacking the<A2,h2> Subargument of<A,h> There exists (A ⊆ A1) such that h and h i is contradictory (i.e., Π∪{h,h i} derives the complementary literal if and only if.<A1,h1> but<A2,h2> What will overturn this is<A1,h1> but<A2,h2> Sub-argument<A,h> Attack with<A1,h1> but<A,h> In the first case,<A1,h1> In the first case, we call it a proper defeater, and in the second case, a blocking defeater.

[0406] At block 4205, the method 4200 includes constructing a dialectical tree from the reversible derivation, with root nodes representing arguments and leaf nodes representing counterarguments. The target argument can be thought of as a DeLP query that is resolved with respect to a dialectical tree that encompasses all possible lines of argument for a given query. The definition of a dialectical tree provides us with an algorithmic perspective for discovering implicit self-attacking relationships in a user's argument.<A0,h0> Let be an argument (target assertion) from program P. For purposes of explanation, block 4205 will be described with respect to FIG.

[0407] Figure 43 is a diagram illustrating an exemplary dialectical tree according to one embodiment. Figure 43 depicts a dialectical tree for the text developed above. Figure 43 includes a dialectical tree 4300 including a root node 4301 and nodes 4302-4307. Dialectical tree 4300 includes:<A0,h0> Based on this, it is defined as follows: 1. The root of the tree (root node 4301) is<A0,h0> is labeled with . 2.N, n ,h n > as a non-root vertex, and Λ=[<A0,h0> ,<A1,h1> ,… n ,h n >] (the sequence of labels of the paths from the root to N).<B0,q0> ,<B1,q1> ,… k ,q k >]To all, n ,h n >. The acceptable line of argument [Λ, i ,q i Each attacker with i ,q i >, there is an arc between N and its child N1.

[0408] The dialectical tree can then be labeled as follows: 1. All leaves (nodes 4302 to 4307) are labeled as U nodes (unrefutable nodes).​​​​​​ 2. Any internal node is labeled as a U-node whenever all of its associated child nodes are labeled as D-nodes. 3. Any internal node is labeled as a D-node whenever at least one of its associated child nodes is labeled as a U-node.

[0409] At block 4206, the method 4200 includes evaluating the dialectical tree by recursively evaluating the counterarguments.

[0410] In the DeLP example, the literal rent_receipt is<A, rent_receipt> = <{ (rent_receipt -< rent_deposit_transaction), (rent_deposit_transaction -< tenant_short_on_money)}, rent_receipt>, for which there are three counterarguments, with three respective lines of argument:

[0411]

number

[0412] (1) and (2) are proper objections, the last one is a blocking objection. The first argument structure has a counterargument, <{rent_deposit_transaction -< tenant_short_on_money}, rent_deposit_transaction), but it is not an objection because the former is more specific. Therefore, there is no objection, and the line of argument ends there.

[0413] B3 above is the blocking rebuttal <{(rent_deposit_transaction -< tenant_short_on_money)}, rent_deposit_transaction>, which is<A, rent_receipt> This is a non-conforming subargument of B2, which cannot be introduced because it would create an impermissible line of argument. B2 has two counterarguments that can be introduced:

[0414]

number

[0415] D1 and C2 have blocking objections, but cannot be introduced because they would make the line of argument impermissible. Therefore, the state rent_receipt cannot be reached because no argument supporting the literal rent_receipt is warranted.

[0416] At block 4207, method 4200 involves identifying the claim supported by the argument as valid in response to determining that none of the counterarguments contradict the revocable derivation. A determination that no contradictory argument exists indicates that the claim is valid, and a determination that contradictory argument exists indicates that the claim is invalid. The rhetorical classifier 102 can then perform an action based on the validation, such as providing a different answer to the user device based on the validity of the claim.

[0417] Argumentative verification results Argument validation is evaluated based on argument detection (using linguistic means) and then validation (logical means). A dataset of 623 legal cases was compiled from Landlord v. Tenant (2018). Each year, this website provides over 700 summaries of recent landlord-tenant court cases and agency decisions. Landlord v. Tenant covers over a dozen courts and agencies, including the New York City Civil Court, the New York City Department of Housing and Community Redevelopment (DHCR), the New York City Board of Environmental Control, and others. The website allows users to gain access to their dynamic database of cases dating back to 1993, as well as the New York Landlord v. Tenant newsletter archive, and to perform searches for specified case summaries. Full decisions and opinion letters are also available from this source.

[0418] A typical case summary reads as follows: "Tenants complained of a reduction in building-wide services. They said that the building superintendent didn't make needed repairs as requested and that the landlord had refused to perform repairs in their apartment. They also complained about building accessibility issues. Among other things, the building side door walkway was reconstructed and made narrower. This made it hard to navigate a wheelchair through that doorway. The DRA ruled against tenants, who appealed and lost." (The tenants complained about a reduction in services throughout the building. They stated that the building manager had not made needed repairs as requested, and that the landlord had refused to make repairs in their apartments. The tenants also complained about accessibility issues in the building. In particular, the building's side doorway had been remodeled and narrowed, making it difficult to pass a wheelchair through the doorway. The DRA found against the tenants, who appealed but lost.) First, we extract sentences containing arguments and then attempt to find assertions conveyed from the DeLP ontology. In the above example, the assertion to be verified is repair_is_done. We then submit this assertion for verification. We obtain assertion validity values ​​from tags on the web page assigned by the judge who heard the case, such as rent_reduction_denied. Table 12 below shows the evaluation results conveyed along with the arguments in the landlord vs. tenant case text.

[0419] [Table 14]

[0420] For the argument detection task, we use this landlord-tenant pair as a positive training set. As a negative dataset, we use a variety of text sources that should not contain argument or opinion data. We also used components of the dataset (Lee, 2001), including Wikipedia, factual news sources, and sections of the corpus such as ['tells'], instructions on how to use software; ['tele'], instructions on how to use hardware; and [news], presenting news articles in an objective and detached style. Further details on the negative, argument-free dataset are available in (Galitsky et al. 2018 and Chapter 10).

[0421] Baseline argument detection methods rely on keywords and syntactic features to detect arguments (Table 13.8). Often, coordinated pairs of communicative actions (where at least one has a negative sentiment polarity associated with the opponent) are a hint that a logical argument is present. This naive method is 29% worse than the top-performing TK-learned CDT method. CDT's SVM TK outperforms RST+CA and RST+full-parsetree (Galitsky, 2017) SVM TK by about 5% due to noisy syntactic data, which is frequently redundant for argument detection.

[0422] While the SVM TK method provides an acceptable F-measure, it does not help explain how the emotional argument identification problem is exactly solved, providing only the final scoring and class labels. The Nearest Neighbor Maximum Common Subgraph algorithm is much more useful in this regard (Galitsky et al., 2015). Comparing the two rows below, we observe that it is possible, but rare, to represent emotional arguments without CA.

[0423] Evaluating the logical arguments extracted from the text, we were interested in cases where a document author provided invalid, inconsistent, or self-contradictory cases. This is important for chatbots as a front-end for CRM systems that focus on customer retention and facilitate communication with customers (Galitsky et al., 2009). We selected the domain of residential real estate complaints and built a DeLP thesaurus for this domain. Automated complaint handling systems can be essential in decision support procedures for property management companies, for example (Constantinos et al., 2003).

[0424] [Table 15]

[0425] In our validity assessment, we focus on target features related to how a given complaint should be dealt with, such as compensation_required, proceed_with_eviction, rent_receipt.

[0426] The validity evaluation results are shown in Table 13. The first and second rows show the results for the simplest complaint with a single rhetorical relation, such as contrast, and a single CA showing the extracted argument-attack relation, respectively. The third and fourth rows show the validation results for legal cases with two non-default rhetorical relations and two CAs of the disagreement type, correspondingly. The fifth row evaluates complaints of average complexity, and the bottom row evaluates the most complex and lengthy complaints in terms of their CDTs. The third column shows the accuracy of the standalone argument validation system for detecting invalid arguments in complaints. Finally, the fourth column shows the accuracy of the integrated argument extraction and validation system.

[0427] In our validity assessment, we focus on the target features (claims) regarding what kind of ruling needs to be issued, such as compensation_required, proceed_with_eviction, rent_receipt, etc. The system decision is based on whether the identified claim is valid or not; if so, the ruling is in favor of this claim, and if not, it is against this claim.

[0428] In these results, recall is low because in the majority of cases, the invalidity of the claim is due to factors other than self-defeating. Precision is relatively high when logical flaws in the argument are established because other factors besides the argument (such as incorrect facts) also contribute, making the entire claim likely invalid. As the complexity of the complaint and its discourse tree increases, F1 first improves because more logical terms are available, and then declines because there is a higher opportunity for inference error due to noisier input.

[0429] It is important for decision support systems to maintain a low false positive rate. While it is acceptable to miss invalid complaints, the confidence level for detected invalid complaints should be fairly high. If a human agent is recommended to view a given complaint as invalid, the agent's expectations should be met most of the time. Although the F1 score of the overall argument detection and validation system is low compared to modern recognition systems, it is still considered usable as a component of a CRM decision support system.

[0430] Syntax Generation Some of the techniques discussed herein, including improved discourse parsers, can use syntactic generalization. Performing syntactic generalization of two sentences involves identifying words in each of the sentences and / or identical parts of speech (POS) in each of the sentences. A lemma refers to a word that does not have associated part-of-speech information. If the lemmas for two words are different but the parts of speech for each word are the same, the parts of speech are part of the generalized result. If the lemmas are the same but the parts of speech are different, the parts of speech are part of the generalized result.

[0431] To illustrate this concept, consider two example natural language expressions. The meaning of each expression is expressed in a logical formula. Syntheses and anti-syntheses of these expressions are constructed. Some words (entities) are mapped to predicates, some words are mapped to their arguments, and some other words do not occur explicitly in the logical formal expression but show the above instantiation of predicates with arguments.

[0432] Consider the following two sentences: "camera with digital zoom" and "camera with zoom for beginners." The following logical predicates are used to express meaning:

[0433] camera(name_of_feature, type_of_users) and zoom(type_of_zoom).

[0434] Note that this is a simplified example and may therefore have a reduced number of arguments compared to more typical examples. Continuing with the example, the above expression can be expressed as: camera(zoom(digital), AnyUser), camera(zoom(AnyZoom), beginner) By notation, variables (unmaterialized values ​​not specified in NL expressions) begin with a capital letter. Given the pair of expressions above, integration computes their most generalized specialization camera(zoom(digital), beginner), and anti-integration computes their most specific generalization camera(zoom(AnyZoom), AnyUser).

[0435] At the syntactic level, these expressions undergo two noun phrase generalizations ('^') as {NN-camera, PRP-with, [digital], NN-zoom [for beginners]}. Expressions in square brackets are removed because they occur in one expression but not the other. The result is the syntactic analog of the semantic generalization: obtain{NN-camera, PRP-with, NN-zoom]}.

[0436] The goal of abstract generalization is to find commonalities between parts of text at various semantic levels. Generalization operations are performed at one or more levels. Examples of levels are the paragraph level, sentence level, phrase level, and word level.

[0437] At each level (except the word level), at the individual word level, the result of the generalization of two expressions is a set of expressions, such that for each pair of expressions where one is less general than the other, the latter is eliminated. The generalization of two sets of expressions is a set of the sets that result from the pairwise generalization of these expressions.

[0438] There is only one generalization for a pair of words: if the words are the same in the same form, the result is the node that has this word in this form. To involve the word2vec model, the following rule is used to calculate the generalization of two different words: if subject1 = subject2, then subject1^subject2 =<subject1, POS(subject1), 1> Otherwise, if they have the same part of speech, then subject1^subject2 =<*,POS(subject1), word2vecDistance(subject1^subject2)>. If the parts of speech are different, the generalization is an empty tuple; it cannot be generalized further.

[0439] For a pair of phrases, the generalization contains all maximal ordered sets of generalization nodes for the words in the phrases such that the word order is preserved. In the following example: "To buy a digital camera today, on Monday." "Digital camera was a good buy today, first Monday of the month." The generalization is<JJ-digital, NN-camera> ,<NN- today, ADV,Monday>}, where the noun phrase generalization is followed by the adverb phrase generalization. The verb buy is excluded from both generalizations because it occurs in a different order in the above phrases. Buy - digital - camera is not a generalization phrase because buy occurs in a different sequence with the other generalization nodes.

[0440] Improved discourse parser A particular aspect relates to an improved discourse parser. Predicting rhetorical relations between two sentences is a goal of discourse analysis, along with text segmentation (dividing sentences into basic discourse units). While a document can be analyzed as a sequence of hierarchical discourse structures, the problem in discourse coherence is how rhetorical relations are signaled by the source text (and can be identified by the parser). For example, rhetorical relations are often signaled by discourse markers such as and, because, however, and while, and when relations contain such markers, they are sometimes classified as explicit relations. Discourse markers are reliable signals of coherence relations.

[0441] However, existing discourse parsers use machine learning techniques and datasets that can be difficult and time-consuming to scale. The task of discourse relation prediction is already complex, compounded by the time-consuming nature of extending these annotated datasets. Thus, many available discourse parsers assign elaboration and join relations when other, more descriptive rhetorical relations would be more suitable. Thus, the recall of establishing other, more specific rhetorical relations can be relatively low.

[0442] However, existing discourse parsers can be improved by performing additional analysis of the discourse tree, e.g., semantic analysis, and adjusting the discourse tree accordingly. In the examples discussed below, a discourse parser is applied to the text, and any resulting elaboration or joint rhetorical relations can be replaced with more suitable rhetorical relations obtained by using semantic analysis (e.g., abstract semantic representation) patterns, if available. Generally, this approach applies to intra-sentence rhetorical relations.

[0443] Figure 44 illustrates a discourse tree and a semantic tree, according to one embodiment. Figure 44 illustrates a discourse tree 4400 and a semantic tree 4410.

[0444] Discourse tree 4400 and semantic tree 4410 respectively represent the following text: "It was a question of life or death for me: I had scarcely enough drinking water to last a week."

[0445] The discourse tree 4410 is represented in text-based form as follows (indentations refer to levels of nesting in the tree): detailed description TEXT:It was a question of life or death for me : detailed description TEXT:I had scarcely enough drinking water TEXT:to last a week. As can be seen from the discourse tree 4410, the second elaboration relation 4412 produced by the discourse parser relating "I had scarcely enough drinking water" and "to last a week" is not as accurate as possible with respect to the text because "to last a week" is simply more of an elaboration of "I had scarcely enough drinking water". This is improved by leveraging AMR relations in the semantic tree 4420. The semantic tree 4410 is also shown below in text-based form.

[0446]

number

[0447] As can be seen in semantic tree 4420, the semantic relation "purpose", identified as relation 4422, has a semantic role related to the verb drink, identified as role 4424. Because discourse tree 4410 and semantic tree 4420 have the entity "drink" in common, it is possible to identify the core EDU ("I had scarcely enough drinking water") that has drink in discourse tree 4400.

[0448] The greater the number of common entities between the discourse tree and the semantic tree template, the better the match for improving the rhetorical relation. Continuing the example, the satellite EDU ("to last a week") is identified and linked with the rhetorical relation "elaboration." Finally, "elaboration" is replaced with "purpose" for a more accurate discourse tree. This link is shown as link 4430.

[0449] Leveraging semantic information improves the discourse tree in certain situations, such as when there is a lack of discourse markers, when discourse markers are ambiguous or misleading, or when the deeper semantic representation of a sentence, such as in AMR, implies a specific rhetorical relationship. Once syntactic similarity between the text being parsed and the AMR patterns is established, the semantic roles from the AMR verbs can be interpreted as the respective rhetorical relationships at the discourse level. This mapping between semantic relationships in AMR and specific rhetorical relationships is established regardless of how the connecting EDUs, core EDUs, and satellite EDUs are connected.

[0450] As a result of manual generalization of the available AMR annotations, a mapping between AMR semantic relations and rhetorical relations was developed and is shown below in Table 14. Table 14 shows examples of semantic roles and corresponding rhetorical relations. In Table 14, the first column lists the rhetorical relations to be detected. The second column represents the AMR semantic relations mapped to the rhetorical relations. The third column provides example sentences that are matched back to the sentence being rhetorically parsed. The fourth column shows the AMR parsing of the template.

[0451] To create this mapping of rhetorical relations to semantic roles, which can be performed offline (e.g., before runtime), a list of rhetorical relations is considered. For each rhetorical relation, the collection of AMR annotations for the particular semantic relation is determined. Once systematic correlations are identified, a corresponding mapping is created, represented by the entries in Table 14. Table 14 shows the rhetorical relations elaborated by the AMR examples.

[0452] [Table 16-1]

[0453] [Table 16-2]

[0454] Table 15 below provides an example of a refined discourse tree in which an elaboration is turned into a specific relation. A template is constructed and refined. The template shows the detected rhetorical relation aspect in bold. The second example shows the actual refinement when the template from the second line below is applied to turn an elaboration into a concession. The syntactic generalization between this template and the sentence is also shown.

[0455] [Table 17]

[0456] To replace the rhetorical relations of an elaboration with those obtained by manual AMR tagging, syntactic similarity is established between the core and satellite basic discourse units of this elaboration and the template. If such similarity is high (patterns from the AMR dataset have been parsed), the elaboration can be overwritten with high confidence. The higher the syntactic similarity score, the higher the confidence that the semantic roles obtained from the pattern accurately describe the rhetorical relations. Without sufficient AMR pattern data and extensive mappings to rhetorical relation data, formal learning of such mappings is difficult. Therefore, this similarity score threshold is used.

[0457] Table 16 shows the co-occurrence values ​​and percentages for lexical, syntactic, and semantic correlations with rhetorical relations. This data helps improve the scoring for "and" and "as" (usually ignored due to syntactic generalizations), as well as for "while," "however," and "because," because they usually have very low scores.

[0458] [Table 18]

[0459] Another example of using semantic relations and roles to refine rhetorical relations is shown in FIG. Figure 45 illustrates a discourse tree and a semantic tree, according to one embodiment. Figure 45 illustrates a discourse tree 4510 and a semantic tree 4520. Discourse tree 4510 represents the text "I ate the most wonderful hamburger that she had ever bought for me." Semantic tree 4520 does not represent the same text as discourse tree 4510. Rather, semantic tree 4520 represents template text that is a good match with the text in discourse tree 4510 and can be used to improve discourse tree 4510.

[0460] The discourse tree 4510 is represented in text form as follows: detailed description TEXT: I ate the most wonderful hamburger TEXT: that she had ever bought for me. As can be seen from discourse tree 4510, the two basic discourse units "I ate the most wonderful hamburger" and "that she had ever bought for me." are connected by the rhetorical relation "elaboration." Therefore, discourse tree 4510 is a good candidate for improvement because "elaboration" may not be the most accurate rhetorical relation.

[0461] The AMR semantic role of compared-to is mapped to the rhetorical relation of comparison. The default discourse parser provides an elaboration that can be turned into a more precise rhetorical relation if the EDU pair with the default rhetorical relation is semantically similar to a template with a specific semantic relation that can be mapped to a rhetorical relation. To establish the precise rhetorical relation between the EDUs in the sentence, a match is attempted against a template found in a set of semantic templates (e.g., an AMR repository). The matched template is for the sentence "It was the most magnificent and stately planet that he had ever seen."

[0462] To match the EDU pair being parsed with a template, the EDU and template are aligned and generalized. In this case, the syntactic generalization between the EDU pair and the template is [VB-* DT-the RBS-most JJ-(wonderful ^magnificent) IN-that PRP-she VB-had RB-ever VB-*], and there is significant evidence that the sentence and pattern being parsed share a common syntactic structure. For example, wonderful ^magnificent generates an abstract adjective with the meaning of what is common between these adjectives. Connection 4530 shows the correspondence between the adjective magnificent in the AMR representation and the adjective wonderful in the original DT.

[0463] Therefore, the elaboration of discourse tree 4500 is replaced with a rhetorical relation of type "comparison". The corrected discourse tree is as follows: comparison TEXT: I ate the most wonderful hamburger TEXT: that she had ever bought for me. Based on the above example, the process of improving the discourse tree will be further explained.

[0464] 46 is a flowchart of an example process 4600 for generating an improved discourse tree, according to one aspect. It should be understood that in some cases, one or more operations in the process 4600 may not be performed. The process 4600 may be performed by the application 102.

[0465] At block 4602, process 4600 includes creating a discourse tree from the text by identifying basic discourse units within the text. At block 4602, process 4600 includes operations substantially similar to block 1502 of process 1500. The determined discourse tree includes nodes, each non-terminal of which represents a rhetorical relationship between two basic discourse units, and each terminal of which is associated with a basic discourse unit.

[0466] At block 4604, process 4600 includes identifying a rhetorical relation of type elaboration or joint in the discourse tree. The rhetorical relation relates two basic discourse units, e.g., a first basic discourse unit and a second basic discourse unit (rather than relating two other rhetorical relations or one rhetorical relation and one basic discourse unit).

[0467] The first basic discourse unit and the second basic discourse unit form a referential sentence. For example, referring again to Figure 45, the first EDU is "I ate the most wonderful hamburger" and the second EDU is "that she had ever bought for me", and the rhetorical relationship (before updating) is "elaboration."

[0468] At block 4606, process 4600 includes determining a syntactic generalization score for each candidate sentence in the set of candidate sentences. As described above in Tables 14 and 15, each candidate sentence has a corresponding semantic relationship (e.g., an AMR representation). In a simplified example, the syntactic generalization score is the number of common entities between the reference sentence and the candidate sentence. Each of the common entities shares a common part of speech between the candidate sentence and the reference sentence. However, the syntactic generalization score may be calculated differently in other aspects, as described below.

[0469] The goal of abstract generalization is to find commonalities between pieces of text at various semantic levels. Generalization can be done at the paragraph, sentence, EDU, phrase, and individual word levels. Except at the word level, the result of generalizing two expressions is a set of expressions. In such a set, for each pair of expressions where one is less common than the other, the latter is eliminated. A generalization of two sets of expressions is a set of sets of expressions that are the result of pairwise generalization of these expressions. For illustrative purposes, Figure 46 will be discussed with reference to Figure 47, which shows generalization, and Figure 48, which shows alignment.

[0470] Figure 47 illustrates a generalization of sentences and templates with known semantic relationships, according to one embodiment. Figure 47 illustrates a generalization of sentence 4710 "If you read a book at night, your knowledge will improve" and template 4720 from Table 14 "If one gets lost in the night, such knowledge is valuable." The resulting generalization 4730 is as follows: [IN-If PRP-* VB-* ... NN-night ... NN-knowledge ] In this template, IN-If PRP-* VB-* is a signature of the semantic relation of :condition() and is also a discourse relation of condition, but there happen to be more general words, such as "NN-night ... NN-knowledge", that may or may not be used to establish similarity between the sentence and the template.

[0471] To determine how to calculate an appropriate generalization score, a computational study was conducted to determine the part-of-speech weights that would yield the most accurate measure of similarity between sentences. The problem was formulated as finding optimal weights for nouns, adjectives, verbs, and their forms (e.g., gerunds and past tense) such that the resulting search relevance was maximized. Search relevance was measured as the deviation in the order of search results from the best for a given query, and the current search order was determined based on the generalization score for a given set of POS weights (with other generalization parameters fixed). As a result of this optimization performed, W VBcommon With the exception of common frequent verbs such as get, take, set, and put, where W = 0.57 NN =1.0, W JJ =0.32, W RB =0.71, W CD =0.64, W VB =0.83, W PRP = 0.35 is obtained. <POS,*> is set to 0.2 (different words but same POS), and W <*,word> = 0.3 (same word, but occurs in two sentences as different POS). {and,as,but,while,however,because} is computed as a default value of 1 normalized to the values ​​in the second column of Table 16. Note that the default syntactic generalization ignores discourse indicators in most cases.

[0472] Then, the generalization score between the reference sentence (ref_sentence) and the candidate template (Template) is calculated by the word Wword ref_sentence and word template can be expressed as a sum over phrases of weighted sums over

[0473]

number

[0474] The maximum generalization can then be defined as the one with the highest score. At the phrase level, generalization begins by finding an alignment between two phrases (as many word correspondences as possible between the two phrases). The alignment operation is performed such that phrase completeness is preserved. For example, two phrases can be aligned only if a correspondence between their initial nouns is established. There are similar completeness constraints for aligning verbs, prepositions, and other types of phrases.

[0475] Figure 48 shows an alignment between two sentences, according to one embodiment. Figure 48 shows an alignment between sentence 4810, which reads "use the screw driver from this tool for fixing heaters," and sentence 4820, which reads "get short screw driver holder for electric heaters." The resulting alignment 4830 is as follows: VB-* JJ-* NN-zoom NN-* IN-for NN-* In some aspects, separate generalizations of the nucleus and satellites can be used to generate an improved discourse tree. For example, a discourse tree is created as in block 4602 of process 4600. From the discourse tree, rhetorical relations are identified. Suitable rhetorical relations are identified. Examples of suitable rhetorical relations include innermost relations of elaboration and joint, as well as nested relations (elaboration on another elaboration [on another elaboration]).

[0476] Core EDUs and satellite EDUs are identified. If they are too complex or long, these EDUs can be reduced in size and / or complexity. The core EDUs are generalized with each template (e.g., Table 14 and / or Table 15). Similar to block 4608, the candidate sentence with the highest generalization score is selected. If the score exceeds a threshold, the satellite EDU corresponding to the rhetorical relation is generalized with the template. If the generalization score of the satellite EDU exceeds a threshold, the rhetorical relation is used to replace the rhetorical relation in the reference sentence. Examples of generalization thresholds are 2.0 (for core) and 3.3 (for satellite).

[0477] Returning to FIG. 46, at block 4608, the process 4600 includes selecting the candidate sentence having the highest syntactic generalization score among the syntactic generalization scores.

[0478] In some aspects, no match is found. For example, the application 102 searches an Abstract Semantic Representation (AMR) dataset (e.g., Table 14 and / or Table 15) to identify that the identified semantic relation is not present in the AMR dataset, and then replaces the rhetorical relation in the discourse tree with the additional semantic relation present in the AMR dataset.

[0479] At block 4610, the process 4600 includes identifying semantic relationships corresponding to the candidate sentence. The semantic relationships correspond to words in the candidate sentence and define roles in the candidate sentence. For example, the semantic relationships in the candidate sentence are identified in Table 14 and / or Table 15.

[0480] At block 4612, process 4600 includes replacing the rhetorical relation with an updated rhetorical relation that corresponds to the semantic relation in the discourse tree, thereby creating an updated discourse tree. A rhetorical relation that matches the semantic relation identified at block 4610 is identified. The identified rhetorical relation is inserted into the discourse tree in place of the rhetorical relation identified at block 4604.

[0481] Levels of generalization: from syntax to semantics to discourse To demonstrate how syntactic generalizations rise from the syntactic level to the semantic level, we can follow Mill's law of direct correspondence (induction) applied to linguistic structures. In his 1843 book, A System of Logic, the British philosopher J.S. Mills wrote, "When two or more instances of the phenomenon under investigation have in common only one circumstance, the circumstance with which all the instances agree is the cause (or effect) of the given phenomenon."

[0482] Consider a linguistic property A of a phrase f. For A to be a necessary condition for some effect E, A must always be present in several phrases that address E. In the field of linguistics, A is a linguistic structure and E is its meaning. Thus, the presence or absence of linguistic properties considered as "possible necessary conditions" in a sentence is verified. Clearly, any linguistic property A that does not exist when meaning E exists cannot be a necessary condition for this meaning E of a phrase.

[0483] For example, the matching method is to find if the word {ABCD} <wxyz>can be expressed as a phrase f1 that occurs with a meaning formally expressed as: <wtuv>Consider also another phrase f2 that occurs with {ABCD}. Now, by applying generalization to the words {ABCD} and {AEFG}, we obtain {A} (here, for the sake of the example, we ignore the syntactic structure of f1 and f2). Thus, we see that the word A is the cause of w (has the sense w). Throughout this book, we will consider linguistic structures that cover ABCD, in addition to the list itself, and apply agreement methods.

[0484] Therefore, we can generate (inductive) semantics by applying syntactic generalization. While semantics cannot be obtained by considering only the syntactic information of a sample, generalizing two or more phrases (samples) yields not only a syntactic structure but also an (inductive) semantic structure. Viewing syntactic generalization as an inductive cognitive procedure, we can formally define the transition from the syntactic level to the semantic level. In this study, we do not confuse syntactic and semantic features to learn classes; instead, we derive semantic features from syntax according to the inductive framework described above.

[0485] Improved parser evaluation False positives and false negatives are identified in refining the rhetorical relations of elaboration by analyzing the functionality of downstream applications of discourse parsing, such as summarization, dialogue management, and argument analysis.

[0486] If the elaboration is a correct relation but is changed to a more specific relation, the obtained search results may not match the query and some sentences may not occur in the result summary as the rules are adjusted for the elaboration. To make the retrieval and summarization system less sensitive to false positives obtained by our refinement system, the matching and selection rules need to be updated to take into account that causes, concessions, and conditions are partial cases of elaboration, and discourse trees with these node labels should be matched accordingly.

[0487] If elaboration is not overridden with a more specific relationship, the accuracy of downstream systems will be adversely affected. If a specific, unique relationship in a question is identified, it must be addressed in the answer, and therefore, if it remains elaboration, the selection of a specific answer that matches the question in style will fail. Less relevant sentences or phrases may be included in the summary, or the number of candidates for such inclusion will be reduced. We conclude that false negatives are worse than false positives.

[0488] Another consideration concerns the genre of the training dataset for discourse parsing. The RST Discourse Treebank (RST-DT, Carlson et al., 2001) contains news articles and is not a good source for modeling text structure in other genres, such as fiction, scientific texts, engineering system descriptions, and legal documents. Applications of discourse parsing in these domains are important. Therefore, while detail may be sufficient for news presentations, more specific structure is needed to model author reasoning in other genres and domains, such as specialized texts.

[0489] [Table 19]

[0490] Four problems are used to evaluate the developed discourse parser refinement (Table 16): 1) Searching for complex and long questions, implementing alignment between questions and answers (Section 2), using the Yahoo!Answers dataset; 2) Finding dialogue-forming utterances from text or documents, using the dataset and techniques developed in Volume 2, Chapter 1. This technique relies heavily on discourse parsing; 3) Generating a dialogue from an initial query (Volume 2, Chapter 1), taking into account rhetorical relations to filter out undesirable utterances (Galitsky and Ilvovsky 2016); 4) Assessing the veracity of a document through its discourse structure (Volume 2, Chapter 6). The structure of the discourse tree, how the document author expresses information, along with which rhetorical relations are used, is important for detecting lies or fake news.

[0491] We can observe that in utterance classification and text classification, we achieve improvements of over 4% by subdividing discourse parser results. However, in search problems, which are less sensitive to the appropriate rhetorical relation, we achieve improvements closer to 3%. We now proceed to evaluate how individual rhetorical relation subdivisions perform (Table 17). We use one AMR corpus for training and another for testing.

[0492] [Table 20]

[0493] Lines 3-8 analyze the relation detection results for individual relation types, such as contrast. The baseline parser recognizes contrast along with cause and condition, but not the remaining relations. This is due to the fact that these specific and rare relations are not well represented in the Discourse TreeBank. Overall, for only these rhetorical relations, the performance of the baseline classifier weighted by the quantity of these relations is quite low. Learning from AMR, it is possible to achieve a performance of 77.7 on this data. Thus, we obtain an improvement of 36.1 for these six relation subdivisions. This is important for a task where the emphasis is on specific relation types and not on the discourse tree structure.

[0494] Overall, because these six relations are rare, the parser improvement results in an average improvement of up to 6% across all rhetorical relations, which is still valuable for the application of discourse parsers.

[0495] Related Work The six rhetorical relations that are the focus of discourse parser improvements are crucial for several applications of retrieval, summarization, and text classification into abstract classes. For retrieval, when a question involves any of these six relations, a good comprehensive answer cannot include only the default rhetorical relations but instead properly communicates to the document author the specific relations at issue. Such an answer cannot simply be a kindred fact answer. The contrast needs to be conveyed by presenting both a thesis and a counter-thesis, perhaps using an argumentative pattern. Causes must be communicated through one form or another of explanation, providing reasons and setting up a framework for inference. Conditions and comparisons must be addressed by communicating and presenting factual data in both parts (If and Then, or Item A vs. Item B).

[0496] Therefore, if these six relationships in a user question are not properly recognized, the communication aspect is likely to present an incorrect answer, leaving the user frustrated with the search, even if it is relevant in terms of the queried entities and their attributes.

[0497] The Penn Discourse Treebank (PDTB, Prasad et al. 2017) is a primary resource for training discourse parsers. Version 3.0 is the third release of the Penn Discourse Treebank project, which aims to annotate the Wall Street Journal section of Treebank-2 with discourse relations. Penn Discourse Treebank Version 3 contains over 53,600 annotated relational tokens. We normalized some pairwise annotations, included new semantics, and performed consistency checks across the corpus. Further details on the development of PDTB are available (PDTB 2019). Because PDTB only includes the news genre, the performance of discourse parsers trained on it in other genres is limited. In this work, we attempted to address these limitations by using a source of semantic relations from which discourse relations can be deduced.

[0498] The PDTB project was inspired by the observation that discourse relations are based on identifiable sets of explicit words or phrases (discourse conjunctions, discourse directives) or simply the adjacency of two sentences. PDTB has been used by many researchers in the field of natural language processing and, more recently, by researchers in psycholinguistics. It has also stimulated the development of similar resources in other languages ​​and domains. There is a strong mismatch between the complexity on which PDTB 3.0 is built and the complexity that discourse parsers can handle. Discourse parsers trained on previous models perform very poorly in recognizing the very specific relations used in PDTB 3.0. Therefore, the improvements described in this chapter are essential for downstream applications of discourse parsing.

[0499] Exemplary Computing System 49 is a simplified diagram showing a distributed system 4900 for implementing one of the above aspects. In the illustrated aspect, the distributed system 4900 includes one or more client computing devices 4902, 4904, 4906, and 4908 configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., via one or more networks 4910. A server 4912 may be communicatively coupled to the remote client computing devices 4902, 4904, 4906, and 4908 via the network 4910.

[0500] In various aspects, server 4912 may be adapted to execute one or more services or software applications provided by one or more of the system's components. The services or software applications may include non-virtual and virtual environments. Virtual environments, whether two-dimensional or three-dimensional (3D) representations, page-based logical environments, or the like, may include those used for virtual events, trade shows, simulators, classrooms, purchasing transactions, and business activities. In some aspects, these services may be provided to users of client computing devices 4902, 4904, 4906, and / or 4908 as web-based or cloud services or under a Software as a Service (SaaS) model. Users operating client computing devices 4902, 4904, 4906, and / or 4908 may then utilize one or more client applications to interact with server 4912 and utilize the services provided by these components.

[0501] In the illustrated configuration, software components 4918, 4920, and 4922 of system 4900 are shown implemented on server 4912. In other aspects, one or more of the components of distributed system 4900 and / or the services provided by these components may be implemented by one or more of client computing devices 4902, 4904, 4906, and / or 4908. A user operating a client computing device may then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible that may differ from distributed system 4900. Thus, the illustrated aspect is an example of a distributed system for implementing the aspect system and is not intended to be limiting.

[0502] Client computing devices 4902, 4904, 4906, and / or 4908 may be handheld mobile devices (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display) running software such as Microsoft Windows® Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and supporting Internet, email, short message service (SMS), BlackBerry®, or other available communication protocols. Client computing devices may also be general-purpose personal computers, including, by way of example, personal and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. The client computing devices may be workstation computers running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS. Alternatively, or additionally, the client computing devices 4902, 4904, 4906, and 4908 may be thin client computers, Internet-enabled gaming systems (e.g., Microsoft Xbox game consoles with or without Kinect gesture input devices), and / or other electronic devices such as personal messaging devices capable of communicating over the network 4910.

[0503] Although the exemplary distributed system 4900 is shown with four client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 4912.

[0504] Network 4910 in distributed system 4900 may be any type of network familiar to those skilled in the art capable of supporting data communications using any of a variety of commercially available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. By way of example only, network 4910 may be a local area network (LAN), such as one based on Ethernet, token ring, etc. Network 4910 may also be a wide area network and the Internet. Network 4910 may include a virtual network, including, but not limited to, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under the Institute of Electrical and Electronics (IEEE) 802.28 suite of protocols, Bluetooth and / or other wireless protocols), and / or any combination thereof, and / or other networks.

[0505] The servers 4912 may be comprised of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or other suitable configurations and / or combinations. The servers 4912 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. One or more flexible pools of logical storage may be virtualized to maintain virtual storage devices for the servers. Virtual networks may be controlled by the servers 4912 using software-defined networking. In various aspects, the servers 4912 may be adapted to execute one or more services or software applications described in the above disclosure. For example, the servers 4912 may correspond to servers for executing the above-described processes according to aspects of the present disclosure.

[0506] Server 4912 may run an operating system, including any of those described above, as well as any commercially available server operating system. Server 4912 may also run any of a variety of additional server and / or middle-tier applications, including a hypertext transport protocol (HTTP) server, a file transfer protocol (FTP) server, a common gateway interface (CGI) server, a JAVA server, a database server, etc. Exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM International Business Machines, etc.

[0507] In some implementations, server 4912 may include one or more applications for analyzing and consolidating data feeds and / or event updates received from users of client computing devices 4902, 4904, 4906, and 4908. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams, including real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 4912 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of client computing devices 4902, 4904, 4906, and 4908.

[0508] Distributed system 4900 may also include one or more databases 4914 and 4916. Databases 4914 and 4916 may reside in a variety of locations. As one example, one or more of databases 4914 and 4916 may reside on non-transitory storage media local to (and / or resident in) server 4912. Alternatively, databases 4914 and 4916 may be remote from server 4912 and communicate with server 4912 via a network-based or dedicated connection. In one set of aspects, databases 4914 and 4916 may reside on a storage-area network (SAN). Similarly, any necessary files for performing functions attributed to server 4912 may be stored locally and / or remotely on server 4912, as appropriate. In one set of aspects, databases 4914 and 4916 may include relational databases, such as those provided by Oracle Corporation, adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0509] 50 is a simplified block diagram of one or more components of a system environment 5000 capable of delivering services provided by one or more components of an aspect of a system as cloud services, according to one aspect of the present disclosure. In the illustrated aspect, the system environment 5000 includes one or more client computing devices 5004, 5006, and 5008 that can be used by users to interact with a cloud infrastructure system 5002 that provides cloud services. The client computing devices can be configured to run client applications, such as a web browser, a proprietary client application (e.g., Oracle Forms), or other applications, that can be used by users of the client computing devices to interact with the cloud infrastructure system 5002 to use services provided by the cloud infrastructure system 5002.

[0510] It should be understood that the cloud infrastructure system 5002 depicted in the figure may have components other than those shown. Moreover, the illustrated aspect is only one example of a cloud infrastructure system that may incorporate aspects of the present invention. In some other aspects, the cloud infrastructure system 5002 may have more or fewer components than those depicted in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0511] Client computing devices 5004, 5006 and 5008 may be similar devices to those described above for 4902, 4904, 4906 and 4908.

[0512] Although the exemplary system environment 5000 is shown with three client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, etc., may interact with the cloud infrastructure system 5002.

[0513] Network 5010 may facilitate communication and exchange of data between clients 5004, 5006, and 5008 and cloud infrastructure system 5002. Each network may be any type of network familiar to those skilled in the art capable of supporting data communication using any of a variety of commercially available protocols, including those described above for network 4910.

[0514] Cloud infrastructure system 5002 may comprise one or more computers and / or servers, which may include those described above for server 4912.

[0515] In certain aspects, the services provided by a cloud infrastructure system may include numerous services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, and managed technical support services. The services provided by the cloud infrastructure system are dynamically scalable to meet the needs of its users. A specific instantiation of a service provided by the cloud infrastructure system is referred to herein as a "service instance." In general, any service available to users from a cloud service provider's system over a communications network such as the Internet is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems comprising the cloud service provider's system are distinct from the customer's own on-premises servers and systems. For example, a cloud service provider's system may host an application, and users may order and use the application on demand over a communications network such as the Internet.

[0516] In some examples, services in a computer network cloud infrastructure may include storage, hosted databases, hosted web servers, protected computer network access to software applications, or other services provided to users by a cloud vendor or otherwise known in the art. For example, a service may include password-protected access to remote storage on the cloud over the Internet. As another example, a service may include a web-services-based hosted relational database and scripting language middleware engine for private use by networked developers. As another example, a service may include access to an email software application hosted on a cloud vendor's website.

[0517] In certain aspects, cloud infrastructure system 5002 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such a cloud infrastructure system is the Oracle Public Cloud offered by the present assignee.

[0518] Large amounts of data, sometimes referred to as big data, can be hosted and / or manipulated by infrastructure systems at multiple levels and at different scales. Such data can include data sets so large and complex that they can be difficult to process using typical database management tools or traditional data processing applications. For example, terabytes of data may be difficult to store, retrieve, and process using personal computers or their rack-based counterparts. Data of this size can be difficult to work with using modern relational database management systems and desktop statistics and visualization packages. They may require massively parallel processing software running thousands of server computers, beyond the architecture of commonly used software tools, to capture, curate, manage, and process the data within acceptable elapsed times.

[0519] Analysts and researchers can store and process extremely large data sets to visualize, detect trends, and / or interact with large amounts of data. Dozens, hundreds, or even thousands of processors linked in parallel can act on such data, thereby displaying it or simulating forces on it or what it represents. These data sets may involve structured data, such as data organized in databases or according to structured models, and / or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing, etc.). By enhancing the ability of a system to relatively quickly converge more (or fewer) computing resources on a target, cloud infrastructure systems can be better utilized to perform tasks on large data sets based on requests from businesses, government agencies, research organizations, private individuals, like-minded groups of individuals or organizations, or other entities.

[0520] In various aspects, cloud infrastructure system 5002 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 5002. Cloud infrastructure system 5002 may provide cloud services through a variety of deployment models. For example, services may be provided under a public cloud model, in which cloud infrastructure system 5002 is owned by an organization that sells cloud services (e.g., owned by Oracle Corporation) and the services are available to the general public or various industry enterprises. As another example, services may be provided under a private cloud model, in which cloud infrastructure system 5002 is operated solely for a single organization and may provide services to one or more entities within that organization. Cloud services may also be provided under a community cloud model, in which cloud infrastructure system 5002 and the services provided by cloud infrastructure system 5002 are shared by several organizations within an associated community. Cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0521] In some aspects, the services provided by cloud infrastructure system 5002 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other categories of services, including hybrid services. A customer may order one or more services provided by cloud infrastructure system 5002 via a subscription order. Cloud infrastructure system 5002 then performs processing to provide the services in the customer's subscription order.

[0522] In some aspects, services provided by the cloud infrastructure system 5002 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, the SaaS platform may provide functionality for building and delivering a suite of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing the SaaS services. By utilizing the services provided by the SaaS platform, customers can utilize applications that run on the cloud infrastructure system. Customers can obtain application services without the need for customers to purchase separate licenses and support. A variety of different SaaS services may be offered. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0523] In some aspects, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall into the PaaS category. Examples of platform services include, but are not limited to, services that enable organizations (e.g., Oracle) to integrate existing applications on a shared, common architecture and the ability to build new applications that leverage shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure to provide the PaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without the need for the customers to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS) and Oracle Database Cloud Service (DBCS).

[0524] By utilizing services provided by a PaaS platform, customers can utilize programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some aspects, platform services provided by a cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion middleware services), and Java cloud services. In one aspect, database cloud services may support a shared service deployment model that enables organizations to pool database resources and provide database-as-a-service to customers in the form of a database cloud. Middleware cloud services may provide customers with a platform for developing and deploying various business applications in the cloud infrastructure system, and Java cloud services may provide customers with a platform for deploying Java applications in the cloud infrastructure system.

[0525] A variety of different infrastructure services may be provided by the IaaS platform in a cloud infrastructure system. The infrastructure services facilitate the management and control of basic computing resources such as storage, network, and other underlying computing resources for customers who use the services provided by the SaaS and PaaS platforms.

[0526] Additionally, in certain aspects, cloud infrastructure system 5002 may include infrastructure resources 5030 for providing resources used to provide various services to customers of the cloud infrastructure system. In one aspect, infrastructure resources 5030 may include a pre-integrated, optimized combination of hardware, such as servers, storage, and networking resources, for running the services provided by the PaaS and SaaS platforms.

[0527] In some aspects, resources in cloud infrastructure system 5002 may be shared by multiple users and dynamically reallocated on a demand basis. Resources may also be allocated to users in different time periods. For example, cloud infrastructure system 5030 may maximize resource utilization by allowing a first set of users in a first time period to utilize resources of the cloud infrastructure system for a specified time period and allowing reallocation of the same resources to another set of users located in a different time period.

[0528] In certain aspects, several internal shared services 5032 may be provided that are shared by various components or modules of, and services provided by, cloud infrastructure system 5002. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services to enable cloud support, email services, notification services, file transfer services, etc.

[0529] In certain aspects, cloud infrastructure system 5002 may provide comprehensive management of cloud services (e.g., SaaS services, PaaS services, and IaaS services) in the cloud infrastructure system. In one aspect, cloud management functionality may include functionality for provisioning, managing, tracking, etc., customer subscriptions received by cloud infrastructure system 5002.

[0530] In one aspect, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 5020, an order orchestration module 5022, an order provisioning module 5024, an order management and monitoring module 5026, and an identity management module 5028. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or other suitable configurations and / or combinations.

[0531] In example operation 5034, a customer using a client device, such as client device 5004, 5006, or 5008, may interact with cloud infrastructure system 5002 by requesting one or more services offered by cloud infrastructure system 5002 and placing an order for a subscription to one or more services provided by cloud infrastructure system 5002. In particular aspects, the customer may access a cloud user interface (UI), i.e., cloud UI 5012, cloud UI 5014, and / or cloud UI 5016, and place a subscription order via these UIs. Order information received by cloud infrastructure system 5002 in response to the customer placing an order may include information identifying the customer and the one or more services offered by cloud infrastructure system 5002 to which the customer intends to subscribe.

[0532] After an order is placed by a customer, the order information is received via the cloud UI 5050, 5014 and / or 5016.

[0533] In operation 5036, the order is stored in order database 5018. Order database 5018 may be one of several databases operated by cloud infrastructure system 5018 and in cooperation with other system elements.

[0534] At operation 5038, the order information is forwarded to the order management module 5020. In some examples, the order management module 5020 may be configured to perform billing and accounting functions related to the order, such as confirming the order and booking the order upon confirmation.

[0535] At operation 5040, information regarding the order is communicated to the order orchestration module 5022. The order orchestration module 5022 may utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some examples, the order orchestration module 5022 may orchestrate the provisioning of resources to support the subscribed service using the services of the order provisioning module 5024.

[0536] In certain aspects, the order orchestration module 5022 enables management of business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. At operation 5042, upon receiving an order for a new subscription, the order orchestration module 5022 sends a request to the order provisioning module 5024 to allocate resources and configure those resources needed to fulfill the subscription order. The order provisioning module 5024 enables allocation of resources for services ordered by a customer. The order provisioning module 5024 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 5000 and the physical implementation layer used to provision resources to provide the requested services. Thus, the order orchestration module 5022 can be decoupled from implementation details such as whether services and resources are actually provisioned on the fly or pre-provisioned and only allocated / assigned when requested.

[0537] In operation 5049 , once the services and resources are provisioned, notification of the provided services may be sent to the customer on the client computing devices 5004 , 5006 and / or 5008 by the order provisioning module 5024 of the cloud infrastructure system 5002 .

[0538] At operation 5051, a customer's subscription order may be managed and tracked by the order management and monitoring module 5026. In some examples, the order management and monitoring module 5026 may be configured to collect usage statistics for the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system up time and system down time.

[0539] In certain aspects, cloud infrastructure system 5000 may include identity management module 5028. Identity management module 5028 may be configured to provide identity services, such as access management and authorization services, in cloud infrastructure system 5000. In some aspects, identity management module 5028 may control information about customers who wish to use services provided by cloud infrastructure system 5002. Such information may include information authenticating the identities of such customers and information describing which actions those customers are authorized to perform on various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). Identity management module 5028 may also include management of descriptive information about each customer and how and by whom this descriptive information may be accessed and modified.

[0540] 51 illustrates an exemplary computer system 5100 capable of implementing various aspects of the present invention. The computer system 5100 can be used to implement any of the computer systems described above. As shown in the figure, the computer system 5100 includes a processing unit 5104 that communicates with several peripheral subsystems via a bus subsystem 5102. These peripheral subsystems may include a processing acceleration unit 5106, an I / O subsystem 5108, a storage subsystem 5118, and a communications subsystem 5124. The storage subsystem 5118 includes a tangible computer-readable storage medium 5122 and a system memory 5110.

[0541] The bus subsystem 5102 provides a mechanism for allowing the various components and subsystems of the computer system 5100 to communicate with each other as intended. While the bus subsystem 5102 is shown schematically as a single bus, alternative implementations of the bus subsystem may utilize multiple buses. The bus subsystem 5102 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured to the IEEE P3086.1 standard.

[0542] Processing unit 5104, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of computer system 5100. Processing unit 5104 may include one or more processors. These processors may include single-core or multi-core processors. In particular aspects, processing unit 5104 may be implemented as one or more independent processing units 5132 and / or 5134, each with a single-core or multi-core processor included therein. In other aspects, processing unit 5104 may be implemented as a quad-core processing unit formed by incorporating two dual-core processors on a single chip.

[0543] In various aspects, processing unit 5104 may execute various programs in response to program code and may maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in processing unit 5104 and / or storage subsystem 5118. Through suitable programming, processing unit 5104 may provide the various functions described above. Computer system 5100 may also additionally include a processing acceleration unit 5106, which may include a digital signal processor (DSP), special purpose processor, etc.

[0544] The I / O subsystem 5108 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen, which may be integrated with a display, a scroll wheel, a click wheel, a dial, buttons, switches, a keypad, an audio input device, along with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may include, for example, a motion detection and / or gesture recognizer, such as a Microsoft Kinect® motion sensor, which allows a user to control and interact with an input device, such as a Microsoft Xbox® 360 game controller, through a natural user interface using gestures and spoken commands. User interface input devices may also include an eye gesture recognizer, such as a Google Glass® blink detector, which detects eye movements from a user (e.g., "blinking" while taking pictures and / or making menu selections) and translates the eye gesture as input to the input device. The user interface input devices may also include a voice recognition sensing device that allows a user to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0545] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphics tablets, audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser range finders, and gaze detection devices. User interface input devices may also include medical imaging input devices such as computed tomography, magnetic resonance imaging, positional emission tomography, and medical ultrasound equipment. User interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0546] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. The display subsystem may be a flat panel display, such as one using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, a projection device, a touch screen, or the like. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 5100 to a user or another computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, voice output devices, and modems.

[0547] Computer system 5100 may include a storage subsystem 5118 that comprises software elements presently shown as located in system memory 5110. System memory 5110 may store program instructions that are loadable and executable on processing unit 5104, as well as data generated during the execution of these programs.

[0548] Depending on the configuration and type of computer system 5100, the system memory 5110 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or presently being operated on and executed by the processing unit 5104. In some implementations, the system memory 5110 may include several different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 5100, such as during start-up, may typically be stored in ROM. By way of example and not limitation, system memory 5110 also shows application programs 5112, program data 5114, and operating system 5116, which may include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), and the like.By way of example, operating system 5116 may include various versions of the Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0549] The storage subsystem 5118 may also provide a tangible, computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of certain aspects. Software (programs, code modules, instructions) that, when executed by a processor, provide the functionality described above may be stored in the storage subsystem 5118. These software modules or instructions may be executed by the processing unit 5104. The storage subsystem 5118 may also provide a repository for storing data used in accordance with the present invention.

[0550] The storage subsystem 5118 may also include a computer-readable storage medium reader 5120 that may further be connected to a computer-readable storage medium 5122. Together and optionally in combination with the system memory 5110, the computer-readable storage medium 5122 may comprehensively represent remote, local, fixed, and / or removable storage devices, as well as storage media for temporarily and / or permanently containing, storing, transmitting, and retrieving computer-readable information.

[0551] The computer-readable storage medium 5122 containing the code or portions of code may include any suitable medium known or used in the art, including, but not limited to, storage and communication media, such as volatile and nonvolatile, removable and non-removable media, implemented in any method or technology for storing and / or transmitting information. This may include tangible, transitory computer-readable storage media, such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage, or other tangible computer-readable media. It may also include intangible, transitory computer-readable media, such as a data signal, data transmission, or any other medium usable to transmit the desired information and accessible by computer system 5100.

[0552] By way of example, computer-readable storage medium 5122 may include a hard disk drive that reads from or writes to non-removable, non-volatile magnetic media, a magnetic disk drive that reads from or writes to removable, non-volatile magnetic disks, and an optical disk drive that reads from or writes to removable, non-volatile optical disks such as CD-ROMs, DVDs and Blu-ray disks or other optical media. Computer-readable storage medium 5122 may include, but is not limited to, Zip drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (...

Claims

1. 1. A method performed by a system to improve the accuracy of a discourse tree, comprising: creating a discourse tree from the text by identifying basic discourse units in the text, the discourse tree including a plurality of nodes, each non-terminal node of the nodes in the discourse tree representing a rhetorical relationship between two basic discourse units, and each terminal node of the nodes in the discourse tree being associated with a basic discourse unit, the method further comprising: identifying a rhetorical relation of type elaboration or joint in the discourse tree, the rhetorical relation relating a first basic discourse unit and a second basic discourse unit, the first basic discourse unit and the second basic discourse unit forming a reference sentence, the method further comprising: determining a syntactic generalization score for each candidate sentence of the set of candidate sentences, each candidate sentence having a corresponding semantic relation, said determining including: identifying one or more common entities between the candidate sentence and the reference sentence; calculating a syntactic generalization score equal to the number of the identified one or more common entities, the method further comprising: selecting a candidate sentence having a highest syntactic generalization score from among the syntactic generalization scores; and identifying semantic relationships corresponding to the candidate sentence, the semantic relationships corresponding to words in the candidate sentence and defining roles in the candidate sentence, the method further comprising: creating an updated discourse tree by replacing the rhetorical relations in the discourse tree with updated rhetorical relations that correspond to the semantic relations.

2. Creating the discourse tree from the text comprises: providing the text to a classification model; 2. The method of claim 1, further comprising: using the classification model to identify the first basic discourse unit, the second basic discourse unit, and the rhetorical relation.

3. The method of claim 1 or 2, wherein the updated rhetorical relationship is one of an end, a means, a cause, or a temporal sequence.

4. forming a response from the updated discourse tree; The method of any one of claims 1 to 3, further comprising: outputting the response to an external device.

5. The method further comprises: forming a first syntactic parse tree from each candidate sentence; and forming a second syntactic parse tree from the reference sentence, wherein identifying the one or more common entities between the candidate sentence and the reference sentence comprises, for each common entity, identifying the common entity in the first syntactic parse tree and the second syntactic parse tree.

6. The method further comprises: forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature; identifying the text as containing an argument by applying a classification model trained to detect arguments to the communicative discourse tree; forming a response from the text and outputting the response to an external device.

7. The method further comprises: forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature; identifying that the text contains arguments corresponding to claims by applying a classification model trained to detect arguments to the communicative discourse tree; and evaluating the consistency of the argument with respect to itself and with respect to domain-defining clauses associated with the domain of the text by solving a logic system, the logic system comprising: (a) a fixed portion including the assertion clause and the domain definition clause; (b) a set of retractable rules from the communicative discourse tree and a variable portion including facts from communicative actions of the communicative discourse tree, the method further comprising: The method of claim 1 , comprising, in response to determining that the evaluated consistency is greater than a threshold, forming a text response from the text and outputting the text response to an external device.

8. 1. A system comprising: a non-transitory computer-readable medium storing computer-executable program instructions; a processing unit communicatively coupled to the non-transitory computer-readable medium for executing the computer-executable program instructions, wherein executing the computer-executable program instructions configures the processing unit to perform operations, the operations including: creating a discourse tree from the text by identifying basic discourse units in the text, the discourse tree including a plurality of nodes, each non-terminal node of the nodes in the discourse tree representing a rhetorical relationship between two basic discourse units, each terminal node of the nodes in the discourse tree being associated with a basic discourse unit, the operation further comprising: identifying a rhetorical relation of type elaboration or joint in the discourse tree, the rhetorical relation relating a first basic discourse unit and a second basic discourse unit, the first basic discourse unit and the second basic discourse unit forming a reference sentence, and the action further comprising: determining a syntactic generalization score for each candidate sentence of the set of candidate sentences, each candidate sentence having a corresponding semantic relation, said determining including: identifying one or more common entities between the candidate sentence and the reference sentence; calculating a syntactic generalization score equal to the number of the identified one or more common entities, the operations further comprising: selecting a candidate sentence having a highest syntactic generalization score from among the syntactic generalization scores; and identifying semantic relationships corresponding to the candidate sentence, the semantic relationships corresponding to words in the candidate sentence and defining roles in the candidate sentence, the operations further comprising: creating an updated discourse tree by replacing the rhetorical relations in the discourse tree with updated rhetorical relations that correspond to the semantic relations.

9. Creating the discourse tree from the text comprises: providing the text to a classification model; and identifying the first basic discourse unit, the second basic discourse unit, and the rhetorical relation using the classification model.

10. The system of claim 8 or 9, wherein the updated rhetorical relationship is one of an end, a means, a cause, or a temporal sequence.

11. The system of any of claims 8 to 10, wherein each of the one or more common entities shares a common portion of speech between the candidate sentence and the reference sentence.

12. The operation further comprises: forming a first syntactic parse tree from each candidate sentence; and forming a second syntactic parse tree from the reference sentence, wherein identifying the one or more common entities between the candidate sentence and the reference sentence comprises, for each common entity, identifying the common entity in the first syntactic parse tree and the second syntactic parse tree.

13. The operation further comprises: forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature; identifying the text as containing an argument by applying a classification model trained to detect arguments to the communicative discourse tree; forming a response from the text and outputting the response to an external device.

14. The operation further comprises: forming a communicative discourse tree from the updated discourse tree by matching each fragment having a verb to a verb signature; identifying that the text contains arguments corresponding to claims by applying a classification model trained to detect arguments to the communicative discourse tree; evaluating the consistency of the argument with respect to itself and with respect to domain-defining clauses associated with the domain of the text by solving a logic system; The logic system comprises: (a) a fixed portion including the assertion clause and the domain definition clause; (b) a variable portion including a set of retractable rules from the communicative discourse tree and facts from the communicative behavior of the communicative discourse tree, the action further comprising: The system of claim 8 , further comprising, in response to determining that the evaluated consistency is greater than a threshold, forming a text response from the text and outputting the text response to an external device.

15. A program for causing a system to execute the method according to any one of claims 1 to 7.

Images