Relying on discourse trees for ontology construction

Abstract

The systems, apparatus, and methods of the present invention include a discourse tree. In one example, the method includes generating a discourse tree. The method includes identifying a central entity from the discourse tree that is associated with an elaboration-type rhetorical relation and corresponds to a topic node that identifies a central entity of the text. The method includes determining a subset of basic discourse units of the discourse tree associated with the central entity. The method includes forming general phrases from the subset of basic discourse units. The method includes forming tuples from the general phrases, where the tuples are ordered sets of words in normal form. The method includes updating an ontology with entities from the identified tuples in response to successfully converting the basic discourse units associated with the identified tuples into a logical representation.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 134,757, filed January 7, 2021, which claims priority to U.S. Patent Application No. 17 / 466,409, filed September 3, 2021, which are incorporated herein by reference in their entireties for all purposes.

[0002] Technical Field FIELD OF THE DISCLOSURE This disclosure relates generally to linguistics. More specifically, this disclosure relates to the use of discourse techniques to form ontologies. [Background technology]

[0003] background Linguistics is the scientific study of language. One aspect of linguistics is the application of computer science to natural human languages, such as English. Due to significant increases in processor speed and memory capacity, the computational application of linguistics is on the rise. For example, computational analysis of linguistic discourse facilitates numerous applications, such as automated agents that can answer user questions. However, such applications are unable to leverage the rich discourse-related information to form ontologies, resulting in inferior performance in answering questions, performing dialogue management, or providing recommendation systems. Summary of the Invention

[0004] A brief overview In general, the systems, apparatus, and methods of the present invention relate to generating or extending ontologies using discourse techniques. As an example, a computer-implemented method determines text in a discourse tree associated with a central entity. The method generalizes the text.

[0005] In one aspect, a method for extending an ontology includes generating a discourse tree from text including fragments, the discourse tree representing rhetorical relationships between the fragments. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two of the fragments, and each terminal node of the discourse tree is associated with one of the fragments. The method further includes identifying a central entity from the discourse tree that is associated with a rhetorical relationship that is not an elaboration or joint type and corresponds to a topic node that identifies a central entity of the text. The method further includes determining from the discourse tree a subset of basic discourse units associated with the central entity. Determining the subset of basic discourse units includes identifying a core basic discourse unit associated with an elaboration type relationship. The method further includes forming general phrases in the text associated with the subset of basic discourse units by identifying one or more elements common to two or more of the basic discourse units in the subset of basic discourse units. The method further includes forming tuples from the general phrases by applying one or more syntactic or semantic templates to the one or more common elements. Each tuple is an ordered set of words in normal form. The method further includes identifying each tuple of the plurality of tuples as having a type including a noun phrase, a verb phrase, an adjective phrase, or a prepositional phrase. The method further includes updating the ontology with entities from the identified tuple in response to successfully transforming a basic discourse unit associated with the identified tuple into a logical expression including a predicate and an argument. The transformation is based on the type of the identified tuple.

[0006] In one aspect, the method further includes locating the entity in the ontology in response to receiving a query from the user device and providing the entity to the user device.

[0007] In one aspect, the method further includes identifying an entity class. Identifying the entity class includes encoding the tuples as a vector representation, providing the vector representation to a machine learning model, and receiving the entity class from the machine learning model.

[0008] In one aspect, providing the entity to the user device includes providing an entity class to the user device.

[0009] In one aspect, identifying the central entity includes locating a root node in a discourse tree. Identifying the central entity includes determining a subset of terminal nodes from the discourse tree that are associated with non-terminal nodes that represent rhetorical relations of the elaboration type and represent core basic discourse units. Identifying the central entity includes calculating, for each node in the subset of terminal nodes, a respective path length from the root node. Identifying the central entity includes identifying, from the subset of terminal nodes, a topic node having a path length that is the minimum length of the paths among the path lengths.

[0010] In one aspect, converting each basic discourse unit associated with one or more of the tuples into a respective logical representation includes identifying a type of the tuple as a noun phrase or a prepositional phrase, extracting one or more of the head nouns or tail nouns as logical predicates, and extracting one or more other words as arguments of the logical predicates.

[0011] In one aspect, converting each basic discourse unit associated with one or more of the tuples into a respective logical representation includes identifying that the type of the tuple is a verb phrase, and extracting the verb of the tuple as a predicate and one or more other words as arguments.

[0012] In one aspect, each tuple includes a predicate, a subject, and an object. In one aspect, the method further includes identifying an entity class of one or more of the tuples corresponding to the general phrase. The entity class represents a category of entities. The updating further includes updating the ontology with the entity class.

[0013] In one aspect, a system includes a non-transitory computer-readable medium having computer-executable program instructions stored thereon and a processing device communicatively coupled to the non-transitory computer-readable medium for executing the computer-executable program instructions. The processing device is configured to perform operations by executing the computer-executable program instructions. The operations include generating a discourse tree from text including fragments that represents rhetorical relationships between the fragments. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two of the fragments, and each terminal node of the discourse tree is associated with one of the fragments. The operations include identifying from the discourse tree a central entity that (i) is associated with an elaboration-type or joint-type rhetorical relationship and (ii) corresponds to a topic node that identifies a central entity of the text. The operations include constructing a communicative discourse tree from the discourse tree by matching each fragment having a verb in the discourse tree with a predetermined verb signature. The operations include identifying from the communicative discourse tree a central entity that is associated with an elaboration-type rhetorical relationship and corresponds to a topic node that identifies a central entity of the text. The operations further include determining, from the communicative discourse tree, a subset of basic discourse units associated with the central entity. Determining the subset of basic discourse units includes identifying a core basic discourse unit associated with an elaboration-type relationship. The operations further include forming a general phrase in text associated with the subset of basic discourse units by identifying one or more elements common to two or more of the basic discourse units in the subset of basic discourse units. The operations further include forming tuples from the one or more common elements by applying one or more syntactic or semantic templates to respective phrases. Each tuple is an ordered set of words in normal form. The operations further include identifying each tuple of the plurality of tuples as having a type including a noun phrase, a verb phrase, an adjective phrase, or a prepositional phrase.The operations further include updating the ontology with entities from the identified tuple in response to successfully transforming the basic discourse units associated with the identified tuple into a logical expression including a predicate and an argument, the transformation being based on a type of the identified tuple.

[0014] The above methods may be implemented as a tangible computer-readable medium and / or operating within a computer processor and associated memory. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 illustrates an exemplary ontology environment according to one aspect. [Figure 2] FIG. 1 illustrates an example of a discourse tree according to one embodiment. [Figure 3] FIG. 10 illustrates yet another example of a discourse tree according to one aspect. [Figure 4] FIG. 1 illustrates an exemplary schema according to an aspect. [Figure 5] FIG. 1 illustrates a node-link representation of a hierarchical binary tree according to one embodiment. [Figure 6] 6 illustrates an exemplary indented text encoding of the representation of FIG. 5, according to one embodiment. [Figure 7] FIG. 1 illustrates an exemplary discourse tree for an example request regarding property taxes, according to one aspect. [Figure 8] FIG. 8 illustrates exemplary responses to the questions shown in FIG. 7. [Figure 9] FIG. 1 illustrates a discourse tree of a formal response, according to one aspect. [Figure 10] FIG. 1 illustrates a discourse tree of raw answers, according to one embodiment. [Figure 11] FIG. 1 illustrates a communicative discourse tree for a first agent's assertion, according to one embodiment. [Figure 12] FIG. 10 illustrates a communicative discourse tree for a second agent's assertion, according to one embodiment. [Figure 13]FIG. 10 illustrates a communicative discourse tree for a third agent's assertion, according to one embodiment. [Figure 14] FIG. 1 illustrates perspective interlacing according to one embodiment. [Figure 15] FIG. 1 illustrates an exemplary process for building a communicative discourse tree, according to one aspect. [Figure 16] FIG. 1 illustrates an example of extracting logical clauses from text according to one aspect of the present disclosure. [Figure 17] FIG. 2 illustrates an example of an entity relationship diagram according to one aspect of the present disclosure. [Figure 18] FIG. 2 illustrates an entity graph and a discourse tree according to one aspect of the present disclosure. [Figure 19] FIG. 10 illustrates an example of an event annotation according to one aspect of the present disclosure. [Figure 20] FIG. 10 illustrates an example of visualization of annotations according to one aspect of the present disclosure. [Figure 21] FIG. 1 illustrates an abstract semantic representation graph and an event taxonomy according to one aspect of the present disclosure. [Figure 22] FIG. 1 illustrates an aggregation of phrases to obtain a hierarchy according to one aspect of the present disclosure. [Figure 23] FIG. 1 illustrates an entity grid matrix according to one aspect of the present disclosure. [Figure 24] 24A and 24B show a syntax tree according to one embodiment of the present disclosure. [Figure 25] 1 is an entity relationship graph according to one aspect of the present disclosure. [Figure 26] 10 is an additional entity relationship graph according to an aspect of the present disclosure. [Figure 27] FIG. 1 illustrates a discourse tree according to one aspect of the present disclosure. [Figure 28] 1 is a flowchart illustrating an exemplary process for enriching an ontology according to one aspect of the present disclosure. [Figure 29] 1 is a simplified diagram illustrating a distributed system for implementing one of the aspects. [Figure 30]FIG. 1 is a simplified block diagram illustrating components of a system environment in which services provided by components of an aspect of the system may be offered as cloud services, according to one aspect. [Figure 31] FIG. 1 illustrates an exemplary computer system in which various aspects of the present invention may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0016] Detailed Description Aspects disclosed herein provide technical improvements in the field of computer-implemented linguistics. More specifically, certain aspects use discourse and other techniques to generate improved ontologies. Ontologies include entities and relationships between pairs of related entities or attributes. Ontologies can be constructed for any knowledge domain, such as law, technology, medicine, etc. Furthermore, ontologies can be used in electronic systems such as decision support systems (DSS) or search tools.

[0017] As an example, in the medical domain, an ontology can map diseases with drug names and treatments. The use of ontologies in medicine is primarily focused on representing medical terminology. For example, medical professionals use ontologies to represent knowledge about disease symptoms and treatments. Pharmaceutical companies use ontologies to represent information about drugs, dosages, and allergies.

[0018] Ontologies are the foundation of many DSSs used to support medical activities, and therefore the quality of the underlying ontologies affects the usage outcomes of DSSs that rely on these ontologies. As a result, automatically constructed medical ontologies (including schema knowledge and individual descriptions) are validated by domain experts. Therefore, traditionally, the construction and adaptation of medical ontologies has relied on close collaboration between domain experts (e.g., medical professionals) and knowledge engineers. While existing automatic ontology construction techniques exist, they are limited to producing partial solutions.

[0019] Thus, technical advantages of some aspects include improved ontologies enabled by the use of discourse techniques that more accurately represent source text. The use of discourse techniques facilitates the selection of more relevant ontology entries from source text compared to previous techniques. Examples of applications that benefit from improved ontologies include search systems, recommendation systems, DSSs, autonomous agents, and diagnostic systems.

[0020] Additionally, some embodiments utilize a Communicative Discourse Tree (CDT), which is a discourse tree that includes communicative acts. By incorporating labels that identify communicative acts, the Communicative Discourse Tree is trained with a richer set of features than simply the syntax of rhetorical relations and elementary discourse units (EDUs). Using such a feature set, additional techniques can be used to develop and / or enhance ontologies, enabling improved automated agents. In doing so, computing systems enable autonomous agents that can intelligently answer questions.

[0021] Specific Definitions As used herein, "rhetorical structure theory" is a field of research and study that has provided a logical foundation that may enable the analysis of discourse coherence.

[0022] As used herein, "discourse tree" or "DT" refers to a structure that expresses rhetorical relations about sentences that are part of a sentence.

[0023] As used herein, a "rhetorical relation," "rhetorical relationship," "coherence relation," or "discourse relation" describes how two segments of discourse are logically connected to each other. Examples of rhetorical relations include elaboration, contrast, and attribution.

[0024] As used herein, a "sentence fragment" or "fragment" is a part of a sentence that can be separated from the rest of the sentence. A fragment is a basic discourse unit. For example, in the sentence "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down the plane," the two fragments are "Dutch accident investigators say that evidence points to pro-Russian rebels" and "as being responsible for shooting down the plane." A fragment can, but does not necessarily, contain a verb.

[0025] As used herein, "signature" or "frame" refers to the properties of verbs in a fragment. Each signature may include one or more thematic roles. For example, in the fragment "Dutch accident investigators say that evidence points to pro-Russian rebels," the verb is "say," and the signature for this particular use of the verb "say" may be an "agent-verb-topic." In this case, "investigators" is the agent and "evidence" is the topic.

[0026] As used herein, "thematic role" refers to a component of a signature that is used to describe the role of one or more words. Continuing with the example above, "agent" and "topic" are thematic roles.

[0027] As used herein, "nuclearity" refers to which text segment, fragment, or span is closer to the center of the writer's purpose. A nucleus is a more central span, and a satellite is a less central one.

[0028] As used herein, "coherency" refers to that which links two rhetorical relationships together.

[0029] As used herein, a "communicative verb" is a verb that indicates communication. For example, the verb "deny" is a communicative verb.

[0030] As used herein, "communicative behavior" describes an action performed by one or more agents and the subject of the agents.

[0031] As used herein, a "claim" is a statement of truth of something. For example, a claim can be, "I'm not responsible for paying rent this month" or "My rent is late."

[0032] As used herein, an "argument" is a reason or set of reasons stated to support a claim. An example of an argument for the above claim is "The necessary repairs were not completed."

[0033] As used herein, "argumentative validity" or "validity" refers to whether the argument supporting a claim is internally consistent. Internal consistency refers to whether the argument is consistent with itself, for example, whether it does not contain two contradictory statements. External consistency refers to whether the argument is consistent with known facts and rules.

[0034] As used herein, a "logic system" or "logic program" is a set of instructions, rules, facts, and other information that can represent an argument for a particular claim. Solving a logic system results in a determination of whether the argument is valid.

[0035] FIG. 1 illustrates an exemplary ontology environment according to one embodiment. FIG. 1 shows a computing device 101, input text 120, and ontology 140. Example computing devices include client computing devices 2902, 2904, 2906, and 2908, as well as client computing devices 3004, 3006, and 3008 shown in FIGS. 29 and 30, respectively. In the illustrated example, computing device 101 accesses input text 120 and uses discourse and other techniques to form and / or update ontology 140. An example of a process for creating ontology entries is process 2800, described with respect to FIG. 28. An ontology contains data or information related to a particular subject area, such as law, engineering, or medicine. An ontology typically contains multiple entries, and each entry may contain logical statements and cross-references to other entries or external sources.

[0036] The computing device 101 includes one or more of an application 122, a discourse parser 104, a machine learning model 124, and training data 125. The application 122 can be configured to perform operations described herein, such as parsing text and applying semantic or syntactic templates to the text. The discourse parser 104 can create a discourse tree and / or a communicative discourse tree. An example of a process for creating a discourse tree is process 1600 described with respect to FIG. 16. The machine learning model 124 can be a classifier, a predictive model, or other type of model. Examples of suitable models include tree kernel models and nearest neighbor models. The machine learning model 124 can be trained using supervised or unsupervised techniques. Training data 125 can be used to train the machine learning model 124. The training data can include positive and negative datasets with associated training labels.

[0037] Rhetorical Structure Theory and Discourse Trees Linguistics is the scientific study of language. For example, linguistics can include the structure of a sentence (syntax), e.g., subject-verb-object, the meaning of a sentence (semantics), e.g., "dog bites man" vs. "man bites dog", and also what speakers do in conversation, i.e., discourse analysis or the analysis of language beyond the sentence.

[0038] The theoretical foundation of discourse (Rhetoric Structure Theory (RST)) can be traced to "Rhetorical structure theory: A Theory of Text organization" by Mann, William, and Thompson, Sandra (Text-Interdisciplinary Journal for the Study of Discourse, 8(3):243-281, 1988). RST enabled discourse analysis in a similar way to how syntax and semantics from programming language theory enabled modern software compilers. More specifically, RST envisions building blocks at at least two levels. The two levels include a first level, such as kernels and rhetorical relations, and a second level, such as structure or schema. A discourse parser or other computer software can parse text into a discourse tree.

[0039] Rhetorical structure theory models the logical organization of a text (the structure used by the writer) by relying on the relationships between parts of the text. RST simulates text coherence by forming a hierarchical, connected structure of the text through a discourse tree. Rhetorical relationships are divided into equivalence classes and subclasses. These relationships maintain across two or more text spans, thereby achieving coherence. These text spans are called elementary discourse units (EDUs). Clauses within a sentence and sentences within a text are logically connected by the author. The meaning of a given sentence is related to the meaning of the previous and following sentences. This logical relationship between clauses is called the coherence structure of the text. RST is one of the most widespread theories of discourse, based on a tree-like discourse structure, the discourse tree (DT). The leaves of the DT correspond to EDUs (successive atomic text spans). Adjacent EDUs are connected by coherence relations (e.g., attribute, sequence) that form higher-level discourse units. These units are further subordinate to this relational link. EDUs linked by a relation are further differentiated based on their relative importance. The core is the core part of the relation, and the satellites are the periphery. As mentioned above, to determine the correct request-response pair, both topic and rhetorical match are analyzed. When a speaker answers a question, such as a phrase or sentence, their response must address the topic of this question. If the question is formulated implicitly, the seed text of the message is used to anticipate an appropriate response that not only stays on topic but also matches the generalized cognitive state about this seed.

[0040] Rhetorical relations As mentioned above, some aspects described herein use communicative discourse trees. Rhetorical relations can be described in a variety of ways. For example, Mann and Thompson describe 23 possible relations in "Rhetorical Structure Theory: A Theory of Text Organization" by C. Mann, William & Thompson, Sandra (1987) ("Mann and Thompson"). Several other relations are also possible.

[0041] [Table 1]

[0042] Some empirical studies assume that the majority of texts are constructed using core-satellite relationships (see Mann and Thompson). However, other relationships do not involve a finite choice of cores. Examples of such relationships are given below.

[0043] [Table 2]

[0044] Figure 2 shows an example of a discourse tree according to one embodiment. Figure 2 includes discourse tree 200. The discourse tree includes text span 201, text span 202, text span 203, relation 210, and relation 228. The numbers in Figure 2 correspond to the three text spans. Figure 3 corresponds to the following text example with three text spans numbered 1, 2, and 3:

[0045] 1. Honolulu, Hawaii will be the site of the 2017 Conference on Hawaiian History.

[0046] 2. It is expected that 200 historians from the US and Asia will attend.

[0047] 3. The conference will be concerned with how the Polynesians sailed to Hawaii.

[0048] For example, relationship 210, or elaboration, describes the relationship between text span 201 and text span 202. Relationship 228 indicates the relationship (elaboration) between text span 203 and text span 204. As shown, text spans 202 and 203 further elaborate on text span 201. In the example above, assuming the purpose is to inform the reader of a conference, text span 1 is the core. Text spans 2 and 3 provide more details about the conference. In Figure 2, horizontal numbers (e.g., 1-3, 1, 2, 3) cover spans of text (possibly composed of further spans), and vertical lines indicate a core or multiple cores. The curved lines represent rhetorical relationships (elaborations), with arrows pointing from the satellites to the core. If only text spans functioned as satellites, rather than as cores, removing the satellites would still leave the text coherent. Removing the core from Figure 2 would make text spans 2 and 3 difficult to understand.

[0049] Figure 3 shows a further example of a discourse tree according to one embodiment. Figure 3 includes constituents 301 and 302, text spans 305-307, relation 310, and relation 328. Relation 310 indicates the relationship (enabling) between constituent 306 and constituent 305, and between constituent 307 and constituent 305. Figure 3 points to the following text spans: 1. The new Tech Report abstracts are now in the journal area of ​​the library near the abridged dictionary.

[0050] 2. Please sign your name by any means that you would be interested in seeing.

[0051] 3. The last day for sign-ups is May 31st. As can be seen, relationship 328 indicates the relationship, or enabling, between entity 307 and entity 306. Figure 3 illustrates that although multiple kernels can be nested, there is only one most kernel text span.

[0052] Discourse tree construction Discourse trees can be generated using a variety of methods. A simple example of a method for building a DT bottom up is as follows: (1) Divide the discourse text into multiple units using (a) and (b) below.

[0053] (a) The unit size may vary depending on the purpose of the analysis. (b) Typically the unit is a clause.

[0054] (2) Examine each unit and each adjacent unit. Is there a relationship between them? (3) If the relationship is maintained, mark the relationship.

[0055] (4) If a relationship does not hold, the unit may be at the boundary of a higher-level relationship. Focus on the relationships that hold between larger units (spans).

[0056] (5) Continue until all units in the text are understood. Mann and Thompson also describe a second level of building block structures called schema applications. In RST, rhetorical relations are not mapped directly onto the text; rather, they are fitted onto structures called schema applications, which are then further fitted to the text. Schema applications are derived from simpler structures called schemas (as shown in Figure 4). Each schema shows how a particular unit of text can be decomposed into smaller textual units. A rhetorical structure tree, or DT, is a hierarchical system of schema applications. Schema applications link several consecutive text spans to create complex text spans. Complex text spans can then be linked by higher-level schema applications. RST asserts that the structure of any coherent discourse can be described by a single rhetorical structure tree, the top-level schema of which creates a span that encompasses the entire discourse.

[0057] FIG. 4 illustrates an exemplary schema according to one embodiment. FIG. 4 illustrates a joint schema as a list of items consisting of a core but no satellites. FIG. 4 illustrates schemas 401-406. Schema 401 illustrates a contextual relationship between text span 410 and text span 428. Schema 402 illustrates a sequence relationship between text span 420 and text span 421, and a sequence relationship between text span 421 and text span 422. Schema 403 illustrates a contrastive relationship between text span 430 and text span 431. Schema 404 illustrates a joint relationship between text span 440 and text span 441. Schema 405 illustrates a motivational relationship between 450 and 451, and an enabling relationship between 452 and 451. Schema 406 illustrates a joint relationship between text span 460 and text span 462. An example of a joint scheme is illustrated in FIG. 4 for the following three text spans: 1. Skies will be partly sunny in the New York metropolitan area today.

[0058] 2. It will be more humid, with temperatures in the middle 80's.

[0059] 3. Tonight will be mostly cloudy, with the low temperature between 65 and 70 degrees Fahrenheit.

[0060] Although Figures 2-4 show discourse trees in several graphs, other representations are possible.

[0061] Figure 5 illustrates a node-link representation of a hierarchical binary tree according to one embodiment. As can be seen from Figure 5, the leaves of the DT correspond to consecutive but non-overlapping spans of text called elementary discourse units (EDUs). Adjacent EDUs are connected by relations (e.g., elaboration, attribute...) to form larger relationally connected discourse units. "Discourse analysis in RST involves two subtasks: discourse segmentation is the task of identifying EDUs, and discourse parsing is the task of linking discourse units into a labeled tree." See Joty, Shafiq R, Giuseppe Carenini, Raymond T Ng, and Yashar Mehdad (2013), "Combining intra- and multi-sentential rhetorical parsing for document-level discourse analysis," ACL (1), pages 486-496.

[0062] Figure 5 shows text spans that are leaves or terminal nodes on a tree, numbered in the order in which they appear throughout the text shown in Figure 6. Figure 5 includes a tree 500. Tree 500 includes, for example, nodes 501-507. The nodes represent relationships. The nodes are either non-terminal nodes, such as node 501, or terminal nodes, such as nodes 502-507. As can be seen, nodes 503 and 504 are related by a joint relationship. Nodes 502, 505, 506, and 508 are nuclei. The dotted lines indicate that the branches or text spans are satellites. These relationships are the nodes in the gray boxes.

[0063] FIG. 6 shows an exemplary indented text encoding for the representation in FIG. 5, according to one embodiment. FIG. 6 includes text 600 and text sequences 602-604. Text 600 is represented in a manner that is more amenable to computer programming. Text sequence 602 corresponds to node 502. Sequence 603 corresponds to node 503. Sequence 604 corresponds to node 504. In FIG. 6, "N" indicates a nucleus and "S" indicates a satellite.

[0064] Discourse Parser Example Automatic discourse segmentation can be performed in various ways. For example, given a sentence, a segmentation model identifies boundaries of composite basic discourse units by predicting whether a boundary should be inserted before each particular token in the sentence. For example, one framework considers each token in the sentence sequentially and independently. In this framework, the segmentation model scans the sentence token by token and uses binary classification, such as a support vector machine or logistic regression, to predict whether it is appropriate to insert a boundary before the token being examined. In another example, the task is a sequential labeling problem. Once the text is segmented into basic discourse units, a sentence-level discourse analysis can be performed to construct a discourse tree. Machine learning techniques can be used.

[0065] In one aspect of the present invention, two Rhetorical Structure Theory (RST) discourse parsers are used: CoreNLPProcessor, which relies on constituent syntax, and FastNLPProcessor, which uses dependency syntax. See "Two Practical Rhetorical Structure Theory Parsers" by Surdeanu, Mihai & Hicks, Thomas & Antonio Valenzuela-Escarcega, Marco (2015).

[0066] Additionally, the two discourse parsers mentioned above, CoreNLPProcessor and FastNLPProcessor, use natural language processing (NLP) for parsing. For example, Stanford CoreNLP presents multi-word primitives that are parts of speech, whether they are names of companies, people, etc.; standardizes dates, times, and numerical quantities; marks sentence structure in terms of phrases and syntactic dependencies; and indicates which noun phrases refer to the same entity. In practice, RST remains a theory of discourse that may work in many cases, but may not work in other cases. Many variables exist that determine what EDUs are in a coherent text, including, but not limited to, what discourse segmenter is used, what relation inventory is used, what relations are selected for the EDUs, the corpus of documents used for training and testing, and even what parser is used. For example, in the aforementioned paper "Two Practical Rhetorical Structure Theory Parsers" by Surdeanu et al., tests must be run on specific corpora using specialized metrics to determine which parser provides better performance. Thus, unlike computer language parsers, which produce predictable results, discourse parsers (and segmenters) can produce unpredictable results, depending on the training and / or testing text corpora. Discourse trees are thus a mixture of predictable techniques (e.g., compilers) and unpredictable techniques (e.g., chemistry, where experimentation is required to determine which combinations will produce the desired results).

[0067] To objectively judge how good a discourse analysis is, a set of metrics is used, such as the Precision / Recall / F1 metric by Daniel Marcu in "The Theory and Practice of Discourse Parsing and Summarization" (MIT Press) (2000). Precision, or positive predictive value, is the fraction of relevant instances among the instances retrieved, while recall (also known as sensitivity) is the fraction of relevant instances retrieved over the total amount of relevant instances. Thus, both precision and recall are based on an understanding and measure of relevance. Suppose a computer program for recognizing dogs in photographs identifies eight dogs in a photograph containing twelve dogs and some cats. Of the eight dogs identified, five are indeed dogs (true positives) and the rest are cats (false positives). The program's precision is 5 / 8 and its recall is 5 / 12. If a search engine returns 30 pages, but only 20 of them are relevant, and does not return an additional 40 relevant pages, its precision is 20 / 30 = 2 / 3 and its recall is 20 / 60 = 1 / 3. So, in this case, precision is "how useful the search results are," and recall is "how complete the results are." The F1 score (F-score or F-criterion) is a measure of the accuracy of a test. It takes into account both the precision and recall of a test to calculate the score: F1 = 2 x (precision x recall) / (precision + recall)), which is the harmonic mean of precision and recall. The F1 score reaches its optimum at 1 (perfect precision and recall) and its worst at 0.

[0068] Autonomous Agents or Chatbots A conversation between person A and person B is a form of discourse. For example, applications such as Facebook® Messenger, WhatsApp®, Slack®, and SMS exist, and conversation between A and B may typically be via messages in addition to more traditional email and voice conversations. A chatbot (sometimes referred to as an intelligent bot or virtual assistant) is an “intelligent” machine that, for example, replaces person B and mimics, to varying degrees, the conversation between the two people. One example of the ultimate goal is for person A to be unable to distinguish whether person B is a human or a machine (the Turing test, developed by Alan Turing in 1950). Artificial intelligence and natural language processing, including discourse analysis and machine learning, have made great strides toward the long-term goal of passing the Turing test. Of course, computers are increasingly able to search and process vast repositories of data and perform complex analyses on the data, including predictive analytics, and the long-term goal is to combine human-like chatbots with computers.

[0069] For example, users can interact with an intelligent bot platform through conversational interactions. This interaction, called a conversational user interface (UI), is a dialogue between the end user and the chatbot, just like a conversation between two people. It can be as simple as the end user saying "Hello" to the chatbot, the chatbot responding "Hi," and the chatbot then asking the user what they need. It can also be a transactional interaction with a banking chatbot, such as transferring money from one account to another, or an informational interaction with an HR chatbot, such as checking a vacation balance, or a FAQ question with a retail chatbot, such as how to process a return. Natural language processing (NLP) and machine learning (ML) algorithms, combined with other approaches, can be used to classify end-user intent. A high-level intent is what the end user wants to accomplish (e.g., get a balance and make a purchase). An intent is essentially a mapping of customer input to units of work for the backend to perform. So based on the phrases uttered by the user in the chatbot, these are mapped to specific distinct use cases or units of work for, for example, balance enquiry, money transfer and tracking expenses - all "use cases" that the chatbot should be able to support and solve, which units of work should be triggered from the free text entry that the end user types in natural language.

[0070] The underlying principle of making an AI chatbot respond like a human is that the human brain can formulate and understand requests, and furthermore, can provide superior responses to human requests far better than a machine can. Therefore, if human B is to be imitated, the chatbot's requests / responses must be significantly improved. So the first part of the problem is how the human brain formulates and understands requests. A model is used for imitating. RST and DT allow a formal and repeatable way to do this.

[0071] At a high level, there are typically two types of requests. Specifically, (1) a request to perform some action, and (2) a request for information (e.g., a question). The first type has a response for which a unit of work is created. The second type has a response to the question (i.e., e.g., a good answer). The answer, for example, in some embodiments, may take the form of an AI that constructs an answer from an extensive knowledge base or from matching the best existing answer by searching the Internet or intranet or other publicly or privately available data sources.

[0072] Communicative discourse trees and rhetorical classifiers Aspects of the present disclosure build a communicative discourse tree and use the communicative discourse tree to analyze whether the rhetorical structure of a request or question matches an answer. More specifically, aspects described herein create representations of request-response pairs, learn these representations, and associate pairs with classes of valid or invalid pairs. In this manner, an autonomous agent can receive a question from a user, process the question by, for example, searching multiple answers, determine a best answer from among the multiple answers, and provide the answer to the user.

[0073] More specifically, to represent the linguistic features of text, the aspects described herein use rhetorical relations and speech acts (or communicative actions). Rhetorical relations are relationships between parts of sentences, typically obtained from a discourse tree. Speech acts are obtained as verbs from a verb resource such as VerbNet. By using both rhetorical relations and communicative actions, the aspects described herein can accurately recognize valid request-response pairs. To do so, the aspects correlate the syntactic structure of a question with the syntactic structure of an answer. Using this structure, a better answer can be determined.

[0074] For example, if an autonomous agent receives an indication from a person that the person wishes to sell an item with a particular characteristic, the autonomous agent should provide search results that not only include the characteristic but also indicate an intent to purchase. In this manner, the autonomous agent has determined the user's intent. Similarly, if an autonomous agent receives a request from a person to share information about a particular item, the search results should include an intent to receive a recommendation. If a person asks the autonomous agent for opinions on a topic, the autonomous agent will share its opinion on the subject rather than soliciting another opinion.

[0075] Analyzing Request and Response Pairs 7 illustrates an exemplary discourse tree for the property tax request example, according to one embodiment. Node labels are relationships, and arrowhead lines point to satellites. The kernel is a solid line. FIG. 7 illustrates the following text:

[0076] Request: "My husbands' grandmother him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund. I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?(My husband's grandmother gave him his grandfather's truck. She transferred the title, but my husband couldn't put the truck in his name because he had an outstanding fine on his license. I wanted it in my name, so I paid the property taxes and got insurance on the truck. When it came time to get the title and get the tag, I didn't have the money to do so. Now, depending on the situation, I won't be able to afford to keep the truck. I went to the insurance agent and they refused to refund me. I'm wondering if I can get the property taxes refunded if I can't get the tag on the truck.)

[0077] Response: "The property tax is assessed on property that you own. Just because you chose to not register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax. If you apply late, there will be penalties on top of the normal taxes and fees. You don't need to register it at the same time, but you absolutely need to title it within the period of time stipulated in state property tax is levied on property you own. Just because you choose not to register it doesn't mean you don't own it; therefore, the tax cannot be refunded. Even if you haven't yet title to the vehicle, you still own it within the taxing district boundaries and must pay the tax. Keep in mind that every state has a limited time to transfer title and pay use tax. If you file late, you will likely be assessed penalties in addition to the regular taxes and fees. You don't have to register the vehicle at the same time, but you must make sure you title it within the time frame required by state law.

[0078] As can be seen from Figure 7, by analyzing the above text, the following results are obtained: "My husband's grandmother gave him his grandfather's truck" is elaborated by the expression "She signed the title over but due to my husband", which is elaborated by "having unpaid fines on his license, he was not able to get the truck put in his name", which is elaborated by the expressions "I wanted to put in my name", "and paid the property tax", and "and got insurance for the truck".

[0079] "My husband's grandmother gave him his grandfather's truck. She signed the title over, but due to my husband having unpaid fines on his license, he was unable to get the truck put in his name. I wanted to put in my name, paid the property tax, and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so" contrasts with "Now, due to circumstances," which is detailed by "I am not going to be able to afford the truck," which is detailed by "I went to the insurance place" and "and was refused a refund."

[0080] "My husbands’ grandmother gave him his grandfather’s truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund." is detailed in "I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?"

[0081] "My husbands' grandmother gave him his grandfather's truck. She signed the title over but due to my husband having unpaid fines on his license, he was not able to get the truck put in his name. I wanted to put in my name and paid the property tax and got insurance for the truck. By the time it came to sending off the title and getting the tag, I didn't have the money to do so. Now, due to circumstances, I am not going to be able to afford the truck. I went to the insurance place and was refused a refund.'' is detailed in ``I am just wondering that since I am not going to have a tag on this truck, is it possible to get the property tax refunded?''.

[0082] "I am just wondering" belongs to the same unit "that" as "is it possible to get the property tax refunded?", which has the condition "since I am not going to have a tag on this truck."

[0083] As can be seen, the main topic theme is "Property Tax on Cars." The question contains a contradiction: on the one hand, all property is taxable; on the other hand, ownership is somewhat incomplete. A good response must address the topic of the question and clarify the contradiction. To do this, the respondent makes a stronger case for the need to pay tax on everything owned, regardless of registration status. This example is an element of a positive training set from the Yahoo! Answers reputation domain. The main topic theme is "Property Tax on Cars." The question contains a contradiction: on the one hand, all property is taxable; on the other hand, ownership is somewhat incomplete. A good answer / response must address the topic of the question and clarify the contradiction. The reader may realize that the question contains a rhetorical relation of contrast, and therefore the answer must match the question in a similar relation to be convincing. In other cases, the answer may appear incomplete even to people who are not experts in the field.

[0084] 8 shows an example response to the question presented in FIG. 7 according to certain embodiments of the present invention. The central core is "The property tax is assessed on property," which is expanded by "that you own." "The property tax is assessed on property that you own" is also a core expanded by "Just because you chose not to register it does not mean that you don't own it, so the tax is not refundable. Even if you have not titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax."

[0085] The core statement, "The property tax is assessed on property that you own. Just because you chose not to register it does not mean that you don't own it, so the tax is not refundable. Even if you haven't titled the vehicle yet, you still own it within the boundaries of the tax district, so the tax is payable. Note that all states give you a limited amount of time to transfer title and pay the use tax." is expanded by the condition, "If you apply late," that "there will be penalties on top of the normal taxes and fees." This is further expanded by the contrast, "but you absolutely need to title it within the period of time stipulated in state law" and "You don't need to register it at the same time."

[0086] By comparing the DT of Figure 7 with the DT of Figure 8, it is possible to determine how well the response (Figure 8) matches the request (Figure 7). In some embodiments of the invention, the above framework is used, at least in part, to determine the DT for request / response and rhetorical fit between the DTs.

[0087] In another example, the question "What does The Investigative Committee of the Russian Federation do?" has at least two answers, for example, the official answer or the actual answer.

[0088] Figure 9 illustrates a discourse tree for an official response, according to one embodiment. As shown in Figure 9, the official response or statement states, "The Investigative Committee of the Russian Federation is the main federal investigating authority which operates as Russia's Anti-corruption agency and has statutory responsibility for inspecting the police forces, combating police corruption and police misconduct, is responsible for conducting investigations into local authorities and federal governmental bodies."

[0089] Figure 10 shows a discourse tree for one embodiment of the raw answers. As shown in Figure 10, an alternative, and potentially more honest, answer is as follows: "The Investigative Committee of the Russian Federation is supposed to fight corruption. However, top-ranking officers of the Investigative Committee of the Russian Federation are charged with the creation of a criminal community. Not only that, but their involvement in large bribes, money laundering, obstruction of justice, abuse of power, extortion, and racketeering has been reported. Due to the activities of these officers, dozens of high-profile cases, including those against criminal lords, had been ultimately ruined." The choice of answer depends on the context. Rhetorical structure allows us to distinguish between "official," "politically correct," template-based answers and "actual," "raw," "reports from the field," or "controversial" answers. See Figures 9 and 10. Sometimes, the question itself can give a hint as to which category of answer is expected. If the question is formulated as a factual or definitional question without a second meaning, an answer in the first category is appropriate. In other cases, if the question has the meaning "tell me what it actually is," the second category is appropriate. In general, after extracting the rhetorical structure from the question, it is easier to select an appropriate answer that will have a similar, consistent, or complementary rhetorical structure.

[0090] The official answer is based on elaborations and joints that are neutral in terms of the controversies the text may contain (see figure). At the same time, the raw answer contains a contrastive relation: this relation between a phrase about what the agent is expected to do and a phrase about what this agent was found to have done is extracted.

[0091] Classification of request-response pairs Application 122 can determine whether a given answer or response, such as an answer from answer database 105 or a public database, is responsive to a given question or request. More specifically, application 122 analyzes whether a request-response pair is accurate or inaccurate by determining one or both of (i) relevance or (ii) rhetorical match between the request and the response. Rhetorical match can be analyzed without taking relevance into account, which can be treated orthogonally.

[0092] The application 122 can determine the similarity between question-answer pairs using a variety of methods. For example, the application 122 can determine the level of similarity between each question and each answer. Alternatively, the application 122 can determine the degree of similarity between a first pair including a question and an answer and a second pair including a question and an answer.

[0093] For example, the application 122 uses a machine learning model 124 that is trained to predict matching or non-matching answers. The application 122 may use two pairs at a time, e.g.<q1,a1> and<q2,a2> , can be processed. The application 122 compares q1 with q2 and a1 with a1 to generate a combined similarity score. Such comparisons allow determining whether an unknown question / answer pair contains the correct answer by assessing its distance from another question / answer pair with a known label. In particular, the unlabeled pair<q2,a2> does not "guess" accuracy based on words or structures shared by q2 and a2, but rather, both q2 and a2 use these words or structures as a basis for determining the accuracy of the labeled pair<q2,a2> Since this method aims to classify answers independently of the domain, it can only exploit the structural consistency between the question and the answer, but cannot exploit the "meaning" of the answer.

[0094] In one aspect, the application 122 uses training data 125 to train the machine learning model 124. In this aspect, the machine learning model 124 is trained to determine the similarity between question and answer pairs, which is a classification problem. The training data 125 may include a positive training set and a negative training set. The training data 125 includes matching request-response pairs in the positive data set and optional or less relevant or appropriate request-response pairs in the negative data set. For the positive data set, different domains are selected with distinct acceptance criteria that indicate whether an answer or response is suitable for the question.

[0095] Each training data set includes a set of training pairs. Each training set includes a question-communicating discourse tree that represents a question and an answer-communicating discourse tree that represents an answer and an expected level of complementarity between the question and the answer. Using an iterative process, application 122 provides the training pairs to machine learning model 124 and receives the level of complementarity from the model. Application 122 calculates a loss function by determining the difference between the determined level of complementarity and the expected level of complementarity for a particular training pair. Based on the loss function, application 122 adjusts the internal parameters of the classification model to minimize the loss function.

[0096] Acceptance standards may vary depending on the application. For example, acceptance standards may be low for community question and answering, automated question and answering, automated customer support systems, manual customer support systems, social network communications, and individual postings, such as consumers' experiences with products, such as surveys and complaints. RR acceptance standards may be high for scientific and technical documents, specialized newspapers and magazines, FAQ-style health and legal documents, and specialized social networks such as "stackoverflow."

[0097] Communicative Discourse Tree (CDT) The application 122 can create, analyze, and compare communicative discourse trees. Communicative discourse trees are designed to combine rhetorical information with speech act structure. CDTs contain arcs labeled with expressions about communicative actions. By combining communicative actions, CDTs enable the modeling of RST relations and communicative actions. CDTs are an epitome of parse chiasm, which combines parse trees for sentences with discourse-level relations between the words and parts of the sentence in a single graph. By incorporating labels that identify speech acts, communicative discourse trees can be trained across a rich set of features, not just the syntactic and rhetorical relations of elementary discourse units (EDUs).

[0098] In one example, a dispute between three parties regarding the causes of the downing of a civilian airliner, Malaysian Airlines Flight 17, is analyzed. An RST representation of the back-and-forth argument is constructed. In this example, three competing agents—Dutch investigators, the Investigative Committee of the Russian Federation, and the self-proclaimed Donetsk People's Republic—exchange their opinions on the issue. This example illustrates a contentious conflict in which each party threatens to accuse the other. To appear more persuasive, each party not only makes its own claims but also formulates responses to reject the other party's claims. To achieve this goal, each party attempts to match the other party's argumentative style and discourse.

[0099] 11 illustrates a communicative discourse tree for a first agent's argument, according to one embodiment. FIG. 11 illustrates a communicative discourse tree 100 representing the following text: "Dutch accident investigators say that evidence points to pro-Russian rebels as being responsible for shooting down the plane. The report indicates where the missile was fired from and identifies who was in control of the territory and pins the downing of MH17 on the pro-Russian rebels." As can be seen in Figure 11, the non-terminal nodes of the CDT are rhetorical relations, and the terminal nodes are the basic discourse units (phrases, sentence fragments) that are the subjects of these relations. Some arcs in the CDT are labeled with expressions about communicative actions, including the agents who are the actors and the subjects of these actions (what is being exchanged). For example, the core node for the elaboration relation (on the left) is labeled say(Dutch, evidence), and the satellite is labeled responsible(rebels, shooting down). These labels are not intended to represent that the subjects of the EDUs are evidence and shooting down, but rather to match this CDT with others in order to find similarities between them. In this case, simply linking these communicative actions by rhetorical relations, rather than providing information about the communicative discourse, is too limited to represent the structure of what is being exchanged and how it is being exchanged. The requirement for RR pairs to have the same or coordinated rhetorical relations is too weak, so matching CDT labels on arcs is necessary in addition to matching nodes.

[0100] The straight edges of this graph are syntactic relations, and the curved arcs are discourse relations such as anaphora, same-entity, subentity, rhetorical relations, and communicative acts. This graph contains much richer information than simply combining parse trees for individual sentences. In addition to CDT, parse chiasm can be generalized at the word, relation, phrase, and sentence levels. Speech acts are logical predicates that represent the agents involved in each speech act and their subject. Arguments for logical predicates are formed according to their semantic roles, as proposed by frameworks such as VerbNet. See "A Large-scale Classification of English Verbs" by Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer (Language Resources and Evaluation Journal, 42(1), 21-40, Springer Netherlands, 2008). and / or "VerbNet overview, extensions, mappings and apps" by Karin Kipper Schuler, Anna Korhonen, and Susan W. Brown, Tutorial, NAACL-HLT: 2009, Boulder, Colorado.

[0101] 12 illustrates a communicative discourse tree for a second agent's assertion, according to one embodiment. FIG. 12 illustrates a communicative discourse tree 1200 representing the following text: "The Investigative Committee of the Russian Federation believes that the plane was hit by a missile, which was not produced in Russia. The committee cites an investigation that established the type of the missile." 13 illustrates a communicative discourse tree for a third agent's claim, according to one embodiment. FIG. 13 illustrates a communicative discourse tree 1300 representing the following text: "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired. It became possible only after three months after the tragedy to say if rebels controlled one or another town." As can be seen from the communicative discourse trees 1100-1300, the responses are not arbitrary. They refer to the same entities as the original text. For example, communicative discourse trees 1200 and 1300 relate to communicative discourse tree 1100. The responses support inconsistencies with inferences and sentiments about these entities and about their behavior.

[0102] More specifically, the replies of participating agents should reflect the communicative discourse of the first seed message. As a simple observation, as the first agent uses attributes to convey its argument, other agents will follow that set and either provide their own attributes or attack the validity of the advocate's attributes, or both. Each CDT pair can be trained to capture a wide variety of features regarding how the communicative structure of the seed message should be preserved in subsequent messages.

[0103] To verify the match of requests and responses, discourse relations or speech acts (communicative actions) alone are often insufficient. As can be seen from the examples shown in Figures 11-13, the discourse structure and dialogue type of the dialogue between agents are useful. However, it is not necessary to analyze the domain of the dialogue (e.g., military conflict or politics) or the subject matter (i.e., entities) of these dialogues.

[0104] Representation of rhetorical relations and communicative actions Two approaches are often used to compute similarity between abstract structures: (1) represent these structures in a numerical space and represent similarity as a number (a statistical learning approach); or (2) use structural representations such as trees and graphs rather than numerical spaces to represent similarity as a maximal common substructure. Representing similarity as a maximal common substructure is called generalization.

[0105] Learning communicative actions aids in the representation and comprehension of arguments. A computational verb lexicon supports the capture of action entities and helps provide a rule-based format for expressing their meaning. Verbs represent the semantics of the described event as well as relational information between participants in that event, projecting a syntactic structure that encodes that information. Verbs, particularly communicative action verbs, are highly variable and can display a rich range of semantic behaviors. Accordingly, verb classification helps learning systems address this complexity by organizing verbs into groups that share core semantic properties.

[0106] VerbNet is one such lexicon that identifies the semantic roles and syntactic pattern features of verbs within each class and articulates the connection between the syntactic patterns and the underlying semantic relationships that can be inferred for all members of the class. See Karin Kipper, Anna Korhonen, Neville Ryant, and Martha Palmer, "Language Resources and Evaluation," Vol. 42, No. 1 (March 2008) 21. Each syntactic frame or verb signature for a class has a corresponding semantic representation that details the semantic relationships between event participants over the course of the event.

[0107] For example, the verb "amuse" is part of a cluster of similar verbs with similar structures of argument (semantic roles), such as amaze, anger, arouse some emotion, disturb, and irritate. The argumentative roles of these communicative actions are Experiencer (usually a living entity), Stimulus, and Result. Each verb can have a semantic class that is distinguished by syntactic features of how this verb appears within a sentence or frame. For example, the frame for "amuse" is as follows, using the following main noun phrase (NP), noun (N), communicative action (V), verb phrase (VP), and adverb (ADV):

[0108] NP V NP. Example: "The teacher amused the children." Syntax: Stimulus V Experiencer. Clauses: amuse(Stimulus, E, Emotion, Experiencer), cause(Stimulus, E), emotional_state(Result(E), Emotion, Experiencer).

[0109] NP V ADV-Middle: Example: "Small children amuse quickly." Syntax: Experiencer V ADV. Clause: amuse(Experiencer, Prop):-, property(Experiencer, Prop), adv(Prop).

[0110] NP V NP-PRO-ARB. Example: "The teacher amused." Syntax: Stimulus V. amuse(Stimulus, E, Emotion, Experiencer): cause(Stimulus, E), emotional_state(Result(E), Emotion, Experiencer).

[0111] NPcause V NP. Example: "The teacher's dolls amused the children." Syntax: Stimulus <+genitive>('s) V Experiencer. amuse(Stimulus, E, Emotion, Experiencer): cause(Stimulus, E), emotional_state(during(E), Emotion, Experiencer).

[0112] NP V NP ADJ. Example: "This performance bored me totally." Syntax: Stimulus V Experiencer Result. amuse(Stimulus, E, Emotion, Experiencer). cause(Stimulus, E), emotional_state(result(E), Emotion, Experiencer), Pred(result(E), Experiencer).

[0113] Communicative actions can be characterized into several clusters, such as verbs with predicate complements (appoint, characterize, dub, declare, conjecture, masquerade, orphan, captain, consider, classify); verbs of perception (see, sight, peer); verbs of mental states (amu se (to entertain, admire, marvel, appeal); verbs of desire (want, long); verbs of judgment (judgment); verbs of evaluation (assess, estimate); verbs of search (hunt, search, stalk, investigate, rummage, ferret); social Verbs of speech (correspond, marry, meet, battle); verbs of communication (transfer (message), inquire, interrogate, tell, manner (speaking), talk, chat, say, complain, advise, confess) These include: to do, lecture, overstate, promise; avoidance verbs (avoid); measurement verbs (register, cost, fit, price, bill); and aspect verbs (begin, complete, continue, stop, establish, sustain).

[0114] The embodiments described herein have advantages over statistical learning models. In contrast to statistical solutions, embodiments using classification systems can provide verbs or verb-like structures that are determined to provide features of interest (such as rhetorical matches). For example, statistical machine learning models represent similarities as numbers, which can make interpretation difficult.

[0115] Representing a request-response pair Representing request-response pairs facilitates classification-based operations based on a pair. In one example, a request-response pair can be represented as a parse chiasm. A parse chiasm represents the parse trees for two or more sentences using discourse-level relationships between the words and parts of the sentences in a graph. See Galitsky (2015). Topical similarity between a question and an answer can be represented as a common subgraph of the parse chiasm. The more common graph nodes there are, the higher the similarity.

[0116] Figure 14 illustrates parse interweaving according to one embodiment. Figure 14 illustrates parse interweaving 1400, which includes parse tree 1401 (for a request) and parse tree 1402 (for the corresponding response).

[0117] Parsetree 1401 poses the following question: "I just had a baby and it looks more like the husband I had my baby with. However it does not look like me at all and I am scared that he was cheating on me with another lady and I had her kid. This child is the best thing that has ever happened to me and I cannot imagine giving my baby to the real mom." Response 1402 represents the following response: "Marital therapists advise on dealing with a child being born from an affair as follows. One option is for the husband to avoid contact but just have the basic legal and financial commitments. Another option is to have the wife fully involved and have the baby fully integrated into the family just like a child from a previous marriage."

[0118] Figure 14 shows a greedy approach to representing linguistic information about a paragraph of text. The straight edges of this graph are syntactic relations, and the curved arcs are discourse relations such as anaphora, same-entity, subentity, rhetorical relations, and communicative acts. The solid arcs are for same-entity / subentity / anaphora relations, and the dotted arcs are for rhetorical relations and communicative acts. The oval labels on the straight edges indicate syntactic relations. Lemmas are written in boxes for the nodes, and the lemma forms are written to the right of the nodes.

[0119] The parse chiasm 1400 contains much richer information than simply combining the parse trees for individual sentences. Navigating through this graph along edges for syntactic relations and arcs for discourse relations allows a given parse chiasm to be converted into a semantically equivalent form for matching with other parse chiasms and performing text similarity assessment tasks. As many links as possible are represented to form a complete formal representation of the paragraph. Each discourse arc generates a pair of parse phrases that can be potential matches.

[0120] The topical similarity between the seed (request) and the response is represented as a common subgraph of the parsing intersection, visualized as a connected cloud. The greater the number of common graph nodes, the greater the similarity. For rhetorical matches, the common subgraph need not be large in a given text. However, the rhetorical relations and communicative actions of the seed and response must be interrelated and correspond.

[0121] Generalizations about communication behavior The similarity between two communicative actions A1 and A2 is defined as an abstract verb that possesses features common to A1 and A2. Defining the similarity between two verbs as an abstract verb-like structure supports inductive learning tasks such as evaluating rhetorical agreement. In one example, the similarity between two common verbs, agree and disagree, can be generalized as follows: agree ^ disagree = verb(Interlocutor, Proposed_action, Speaker). In this case, the Interlocutor is the person who proposes the Proposed_action to the Speaker, to whom the Speaker conveys his or her response. Furthermore, the proposed_action is the action the Speaker would perform if accepting or rejecting the request or proposal, and the Speaker is the person to whom a particular action is proposed and who responds to the request or proposal made.

[0122] In yet another example, the similarity between the verbs agree and explain is as follows: agree ^ explain = verb(Interlocutor, *, Speaker). The subject of a communicative act is generalized in the context of the communicative act, but not in other "physical" acts. Thus, the aspect generalizes individual occurrences of the communicative act with the corresponding subject.

[0123] Additionally, a sequence of communicative acts that represents a dialogue can be compared with other such sequences of similar dialogues. In this manner, the dynamic discourse structure of the dialogue as well as the meaning of individual communicative acts (as opposed to its static structure reflected by rhetorical relations) are expressed. Generalization is a complex structural expression that occurs at each level: the lemmas of communicative acts are generalized by lemmas, and their semantic roles are generalized by each other.

[0124] Communicative acts are used by text authors to indicate the structure of a dialogue or conflict. See Searle, JR (1969), "Speech Acts: An Essay in the Philosophy of Language" (Cambridge University Press). The subject is generalized in the context of these acts, not in other "physical" actions. Thus, individual occurrences of communicative acts are generalized in their subjects and in pairs of them as discourse "steps."

[0125] The generalization of communicative actions can also be considered in terms of matching verb frames such as VerbNet. Communicative links reflect discourse structures related by participation (or mention) rather than by a single agent in a text. Links form sequences that connect words about communicative actions (verbs or multiple words that implicitly indicate one's communicative intention).

[0126] A communicative act includes an actor, one or more agents acting on the actor, and a phrase that describes the characteristics of this act. A communicative act can be described as a function of form: verb (agent, subject, cause). In this case, the verb characterizes some type of interaction between the agents involved (e.g., explain, confirm, remind, disagree, deny, etc.). The subject refers to the information conveyed or the object stated. The cause refers to the motivation or explanation for the subject.

[0127] A scenario (a labeled, directed graph) is a subgraph of the parsed intersection G = (V, A), where V = {action1, action2, ...actionn} is a finite set of vertices corresponding to communicative actions, and A is a finite set of labeled arcs (ordered pairs of vertices) classified as follows:

[0128] Each arc actioni, actionj∈Asequence corresponds to two actions vi, agi, si, ci and vj, agj, sj, cj that refer to the same subject (e.g., sj = si or different subjects). Each arc actioni, actionj∈Acause corresponds to an aggression relation between actioni and actionj, indicating that the cause of actioni is in conflict with the subject or cause of actionj.

[0129] The subgraphs of a parse crossover associated with a scenario of interaction between agents have several notable characteristics. For example, (1) all vertices are ordered in time, and thus every vertex (except the initial and terminal vertices) has one incoming arc and one outgoing arc. (2) For an Asequence arc, at most one incoming arc and one outgoing arc are allowed. (3) For an Acause arc, there can be many incoming arcs as well as many outgoing arcs from a given vertex. The vertices involved may be associated with different agents or the same agent (i.e., if this agent contradicts itself). To compute the similarity between a parse crossover and their communication behaviors, the strict correspondences between the induced subgraphs, subgraphs with similar labels of arcs and the same configuration, and vertices are analyzed.

[0130] The following similarities exist by analyzing the arcs of communicative actions in Peircean crossovers: (1) one communicative action with a theme from T1 compared to another communicative action with a theme from T2 (arc of communicative actions is not used), and (2) one pair of communicative actions with a theme from T1 compared to another pair of communicative actions from T2 (arc of communicative actions is used).

[0131] The generalization of two different communicative actions is based on their attributes, see Galitsky et al. (2013). As can be seen from the example illustrated in Figure 14, one communicative action from T1, namely cheating(husband, wife, another lady), can be compared with a second communicative action from T2, namely avoid(husband, contact(husband, another lady). The generalization results in a communicative_action(husband,*), which introduces a constraint on A of the form: if a given agent (= husband) is mentioned as the subject of a CA in Q, then he / she should also be the subject of (possibly another) CA in A. Two communicative actions can always be generalized, but this is not the case with their subjects. If the result of their generalization is empty, then the result of the generalization of communicative actions with these subjects is also empty.

[0132] Generalization of the RST relation Some relations between discourse trees can be generalized, such as arcs that represent the same type of relation (denotation relations such as contrast, thematic relations such as conditional, and multi-nuclear relations such as list). A nucleus or a situation indicated by a nucleus is denoted by "N". A satellite or a situation indicated by a satellite is denoted by "S". "W" denotes the writer. "R" denotes the reader (listener). Situations are proposals, completed or ongoing actions, and communicative actions and states (including beliefs, desires, approve, explain, reconcile, etc.). The generalization of the two RST relations with the above parameters is expressed as follows: rst1(N1,S1,W1,R1) ^ rst2(N2,S2,W2,R2)=(rst1 ^ rst2)(N1 ^ N2,S1 ^ S2,W1 ^ W2,R1 ^ R2). The texts in N1, S1, W1, and R1 are generalized as phrases. For example, rst1 ^ rst2 can be generalized as follows: (1) If relation_type(rst1) != relation_type(rst2), the generalization is empty. (2) Otherwise, the signature of the rhetorical relation is generalized as a sentence. sentence(N1,S1,W1,R1) ^ sentence(N2,S2,W2,R2) See Iruskieta, Mikel, Iria da Cunha, and Maite Taboada, "A qualitative comparative method for rhetorical structures: identifying different discourse structures in multilingual corpora" (Lang Resources & Evaluation. June 2015, Volume 49, Issue 2).

[0133] For example, the meaning of rst-background ^ rst-enablement= (S increases the ability of R to comprehend an element in N) ^ (R comprehending S increases the ability of R to perform the action in N) = increase-VB the-DT ability-NN of-IN R-NN to-IN.

[0134] Also, since the rst-background ^ rst-enablement relations vary, the RST relation part is empty. Then, expressions that are verbal definitions of each RST relation are generalized. For example, for each word, or for a placeholder for a word such as agent, this word (with its POS) is kept if it is the same in each input phrase, and excluded if it is different between these phrases. The resulting expressions can be interpreted as the common meaning between two different formally obtained definitions of the RST relation.

[0135] The two arcs between the question and answer shown in Figure 14 represent generalization instances based on the RST relation "RST-contrast." For example, "I just had a baby" is an RST-contrast with "it does not look like me," which in turn is related to "husband to avoid contact," which is an RST-contrast with "have the basic legal and financial commitments." As can be seen from the above, the answer does not necessarily have to be similar to the verb phrase in the question, but the rhetorical structures of the question and answer are similar. Not all phrases in the answer necessarily match phrases in the question. For example, mismatched phrases have a specific rhetorical relationship with phrases in the answer that are related to phrases in the question.

[0136] Building a discourse tree for communication 15 illustrates an exemplary process for building a communicative discourse tree, according to one aspect. Application 122 may implement process 1500. As discussed above, a communicative discourse tree enables improved search engine results.

[0137] In block 1501, process 1500 includes accessing sentences containing fragments. At least one fragment includes a verb and multiple words. Each word includes the roles of the multiple words within the fragment. Each fragment is a basic discourse unit. For example, application 122 accesses a sentence such as, "Rebels, the self-proclaimed Donetsk People's Republic, deny that they controlled the territory from which the missile was allegedly fired," as described in connection with FIG. 13.

[0138] Continuing with the above example, application 122 determines that the sentence contains several fragments. For example, a first fragment is "Rebels, ..., deny." A second fragment is "that they controlled the territory." A third fragment is "from which the missile was allegedly fired." Each fragment contains a verb (e.g., "deny" for the first fragment and "controlled" for the second fragment). However, a fragment need not contain a verb.

[0139] At block 1502, process 1500 includes generating a discourse tree representing rhetorical relationships between the sentence fragments. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two of the sentence fragments, and each terminal node of the discourse tree is associated with one of the sentence fragments.

[0140] Continuing with the example above, application 122 generates a discourse tree such as that shown in Figure 13. For example, the third fragment, "from which the missile was allegedly fired," recounts "that they controlled the territory." Both the second and third fragments are concerned with attributing what happened (i.e., the attack could not have been insurgent because insurgents did not control the territory).

[0141] At block 1503, process 1500 includes accessing a number of verb signatures. For example, application 122 accesses a list of verbs, such as from VerbNet. Each verb matches or is related to the verb of the fragment. For example, for the first fragment, the verb is "deny." Therefore, application 122 accesses a list of verb signatures related to the verb "deny."

[0142] As described above, each verb signature includes one or more of the verb and thematic roles of the fragment. For example, a signature may include one or more of a noun phrase (NP), a noun (N), a communicative act (V), a verb phrase (VP), or an adverb (ADV). Thematic roles describe the relationship between a verb and associated words. For example, "The teacher amused the children" has a different signature from "small children amuse quickly." For the first fragment, the verb "deny," the application 122 accesses a list of frames or verb signatures for verbs that match "deny." The list is "NP V NP to be NP," "NP V that S," and "NP V NP."

[0143] Each verb signature includes a thematic role. The thematic role refers to the role of the verb in the sentence fragment. The application 122 determines the thematic role for each verb signature. Exemplary thematic roles include "actor," "agent," "asset," "attribute," "beneficiary," "cause," "location destination source," "destination," "source," "location," "experiencer," "extent," "instrument," "material and product," "material," "product," "patient," "predicate," "recipient," "stimulus," "theme," "time," or "topic."

[0144] Process 1500 includes, for each of the verb signatures, determining the number of thematic roles of each signature that match the roles of words in the fragment at block 1504. For the first fragment, application 122 determines that the verb "deny" has only three roles: "agent," "verb," ​​and "theme."

[0145] Process 1500 includes selecting a particular verb signature from the verb signatures based on the particular verb signature with the most matches at block 1505. For example, referring again to Figure 13, in the first fragment, "the rebels deny that they controlled the territory," "deny" matches the verb signature deny "NP V NP," and "control" matches control(rebels, territory). Verb signatures are nested, resulting in the nested signature "deny(rebel, control(rebel, territory))."

[0146] Request-Response Representation Request-response pairs can be analyzed singly or as pairs. In one example, request-response pairs can be spliced ​​together. When splicing together, rhetorical agreement is expected to hold not only between consecutive members, but also between triplet members and 4-tuple members. Discourse trees can be constructed for texts that represent sequences of request-response pairs. For example, in the domain of customer complaints, the request and response are in the same text from the perspective of the complaining customer. A customer complaint text can be divided into request and response text portions, which can then form pairs of positive and negative data sets. In one example, all text about the advocate and all text about the opponent are combined. The first sentence of each of the following paragraphs would form the request portion (which would contain three sentences), and the second sentence of each paragraph would form the response portion (which in this example would contain three sentences).

[0147] Building Ontologies from Text Certain aspects relate to ontology building using discourse trees and / or Communicative Discourse Trees (CDTs). Furthermore, such techniques can improve the extraction of related text and / or entities within text for ontology entries. Improved ontologies improve downstream performance of downstream applications, such as search systems.

[0148] Medical records can be one of the most valuable sources of information and data regarding a patient's care. Such records contain important items such as eligibility criteria, summary diagnoses, and prescribed medications, which are typically recorded in unstructured, free-text format. Extracting medical or clinical information from medical records has become a significant challenge, especially with the introduction of electronic health records. These records are typically stored as text documents and contain valuable unstructured information that is essential for better decision-making regarding patient care. The three main elements that can be extracted from these clinical records are entities, attributes, and association relationships.

[0149] Automatic recognition of medical entities in unstructured text is a key component of biomedical information retrieval systems. Application examples include analysis of unstructured text in electronic medical records and knowledge discovery in biomedical literature. Many medical terms are structured as ontologies, which add relationships between entities and often include multiple synonyms per term.

[0150] The quality and consistency of an ontology automatically extracted from text determines the accuracy of a decision support system (DSS). The bottleneck in building a concise, robust, and complete ontology is the lack of a mechanism for extracting ontology entries from reliable and authoritative parts of a document. To build an ontology, reliable text fragments that represent the central points of the text are used. Furthermore, constructing entries from additional comments, explanations, examples, instances, and other unimportant parts of the text is avoided. The disclosed technology uses discourse analysis (which has proven useful for tasks such as summarization) to select discourse units that generate ontology entries.

[0151] Typically, the information retrieved is a set of entities connected by relationships. Information presented in this form is useful for many applications (biomedical text mining, ontology learning, and question answering). Ontologies structure knowledge as a set of terms with edges between terms labeled as relationships to evoke meaningful information. Ontologies serve as the backbone of the Semantic Web concept, which aims to provide meaningful information on the Web. Ontologies can benefit from information extraction in terms of ontology development or population (also known as ontology learning and population).

[0152] Discourse techniques can be used to develop ontologies or improve existing ones. Discourse generally contains sequences of sentences, but discourse can also be found within a single sentence; for example, a connected sequence of contingencies such as actions, result states, and events. The pattern formed by the sentences of a discourse provides additional information than the sum of the separate parts of the discourse. For example, each sentence in the example "Don't worry about the water source exhausting today. It is already tomorrow in some parts of the Earth." is a simple assertion. The second sentence is connected to the first sentence by the rhetorical relation Reason for not worrying.

[0153] Discourse analysis allows speakers to specify what they are doing by making use of linguistic features: 1) You are talking about something that has been said before in the same discourse; 2) indicates relations that hold between states, events, beliefs, etc. presented in the discourse; or 3) Changing to a new topic or resuming a previous topic of discourse.

[0154] Linguistic features that allow speakers to specify relationships that hold between states, events, beliefs, etc. presented in discourse include subordinating conjunctions such as "until" or "unless" and discourse adverbs such as "as a result," as in "Birds have wings. As a result, they can fly unless they are too heavy and wings cannot support their weight." Linguistic features in discourse also give speakers opportunities (including what are called cue phrases or boundary features) to specify a change to a new topic or a resumption of a previous topic.

[0155] The discourse, 1) Individual sentences communicate more than just themselves through their relationships with one another; or 2) take advantage of special features of language that make discourse more understandable; It can be associated with a sequence of statements.

[0156] Discourse can be structured by topics, each of which consists of a set of entities and a limited range of things that can be said about them. Topic structures are common in expository texts such as those found in school textbooks, encyclopedias, and reference books. Topics are characterized by the problem they address. Each topic contains a set of entities that can (but need not) vary from topic to topic. Here, the entities consist of a glider; then the glider, its pilot, and passenger; then the glider and its means of propulsion; then the glider and its launch mechanism. Aspects of this structure are modeled as entity chains.

[0157] An entity chain contains a sequence of expressions that refer to the same entity. For example, a text about the flu may have several entity chains: flu → virus → vitamins → those people → drink more fluids → high fever. Where one sequence of entity chains ends and another set begins can be used as an indicator that the discourse has moved from one topic-oriented segment to another. This is useful for tuple extraction logic in the process of forming an ontology from text.

[0158] Some discourse units have a higher value for ontology entries than others. For example, "I went to see a pulmonologist, because I suspected tuberculosis, as my mom asked me to do. A pulmonary specialist was consulted due to suspicion of tuberculosis. Consider the sentence "I went to see a doctor."

[0159] 16 illustrates an example of extracting logical clauses from text according to one embodiment of the present disclosure. Figure 16 shows a discourse tree 1610 and an Abstract Meaning Representation (AMR) tree 1620 linked by relations 1630 and 1640.

[0160] Discourse tree 1610 represents the text "I went to see a pulmonologist because I suspected tuberculosis as my mom asked me to do." Discourse tree 1620 includes basic discourse unit 1612 ("I went to see a pulmonologist"), basic discourse unit 1614 ("because I suspected tuberculosis"), and basic discourse unit 1616 ("as my mom asked me to do").

[0161] The first part of the sentence, "I went to see a pulmonologist because I suspected tuberculosis," is useful, but the second part of the sentence, "as my mom asked me to do," is not. Therefore, the first part of the sentence will be used when forming an ontology. The core basic discourse unit can point to the more useful parts of the sentence.

[0162] The AMR tree 1620 includes two parts: part 1620 and part 1622. Part 1620 corresponds to basic discourse units 1612 and 1614, and part 1622 corresponds to basic discourse unit 1616. Part 1624 corresponds to basic discourse unit 1616.

[0163] Logical clauses can be reliable hints for extracting and formalizing assertions from text. Part 1620 is extracted as a candidate for ontology entry. As long as there are explicit logical connections between phrases, these connections should be captured in the ontology. Conversely, unrelated situations or facts such as part 1622 should not be included. Logical clauses such as the following are extracted:

[0164]

number

[0165] Figure 17 illustrates an example of an entity-relationship diagram according to one embodiment of the present disclosure. An entity-relationship diagram illustrates relationships between entities in a text, an ontology, or stored in a database. An entity, in this context, is an object, a component of data. Figure 17 illustrates entity-relationship diagrams 1710, 1720, 1730, and 1740. Figure 17 corresponds to the text, "Oxygen saturation refers to the amount of oxygen that's in your bloodstream. The body requires a specific amount of oxygen in your blood to function properly. The normal range of oxygen saturation for adults is 94 to 99 percent. However, if your oxygen saturation level is below 90 percent, you will likely require supplemental oxygen, which is prescribed by your primary care doctor or pulmonologist."

[0166] As can be seen, entity relationship diagram 1710 corresponds to the text, "Oxygen saturation refers to the amount of oxygen that's in your bloodstream." Entity relationship diagram 1720 corresponds to the text, "The body requires a specific amount of oxygen in your blood to function properly." Entity relationship diagram 1730 corresponds to the text, "The normal range of oxygen saturation for adults is 94 to 99 percent." Entity relationship diagram 1740 corresponds to the text, "However, if your oxygen saturation level is below 90 percent, you will likely require supplemental oxygen, which is prescribed by your primary care doctor or pulmonologist."

[0167] The entity relationship diagram can provide a set of candidate tuples to be extracted from the text. A tuple is an ordered set of words in normal form (a phrase minus linguistic information). Thus, a tuple will be shorter in length than the sentence from which it is formed. For example, referring again to entity relationship diagram 1710, the entities "oxygen saturation" and "amount of oxygen" are related by a subject relationship.

[0168] An example of a discourse tree is shown below: A discourse tree can provide a logical view of which text fragments are authoritative and reliable sources of relationships that should be recorded in an ontology (indicated by underlining) and which fragments are not (indicated by brackets []).

[0169]

number

[0170] Defining phrases present in the discourse tree below elaboration are less important and less informative. For example, from the basic discourse units "Oxygen saturation refers to the amount of oxygen" and "that is in your bloodstream", we can obtain:

[0171]

number

[0172] However, this would give an unreliable synonymy between saturation and amount. The term "amount" is not a central entity. In this particular case, "saturation" is a specific term and is associated with the very broad term "amount." Therefore, "amount" should not form an ontology entry because it is too general and can be associated with any value. According to discourse analysis, this link should not be an ontology entry. For example, definitions of entities or attributes that are assumed or explained outside the text should not be extracted from a given text. If a text is about the extension of a tax return filing, an ontology should capture the association between "tax" and "extension of time," but not the association between "tax" and "return," which could be assumed and possibly extracted from an introductory accounting textbook.

[0173] In contrast, more significant phrases for forming ontology entries occur in EDUs for non-trivial relations other than elaboration and joint:

[0174]

number

[0175] 18 illustrates an entity graph and a discourse tree according to one embodiment of the present disclosure. FIG. 18 illustrates an entity relationship graph 1810 and a discourse tree 1820.

[0176] The entity relationship graph 1810 shows the intra-entity relationships of the phrases "... about how you are taking the medication" and "The application must be downloaded onto your smartphone before you start the medication." More specifically, "you" is related to "are taking" by the relation subject, which is related to "medication" by the relation object.

[0177] Discourse tree 1820 represents the text, "The tablets that contain a small sensor come with a patch that detects a signal from the tablet and a smartphone application to display information about how you are taking the medication. The application must be downloaded onto your smartphone before you start the medication. Apply your patch to the left side of the body above the lower edge of the rib cage only when prompted by the smartphone app instructions."

[0178] The texts identified with dotted lines, e.g., texts 1822, 1824, 1826, and 1828, indicate central phrases for which extracted relations are useful and which express the central topic of this text. In contrast, other texts in discourse tree 120 contain phrases for which entity tuples should not be generated, since these other texts are useful only when attached to the central phrases.

[0179] In the discourse tree, the central phrase tablet-contain-sensor corresponds to the core EDU of the top-level rhetorical relation of elaboration. This phrase talks about the central topic of this text, the tablet, and its predicate and object / attribute "contain a small sensor." Another important phrase associated with the main entity node "The tablets" is "to display information about how you are taking the medication."

[0180] Satellite EDUs contain phrases that cannot be properly interpreted in standalone mode. "Come with a patch that detects a signal" must be interpreted in the context of a tablet. Otherwise, the hypothetical ontology entry detect(patch, signal) is too general and does not necessarily stand on its own. A coherent ontology should not generalize from this expression. While the core EDU is interpretable and can form an ontology entry on its own, satellite EDUs should not form an ontology entry.

[0181] Finally, the following ontology entries can be extracted:

[0182]

number

[0183] Events can be annotated, for example, representations can be annotated to indicate biomedical events, which are defined as changes in the state or properties of a physical entity.

[0184] 19 illustrates an example of event annotation according to one embodiment of the present disclosure. FIG. 19 illustrates an entity graph 1900 annotated with events such as "cause" or "subject." An event is an interaction between entities. In the general domain, an event is formalized by the event calculus as a sequence of states with preconditions and effect conditions. Examples of events include chemical reactions, interactions between proteins, interactions between proteins and DNA, or any other type of interaction between entities.

[0185] An event annotation is a textual association of any number of entities in a particular role (theme, cause, etc.). Annotation tags may overlap with rhetorical relations.

[0186] Entity graph 1900 is an annotated version of the statement, "The binding of I kappa B / MAD-3 to NF-kappa B p65 is sufficient to retarget NF-kappa B p65 from the nucleus to the cytoplasm."

[0187] 20 shows an example of an annotation visualization according to one embodiment of the present disclosure. Figure 20 includes visualizations 2010, 2020, 2030, and 2040, which together show the annotation for the sentence "The binding of I kappa B / MAD-3 to NF-kappa B p65 is sufficient to retarget NF-kappa B p65 from the nucleus to the cytoplasm," also shown in Figure 19.

[0188] Visualizations 2020, 2030, and 2040 display the event annotations added to sentences. The original sentence is displayed inside each of those boxes to indicate the text span that belongs to the corresponding annotation. Biological entities that were previously annotated during term annotation are displayed. For example, "I kappa B / MAD-3" and "NF-kappa B p65" are protein molecules. "Nucleus" and "cytoplasm" are components of cells. Terms are represented as n-tuples of attribute-value pairs, as follows: ·(Id: T36, Class: Protein_molecule, Name: I kappa B / MAD-3) ·(Id: T37, Class: Protein_molecule, Name: NF-kappa B p65) ·(Id: T38, Class: Protein_molecule, Name: NF-kappa B p65) ·(Id: T39, Class: Cell_component, Name: nucleus) ·(Id: T40, Class: Cell_component, Name: cytoplasm) The first event, E5, represents the binding of two entities, T36 (I kappa B / MAD-3) and T37 (NF-kappa B p65). This indicates a reference to the binding event. The subject of an event is an attribute or slot that is filled by the entity or entities whose properties are affected by the event. The second event, E6, represents the localization of the protein, T38. The textual indications "retarget" and "to the cytoplasm" are marked up as key expressions covering the event type and the location associated with the event, respectively. The last event, E7, is a causal relationship between E5 and E6. That is, the binding event (E5) of the two proteins "causes" the localization event (E6) of one of the two proteins. This causal relationship is represented as an event of type Positive_regulation.

[0189] Regulation has a broader definition than regulatory events in the strict biological sense, e.g., catalysis, inhibition, up / down regulation, etc. General causal relationships between events can be coded; the expression "is sufficient to" provides a syntactic cue to the causal relationship.

[0190] 21 illustrates an abstract semantic representation graph 2110 and an event taxonomy 2120 for a visualization 2000, according to one embodiment of the present disclosure.

[0191] The event taxonomy 2120 shows ontology entities as rectangular boxes and event entities as circles. Arrows indicate links between events and subjects. The link between events and causes is between "Regulation" and "Binding." The link between events and locations is between "localization" and "cytoplasm."

[0192] Phrase aggregation takes a list of phrases and merges synonymous and related phrases to form meaningful ontology entries. The aggregator outputs a hierarchy of phrase entities obtained by generalizing phrase instances. Phrase aggregation can include multiple functions, such as phrase filters and phrase groupers. Phrase filters can include sentiment filters, phrase filter types (NP, VP), phrase length filters (2-6 words), noun entity filters (no proper nouns), occurrence POS filters (no CD, no PRP, etc.), frequency analysis filters, forbidden phrase (manually set) filters, and phrase normalization filters. Phrase groupers include a leading noun phrase extractor, a phrase generalizer, a phrase combiner, a phrase aggregator, and a classifier. Phrase aggregation results in general phrases.

[0193] The following phrase filtering rules can be used: 1) Extract only noun phrases, verb phrases, and prepositional phrases; 2) Exclude emotional phrases because they are sometimes used in opinionated contexts; 3) Name entities are excluded because they cannot be generalized across properties, although certain types of proper nouns related to health domain-specific relations such as affect / cure / drug-for / followed-by are included; 4) Numbers and prepositions are excluded; 5) Phrase length is limited; 6) phrases with too high and too low frequency are removed; 7) Avoid phrases beginning with articles if they are short; or 8) Non-word strings are cleaned / normalized.

[0194] Once extracted, phrases are clustered and aggregated to obtain reliable repeated instances. Phrases that appear only once are less reliable and are considered "noise." For example, a hierarchy can be formed from the list of phrases. Insulin-dependent diabetes Adult-onset dependent diabetes mellitus Diabetes mellitus, a condition characterized by a near-total deficiency of insulin Diabetes with almost complete insulin deficiency and a strong hereditary component The extraction of head nouns can occur as follows: if two phrases have the same head noun, the phrases can be combined into one category; if two phrases within a category have other nouns or adjectives in common besides the head noun, a subcategory can be formed from these common nouns; in this regard, an inductive cognitive procedure is employed to find commonalities between data samples, and head nouns such as diabetes are retained.

[0195] 22 illustrates the aggregation of phrases to obtain a hierarchy according to one embodiment of the present disclosure. The aggregation of phrases 2200 illustrates the following classes, subclasses, and sub-subclasses: diabetes Melitas Insulin dependence An entity grid is used to aid in extracting relationships. Coherent texts combine sentences to express meaning as a whole; that is, the interpretation of a sentence usually depends on the meaning of adjacent sentences. A coherence model that can distinguish coherent from incoherent text can use this performance in a wide range of applications, such as text generation, summarization, and coherence scoring. A coherence model can tell which phrases and sentences are good sources for ontology entries and which phrases are not. Coherence is measured in various discourse models, such as Rhetorical Structure Theory (RST). In RST, coherence can be measured as the average confidence score of recognized rhetorical relationships. In other discourse theories, coherence can be measured as entity propagation. If entities suddenly appear and disappear in the text, the coherence score is low.

[0196] Entity grids represent text as a grid that captures how the grammatical roles of different entities change from sentence to sentence. The grid is then converted into a feature vector containing local entity transition probabilities, allowing machine learning models to learn the importance of each entity's occurrence.

[0197] FIG. 23 illustrates an entity grid matrix according to one embodiment of the present disclosure. FIG. 23 shows an entity grid matrix 2310 and the resulting sentence annotations 2320. As can be seen, the entity grid matrix contains a column for each sentence s0-s3, with the entities. It is desirable to extract the most complete tuple of objects connected by relationships. The matrix references four sentences s0, s1, s2, and s3 as follows: Eaton Corp. said it sold its Pacific Sierra Research unit to a company formed by employees of that unit. s1:Terms were not disclosed. s2:Pacific Sierra, based in Los Angeles, has 200 employees and supplies professional services and advanced products to industry. s3:Eaton is an automotive parts, controls and aerospace electronics concern. For each sentence, a given entity is selected with "S" if the entity is the subject, "O" if it is the object, "X" if it is some other kind of entity, or "-" if there is none. Thus, for sentence s0, "company" is marked because it is present in the sentence.

[0198] Annotation 2320 of the sentence includes marking various references to "Eaton" and "Pacific Sierra Research" as organizations, which are types of entities.

[0199] The following tuples are extracted:

[0200]

number

[0201] 24A and 24B show syntax trees according to one embodiment of the present disclosure. 24 includes syntax trees 2410, 2420, 2430, and 2440. The syntax trees correspond to sentences s0, s1, s2, and s3 referenced in FIG. 23.

[0202] Syntax tree 2410 represents the text "Eaton Corp. said it sold its Pacific Sierra Research unit to a company formed by employees of that unit." Syntax tree 2420 represents the text "Terms were not disclosed." Syntax tree 2430 represents the text "Pacific Sierra, based in Los Angeles, has 200 employees and supplies professional services and advanced products to industry." Syntax tree 2440 represents the text "Eaton is an automotive parts, controls, and aerospace electronics concern."

[0203] 25 illustrates an entity relationship graph according to one embodiment of the present disclosure. FIG. 25 illustrates an entity relationship graph 2500 for sentences s0, s1, s2, and s3 referenced in FIGS. 23 and 24.

[0204] 26 illustrates an additional entity relationship graph according to one embodiment of the present disclosure. Figure 26 illustrates entity relationship graph 2600 showing the entities in sentence s3. Compared to entity relationship graph 2500, entity relationship graph 2600 is further annotated with details such as organization type.

[0205] 27 shows a discourse tree according to one embodiment of the present invention. Discourse tree 2700 corresponds to the text described with respect to FIGS.

[0206] 28 is a flowchart illustrating an example process 2800 for enriching an ontology according to one aspect of the present disclosure. The process 2800 may be implemented by the application 122.

[0207] Process 2800 includes generating a discourse tree from a text containing fragments, the discourse tree representing rhetorical relationships between the fragments, at block 2801. The discourse tree includes nodes, each non-terminal node representing a rhetorical relationship between two fragments of the fragments, and each terminal node of the discourse tree is associated with one of the fragments. Process 2800 includes operations at block 2801 substantially similar to blocks 1501 and 1502 of process 1500.

[0208] In some cases, a communicative discourse tree (CDT) is generated in block 2801. In that case, process 2800 includes operations substantially similar to blocks 1501-1505 of process 1500. In some cases, the CDT can inform the operations performed in process 2800 beyond the capabilities of the discourse tree alone. For example, the communicative actions of the CDT generated in block 2801 can form the subject of the corresponding base discourse unit. For example, the subject "she" in the sentence "she told me the sky is blue" can be identified as important by the corresponding communicative action. From this, it can be shown that even the subject of a satellite base discourse unit, traditionally considered to be less informative, is actually useful. In this respect, communicative actions can override the traditional view of extracting text solely from the core EDU.

[0209] Returning to process 2800, process 2800 includes, at block 2802, identifying from the discourse tree a central entity that (i) is associated with an elaboration-type rhetorical relation and (ii) corresponds to a topic node that identifies a central entity of the text. The topic node is identified from the discourse tree (or communicative discourse tree). To identify the topic, application 122 calculates the respective path length from the root node for each of the terminal nodes of the tree. As shown in FIG. 27, the root node is the first "elaboration" node.

[0210] Continuing with the example, application 122 identifies topic nodes from terminal nodes by identifying nodes with a path length that is the minimum length of the paths. Next, application 122 determines topics of the discourse tree from topic nodes by extracting noun phrases from the core basic discourse units associated with the topic nodes. The noun phrases are central entities.

[0211] Referring again to Figure 27, an example of a central entity is "Eaton Corp." This central entity can be found in the basic discourse unit "Eaton Corp. said." As another example, in the text "sky is blue," the central entity is identified as "sky."

[0212] Process 2800 includes determining a subset of basic discourse units associated with the central entity from the discourse tree at block 2803. The discourse tree may have one or more basic discourse units associated with the central entity.

[0213] Determining the association of basic discourse units with a central entity may include text analysis of basic discourse units associated with a core type and associated with a non-trivial rhetorical relationship. An example of a non-trivial relationship is a relationship that is neither an elaboration nor a joint type. For example, application 122 identifies from the discourse tree basic discourse units that are (i) of a core type and (ii) associated with or connected to a rhetorical relationship that is not a predefined relationship, e.g., not an "elaboration" or "joint" type.

[0214] Referring again to Figure 27, basic discourse units of the nuclear type include "It sold its Pacific Sierra...", "Pacific Sierra...", and "Eaton is an automotive part...".

[0215] Process 2800 includes identifying, in text associated with the subset of basic discourse units, one or more elements that are common to two or more of the basic discourse units in the subset of basic discourse units, at block 2804. Identifying the common elements may include generalizations.

[0216] For two words with the same part of speech (POS), their generalization is the same word with the POS. If the parts of speech are different for the lemmas of two words, the POS is the same but the POS remains in the result. If the lemmas are the same but the POS is different, the lemmas remain in the result. Lemmas represent words that do not contain any associated part of speech information.

[0217] To illustrate this concept, consider two example natural language expressions. The meaning of each expression is expressed in a logical formula. Syntheses and anti-syntheses of these expressions are constructed. Some words (entities) are mapped to predicates, some words are mapped to their arguments, and some other words do not occur explicitly in the logical formal expression but show the above instantiation of predicates with arguments.

[0218] Consider the following two sentences: "camera with digital zoom" and "camera with zoom for beginners." The following logical predicates are used to express their meaning: camera(name_of_feature, type_of_users) and zoom(type_of_zoom).

[0219] Note that this is a simplified example and may therefore have a reduced number of arguments compared to more typical examples. Continuing with the example, the above expression can be expressed as: camera(zoom(digital), AnyUser), camera(zoom(AnyZoom), beginner) By notation, variables (non-entified values ​​not specified in the NL expression) begin with a capital letter. Given the pair of expressions above, synthesis computes their most generalized specialization camera(zoom(digital), beginner), and anti-synthesis computes their most specific generalization camera(zoom(AnyZoom), AnyUser).

[0220] At the syntactic level, these expressions undergo two noun phrase generalizations ('^') as {NN-camera, PRP-with, [digital], NN-zoom [for beginners]}. Expressions in square brackets are removed because they occur in one expression but not the other. The result is the syntactic analog of the semantic generalization: obtain{NN-camera, PRP-with, NN-zoom]}.

[0221] The goal of abstract generalization is to find commonalities between parts of text at various semantic levels. Generalization operations are performed at one or more levels. Examples of levels are the paragraph level, sentence level, phrase level, and word level.

[0222] At each level (except the word level), at the individual word level, the result of the generalization of two expressions is a set of expressions, such that for each pair of expressions where one is less general than the other, the latter is eliminated. The generalization of two sets of expressions is a set of the sets that result from the pairwise generalization of these expressions.

[0223] For a pair of words, there is only one generalization: if the words are the same in the same form, the result is the node that has this word in this form. To involve the word2vec model (Mikolov et al., 2015), the following rule is used to calculate the generalization of two different words: if subject1 = subject2, then subject1^subject2 =<subject1, POS(subject1), 1> Otherwise, if they have the same part of speech, then subject1^subject2 =<*,POS(subject1), word2vecDistance(subject1^subject2)>. If the parts of speech are different, the generalization is an empty tuple; it cannot be generalized further.

[0224] For a pair of phrases, the generalization contains all maximal ordered sets of generalization nodes for the words in the phrases such that the word order is preserved. In the following example: "To buy a digital camera today, on Monday." To enter.” "Digital camera was a good buy today, the first Monday of the month." The camera was on sale today, the first Monday of the month.") The generalization is<JJ-digital, NN-camera> ,<NN- today, ADV,Monday>}, where a noun phrase generalization is followed by an adverb phrase generalization. The verb buy is excluded from both generalizations because it occurs in a different order in the above phrases. Buy - digital - camera is not a generalization phrase because buy occurs in a different order than the other generalization nodes.

[0225] "Movie from Spain" and "movie from Italy." In another example, the generalization would be "movie from [COUNTRY]", Everything that is common is kept and everything that is different is removed.

[0226] The rationale for removing common elements is that it is useful to locate elements in multiple sources to increase reliability, so only common elements are kept.

[0227] Process 2800 includes forming tuples from the general phrases by applying one or more syntactic or semantic templates to each phrase at block 2805. Examples of templates include: <drug_entity1> is a generic substitute for<drug entity2> (<drug_entity1> teeth<drug entity2> It is a generic alternative to <entity1>concentration is affected by dissolution of <entity 2>( <entity1>The concentration of<entity 2> is affected by the dissolution of <entity1>is a <class_of_entity>( <entity1>teeth<class_of_entity> ) As mentioned above, a tuple is a canonical form of a phrase with linguistic information removed. The canonical form includes verbs being infinitives and nouns being singular nominatives. For example, consider the text "all skies are blue," "sky is blue," and "sky has a blue color." These phrases are represented by the noun-adjective tuple "sky blue." Note that each tuple has a one-to-one correspondence with a phrase.

[0228] The process 2800 includes, at block 2806, identifying a tuple of the plurality of tuples as having a type that includes a noun phrase, a verb phrase, an adjective phrase, or a prepositional phrase.

[0229] A noun phrase (NP) is a syntactic element (such as a clause) that has a noun function (such as the subject of a verb or the object of a verb or preposition). An example of a noun phrase in the sentence is "I found the owner of the dog," where "the owner of the dog" is a noun phrase. A verb phrase is the part of the sentence "He appears on screen as an actor," which contains both a verb and a direct or indirect object (a subordinate of the verb). An adjective phrase is a group of words beginning with an adjective that describes a noun or pronoun. An example of an adjective phrase is "She is rather fond of skiing." A prepositional phrase is a modifying phrase containing a preposition and its object: "Before going home, go to the store."

[0230] The application 122 can identify whether the text contains a noun phrase, a verb phrase, an adjective phrase, or a prepositional phrase. For example, a syntax tree can be formed from the text. From the syntax tree, which identifies parts of speech, the application 122 can determine the type of phrase. If the tree contains a verb, the phrase is a verb phrase. If the phrase begins with an adjective, the phrase is an adjective phrase. If the phrase begins with a preposition, the phrase is a prepositional phrase. Otherwise, the phrase is a noun phrase.

[0231] In some cases, machine learning techniques can be used to determine whether a given phrase is a noun phrase, a verb phrase, an adjective phrase, or a prepositional phrase. For example, the phrase is provided to a trained machine learning model 124, which outputs a phrase type classification.

[0232] At block 2807, process 2800 includes updating the ontology with entities from the identified tuple in response to successfully converting the basic discourse unit associated with the identified tuple into a logical expression including a predicate and an argument based on the type of the identified tuple.

[0233] As used herein, a logical predicate expresses a property or a relationship. For example, consider the phrase "sky blue," where "sky" is the head noun, so the word "sky" is the predicate. A predicate can be expressed as:

[0234]

number

[0235] The transformation is based on the type of the identified tuple. For example, if the tuple is a noun phrase or a prepositional phrase, application 122 extracts one or more of the head nouns or tail nouns as logical predicates and one or more other words as arguments. If the tuple is a verb phrase, application 122 extracts the verb of the tuple as a predicate and one or more other words as arguments.

[0236] Adjective or prepositional phrases give simpler facts as ontology entries:

[0237]

number

[0238] If the tuple indicates an adjective or prepositional phrase, a search for an inner verb phrase within the adjective or prepositional phrase is performed. If there is a verb phrase embedded in the prepositional or adjective phrase, a tuple is formed from the inner verb phrase. If there is no inner verb phrase, no ontology entry is performed.

[0239] If the phrase cannot be converted to a logical expression, the phrase cannot be used, in which case the process 2800 can be repeated to find other candidate ontology entries.

[0240] Downstream applications include search systems, recommendation systems, decision support systems (DSS), and diagnostic systems. For example, application 122 can receive a query from a user device. Example queries include medical treatment questions. In response to receiving the query, application 122 can locate an entity in the ontology and provide the entity to the user device.

[0241] In one aspect, an entity may have a class. Examples of classes include entity classes such as "lab test," "medicine," and "protein." Identifying the class may include using a "word2vec" approach, such as a machine learning model 124. The machine learning model 124 can be trained to identify the entity class. For example, the application 122 encodes the tuples as vector representations and provides the vector representations to the machine learning model 124. The machine learning model then provides the determined entity class to the application 122. The ontology can be updated with the entity class and / or the entity class can be provided to the user device.

[0242] In one aspect, additional grouping can be performed. For example, tuples of the same kind are grouped together to produce reliable and useful ontology entries and minimize inconsistencies: noun phrases are grouped with noun phrases, verbs with verbs, and propositions with propositions. A subsequent aggregation component generalizes the tuples to avoid entries that are too specific and noisy to be reliably applied with sufficient confidence.

[0243] A dictionary manager with synonym identification helps generalize tuples that have the same meaning but are expressed using different words. Inference is used to cover words and multi-words that are not synonyms but imply each other in the context of other words.

[0244] evaluation Although complex domain-specific medical Q / A datasets are available, such as the MCTest, Biological Process Modeling, BioASQ, and InsuranceQA datasets, their size is limited (500-10K) due to the complexity of the task or the need for expert annotations that cannot be crowdsourced or collected from the web. To track the contribution of each ontology building step, seven datasets with different complexity of questions, texts, and their associations are combined. The characteristics of the Q / A datasets are listed in Table 3.

[0245] [Table 3-1]

[0246] [Table 3-2]

[0247] When ontology entries are arbitrarily extracted from noisy data, some entries will contradict each other. The frequency of contradictions indirectly indicates the error rate of tuple extraction and overall ontology formation. Examples of contradictory entries include the following (the third argument should be distinguished):

[0248]

number

[0249] Ontology entries are extracted from the answer. Each resulting entry then attempts to find other entries in the given ontology that contradict the given entry. If at least one such entry is found, the entry is considered contradictory. The percentage of contradictory entries across the ontology is counted and expressed as a percentage of all ontology entries. As a baseline, we evaluate an ontology in which entries are extracted from all text parts and left as is without any refinements. We then apply various refinements and track whether they affect the consistency of the ontology.

[0250] [Table 4]

[0251] How each ontology improvement is evaluated affects the resulting ontology consistency (Table 4). At each refinement step, the discrepancy value is normalized with respect to the total number of ontology entries to reduce the number of entries by removing entries that are deemed unreliable. Each step has its own mechanism for reducing entries that are expected to be noisy, unreliable, or misleading.

[0252] We see that by adding rules for extracting ontology entries, the resulting ontology becomes cleaner, more robust, and more consistent. By employing all measures to reduce inconsistencies, the inconsistency rate of inconsistent ontology entries is less than 1% in most domains. The domains with the least inconsistencies are MedQuAD and emrQA. The worst performing is Electronic Health Records (bottom row).

[0253] We evaluate search accuracy across a number of health-related datasets where the search is supported by an ontology, with the complexity of the ontology support varying (Table 4). Since a single best answer is obtained for each evaluation dataset, search relevance is measured as F1.

[0254] [Table 5]

[0255] We observe that each enhancement to ontology construction results in a small improvement in search relevance (F1). These improvements in the 2% range can be difficult to distinguish from random deviations. However, the overall improvement from the ontology is significant, at over 10%. Our ablation experiments demonstrate that each step of discourse processing, aggregation, matching, and validation is critical and should not be omitted.

[0256] Although ontology-assisted search cannot be expressed as a machine learning task, we have learned important lessons from an industrial evaluation of learning transfer frameworks in our previous work (Galitsky 2019). Building ontologies through web mining and applying them to specific vertical domains can be viewed as inductive transfer / multitask learning using feature representation and relational knowledge transfer approaches. We evaluated ontologies constructed from diverse sources, such as blogs (Galitsky and Kovalerchuk 2006), forums, chats, opinion data (Galitsky and McKenna 2017), and customer support data, as being suitable for processing user queries in product search and recommendation in vertical domains such as shopping and entertainment on eBay.com and finance. Because ontology learning in this study is performed in vertical domains with limited terminology ambiguity, sufficient search accuracy can be achieved in a fully automated setting.

[0257] Advanced systems supporting clinical decisions are particularly attractive in emergency departments, perhaps because the criticality of the situation demands the fastest and most accurate solution possible. The use of TM plays a key role in the development of intelligent systems supporting decision-making in emergency medicine, and its application is already becoming a reality. (Portela et al. 2014) presents a special system for emergency services that guides medical personnel in the correct decision-making process to establish clinical priorities. This complex process was carried out thanks to TM technology, which extracts relevant data from electronic medical records, clinical tests, or treatment plans (Gupta and Lehal 2009).

[0258] Our evaluation shows that relying on discourse analysis does improve the quality of an ontology in terms of: 1) fewer contradictions; 2) High relevance of search results.

[0259] When ontology entries are extracted from important and useful parts of text, rather than from all text, ontology results become more reliable for search and decision making.

[0260] Example of an Exemplary Computing System 29 is a simplified diagram illustrating a distributed system 2900 for implementing one of the above-described aspects. In the illustrated aspect, the distributed system 2900 includes one or more client computing devices 2902, 2904, 2906, and 2908 configured to execute and operate client applications, such as a web browser, a proprietary client (e.g., Oracle Forms), etc., via one or more networks 2910. A server 2912 may be communicatively coupled to the remote client computing devices 2902, 2904, 2906, and 2908 via the network 2910.

[0261] In various aspects, server 2912 may be adapted to execute one or more services or software applications provided by one or more of the system's components. The services or software applications may include non-virtual and virtual environments. Virtual environments, whether two-dimensional or three-dimensional (3D) representations, page-based logical environments, or the like, may include those used for virtual events, trade shows, simulators, classrooms, purchasing transactions, and business activities. In some aspects, these services may be provided to users of client computing devices 2902, 2904, 2906, and / or 2908 as web-based or cloud services or under a Software as a Service (SaaS) model. Users operating client computing devices 2902, 2904, 2906, and / or 2908 may then utilize one or more client applications to interact with server 2912 and utilize the services provided by these components.

[0262] In the illustrated configuration, software components 2918, 2920, and 2922 of system 2900 are shown implemented on server 2912. In other aspects, one or more of the components of distributed system 2900 and / or the services provided by these components may be implemented by one or more of client computing devices 2902, 2904, 2906, and / or 2908. A user operating a client computing device may then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be understood that a variety of different system configurations are possible that may differ from distributed system 2900. Thus, the illustrated embodiment is an example of a distributed system for implementing one embodiment of the system and is not intended to be limiting.

[0263] Client computing devices 2902, 2904, 2906, and / or 2908 may be handheld mobile devices (e.g., iPhone®, mobile phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display) running software such as Microsoft Windows® Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android®, BlackBerry 10, Palm OS, etc., and may support Internet, email, short message service (SMS), BlackBerry®, or other available communication protocols. Client computing devices may also be general-purpose personal computers, including, by way of example, personal and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. The client computing devices may be workstation computers running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS. Alternatively, or additionally, the client computing devices 2902, 2904, 2906, and 2908 may be thin client computers, Internet-enabled gaming systems (e.g., Microsoft Xbox game consoles with or without Kinect gesture input devices), and / or other electronic devices such as personal messaging devices capable of communicating over the network 2910.

[0264] Although the exemplary distributed system 2900 is shown with four client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, may interact with the server 2912.

[0265] Network 2910 in distributed system 2900 may be any type of network familiar to those skilled in the art capable of supporting data communications using any of a variety of commercially available protocols, including, but not limited to, TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Packet Exchange), AppleTalk, etc. By way of example only, network 2910 may be a local area network (LAN), such as one based on Ethernet, token ring, etc. Network 2910 may also be a wide area network and the Internet. Network 2910 may include a virtual network, including, but not limited to, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., a network operating under the Institute of Electrical and Electronics (IEEE) 802.29 suite of protocols, Bluetooth and / or other wireless protocols), and / or any combination thereof, and / or other networks.

[0266] Servers 2912 may be comprised of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, UNIX servers, mid-range servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or other suitable configurations and / or combinations. Servers 2912 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. One or more flexible pools of logical storage may be virtualized to maintain virtual storage devices for the servers. Virtual networks may be controlled by servers 2912 using software-defined networking. In various aspects, servers 2912 may be adapted to execute one or more services or software applications described in the above disclosure. For example, servers 2912 may correspond to a server for executing the above-described processes according to one embodiment of the present disclosure.

[0267] Server 2912 may run an operating system, including any of those described above, as well as any commercially available server operating system. Server 2912 may also run any of a variety of additional server and / or middle-tier applications, including a hypertext transport protocol (HTTP) server, a file transfer protocol (FTP) server, a common gateway interface (CGI) server, a JAVA server, a database server, etc. Exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM International Business Machines, etc.

[0268] In some implementations, server 2912 may include one or more applications for analyzing and consolidating data feeds and / or event updates received from users of client computing devices 2902, 2904, 2906, and 2908. By way of example, the data feeds and / or event updates may include, but are not limited to, Twitter® feeds, Facebook® updates, or real-time updates received from one or more third-party sources and continuous data streams, including real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 2912 may also include one or more applications for displaying the data feeds and / or real-time events via one or more display devices of client computing devices 2902, 2904, 2906, and 2908.

[0269] Distributed system 2900 may also include one or more databases 2914 and 2916. Databases 2914 and 2916 may reside in a variety of locations. As one example, one or more of databases 2914 and 2916 may reside on non-transitory storage media local to (and / or resident in) server 2912. Alternatively, databases 2914 and 2916 may be remote from server 2912 and communicate with server 2912 via a network-based or dedicated connection. In one set of embodiments, databases 2914 and 2916 may reside on a storage-area network (SAN). Similarly, any necessary files for performing functions attributed to server 2912 may be stored locally on server 2912 and / or remotely, as appropriate. In one set of embodiments, databases 2914 and 2916 may include relational databases, such as those offered by Oracle Corporation, adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0270] 30 is a simplified block diagram of one or more components of a system environment 3000 capable of delivering services provided by one or more components of an embodiment of the system as cloud services, according to one embodiment of the present disclosure. In the embodiment shown, system environment 3000 includes one or more client computing devices 3004, 3006, and 3008 that can be used by users to interact with a cloud infrastructure system 3002 that provides cloud services. The client computing devices can be configured to run client applications, such as a web browser, a proprietary client application (e.g., Oracle Forms), or other applications, that can be used by users of the client computing devices to interact with cloud infrastructure system 3002 to use services provided by cloud infrastructure system 3002.

[0271] It should be understood that the cloud infrastructure system 3002 depicted in the figure may have other components than those shown. Furthermore, the depicted embodiment is only one example of a cloud infrastructure system that may incorporate aspects of the present invention. In some other embodiments, cloud infrastructure system 3002 may have more or fewer components than those depicted in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0272] Client computing devices 3004, 3006 and 3008 may be similar devices to those described above for 2902, 2904, 2906 and 2908.

[0273] Although the exemplary system environment 3000 is shown with three client computing devices, any number of client computing devices may be supported. Other devices, such as devices with sensors, etc., may interact with the cloud infrastructure system 3002.

[0274] Network 3010 may facilitate communication and exchange of data between clients 3004, 3006, and 3008 and cloud infrastructure system 3002. Each network may be any type of network familiar to those skilled in the art capable of supporting data communication using any of a variety of commercially available protocols, including those described above for network 2910.

[0275] Cloud infrastructure system 3002 may comprise one or more computers and / or servers, which may include those described above for server 2912.

[0276] In certain aspects, the services offered by a cloud infrastructure system may include numerous services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, and managed technical support services. The services offered by the cloud infrastructure system are dynamically scalable to meet the needs of its users. A specific instantiation of a service offered by the cloud infrastructure system is referred to herein as a "service instance." In general, any service available to users from a cloud service provider's system over a communications network such as the Internet is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems comprising the cloud service provider's system are distinct from the customer's own on-premises servers and systems. For example, a cloud service provider's system may host an application, and users may order and use the application on demand over a communications network such as the Internet.

[0277] In some examples, services in a computer network cloud infrastructure may include storage, hosted databases, hosted web servers, protected computer network access to software applications, or other services provided to users by a cloud vendor or otherwise known in the art. For example, a service may include password-protected access to remote storage on the cloud over the Internet. As another example, a service may include a web-services-based hosted relational database and scripting language middleware engine for private use by networked developers. As another example, a service may include access to an email software application hosted on a cloud vendor's website.

[0278] In certain embodiments, cloud infrastructure system 3002 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. One example of such a cloud infrastructure system is the Oracle Public Cloud offered by the present assignee.

[0279] Large amounts of data, sometimes referred to as big data, can be hosted and / or manipulated by infrastructure systems at multiple levels and at different scales. Such data can include data sets so large and complex that they can be difficult to process using typical database management tools or traditional data processing applications. For example, terabytes of data may be difficult to store, retrieve, and process using personal computers or their rack-based counterparts. Data of this size can be difficult to work with using modern relational database management systems and desktop statistics and visualization packages. They may require massively parallel processing software running thousands of server computers, beyond the architecture of commonly used software tools, to capture, curate, manage, and process the data within acceptable elapsed times.

[0280] Analysts and researchers can store and process extremely large data sets to visualize, detect trends, and / or interact with large amounts of data. Dozens, hundreds, or even thousands of processors linked in parallel can act on such data, thereby displaying it or simulating forces on it or what it represents. These data sets may involve structured data, such as data organized in databases or according to structured models, and / or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing, etc.). By enhancing the ability of aspects to relatively quickly converge more (or fewer) computing resources on a target, cloud infrastructure systems can be better utilized to perform tasks on massive data sets based on requests from businesses, government agencies, research organizations, private individuals, like-minded groups of individuals or organizations, or other entities.

[0281] In various aspects, cloud infrastructure system 3002 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by cloud infrastructure system 3002. Cloud infrastructure system 3002 may provide cloud services through a variety of deployment models. For example, services may be provided under a public cloud model, in which cloud infrastructure system 3002 is owned by an organization that sells cloud services (e.g., owned by Oracle Corporation) and the services are available to the general public or various industry enterprises. As another example, services may be provided under a private cloud model, in which cloud infrastructure system 3002 is operated solely for a single organization and may provide services to one or more entities within that organization. Cloud services may also be provided under a community cloud model, in which cloud infrastructure system 3002 and the services provided by cloud infrastructure system 3002 are shared by several organizations within an associated community. Cloud services may also be provided under a hybrid cloud model, which is a combination of two or more different models.

[0282] In some aspects, the services provided by cloud infrastructure system 3002 may include one or more services offered under the Software as a Service (SaaS) category, the Platform as a Service (PaaS) category, the Infrastructure as a Service (IaaS) category, or other categories of services, including hybrid services. A customer may order one or more services provided by cloud infrastructure system 3002 via a subscription order. Cloud infrastructure system 3002 then performs processing to provide the services in the customer's subscription order.

[0283] In some embodiments, services provided by the cloud infrastructure system 3002 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, the SaaS platform may provide functionality for building and delivering a suite of on-demand applications on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing the SaaS services. By utilizing the services provided by the SaaS platform, customers can utilize applications that run on the cloud infrastructure system. Customers can obtain application services without the need for customers to purchase separate licenses and support. A variety of different SaaS services may be offered. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility for large organizations.

[0284] In some aspects, platform services may be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (e.g., Oracle) to integrate existing applications on a shared, common architecture and the ability to build new applications that leverage shared services provided by the platform. The PaaS platform may manage and control the underlying software and infrastructure to provide the PaaS services. Customers can obtain PaaS services provided by the cloud infrastructure system without the need for the customers to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0285] By utilizing services provided by a PaaS platform, customers can utilize programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion middleware services), and Java cloud services. In one embodiment, the database cloud services may support a shared service deployment model that enables organizations to pool database resources and provide database-as-a-service to customers in the form of a database cloud. The middleware cloud services may provide customers with a platform for developing and deploying various business applications in the cloud infrastructure system, and the Java cloud services may provide customers with a platform for deploying Java applications in the cloud infrastructure system.

[0286] A variety of different infrastructure services may be provided by the IaaS platform in a cloud infrastructure system. The infrastructure services facilitate the management and control of basic computing resources such as storage, network, and other underlying computing resources for customers who use the services provided by the SaaS and PaaS platforms.

[0287] Also, in certain aspects, cloud infrastructure system 3002 may include infrastructure resources 3030 for providing resources used to provide various services to customers of the cloud infrastructure system. In one aspect, infrastructure resources 3030 may include a pre-integrated, optimized combination of hardware, such as servers, storage, and networking resources, for running the services provided by the PaaS and SaaS platforms.

[0288] In some aspects, resources in cloud infrastructure system 3002 may be shared by multiple users and dynamically reallocated on a demand basis. Resources may also be allocated to users in different time periods. For example, cloud infrastructure system 3002 may maximize resource utilization by allowing a first set of users in a first time period to utilize the resources of the cloud infrastructure system for a specified amount of time and allowing reallocation of the same resources to another set of users located in a different time period.

[0289] In certain aspects, several internal shared services 3032 may be provided that are shared by various components or modules of, and services provided by, cloud infrastructure system 3002. These internal shared services may include, but are not limited to, security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling cloud support, email services, notification services, file transfer services, etc.

[0290] In certain aspects, cloud infrastructure system 3002 may provide comprehensive management of cloud services (e.g., SaaS services, PaaS services, and IaaS services) in the cloud infrastructure system. In one aspect, cloud management functionality may include functionality for provisioning, managing, tracking, etc., customer subscriptions received by cloud infrastructure system 3002.

[0291] In one aspect, as shown in the figure, cloud management functionality may be provided by one or more modules, such as an order management module 3020, an order orchestration module 3022, an order provisioning module 3024, an order management and monitoring module 3026, and an identity management module 3028. These modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or other suitable configurations and / or combinations.

[0292] In example operation 3034, a customer using a client device, such as client device 3004, 3006, or 3008, may interact with cloud infrastructure system 3002 by requesting one or more services offered by cloud infrastructure system 3002 and placing an order for a subscription to one or more services provided by cloud infrastructure system 3002. In particular aspects, the customer may access cloud user interfaces (UIs), i.e., cloud UI 3012, cloud UI 3014, and / or cloud UI 3016, and place a subscription order via these UIs. Order information received by cloud infrastructure system 3002 in response to the customer placing an order may include information identifying the customer and the one or more services offered by cloud infrastructure system 3002 to which the customer intends to subscribe.

[0293] After an order is placed by a customer, the order information is received via the cloud UI 3030, 3014 and / or 3016.

[0294] In operation 3036, the order is stored in order database 3018. Order database 3018 may be one of several databases operated by cloud infrastructure system 3002 and in cooperation with other system elements.

[0295] At operation 3038, the order information is forwarded to the order management module 3020. In some examples, the order management module 3020 may be configured to perform billing and accounting functions related to the order, such as confirming the order and booking the order upon confirmation.

[0296] At operation 3040, information regarding the order is communicated to order orchestration module 3022. Order orchestration module 3022 may utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some examples, order orchestration module 3022 may orchestrate the provisioning of resources to support the subscribed service using the services of order provisioning module 3024.

[0297] In certain aspects, the order orchestration module 3022 enables management of business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. At operation 3042, upon receiving an order for a new subscription, the order orchestration module 3022 sends a request to the order provisioning module 3024 to allocate resources and configure those resources needed to fulfill the subscription order. The order provisioning module 3024 enables allocation of resources for services ordered by a customer. The order provisioning module 3024 provides a level of abstraction between the cloud services provided by the cloud infrastructure system 3002 and the physical implementation layer used to provision resources to provide the requested services. Thus, the order orchestration module 3022 can be decoupled from implementation details such as whether services and resources are actually provisioned on the fly or pre-provisioned and only allocated / assigned when requested.

[0298] In operation 3042 , once the services and resources are provisioned, a notification of the provided services 3044 may be sent to the customer on the client computing devices 3004 , 3006 and / or 3008 by the order provisioning module 3024 of the cloud infrastructure system 3002 .

[0299] At operation 3046, the customer's subscription order may be managed and tracked by the order management and monitoring module 3026. In some examples, the order management and monitoring module 3026 may be configured to collect usage statistics for the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system up time and system down time.

[0300] In particular aspects, cloud infrastructure system 3002 may include identity management module 3028. Identity management module 3028 may be configured to provide identity services, such as access management and authorization services, in cloud infrastructure system 3002. In some aspects, identity management module 3028 may control information about customers who wish to use services offered by cloud infrastructure system 3002. Such information may include information authenticating the identities of such customers and information describing which actions those customers are authorized to perform on various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). Identity management module 3028 may also include management of descriptive information about each customer and how and by whom this descriptive information may be accessed and modified.

[0301] 31 illustrates an exemplary computer system 3100 upon which various aspects of the present invention may be implemented. The computer system 3100 may be used to implement any of the computer systems described above. As shown, the computer system 3100 includes a processing unit 3104 that communicates with several peripheral subsystems via a bus subsystem 3102. These peripheral subsystems may include a processing acceleration unit 3106, an I / O subsystem 3108, a storage subsystem 3118, and a communications subsystem 3124. The storage subsystem 3118 includes a tangible computer-readable storage medium 3122 and a system memory 3110.

[0302] The bus subsystem 3102 provides a mechanism for allowing the various components and subsystems of the computer system 3100 to communicate with each other as intended. While the bus subsystem 3102 is shown schematically as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 3102 may be any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured to the IEEE P3186.1 standard.

[0303] Processing unit 3104, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of computer system 3100. Processing unit 3104 may include one or more processors. These processors may include single-core or multi-core processors. In particular aspects, processing unit 3104 may be implemented as one or more independent processing units 3132 and / or 3134, each with a single-core or multi-core processor included in the processing unit. In other aspects, processing unit 3104 may be implemented as a quad-core processing unit formed by incorporating two dual-core processors on a single chip.

[0304] In various aspects, the processing unit 3104 may execute various programs in response to program code and may maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in the processing unit 3104 and / or the memory subsystem 3118. Through suitable programming, the processing unit 3104 may provide the various functions described above. The computer system 3100 may also additionally include a processing acceleration unit 3106, which may include a digital signal processor (DSP), special purpose processor, etc.

[0305] The I / O subsystem 3108 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens, which may incorporate displays, scroll wheels, click wheels, dials, buttons, switches, keypads, audio input devices, as well as voice command recognition systems, microphones, and other types of input devices. User interface input devices may include motion detection and / or gesture recognizers, such as a Microsoft Kinect® motion sensor, which allows a user to control and interact with an input device, such as a Microsoft Xbox® 360 game controller, through a natural user interface using gestures and spoken commands. User interface input devices may also include eye gesture recognizers, such as a Google Glass® blink detector, which detects eye movements from a user (e.g., "blinking" while taking pictures and / or making menu selections) and translates the eye gestures as input to the input device. The user interface input devices may also include a voice recognition sensing device that allows a user to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0306] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads and graphics tablets, audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser range finders, and gaze detection devices. User interface input devices may also include medical imaging input devices such as computed tomography, magnetic resonance imaging, positional emission tomography, and medical ultrasound equipment. User interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0307] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. The display subsystem may be a flat panel display, such as one using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, a projection device, a touch screen, or the like. In general, use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 3100 to a user or another computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, automobile navigation systems, plotters, voice output devices, and modems.

[0308] Computer system 3100 may include a storage subsystem 3118 that comprises software elements currently shown as residing in system memory 3110. System memory 3110 may store program instructions that are loadable and executable on processing unit 3104, as well as data generated during the execution of these programs. Depending on the configuration and type of computer system 3100, the system memory 3110 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or presently being operated on and executed by the processing unit 3104. In some implementations, the system memory 3110 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 3100, such as during start-up, may typically be stored in ROM. By way of example and not limitation, system memory 3110 also illustrates application programs 3112, program data 3114, and an operating system 3116, which may include client applications, web browsers, mid-tier applications, relational database management systems (RDBMS), and the like.By way of example, operating system 3116 may include various versions of the Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0309] The storage subsystem 3118 may also provide a tangible, computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of certain aspects. Software (programs, code modules, instructions) that, when executed by a processor, provide the functionality described above may be stored in the storage subsystem 3118. These software modules or instructions may be executed by the processing unit 3104. The storage subsystem 3118 may also provide a repository for storing data used in accordance with the present invention.

[0310] The storage subsystem 3118 may also include a computer-readable storage medium reader 3120 that may further be connected to a computer-readable storage medium 3122. Together and optionally in combination with the system memory 3110, the computer-readable storage medium 3122 may comprehensively represent remote, local, fixed, and / or removable storage devices, as well as storage media for temporarily and / or permanently containing, storing, transmitting, and retrieving computer-readable information.

[0311] The computer-readable storage medium 3122 containing the code or portions of code may include any suitable medium known or used in the art, including, but not limited to, storage and communication media, such as volatile and nonvolatile, removable and non-removable media, implemented in any method or technology for storing and / or transmitting information. This may include tangible, transitory computer-readable storage media, such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage, or other tangible computer-readable media. It may also include intangible, transitory computer-readable media, such as a data signal, data transmission, or any other medium usable to transmit the desired information and accessible by computer system 3100.

[0312] By way of example, computer-readable storage medium 3122 may include hard disk drives that read from or write to non-removable, non-volatile magnetic media, magnetic disk drives that read from or write to removable, non-volatile magnetic disks, and optical disk drives that read from or write to removable, non-volatile optical disks such as CD-ROMs, DVDs, and Blu-ray disks or other optical media. Computer-readable storage medium 3122 may include, but is not limited to, Zip drives, flash memory cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital video tapes, etc. The computer-readable storage media 3122 may also include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, SSDs based on volatile memory such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and their associated computer-readable media may provide computer-readable instructions, data structures, program modules, and other data to the computer system 3100.

[0313] The communications subsystem 3124 provides an interface with other computer systems and networks. The communications subsystem 3124 serves as an interface for receiving data from other systems and transmitting data from the computer system 3100 to other systems. For example, the communications subsystem 3124 may enable the computer system 3100 to connect to one or more devices via the Internet. In some aspects, the communications subsystem 3124 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular technologies, advanced data network technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution)), Wi-Fi (IEEE 802.30 family of standards or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some aspects, the communications subsystem 3124 may provide a wired network connection (e.g., Ethernet) in addition to or instead of a wireless interface.

[0314] Also, in some embodiments, the communications subsystem 3124 may receive incoming communications in the form of structured and / or unstructured data feeds 3126, event streams 3128, event updates 3130, etc., on behalf of one or more users who may be using the computer system 3100.

[0315] As an example, the communications subsystem 3124 may be configured to receive data feeds 3126, such as web feeds such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, etc., in real time from users of social media networks and / or other communications services, and / or to receive real-time updates from one or more third-party sources.

[0316] Additionally, the communications subsystem 3124 may be configured to receive data in the form of a continuous data stream. The data may include an event stream 3128 of real-time events and / or event updates 3130 that may be continuous or may be essentially unbounded with no clear ends. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc.

[0317] The communications subsystem 3124 may also be configured to output structured and / or unstructured data feeds 3126, event streams 3128, event updates 3130, etc. to one or more databases that may communicate with one or more streaming data source computers coupled to the computer system 3100.

[0318] The computer system 3100 may be one of a variety of types, including a handheld mobile device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head-mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or other data processing system.

[0319] Due to the ever-changing nature of computers and networks, the description of computer system 3100 shown in the figure is intended only as a specific example. Many other configurations are possible having more or fewer components than the system shown in the figure. For example, customized hardware may be used and / or particular elements may be implemented in hardware, firmware, software (including applets), or a combination. Additionally, connections to other computing devices, such as network input / output devices, may be utilized. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will recognize other means and / or methods for implementing various aspects.

[0320] In the foregoing specification, aspects of the invention have been described with reference to specific embodiments thereof, but those skilled in the art will recognize that the invention is not limited thereto. Various features and aspects of the invention described above may be used individually or together. Moreover, aspects may be utilized in a number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Accordingly, the specification and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. 1. A computer-implemented method for responding to queries from a user device using an ontology, comprising: generating an ontology entry for the ontology for each text corpus of the plurality of text corpora, wherein the generating includes: forming a discourse tree representing rhetorical relations between basic discourse units for a text corpus including basic discourse units, the discourse tree including a plurality of nodes, each non-terminal node representing a rhetorical relation between two of the basic discourse units, and each terminal node of the nodes of the discourse tree being associated with one of the basic discourse units, the method further comprising: identifying from the discourse tree central entities that (i) are associated with rhetorical relations of an elaboration type and (ii) correspond to topic nodes that identify central entities of the text corpus; determining a subset of basic discourse units associated with the central entity by identifying, from the discourse tree, core basic discourse units associated with rhetorical relations that are neither elaboration nor joint type; forming a general phrase by identifying, in text associated with the subset of basic discourse units, one or more elements that are common to two or more of the basic discourse units in the subset of basic discourse units; forming tuples from the general phrases by applying one or more syntactic or semantic templates to each phrase, each tuple being an ordered set of words in normal form, the method further comprising: adding entities from the identified tuples to the ontology in response to converting basic discourse units associated with the formed tuples into logical expressions including predicates and arguments; In response to receiving a query from the user device, retrieving, from the ontology, a first entity and a second entity from the plurality of ontology entries based on the query; forming a first response including the first entity and a second response including the second entity; For the first response, determining a first score representing one or both of (i) relevance or (ii) rhetorical match between the query and the first response by applying a machine learning model to the query and the first response; For the second response, determining a second score representing one or both of (i) relevance or (ii) rhetorical match between the query and the second response by applying the machine learning model to the query and the second response; and in response to identifying the first score being greater than the second score, providing the first response to the user device.

2. 10. The method of claim 1, further comprising identifying each tuple of the plurality of tuples as having a type comprising: (i) a noun phrase; (ii) a verb phrase; (iii) an adjective phrase; or (iv) a prepositional phrase, and wherein the converting is based on the type of the tuple formed.

3. Entity classes, encoding the tuples as vector representations; providing the vector representation to a machine learning model; and receiving the entity class from the machine learning model, and wherein providing the entity to the user device includes providing the entity class to the user device.

4. Identifying the central entity comprises: locating a root node in the discourse tree; determining from the discourse tree a subset of terminal nodes that (i) are associated with non-terminal nodes that represent rhetorical relations of the elaboration type and (ii) represent core basic discourse units; calculating, for each node of the subset of terminal nodes, a respective path length from the root node; and identifying a topic node from the subset of terminal nodes that has a path length that is a minimum length of a path among the path lengths.

5. converting each basic discourse unit associated with one or more of said tuples into a respective logical representation, Identifying the type of the tuple as a noun phrase or a prepositional phrase; extracting one or more head nouns or tail nouns as logical predicates; and extracting one or more other words as arguments of said logical predicate.

6. converting each basic discourse unit associated with one or more of said tuples into a respective logical representation, Identifying that the type of the tuple is a verb phrase; Extracting a verb from the tuple as a logical predicate and extracting one or more other words as arguments of the logical predicate.

7. The method of any one of claims 1 to 6, wherein each tuple includes one or more of a predicate, a subject, and an object.

8. 8. The method of claim 1, further comprising identifying an entity class of one or more of the tuples corresponding to the general phrase, the entity class representing a category of the entity, and wherein adding comprises updating the ontology with the entity class.

9. 1. A system comprising: a non-transitory computer-readable medium having computer-executable program instructions stored thereon; a processing device communicatively coupled to the non-transitory computer-readable medium for executing the computer-executable program instructions, wherein executing the computer-executable program instructions configures the processing device to perform operations, the operations including: generating an ontology entry for the ontology for each text corpus of the plurality of text corpora, said generating including: forming a discourse tree representing rhetorical relations between basic discourse units for a text corpus including basic discourse units, the discourse tree including a plurality of nodes, each non-terminal node representing a rhetorical relation between two of the basic discourse units, each terminal node of the nodes of the discourse tree being associated with one of the basic discourse units, the operation further comprising: identifying from the discourse tree central entities that (i) are associated with rhetorical relations of an elaboration type and (ii) correspond to topic nodes that identify central entities of the text corpus; determining a subset of basic discourse units associated with the central entity by identifying, from the discourse tree, core basic discourse units associated with rhetorical relations that are neither elaboration nor joint type; forming a general phrase by identifying, in text associated with the subset of basic discourse units, one or more elements that are common to two or more of the basic discourse units in the subset of basic discourse units; forming tuples from said general phrases by applying one or more syntactic or semantic templates to each phrase, each tuple being an ordered set of words in normal form, said operation further comprising: adding entities from the identified tuples to the ontology in response to converting basic discourse units associated with the formed tuples into logical expressions including predicates and arguments; In response to receiving a query from the user device, retrieving, from the ontology, a first entity and a second entity from the plurality of ontology entries based on the query; forming a first response including the first entity and a second response including the second entity; For the first response, determining a first score representing one or both of (i) relevance or (ii) rhetorical match between the query and the first response by applying a machine learning model to the query and the first response; For the second response, determining a second score representing one or both of (i) relevance or (ii) rhetorical match between the query and the second response by applying the machine learning model to the query and the second response; and in response to identifying the first score being greater than the second score, providing the first response to the user device.

10. 10. The system of claim 9, wherein the instructions, when executed by the processing device, further cause the processing device to perform operations including identifying each tuple of the plurality of tuples as having a type comprising: (i) a noun phrase; (ii) a verb phrase; (iii) an adjective phrase; or (iv) a prepositional phrase; and wherein the converting is based on the type of the tuple formed.

11. When executed by the processing device, the instructions further cause the processing device to: encoding the tuples as vector representations; providing the vector representation to a machine learning model; and receiving the entity class from the machine learning model, and providing the entity to the user device includes providing the entity class to the user device.

12. Identifying the central entity comprises: locating a root node in the discourse tree; determining from the discourse tree a subset of terminal nodes that (i) are associated with non-terminal nodes that represent rhetorical relations of the elaboration type and (ii) represent core basic discourse units; calculating, for each node of the subset of terminal nodes, a respective path length from the root node; and identifying a topic node from the subset of terminal nodes that has a path length that is a minimum length of a path among the path lengths.

13. converting each basic discourse unit associated with one or more of said tuples into a respective logical representation, Identifying the type of the tuple as a noun phrase or a prepositional phrase; extracting one or more head nouns or tail nouns as logical predicates; and extracting one or more other words as arguments of the logical predicate.

14. converting each basic discourse unit associated with one or more of said tuples into a respective logical representation, Identifying that the type of the tuple is a verb phrase; Extracting a verb from the tuple as a logical predicate and extracting one or more other words as arguments of the logical predicate.

15. A program comprising computer-implementable instructions which, when executed by a processing device, causes said processing device to perform the method of any one of claims 1 to 8.

Images