Computer systems, computer programs, and computer-implemented methods (identification and extraction of causal knowledge)

Abstract

To provide a computer system and method that can identify cause-effect pairs and / or extract causal knowledge from a corpus.SOLUTION: In a platform computing system 100, an artificial intelligence model is leveraged to determine respective probabilities that variant sentences are inferred from candidate text, calculate a statistical measure of the respective probabilities, and assess the calculated statistical measure to ascertain whether the first and second phrases possess a causal relationship or non-causal relationship to one another. A knowledge base including one or more pairs of cause phrase-effect phrase pairs is populated with the first and second phrases possessing the causal relationship.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] This embodiment utilizes an artificial intelligence (AI) model to identify cause-and-effect pairs, extract causal relationship knowledge from a corpus, or both, particularly, No teacher or effectively No teacher This relates to systems, computer program products, and methods for implementing them on computers.

[0002] Developing databases containing high-quality sets of cause-and-effect pairs has many potential benefits and applications. Such sets can be further processed for various purposes, particularly in AI systems, and can be represented as causal knowledge graphs, which can be used as input for decision assistance or predictive analysis. Constructing sets of cause-and-effect pairs requires extracting causal knowledge from natural language descriptions of such knowledge in text documents and corpora. Such extraction presents challenging problems related to its broad application in AI systems. With a teacher A major problem when applying knowledge extraction methods is the need for large, manually annotated corpora. Manual corpus annotation is not suitable for extracting large amounts of general causal knowledge.

[0003] Developing a system, computer program product, and computer implementation method that can output a high-quality set of cause-and-effect pairs would represent a significant advance in technical error. In particular, in exemplary embodiments, this system, computer program product, and computer implementation method would substantially or completely No teacher It can operate on this. [Overview of the project] [Problems that the invention aims to solve]

[0004] These embodiments include systems, computer program products, and methods for using artificial intelligence (AI) models to extract causal knowledge from a corpus, identify cause-and-effect pairs, or both. This summary of the invention is provided to introduce a selection of representative concepts in the simplified forms further described below for carrying out the invention. This summary of the invention is not intended to identify any important or essential features of the subject matter of the claims, nor is it intended to be used in any way that would limit the scope of the subject matter of the claims. [Means for solving the problem]

[0005] In one embodiment, the computer system comprises a processor operably coupled to memory, and an artificial intelligence (AI) platform communicating with the processor and memory. The AI ​​platform includes a sentence assembler, an AI model, and a director. The sentence assembler is configured to access candidate text and candidate pairs of first and second clauses. The Line 1 and Line 2 Candidate vs to multiple cause-and-effect patterns Assignment The system is configured to generate multiple variant sentences containing the first and second phrases. The AI ​​model is configured to determine the probability that each variant sentence is inferred from the candidate text, calculate a statistical scale for each probability, and then assess the calculated statistical scale to determine whether the first and second phrases are causally or non-causally related to each other. The director is configured to input candidate texts containing causally related first and second phrases into the knowledge base.

[0006] In another embodiment, the computer system comprises a processor operablely coupled to memory and an artificial intelligence (AI) platform communicating with the processor and memory. The AI ​​platform includes a natural language processing (NLP) model, an AI model, and a director. The NLP model is configured to generate natural language (NL) questions using first phrases that represent cause or effect. The AI ​​model extracts one or more candidate second phrases from candidate text, and one or more candidate second phrases constitute the NL question. The answer to The system is configured to determine the probability of each phrase and select the candidate second phrase with the highest probability as having a causal relationship with the first phrase. The director is configured to input the first phrase and the selected second phrase into a knowledge base.

[0007] In yet another embodiment, a computer program product is provided. This computer program product includes a computer-readable storage medium embodying program code. The program code accesses candidate text, extracts candidate pairs of a first and second clause from the candidate text, and The Line 1 and Line 2 Candidate vs to multiple cause-and-effect patterns Assignment The processor can then execute the following: generate multiple variant sentences containing the first and second clauses. The program code can further execute the processor to determine the probability that each variant sentence is inferred from the candidate text, calculate a statistical scale for each probability, assess the calculated statistical scale to determine whether the first and second clauses are causally or non-causally related to each other, and input candidate texts containing causally related first and second clauses into the knowledge base.

[0008] In a further embodiment, a computer-implemented method is provided. Access candidate text and candidate pairs of the first and second clauses. The first and second clauses are arranged into multiple cause-and-effect patterns. AssignmentThen, a plurality of modified sentences including the first sentence and the second sentence are generated. An artificial intelligence model is utilized to determine the respective probabilities that the modified sentences are inferred from the candidate text, calculate a statistical measure of each probability, and examine the calculated statistical measure to confirm whether the first sentence and the second sentence have a causal relationship or a non-causal relationship with each other. The candidate text including the first sentence and the second sentence having a causal relationship is input into the knowledge base.

[0009] In still a further aspect, a method implemented on a computer is provided. A natural language processing model is used to generate a natural language (NL) question using a first sentence representing a cause or a result. An artificial intelligence (AI) model is utilized. Utilizing the AI model includes extracting one or more candidate second sentences from the candidate text, and determining the respective probabilities that the one or more candidate second sentences are in the NL question The answer to that is and selecting the candidate second sentence having the highest probability as the one having a causal relationship with the first sentence. The first sentence and the selected second sentence are input into the knowledge base.

[0010] From the following detailed description of the exemplary embodiments, these and other features and advantages will become apparent in conjunction with the accompanying drawings that illustrate various systems, subsystems, devices, apparatuses, models, processes, and methods of further aspects.

Brief Description of the Drawings

[0011] The drawings referred to in this specification form a part of the specification and are incorporated herein by reference. Unless otherwise specified, the features shown in the drawings are intended to illustrate only some embodiments and not all embodiments.

[0012] [Figure 1] A schematic diagram of a computer system that enables identification and verification of causal pairs and creation or enhancement or both of a knowledge base is shown.

[0013] [Figure 2]Figure 1 shows a block diagram illustrating the AI ​​platform tools and their associated application programming interfaces (APIs).

[0014] [Figure 3A] A flowchart is shown to determine whether a candidate phrase pair has a causal or non-causal relationship, according to an exemplary embodiment. [Figure 3B] A flowchart is shown to determine whether a candidate phrase pair has a causal or non-causal relationship, according to an exemplary embodiment.

[0015] [Figure 4A] This flowchart shows how to associate candidate cause or effect phrases with their corresponding effects or causes. [Figure 4B] This flowchart shows how to associate candidate cause or effect phrases with their corresponding effects or causes.

[0016] [Figure 5A] This flowchart shows how to determine whether candidate sentences extracted from a text corpus contain causal relationships. [Figure 5B] This flowchart shows how to determine whether candidate sentences extracted from a text corpus contain causal relationships.

[0017] [Figure 6] Figures 1 through 5B show block diagrams illustrating examples of cloud-based support system computer systems / servers for implementing the systems and processes described above.

[0018] [Figure 7] A block diagram illustrating a cloud computing environment is shown.

[0019] [Figure 8]This block diagram illustrates the set of function abstraction model layers provided by the cloud computing environment. [Modes for carrying out the invention]

[0020] It will be readily apparent that the components of exemplary embodiments, as generally described and illustrated in the figures of this specification, can be arranged and designed in a wide variety of different configurations. Therefore, the following detailed description of the systems, computer program products, and methods described herein, as well as embodiments of other aspects, as presented in this description and accompanying figures, is not intended to limit the scope of embodiments as defined in the claims, but merely represents selected embodiments.

[0021] Throughout this specification, references to “selected embodiments,” “one embodiment,” or “a certain embodiment” mean that the specific features, structures, or characteristics described in relation to these embodiments are included in at least one embodiment. Therefore, phrases such as “in selected embodiments,” “in one embodiment,” or “in a certain embodiment” appearing in various places throughout this specification do not necessarily refer to the same embodiment. It should be understood that these various embodiments may be combined with each other and modified using the embodiments.

[0022] The embodiments illustrated will be best understood by referring to the drawings. Throughout, similar parts are denoted by similar numbers. The following description is intended to be merely illustrative and simply illustrates specific selected embodiments of devices, systems, and processes that correspond to embodiments such as those described in the claims of this specification.

[0023] Referring to Figure 1, a schematic diagram of a platform computing system (100) is depicted. In an exemplary embodiment, the system (100) includes or incorporates an artificial intelligence (AI) platform (150). As shown, a server (110) communicates with multiple computing devices (180), (182), (184), (186), (188), and (190) across a network connection (105). so It will be provided. Server (110) A processing unit (also referred to herein as a processor) (112) It communicates with memory (116) via the bus (114). like The configuration includes a server (110) connected to one or more computing devices (180), (182), (184), (186), (188), and (190) via a network (105) with an AI platform (150) for cognitive computing, including natural language processing (NLP) and machine learning (ML). More specifically, the computing devices (180), (182), (184), (186), (188), and (190) communicate with each other and with other devices or components via one or more wired data communication links or wireless data communication links or both, where each communication link may include one or more such as wires, routers, switches, transmitters, or receivers. In this networked configuration, the server (110) and the network connection (105) enable the detection, recognition, and resolution of communications. Other embodiments of the server (110) may be used in conjunction with components, systems, subsystems, or devices other than those described herein, or combinations thereof.

[0024] This specification distinguishes between causal relationships and non-causal relationships, CauseThe AI ​​platform (150) is shown, comprising tools for inputting relational information into a knowledge base. The tools include, but are not limited to, a sentence assembler (152), an AI model (154) including a machine learning model (MLM) in an exemplary embodiment, a natural language processing (NLP) model (156), and a director (158). Figure 1 shows each of the tools (152), (154), (156), and (158) as part of the AI ​​platform (150), but it should be understood that in some embodiments, one or a combination of the tools (152), (154), (156), and (158) are not necessarily part of the AI ​​platform (150) or operate with AI. In the exemplary embodiment, the sentence assembler (152) is non-AI; that is, the functionality of the sentence assembler (152) is performed without using artificial intelligence. In the exemplary embodiment, the AI ​​platform (150) is, No teacher or effectively No teacher It can operate with this.

[0025] Artificial intelligence (AI) is people Related to Computers and Computer behavior Computer science This relates to the field of AI. AI refers to intelligence where a machine can make informed decisions and maximize its chances of success on a given topic. More specifically, AI can learn from datasets to solve problems and provide relevant recommendations. For example, in the field of AI computer systems, natural language systems (such as IBM Watson® artificial intelligence computer system or other natural language question answering systems) process natural language based on knowledge acquired by the system. To process natural language, the system, De Database or knowledge While training can be done using data derived from a corpus, the resulting outcomes may be inappropriate or inaccurate for various reasons.

[0026] Machine learning (ML), a subset of AI, uses algorithms to learn from data and based on this data prediction of raw AI refers to intelligence where machines can make informed decisions, maximizing the chances of success on a given topic. More specifically, AI can learn from datasets to solve problems and provide relevant recommendations. Cognitive computing combines computer science and cognitive science. Melt It is a combination of cognitive computing, which uses minimal data, visual recognition, and natural language processing. Using a self-learning algorithm It solves problems, and humans Process by Optimize.

[0027] At the core of AI and the associated logical thinking lies the concept of similarity. The process of understanding natural language and objects can be challenging. ,relationship Logical thinking from a specific perspective is necessary. Structures that include static and dynamic structures are necessary. fixed It defines a determined output or action for a given input. More specifically, the determined output or action is based on explicit or inherent relationships within the structure. It relies on an appropriate dataset to construct these structures.

[0028] This specification describes an AI platform (150) configured to receive input (102) from one or more sources. For example, the AI ​​platform (150) may receive input (e.g., candidate pairs, candidate phrases, or candidate text, or a combination thereof) from one or more of several computing devices (180), (182), (184), (186), (188), and (190) across a network (105). Furthermore, as shown herein, the AI ​​platform (150) is operably coupled with a first knowledge base, KnowledgeBase0 (160), and a second knowledge base, KnowledgeBase1 (170), each of which is also referred to herein as a corpus or database. Figure 1 shows two knowledge bases (160) and (170), but it should be understood that variations of system (100) may be employed. For example, KnowledgeBase0 (160) and KnowledgeBase1 (170) may be combined to form a single knowledge base. In another variation, either KnowledgeBase0(160) and / or KnowledgeBase1(170), or both, contain multiple knowledge bases. In yet another variation, a further knowledge base containing other information communicates with the AI ​​platform(150).

[0029] According to an exemplary embodiment, the AI ​​platform (150) is configured to access candidate pairs of first and second clauses (162), candidate clauses (164), or candidate text (166), or combinations thereof, as input data for processing.

[0030] In exemplary embodiments, the first and second (candidate) phrases are noun phrases. In other exemplary embodiments, the cause and result phrases are not limited to noun phrases. figureThis further includes prepositional phrases that describe noun phrases. As a non-restrictive example, consider the candidate text: "There is a great concern about the effect of rising gasoline prices, due to supply shortages and cyberattacks." Try it In this example, The The part of the sentence that indicates a cause-and-effect relationship contains the noun phrase "a great concern," followed by the descriptive prepositional phrase "about the effect of rising gasoline prices." In an exemplary embodiment, the NLP model (156) extracts the noun phrase and the descriptive prepositional phrase as either a cause-and-effect phrase or an effect-and-effect phrase. According to another embodiment, the candidate cause-and-effect pairs, the first and second phrases, are annotated with tags that identify a class of semantic relationship between the first and second phrases. Examples of classes of semantic relationship include temporal, correlational, hypothetical, and so on.

[0031] [Examples of candidate pair embodiments]

[0032] In an exemplary embodiment, the sentence assembler (152) of the AI ​​platform (150) is configured to access one or more candidate pairs (162) and candidate text (166) (typically a candidate sentence) containing these candidate pairs (162) from KnowledgeBase0 (160), directly from KnowledgeBase0 (160), or indirectly through one or more of the computing devices (180), (182), (184), (186), (188), and (190). Each candidate pair (162) includes at least a first and second phrase that represent a candidate cause-and-effect semantic pair formation. In another exemplary embodiment, the NLP model (156) is configured to access and receive the candidate text (166) from KnowledgeBase0 (160) and is configured to parse the candidate text (166) to identify candidate pairs therein.

[0033] The statement assembler (152), shown in Figure 1 as part of the AI ​​platform (150), is configured in alternative exemplary embodiments to access multiple cause-and-effect patterns (168) that are neither AI-based nor part of the AI ​​platform (150). In exemplary embodiments, the cause-and-effect patterns (168) are syntactically distinct from one another. Examples of cause-and-effect patterns (168) include: "X causes Y", "X is the reason for Y", "Because of X, Y", "X leads to Y", "if X, then Y", "the effect of X is Y", "Y is a result of X", etc. In exemplary embodiments, the statement assembler (152) accesses each of the first and second clauses of a candidate pair (162) to multiple different cause-and-effect patterns (168). Insertion Alternatively, it is configured to generate multiple variant statements by substitution.

[0034] In one embodiment, an AI model (e.g., MLM) (154) is configured to receive a modified sentence from a sentence assembler (152). In an exemplary embodiment, the AI ​​model (154) is an inference model, such as a natural language inference (NLI) model. For each modified sentence, the AI ​​model (154) is configured to determine the probability that each modified sentence is inferred from a candidate text, e.g., a candidate text. According to an exemplary embodiment, this probability may be expressed as a score, such as a confidence score between 0 and 1. For example, for the modified sentence "A summer of storms across the southeast caused an increase in property insurance claims," ​​the AI ​​model (154) assesses the probability that this modified sentence is inferred from the candidate text.

[0035] According to one embodiment, the AI ​​model (e.g., MLM) is, this Designed for the task Ta It is trained using training data. For example, in one embodiment, a natural language inference (NLI) model is trained using training data curated by a human (e.g., a subject expert), which includes pairs of sentences marked with labels such as “implication,” “contradiction,” or “neutral.” As a non-limiting example of a training instance, the first sentence provided is “A significant drop in price was observed after the statement was issued.” in A common pattern is for "statement" to be the first line and "significant price drop" to be the second line. original Using one of the cause-and-effect patterns "The statement caused a major decrease in prices," etc. Construct a second sentence and evaluate whether the input sentence is implicative, contradictory, or neutral to the second sentence.

[0036] In an exemplary embodiment, the AI ​​model (154) calculates the probability of each of the transformed sentences. Regarding It is further configured to calculate statistical measures such as the mean or median. total The calculated statistical scale Whether the first and second clauses of the candidate pair have a causal or non-causal relationship. An assessment is performed to determine whether the condition is met. In one embodiment, this assessment includes determining whether the calculated statistical measure meets a predetermined threshold, such as a statistical measure of 0.5 or greater.

[0037] In an exemplary embodiment, the director (158) communicates with KnowledgeBase1 (170), and KnowledgeBase1 (170) in Figure 1 contains multiple verified cause-and-effect pairs, for example, Cause-Effect Pair0 (1720), Cause-Effect Pair1 (1721), and Cause-Effect Pair N-1 (172 N-1 The data is described with pre-filled values, where N can be any integer. In an exemplary embodiment, the director (158) is configured to further input the candidate text and candidate pair into KnowledgeBase1(170) as verified cause-and-effect pairs if it is verified that the candidate pair has a causal relationship. On the other hand, the director (158) is configured not to further input the candidate pair into KnowledgeBase1(170) if it is assessed that the candidate pair does not have a causal relationship.

[0038] Validated candidate pairs have various uses and applications. In an exemplary embodiment, candidate pairs validated to have a causal relationship may be used to train an AI model. In another embodiment, candidate pairs validated to have a causal relationship may be used to predict (or forecast) future events. For example, suppose there is a "protest" taking place tomorrow, and the AI ​​model can determine (or not determine) a causal relationship between the "protest" and "violence". Derivation If so, the verified candidate pairs may be used to generate warnings, if possible.

[0039] [Examples of candidate phrases]

[0040] In an exemplary embodiment, the NLP model (156) is configured to access one or more first clauses (164) from KnowledgeBase0(160), directly from KnowledgeBase0(160), or indirectly through one or more of the computing devices (180), (182), (184), (186), (188), and (190). In an exemplary embodiment, each of the accessed first clauses (164) represents candidate text, e.g., a cause clause or result clause of a candidate sentence. In another exemplary embodiment, the NLP model (156) is configured to access candidate text (166) from KnowledgeBase0(160) and to parse candidate text (166) to extract first clauses therefrom. In an exemplary embodiment, the NLP model (156) uses the accessed first clauses (164) to derive or The first phrase accessed Generates natural language (NL) questions that include [the specified element].

[0041] In an exemplary embodiment, the AI ​​model (154) includes a question-and-answer (QA) model. The QA model may be pre-trained, for example, using question-and-answer pairs reviewed by an SME. The AI ​​model (154) (e.g., the QA model) is configured to extract one or more candidate second phrases from candidate text. According to one embodiment, the accessed first phrase (164) represents a cause and the candidate second phrases represent one or more consequences. According to another embodiment, the accessed first phrase (164) represents a consequence and the candidate second phrases represent one or more causes.

[0042] The AI ​​model (154) considers each candidate second phrase to be relevant to the NL question. Opposite Correct That is the answer.It is configured to determine the probability of each of whether or not. According to an exemplary embodiment, this probability may be expressed as a score, such as a confidence score between 0 and 1. In one embodiment, each score represents a measure of the likelihood that the accessed first clause and the candidate second clause have a cause-and-effect relationship with each other. Based on this probability or score, at least one correspondence Attached A candidate pair is selected. According to an exemplary embodiment, the probability (e.g., score) of the selected associated candidate pair is such that the accessed first clause and the candidate second clause have a cause-and-effect relationship. Most High possibility This reflects the situation. For example, if the scores are between 0 and 1, with 0 being the lowest score and 1 being the highest score, the pair with the highest score (i.e., closest to 1) will be selected.

[0043] In an exemplary embodiment, the director (158) communicates with KnowledgeBase1 (170), and KnowledgeBase1 (170) in Figure 1 contains multiple verified cause-and-effect pairs, for example, Cause-Effect Pair0 (1720), Cause-Effect Pair1 (1721), and Cause-Effect Pair N-1 (172 N-1 The data is depicted with the values ​​pre-filled, and N can be any integer. In an exemplary embodiment, the director (158) is the corresponding with the highest score. Attached Candidate pairs (or multiple pairs with the highest scores) Attached The system is configured to further input candidate pairs into KnowledgeBase1(170). On the other hand, the director (158) is configured not to further input the first and second clauses, which have a non-causal relationship, into KnowledgeBase1(170).

[0044] The director (158) is a pair of one or more cause-and-effect clauses, for example, Cause-Effect Pair0 (1720), Cause-Effect Pair1 (1721), and Cause-Effect Pair in Figure 1. N(172 N It communicates with KnowledgeBase1(170), including the one with the highest score. Director(158) is the one with the highest score. Attached By inputting candidate pairs into KnowledgeBase1(170), the number of cause-and-effect pairs to be input into KnowledgeBase1(170) is gradually It is designed to increase the target.

[0045] Validated pairs have a variety of uses and applications. In an exemplary embodiment, candidate pairs validated for causality may be used to train an AI model. In another embodiment, candidate pairs validated for causality may be used to predict (or forecast) future events, as described above.

[0046] [Examples of candidate text (without candidate pair / phrase seed)]

[0047] According to another exemplary embodiment, the AI ​​platform (150) accesses candidate text (166). In an exemplary embodiment, the AI ​​platform (150) is configured to access candidate text (166) from KnowledgeBase0 (160), either directly from KnowledgeBase0 (160) or indirectly through one or more of several computing devices (180), (182), (184), (186), (188), and (190). According to the exemplary embodiment, the NLP model (156) is configured to extract candidate sentences containing a first phrase and a second phrase from candidate text (166). According to another exemplary embodiment, the NLP model (156) is configured to extract candidate phrases and pair all cause-and-effect combinations to form candidate cause-and-effect pairs. According to the exemplary embodiment, phrase extraction is performed using NPFST (or NP), an algorithm for extracting noun phrases from sentences using a finite state transformer. According to another exemplary embodiment, phrase extraction is performed using CP.NPFST (or CP), which is based on sentence element construct analysis to extract not only noun phrases but all types of phrases. See Handler et al., "Bag of What? Simple Noun Phrase Extraction for Text Analysis," Proceedings of the First Workshop of NLP and Computational Social Science, 114-124 (2016).

[0048] In an exemplary embodiment, candidate text (166) may include structured data, unstructured data, or both. For example, candidate text (166) may include the sentence, "Even though secondary school summer break was shortened, a lowering of gas prices lead to a rise in summer vacation travel." In one embodiment, NLP model (156) identifies a first candidate pair (P1) showing a non-causal relationship between the first phrase, "secondary school summer break was shortened," and the second phrase, "a rise in summer vacation travel." NLP model (156) also identifies a second candidate pair (P2) showing a causal relationship between the first phrase, "a lowering of gas prices," and the second phrase, "a rise in summer vacation travel."

[0049] In an exemplary embodiment, the NLP model (156) is provided with access to multiple causal relationship patterns (169). In an exemplary embodiment, the causal relationship patterns (169) are syntactically distinct from one another. The cause-and-effect patterns (168) and the causal relationship patterns (169) (as described above in relation to exemplary embodiments of candidate pairs) may be the same as or different from one another, or they may partially overlap in such a way that some patterns are shared by both sets (168) and (169), and other patterns are included in only one set (168) or (169) but not the other.

[0050] Dunietz et al.,The BECauSE Corpus 2.0:Annotating Causality and Overlapping Relations,Proceedings of the 11th The Linguistic Annotation Workshop, 95-104 (2017), provides non-restrictive examples of causal structure patterns. By removing causal structure patterns that are general regardless of source and often represent weak causal indicators (e.g., "Y by X"), a set of causal patterns (169) can be generated. The following table provides a non-exclusive list of exemplary causal patterns. [Table 1]

[0051] The NLP model (156) is configured to lemmatize each of the causal relationship pattern (169) and the candidate sentences from the candidate text (166). According to one embodiment, the lemmatization is performed on the verbs of the causal relationship pattern (169) (for example, in the case of item 1 in the table above, "causes" and "caused") and the candidate sentences from the candidate text (166). Each This involves converting to the base form (for example, "cause"). For instance, lemmatizing a verb in the cause-and-effect pattern "X causes Y" results in the lemmatized cause-and-effect pattern "X cause Y". Lemmatizing verbs makes it easier to associate cause-and-effect patterns with candidate sentences.

[0052] The NLP model (156) is further configured to convert the lemmatized causal relationship pattern into a lemmatized regular expression, for example, "(.*) causes (.*): (.*)cause(.*)". The lemmatized regular expression is compared with lemmatized candidate sentences (166) to find one or more corresponding ones. In an exemplary embodiment, matching involves identifying regular expressions and candidate sentences that have the same base form of verbs, e.g., "cause", "trigger", "result", "attribute", etc. In an exemplary embodiment, corresponding lemmatized candidate sentences are subject to verification, for example, that the first candidate pair P1 described above shows a non-causal relationship and the second candidate pair P2 described above shows a causal relationship. According to an exemplary embodiment, verification involves adding the first and second clauses of the corresponding lemmatized candidate sentence to the candidate pair (162) and processing the candidate pair as described above in relation to the exemplary embodiment of the candidate pair.

[0053] While the above description refers to an NLP model (156) that performs multiple tasks in a particular exemplary embodiment, it should be understood that NLP model (156) may include multiple NLP models, each to be assigned one or more specific tasks, and that multiple NLP models are collectively referred to as NLP model (156).

[0054] In some exemplary embodiments, the server (110) may be an IBM Watson® system available from International Business Machines Corporation in Armonk, New York, which is augmented using the mechanisms of the exemplary embodiments described below. The tools, collectively referred to as the statement assembler (152), AI model (154), NLP model (156), and director (158), are shown as being embodied in or integrated into the AI ​​platform (150) of the server (110). In some embodiments, the tools may be implemented in a separate computing system (e.g., server 190) connected to the server (110) across a network (105). Wherever they are embodied, the tools function to assist in the identification of cause-and-effect pairs and the construction of a knowledge base of cause-and-effect pairs.

[0055] The types of information processing systems that can utilize the AI platform (150) range from small handheld devices such as handheld computers / cell phones (180) to large mainframe systems such as mainframe computers (182). Examples of handheld computers (180) include personal digital assistants (PDAs), personal entertainment devices such as MP4 players, portable TVs, and compact disc players. Other examples of information processing systems include pen or tablet computers (184), laptop or notebook computers (186), personal computer systems (188), and servers (190). As shown, these various information processing systems may be networked together using a computer network (105). The types of computer networks (105) that can be used to interconnect these various information processing systems include local area networks (LANs), wireless local area networks (WLANs), the Internet, public switched telephone networks (PSTNs), other wireless networks, and any other network topology that can be used to interconnect information processing systems. Many of the information processing systems include non-volatile data stores such as hard drives or non-volatile memory or both. Some of the information processing systems may use separate non-volatile data stores (e.g., the server (190) utilizes a non-volatile data store (190 A ), and the mainframe computer (182) utilizes a non-volatile data store (182a)). The non-volatile data store (182 A ) may be a component external to the various information processing systems or may be within one of the information processing systems.

[0056] The information handling systems employed to support the AI ​​platform (150) may take many forms, some of which are shown in Figure 1. For example, the information handling system may take the form of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. In addition, the information handling system may take other form factors such as a personal digital assistant (PDA), a gaming device, an ATM machine, a portable telephone device, a communication device, or other devices including a processor and memory. Furthermore, while the information handling system may embody a northbridge / southbridge controller architecture, it will be noted that other architectures may also be employed.

[0057] In the art, an Application Programming Interface (API) is understood as a software intermediary between two or more applications. With respect to the AI ​​platform (150) shown and described in Figure 1, one or more APIs may be used to support one or more of the tools (152), (154), (156), and (158), and their associated functions. Referring to Figure 2, a block diagram (200) is provided showing the tools (152), (154), (156), and (158), and their associated APIs. As shown, several tools are embedded within the AI ​​platform (205). These tools include a sentence assembler (252) associated with API0 (212), an artificial intelligence model (e.g., a machine learning model) (254) associated with API1 (222), a natural language processing model (256) associated with API2 (232), and a director (258) associated with API3 (242). Each of these APIs may be implemented in one or more language and interface specifications.

[0058] As shown, API0(212) is configured to support and enable the functions represented by Sentence Assembler(252). API0(212) provides functional support for constructing sentences from first and second clauses using cause-and-effect patterns. API1(222) provides functional support for extracting candidate sentences from a corpus, lemmatizing sentences and causal patterns, generating regular expressions, and generating natural language questions. API2(232) provides functional support for eliciting inferences, generating question-and-answer pairs, and providing statistical measures and scores. API3(242) provides functional support for inputting cause-and-effect pairs into a knowledge base. As shown, each of APIs (212), (222), (232), and (242) is operably coupled with an API orchestrator (260), also known as the orchestration layer, which is understood in the art to function as an abstraction layer for transparently threading the separate APIs together. In one embodiment, the functions of the separate APIs may be coupled or combined. In another embodiment, the functions of the separate APIs may be further divided into further APIs. Thus, the API configurations shown herein should not be considered limiting. Accordingly, the functionality of a tool may be embodied or supported by its respective APIs, as shown herein.

[0059] Referring together to Figures 3A and 3B, a flowchart (300) is provided illustrating an embodiment of the process (or method) for determining whether a candidate phrase pair has a causal or non-causal relationship.

[0060] In Figure 3A, a corpus containing text is accessed (302), and a database of cause-and-effect patterns is accessed (304). In an exemplary embodiment, the cause-and-effect patterns are syntactically distinct from one another. The text of the accessed corpus is preprocessed to identify one or more candidate sentences. Each of the one or more candidate sentences contains a candidate pair containing a first clause and a second clause (306). The number of candidate pairs is identified and assigned to a variable M. TOTAL Allocate (308) and initialize the corresponding CandidatePair count variable M (310).

[0061] C andidatePair M The first and second clauses are applied to each of the accessed cause and effect patterns. Assignment By doing so, CandidatePair M from Transformed sentence It is constructed (312). Identify the number of transformation sentences and assign it to the variable N. TOTAL Assign (314) and initialize the corresponding VariantSentence count variable N (316). Use an AI model such as a natural language inference model, for example a machine learning model (MLM), to determine VariantSentence N The probability of a sentence being inferred from the corpus text is determined (318). The VariantSentence count variable N is incremented (320), and the incremented variable N is equal to N TOTAL Determine whether it is greater (322). A negative response to decision (322) is interpreted as indicating that further transformations remain for processing, thereby returning to step (318) for processing of the next transformation. A positive response to decision (322) is CandidatePair M This is interpreted as indicating that all variant sentences have been processed.

[0062] Referring to Figure 3B, a positive response to decision (322) results in the CandidatePair shown in Figure 3B. M From VariantSentence1 to VariantSentenceNTOTAL A statistical scale of the probability up to (324) is calculated. Examples of statistical scales include the mean and the median. It is determined whether the statistical scale meets the threshold (326). A positive response to the decision (326) is interpreted as indicating that a causal relationship exists between the first and second clauses of the candidate pair. M The first and second clauses of the statement are identified as having a causal relationship (328). If there is a negative response to the decision (326), the CandidatePair M The first and second clauses are identified as having no causal relationship (330). In any case, step (328) Also (330) , in other words, a negative response from the decision in step (326) either to Next, the CandidatePair count variable M is incremented (332), and the incremented count variable M becomes M TOTAL The process determines whether the result is greater than (334). A negative response to the decision (334) is interpreted as indicating that further candidate pairs remain for processing, thereby returning to step (312). A positive response to the decision (334) is interpreted as indicating that all candidate pairs have been processed, thereby proceeding to step (336). In (336), the identified candidate statements containing candidate pairs with causal relationships are compiled and printed. The printed candidate statements are then used to populate a database, for example, KnowledgeBase1(170) in Figure 1 (338).

[0063] Referring together to Figures 4A and 4B, flowcharts (400) are provided, each illustrating an embodiment of a process (or method) for identifying a corresponding result or cause clause, starting with text (e.g., one or more sentences) and a candidate cause or result clause.

[0064] In Figure 4A, the corpus containing text is accessed (402). One or more first phrases representing cause or effect are also accessed (404). The number of first phrases accessed is identified and assigned to the variable X. TOTALAllocate (406) and initialize the corresponding FirstPhrase count variable X (408).

[0065] First Phrase X The text is presented to a natural language processing (NLP) model to create a natural language (NL) question (410). The NL question is presented to an AI model, such as a question and answer (QA) model, for example, a machine learning model (MLM), to extract one or more candidate second phrases from the corpus text (412). FirstPhrase X This corresponds to each of the one or more candidate second phrases. attach And, to respond Attached Provide candidate pairs (414). Attached Each candidate pair is assigned a cause-and-effect relationship score (416). In an exemplary embodiment, the AI ​​model assigns the score. FirstPhrase X And the second candidate clause has a cause-and-effect relationship. Most High possibility Correspondence with one or more scores that reflect Attached Select a candidate pair (418).

[0066] Referring to Figure 4B, the correspondence of the selected high score Attached Verification is performed to determine whether the candidate pairs are causally related (420). According to one embodiment, verification (420) is performed on the correspondence of the selected high score. Attached This may involve analyzing the candidate pairs by applying steps (308) to (336) in Figure 3A. The FirstPhrase count variable X is incremented (422), and the FirstPhrase X The incremented value of X TOTAL Determine whether it is greater (424). A negative response to decision (424) is interpreted as indicating that further accessed first clauses remain for processing, thereby returning to step (410). A positive response to decision (424) is interpreted as indicating that all accessed first clauses have been processed. Selected high-scoring correspondence AttachedThe pairs are compiled and output (426) and used to insert them into the database (428).

[0067] Referring together to Figures 5A and 5B, a flowchart (500) is provided illustrating an embodiment of a process (or method) for determining whether candidate sentences accessed from a text-containing corpus have a cause-and-effect relationship.

[0068] In Figure 5A, a database of causal relationship patterns (e.g., the causal relationship pattern (169) in Figure 1 described above) is accessed (502). In an exemplary embodiment, the causal relationship patterns are syntactically distinct from one another. The causal relationship patterns are lemmatized (504). In an exemplary embodiment, the verbs of the causal relationship patterns are lemmatized. For example, the verb "causes" in the causal relationship pattern "X causes Y" and the verb "caused" in the causal relationship pattern "X caused Y" are converted to the base form of the verb "cause". In an exemplary embodiment, an NLP model is used for lemmatization. The lemmatized causal relationship phrase (e.g., X cause Y) is converted to a regular expression (e.g., "(.*) causes (.*)" (506). The number of regular expressions is identified and assigned to the variable Z. TOTAL Allocate (508) and initialize the lemmatized regular expression count variable Z (510).

[0069] Access a corpus containing text (512). Identify one or more candidate sentences from the corpus of text (514). For each sentence, extract one or more candidate pairs of a first and second phrase (516). In an exemplary embodiment, the first and second phrases contain noun phrases. In another exemplary embodiment, either or both of the first and second phrases contain a noun phrase and an explanatory prepositional phrase. In yet another embodiment, neither or both of the first and second phrases contain noun phrases. Identify the number of candidate sentences and assign a variable Y to it. TOTALAllocate (518). Initialize the count variable Y of CandidateSentence (520).

[0070] Candidate Sentence Y It is also lemmatized (522). As shown in Figure 5B, the lemmatized CandidateSentence Y Determine whether one or more pairs of the first and second clauses of the lemmatized regular expression correspond to the lemmatized regular expression (524). In an exemplary embodiment, determination (524) is whether the lemmatized CandidateSentence Y Each pair of the first and second clauses is a lemmatized Regex Z This includes assessing whether it has the same lemmatized verb. An affirmative response to decision (524) is interpreted as indicating that the first and second phrase pairs of the candidate sentences are potentially causally related. Decision (524) in affirmative expression but answer So If this happens, CandidateSentence Y The candidate pairs of the and phrases are added to a list, which in one embodiment is a file (526), ​​and the Regex count variable Z is incremented (528). On the other hand, the non-affirmative expression is determined (524). but answer So The process then proceeds to step (528) incrementing the Regex count variable Z.

[0071] After incrementing the Regex count variable Z (528), the incremented Regex count variable Z becomes Z TOTAL Determines whether it is greater than (530). A negative answer to the decision (530) is all of the lemmatized regular expression. In contrast Candidate Sentence Y This is interpreted as indicating that no decision has been made regarding the response, and as a result, the process is as follows: CandidateSentence Y And an incremented regular expression, for example, Regex ZReturn to decision (524) to determine whether a corresponding exists between them. An affirmative response to decision (530) is CandidateSentence Y This is interpreted as indicating that the entire regular expression matching process has been completed, and as a result, the candidate statement variable is incremented (532).

[0072] The incremented variable Y is Y TOTAL Determine whether it is greater (534). A negative response to decision (534) is interpreted as indicating that not all of the lemmatized candidate sentences have been processed, thereby the process increments the CandidateSentence Y Return to step (522) for lemmatization. An affirmative response to decision (534) is interpreted as indicating that all lemmatized candidate sentences have been processed (i.e., correspondence decisions (524) have been made for all lemmatized candidate sentences for all lemmatized regular expressions). Optionally, to verify that the first and second clauses of a candidate sentence have a cause-and-effect relationship, send the list of candidate sentences with identified candidate pairs created in step (526) to step (308) in Figure 3A (536).

[0073] Certain exemplary embodiments of the systems, methods, and computer program products described herein generate high-quality sets of cause-and-effect pairs in a substantially or fully unsupervised automated manner. The exemplary embodiments further involve the use of cause-and-effect pairs for further processing, their representation as causal knowledge graphs, and their use for decision-making assistance or predictive analysis.

[0074] The tools and APIs shown in Figures 1 and 2, respectively, and the processes shown in Figures 3A, 3B, 4A, 4B, 5A, and 5B, are used to illustrate and explain how causal relationships are identified and verified. The functional tools (152), (154), (156), and (158), and the embodiments of their associated functions, may be embodied in a single computer system / server, or in some embodiments, they may be configured in a cloud-based system that shares computing resources. Referring to Figure 6, a block diagram (600) is provided showing an example of a computer system / server (602) for implementing the processes described above with respect to Figures 3A to 5B. Hereinafter, the computer system / server (602) will be referred to as a host (602) that communicates with a cloud-based support system. The host (602) can operate using the environment or configuration of a number of other general-purpose or special-purpose computing systems. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with the host (602) include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and file systems (e.g., distributed storage environments and distributed cloud computing environments) that include any of the above systems, devices, and their equivalents.

[0075] A host (602) may be described in the general context of computer system executable instructions, such as program modules, that are executed by a computer system. Generally, a program module may include routines, programs, objects, components, logic, and data structures that perform a specific task or implement a specific abstract data type. A host (602) may be implemented in a distributed cloud computing environment (610) where tasks are performed by remote processing devices linked over a communication network. In a distributed cloud computing environment, program modules may reside on both local computer system storage media, including memory storage devices, and remote computer system storage media.

[0076] As shown in Figure 6, the host (602) is represented in the form of a general-purpose computing device. The components of the host (602) may include, but are not limited to, one or more processors or processing units (604), for example, a hardware processor, system memory (606), and a bus (608) that connects various system components, including the system memory (606), to the processing units (604). The bus (608) represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of the various bus architectures. Examples, but not limited to, such architectures include the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MCA) bus, the Enhanced ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus. The host (602) typically includes various computer system-readable media. Such media may be any available media accessible from the host (602), and may include both volatile and non-volatile media, as well as removable and non-removable media.

[0077] System memory (606) may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) (630) or cache memory (632) or both. As merely an example, a storage system (634) may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown; usually called a “hard drive”). Not shown, a magnetic disk drive may be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a “floppy disk”), and an optical disk drive may be provided for reading from or writing to a removable, non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical medium. In such cases, each may be connected to a bus (608) by one or more data medium interfaces.

[0078] A program / utility (640) having a set (at least one) of program modules (642) may be stored in system memory (606), as well as, but not limited to, an operating system, one or more application programs, other program modules, and program data. Each of the operating system, one or more application programs, other program modules, and program data, or any combination thereof, may include an implementation of a networking environment. The program modules (642) generally perform functions or methods, or both, of embodiments for supporting and enabling reinforcement learning through random action replay of natural language (NL). For example, a set of program modules (642) may include tools (152), (154), (156), or (158), or a combination thereof, as described in Figure 1.

[0079] The host (602) may communicate with one or more external devices (614), such as a keyboard or pointing device, a display (624), one or more devices that allow a user to interact with the host (602), or any device that allows the host (602) to communicate with one or more other computing devices (e.g., a network card, a modem), or a combination thereof. Such communication may occur via an input / output (I / O) interface (622). Furthermore, the host (602) may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via a network adapter (620). As depicted, the network adapter (620) communicates with other components of the host (602) via a bus (608). In one embodiment, multiple nodes of a distributed file system (not shown) communicate with the host (602) via the I / O interface (622) or via the network adapter (620). Although not shown, it should be understood that other hardware and / or software components may be used in conjunction with the host (602). Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems.

[0080] In this document, the terms “computer program medium,” “computer-usable medium,” and “computer-readable medium” are generally used to refer to media such as RAM (630), cache (632), and storage systems (634), including removable storage drives and system memory (606), which include hard disks installed on hard disk drives.

[0081] The computer program (also called computer control logic) is stored in system memory (606). The computer program may be received via a communication interface such as a network adapter (620). When such a computer program is executed, it enables the computer system to perform features of this embodiment as described herein. In particular, when the computer program is executed, it enables the processing unit (604) to perform features of the computer system. Thus, such a computer program represents a controller of the computer system.

[0082] In one embodiment, the host (602) is a node in a cloud computing environment. As is known in the Art, cloud computing is a service delivery model to enable convenient on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and deprovisioned with minimal administrative effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models. Examples of such characteristics are as follows:

[0083] On-demand self-service: Cloud consumers can unilaterally provision computing functions such as server time and network storage automatically as needed, without requiring human interaction with a service provider.

[0084] Broad network access: Functionality can be accessed over the network, and through standard mechanisms that facilitate use by heterogeneous thin client platforms or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0085] Resource pooling: A provider pools its computing resources to serve multiple consumers using a multi-tenant model. Multiple different physical and virtual resources are dynamically allocated and reallocated according to demand. Consumers generally have no control over or awareness of the exact location of the resources provided, although they may be able to specify the location at a higher layer of abstraction (e.g., country, state, or data center), thus demonstrating location independence.

[0086] Rapid Adaptability: Features can be provisioned quickly and adaptively, sometimes automatically, to scale out rapidly and de-provisioned to scale in rapidly. Consumers often see an unlimited number of features available for provisioning and can purchase any number at any time.

[0087] Measured Services: Cloud systems automatically control and optimize resource usage by leveraging instrumentation capabilities at some abstraction layer appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported to provide transparency to both the providers and consumers of the services used.

[0088] The service model is as follows:

[0089] Software as a Service (SaaS): The functionality provided to consumers is the use of a provider's applications running on cloud infrastructure. These applications are accessible from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application functionalities, with the exception of potentially limited user-specific application configuration settings.

[0090] Platform as a Service (PaaS): The functionality offered to consumers is the deployment of applications they have created or acquired, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but they do control the applications being deployed and, in some cases, the configuration of the application hosting environment.

[0091] Infrastructure as a Service (IaaS): The functionality provided to consumers is the provisioning of processing, storage, networking, and other fundamental computing resources that enable consumers to deploy and run any software they choose (which may include operating systems and applications). Consumers do not manage or control the underlying cloud infrastructure, but they do control the operating system, storage, and deployed applications, and, in some cases, have limited control over selected networking components (e.g., host firewalls).

[0092] The deployment model is as follows:

[0093] Private Cloud: Cloud infrastructure is operated solely for the organization. It may be managed by the organization or a third party and may reside on-premises or off-premises.

[0094] Community Cloud: A cloud infrastructure is shared by several organizations to support a specific community that shares common interests (e.g., mission, security requirements, policies, and compliance considerations). It may be managed by an organization or a third party and may reside on-premises or off-premises.

[0095] Public cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0096] Hybrid cloud: A configuration of two or more clouds (private, community, or public) where the cloud infrastructure remains a separate entity, but they are bound together by standard or patented technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0097] Cloud computing environments are service-oriented environments that emphasize statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0098] Referring here to Figure 7, an exemplary cloud computing network (700) is shown. The cloud computing network (700) includes a cloud computing environment (750) having one or more cloud computing nodes (710) that can communicate with local computing devices used by cloud consumers. Examples of these local computing devices include, but are not limited to, personal digital assistants (PDAs) or cellular phones (754A), desktop computers (754B), laptop computers (754C), or automotive computer systems (754N), or a combination thereof. Individual nodes within a cloud computing node (710) may communicate with each other further. They may be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment (700) to provide infrastructure, a platform, or software, or a combination thereof, as a service, eliminating the need for cloud consumers to maintain resources on their local computing devices. The types of computing devices (754A-N) shown in Figure 7 are for illustrative purposes only, and it should be noted that the cloud computing environment (750) can communicate with any type of computerized device via any type of network or network-addressable connection or both (for example, using a web browser).

[0099] Referring now to Figure 8, the set of function abstraction layers (800) provided by the cloud computing network in Figure 7 is shown. It should be understood in advance that the components, layers, and functions shown in Figure 8 are for illustrative purposes only and embodiments are not limited thereto. As depicted, the following layers and corresponding functions are provided: hardware and software layers (810), virtualization layer (820), management layer (830), and workload layer (840).

[0100] The hardware and software layer (810) includes hardware and software components. Examples of hardware components include mainframes, such as IBM® zSeries® systems; RISC (Reduced Instruction Set Computer) architecture-based servers, such as IBM pSeries® systems, IBM xSeries® systems, and IBM BladeCenter® systems; storage devices; and network and networking components. Examples of software components include network application server software, such as IBM WebSphere® application server software; and database software, such as IBM DB2® database software (IBM, zSeries, pSeries, xSeries, BladeCenter, WebSphere, and DB2 are trademarks of International Business Machines Corporation, registered in numerous jurisdictions worldwide).

[0101] The virtualization layer (820) provides an abstraction layer from which examples of virtual entities may be provided, such as virtual servers, virtual storage, virtual networks including virtual private networks, virtual applications and operating systems, and virtual clients.

[0102] In one example, the management layer (830) may provide functions such as resource provisioning, metering and pricing, a user portal, service layer management, and SLA planning and execution. Resource provisioning provides the dynamic procurement of computing and other resources used to perform tasks within the cloud computing environment. Metering and pricing provides cost tracking as resources are used within the cloud computing environment and invoicing or invoice creation for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification of cloud consumers and tasks, as well as protection of data and other resources. The user portal provides consumers and system administrators with access to the cloud computing environment. Service layer management provides the allocation and management of cloud computing resources to ensure that the required service layers are met. Service layer agreement (SLA) planning and execution provides the proactive preparation and procurement of cloud computing resources for anticipated future requirements in accordance with the SLA.

[0103] The workload layer (840) provides examples of functions for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include, but are not limited to, mapping and navigation, software development and lifecycle management, provision of virtual classroom education, data analysis processing, transaction processing, and identification and extraction of causal knowledge.

[0104] While specific embodiments of this embodiment are shown and described, it will be apparent to those skilled in the art that changes and modifications can be made based on the teachings herein without departing from the embodiments and their broader aspects. Therefore, the appended claims should encompass, within their scope, all such changes and modifications that fall within the true spirit and scope of the embodiments. Furthermore, it should be understood that embodiments are defined solely by the appended claims. Where a particular number of claim elements to be introduced is intended, such intention is explicitly stated in the claims; where such statement is absent, it will be apparent to those skilled in the art that no such limitation exists. As a non-restrictive example, for the sake of understanding, the following appended claims include the use of the introductory phrases “at least one” and “one or more” for introducing claim elements. However, the use of such phrasing should not be interpreted as suggesting that the introduction of a claim element with the indefinite article "a" or "an" limits any particular claim containing such introduced claim element to embodiments containing only one such element, even if the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" are included in the same claim. The same applies to the use of definite articles in claims. As used herein, the term "and / or" means either one or both (or any combination or all of the terms or expressions mentioned), for example, "A, B, and / or C" includes A only, B only, C only, A and B, A and C, B and C, and A, B, and C.

[0105] This embodiment may be a system, method, or computer program product, or a combination thereof. In addition, selected embodiments of this embodiment may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment that combines software and / or hardware embodiments, all of which may be generally referred to herein as “circuits,” “modules,” or “systems.” Furthermore, embodiments of this embodiment may take the form of a computer program product embodied in one or more computer-readable storage media having computer-readable program instructions that cause a processor to execute embodiments of this embodiment. Thus embodied, the disclosed systems, methods, or computer program products, or combinations thereof, operate to provide improved identification and verification of causal pairs.

[0106] A computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing, but is not limited to the following. A non-exclusive list of more specific examples of computer-readable storage mediums includes portable computer diskettes, hard disks, dynamic or static random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), magnetic storage devices, portable compact disk read-only memory (CD-ROM), digital multipurpose disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved raised structures on which instructions are recorded, and any suitable combination of the foregoing. When used herein, computer-readable storage media should not be construed as themselves being radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or transient signals such as electrical signals transmitted through wires.

[0107] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or they may be downloaded to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers those computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0108] The computer-readable program instructions for performing the operation of this embodiment may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java®, Smalltalk®, or C++, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially as a standalone software package on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer, server, or cluster of servers. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), and the connection may be made to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute a computer-readable program instruction by personalizing the electronic circuit using state information of the computer-readable program instruction in order to perform an aspect of this embodiment.

[0109] This specification describes aspects of the embodiment with reference to flowcharts, block diagrams, or both, of methods, apparatus (systems), and computer program products according to the embodiment. It will be seen that each block in the flowchart or block diagram, or both, and combinations of blocks in the flowchart or block diagram, or both, can be implemented by computer-readable program instructions.

[0110] By providing these computer-readable program instructions to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, and manufacturing a machine, instructions executed via the processor of the computer or other programmable data processing device may create means for implementing functions / operations specified in one or more blocks of a flowchart or block diagram, or both. By storing these computer-readable program instructions in a computer-readable storage medium capable of instructing a computer, a programmable data processing device, or other device, or a combination thereof, to function in a particular manner, the computer-readable storage medium storing the instructions may contain a product containing instructions that implement modes of functions / operations specified in one or more blocks of a flowchart or block diagram, or both.

[0111] Computer-readable program instructions may be loaded onto a computer, other programmable data processing device, or other device to execute a series of operable steps on the computer, other programmable device, or other device to generate a computer-implemented process, thereby enabling the instructions executed on the computer, other programmable device, or other device to implement a function / operation specified in one or more blocks of a flowchart or block diagram or both.

[0112] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this embodiment. In this context, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may be performed in an order different from that shown in the diagram. For example, depending on the related functions, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. Further blocks not shown in the diagram may be included, for example, before, after, or simultaneously with one or more shown blocks. It will also be noted that each block in the block diagram or flowchart diagram, or both, and any combination of blocks in the block diagram or flowchart diagram, or both, may be implemented by a special-purpose hardware-based system that performs a specified function or operation, or a combination of special-purpose hardware and computer instructions.

[0113] While this specification has described specific embodiments for illustrative purposes, it will be evident that various modifications can be made without departing from the spirit and scope of these embodiments. In particular, the identification and verification of causal pairs may be performed by multiple different computing platforms or across multiple devices. Furthermore, data storage or corpus or both may be localized, remote, or spread across multiple systems. Accordingly, the scope of protection of these embodiments is limited only by the following claims and their equivalents.

Claims

1. A computer system, A computer processor that is coupled to memory in an operable manner, A statement assembler operablely associated with the aforementioned computer processor, Access the candidate text and candidate pairs of the first and second clauses, and, A statement assembler that generates multiple modified statements containing the first and second clauses by substituting the first and second clauses into multiple cause-and-effect patterns, An artificial intelligence (AI) platform that communicates with the computer processor and the memory, wherein the AI ​​platform is It is an AI model, Determine the probability that each of the aforementioned multiple modified sentences is inferred from the candidate text. The statistical scale for each of the aforementioned probabilities is calculated, and, An AI model configured to assess the calculated statistical scale and determine whether the first and second clauses are causally or non-causally related to each other, A director configured to input the candidate text, which includes the first and second phrases having the aforementioned causal relationship, into a knowledge base. An AI platform that has A computer system equipped with the following features.

2. The computer system according to claim 1, wherein the AI ​​model includes an inference model.

3. A natural language processing model (NLP model) that communicates with a corpus containing the aforementioned candidate texts, and is configured to extract the aforementioned candidate pairs from the corpus. The computer system according to claim 1, further comprising:

4. The aforementioned NLP model is Lemma-forming multiple causal relationship patterns, Convert the aforementioned lemmatized causal relationship patterns into regular expressions, The candidate text is lemmatized, Determine whether the lemmatized candidate text corresponds to one or more of the regular expressions, and Identifying the candidate pair from the lemmatized candidate text and one or more of the corresponding regular expressions. The computer system according to claim 3, further configured as follows.

5. The aforementioned NLP model is The computer system according to claim 3, configured to extract noun phrases and prepositional phrases describing the noun phrases from the corpus, wherein the noun phrases and the prepositional phrases collectively represent the first phrase or the second phrase.

6. The computer system according to any one of claims 1 to 5, wherein the director is configured to train the AI ​​model using the candidate text comprising the first and second clauses having the causal relationship.

7. A computer system, A computer processor that is coupled to memory in an operable manner, The system comprises an artificial intelligence (AI) platform that communicates with the computer processor and the memory, and the AI ​​platform is A natural language processing model (NLP model) configured to generate natural language (NL) questions using a first phrase that represents cause or effect, Extract one or more candidate second phrases from the candidate text, Determine the probability that each of the one or more candidate second phrases is an answer to the NL question, and A first AI model is configured to select the candidate second phrase with the highest probability as having a causal relationship with the first phrase, A director configured to input the first phrase and the selected second phrase into a knowledge base. Having, Computer system.

8. The computer system according to claim 7, wherein the first AI model includes a question and answer (QA) model.

9. The NLP model communicates with a corpus containing the candidate text, and the NLP model is further configured to extract the first phrase from the corpus. The computer system according to claim 7.

10. The aforementioned AI platform is Access the first clause and the selected second clause, and, A statement assembler configured to generate multiple modified statements containing the first clause and the selected second clause by substituting the first clause and the selected second clause into multiple cause-and-effect patterns, Determine the probability that each of the aforementioned multiple modified sentences is inferred from the candidate text. The statistical scale for each of the aforementioned probabilities is calculated, and, A second AI model configured to assess the calculated statistical scale and determine whether the first clause and the selected second clause are causally or non-causally related to each other. The computer system according to claim 7, further comprising:

11. The computer system according to any one of claims 7 to 10, wherein the director is configured to train the AI ​​model using the first clause and the selected second clause.

12. In the processor, Steps to access candidate text, A procedure for extracting candidate pairs of the first and second phrases from the aforementioned candidate text, A procedure for generating multiple modified sentences containing the first and second clauses by substituting the extracted first and second clauses into multiple cause-and-effect patterns, A procedure for determining the probability that each of the aforementioned multiple modified sentences is inferred from the candidate text, The procedure for calculating the statistical scale of each of the aforementioned probabilities, A procedure for assessing the calculated statistical scale to determine whether the first and second clauses are causally or non-causally related to each other, and, A computer program for causing a computer to perform a procedure for inputting candidate text, which includes the first and second clauses having the aforementioned causal relationship, into a knowledge base.

13. The computer program according to claim 12, which causes the processor to further perform the procedure of extracting candidate pairs from a corpus.

14. The aforementioned processor, Procedures for lemmatizing multiple causal relationship patterns, A procedure for converting the aforementioned lemmatized causal relationship patterns into regular expressions, The procedure for lemmatizing the aforementioned candidate text, A procedure for determining whether the lemmatized candidate text corresponds to one or more of the regular expressions, and The computer program according to claim 13, for further performing a step of identifying the candidate pair from the lemmatized candidate text and the corresponding one or more regular expressions.

15. The aforementioned processor, A computer program according to claim 13 or 14 for causing a procedure to extract noun phrases and prepositional phrases describing the noun phrases from the corpus, wherein the noun phrases and the prepositional phrases collectively represent the first phrase or the second phrase.

16. A method implemented by a computer, The steps include accessing candidate text and candidate pairs of the first and second phrases, The steps include: substituting the first and second clauses into a plurality of cause-and-effect patterns to generate a plurality of modified sentences containing the first and second clauses; By utilizing AI models, Determine the probability that each of the aforementioned multiple modified sentences is inferred from the candidate text. The statistical scale for each of the aforementioned probabilities is calculated, and, The steps include: assessing the calculated statistical scale to determine whether the first and second clauses are causally or non-causally related to each other; The step of inputting the candidate text, which includes the first and second phrases having the aforementioned causal relationship, into a knowledge base. A method implemented on a computer equipped with [a certain feature].

17. The method implemented on a computer according to claim 16, wherein the AI ​​model includes an inference model.

18. A step of extracting the candidate pairs from a corpus, wherein the corpus includes the candidate texts. A computer-implemented method according to claim 16, further comprising:

19. The stage of lemmatizing multiple causal relationship patterns, The steps include converting the aforementioned lemmatized causal relationship patterns into regular expressions, The step of lemmatizing the aforementioned candidate text, The step of determining whether the lemmatized candidate text corresponds to one or more of the regular expressions, A step of identifying the candidate pair from the lemmatized candidate text and the corresponding one or more regular expressions. A computer-implemented method according to claim 18, further comprising the following:

20. A step of extracting noun phrases and prepositional phrases describing the noun phrases from the corpus, wherein the noun phrases and prepositional phrases represent the first phrase or the second phrase as a whole. A computer-implemented method according to claim 18, further comprising the following:

21. A computer-implemented method according to any one of claims 16 to 20, further comprising the step of training the AI ​​model using the candidate text comprising the first and second clauses having the aforementioned causal relationship.

22. A method implemented by a computer, The process involves generating a natural language (NL) question using a first phrase that expresses cause or effect, This is the stage where the first AI model is utilized. The steps include: extracting one or more candidate second phrases from the candidate text; The steps include determining the probability that each of the one or more candidate second phrases is an answer to the NL's question, A step of utilizing a first AI model, which includes the step of selecting the candidate second phrase having the highest probability as having a causal relationship with the first phrase, The step of inputting the first phrase and the selected second phrase into a knowledge base. A method implemented on a computer equipped with [a certain feature].

23. The method implemented on a computer according to claim 22, wherein the first AI model includes a question and answer (QA) model.

24. The computer-implemented method according to claim 22, further comprising the step of training the AI ​​model using the first clause and the selected second clause.

25. A step of substituting the first clause and the selected second clause into a plurality of cause-and-effect patterns to generate a plurality of modified sentences including the first clause and the selected second clause, At the stage of utilizing the second AI model, The steps include determining the probability that each of the multiple modified sentences is inferred from the candidate text, A step of utilizing a second AI model, which includes a step of calculating the statistical scale of each of the aforementioned probabilities, A step of assessing the calculated statistical scale to determine whether the first clause and the selected second clause are causally or non-causally related to each other. A computer-implemented method according to any one of claims 22 to 24, further comprising:

Images