Domain-specific information extraction from documents

Abstract

Extracting information from a domain-specific document can include performing named entity recognition on the document. A regular expression tagger can tag regular expressions in the domain-specific document without a pre-annotated training set for the regular expression tagger. The defined domain-specific terms can be expanded with terms and phrases related to the defined domain-specific terms based on rules specified for desired information about the named entity to be extracted. A dictionary-based tagger can tag the defined and expanded domain-specific terms and phrases in multiple domain-specific documents. Named entity recognition conflicts between the regular expression tagger and the dictionary-based tagger can be resolved. Desired information about the named entity can be extracted. Classification can be performed to identify whether the domain-specific document is relevant to the desired information.

Description

[Technical Field]

[0001] This disclosure relates to domain-specific information extraction from documents. Such techniques can be particularly useful for gaining insight and improving productivity, reliability, and security. [Background technology]

[0002] In some domains (e.g., chemical production), it may be common for humans to manually review documents to extract information of interest. However, this may be inefficient and insufficient for scaling information extraction solutions across diverse applications across an enterprise. Furthermore, the collected and / or extracted information may be incomplete because only a subset of available documents may be examined. For example, a manufacturing department may be tasked with mining reliability-related data to identify common root causes of unplanned events. As another example, an environmental, health, and safety department may be tasked with investigating the root causes of safety incidents. As yet another example, a supply chain department may be tasked with determining which customers are complaining and why. A human resources department may be concerned with issues that negatively impact employee experience. Documents available to an enterprise may contain relevant information, but due to time and resource constraints, they may not be examined in sufficient detail to reveal the relevant information to the enterprise. The data used to draw conclusions may be incomplete.

[0003] Natural language processing (NLP) is a subfield of linguistics, computer science, and artificial intelligence that deals with the interaction between computers and human language, particularly how to program computers to process and analyze large amounts of natural language data. Information extraction is the task of automatically extracting structured information from unstructured and / or semi-structured machine-readable documents and / or other electronically represented sources. While commercially available open-source NLP and / or information extraction methods exist, training such models tends to be data-demanding and data-specific. For example, most NLP models require a large amount of annotated documents to train the NLP model. In domain-specific contexts, there may not be enough annotated documents or other training material available for the NLP model to perform with the desired accuracy. Domain-specific contexts may use domain-specific terminology that differs from other domains. Summary of the Invention

[0004] The present disclosure is directed to extracting information from text-based, domain-specific documents, which can help companies gain insights in a wide range of applications. In the chemical industry, examples of such applications include identifying the root causes of production losses, safety incidents, and customer complaints. Examples of information that may be desired to be extracted include unit operations, chemical properties, human injuries, etc. The present disclosure is also directed to identifying documents containing desired information (e.g., whether a particular document describes an incident in which a human was injured by mobile equipment rather than other factors). Research and development related to knowledge extraction, context modeling, and document classification have numerous applications. The results of information extraction can be used to improve operational reliability, safety, and / or customer experience, among other benefits. Many companies may have large volumes of documents stored in databases, making manual review of them prohibitively expensive and inefficient. Currently available information extraction solutions cannot extract desired information in domain-specific contexts with satisfactory accuracy without significant additional training with labeled data.

[0005] To address such concerns, at least one embodiment provides a framework for automated information extraction for domain-specific context. Such a framework can include a pre-trained component for extracting information (e.g., locations and business names), a self-tuning component that can be trained on a relatively small number of annotated data sources containing domain-specific terminology, and an insight extraction component that builds on the collected information. With respect to the annotated data sources or training materials, "relatively small" means fewer than those recommended or required by existing NLP solutions.

[0006] The above summary is not intended to describe each disclosed embodiment or every implementation. More particularly, the present specification exemplifies exemplary embodiments. In several places throughout the application, guidance is provided through lists of examples, which examples can be used in various combinations. In each instance, the recited list serves only as a representative group and should not be interpreted as an exclusive list. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram of a framework for domain-specific document information extraction and classification. [Figure 2] FIG. 1 is a block diagram of a more detailed framework for domain-specific document information extraction and classification. [Figure 3] FIG. 1 is a block diagram of a framework for named entity recognition. [Figure 4] An example of the results of information extraction from domain-specific documents is shown below. [Figure 5] FIG. 1 is a flow diagram illustrating a method for domain-specific information extraction from documents. [Figure 6] 1 illustrates an example machine upon which a set of instructions may be executed to cause the machine to perform various methodologies discussed herein. DETAILED DESCRIPTION OF THE INVENTION

[0008] Information extraction involves processing natural language text documents to return structured information of interest. There are various subtasks of information extraction, such as named entity recognition, coreference resolution, relation extraction, and knowledge-based reasoning. These tasks can be performed in different ways, including pattern matching approaches, dictionary-based approaches, machine learning-based approaches, and deep learning-based approaches. Most existing solutions require a significant amount of training data to operate accurately on domain-specific documents. Some domains may not have enough training data available for existing models to operate accurately.

[0009] At least one embodiment includes a language processing framework that can extract information of interest with high accuracy and small amounts of training data compared to available NLP solutions. Multiple language processing approaches can be integrated into a single solution to provide described benefits, such as improved safety, reliability, customer experience, and employee experience.

[0010] The figures herein follow a numbering convention in which the first digit or digits correspond to the figure number of the figure, and the remaining digits identify an element or component within the figure. Similar elements or components between different numbers can be identified by using similar numbers. For example, 102 can designate element "02" in FIG. 1, and a similar element can be designated as 202 in FIG. 2. Similar elements within figures may be referenced with a hyphen and an extra number or letter. Such similar elements may generally be referenced without the hyphen and extra number or letter. For example, elements 108-1 and 108-2 in FIG. 1 may be collectively referred to as 108. As will be understood, elements shown in various embodiments herein may be added, interchanged, and / or eliminated to provide additional embodiments. Additionally, it will be understood that the proportions and relative scales of the elements provided in the figures are intended to illustrate certain embodiments of the present invention and should not be construed in a limiting sense.

[0011] 1 is a block diagram of a framework for information extraction and classification of domain-specific documents 102. The framework can include domain-specific documents 102, various modules 104, 106, and applications 108. As used herein, a "module" can include program instructions and / or hardware, but at least includes program instructions. In this context, an "application" refers to what the framework is used for (e.g., information extraction, document classification, etc.).

[0012] As shown, the domain-specific documents 102 are primarily text documents, but may also include graphs, images, or other non-text fields. The domain-specific documents 102 are electronic documents that can be stored in a database. The domain-specific documents 102 contain domain-specific terms. Existing pre-trained NLP models cannot recognize the domain-specific terms. It can also be difficult to customize or modify an NLP model to identify specific information of interest within the domain-specific documents 102. Furthermore, available datasets from the domain-specific documents 102 are too small to accommodate existing NLP models. For example, common available NLP models suggest training a classification model using more than 50 “true” documents per class, where “true” documents are documents that should be classified as interesting by the classification model. Without such training, the NLP model will not accurately or efficiently classify other documents. One example of why insufficient data (or documents) may exist is that a company may not have many accidents and therefore may not have many accident reports that are useful in the environmental health and safety domain. If an NLP model is inaccurate in classifying documents, it may not be as useful or efficient as having a human manually perform the review and classification process.

[0013] According to at least one embodiment, the modules of the framework can be used for different applications. For example, the named entity recognition module 104 can be used for both the information extraction application 108-1 and the document classification application 108-2. In the example shown in FIG. 1 , the classification module 106 is used for the classification application 108-2 but not for the information extraction application 108-1. In this example, the document classification application 108-2 can also include information extraction, while the information extraction application 108-1 does not include document classification. Examples of domain-specific information extraction applications 108-1 at the entity level include identifying chemical properties and identifying unit operations, among others. Examples of domain-specific document classification applications 108-2 are classifying documents as related to a particular safety event or as related to a particular reliability event. For example, the document classification application 108-2 can be run on each of the domain-specific documents 102 to determine whether each of them is related to a particular safety event. The document classification module 104 and the classification module 106 are described in more detail with respect to FIG. 2 .

[0014] FIG. 2 is a block diagram of a more detailed framework for information extraction and classification of domain-specific documents 202. The framework may include an optional preprocessing module 210. The preprocessing module 210 may be configured to break down the domain-specific documents 202 into words and restore them to their corresponding stems for future analysis and modeling. The preprocessing module 210 may include a sentence segmentation submodule 212, a tokenization submodule 214, a lemmatization submodule 216, and a part-of-speech (POS) tagging module 218. The sentence segmentation submodule 212 may be configured to identify where sentences begin and end in the domain-specific documents 202, thereby segmenting the text into sentences. The tokenization submodule 214 may be configured to define sections of strings of input characters, thereby identifying groups of characters as words. The lemmatization submodule 216 may be configured to group word forms (e.g., verb conjugations) together so that they can be analyzed as a single term. The POS tagging submodule 218 may be configured to mark words in text as corresponding to parts of speech based on definitions and context. For example, the POS tagging submodule 218 may be configured to identify words as nouns, verbs, adverbs, adjectives, etc. The preprocessing module 210 may operate on the domain-specific documents 202 before passing the results of the operations of the preprocessing module 210 to the named entity recognition module 204.

[0015] The named entity recognition module 204 may be configured to extract information of interest (named entities) from the domain-specific documents 202 by performing named entity recognition on the domain-specific documents 202. Examples of named entities in the secure domain include mobile devices and humans. The named entity recognition module 204 includes a regular expression tagger sub-module 220, a dictionary-based tagger sub-module 222, a dictionary extension sub-module 224, and a named entity recognition (NER) coordinator 226. In some embodiments, the named entity recognition module 204 may also include an optional NLP tagger module 228.

[0016] The regular expression tagger submodule 220 can be configured to extract information based on domain-specific patterns without a pre-annotated training dataset. Domain-specific patterns can be naming conventions for entities within a company, such as project numbers, positions, and identification numbers (as in the following example). Regular expressions can be defined for such patterns, so that additional model training using annotated data is not required to identify the patterns. A regular expression is a sequence of characters that specifies a search pattern in text. An example of a domain-specific pattern is "FT-XXX-YYY" for flow meter YYY at location XXX. The regular expressions used by the regular expression tagger submodule 220 can be manually defined by a user and / or a domain expert. The regular expression tagger submodule 220 can receive domain-specific documents 202 directly or via the preprocessing module 210. The output of the regular expression tagger submodule 220 proceeds to the NER coordinator submodule 226.

[0017] The dictionary-based tagger submodule 222 may be configured to identify domain-specific terms within the domain-specific documents 202. The dictionary-based tagger submodule 222 does not require a model training step, which advantageously reduces the amount of data required for training an accurate information extraction framework. The dictionary-based tagger submodule 222 may be populated with domain-specific code for matching domain-specific N-gram phrases. In some embodiments, the dictionary-based tagger submodule 222 may be manually populated by a user and / or a domain expert. The dictionary-based tagger submodule 222 may receive the domain-specific documents 202 via the preprocessing module 210. The output of the dictionary-based tagger submodule 222 proceeds to the NER coordinator submodule 226 and the dictionary expansion submodule 224.

[0018] The dictionary expansion submodule 224 can be configured to enable a user to quickly and accurately expand domain-specific terms (e.g., N-gram terms) in the dictionary-based tagger submodule 222 based on user-specified rules. The initial dictionary established by a subject matter expert may not cover all relevant named entities. For example, it may include "truck" and "operator" but not "forklift" and "employee." The dictionary expansion submodule 224 can identify potential named entities of interest and provide additional named entities to the dictionary-based tagger submodule 222 to improve the resulting accuracy. The dictionary expansion submodule 224 can operate based on given rules, such as "consider nouns according to patterns of mobile equipment and contact means." These noun phrases are likely to be related to humans (in the safety domain, where the named entity of interest is human) because, in a safety report, an incident may be contact between mobile equipment and a human. Such an approach is more effective than some previous approaches that essentially find all noun phrases. The dictionary expansion submodule 224 works with other types of words, phrases, and other types of relationships. The dictionary expansion sub-module 224 may operate in a semi-automatic manner, where words, phrases, or relationships are identified by the sub-module and then optionally confirmed by a user. The dictionary expansion sub-module 224 may receive the domain-specific documents 202 via the preprocessing module 210. The output of the dictionary expansion sub-module 224 proceeds to the NER coordinator sub-module 226 and the dictionary-based tagger sub-module 222. Thus, the dictionary expansion sub-module 224 may function to expand the vocabulary of the dictionary-based tagger sub-module 222, which may improve the rules and identification of words and phrases by the dictionary expansion sub-module 224.

[0019] The NLP tagger submodule 228 is an optional function of the framework. The NLP tagger submodule 228 can use an artificial neural network (ANN) to identify named entities and / or map items in the text of the domain-specific document 202 to named entities and help identify possible relationships between the items in the text and the named entities. Generally, the NLP tagger requires a significant amount of training data to achieve high accuracy. For named entities that have sufficient training data and are not domain-specific (e.g., for the chemical industry), such as places and / or institutions, a pre-trained NLP tagger submodule 228 can be employed. For other domains that have insufficient training data and are domain-specific (e.g., for the chemical industry), the NLP tagger submodule 228 can be omitted or bypassed for the framework. The NLP tagger submodule 228 can receive the domain-specific document 202 via the preprocessing module 210. The output of the NLP tagger submodule 228 proceeds to the NER coordinator submodule 226.

[0020] An ANN can process information by modeling a network of neurons, such as neurons in a human brain, to process information (e.g., stimuli) being sensed in a particular environment. Similar to a human brain, a neural network typically includes multiple neuron topologies (e.g., which may be referred to as artificial neurons). An ANN operation refers to an operation that uses artificial neurons to process inputs to perform a given task. An ANN operation may include implementing various machine learning algorithms to process the inputs.

[0021] Deep learning is a type of machine learning that has been enabled by improved computational power, data availability, and software tools. Deep learning applies ANNs to accomplish tasks previously thought impossible for computers to perform. The "deep" in deep learning refers to the use of multiple layers in an ANN. These layers extract successively higher-level features from raw inputs. Each layer of an ANN can be represented as a column of nodes. The nodes, which may correspond to artificial neurons, can receive various inputs. Interconnection regions can connect nodes between different layers and can be represented by lines connecting the nodes. Nodes can receive inputs from other nodes through interconnection regions. Interconnection regions can connect each node in a first layer to each node in a second layer. ANNs can be configured with a training process in which the various connections in the interconnection regions are assigned weight values ​​or updated with new weight values ​​used in operations or calculations at the nodes. The training process can vary depending on the particular application or use of the ANN. For example, an ANN can be trained for language processing and / or information extraction, as described herein, or another processing or computation task.

[0022] The ANN may include an output layer represented by a final row of nodes. The final row of nodes may be referred to as an output node. Each of the output nodes may be coupled to receive inputs from nodes in a previous layer of nodes. The process of receiving available outputs at the output layer of output nodes as a result of inputs provided to nodes in the first layer may be referred to as inference or forward propagation. That is, input signals representing some real-world phenomenon or application may be provided to the trained ANN, and results may be output through inference occurring as a result of computations enabled by the various nodes and interconnections. In the case of an ANN trained for named entity recognition, the input may be a signal representing text from a domain-specific document 202 (or a preprocessed document), and the output may be a signal representing a named entity.

[0023] The NER coordinator sub-module 226 may be configured to determine the final named entity assignment. Because multiple taggers are employed within the framework (e.g., the regular expression tagger sub-module 220 and the dictionary-based tagger sub-module 222), conflicts between named entities recognized by the taggers may occur. The NER coordinator sub-module 226 may resolve these conflicts by having the final named entity assignment authority for the domain-specific document 202. For example, the NLP tagger sub-module 228 may tag "Dow" as a person, while the dictionary-based tagger 222 may tag "Dow" as an organization. The NER coordinator sub-module 226 may resolve this exemplary conflict by tagging "Dow" as the organization. In some embodiments, the NER coordinator sub-module 226 may resolve conflicts by assigning weights to each tagger (e.g., the dictionary-based tagger sub-module 222 may be assigned a greater weight than the NLP tagger sub-module 228). Among the NLP taggers, higher weights may be assigned to those with better performance on a small test dataset. A calibration dataset may be used to test the accuracy of different taggers, and weights for each may be assigned based on their different levels of accuracy. In general, the dictionary-based tagger submodule 222 and the regular expression tagger submodule 220 may be assigned higher weights than NLP taggers due to their inherently higher reliability based on domain-specific knowledge. The NER coordinator submodule 226 may receive input from the regular expression tagger submodule 220, the dictionary-based tagger submodule 222, and the NLP tagger submodule 228 (if the framework is so equipped). In some applications, the NER coordinator submodule 226 may be configured to provide the extracted information 208 as an output of the framework (e.g., when the extracted information 208 is one or more named entities).In some applications, the NER coordinator sub-module 226 may provide output to the classification module 206 (e.g., when the extracted information 208 is document classification).

[0024] The classification module 206 may be configured to determine whether each of the domain-specific documents 202 is of interest. For example, in the safety domain, the classification module 206 may find documents related to injuries caused by mobile devices. In one such example, the named entity recognition module 204 identifies named entities of humans and mobile devices, and the classification module 206 then identifies information in the domain-specific documents 202 related to the named entities to determine whether the documents are of interest. The classification module 206 includes a relationship-based classifier sub-module 230. In some embodiments, the classification module 206 may also include an NLP classifier sub-module 232 and a classification coordinator sub-module 234.

[0025] The relationship-based classifier submodule 230 can be configured to apply relationships between named entities to classify the domain-specific documents 202. The configuration of the relationship-based classifier submodule 230 can be based on input from experts, so that its classifications do not need to be learned using a training dataset. For example, with respect to the safety domain, the relationship between a mobile device and a human connected by contact can be applied to classify the domain-specific document 202 as safety-related. In this example, the relationship used by the relationship-based classifier submodule 230 is that if the report mentions the named entity in either the sequence "mobile device," "contact means," or "human," or the sequence "human," "contact means," and "mobile device," the report is related to the safety domain. The relationship-based classifier submodule 230 can be accurate without requiring significant or any training data. To the extent that the relationship-based classifier submodule 230 is trained, it is trained at the input document level. The relationship-based classifier submodule 230 is generally a rule-based classifier that does not extrapolate information beyond the relationships specified in its underlying programming. The relationship-based classifier sub-module 230 can receive the output of the named entity recognition module 204 (e.g., named entities) and the domain-specific documents 202. The output of the relationship-based classifier sub-module 230 goes to the classification coordinator sub-module 234 or is output directly as extracted information 208.

[0026] The NLP classifier submodule 232 is an optional feature of the framework. The NLP classifier submodule 232 can be configured to classify natural language inputs by converting them into semantic categories using characteristic vocabulary in the domain-specific documents 202. The use of the NLP classifier submodule 232 requires relatively more data than the operation of the relationship-based classifier submodule 230. However, the NLP classifier submodule 232 can extrapolate from the characteristic vocabulary in the domain-specific documents 202 using an ANN to learn based on previous inputs and results. The integration of the NLP classifier submodule 232 and the relationship-based classifier submodule 230 in the classification module 206 can provide more accurate results than either submodule operating on its own. The NLP classifier submodule 232 can receive the output of the named entity recognition module 204 (e.g., named entities) and the domain-specific documents 202. The output of the NLP classifier submodule 232 proceeds to the classification coordinator submodule 234.

[0027] The classification coordinator submodule 234 is an optional function of the framework that may be used when the classification module 206 includes the use of the NLP classifier submodule 232. The classification coordinator submodule 234 may be configured to determine a final classification for the domain-specific document 202. For embodiments in which multiple classifiers are employed in the framework (e.g., the relationship-based classifier submodule 230 and the NLP classifier submodule 232), conflicts in classifications determined by the classifiers may occur. The classification coordinator submodule 234 can resolve these conflicts by having the final classification authority for the domain-specific document 202. To resolve conflicts, the classification coordinator submodule 234 can assign weights to different classifiers. For example, relationship-based classifications may be assigned a greater weight than NLP classifications. Weights among different NLP classifications can be assigned based on testing the performance results of the NLP classifiers. The classification coordinator submodule 234 may receive input from the relationship-based classifier submodule 230 and the NLP classifier submodule 232. The classification coordinator sub-module 234 may be configured to provide the extracted information 208 as an output of the classification coordinator sub-module 234 and the framework (eg, where the extracted information 208 is a document classification).

[0028] The framework for information extraction can incorporate domain-specific terminology via the regular expression tagger submodule 220, the dictionary-based tagger submodule 222, the dictionary expansion submodule 224, and the relationship-based classifier submodule 230. Pre-trained NLP models can be incorporated into the framework via the NLP tagger submodule 228 and / or the NLP classifier submodule 232 to provide non-domain-specific insights. The NER coordinator submodule 226 and / or the classification coordinator submodule 234 can resolve conflicts from different information extraction methodologies. The framework can automatically extract information with high accuracy using minimal training data sets, as opposed to some previous approaches that require training NLP models for domain-specific tasks. The integration of the domain-specific submodules with the NLP submodule can provide greater accuracy than existing off-the-shelf solutions while using less training data than required by existing solutions. The domain-specific submodules are flexible and highly customizable for different domain-specific tasks, requiring minimal training data to be tailored to those different domain-specific tasks.

[0029] The information extraction framework was tested in two different case studies: the first was mining safety incidents related to human-mobile equipment contact points, and the second was mining reliability incidents in a chemical production business. The use of the framework is not limited to these domains.

[0030] In the case of safety incident information extraction, the goal was to identify all incidents related to human injuries caused by mobile equipment in order to reduce future such incidents. The framework accurately identified 35 of 41 human-mobile equipment contact incidents out of a total of approximately 9,000 incidents in the database, taking 5 minutes to complete the task. Each incident was identified using domain-specific documentation. A commercially available tool accurately identified 21 of 41 target incidents, taking 22 minutes to complete the task. The disclosed framework's superior results are attributed to the customized named entity recognition module 204 and customized classification module 206, which are not included in the commercially available model. The commercially available model documentation required at least 50 incidents of each type to be identified in the training set, which exceeded the amount available in the 9,000 incident dataset. The results identified the mobile equipment most likely to injure workers, thereby reducing what would otherwise be a study of all mobile equipment to focus on those most relevant to the problem. This also eliminated the need for time-consuming manual review of 9,000 domain-specific documentation. Technology improvements, best practices, and potential operating procedure changes for the types of equipment identified through text data mining can be evaluated to protect workers from contact with this equipment and reduce such incidents.

[0031] In the second case study, the goal was to extract equipment failures from reliability documentation to identify and prioritize improvement opportunities. The framework identified a similar amount of named entities (e.g., failure, equipment, etc.) compared to the commercial tool. The framework recognized domain-specific terms and phrases more accurately than the commercial tool.

[0032] FIG. 3 is a block diagram of a framework for named entity recognition. The framework for named entity recognition can receive a domain-specific document 302, such as a new incident report. Named entity recognition can be performed by one or more taggers, such as a dictionary-based tagger 322, similar to the dictionary-based tagger sub-module 222 shown in FIG. 2. The dictionary 336 of the dictionary-based tagger 322 can include definitions of named entities 338 and relationships 340 between the named entities. Non-limiting examples of named entities 338 include mobile device 338-1, contact means 338-2, and person 338-3. A typical representation of the relationships between the named entities is shown at 340 as “entity 1 + entity 2 + entity 3.” After operating on the domain-specific document 302, the dictionary-based tagger 322 can output an indication of tagged entities 342 identified from the domain-specific document 302. As shown at 343, classification can be performed to determine whether the document 302 with the tagged entities is relevant to the desired information.

[0033] Box 344 of Figure 3 shows some example definitions of named entities. For example, the named entity "mobile equipment" includes the definition "truck." The named entity "contact method" includes the definitions "hit" and "strike." The named entity "human" includes "employee" and "EE." Box 344 also shows an example of extracted information and / or document classification: "Mobile Equipment Incident?" The illustrated example results are both "yes," although "no" is also a possible result, although not specifically shown.

[0034] 3 includes an example snippet of text 346 from a domain-specific document 302. The text reads as follows: Person 1 EE was checking the condition of a trailer. While standing on the dock, the EE turned and noticed a forklift coming. The forklift driver was distracted looking down at the scanner while the forklift was moving. The box load struck the EE at 1-2 mph. The EE lost balance and fell onto his back. Both EEs were taken to a medical facility for follow-up testing. RTWNR

[0035] Text 346 has been run through tagger 322 to recognize various named entities. The human named entity category is identified by a single underline as various instances of "EE." The mobile equipment named entity category is identified by a double underline as instances of "trailer," "forklift," and "pack of boxes." The category of means of contact with the named entity is identified by boxing as instances of "contacted." Relationships between named entities are identified as shown in bold font as instances of "pack of boxes contacted EE at 1-2 mph."

[0036] 4 shows an example of the results of information extraction from a domain-specific document. The domain-specific document contains text 446 that reads: The employee was removing trash from the dock by bay number 5. As he was removing trash, the forklift driver exited the trailer onto the forklift after loading a box. At this time, the forklift tire grazed the employee's foot. The employee was taken to Prime Medical where he was diagnosed with a foot contusion. An investigation is underway to verify the employee's statements.

[0037] Various instances of the named entity "human" 448-1 are identified by single underlines as "employee," "he," "driver," and "employee's." Various instances of the named entity "mobile equipment" 448-2 are identified by double underlines as "forklift" and "trailer." An instance of the relationship between the named entity "contact means" 450 is identified as "grazed." Specifically, in line 3 of text 446, contact means 450 "graze" occurs between the named entity "forklift" of mobile equipment 448-2 and the human named entity 448-1 "employee."

[0038] FIG. 5 is a flow diagram illustrating a method for information extraction from domain-specific documents. At block 552, the method may include performing named entity recognition on the domain-specific documents. The named entity recognition may include tagging regular expressions in the domain-specific documents without a pre-annotated training set at block 554. At block 556, the named entity recognition may include expanding defined domain-specific terms with terms and phrases related to the defined domain-specific terms based on rules specified for desired information about the named entities to be extracted. At block 558, the named entity recognition may include tagging the defined and expanded domain-specific terms and phrases in the domain-specific documents. At block 560, the named entity recognition may include resolving named entity recognition conflicts between regular expression tagging and dictionary-based tagging. In some embodiments, the named entity recognition may further include NLP tagging the multiple domain-specific documents. In such embodiments, the conflict resolution at block 560 may further resolve any additional named entity recognition conflicts introduced by the NLP tagging.

[0039] At block 562, the method may include extracting desired information about the named entity. Extracting the desired information may include identifying fault information related to a chemical process. Extracting the desired information may include identifying domain-specific documents related to a safety incident. Other domain-specific examples of extracting the desired information are possible.

[0040] Although not specifically shown in Figure 5, the method may further include preprocessing the domain-specific documents before performing named entity recognition. The domain-specific documents may be preprocessed by sentence segmentation, tokenization, lemmatization, and part-of-speech tagging of the domain-specific documents.

[0041] Although not specifically shown in FIG. 5 , the method may further include performing classification to identify whether each of the plurality of domain-specific documents is relevant to the desired information after resolving the named entity recognition conflicts. Performing classification may include relationally classifying each of the plurality of domain-specific documents based on application of relationships between the named entity recognition results, NLP-processing classifying each of the plurality of domain-specific documents, and resolving classification conflicts between the relational classification and the NLP classification. The NLP classification may be performed by an NLP classifier submodule trained at the document level based on characteristic vocabulary within the domain-specific documents. The relational classification may be performed by a relationship-based classifier submodule that does not require training data.

[0042] The method illustrated in FIG. 5 may be performed by processing logic, which may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, device hardware, integrated circuits, etc.), software (e.g., instructions executing on a processing device), or a combination thereof. In some embodiments, the method is performed by or using the system illustrated in FIG. 6. Although shown in a particular sequence or order, unless otherwise specified, the order of processes may be modified. Therefore, the illustrated embodiment should be understood as merely an example, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. Furthermore, one or more processes may be omitted in various embodiments. Thus, not all processes are required in all embodiments. Other process flows are possible.

[0043] 6 illustrates an example machine 600 upon which a set of instructions may be executed to cause the machine 600 to perform various methodologies discussed herein. The machine 600 may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine 600 may operate as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or client machine in a cloud computing infrastructure or environment, or as a server or client machine in a client-server network environment.

[0044] Machine 600 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web appliance, server, network router, switch, or bridge, or any machine capable of executing a set of instructions (sequentially or otherwise) that specify actions to be taken by the machine. Moreover, although a single machine 600 is illustrated, the term "machine" should also be taken to include any collection of machines that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0045] The exemplary machine 600 includes a processing device 670, a main memory 672 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM), or Rambus DRAM (RDRAM)), a static memory 674 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 676 that communicate with each other via a bus 678.

[0046] Processing device 670 represents one or more general-purpose processing devices, such as a microprocessor, central processing unit (CPU), etc. More specifically, processing device may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other sets of instructions or a combination of sets of instructions. Processing device 670 may also be one or more special-purpose processing devices, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. Processing device 670 is configured to execute instructions 686 to perform the operations and steps discussed herein. Machine 600 may further include a network interface device 1012 for communicating over network 682.

[0047] Data storage system 676 may include a machine-readable storage medium 684 (also known as a computer-readable medium) that stores one or more sets of instructions 686 or software that embody any one or more of the methodologies or functions described herein. Instructions 686 may also reside, completely or at least partially, within main memory 672 and / or within processing device 670 during execution thereof by machine 600, with main memory 672 and processing device 670 also constituting machine-readable storage media.

[0048] In one embodiment, instructions 686 include instructions for implementing functionality corresponding to the framework for information extraction from domain-specific documents described herein. While machine-readable storage medium 684 is shown to be a single medium in one exemplary embodiment, the term "machine-readable storage medium" should be interpreted to include a single medium or multiple media that store one or more sets of instructions. The term "machine-readable storage medium" should also be interpreted to include any medium that can store or encode a set of instructions for execution by a machine, causing the machine to perform any one or more of the methodologies of the present disclosure. Accordingly, the term "machine-readable storage medium" should be interpreted to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0049] As used herein, the singular forms "a," "an," and "the" include both singular and plural referents unless the context clearly dictates otherwise. Furthermore, the word "may" is used throughout this application in its permissive (i.e., potential, can) sense rather than its obligatory (i.e., must) sense. The term "comprises" and its derivatives mean "including, but not limited to." The term "coupled," unless otherwise specified, means directly or indirectly connected, and can include wireless connections.

[0050] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even if only a single embodiment is described with respect to a particular feature. The example features provided in this disclosure are intended to be illustrative rather than limiting, unless otherwise stated. The above description is intended to cover such alternatives, modifications, and equivalents as would be apparent to one skilled in the art having the benefit of this disclosure.

[0051] The scope of the present disclosure includes any feature or combination of features (either explicit or implicit) disclosed herein, or any generalization thereof, whether or not it alleviates any or all of the problems addressed herein. Various advantages of the present disclosure have been described herein, but embodiments may provide some, all, or none of such advantages, or may provide other advantages.

[0052] In the Detailed Description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the disclosed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Accordingly, the following claims are incorporated into the Detailed Description herein, with each claim standing on its own as a separate embodiment.

Claims

1. 1. A computer-implemented method for domain-specific information extraction from documents, comprising: Performing named entity recognition on a plurality of domain-specific documents, tagging, by a regular expression tagger, regular expressions in the plurality of domain-specific documents without a pre-annotated training set for the regular expression tagger; expanding the defined domain-specific terms with terms and phrases related to the defined domain-specific terms based on rules specified for desired information about the named entity to be extracted; tagging the defined domain-specific terms and phrases and the expanded domain-specific terms and phrases in the plurality of domain-specific documents with a dictionary-based tagger; resolving named entity recognition conflicts between the regular expression tagger and the dictionary-based tagger; and extracting the desired information about the named entity.

2. The method of claim 1 , wherein extracting the desired information comprises identifying fault information associated with a chemical process.

3. The method of claim 1 , wherein extracting the desired information comprises identifying those of the plurality of domain-specific documents that are relevant to a safety incident.

4. performing classification to identify whether each of the plurality of domain-specific documents is relevant to the desired information after resolving the named entity recognition conflicts, wherein performing classification includes: Relationally classifying each of the plurality of domain-specific documents based on application of relationships between results of the named entity recognition; natural language processing (NLP) classification of each of the plurality of domain-specific documents; and resolving classification conflicts between the relational classification and the NLP classification.

5. 5. The method of claim 4, wherein NLP classifying comprises NLP classifying with an NLP classifier submodule trained at a document level based on characteristic vocabulary within the plurality of domain-specific documents.

6. The method of claim 4 , wherein classifying the relationships includes classifying the relationships by a relationship-based classifier submodule that does not require training data.

7. performing named entity recognition further comprises NLP tagging the plurality of domain-specific documents; The method of claim 1 , wherein resolving named entity recognition conflicts further comprises resolving conflicts among the NLP tagging, the regular expression tagging, and the dictionary tagging.

8. the plurality of domain-specific documents, Segmenting the plurality of domain-specific documents into sentences; tokenizing the plurality of domain-specific documents; lemmatizing the plurality of domain-specific documents; The method of claim 1 , further comprising preprocessing the plurality of domain-specific documents by:

9. 1. A system for domain-specific information extraction from documents, comprising: A memory, a preprocessing module configured to decompose the domain-specific document into words; a named entity recognition module, a regular expression tagger sub-module configured to tag groups of words as named entities in the domain-specific documents based on domain-specific patterns; a dictionary-based tagger sub-module configured to tag domain-specific terms and phrases in the domain-specific documents as named entities; a dictionary expansion sub-module configured to expand the domain-specific terms in the dictionary-based tagger sub-module based on specified rules; a natural language processing (NLP) tagger sub-module configured to tag named entities within the domain-specific documents; a named entity recognition coordinator configured to resolve named entity assignment conflicts among the regular expression tagger sub-module, the dictionary-based tagger sub-module, and the NLP tagger sub-module; a classification module, a relation-based classifier sub-module configured to apply the relations between the named entities to classify the domain-specific documents; an NLP classifier sub-module configured to classify the domain-specific documents based on characteristic vocabulary within the domain-specific documents; a classification module including a classification coordinator sub-module configured to resolve classification conflicts between the relationship-based classifier sub-module and the NLP classifier sub-module; and a processor, coupled to the memory, configured to execute the modules and sub-modules stored in the memory.

10. 10. The system of claim 9, wherein the processor is optionally configured to execute the named entity recognition module without executing the NLP tagger sub-module.

11. 10. The system of claim 9, wherein the processor is optionally configured to execute the classification module without executing the NLP classifier sub-module and the classification coordinator sub-module.

12. 10. The system of claim 9, wherein the processor is configured to execute the regular expression tagger sub-module without a pre-annotated training dataset.

13. 10. The system of claim 9, wherein the NLP tagger sub-module is pre-trained without domain-specific data.

14. 14. The system of claim 13, wherein the NLP classifier sub-module is trained at the domain-specific document level without a separate domain-specific classification training dataset.

15. The pre-processing module includes: a sentence segmentation submodule configured to segment the text of the domain-specific document into sentences; a tokenization sub-module configured to identify groups of characters in the domain-specific document as words; a lemmatization sub-module configured to group word variants together as a single term in the domain-specific document; and a part-of-speech tagging sub-module configured to mark words in the text of the domain-specific document as corresponding to parts of speech.

Images