Extracting information from domain-specific documents

Abstract

Extracting information from a domain-specific document may include performing named entity identification on the domain-specific document. A regular expression marker may mark a regular expression in the domain-specific document without a pre-annotated training set for the regular expression marker. The defined domain-specific terms may be augmented with terms and phrases related to the defined domain-specific terms based on rules specified for desired information about the named entity to be extracted. A dictionary-based marker may mark the defined domain-specific terms and phrases as well as the augmented domain-specific terms and phrases in the plurality of domain-specific documents. Conflicts in named entity recognition between the regular expression marker and the dictionary-based marker may be resolved. The desired information about the named entity may be extracted. Classification may be performed to identify whether the domain-specific document is related to the desired information.

Description

Technical Field

[0001] The present disclosure relates to extracting information from domain-specific documents. Such techniques may be particularly useful for gaining insights and improving productivity, reliability, and security. Background Art

[0002] In some areas (e.g., chemical production), humans often manually review documents to extract information of interest. However, this may be ineffective and insufficient to scale information extraction solutions across different applications across the company. In addition, the information collected and / or extracted may be incomplete because only a subset of the available documents can be examined. For example, the manufacturing department may be assigned the task of mining reliability-related data to identify the common root causes of unplanned events. As another example, the environment, health, and safety department may be assigned the task of investigating the root causes of safety incidents. As yet another example, the supply chain department may be assigned the task of determining which customers are complaining and the reasons for the complaints. The human resources department may be interested in issues that lead to poor employee experience. The documents available to the company may contain relevant information, but due to time and resource constraints, it may not be possible to conduct a sufficiently detailed examination to reveal the relevant information to the company. The data used to draw conclusions may be incomplete.

[0003] Natural language processing (NLP) is a branch of linguistics, computer science, and artificial intelligence that deals with the interaction between computers and human language, and in particular, how to program computers to process and analyze large amounts of natural language data. Information extraction is the task of automatically extracting structured information from unstructured and / or semi-structured machine-readable documents and / or other electronically represented sources. There are commercial and open source NLP and / or information extraction methods, however, training such models tends to be data-intensive and data-specific. For example, most NLP models require a large amount of annotated documents to train the NLP model. In domain-specific contexts, there may not be enough annotated documents or other training material available for the NLP model to perform with the desired accuracy. Domain-specific contexts may use domain-specific terminology that differs from other domains. Summary of the Invention

[0004] The present disclosure relates to extracting information from text-based domain-specific documents, which can help companies gain insights in a wide range of applications. In the field of chemical companies, examples of such applications include identifying the root causes of production losses, safety incidents and customer complaints. Examples of information that may be expected to be extracted include unit operations, chemical properties, human injuries, etc. The present disclosure also relates to identifying documents that include expected information (for example, whether a particular document describes an accident in which a human being was injured by mobile equipment rather than other factors). There are many applications in research and development related to knowledge extraction, context modeling and document classification. The results of information extraction can be used to improve operational reliability, safety and / or customer experience, etc. Many companies may have a large number of documents stored in a database, making manual review of these documents too expensive and ineffective. Currently available information extraction solutions cannot extract the expected information in a domain-specific context with satisfactory accuracy without a large amount of additional training using labeled data.

[0005] To address such issues, at least one embodiment provides a framework for automatic information extraction for domain-specific contexts. Such a framework may include a pre-trained component for extracting information (e.g., location and company name), a self-tuning component that can be trained using a relatively small amount of annotated data sources containing domain-specific terminology, and an insight extraction component that builds upon the collected information. With respect to the annotated data source or training material, "relatively small" means less than recommended or required by existing NLP solutions.

[0006] The above summary of the invention is not intended to describe each disclosed embodiment or every specific implementation. The following description more particularly illustrates exemplary embodiments. In several places throughout this application, guidance is provided by lists of examples, which can be used in various combinations. In each case, the enumerated list serves only as a representative group and should not be construed as an exclusive list. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A block diagram of a framework for information extraction and classification of domain-specific documents.

[0008] Figure 2 is a block diagram of a more detailed framework for information extraction and classification of domain-specific documents.

[0009] Figure 3 is a block diagram of a framework for named entity recognition.

[0010] Figure 4 Example results for extracting information from domain-specific documents are illustrated.

[0011] Figure 5 is a flow chart illustrating a method for extracting information from domain-specific documents.

[0012] Figure 6 An example machine is illustrated in which a set of instructions may be executed, the set of instructions being operable to cause the machine to perform the various methodologies discussed herein. DETAILED DESCRIPTION

[0013] Information extraction involves processing natural language text documents to return interesting structured information. There are various information extraction subtasks, such as named entity recognition, coreference resolution, relation extraction, and knowledge-based reasoning. These tasks can be performed in different ways, including pattern matching methods, dictionary-based methods, machine learning-based methods, and deep learning-based methods. Most existing solutions require large amounts of training data to operate correctly on domain-specific documents. Some domains may not have sufficient training data available for existing models to operate accurately.

[0014] At least one embodiment includes a language processing framework that can extract interesting information with high accuracy and minimal training data compared to available NLP solutions. Multiple language processing methods can be integrated into a single solution to provide the described advantages, such as improved security, reliability, customer experience, and employee experience.

[0015] The figures herein follow a numbering convention in which the first digit or digits correspond to the figure number of the drawing and the remaining digits identify an element or component in the drawing. Similar elements or components between different figures may be identified by using similar numerals. For example, 102 may represent Figure 1 Component "02" in the example, and similar components can be found in Figure 2 202. Similar elements within the figures may be represented with a hyphen and additional numbers or letters. Such similar elements may generally be represented without a hyphen and additional numbers or letters. For example, Figure 1 1 and 108-2 in the drawings may be collectively referred to as 108. As will be appreciated, the elements shown in the various embodiments herein may be added, exchanged, and / or eliminated to provide a plurality of additional embodiments. Furthermore, as will be appreciated, the proportions and relative proportions of the elements provided in the drawings are intended to illustrate certain embodiments of the present invention and should not be construed as limiting.

[0016] Figure 1 1 is a block diagram of a framework for information extraction and classification of domain-specific documents 102. The framework may include domain-specific documents 102, various modules 104, 106, and applications 108. As used herein, a "module" may include program instructions and / or hardware, but at least includes program instructions. In this context, an "application" refers to the purpose of the framework (e.g., information extraction, document classification, etc.).

[0017] As indicated, domain-specific documents 102 are primarily text documents, although they may include graphics, images, or other non-text fields. Domain-specific documents 102 are electronic documents that can be stored in a database. Domain-specific documents 102 contain domain-specific terminology. Existing pre-trained NLP models are unable to recognize domain-specific terminology. It may also be difficult to customize or otherwise modify an NLP model to be able to identify specific information of interest in domain-specific documents 102. In addition, the available datasets from domain-specific documents 102 are too small to accommodate existing NLP models. For example, popular and available NLP models recommend using more than 50 "true" documents per class to train a classification model, where "true" documents are documents that should be classified as interesting by the classification model. Without such training, the NLP model will not accurately or efficiently classify other documents. One example of why there may be insufficient data (or documents) is that a company may not have a large number of accidents, and therefore does not have a large number of accident reports to provide assistance in the field of environmental health and safety. If an NLP model is inaccurate in its document classification, it may not be more useful or efficient than having humans manually perform the review and classification process.

[0018] According to at least one embodiment, the modules of the framework can be used for different applications. For example, the named entity recognition module 104 can be used for both the information extraction application 108-1 and the document classification application 108-2. Figure 1 In the illustrated example, the classification module 106 is used for the classification application 108-2, but not for the information extraction application 108-1. In this example, the document classification application 108-2 may also involve information extraction, but the information extraction application 108-1 does not involve document classification. Examples of entity-level domain-specific information extraction applications 108-1 include identifying chemical properties and identifying unit operations, etc. An example of a domain-specific document classification application 108-2 is to classify a document as being related to certain security events or being related to certain reliability events. For example, the document classification application 108-2 may be run on each of the domain-specific documents in the domain-specific documents 102 to determine whether each of the domain-specific documents is related to a specific security event. About Figure 2 The document classification module 104 and the classification module 106 are described in more detail.

[0019] Figure 22 is a block diagram of a more detailed framework for information extraction and classification of domain-specific documents 202. The framework may include an optional pre-processing module 210. The pre-processing module 210 may be configured to decompose the domain-specific document 202 into words and restore them to corresponding stems for future analysis and modeling. The pre-processing module 210 may include a sentence segmentation submodule 212, a tokenization submodule 214, a lemmatization submodule 216, and a part-of-speech (POS) tagging module 218. The sentence segmentation submodule 212 may be configured to identify where sentences in the domain-specific document 202 begin and end, thereby segmenting the text into sentences. The tokenization submodule 214 may be configured to divide a string of input characters into sections, thereby identifying groups of characters as words. The lemmatization submodule 216 may be configured to group inflected forms of words (e.g., verb conjugations) together so that these inflected forms can be analyzed as a single term. The POS tagging submodule 218 may be configured to label words in the text as corresponding to parts of speech based on definition and context. For example, the POS tagging submodule 218 can be configured to identify words as nouns, verbs, adverbs, adjectives, etc. The pre-processing module 210 can operate on the domain-specific document 202 before passing the results of the operations of the pre-processing module 210 to the named entity recognition module 204 .

[0020] The named entity recognition module 204 can be configured to extract interesting information (named entities) from the domain-specific document 202 by performing named entity recognition on the domain-specific document 202. Examples of named entities in the security domain include mobile devices and humans. The named entity recognition module 204 includes a regular expression tokenizer submodule 220, a dictionary-based tokenizer submodule 222, a dictionary expansion submodule 224, and a named entity recognition (NER) coordinator 226. In some embodiments, the named entity recognition module 204 may also include an optional NLP tokenizer module 228.

[0021] The regular expression tokenizer submodule 220 can be configured to extract information based on domain-specific patterns in the absence of a pre-annotated training dataset. Domain-specific patterns can be naming conventions for entities within a company, such as project numbers, positions, and identification numbers (as in the example below). Regular expressions can be defined for such patterns so that additional model training using annotated data is not required to identify the pattern. A regular expression is a sequence of characters that specifies a search pattern in text. An example of a domain-specific pattern is "FT-XXX-YYY" for flow meter YYY at location XXX. The regular expressions used by the regular expression tokenizer submodule 220 can be manually defined by a user and / or domain expert. The regular expression tokenizer submodule 220 can receive the domain-specific document 202 directly or via the preprocessing module 210. The output of the regular expression tokenizer submodule 220 goes to the NER coordinator submodule 226.

[0022] The dictionary-based tagger submodule 222 can be configured to identify domain-specific terms in the domain-specific documents 202. The dictionary-based tagger submodule 222 does not require a model training step, which advantageously reduces the amount of data required compared to training an accurate information extraction framework. The dictionary-based tagger submodule 222 can be populated with domain-specific codes for matching domain-specific N-gram phrases. In some embodiments, the dictionary-based tagger submodule 222 can be manually populated by a user and / or a domain expert. The dictionary-based tagger submodule 222 can receive the domain-specific documents 202 via the preprocessing module 210. The output of the dictionary-based tagger submodule 222 goes to the NER coordinator submodule 226 and to the dictionary expansion submodule 224.

[0023] The dictionary expansion submodule 224 can be configured to allow the user to quickly and accurately expand the domain-specific terms (e.g., N-gram phrases) in the dictionary-based tagger submodule 222 based on user-specified rules. The initial dictionary established by the subject matter expert may not cover all relevant named entities. For example, the initial dictionary may include "truck" and "operator", but may not include "forklift" and "employee". The dictionary expansion submodule 224 can identify potential named entities of interest and provide additional named entities to the dictionary-based tagger submodule 222 to improve the accuracy achieved. The dictionary expansion submodule 224 can operate based on a given rule, such as "consider nouns that follow the pattern of mobile equipment and contact patterns." These noun phrases (in the security field, where the named entity of interest is a human) may be related to humans because, in a safety report, an incident may be a contact between a mobile equipment and a human. Such an approach is more efficient than some previous approaches that find substantially all noun phrases. The dictionary expansion submodule 224 works with other types of words, phrases, and other types of relationships. The dictionary expansion submodule 224 can operate in a semi-automatic manner, in which words, phrases, or relationships can be identified by the submodule and then optionally confirmed by a user. The dictionary expansion submodule 224 can receive the domain-specific document 202 via the pre-processing module 210. The output of the dictionary expansion submodule 224 goes to the NER coordinator submodule 226 and to the dictionary-based tokenizer submodule 222. Thus, the dictionary expansion submodule 224 can be used to expand the dictionary of the dictionary-based tokenizer submodule 222, and the dictionary-based tokenizer submodule 222 can improve the rules and the identification of phrases by the dictionary expansion submodule 224.

[0024] NLP marker submodule 228 is an optional feature of the framework. NLP marker submodule 228 can use artificial neural network (ANN) to help identify named entities and / or map the project in the text of domain-specific document 202 to named entities and what relationship may be between the project in the text and the named entity. Generally speaking, NLP markers need a large amount of training data to achieve high accuracy. For named entities (such as position and / or mechanism) with enough training data and not domain-specific (for example, for chemical industry), pre-trained NLP marker submodule 228 can be adopted. For other fields with insufficient training data and domain-specific (for example, for chemical industry), NLP marker submodule 228 can be omitted or bypassed for framework. NLP marker submodule 228 can receive domain-specific document 202 via pre-processing module 210. The output of NLP marker submodule 228 goes to NER coordinator submodule 226.

[0025] ANNs can process information by modeling a network of neurons (such as those in the human brain) to process information (e.g., stimuli) sensed in a specific environment. Similar to the human brain, neural networks typically include a multi-neuron topology (e.g., which can be referred to as artificial neurons). ANN operations refer to operations that use artificial neurons to process inputs to perform a given task. ANN operations can involve executing various machine learning algorithms to process inputs.

[0026] Deep learning is a type of machine learning that has been enabled by improvements in computing power, data availability, and software tools. Deep learning can apply artificial neural networks (ANNs) to accomplish tasks once thought impossible for computers to perform. The "depth" in deep learning refers to the use of multiple layers in the ANN. These layers successively extract higher-order features from the raw input. Each layer of the ANN can be represented as a column of nodes. Nodes, which can correspond to artificial neurons, can receive various inputs. Interconnected regions can couple nodes between different layers, as indicated by the lines coupling the nodes. Nodes can receive input from other nodes via the interconnected regions. An interconnected region can couple each node in the first layer to each node in the second layer. The ANN can be configured during a training process, where various connections in the interconnected regions are assigned weight values or updated with new weight values for operations or calculations at the nodes. The training process can vary depending on the specific application or use of the ANN. For example, the ANN can be trained for language processing and / or information extraction, as described herein, or for another processing or computational task.

[0027] The ANN may include an output layer represented by a last column of nodes. The last column of nodes may be referred to as output nodes. Each of the output nodes may be coupled to receive input from a node in a previous layer of node layers. The process of receiving a usable output at the output layer of the output nodes as a result of the input fed to the nodes of the first layer may be referred to as inference or forward propagation. That is, an input signal representing some real-world phenomenon or application may be fed into the trained ANN, and the result may be output through inference that occurs as a result of calculations enabled by the various nodes and interconnections. In the case of an ANN trained for named entity recognition, the input may be a signal representing the text of a domain-specific document 202 (or pre-processed document), and the output may be a signal representing a named entity.

[0028] The NER coordinator submodule 226 can be configured to determine the final named entity assignment. Because multiple taggers (e.g., regular expression tagger submodule 220 and dictionary-based tagger submodule 222) are used in the framework, conflicts of named entities identified by the taggers may occur. The NER coordinator submodule 226 can resolve these conflicts by having the final named entity assignment authority for the domain-specific document 202. For example, the NLP tagger submodule 228 can mark "Dow" as a person, while the dictionary-based tagger 222 can mark "Dow" as an organization. The NER coordinator submodule 226 can resolve this example conflict by marking "Dow" as an organization. In some embodiments, the NER coordinator submodule 226 can resolve the conflict by assigning weights to each tagger (e.g., the dictionary-based tagger submodule 222 may be assigned a greater weight than the NLP tagger submodule 228). Among the NLP taggers, higher weights can be assigned to those taggers with better performance in a small test data set. The calibration dataset can be used to check the accuracy of different taggers, and corresponding weights can be assigned based on those different accuracy levels. Generally speaking, the dictionary-based tagger submodule 222 and the regular expression tagger submodule 220 can be assigned a larger weight than the NLP tagger due to their inherent greater reliability based on domain-specific knowledge. The NER coordinator submodule 226 can receive input from the regular expression tagger submodule 220, the dictionary-based tagger submodule 222, and the NLP tagger submodule 228 (if the framework is so equipped). For some applications, the NER coordinator submodule 226 can be configured to provide the extracted information 208 as an output of the framework (e.g., wherein the extracted information 208 is one or more named entities). For some applications, the NER coordinator submodule 226 can provide an output to the classification module 206 (e.g., wherein the extracted information 208 is a document classification).

[0029] The classification module 206 can be configured to determine whether each domain-specific document in the domain-specific documents 202 is of interest. For example, in the security domain, the classification module 206 can find documents related to injuries caused by mobile equipment. In such an example, the named entity recognition module 204 will identify the named entities of humans and mobile equipment, and then the classification module 206 will identify information in the domain-specific documents 202 related to the named entities to determine whether the document is of interest. The classification module 206 includes a relationship-based classifier submodule 230. In some embodiments, the classification module 206 may also include an NLP classifier submodule 232 and a classification coordinator submodule 234.

[0030] The relationship-based classifier submodule 230 can be configured to apply relationships between named entities to classify domain-specific documents 202. The configuration of the relationship-based classifier submodule 230 can be based on input from experts, such that its classification does not require learning from a training dataset. For example, with respect to the security domain, a relationship between mobile equipment and people connected by contact can be applied to classify the domain-specific document 202 as security-related. In this example, the relationship used by the relationship-based classifier submodule 230 is: if a report mentions a named entity in any of the following sequences: "mobile equipment," "contact method," "human," or "human," "contact method," "mobile equipment," then the report is security-related. The relationship-based classifier submodule 230 can be accurate without requiring a large amount of or any training data. To the extent that the relationship-based classifier submodule 230 is trained, it is trained at the input document level. The relationship-based classifier submodule 230 is a rule-based classifier that generally does not extrapolate information beyond the relationships specified in its underlying programming. The relation-based classifier submodule 230 may receive the output of the named entity recognition module 204 (eg, named entities) and the domain-specific documents 202. The output of the relation-based classifier submodule 230 goes to the classification coordinator submodule 234 or is directly output as the extracted information 208.

[0031] The NLP classifier submodule 232 is an optional feature of the framework. The NLP classifier submodule 232 can be configured to use the feature vocabulary in the domain-specific document 202 to classify these feature vocabulary by converting the natural language input into semantic categories. The use of the NLP classifier submodule 232 is relatively more data-intensive than the operation of the relationship-based classifier submodule 230. However, the NLP classifier submodule 232 can use ANN to learn based on previous inputs and results and extrapolate from the feature vocabulary in the domain-specific document 202. The integration of the NLP classifier submodule 232 and the relationship-based classifier submodule 230 in the classification module 206 can provide more accurate results than any submodule operating independently. The NLP classifier submodule 232 can receive the output of the named entity recognition module 204 (e.g., named entities) and the domain-specific document 202. The output of the NLP classifier submodule 232 goes to the classification coordinator submodule 234.

[0032] The classification coordinator submodule 234 is an optional feature of the framework that can be used when the classification module 206 includes the use of the NLP classifier submodule 232. The classification coordinator submodule 234 can be configured to determine the final classification of the domain-specific document 202. For embodiments in which multiple classifiers are employed in the framework (e.g., the relationship-based classifier submodule 230 and the NLP classifier submodule 232), conflicts in the classifications determined by the classifiers may arise. The classification coordinator submodule 234 can resolve these conflicts by having final classification authority for the domain-specific document 202. The classification coordinator submodule 234 can assign weights to different classifiers to resolve conflicts. For example, the relationship-based classification can be assigned a greater weight than the NLP classification. Among the different NLP classifications, weights can be assigned based on test performance results of the NLP classifiers. The classification coordinator submodule 234 can receive input from the relationship-based classifier submodule 230 and the NLP classifier submodule 232. The classification coordinator submodule 234 can be configured to provide the extracted information 208 as an output of the classification coordinator submodule 234 and an output of the framework (eg, where the extracted information 208 is a document classification).

[0033] The framework for information extraction can incorporate domain-specific terminology via a regular expression tokenizer submodule 220, a dictionary-based tokenizer submodule 222, a dictionary expansion submodule 224, and a relationship-based classifier submodule 230. Pre-trained NLP models can be incorporated into the framework via an NLP tokenizer submodule 228 and / or an NLP classifier submodule 232 to provide non-domain-specific insights. An NER coordinator submodule 226 and / or a classification coordinator submodule 234 can resolve conflicts between different information extraction methods. Compared to some previous approaches that would otherwise require training NLP models for domain-specific tasks, the framework can automatically extract information with high accuracy using minimal training data sets. The integration of domain-specific submodules with the NLP submodule can provide higher accuracy than existing commercially available solutions while using less training data than existing solutions. The domain-specific submodules are flexible and highly customizable for different domain-specific tasks and require minimal training data to adjust for those different domain-specific tasks.

[0034] The information extraction framework was tested in two different case studies. The first case study focused on mining safety incidents related to how people and mobile equipment interact. The second case study focused on mining reliability incidents in a chemical production operation. The framework's use is not limited to these domains.

[0035] In the case of safety incident information extraction, the goal is to identify all incidents associated with human injuries caused by mobile equipment in order to reduce such incidents in the future. The framework correctly identified 35 of 41 types of human-mobile equipment contact modes in a database of approximately 9,000 total incidents, taking 5 minutes to complete the task. Each incident was identified by domain-specific documentation. A commercially available tool correctly identified 21 of the 41 target incidents, taking 22 minutes to complete the task. The superior results of the disclosed framework are attributed to a custom named entity recognition module 204 and a custom classification module 206, which are not included in the commercially available model. The commercially available model documentation requires that at least 50 incidents of each type be identified in the training set, which exceeds the amount available in the 9,000-incident dataset. The results identify the mobile equipment most likely to injure workers, thereby reducing the research that would otherwise be conducted on all mobile equipment to focus on those most relevant to the problem. This also avoids the need to manually review 9,000 domain-specific documents, which would otherwise be time-consuming. Technology improvements, best practices, and potential operational procedure changes for the types of equipment identified through text data mining can be evaluated to protect workers from means of contact with that equipment and reduce such incidents.

[0036] In the second case study, the goal was to extract equipment failures from reliability documentation to reveal and prioritize improvement opportunities. Compared to commercially available tools, the framework identified a similar number of named entities (e.g., failure, equipment, etc.). The framework also identified domain-specific terms and phrases more accurately than commercially available tools.

[0037] Figure 3 3 is a block diagram of a framework for named entity recognition. The framework for named entity recognition may receive a domain-specific document 302, such as a new accident report. Named entity recognition may be performed by one or more taggers, such as a dictionary-based tagger 322, similar to a Figure 2 3. The dictionary-based tagger 322 may be executed as illustrated by the dictionary-based tagger submodule 222. The dictionary 336 of the dictionary-based tagger 322 may include definitions of named entities 338 and relationships 340 between named entities. Non-limiting examples of named entities 338 include mobile equipment 338-1, contact mode 338-2, and human 338-3. A general representation of relationships between named entities is illustrated at 340 as "entity 1 + entity 2 + entity 3". After operating on the domain-specific document 302, the dictionary-based tagger 322 may output an indication of tagged entities 342 identified from the domain-specific document 302. Classification may be performed to determine whether the document 302 with the tagged entities is relevant to the desired information, as illustrated at 343.

[0038] Figure 3 Box 344 in the example illustrates some examples of named entity definitions. For example, the named entity "mobile equipment" includes the definition "truck." The named entity "contact mode" includes the definitions "hit" and "hit by." The named entity "human" includes "employee" and "EE." Box 344 also illustrates an example of extracted information and / or document classification: "Did a mobile equipment accident occur?" The example results illustrated are all "yes," but, although not specifically illustrated, "no" is also a possible result.

[0039] Figure 3 An example snippet of text 346 from domain-specific document 302 is included. The text reads as follows:

[0040] Personnel 1, EE, was verifying the status of a trailer. While standing on the dock, EE turned and noticed a forklift approaching. While the forklift was moving, the forklift driver became distracted while looking down at a scanner. The boxed load struck EE at a speed of 1 to 2 mph. EE lost his balance and fell on his back. Both EEs were taken to the medical department for post-incident testing.

[0041] RTWNR

[0042] Text 346 has been processed by tagger 322, and various named entities have been identified. The category of human named entities has been identified as various instances of "EE" by single underlining. The category of mobile equipment named entities has been identified as instances of "trailer," "forklift," and "boxed load" by double underlining. The category of contact mode named entities has been identified as instances of "contacted" by framing. Relationships between named entities have been identified as instances shown in bold font as "boxed load contacts EE at a speed of 1 mph to 2 mph."

[0043] Figure 4 The example result of extracting information from a domain-specific document is shown. The domain-specific document includes text 446, the content of which is as follows:

[0044] An employee was removing trash from the dock at Berth 5. While he was removing the trash, a forklift driver was removing a trailer from the forklift after loading boxes. The forklift tire bruised the employee's foot. The employee was taken to Prime Medical Center and diagnosed with a foot contusion. An investigation is ongoing to verify the employee's statement.

[0045] Various instances of the named entity "human" 448-1 have been identified by single underscores as "employee," "he," "driver," and "employee's." Various instances of the named entity "mobile equipment" 448-2 have been identified by double underscores as "forklift" and "trailer." An instance of the relationship between the named entities "contact pattern" 450 is identified as "scratched." Specifically, the contact pattern 450 "scratched" occurs between the mobile equipment 448-2 named entity "forklift" and the human named entity 448-1 "employee" in line 3 of the text 446.

[0046] Figure 5is a flow chart illustrating a method for extracting information from domain-specific documents. At block 552, the method may include performing named entity recognition on the domain-specific documents. Named entity recognition may include, at block 554, tagging regular expressions in the domain-specific documents without a pre-annotated training set. At block 556, named entity recognition may include augmenting defined domain-specific terms with terms and phrases related to the defined domain-specific terms based on rules specified for the desired information about the named entities to be extracted. At block 558, named entity recognition may include tagging the defined domain-specific terms and phrases and the augmented domain-specific terms and phrases in the domain-specific documents. At block 560, named entity recognition may include resolving conflicts in named entity recognition between regular expression tagging and dictionary-based tagging. In some embodiments, named entity recognition may also include performing NLP tagging on multiple domain-specific documents. In such embodiments, the conflict resolution in block 560 may further resolve any additional conflicts in named entity recognition introduced by the NLP tagging.

[0047] At block 562, the method may include extracting desired information about the named entity. Extracting the desired information may include identifying failure information related to the chemical process. Extracting the desired information may include identifying domain-specific documents related to safety incidents. Other domain-specific examples of extracting the desired information are possible.

[0048] Despite Figure 5 Although not specifically illustrated in the embodiment, the method may further include: pre-processing the domain-specific document before performing named entity recognition. The domain-specific document may be pre-processed by performing sentence segmentation, tokenization, lemmatization, and part-of-speech tagging on the domain-specific document.

[0049] Despite Figure 5 Although not specifically illustrated in the text, the method may further include: performing classification after resolving conflicts in named entity recognition to identify whether each of the multiple domain-specific documents is relevant to the desired information. Performing classification may include: performing relational classification on each of the multiple domain-specific documents based on the application of relationships between the results of named entity recognition; performing NLP processing classification on each of the multiple domain-specific documents; and resolving classification conflicts between relational classification and NLP classification. The NLP classification may be performed by an NLP classifier submodule trained at the document level based on feature vocabulary in the domain-specific documents. The relational classification may be performed by a relation-based classifier submodule that does not require training data.

[0050] Figure 5The illustrated methods may be performed by processing logic that may include hardware (e.g., a processing device, circuitry, dedicated logic, programmable logic, microcode, hardware of a device, integrated circuit, etc.), software (e.g., instructions run or executed on a processing device), or a combination thereof. In some embodiments, the methods are performed by Figure 6 The illustrated system is performed or performed using the system. Although shown in a particular order or sequence, the order of the processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be performed in a different order, and some processes may be performed in parallel. Additionally, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are possible.

[0051] Figure 6 The example machine 600 is illustrated as an example machine within which a set of instructions, for performing the various methodologies discussed herein, may be executed. The machine 600 may be connected (e.g., using a network) to other machines in a LAN, an intranet, an extranet, and / or the Internet. The machine 600 may operate in the capacity of a server or a client machine in server-client network environment, as a peer machine in a peer-to-peer (or distributed) network environment, or as a server or a client machine in a cloud computing infrastructure or environment.

[0052] The machine 600 may be a personal computer (PC), a tablet PC, a digital video set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network appliance, a server, a network router, a switch or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying actions to be taken by the machine. Furthermore, while a single machine 600 is illustrated, the term "machine" shall also be construed to include any collection of machines that individually or jointly execute one (or more) sets of instructions to perform any one or more of the methodologies discussed herein.

[0053] The example machine 600 includes a processing device 670, a main memory 672 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM)), etc.), a static memory 674 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 676, which communicate with each other via a bus 678.

[0054] The processing device 670 represents one or more general-purpose processing devices, such as a microprocessor or a central processing unit (CPU). More specifically, the processing device can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing device 670 can also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor. The processing device 670 is configured to execute instructions 686 to perform the operations and steps discussed herein. The machine 600 can also include a network interface device 1012 for communicating via the network 682.

[0055] The data storage system 676 may include a machine-readable storage medium 684 (also referred to as a computer-readable medium) having stored thereon one or more sets of instructions 686 or software embodying any one or more of the methodologies or functions described herein. The instructions 686 may also reside, completely or at least partially, within the main memory 672 and / or within the processing device 670 during execution by the machine 600, the main memory 672 and the processing device 670 also constituting machine-readable storage media.

[0056] In one embodiment, instructions 686 include instructions for implementing functionality corresponding to the framework described herein for extracting information from domain-specific documents. Although machine-readable storage medium 684 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be taken to include a single medium or multiple media that store one or more instruction sets. The term "machine-readable storage medium" should also be taken to include any medium that is capable of storing or encoding an instruction set that is executed by a machine and causes the machine to perform any one or more methods of the present disclosure. Therefore, the term "machine-readable storage medium" should be taken to include, but is not limited to, solid-state memories, optical media, and magnetic media.

[0057] As used herein, the singular forms "a", "an", and "the" include both singular and plural referents unless the context clearly dictates otherwise. Furthermore, throughout this application, the word "may" is used in a permissive sense (i.e., likely to, able to), rather than in a mandatory sense (i.e., must). The term "include" and its derivatives mean "including but not limited to". The term "coupled" means connected directly or indirectly, and may include wireless connections unless otherwise specified.

[0058] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to a particular feature. Unless otherwise stated, the examples of features provided in this disclosure are intended to be illustrative and not restrictive. The above description is intended to encompass such alternatives, modifications, and equivalents as would be apparent to one skilled in the art having the benefit of this disclosure.

[0059] The scope of the present disclosure includes any feature or combination of features disclosed herein (explicitly or implicitly), or any generalization thereof, whether or not it mitigates any or all of the problems addressed herein. Various advantages of the present disclosure have been described herein, but embodiments may provide some, all, or none of these advantages, or may provide other advantages.

[0060] In the foregoing detailed description, some features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the disclosed embodiments must utilize more features than are expressly recited in each claim. Rather, as the following claims reflect, the inventive subject matter does not lie in all features of a single disclosed embodiment. Therefore, the following claims are hereby incorporated into the detailed description, with each claim standing on its own as a separate embodiment.

Claims

1. A computer-implemented method for extracting information from a domain-specific document, the computer-implemented method comprising: Perform named entity recognition on multiple domain-specific documents by: tokenizing, by a regular expression tokenizer, regular expressions in the plurality of domain-specific documents without a pre-annotated training set for the regular expression tokenizer; augmenting the defined domain-specific terms with terms and phrases related to the defined domain-specific terms based on rules specified for desired information about the named entities to be extracted; tagging the defined domain-specific terms and phrases and the expanded domain-specific terms and phrases in the plurality of domain-specific documents by a dictionary-based tagger; as well as Resolving conflicts in named entity recognition between the regular expression tokenizer and the dictionary-based tokenizer; as well as The desired information about the named entity is extracted.

2. The method according to claim 1, wherein extracting the desired information comprises: Identifies fault information related to chemical processes.

3. The method according to claim 1, wherein extracting the desired information comprises: Those domain-specific documents of the plurality of domain-specific documents that are relevant to the security incident are identified.

4. The method according to claim 1, further comprising: After resolving the conflicts in the named entity recognition, performing classification to identify whether each of the plurality of domain-specific documents is relevant to the desired information, wherein performing classification comprises: performing relationship classification on each of the plurality of domain-specific documents based on application of the relationship between the results of the named entity recognition; performing natural language processing (NLP) classification on each of the plurality of domain-specific documents; and Resolving classification conflicts between the relational classification and the NLP classification. 5 . The method of claim 4 , wherein the NLP classification comprises NLP classification performed by an NLP classifier submodule trained at a document level based on feature vocabularies in the plurality of domain-specific documents. The method of claim 4 , wherein the relationship classification comprises relationship classification by a relationship-based classifier submodule that does not require training data.

7. The method of claim 1 , wherein performing named entity recognition further comprises: performing NLP tagging on the plurality of domain-specific documents; and Wherein resolving conflicts in named entity recognition further comprises: resolving conflicts among the NLP tags, the regular expression tags and the dictionary tags.

8. The method of claim 1 , further comprising pre-processing the plurality of domain-specific documents by: Performing sentence segmentation on the multiple domain-specific documents; tokenizing the plurality of domain-specific documents; performing lemmatization on the plurality of domain-specific documents; as well as Part-of-speech tagging is performed on the plurality of domain-specific documents.

9. A system for extracting information from domain-specific documents, the system comprising: a memory storing: a preprocessing module configured to decompose the domain-specific document into words; A named entity recognition module, the named entity recognition module comprising: a regular expression tokenizer submodule configured to tokenize word groups as named entities in the domain-specific document based on a domain-specific pattern; a dictionary-based tokenizer submodule configured to tokenize domain-specific terms and phrases as named entities in the domain-specific documents; a dictionary expansion submodule configured to expand the domain-specific terms in the dictionary-based tokenizer submodule based on specified rules; a natural language processing (NLP) tagger submodule configured to tag named entities in the domain-specific document; a named entity recognition coordinator configured to resolve conflicts in named entity assignments between the regular expression tokenizer submodule, the dictionary-based tokenizer submodule, and the NLP tokenizer submodule; A classification module, the classification module comprising: a relation-based classifier submodule, the relation-based classifier submodule being configured to apply the relations between the named entities to classify the domain-specific documents; an NLP classifier submodule configured to classify the domain-specific document based on characteristic vocabulary in the domain-specific document; and a classification coordinator submodule configured to resolve classification conflicts between the relationship-based classifier submodule and the NLP classifier submodule; and A processor is coupled to the memory and configured to execute the modules and sub-modules stored in the memory.

10. The system of claim 9, wherein the processor is configured to optionally execute the named entity recognition module without executing the NLP tagger submodule.

11. The system of claim 9, wherein the processor is configured to optionally execute the classification module without executing the NLP classifier submodule and the classification coordinator submodule.

12. The system of claim 9, wherein the processor is configured to execute the regular expression tokenizer submodule without a pre-annotated training dataset.

13. The system of claim 9, wherein the NLP tagger submodule is pre-trained without domain-specific data.

14. The system of claim 13, wherein the NLP classifier submodule is trained at a domain-specific document level without a separate domain-specific classification training dataset.

15. The system of claim 9, wherein the pre-processing module comprises: a sentence segmentation submodule configured to segment the text of the domain-specific document into sentences; a tokenization submodule configured to identify character groups in the domain-specific document as words; a lemmatization submodule configured to group inflected forms of words together into a single term in the domain-specific document; and A part-of-speech tagging submodule is configured to tag words in the text of the domain-specific document as corresponding to parts of speech.

Images