Hybrid intelligence for scalable and effective knowledge discovery and representation

Abstract

Despite their importance, it is recognized herein that current approaches to discovering and developing knowledge representations that include ontologies, semantic data, and knowledge graphs lack efficiencies and capabilities. For example, such discovery and development typically include cumbersome tasks that often require significant manual effort. In an example aspect, a knowledge computing system queries one or more experts or expert systems while generating and updating a knowledge representation.

Description

HYBRID INTELLIGENCE FOR SCALABLE AND EFFECTIVE KNOWLEDGE DISCOVERY AND REPRESENTATIONBACKGROUND

[0001] Ontologies, semantic data, and knowledge graphs are crucial components towards developing machine- interpretable and scalable data representation frameworks. These technologies involve capturing and encoding structured knowledge, and facilitating sophisticated information retrieval, data integration, and inferencing processes. The deployment of these technologies in various industries (e.g., healthcare, smart cities, etc.) has had a significant impact on the way organizations manage, analyze, and utilize data, and has enabled algorithmic interpretation and reasoning over complex data, thereby leading to more effective and intelligent decision-making systems. One such example, the Gene Ontology project, has greatly impacted the biomedical domain by providing a standardized vocabulary for describing gene functions, thereby enabling researchers to perform meta-analyses and uncover novel insights. By way of another example, in the financial sector, the Financial Industry Business Ontology (FIBO) serves as a foundation for data integration, risk analysis, and regulatory compliance. These and many more examples demonstrate that ontologies, semantic data, and knowledge graphs can play an important role in increasing the overall quality of products and services, thereby promoting innovation.

[0002] Despite their importance, it is recognized herein that current approaches to discovering and developing such ontologies, semantic data, and knowledge graphs lack efficiencies and capabilities. For example, such discovery and development typically include cumbersome tasks that often require significant manual effort.SUMMARY

[0003] Methods and systems are disclosed for generating or maintaining a knowledge representation of an industrial system that defines an ontology, semantic data, and a knowledgegraph. In an example aspect, a knowledge computing system queries one or more experts or expert systems while generating the knowledge representation.

[0004] In an example aspect, a knowledge computing system can include a memory having a plurality of application modules stored thereon, and a processor for executing application modules. The application modules can include an Al module configured to perform various operations. The operations include, in some examples, determining a domain associated. Based on the domain, the module can select a set of documents associated with the domain. The Al module can parse the set of documents so as to construct the knowledge representation based on the set of documents. While parsing the documents, the Al module can identify a concept associated with an ambiguity or a conflict. Responsive to identifying the concept, the Al module can send a query to a system or expert associated with the domain. Responsive to the query, the Al module can receive a response that eliminates the ambiguity or conflict. Furthermore, the Al module can define a natural language processor, and the Al module can train or tune the natural language processor with the response.

[0005] In another example aspect, the Al module can, while parsing the set of documents, determine an interpretation of text within the documents, and determine a confidence level associated with the interpretation. The Al module can compare the confidence level to a predetermined threshold, and can make a determination that the confidence level is less than the predetermined threshold. Responsive to the determination, the Al module can prompt an expert of the domain to provide disambiguation input related to the interpretation. The Al module can receive the disambiguation input. Based on the disambiguation input, the Al module can adjust the confidence level associated with the interpretation. Furthermore, the Al module can tune or train the natural language processor based on the disambiguation input. Thus, without being bound by theory, the Al module can selectively prompt or query experts or expert systems, so as to reduce manual effort that is required in constructing high-fidelity knowledge representations.

[0006] BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The foregoing and other aspects of the present invention are best understood from the following detailed description when read in connection with the accompanying drawings. For thepurpose of illustrating the invention, there is shown in the drawings embodiments that are presently preferred, it being understood, however, that the invention is not limited to the specific instrumentalities disclosed. Included in the drawings are the following Figures:

[0008] FIG. 1 is a block diagram of a knowledge computing system in accordance with an example embodiment, wherein the knowledge computing system is configured to generate knowledge representations for a given industrial system.

[0009] FIG. 2 shows an example of a computing environment within which embodiments of this disclosure may be implemented.DETAILED DESCRIPTION

[0010] As an initial matter, it is recognized herein that current approaches to generating knowledge representations (e.g., models) that include ontologies, semantic data, and knowledge graphs for industrial systems do not address various technical problems associated with increasing efficiencies and capabilities. As an example, an ontology for a building automation domain can define or describe how different building applications, data, and devices are related, among other descriptions of knowledge.

[0011] For example, in some cases, the inherent complexity of domain knowledge, the need for expert input, or the heterogeneity of data sources can present challenges in constructing models that define ontologies, semantic data, and knowledge graphs. It is further recognized herein that maintaining these models can define continuous and cumbersome tasks because, for example, knowledge that is represented continuously evolves as new concepts emerge and as human understanding evolves. Thus, in accordance with various embodiments, machine learning and natural language processing are used so as to reduce the reliance on human expertise for the discovery and construction of machine-interpretable and scalable ontologies and knowledge graphs.

[0012] By way of background, current manual approaches to discovering and constructing knowledge representations, for instance knowledge graphs, rely on experts in a specific domain to create a formal and explicit specification of the shared knowledge in their domain of expertise. This can require the definition of concepts, relationships, and constraints to capture a high- fidelity representation of the structure of the domain in question. There are several tools, such as Protege, which facilitate the creation and visualization of knowledge for the development ofontologies. Such labor-intensive manual approaches can be expensive and time-consuming, among other drawbacks. For example, such knowledge representations are often complicated with many elements and many dependencies between the elements, such that various technical challenges need to be addressed to maintain the completeness and correctness of all elements and their respective relationships (e.g., dependencies). Fully automatic approaches can rely entirely on algorithmic knowledge discovery and representation using a combination of text mining and machine learning techniques. It is recognized herein, however, that such approaches often generate incomplete and low-fidelity ontologies. In other examples, there is a reliance on the collective intelligence of a community of users for the development and maintenance of ontologies, knowledge graphs, or semantic data, so as to define a collaborative approach or platform (e.g., DBpedia, Wikidata). Such collaborative approaches, however, often introduce significant coordination challenges and require the consensus of many experts, often with conflicting perspectives, thereby creating a slow and difficult process.

[0013] Referring now to FIG. 1, an example knowledge computing system 100 can be configured to generate ontologies while prompting a human user to assist, so as to define a human in the loop (FUTL) machine learning system. The system 100 can include a neural network or artificial intelligence (Al) module 102 configured to programmatically access various documents 104. The documents 104 can define any existing domain documentation, such as rules, current standards, technical specifications, and the like. By way of example, when generating a knowledge representation in the train or rail domain, the documents 104 can include standards, glossaries, requirements specifications, design documents, and the like that include content and concepts from the rail or train domain. Each document 104 can be scanned by the Al module 102 so that the Al module 102 can extract content from the documents 104. The Al module 102 can identify new content, and can update a knowledge base or representation 108 with the new content. In some cases, the Al module 104 can determine which concepts from the documents 104 are related to each other, and which information is relevant to the given domain, so that the information is added to the knowledge representation.

[0014] The knowledge computing system 100 can further include a user device or expert system 106 and one or more processors and memory having stored thereon applications, agents, and computer program modules including, for example, the Al module 102. It will be appreciated that the program modules, applications, computer-executable instructions, code, orthe like depicted in FIG. 1 are merely illustrative and not exhaustive, and that processing described as being supported by any particular module may alternatively be distributed across multiple modules or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code may be provided to support functionality provided by the program modules, applications, or computer-executable code depicted in FIG. 1 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program modules depicted in FIG. 1 may be performed by a fewer or greater number of modules, or functionality described as being supported by any particular module may be supported, at least in part, by another module. In addition, program modules that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality described as being supported by any of the program modules depicted in FIG. 1 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0015] Still referring to FIG. 1, in various examples, at 101, the Al module 102 can request input, via the user device 106, from a user. As the system 100, for instance the Al module 102, parses the documentation 104 and constructs the knowledge base 108, it may encounter conflicts. For example, different documents might provide information that conflicts with each other (e.g., conflicting definitions), or different terms might be used to refer to the same concept. By way of further example, the Al module 102 might identify a low certainty associated with how concepts are related or connected, or a low certainty associated with whether a given term is relevant to the domain. In these cases that involve a conflict or low certainty, the Al module can determine that there is a conflict or low certainty. Based on such determinations, the Al module 102 can interact with the expert system 106 to receive guidance, rather than performing a best effort, it can resolve its uncertainty by interacting with the user.

[0016] In some cases, the user can define a pool of domain experts. In an example, the input that is requested relates to parsing and extracting a domain ontology. At 103, one or more experts in the domain, via the user device or expert system 106, can provide a list of documents 106 related to the domain ontology. In some examples, experts in the domain can selectivelyprovide an initial list of documents for the Al module 102 to establish a foundational understanding of the domain ontology. Continuing with the example, after parsing the documents, the Al module 102 might identify areas that are missing, unclear, or ambiguous. To address these gaps, the Al module 102 may require further specific inputs, for instance through direct interactions with the user or expert systems, or by requesting the provision of additional documents related to specific aspects in the domain. At 105, the Al module 102 can parse the documents 106 so as to construct the knowledge representation 108 that can define a knowledge graph and ontology. In particular, the Al module 102 can perform natural language processing, entity recognition, relation extraction, and graph construction. A natural language processor (NLP), such as the Stanford NLP, can be specifically trained for task such as named entity recognition (NER) or relation extraction by using annotated data. Performing NER can include performing an information extraction task that classifies named entities into predefined categories. Example categories include, without limitation, person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc. NER can involve document parsing that includes various sub-activities, such as, for example and without limitation: tokenization, part-of-speech tagging, dependency parsing, and entity recognition and classification. Tokenization includes breaking down documents 104 into individual words or tokens. During part-of-speech tagging, each token in a sentence can be tagged with its grammatical role, such as noun, verb, adjective, etc. During dependence parsing, the Al module 102 can identify grammatical relationships between tokens, so as to understand the structure of the sentence. Entity recognition and entity classification can include the system using a combination of rule-based methods, statistical models, and deep learning models, to identify named entities in the text that can be classified into predefined categories. For example, a name can be classified as a person or the name of a country can be classified as a location.

[0017] For example, the list of documents 106 can define domain-specific initial training data for the Al module 102. Such training data can define concepts and relationships that the experts perceive as important for exploration. Based on the Al module 102 parsing the documents 106, the Al module 102 can request validation and disambiguation from the experts (at 101). In some examples, the Al module 102 can define a text-based interface via which the Al module 102 can request confirmation or correction of scenarios having low certainty, conflict, or ambiguity. By way of example, if during parsing the documents 102 the Al module 102encounters to words Georgia and Jordan, the Al module 102 might use the text-based interface to query the expert system to confirm whether the names refer to a country / state (location) or person. At 103, based on the request, the experts, for instance via the user device 104 that is communicatively coupled to the Al module 102, can provide validation and disambiguation. Thus, in some cases, the Al module 102 and a human expert can continuously interact so as to seek each other’s guidance when and where it is most needed over an end-to-end process, rather than at a specific step of the process. The Al module 102 can request human intervention early in the process of generating knowledge representations, so as to address any uncertainties as they arise and perform initial verification to decrease ambiguity and minimize the necessity for later manual correction. Further, based on the request, an expert or the expert system can adjust confidence thresholds.

[0018] Confidence thresholds can be used by the Al module 102 to determine when to seek disambiguation input from users or external sources. Confidence thresholds can be helpful in steering the frequency of such disambiguation requests in order not to overwhelm users. In an example, for each prediction or entity recognition, the NLP of the Al module 102 can compute a confidence score. This score represents the system's certainty in its prediction. The Al module 102 can define a threshold value for the confidence scores, below which the system considers its predictions as uncertain or ambiguous. When the Al module 102 encounters a prediction with a confidence score below the threshold, it can prompt the user or the expert system (or another system) for disambiguation. As the system 100, in particular the Al module 102, receives disambiguation input, it can auto-adjust its thresholds. The thresholds can also be adjusted by a user if they determine that the NLP is making too many errors or asking too often for input. The disambiguation inputs from users can also be used as feedback to retrain or fine-tune the Al module 102, in particular the NLP model, based on context. By way of example, in a sports article, the mention of "Jordan" might have a higher confidence of referring to the basketball player Michael Jordan than the country of Jordan.

[0019] With continuing reference to FIG. 1, at 107, based on the parsing the documents 104 and input received from the experts, the Al module 102 can generate the knowledge representation or model 108 that defines an ontology, semantic data, and a knowledge graph. In various examples, the Al module 102 incorporates the feedback from the experts into the knowledge representation, so as to continuously update and propagate change throughout theknowledge representation 108. In an example, the Al module 102 can generalize feedback and adjust model weights. For example, when the Al module 102 receives feedback, the Al module 102 can process the feedback as new data points that updates the Al module, for instance an NLP model. Thus, the Al module can compute the loss, calculate the gradient of the loss function with respect to each weight in the model, and determine in which direction the weight needs to be adjusted to reduce the loss, etc. In some examples, the feedback defines explicit interactive annotations and labels in the data to provide context information on relevance, correctness, and importance properties of different concepts. Furthermore, 101 and 103 can define validation checkpoints and feedback loops where users can review learning progress and provide refinement suggestions.

[0020] Thus, as described herein, a knowledge computing system can include a memory having a plurality of application modules stored thereon, and a processor for executing application modules. The application modules can include an Al module configured to perform various operations. The operations include, in some examples, determining a domain associated. Based on the domain, the module can select a set of documents associated with the domain. The Al module can parse the set of documents so as to construct the knowledge representation based on the set of documents. While parsing the documents, the Al module can identify a concept associated with an ambiguity or a conflict. Responsive to identifying the concept, the Al module can send a query to a system or expert associated with the domain. Responsive to the query, the Al module can receive a response that eliminates the ambiguity or conflict. Furthermore, the Al module can define a natural language processor, and the Al module can train or tune the natural language processor with the response.

[0021] In another example aspect, the Al module can, while parsing the set of documents, determine an interpretation of text within the documents, and determine a confidence level associated with the interpretation. The Al module can compare the confidence level to a predetermined threshold, and can make a determination that the confidence level is less than the predetermined threshold. Responsive to the determination, the Al module can prompt an expert of the domain to provide disambiguation input related to the interpretation. The Al module can receive the disambiguation input. Based on the disambiguation input, the Al module can adjust the confidence level associated with the interpretation. Furthermore, the Al module can tune or train the natural language processor based on the disambiguation input. Thus, without beingbound by theory, the Al module can selectively prompt or query experts or expert systems, so as to reduce manual effort that is required in constructing high-fidelity knowledge representations.

[0022] FIG. 2 illustrates an example of a computing environment within which embodiments of the present disclosure may be implemented. A computing environment 600 includes a computer system 610 that may include a communication mechanism such as a system bus 621 or other communication mechanism for communicating information within the computer system 610. The computer system 610 further includes one or more processors 620 coupled with the system bus 621 for processing the information. The safety computing system 200 may include, or be coupled to, the one or more processors 620.

[0023] The processors 620 may include one or more central processing units (CPUs), graphical processing units (GPUs), or any other processor known in the art. More generally, a processor as described herein is a device for executing machine-readable instructions stored on a computer readable medium, for performing tasks and may comprise any one or combination of, hardware and firmware. A processor may also comprise memory storing machine-readable instructions executable for performing tasks. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and / or by routing the information to an output device. A processor may use or comprise the capabilities of a computer, controller or microprocessor, for example, and be conditioned using executable instructions to perform special purpose functions not performed by a general purpose computer. A processor may include any type of suitable processing unit including, but not limited to, a central processing unit, a microprocessor, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), a System-on-a-Chip (SoC), a digital signal processor (DSP), and so forth. Further, the processor(s) 620 may have any suitable microarchitecture design that includes any number of constituent components such as, for example, registers, multiplexers, arithmetic logic units, cache controllers for controlling read / write operations to cache memory, branch predictors, or the like. The microarchitecture design of the processor may be capable of supporting any of a variety of instruction sets. A processor may be coupled (electrically and / or as comprising executable components) with any other processor enabling interaction and / or communication there-between. A user interfaceprocessor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.

[0024] The system bus 621 may include at least one of a system bus, a memory bus, an address bus, or a message bus, and may permit exchange of information (e.g., data (including computer-executable code), signaling, etc.) between various components of the computer system 610. The system bus 621 may include, without limitation, a memory bus or a memory controller, a peripheral bus, an accelerated graphics port, and so forth. The system bus 621 may be associated with any suitable bus architecture including, without limitation, an Industry Standard Architecture (ISA), a Micro Channel Architecture (MCA), an Enhanced ISA (EISA), a Video Electronics Standards Association (VESA) architecture, an Accelerated Graphics Port (AGP) architecture, a Peripheral Component Interconnects (PCI) architecture, a PCI-Express architecture, a Personal Computer Memory Card International Association (PCMCIA) architecture, a Universal Serial Bus (USB) architecture, and so forth.

[0025] Continuing with reference to FIG. 2, the computer system 610 may also include a system memory 630 coupled to the system bus 621 for storing information and instructions to be executed by processors 620. The system memory 630 may include computer readable storage media in the form of volatile and / or nonvolatile memory, such as read only memory (ROM) 631 and / or random access memory (RAM) 632. The RAM 632 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The ROM 631 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 630 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 620. A basic input / output system 633 (BIOS) containing the basic routines that help to transfer information between elements within computer system 610, such as during start-up, may be stored in the ROM 631. RAM 632 may contain data and / or program modules that are immediately accessible to and / or presently being operated on by the processors 620. System memory 630 may additionally include, for example, operating system 634, application modules 635, and other program modules 636. Application modules 635 may include aforementioned modules described for FIG. 1 and may also include a user portal for development of the application program, allowing input parameters to be entered and modified as necessary.

[0026] The operating system 634 may be loaded into the memory 630 and may provide an interface between other application software executing on the computer system 610 and hardware resources of the computer system 610. More specifically, the operating system 634 may include a set of computer-executable instructions for managing hardware resources of the computer system 610 and for providing common services to other application programs (e.g., managing memory allocation among various application programs). In certain example embodiments, the operating system 634 may control execution of one or more of the program modules depicted as being stored in the data storage 640. The operating system 634 may include any operating system now known or which may be developed in the future including, but not limited to, any server operating system, any mainframe operating system, or any other proprietary or non-proprietary operating system.

[0027] The computer system 610 may also include a disk / media controller 643 coupled to the system bus 621 to control one or more storage devices for storing information and instructions, such as a magnetic hard disk 641 and / or a removable media drive 642 (e.g., floppy disk drive, compact disc drive, tape drive, flash drive, and / or solid state drive). Storage devices 640 may be added to the computer system 610 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire). Storage devices 641, 642 may be external to the computer system 610.

[0028] The computer system 610 may include a user input interface or graphical user interface (GUI) 661, which may comprise one or more input devices, such as a keyboard, touchscreen, tablet and / or a pointing device, for interacting with a computer user and providing information to the processors 620.

[0029] The computer system 610 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 620 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 630. Such instructions may be read into the system memory 630 from another computer readable medium of storage 640, such as the magnetic hard disk 641 or the removable media drive 642. The magnetic hard disk 641 and / or removable media drive 642 may contain one or more data stores and data files used by embodiments of the present disclosure. The data store 640 may include, but are not limited to, databases (e.g., relational, object-oriented, etc.), file systems, flat files, distributed data stores in which data is stored on more than one node of a computer network,peer-to-peer network data stores, or the like. Data store contents and data files may be encrypted to improve security. The processors 620 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 630. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

[0030] As stated above, the computer system 610 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processors 620 for execution. A computer readable medium may take many forms including, but not limited to, non-transitory, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as magnetic hard disk 641 or removable media drive 642. Non-limiting examples of volatile media include dynamic memory, such as system memory 630. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the system bus 621. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0031] Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0032] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, may be implemented by computer readable medium instructions.

[0033] The computing environment 600 may further include the computer system 610 operating in a networked environment using logical connections to one or more remote computers, such as remote computing device 680. The network interface 670 may enable communication, for example, with other remote devices 680 or systems and / or the storage devices 641, 642 via the network 671. Remote computing device 680 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 610. When used in a networking environment, computer system 610 may include modem 672 for establishing communications over a network 671, such as the Internet. Modem 672 may be connected to system bus 621 via user network interface 670, or via another appropriate mechanism.

[0034] Network 671 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 610 and other computers (e.g., remote computing device 680). The network 671 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-6, or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generallyknown in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 671.

[0035] It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in FIG. 2 as being stored in the system memory 630 are merely illustrative and not exhaustive and that processing described as being supported by any particular module may alternatively be distributed across multiple modules or performed by a different module. In addition, various program module(s), script(s), plug-in(s), Application Programming Interface(s) (API(s)), or any other suitable computer-executable code hosted locally on the computer system 610, the remote device 680, and / or hosted on other computing device(s) accessible via one or more of the network(s) 671, may be provided to support functionality provided by the program modules, applications, or computer-executable code depicted in FIG. 2 and / or additional or alternate functionality. Further, functionality may be modularized differently such that processing described as being supported collectively by the collection of program modules depicted in FIG. 2 may be performed by a fewer or greater number of modules, or functionality described as being supported by any particular module may be supported, at least in part, by another module. In addition, program modules that support the functionality described herein may form part of one or more applications executable across any number of systems or devices in accordance with any suitable computing model such as, for example, a client-server model, a peer-to-peer model, and so forth. In addition, any of the functionality described as being supported by any of the program modules depicted in FIG. 2 may be implemented, at least partially, in hardware and / or firmware across any number of devices.

[0036] It should further be appreciated that the computer system 610 may include alternate and / or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly, it should be appreciated that software, firmware, or hardware components depicted as forming part of the computer system 610 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory 630, it should be appreciated that functionality described as being supported by the program modules may be enabled by any combination of hardware, software, and / or firmware. Itshould further be appreciated that each of the above-mentioned modules may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and / or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other modules. Further, one or more depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and / or additional functionality. Moreover, while certain modules may be depicted and described as sub-modules of another module, in certain embodiments, such modules may be provided as independent modules or as sub-modules of other modules.

[0037] Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and / or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like can be additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”

[0038] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block mayoccur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method of generating a knowledge representation that defines an ontology, semantic data, and a knowledge graph, the method comprising: determining a domain associated with the knowledge representation; based on the domain, selecting a set of documents associated with the domain; parsing the set of documents so as to construct the knowledge representation based on the set of documents.

2. The computer-implemented method as recited in claim 1 , the method further comprising: while parsing the documents, identifying a concept associated with an ambiguity or a conflict.

3. The computer-implemented method as recited in claim 2, the method further comprising: responsive to identifying the concept, sending a query to a system or expert associated with the domain.

4. The computer-implemented method as recited in claim 3, the method further comprising: responsive to the query, receiving a response that eliminates the ambiguity or conflict.

5. The computer-implemented method as recited in claim 4, the method further comprising: training a natural language processor with the response.

6. The computer-implemented method as recited in claim 1, the method further comprising: while parsing the set of documents, determining an interpretation of text within the documents; determining a confidence level associated with the interpretation; andcomparing the confidence level to a predetermined threshold.

7. The computer-implemented method as recited in claim 6, the method further comprising: making a determination that the confidence level is less than the predetermined threshold; and responsive to the determination, prompting an expert of the domain to provide disambiguation input related to the interpretation.

8. The computer-implemented method as recited in claim 7, the method further comprising: based on the disambiguation input, adjusting the confidence level associated with the interpretation.

9. The computer-implemented method as recited in claim 7, the method further comprising: receiving the disambiguation input; and tuning a natural language processor based on the disambiguation input.

10. A knowledge computing system comprising: a memory having a plurality of application modules stored thereon; and a processor for executing the application modules, the application modules comprising an artificial intelligence (Al) module configured to: determine a domain associated with a knowledge representation; based on the domain, select a set of documents associated with the domain; parse the set of documents so as to construct the knowledge representation based on the set of documents, the knowledge representation defining an ontology, semantic data, and a knowledge graph,11. The knowledge computing system as recited in claim 10, the Al module further configured to:while parsing the documents, identify a concept associated with an ambiguity or a conflict; and responsive to identifying the concept, send a query to a system or expert associated with the domain.

12. The knowledge computing system as recited in claim 11, the Al module further configured to: responsive to the query, receive a response that eliminates the ambiguity or conflict.

13. The knowledge computing system as recited in claim 12, wherein the Al module defines a natural language processor, the Al module further configured to: train the natural language processor with the response.

14. The knowledge computing system as recited in claim 10, the Al module further configured to: while parsing the set of documents, determine an interpretation of text within the documents; determine a confidence level associated with the interpretation; compare the confidence level to a predetermined threshold; make a determination that the confidence level is less than the predetermined threshold; and responsive to the determination, prompt an expert to provide disambiguation input related to the interpretation.

15. The knowledge computing system as recited in claim 14, the Al module further configured to: receive the disambiguation input; based on the disambiguation input, adjust the confidence level associated with the interpretation; andtuning a natural language processor based on the disambiguation input.

Images