A method, system, device and medium for automatically obtaining concepts in a domain knowledge base

Abstract

The present invention discloses a method, system, device and medium for automatically obtaining concepts in a domain knowledge base, including: parsing semi-structured documents based on DOM to obtain keyword information of the documents; crawling Web page data based on the keyword information; dynamically parsing the Web page data, extracting triples, and constructing an ontology; constructing a lifelong machine learning model based on the ontology; and obtaining unnamed concepts based on the lifelong machine learning model and expanding them into the ontology knowledge base in the form of triples. The present invention can effectively integrate the knowledge structure, making it more convenient for users to understand and use. At the same time, when constructing the ontology, it can infer complex logical relationships based on existing concepts and attributes, enabling the program to easily understand the semantic logic. And in the process of continuous learning of the system, it realizes the dynamic acquisition and integration of web information knowledge, continuously expands the domain knowledge base, and timely updates the constructed knowledge concept structure.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge management, and relates to a method, system, device and medium for automatically obtaining concepts in a domain knowledge base. Background Art

[0002] In a knowledge-based society, learning is not only a systematic and standardized activity in educational institutions such as schools, but also a process of problem-solving in the working and living environments. As the main way for individuals to obtain knowledge resources currently, the Internet has a huge amount of information with uneven quality. Therefore, in order to ensure the learning effect in a non-systematic learning environment, certain technical means and authoritative knowledge information resources are needed to establish a corresponding domain knowledge base.

[0003] Knowledge management technology plays a key role in the knowledge-driven process. Knowledge management for individuals or organizations involves various tasks, including creating, sharing, using, and managing knowledge and information. It is a multi-disciplinary field aiming to achieve goals through the optimal utilization of knowledge. Existing knowledge management technologies are diverse, including from designing concepts representing knowledge in a certain domain to developing knowledge management tools and applying knowledge to solve problems. A fundamental element of healthcare knowledge is to collect and integrate healthcare knowledge from various information resources, such as professional medical organizations, healthcare portals, and social media communities related to health. In an e-health environment, knowledge collection is a social cooperation process of integrating and creating a knowledge system, and also a process of using this knowledge to connect relevant resources (such as health services). There are many known IT artifacts that support the creation, storage, and retrieval of knowledge, such as knowledge bases. With the support of the analysis and reasoning capabilities of knowledge bases, knowledge management in healthcare can be used for decision-making and decision support, as well as the ability and willingness to use health technologies.

[0004] A broad application area of Lifelong Learning (LL) is Information Extraction (IE); the goal of IE is to continuously extract and accumulate as much useful information and knowledge as possible. In other words, the extraction process is naturally continuous and cumulative. The previously extracted information can help extract more and higher-quality information later, which matches the goal of LL. The Knowledge Base (KB) of LL will save the extracted information and other useful information for machine reading. Among them, the most famous lifelong information extraction system is NELL, which defines an ontology of a set of target classes and relationships to be learned in the form of a predicate set, a small number of seed training samples for each class and relationship, and a set of constraints coupling various classes and relationships.

[0005] There are four main subsystem components in NELL for performing extraction and learning, namely the Coupled Pattern Learner (CPL), Coupled SEAL (CSEAL), Coupled Morphological Classifier (CMC), and Rule Learner (RuleL). The extractor of the CPL subsystem mainly uses the context pattern in the reading stage to simultaneously extract generic and relational instances from unstructured free Web texts; CSEAL is an extraction and learning system that uses wrapper induction methods to extract facts from semi-structured web pages; the CMC classifier is used to determine whether the candidate facts / confidences extracted by other components belong to their respective genera; RuleL is responsible for the generalization of multi-relations, that is, the learned rules are used to infer new relational instances from other relational instances already in the knowledge base.

[0006] Since NELL is a semi-supervised information extraction system and there are only a small number of labeled training samples for each learning task, it is not sufficient to learn an accurate extractor to extract reliable knowledge. Without reliable knowledge support, NELL cannot sustain for LL because using incorrect knowledge in future learning is very harmful. Therefore, different types of data sources are used to extract different types of relevant knowledge and at the same time to constrain the learning tasks. However, as an autonomous learning agent, there are still certain problems: continuously learning and reading the Web 24 hours a day, with high costs; no self-reflection and clear learning goals, resulting in poor effects for specialized domain learning; relying on network redundant information, so the relationship between consistency and correctness is not unified. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems in the prior art and provide a method, system, device, and medium for automatically obtaining concepts in a domain knowledge base, which can reduce learning costs, clarify learning goals, and improve learning effects.

[0008] To achieve the above object, the present invention adopts the following technical solutions:

[0009] A method for automatically obtaining concepts in a domain knowledge base includes:

[0010] Parsing semi-structured documents based on DOM to obtain keyword information of the documents;

[0011] Crawling Web page data based on the keyword information;

[0012] Dynamically parsing the Web page data, extracting triples, and constructing an ontology;

[0013] Constructing a lifelong machine learning model based on the ontology;

[0014] Obtaining unnamed concepts based on the lifelong machine learning model and expanding them into the ontology knowledge base in the form of triples.

[0015] A further improvement of the present invention lies in that:

[0016] The semi-structured documents include: UMLS, SNOMED-CT, and MeSH;

[0017] Parse the semi-structured documents based on DOM to obtain the keyword information of the documents. Specifically:

[0018] Parse the semi-structured data of UMLS, SNOMED CT, and MeSH through DOM to extract the relationship information of the descriptions, concepts, and terms of the keywords.

[0019] Crawl the Web page data based on the keyword information. Specifically:

[0020] Based on the description information of the keywords parsed by DOM, search for article information limited to the keywords in the Web page, crawl the article information, obtain the title, publication time, and content of the article, create an output file, and write the results to the local es library.

[0021] Dynamically parse the Web page data to extract triples. Specifically:

[0022] Dynamically parse the Web page data based on the entity resolution technology in natural language processing;

[0023] Construct a semantic template that matches the crawled unstructured / semi-structured data, convert the semi-structured data parsed by DOM into structured data through regular expression rules, extract <S, P, O> from the crawled Web page data, and construct the entity and its attribute relationships required for the ontology.

[0024] Construct an ontology. Specifically:

[0025] Use the structured data and the <S, P, O> triples extracted from the Web resources as the data sources for ontology construction;

[0026] Metadata acquisition: Use Jsoup to parse and obtain the HTML text content to obtain metadata; the obtained metadata is the text paragraph information in the Web page;

[0027] Generate the assertion layer: Preprocess the metadata to generate assertions;

[0028] Semantic concept extraction layer: Extract the keyword concepts in the text content;

[0029] Semantic relationship extraction layer: Abstract the triples in the RDF specification from each assertion, and the information content of each part of the triples needs to use the terms or concepts extracted by the semantic concept extraction layer;

[0030] RDF data is a collection of statements, called triples, in the form <S, P, O>, where S represents the subject, P is the predicate, and O refers to the object; each triple describes the relationship between S and O; the set of triples is represented as a directed acyclic graph, where the nodes represent S and O, and the edges represent the predicates that connect S and O.

[0031] A sentence contains triples; where the triples are in the form <S, P, O>, S and O are entities, and P is the relationship between the two entities.

[0032] Ontology language representation layer: Use RDF and OWL languages to permanently store the existing set of triples, that is, store the data source; enable the semantic relationships obtained by the upper layer to be represented by the ontology language, and output the learned knowledge as an ontology file.

[0033] Complete ontology construction based on the Protégé tool, start the Hermit inference engine to infer semantic logical relationships, and obtain semantic logic.

[0034] Based on the ontology, build a lifelong machine learning model, specifically:

[0035] Receive the tasks sent by the task manager.

[0036] Based on the previous task information in the ontology knowledge base, learn the tasks and store them in the ontology knowledge base.

[0037] The N learning tasks executed by the learner and the data sets corresponding to each learning task. When receiving the (N + 1)-th task T N+1 and its corresponding data set D N+1 from the task manager, the learner uses the historical knowledge in the ontology knowledge base to learn T N+1 ; the ontology knowledge base maintains the previously learned knowledge and accumulates knowledge by learning previous tasks; when learning T N+1 is completed, update the ontology knowledge base.

[0038] Based on the lifelong machine learning model, obtain unnamed concepts, specifically: use frequent pattern and association rule mining to discover unknown phenomena in existing concepts.

[0039] The FP-tree algorithm scans the existing concept structure twice to construct an FP-tree and find frequent item sets through it; the process of the FP-growth algorithm is as follows:

[0040] The first scan: Scan the data set to obtain the item set with a frequency of 1, define the minimum support, delete the items with a support less than the minimum support, and then sort the entries in the original data set in descending order according to the item set.

[0041] The second scan creates the header table and the FP tree in descending order from top to bottom;

[0042] For each item, find its conditional pattern base, recursively call the tree structure, and delete the items with a support less than the minimum support; if the final tree structure presents a single path, directly list all combinations; if it is not a single path, continue to call the tree structure until a single path is formed;

[0043] Discover the frequent item sets and association rules among multiple concept entities in the PIS through the frequent item sets, and obtain the unnamed concepts.

[0044] A concept automatic acquisition system in a domain knowledge base, comprising:

[0045] A first acquisition module, which acquires the keyword information of the document based on DOM parsing of the semi-structured document;

[0046] A crawling module, which crawls Web page data based on the keyword information;

[0047] An analysis module, which is used to dynamically analyze the Web page data, extract triples, and construct an ontology;

[0048] A construction module, which constructs a lifelong machine learning model based on the ontology;

[0049] A second acquisition module, which acquires the unnamed concepts based on the lifelong machine learning model and expands them into the ontology knowledge base in the form of triples.

[0050] A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the steps of the above method are implemented.

[0051] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The present invention parses semi-structured documents through DOM to obtain the keyword information of the documents, crawls Web page data according to the keyword information, and performs dynamic parsing to extract triples, thereby effectively integrating the knowledge structure, making it more convenient for users to understand and use. At the same time, the ontology is constructed, and complex logical relationships can be inferred based on existing concepts and attributes, enabling the program to easily understand semantic logic. And in the process of continuous learning of the system, the dynamic acquisition and integration of web information knowledge are realized, the domain knowledge base is continuously expanded, and the constructed knowledge concept structure is also updated in a timely manner.

[0054] According to the lifelong machine learning model, the present invention continuously obtains unnamed unknown concepts, updates the knowledge concept structure in a timely manner, and stores the updated concepts in the ontology knowledge base to enrich the content of the ontology knowledge base. The present invention can help users improve their ability to ask questions and select relevant information, understand the relationships between various entities, reduce the learning cost, clarify the learning objectives, and improve the learning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is a flowchart of the method for automatically obtaining concepts in the domain knowledge base of the present invention;

[0057] Figure 2 It is a schematic structural diagram of the automatic acquisition of concepts in the domain knowledge base of the present invention;

[0058] Figure 3 It is a flowchart of the dynamic acquisition and storage of Web resources of the present invention;

[0059] Figure 4 It is a schematic diagram of the construction of the domain ontology based on the MeSH thesaurus of the present invention;

[0060] Figure 5 It is a fragment diagram of the domain ontology in the medical and health field based on MeSH of the present invention;

[0061] Figure 6 It is a system structure diagram of the system architecture of the lifelong machine learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. Components of the embodiments of the present invention usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0063] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0064] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0065] In the description of the embodiments of the present invention, it should be noted that if terms such as "upper", "lower", "horizontal", "inner", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, or the orientations or positional relationships in which the inventive product is customarily placed during use, it is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention. In addition, terms such as "first", "second", etc. are only used for descriptive distinction and cannot be construed as indicating or implying relative importance.

[0066] In addition, if the term "horizontal" appears, it does not mean that the component is required to be absolutely horizontal, but it can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but it can be slightly inclined.

[0067] In the description of the embodiments of the present invention, it should also be noted that unless otherwise clearly specified and defined, if terms such as "set", "installed", "connected", "connected to" are understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0068] The following further describes the present invention in detail with reference to the accompanying drawings:

[0069] SeeFigure 1 , this invention discloses a method for automatically obtaining concepts in a domain knowledge base, including:

[0070] S101, Parse semi-structured documents based on DOM to obtain keyword information of the documents.

[0071] The semi-structured documents include: UMLS, SNOMED-CT, and MeSH;

[0072] UMLS is the Unified Medical Language System; SNOMED-CT is the Systematized Nomenclature of Medicine-Clinical Terms; MeSH is the Medical Subject Headings;

[0073] Parse semi-structured documents based on DOM to obtain keyword information of the documents, specifically:

[0074] Parse the semi-structured data of UMLS, SNOMED CT, and MeSH through DOM, and extract the relationship information of the descriptions, concepts, and terms of the keywords.

[0075] S102, Crawl Web page data based on the keyword information.

[0076] Search for article information limited to the keywords in the Web based on the description information of the keywords parsed by DOM, crawl the article information, obtain the title, publication time, and content of the article, create an output file, and write the results to the local es library.

[0077] S103, Dynamically parse the Web page data, extract triples, and construct an ontology.

[0078] Construct a semantic template that matches the crawled unstructured / semi-structured data, convert the semi-structured data parsed by DOM into structured data through regular expression rules, extract <S, P, O> from the crawled Web resources, and construct the entity and its attribute relationships required for the ontology.

[0079] Construct an ontology, specifically:

[0080] Use the structured data and the <S, P, O> triples extracted from the Web resources as the data sources for ontology construction;

[0081] Metadata acquisition: Use Jsoup to parse and obtain the HTML text content to obtain metadata; the obtained metadata is the text paragraph information in the web page;

[0082] Assertion Generation Layer: Preprocess the metadata to generate assertions;

[0083] Semantic Concept Extraction Layer: Extract keyword concepts from the text content;

[0084] Semantic Relationship Extraction Layer: Abstract triples in the RDF specification from each assertion. The information content of each part of the triple needs to use the terms or concepts extracted by the Semantic Concept Extraction Layer;

[0085] Ontology Language Representation Layer: Use RDF and OWL languages to permanently store the existing triple set, so that the semantic relationships obtained in the previous layer can be represented by the ontology language, and output the learned knowledge as an ontology file;

[0086] RDF data is a collection of states, called triples, in the form of <S, P, O>. S represents the subject, P is the assertion, and O refers to the object; each triple describes the relationship between S and O; the triple set is represented as a direct canonical graph, where the nodes in the graph represent S and O, and the edges represent assertions, connecting S and O;

[0087] A sentence contains triples; where the triples are in the form of <S, P, O>, S and O are entities, and P is the relationship between the two entities;

[0088] Complete ontology construction based on the Protégé tool, start the Hermit reasoner to infer semantic logical relationships, and obtain semantic logic.

[0089] S104, Based on the ontology, construct a lifelong machine learning model.

[0090] Receive the tasks sent by the task manager;

[0091] Based on the previous task information in the ontology knowledge base, learn the tasks and store them in the ontology knowledge base;

[0092] The N learning tasks executed by the learner and the data sets corresponding to each learning task. When receiving the (N + 1)-th task T N+1 and its corresponding data set D N+1 from the task manager, the learner uses the historical knowledge in the ontology knowledge base to assist in learning T N+1 The ontology knowledge base maintains the previously learned knowledge and accumulates knowledge by learning previous tasks; when the learning of T N+1 is completed, update the ontology knowledge base.

[0093] S105, Based on the lifelong machine learning model, obtain unnamed concepts and expand them into the ontology knowledge base in the form of triples.

[0094] Mining unknown phenomena in existing concepts using frequent patterns and association rules;

[0095] The FP-tree algorithm scans the existing concept structure twice to construct the FP-tree and find the frequent item sets through it. The process of the FP-growth algorithm is as follows:

[0096] In the first scan, the dataset is scanned to obtain the item sets with a frequency of 1. The minimum support is defined, and the items with a frequency less than the minimum support are deleted. Then, the entries in the original dataset are sorted in descending order according to the item sets.

[0097] In the second scan, the item header table and the FP-tree are created in descending order from top to bottom.

[0098] For each item, find its conditional pattern base, recursively call the tree structure, and delete the items with a frequency less than the minimum support. If the final tree structure shows a single path, list all combinations directly; if it is not a single path, continue to call the tree structure until a single path is formed.

[0099] Discover the frequent item sets and association rules among multiple concept entities in the PIS through the frequent item sets, and obtain the unnamed concepts.

[0100] The domain knowledge base includes an ontology knowledge base and Web page data. The Web page data includes the title, publication time, and content of the article.

[0101] See Figure 2 , the present invention discloses a concept automatic acquisition system in a domain knowledge base, including:

[0102] The first acquisition module, which acquires the keyword information of the document based on DOM parsing of the semi-structured document;

[0103] The crawling module, which crawls the Web page data based on the keyword information;

[0104] The parsing module, which is used to dynamically parse the Web page data, extract triples, and construct an ontology;

[0105] The construction module, which constructs a lifelong machine learning model based on the ontology;

[0106] The second acquisition module, which acquires the unnamed concepts based on the lifelong machine learning model and expands them into the ontology knowledge base in the form of triples.

[0107] A method for automatically acquiring concepts in a domain knowledge base, including:

[0108] Step 1: Crawl relevant reports, articles, etc. from authoritative Web sites based on semi-structured concepts such as UMLS, SNOMED CT, and MeSH;

[0109] 1) DOM parsing of semi-structured document files

[0110] Parse semi-structured data such as UMLS, SNOMED CT, and MeSH through DOM, and extract relationship information such as descriptions, concepts, and terms of keywords: Parse the tagged document into a DOM tree, and the content in the tree is encapsulated as node objects. The belonging of the tags is reflected according to the hierarchical relationship of the tags, forming a tree structure. Therefore, the document object formed by DOM parsing the document is called a dom tree, and the tags, text, and even attributes in the tree are called nodes. In this system, a DOM parser is used to convert the XML data format downloaded from Mesh into an object that can be accessed by JavaScript.

[0111] 2) Crawler to crawl Web page data

[0112] See Figure 3 , search for article information limited to these keywords in WebMD through the description information of the keywords parsed from DOM, and store the processed page information in the local es database.

[0113] Step 2: Use technologies such as entity resolution in natural language processing (NLP) to dynamically parse the resources in the Web, extract triples <S, P, O>, and continuously populate the ontology;

[0114] 1) Obtain structured data with the help of regular expressions

[0115] Based on the entity resolution technology in natural language processing (NLP), dynamically parse the Web page data; construct a semantic template that matches the crawled unstructured / semi-structured data, and then convert the semi-structured data parsed by DOM into structured data by writing regular expression rules, and extract the entities and their attribute relationships required for constructing the ontology <S, P, O> from the crawled Web resources;

[0116] 2) Standardized processing for constructing an ontology in the medical and health field

[0117] See Figure 4, the success of domain ontology construction depends on the participation of domain experts and the accuracy and consistency of the description of domain knowledge. Based on this characteristic of domain ontology, various construction approaches and methods have been proposed in the industry, and one of them is to construct ontology based on thesaurus. Because thesaurus is a description of domain concepts based on the knowledge of a discipline, and it is a relatively complete knowledge organization system for the knowledge of a discipline. As the domain knowledge itself changes, the thesaurus continuously adds and deletes subject terms and improves its semantic relationships. From the changes in the terms it collects, the development direction and hot topics of the domain can be discovered, laying a foundation for reusing existing domain knowledge and quickly constructing domain ontology.

[0118] See Figure 5 , the construction process of domain ontology is as follows:

[0119] The structured data obtained and the <S, P, O> triples extracted from Web resources are used as the data sources for ontology construction;

[0120] Use development interfaces related to web crawlers such as Jsoup to multi-threadedly crawl the information of search result pages and perform automatic ontology construction. The specific steps are as follows:

[0121] Metadata acquisition: Use Jsoup to parse and obtain the HTML text content. The metadata obtained is the text paragraph information in the web page. The acquisition of metadata is the first process of ontology construction;

[0122] Generate the assertion layer: After preprocessing the metadata obtained in the first layer, generate assertions;

[0123] Semantic concept extraction layer: In step one, the keywords have been parsed. In step two, the unstructured keyword structure is structured and obtained, and these keyword concepts can be extracted from the text information;

[0124] Semantic relationship extraction layer: This layer abstracts the triples in the RDF specification from each assertion. The information content of each part of the triple needs to use the terms or concepts extracted by the semantic concept extraction layer;

[0125] Ontology language representation layer: This layer uses RDF and OWL languages to permanently store the existing triple set, so that the semantic relationships obtained in the previous layer can be represented by ontology language, and the learned knowledge is output as an ontology file;

[0126] 3) After completing the ontology construction through the Protégé tool, start the Hermit inference engine to infer complex logical relationships, enabling the program to easily understand the semantic logic.

[0127] See Figure 6, Step 3: Through continuous learning, extract the phenomena that have not been named as concepts from the existing phenomena using frequent patterns, and after naming them by domain experts, add them to the initial concept structure, and repeat Step 2 to continue expanding the ontology.

[0128] 1) Learning task

[0129] Through continuous learning, the ability to use the accumulated learned knowledge to assist future learning enables the system to continuously improve its ability to discover new tasks during the process of continuous learning. Since current lifelong learning algorithms can only target specific types of tasks and data, it is necessary to structurally process the concepts before learning. And establish some other sub-components in the KB, such as the Past Information Store (PIS), to store the result models, patterns, or other forms of output information generated by previous learning. For our specific domain KB, manual intervention can be carried out from time to time to decide what knowledge to retain to assist users' future learning.

[0130] Lifelong learning model: Starting from the task manager allocating a new task to the Knowledge-Based Learner (KBL), this task can be given or automatically recognized. Then, the KBL uses the knowledge stored in the Knowledge Base (KB) to generate a model for the user and sends the information or knowledge that needs to be retained to the KB for future use. During the application process, the system can also discover new tasks and learn during work. Some knowledge obtained from the application can also be stored to assist future learning tasks.

[0131] At any point in time in the lifelong learning model, the learner has executed N learning tasks T1, T2,..., T N , and these tasks are also called previous learning tasks, and there are their respective corresponding datasets D1, D2,..., D N . These tasks can be of different types or belong to different domains. When encountering the N + 1th task T N+1 , which is called a new task, and its corresponding dataset D N+1 , the learner can use the historical knowledge in the knowledge base to assist in learning T N+1 . This task can be given or detected by the system itself. The knowledge base maintains the previously learned knowledge and accumulates knowledge by learning previous tasks. When the learning of T N+1 is completed, according to the knowledge learned from T N+1 , update the ontology knowledge base; and expand it to the ontology knowledge base in the form of triples.

[0132] 2) Mine unknown phenomena in existing concepts using frequent patterns and association rules; The FP-tree algorithm scans the existing concept structure twice to construct the FP-tree and find the frequent item sets through it;

[0133] The process of the FP-growth algorithm is as follows:

[0134] In the first scan of the data set, obtain the item sets with a frequent item count of 1, define the minimum support (the minimum number of times an item appears), delete those items with a count less than the minimum support, and then sort the entries in the original data set in descending order according to the item set.

[0135] In the second scan, create the item header table and the FP-tree in descending order from top to bottom.

[0136] For each item in the order from bottom to top, find its conditional pattern base (CPB), recursively call the tree structure, and delete the items with a count less than the minimum support. If the final tree structure presents a single path, directly list all combinations; for non-single-path ones, continue to call the tree structure until a single path is formed.

[0137] Discover the frequent item sets and association rules among multiple concept entities in the PIS through the frequent item sets, so that this phenomenon is named by domain experts, and then new concept names are generated.

[0138] The terminal device provided by an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps in the above-mentioned method embodiments. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-mentioned device embodiments.

[0139] The computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention.

[0140] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0141] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0142] The memory can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory, and by invoking the data stored in the memory, the processor implements various functions of the terminal device.

[0143] If the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0144] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for automatically obtaining concepts in a domain knowledge base, characterized in that, Including: Parse semi-structured documents based on DOM to obtain keyword information of the documents; Crawl Web page data based on the keyword information; Dynamically parse the Web page data, extract triples, and construct an ontology; Construct a lifelong machine learning model based on the ontology; Based on the lifelong machine learning model, obtain unnamed concepts and expand them into the ontology knowledge base in the form of triples; The constructing of the lifelong machine learning model based on the ontology is specifically as follows: Receive the tasks sent by the task manager; Learn the tasks based on the previous task information in the ontology knowledge base and store them in the ontology knowledge base; For the N learning tasks executed by the learner and the data sets corresponding to each learning task, when receiving the (N + 1)-th task T_(N+1) and its corresponding data set D_(N+1) sent by the task manager, the learner uses the historical knowledge in the ontology knowledge base to learn T_(N+1). The ontology knowledge base maintains the previously learned knowledge and accumulates knowledge by learning previous tasks; when the learning of T_(N+1) is completed, the ontology knowledge base is updated; The obtaining of the unnamed concepts based on the lifelong machine learning model is specifically as follows: Use frequent pattern and association rule mining to discover unknown phenomena in existing concepts; The FP-tree algorithm scans the existing concept structure twice to construct an FP-tree and find frequent item sets through it. The process of the FP-growth algorithm is as follows: The first scan: Scan the data set to obtain the item set with a frequent item of 1, define the minimum support, delete the items less than the minimum support, and then sort the entries in the original data set in descending order according to the item set; The second scan: Create an item header table and an FP-tree in descending order from top to bottom; For each item, find its conditional pattern base, recursively call the tree structure, and delete the items less than the minimum support; if the finally presented tree structure is a single path, directly list all combinations; for non-single-path ones, continue to call the tree structure until a single path is formed; Discover the frequent item sets and association rules among multiple concept entities in the PIS through the frequent item sets to obtain unnamed concepts.

2. The method for automatically obtaining concepts in the domain knowledge base according to claim 1, wherein The semi-structured documents include: UMLS, SNOMED-CT, and MeSH; Parse semi-structured documents based on DOM to obtain keyword information, specifically as follows: Parse the semi-structured data of UMLS, SNOMED CT, and MeSH through DOM, and extract the relationship information of the descriptions, concepts, and terms of the keywords.

3. The method for automatically obtaining concepts in the domain knowledge base according to claim 2, characterized in that, The crawling of Web page data based on the keyword information is specifically as follows: Based on the description information of the keywords parsed by DOM, search for article information limited to the keywords in the Web page, crawl the article information, obtain the title, publication time, and content of the article, create an output file, and write the results to the local es library.

4. The method for automatically obtaining concepts in the domain knowledge base according to claim 3, characterized in that, The dynamic parsing of the Web page data to extract triples is specifically as follows: Dynamically parse the Web page data based on the entity parsing technology in natural language processing; Construct a semantic template that matches the crawled unstructured / semi-structured data, convert the semi-structured data parsed by DOM into structured data through regular expression rules, extract <S, P, O> from the crawled Web page data, and construct the entities and their attribute relationships required for the ontology.

5. The method for automatically obtaining concepts in the domain knowledge base according to claim 4, characterized in that, The construction of the ontology is specifically as follows: Use the structured data and the <S, P, O> triples extracted from Web resources as the data sources for ontology construction; Metadata acquisition: Use Jsoup to parse and obtain the HTML text content to obtain metadata; the obtained metadata is the text paragraph information in the web page; Generate the assertion layer: Preprocess the metadata to generate assertions; Semantic concept extraction layer: Extract the keyword concepts in the text content; Semantic relationship extraction layer: Abstract the triples in the RDF specification from each assertion, and the information content of each part of the triples needs to use the terms or concepts extracted by the semantic concept extraction layer; RDF data is a collection of states, called triples, in the form of <S, P, O>. S represents the subject, P is the assertion, and O refers to the object; each triple describes the relationship between S and O; the triple set is represented as a direct canonical graph, where the nodes in the graph represent S and O, and the edges represent assertions, connecting S and O; A sentence contains triples; where the triples are in the form of <S, P, O>, S and O are entities, and P is the relationship between the two entities; Ontology language representation layer: Use RDF and OWL languages to permanently store the existing triple set, that is, store the data source; enable the semantic relationships obtained in the previous layer to be represented by the ontology language, and output the learned knowledge as an ontology file; Complete the ontology construction based on the Protégé tool, start the Hermit inference engine to infer the semantic logical relationships, and obtain the semantic logic.

6. A concept automatic acquisition system in a domain knowledge base, characterized in that, Obtained based on the concept automatic acquisition method in the domain knowledge base according to any one of claims 1-5, including: The first acquisition module, which acquires the keyword information of the document by parsing the semi-structured document based on DOM; The crawling module, which crawls the Web page data based on the keyword information; The parsing module, which is used to dynamically parse the Web page data, extract triples, and construct the ontology; The construction module, which constructs a lifelong machine learning model based on the ontology; The second acquisition module, which acquires the unnamed concepts based on the lifelong machine learning model and expands them into the ontology knowledge base in the form of triples.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-5.

Images