Intelligent semantic analysis knowledge graph system for multi-source heterogeneous data

Abstract

The invention relates to an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data. The system comprises a data preprocessing module used for preprocessing to-be-processed multi-source heterogeneous data to obtain format-standardized text data; the semantic analysis module is used for performing natural language processing on the text data to obtain words, and then performing vocabulary layer, syntactic layer and semantic layer analysis on the text data and the words to obtain structured data; the structured data comprises a triple consisting of entities, relationships and events and a network diagram of the entities and the relationships; the model structure mapping module is used for mapping the structured data into a predefined structure of a knowledge graph to obtain structured graph data; and the knowledge graph module analyzes entities and relationships in the graph data, performs reasoning to obtain potential relationships in the graph data by utilizing a logic rule and a machine learning algorithm, realizes cross-data-source and cross-document deep semantic association, and improves the efficiency of large-scale data processing and knowledge management.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data. Background Art

[0002] With the rapid development of information technology, the diversity and complexity of data are increasing year by year. Massive heterogeneous data such as text data, image data, and table data are difficult to analyze and utilize efficiently through traditional means. Although existing natural language processing technology can extract structured information from text to a certain extent, it is often limited to a single data source and shallow semantic analysis. It cannot meet the needs of deep semantic association across data sources and multi-type files, cannot reveal the complex logical relationship between data, and is difficult to support knowledge discovery and reasoning.

[0003] As an effective way to express and associate knowledge, knowledge graphs have been widely used in various intelligent systems in recent years. By constructing knowledge graphs, information from different sources can be organically combined to form a structured, queryable and visual knowledge network. However, existing knowledge graph construction technologies usually require a lot of manual intervention, cannot automatically process multi-source heterogeneous data, and have many deficiencies in semantic association, graph reasoning and updating.

[0004] Semantic analysis technology combines natural language processing, machine learning and knowledge graph technology to achieve deep semantic understanding of text. However, it cannot accurately analyze and understand text in special scenarios, and the concepts, entities and relationships cannot be accurately extracted to achieve structured expression of information. Knowledge graphs link various types of information through the structure of graphs, becoming the basis for understanding knowledge, reasoning and data visualization. Current technology urgently needs an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data, which can semantically process various types of text data based on a predefined model structure and construct a knowledge graph, and has the ability to analyze, reason and visualize.

[0005] Existing technologies have the following main deficiencies when dealing with semantic analysis and knowledge graph construction of large-scale, multi-source heterogeneous data: Limited semantic analysis capabilities: Existing natural language processing tools have difficulty processing deep semantics in complex documents, especially when it comes to cross-document semantic associations.

[0006] Low degree of automation: The construction of knowledge graphs often relies on manual definition and lacks automated semantic information extraction and graph generation mechanisms, making it impossible to effectively cope with the needs of automated processing of massive heterogeneous data.

[0007] Lack of cross-document association analysis capabilities: Existing systems usually only process a single data source and find it difficult to effectively integrate data from multiple sources for semantic association and relational reasoning.

[0008] Insufficient reasoning and decision support capabilities: Existing knowledge graph systems lack intelligent semantic reasoning capabilities and are unable to provide users with automatic reasoning and decision support for complex relationships.

[0009] Therefore, it is of great practical significance to develop a system that can perform intelligent semantic analysis on multi-source heterogeneous data and build and update knowledge graphs through automated means. Summary of the invention

[0010] In view of the technical problems existing in the prior art, the present invention provides a knowledge graph system for intelligent semantic analysis of multi-source heterogeneous data, which aims to perform intelligent semantic analysis on various types of heterogeneous data (such as text, pictures, tables, etc.) through advanced natural language processing and deep learning technologies, automatically extract important entities and relationships in the data, and build and update knowledge graphs that meet the needs of the field. The system realizes deep semantic association across data sources and documents through automated semantic reasoning and model structure mapping functions, improves the efficiency of large-scale data processing and knowledge management, and makes up for the deficiencies in semantic information extraction, structured representation, and association analysis capabilities in the prior art.

[0011] According to a first aspect of the present invention, there is provided an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data, comprising: a data preprocessing module, a semantic analysis module, a model structure mapping module, a semantic reasoning module and a knowledge graph module; The data preprocessing module is used to preprocess the multi-source heterogeneous data to be processed to obtain text data with standardized format; The semantic analysis module is used to perform natural language processing on the text data to obtain each word, and then perform vocabulary, syntax and semantic analysis on the text data and the words to obtain structured data; the structured data includes: triples composed of entities, relations and events, and network diagrams of entities and relations; The model structure mapping module is used to map the structured data into a predefined structure of the knowledge graph to obtain structured graph data; The knowledge graph module analyzes the entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer the potential relationships existing in the graph data.

[0012] Based on the above technical solution, the present invention can also make the following improvements.

[0013] Optionally, the data preprocessing module includes: a data access layer, a data processing layer and a standardized data processing layer; The data access layer is used to receive the multi-source heterogeneous data from various sources; The data processing layer performs differentiated parsing processing according to the categories of the multi-source heterogeneous data and provides a standard processing interface, wherein the categories include: unstructured data, semi-structured data and structured data; The standardized data processing layer standardizes all parsed data to generate structured data in a unified format.

[0014] Optionally, the semantic analysis module includes: a text preprocessing layer; The process of the text preprocessing layer performing natural language processing on the text data includes: dividing the text data into individual words or phrases, removing stop words, and assigning a part-of-speech tag to each word.

[0015] Optionally, the semantic analysis module includes: an analysis layer; The analysis layer is used to perform vocabulary, syntax and semantic analysis on the text data and the words, extract semantic relations and enhance the semantics of the domain knowledge base; The process of conducting vocabulary-level analysis includes: conducting basic semantic analysis on each word or phrase in the text, performing named entity recognition, and word vector representation; The process of conducting syntactic analysis includes: paying attention to the grammatical structure in the text and analyzing the relationship between words; The process of semantic analysis includes: digging deep into the text's deep semantic information, understanding the overall meaning of the sentence and the relationship between the context; The process of extracting semantic relations includes: extracting entities and relations in the text; The process of semantic enhancement of the domain knowledge base includes: performing semantic enhancement in combination with a domain-specific knowledge base.

[0016] Optionally, the semantic analysis module includes: a semantic analysis result standardization output layer; The semantic parsing result standardized output layer is used to convert the results of the semantic analysis into the structured data, and then use entities as nodes and relationships as edges to convert the semantic information into a network diagram of nodes and edges to represent it in the structure of a knowledge graph.

[0017] Optionally, the model structure mapping module includes: a model definition unit, an entity mapping unit, a relationship mapping unit, a domain knowledge integration unit and a dynamic expansion and update unit; The model definition unit is used to set model-related attributes and dynamically set parameters to provide intervention optimization methods; The entity mapping unit is used to map the entities identified by the semantic analysis module to the nodes of the knowledge graph; The relationship mapping unit is used to map the semantic relationship between different entities into edges in the graph and specify a relationship type for each edge; The event mapping unit is used to map the event as a special entity into the knowledge graph, and represent each element in the event in the form of nodes and relationships; The domain knowledge integration unit is used to combine the structure of the domain-specific knowledge base so that domain-specific entities and relationships can be accurately mapped and associated with existing knowledge; The dynamic expansion and update unit is used to automatically expand the existing graph structure according to new data, update existing entities or relationships, and maintain the timeliness of knowledge.

[0018] 8. The system according to claim 1, characterized in that the system further comprises: a semantic reasoning module; The semantic reasoning module is used to infer potential knowledge and implicit relationships based on the structured knowledge graph.

[0019] Optionally, the reasoning of the semantic reasoning module includes: potential relationship reasoning, multi-hop reasoning, reasoning rule base and ontology-supported reasoning; The potential relationship reasoning includes: analyzing entities and relationships in the knowledge graph, using logic rules and machine learning algorithms to infer potential relationships that are not explicitly marked but actually exist in the knowledge graph; The multi-hop reasoning includes: reasoning out the final implicit relationship by analyzing multiple intermediate nodes; The inference rule base includes: an inference rule base based on each set field to guide the inference process, wherein the inference rule base contains logical rules between entities and relations within the field; The reasoning supported by the ontology includes: performing concept-level reasoning in combination with the ontology.

[0020] Optionally, the knowledge graph module analyzes entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer potential relationships in the graph data; The processing process of the knowledge graph module includes: integrating multi-source data to uniformly construct the knowledge graph structure, classifying and removing duplicates, and constructing the knowledge graph; Store the constructed knowledge graph into the knowledge base; Construct a query interface for the knowledge base that allows users to retrieve specific knowledge or relationships; The knowledge graph is presented in the form of nodes and edges, and a graphical interface is constructed to visualize the knowledge graph.

[0021] The invention provides an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data, and its beneficial effects include: 1) unified parsing capability of multi-source heterogeneous data; the system can process heterogeneous data from multiple sources, including multiple data types such as text, pictures, and tables, automatically identify key information in different data, realize unified semantic analysis across data sources, solve the bottleneck of heterogeneous data integration and processing in the prior art, and greatly improve the analysis efficiency in large-scale data environment. 2) high accuracy of multi-level semantic analysis; through natural language processing (NLP) and deep learning technology, the system can perform multi-level semantic analysis, including named entity recognition, relationship extraction, event recognition and other functions. And combined with the knowledge base of professional fields, it can more accurately identify field-specific terms and complex semantic associations, greatly improve the parsing capability of complex texts and field-specific terms, solve the shortcomings of the prior art in processing field expertise, and is suitable for semantic analysis scenarios in various industries, such as law, medical care, finance, etc. 3) automatic generation and dynamic update of knowledge graphs; the system supports automatic mapping of analyzed semantic information to the knowledge graph model to generate a structured knowledge graph without manual intervention by users. In addition, the system supports the dynamic expansion and real-time update of the knowledge graph, and can automatically integrate new data content. It greatly reduces the need for manual participation, realizes the automatic construction and real-time update of the knowledge graph, adapts to the dynamically changing data environment, and meets the needs of enterprises for real-time data analysis and knowledge management. 4) Cross-document and cross-model semantic association analysis: The system can analyze and integrate the semantic information between multiple data sources, derive the semantic associations and potential relationships across documents and models, and form a global knowledge network. It fills the gap in cross-document and multi-model association analysis in the existing technology, significantly enhances the global cognitive ability of the knowledge graph, and provides more comprehensive and accurate knowledge management and analysis support for enterprises or institutions. 5) Automated semantic reasoning and decision support: By integrating graph algorithms and logical reasoning engines, the system can perform automated reasoning based on the knowledge graph, derive implicit semantic relationships, help users discover potential knowledge or associations, and provide intelligent decision support. It enhances the application value of the knowledge graph, especially in fields that require complex reasoning such as medical diagnosis and legal case analysis, the system can provide more intelligent decision support functions. 6) Dynamic integration of domain-specific knowledge bases: The system supports the integration of domain-specific knowledge bases, and can build and analyze knowledge graphs based on specific industry needs (such as medical, legal, and financial). The knowledge base supports dynamic expansion and is automatically updated as new data is input. This improves the system's professionalism and adaptability in specific fields, enhances the understanding and reasoning capabilities of domain knowledge, and meets the personalized needs of various industries. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A structural diagram of a knowledge graph system for intelligent semantic analysis of multi-source heterogeneous data provided by the present invention; Figure 2 A flowchart of a knowledge graph method for intelligent semantic analysis of multi-source heterogeneous data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] Figure 1 The flow chart of the knowledge graph system for intelligent semantic analysis of multi-source heterogeneous data provided by the present invention is as follows: Figure 1 As shown, the system includes: a data preprocessing module, a semantic analysis module, a model structure mapping module, a semantic reasoning module and a knowledge graph module.

[0025] The data preprocessing module is used to preprocess the multi-source heterogeneous data to be processed to obtain text data with standardized format.

[0026] In specific implementation, the system collects data from different data sources and converts each data into an analyzable format through preprocessing steps.

[0027] The semantic analysis module is used to perform natural language processing on text data to obtain individual words, and then perform lexical, syntactic and semantic analysis on the text data and words to obtain structured data; structured data includes: triplets consisting of entities, relationships and events, and network diagrams of entities and relationships.

[0028] The model structure mapping module is used to map structured data into the predefined structure of the knowledge graph to obtain structured graph data.

[0029] The knowledge graph module analyzes the entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer the potential relationships in the graph data.

[0030] The present invention provides a knowledge graph system for intelligent semantic analysis of multi-source heterogeneous data, which aims to perform intelligent semantic analysis on various types of heterogeneous data (such as text, pictures, tables, etc.) through advanced natural language processing and deep learning technologies, automatically extract important entities and relationships in the data, and build and update knowledge graphs that meet field requirements. The system realizes deep semantic associations across data sources and documents through automated semantic reasoning and model structure mapping functions, improves the efficiency of large-scale data processing and knowledge management, and makes up for the deficiencies in semantic information extraction, structured representation, and association analysis capabilities in existing technologies.

[0031] Example 1 Embodiment 1 provided by the present invention is an embodiment of an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data provided by the present invention, such as Figure 2 The flowchart of a method for intelligent semantic analysis of knowledge graph of multi-source heterogeneous data provided by an embodiment of the present invention is shown in FIG. Figure 1 and Figure 2 It can be seen that the embodiment of the system includes: a data preprocessing module, a semantic analysis module, a model structure mapping module, a semantic reasoning module and a knowledge graph module.

[0032] The data preprocessing module is used to preprocess the multi-source heterogeneous data to be processed to obtain text data with standardized format.

[0033] In a possible implementation manner, the data preprocessing module includes: a data access layer, a data processing layer and a standardized data processing layer.

[0034] The data access layer is used to receive multi-source heterogeneous data from various sources.

[0035] In the specific implementation, access processing is performed according to the sources of different data. For example, for the access of unstructured files, a visual interface can be used for uploading and importing, or an API can be used for importing. Semi-structured data can be accessed specifically according to the source of the data, not limited to communication protocols such as HTTP and TCP.

[0036] The data processing layer distinguishes and parses multi-source heterogeneous data according to their categories and provides standard processing interfaces. The categories include: unstructured data, semi-structured data, and structured data.

[0037] In specific implementation, the data processing layer 120 processes data files from different sources. Unstructured data includes but is not limited to PDF, WORD, TEXT, pictures and web pages, etc. Semi-structured data includes but is not limited to JSON, XML, API object instances, etc. Structured data includes but is not limited to manual input data of dynamic form functions provided by the system, data content of model instances, various database structured data, etc. The processing interface provided to the outside is extensible, developed in JAVA, and dynamically connected to jar.

[0038] The standardized data processing layer standardizes all parsed data and generates structured data in a unified format to facilitate analysis by the semantic analysis model.

[0039] The semantic analysis module is used to perform natural language processing on text data to obtain individual words, and then perform lexical, syntactic and semantic analysis on the text data and words to obtain structured data; structured data includes: triplets consisting of entities, relationships and events, and network diagrams of entities and relationships.

[0040] In a possible embodiment, the semantic analysis module includes: a text preprocessing layer.

[0041] The process of the text preprocessing layer performing natural language processing on text data includes: segmenting the text data into individual words or phrases, removing stop words, and assigning part-of-speech tags to each word.

[0042] In a specific implementation, after receiving the standardized text data, the semantic analysis module 200 performs preprocessing through the text preprocessing layer 210. The main steps of the preprocessing include: Tokenization: Segmenting the original text into individual words or phrases as the basic units for subsequent processing. Removing stop words: Removing common words with little semantic contribution, such as "de", "shi", "he", etc., to reduce the interference of noisy data. Part-of-Speech Tagging (POS Tagging): Assigning part-of-speech tags (such as nouns, verbs, adjectives, etc.) to each word to provide a basis for subsequent syntactic and semantic parsing.

[0043] In a possible embodiment, the semantic analysis module includes: an analysis layer.

[0044] In a specific implementation, the analysis layer 220 is the core of the semantic analysis module 200, performing lexical layer analysis, syntactic layer analysis, semantic layer analysis, semantic relation extraction, and semantic enhancement of the domain knowledge base.

[0045] The analysis layer is used to perform lexical layer, syntactic layer, and semantic layer analysis on text data and words, semantic relation extraction, and semantic enhancement of the domain knowledge base.

[0046] The process of performing lexical layer analysis includes: performing basic semantic analysis on each word or phrase in the text, performing named entity recognition, and word vector representation.

[0047] The process of performing syntactic layer analysis includes: paying attention to the grammatical structure in the text and analyzing the relationships between words.

[0048] The process of performing semantic layer analysis includes: deeply mining the deep semantic information of the text, understanding the overall meaning of the sentence and the associations between contexts.

[0049] The process of semantic relation extraction includes: further extracting entities and relationships in the text.

[0050] The process of semantic enhancement of the domain knowledge base includes: combining domain-specific knowledge bases for semantic enhancement, which can improve the accuracy of semantic analysis.

[0051] In a possible embodiment, the semantic analysis module includes: a semantic parsing result standardization output layer.

[0052] The standardized output layer of semantic parsing results is used to convert the results of semantic analysis into structured data, and then use entities as nodes and relationships as edges to convert semantic information into a network diagram of nodes and edges and represent it in the structure of a knowledge graph.

[0053] In the specific implementation, the semantic parsing result is standardized and output 230, and the result of the semantic analysis is converted into a structured data form, mainly including the triple representation of entities, relationships and events: each entity, relationship, event is represented as a triple (subject-predicate-object), such as (Zhang San, professor, university), semantic network representation: through the structural representation of the knowledge graph, the semantic information is converted into a network diagram of nodes (entities) and edges (relationships), which is convenient for subsequent knowledge graph construction and reasoning.

[0054] The core task of the model structure mapping module 300 is to automatically map the entities, relationships, events and other information extracted by the semantic analysis module to the predefined structure of the knowledge graph. The focus of this process is to convert unstructured natural language data into structured graph data and ensure that semantic information is accurately and efficiently expressed in the knowledge graph.

[0055] The model structure mapping module is used to map structured data into the predefined structure of the knowledge graph to obtain structured graph data.

[0056] In a possible implementation manner, the model structure mapping module includes: a model definition unit, an entity mapping unit, a relationship mapping unit, a domain knowledge integration unit and a dynamic expansion and update unit.

[0057] The model definition unit is used to set model-related properties in advance and dynamically set parameters to provide intervention optimization methods; in specific implementations, the intervention optimization methods include regularization, scripts, etc.

[0058] The entity mapping unit is used to map the entities (such as names of people, places, and organizations) identified by the semantic analysis module to nodes in the knowledge graph.

[0059] The relationship mapping unit is used to map the semantic relationships between different entities into edges in the graph and specify the relationship type for each edge (such as "employed by", "located in", "subsidiary", etc.).

[0060] The event mapping unit is used to map events as special entities into the knowledge graph, and to represent the various elements in the event (such as subject, action, time, place, etc.) in the form of nodes and relationships.

[0061] The domain knowledge integration unit is used to combine the structure of the domain-specific knowledge base so that domain-specific entities and relations can be accurately mapped and associated with existing knowledge.

[0062] Since the knowledge graph is dynamically changing, the dynamic expansion and update unit is used to automatically expand the existing graph structure according to new data, update existing entities or relationships, and maintain the timeliness of knowledge.

[0063] The knowledge graph module analyzes the entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer the potential relationships in the graph data.

[0064] In one possible implementation manner, the knowledge graph module analyzes entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer potential relationships existing in the graph data.

[0065] The processing process of the knowledge graph module includes: integrating multi-source data into a unified structure according to the knowledge graph, classifying and removing duplicates, and constructing the knowledge graph.

[0066] Store the constructed knowledge graph in the knowledge base; commonly used storage methods include graph databases (such as Neo4j, JanusGraph): used to efficiently store and query graph data, and relational databases: can be used to store structured triple data.

[0067] Build a query interface for the knowledge base that allows users to retrieve specific knowledge or relationships; the system supports complex query statements, allowing users to filter and analyze information in the knowledge graph based on different conditions.

[0068] The knowledge graph is presented in the form of nodes and edges, and a graphical interface is constructed to visualize the knowledge graph. The system allows users to intuitively observe the relationship between knowledge. Users can perform interactive queries through natural language input or a graphical interface, and the system dynamically generates query results based on user requests.

[0069] In a possible implementation manner, the system further includes: a semantic reasoning module.

[0070] The semantic reasoning module is used to infer potential knowledge and implicit relationships based on the structured knowledge graph.

[0071] In a possible implementation manner, the reasoning of the semantic reasoning module includes: potential relationship reasoning, multi-hop reasoning, reasoning rule base and ontology-supported reasoning.

[0072] Latent relationship reasoning includes: analyzing the entities and relationships in the knowledge graph, using logical rules and machine learning algorithms to infer latent relationships that are not explicitly labeled in the knowledge graph but actually exist; for example, based on the two facts that "Zhang San is a professor at Peking University" and "Peking University is located in Beijing, China", the system can infer the latent relationship that "Zhang San is located in Beijing, China".

[0073] Multi-hop reasoning includes: inferring the final implicit relationship by analyzing multiple intermediate nodes; for example, "Zhang San once co-authored a paper with Li Si", and "Li Si once worked as a consultant in a certain company", the system can deduce the conclusion that "Zhang San has a potential connection with a certain company" through these two-hop relationships.

[0074] The inference rule base includes: the inference rule base based on each field set to guide the inference process, and the inference rule base contains the common logical rules between entities and relationships in the field; for example, "if A is an employee of B, and B is located in C, then A may also be located in C." It is recommended to use the rule engine + dynamic script method to intervene in dynamic optimization.

[0075] Reasoning supported by ontology includes: Concept hierarchical reasoning in combination with ontology. Ontology provides a structured description of domain knowledge and defines the hierarchy and relationship between concepts. Through ontology, the system can perform concept hierarchical reasoning. For example, "cat" and "dog" both belong to "mammal". If certain attributes of "mammal" exist in the system, the reasoning module can apply these attributes to "cat" and "dog".

[0076] The core function of the semantic reasoning module is that through reasoning, the system can not only retrieve explicit relationships, but also discover deeper connections hidden in the data, thereby facilitating the discovery and expansion of knowledge.

[0077] The embodiment of the present invention provides an intelligent semantic analysis knowledge graph system for multi-source heterogeneous data, and its beneficial effects include: 1) unified parsing capability of multi-source heterogeneous data; the system can process heterogeneous data from multiple sources, including multiple data types such as text, pictures, and tables, automatically identify key information in different data, realize unified semantic analysis across data sources, solve the bottleneck of heterogeneous data integration and processing in the prior art, and greatly improve the analysis efficiency in large-scale data environments. 2) High accuracy of multi-level semantic analysis; through natural language processing (NLP) and deep learning technology, the system can perform multi-level semantic analysis, including named entity recognition, relationship extraction, event recognition and other functions. And combined with the knowledge base of professional fields, it can more accurately identify field-specific terms and complex semantic associations, greatly improve the parsing capability of complex texts and field-specific terms, solve the shortcomings of the prior art in processing field expertise, and is suitable for semantic analysis scenarios in various industries, such as law, medical care, finance, etc. 3) Automatic generation and dynamic update of knowledge graphs; the system supports automatic mapping of analyzed semantic information to the knowledge graph model to generate a structured knowledge graph without manual intervention by users. In addition, the system supports the dynamic expansion and real-time update of the knowledge graph, and can automatically integrate new data content. It greatly reduces the need for manual participation, realizes the automatic construction and real-time update of the knowledge graph, adapts to the dynamically changing data environment, and meets the needs of enterprises for real-time data analysis and knowledge management. 4) Cross-document and cross-model semantic association analysis: The system can analyze and integrate the semantic information between multiple data sources, derive the semantic associations and potential relationships across documents and models, and form a global knowledge network. It fills the gap in cross-document and multi-model association analysis in the existing technology, significantly enhances the global cognitive ability of the knowledge graph, and provides more comprehensive and accurate knowledge management and analysis support for enterprises or institutions. 5) Automated semantic reasoning and decision support: By integrating graph algorithms and logical reasoning engines, the system can perform automated reasoning based on the knowledge graph, derive implicit semantic relationships, help users discover potential knowledge or associations, and provide intelligent decision support. It enhances the application value of the knowledge graph, especially in fields that require complex reasoning such as medical diagnosis and legal case analysis, the system can provide more intelligent decision support functions. 6) Dynamic integration of domain-specific knowledge bases: The system supports the integration of domain-specific knowledge bases, and can build and analyze knowledge graphs based on specific industry needs (such as medical, legal, and financial). The knowledge base supports dynamic expansion and is automatically updated as new data is input. This improves the system's professionalism and adaptability in specific fields, enhances the understanding and reasoning capabilities of domain knowledge, and meets the personalized needs of various industries.

[0078] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0079] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0080] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0084] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent semantic analysis knowledge graph system for multi-source heterogeneous data, characterized in that: The system includes: a data preprocessing module, a semantic analysis module, a model structure mapping module, a semantic reasoning module and a knowledge graph module; The data preprocessing module is used to preprocess the multi-source heterogeneous data to be processed to obtain text data with standardized format; The semantic analysis module is used to perform natural language processing on the text data to obtain each word, and then perform vocabulary, syntax and semantic analysis on the text data and the words to obtain structured data; the structured data includes: triples composed of entities, relations and events, and network diagrams of entities and relations; The model structure mapping module is used to map the structured data into a predefined structure of the knowledge graph to obtain structured graph data; The knowledge graph module analyzes the entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer the potential relationships existing in the graph data.

2. The system according to claim 1, characterized in that The data preprocessing module includes: a data access layer, a data processing layer and a standardized data processing layer; The data access layer is used to receive the multi-source heterogeneous data from various sources; The data processing layer performs differentiated parsing processing according to the categories of the multi-source heterogeneous data and provides a standard processing interface, wherein the categories include: unstructured data, semi-structured data and structured data; The standardized data processing layer standardizes all parsed data to generate structured data in a unified format.

3. The system according to claim 1, characterized in that The semantic analysis module includes: a text preprocessing layer; The process of the text preprocessing layer performing natural language processing on the text data includes: dividing the text data into individual words or phrases, removing stop words, and assigning a part-of-speech tag to each word.

4. The system according to claim 1, characterized in that The semantic analysis module includes: an analysis layer; The analysis layer is used to perform vocabulary, syntax and semantic analysis on the text data and the words, extract semantic relations and enhance the semantics of the domain knowledge base; The process of conducting vocabulary-level analysis includes: conducting basic semantic analysis on each word or phrase in the text, performing named entity recognition, and word vector representation; The process of conducting syntactic analysis includes: paying attention to the grammatical structure in the text and analyzing the relationship between words; The process of semantic analysis includes: digging deep into the text's deep semantic information, understanding the overall meaning of the sentence and the relationship between the context; The process of extracting semantic relations includes: extracting entities and relations in the text; The process of semantic enhancement of the domain knowledge base includes: performing semantic enhancement in combination with a domain-specific knowledge base.

5. The system according to claim 1, characterized in that The semantic analysis module includes: a semantic analysis result standardization output layer; The semantic parsing result standardized output layer is used to convert the results of the semantic analysis into the structured data, and then use entities as nodes and relationships as edges to convert the semantic information into a network diagram of nodes and edges to represent it in the structure of a knowledge graph.

6. The system according to claim 1, characterized in that The model structure mapping module includes: a model definition unit, an entity mapping unit, a relationship mapping unit, a domain knowledge integration unit and a dynamic expansion and update unit; The model definition unit is used to set model-related attributes and dynamically set parameters to provide intervention optimization methods; The entity mapping unit is used to map the entities identified by the semantic analysis module to the nodes of the knowledge graph; The relationship mapping unit is used to map the semantic relationship between different entities into edges in the graph and specify a relationship type for each edge; The event mapping unit is used to map the event as a special entity into the knowledge graph, and represent each element in the event in the form of nodes and relationships; The domain knowledge integration unit is used to combine the structure of the domain-specific knowledge base so that domain-specific entities and relationships can be accurately mapped and associated with existing knowledge; The dynamic expansion and update unit is used to automatically expand the existing graph structure according to new data, update existing entities or relationships, and maintain the timeliness of knowledge.

7. The system according to claim 1, characterized in that The system also includes: a semantic reasoning module; The semantic reasoning module is used to infer potential knowledge and implicit relationships based on the structured knowledge graph.

8. The system according to claim 7, characterized in that The reasoning of the semantic reasoning module includes: potential relationship reasoning, multi-hop reasoning, reasoning rule base and ontology-supported reasoning; The potential relationship reasoning includes: analyzing entities and relationships in the knowledge graph, using logic rules and machine learning algorithms to infer potential relationships that are not explicitly marked but actually exist in the knowledge graph; The multi-hop reasoning includes: reasoning out the final implicit relationship by analyzing multiple intermediate nodes; The inference rule base includes: an inference rule base based on each set field to guide the inference process, wherein the inference rule base contains logical rules between entities and relations within the field; The reasoning supported by the ontology includes: performing concept-level reasoning in combination with the ontology.

9. The system according to claim 1, characterized in that The knowledge graph module analyzes entities and relationships in the graph data, and uses logical rules and machine learning algorithms to infer potential relationships in the graph data; The processing process of the knowledge graph module includes: integrating multi-source data to uniformly construct the knowledge graph structure, classifying and removing duplicates, and constructing the knowledge graph; Store the constructed knowledge graph into the knowledge base; Construct a query interface for the knowledge base that allows users to retrieve specific knowledge or relationships; The knowledge graph is presented in the form of nodes and edges, and a graphical interface is constructed to visualize the knowledge graph.

Images