Large model and knowledge graph fusion method, application method and system

Abstract

The invention provides a large model and mapping knowledge domain fusion method, application method and system, and the method comprises the steps: carrying out the mapping knowledge domain embedding of an obtained mapping knowledge domain to generate mapping knowledge domain embedded information, and generating a natural language question based on the mapping knowledge domain embedded information; the knowledge graph embedded information and the natural language question sentences are input into an entity relation learning model, entity expressions and relation expressions corresponding to the natural language question sentences are output, then sub-graphs corresponding to the natural language question sentences are extracted based on the natural language question sentences, the entity expressions and the relation expressions, and the sub-graphs are converted into natural languages one by one. And combining the entity relationship learning model and the knowledge graph embedded information into the large model to generate a target reasoning model so as to realize fusion of the knowledge graph and the large model. Therefore, the model can extract key information and perform deep understanding and reasoning, the reasoning ability of the model in processing complex problems is enhanced, and the definition and interpretability of subsequent output contents of the model are further improved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graphs, and specifically to a method and system for fusing a large model with a knowledge graph. Background Art

[0002] Currently, various fields have provided scenarios and applications for the combination of big models and knowledge graphs. This combination has brought new perspectives and possibilities to the field of artificial intelligence.

[0003] The "Wen Xin Yi Yan" model has powerful natural language processing and logical reasoning capabilities through pre-training with a large-scale corpus. In terms of knowledge reasoning, the "Wen Xin Yi Yan" model uses technologies such as deep learning attention mechanism and feedforward neural network to perform efficient semantic analysis and reasoning. In addition, a knowledge-enhanced deep question-answering system "ERNIE Bot" has been launched. The system uses natural language processing and deep learning technology to analyze and reason questions, retrieve and extract relevant information from a large knowledge base, and get the correct answer.

[0004] The cognitive model of "Spark" takes natural language understanding as its core, combines multimodal perception and cognitive computing capabilities, and can deeply understand and explain human language behavior. In terms of knowledge reasoning, the cognitive model of "Spark" uses technologies such as recurrent neural networks and attention mechanisms in deep learning technology, which can analyze, understand and reason about problems, retrieve and extract relevant information from a large knowledge base, and obtain the correct answer.

[0005] The LaMDA model has powerful natural language processing and logical reasoning capabilities through pre-training on a large-scale corpus. In terms of knowledge reasoning, the LaMDA model uses technologies such as Transformer and attention mechanism in deep learning technology to analyze, understand and reason about problems, retrieve and extract relevant information from a large knowledge base, and obtain the correct answer.

[0006] In addition, an intelligent reasoning technology based on the BERT model uses technologies such as Transformer and bidirectional pre-training in natural language processing and deep learning technology to analyze, understand and reason about questions, retrieve and extract relevant information from a large knowledge base to obtain the correct answer.

[0007] Technology companies and research institutions at home and abroad are exploring intelligent reasoning technologies that integrate large models with knowledge graphs, and are using different deep learning and natural language processing technologies to improve the ability and efficiency of intelligent reasoning. The continuous development of these technologies will further promote the development and application of intelligent reasoning technologies.

[0008] Although some companies are trying to combine knowledge graphs with large language models to improve the understanding and reasoning capabilities of intelligent systems, their exploration of how to better enable knowledge graphs to provide knowledge for large language models is still in the initial stage. Their methods have the following shortcomings:

[0009] 1. Insufficient depth of knowledge fusion: Existing technologies may not fully realize the deep fusion between knowledge graphs and large language models, resulting in limited knowledge application and reasoning capabilities.

[0010] 2. Poor dynamic updating and adaptability: Existing technologies may perform poorly in terms of dynamic updating of knowledge graphs and real-time adaptability of intelligent systems, making it difficult to quickly respond to changes in the information environment.

[0011] 3. Low explainability: Existing technologies may be insufficient in the explainability of intelligent question-answering systems, making it difficult for users to understand the system’s reasoning process and the basis of the results.

[0012] 4. Limitations in cross-domain applications: Existing technologies may have limitations in cross-domain applications and may be difficult to adapt to the specific needs and knowledge structures of different fields. Summary of the invention

[0013] In order to solve the problem that the existing technology lacks the ability to understand and reason in the fusion of large models and knowledge graphs, the present invention proposes a method and system for fusing large models with knowledge graphs, and an application method and system for a target reasoning model based on knowledge graph fusion, including:

[0014] In the first aspect, a method for fusing a large model with a knowledge graph is provided, including:

[0015] Obtain a knowledge graph and an entity relationship learning model, and perform knowledge graph embedding on the knowledge graph to generate knowledge graph embedding information;

[0016] Generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output an entity representation and a relationship representation corresponding to the natural language question;

[0017] Extracting a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation;

[0018] The subgraph is converted into natural language and combined with the entity relationship learning model and the knowledge graph embedding information into the big model to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the big model.

[0019] Preferably, extracting a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation includes:

[0020] Extracting output information corresponding to the natural language question from the knowledge graph embedding information based on the context of the natural language question;

[0021] Determine the extended information corresponding to the output information based on the relationship representation and the entity representation;

[0022] A subgraph is generated based on the output information and the extended information.

[0023] Preferably, said embedding the knowledge graph into the knowledge graph to generate knowledge graph embedding information comprises:

[0024] Constructing a relation embedding space by using the TransR method, and mapping entities of the knowledge graph to the relation embedding space;

[0025] Establishing a projection matrix based on the relationship between the triples of the knowledge graph;

[0026] Calculating the distance between the head entity and the tail entity in the triplet by a scoring formula;

[0027] Based on the triples in the knowledge graph, replacing the original head entity or tail entity with a random entity, constructing a negative example set of the triples;

[0028] Knowledge graph embedding information is generated according to the relation embedding space, the projection matrix, the distance between the head entity and the tail entity, and the negative example set of the triples.

[0029] Preferably, the step of inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model and outputting the entity representation and relationship representation corresponding to the natural language question includes:

[0030] Inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, and learning the entity relationship representation in the knowledge graph embedding information through the entity relationship learning model;

[0031] Based on the natural language question, the entity representation and relationship representation corresponding to the natural language question are determined from the knowledge graph embedding information through the entity relationship learning model, and output.

[0032] Secondly, a large model and knowledge graph fusion system is provided, including:

[0033] An embedding module, used to obtain a knowledge graph and an entity relationship learning model, and to embed the knowledge graph into a knowledge graph to generate knowledge graph embedding information;

[0034] An output module, configured to generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output an entity representation and a relationship representation corresponding to the natural language question;

[0035] An extraction module, configured to extract a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation;

[0036] A fusion module is used to convert the subgraph into natural language and integrate it into the large model by combining the entity relationship learning model and the knowledge graph embedding information to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the large model.

[0037] Preferably, the extraction module is further configured to:

[0038] Extracting output information corresponding to the natural language question from the knowledge graph embedding information based on the context of the natural language question;

[0039] Determine the extended information corresponding to the output information based on the relationship representation and the entity representation;

[0040] A subgraph is generated based on the output information and the extended information.

[0041] Preferably, the embedded module is further configured as follows:

[0042] Constructing a relation embedding space by using the TransR method, and mapping entities of the knowledge graph to the relation embedding space;

[0043] Establishing a projection matrix based on the relationship between the triples of the knowledge graph;

[0044] Calculating the distance between the head entity and the tail entity in the triplet by a scoring formula;

[0045] Based on the triples in the knowledge graph, replacing the original head entity or tail entity with a random entity, constructing a negative example set of the triples;

[0046] Knowledge graph embedding information is generated according to the relation embedding space, the projection matrix, the distance between the head entity and the tail entity, and the negative example set of the triples.

[0047] Preferably, the fusion module is further configured to:

[0048] Inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, and learning the entity relationship representation in the knowledge graph embedding information through the entity relationship learning model;

[0049] Based on the natural language question, the entity representation and relationship representation corresponding to the natural language question are determined from the knowledge graph embedding information through the entity relationship learning model, and output.

[0050] Thirdly, an application method of a target reasoning model based on knowledge graph fusion is provided, including:

[0051] Obtaining a context corresponding to the user question and answer information, and extracting question and answer keywords from the user question and answer information;

[0052] Substitute the question-and-answer keywords into the pre-built target reasoning model to obtain the user question-and-answer output information corresponding to the user question-and-answer information

[0053] Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any one of claims 1 to 4.

[0054] Preferably, the step of substituting the question and answer keywords into a pre-built target reasoning model to obtain user question and answer output information corresponding to the user question and answer information includes:

[0055] Outputting entity representation and relationship representation corresponding to the user question and answer information based on the question and answer keywords through an entity relationship learning model in a pre-built target reasoning model;

[0056] Extracting a first question-answer subgraph corresponding to the user question-answer information based on the entity representation, the relationship representation, and the context of the user question-answer information;

[0057] User question and answer output information corresponding to the user question and answer information is generated based on the first question and answer subgraph.

[0058] Preferably, after generating the user question and answer output information corresponding to the user question and answer information based on the first question and answer subgraph, the method further includes:

[0059] After receiving the user updated question and answer information, extracting the user updated keywords from the user updated question and answer information according to the similarity between the user question and answer information and the user updated question and answer information;

[0060] Extracting a second question-and-answer subgraph corresponding to the user-updated question-and-answer information based on the user-updated keyword and the first question-and-answer subgraph;

[0061] Output the user question and answer output information corresponding to the user updated question and answer information based on the second question and answer subgraph.

[0062] Preferably, the obtaining of the context corresponding to the user question and answer information includes:

[0063] The knowledge graph corresponding to the user question and answer information is retrieved in a search enhancement manner to obtain question and answer extension information, and the question and answer extension information is used as the context corresponding to the user question and answer information.

[0064] Fourthly, an application system of a target reasoning model based on knowledge graph fusion is provided, including:

[0065] An information extraction module, used to obtain the context corresponding to the user question and answer information, and extract question and answer keywords from the user question and answer information;

[0066] An output information module, used to substitute the question-and-answer keywords into a pre-built target reasoning model to obtain user question-and-answer output information corresponding to the user question-and-answer information;

[0067] Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any of the above.

[0068] Preferably, the output information module is further used to:

[0069] Outputting entity representation and relationship representation corresponding to the user question and answer information based on the question and answer keywords through an entity relationship learning model in a pre-built target reasoning model;

[0070] Extracting a first question-answer subgraph corresponding to the user question-answer information based on the entity representation, the relationship representation, and the context of the user question-answer information;

[0071] User question and answer output information corresponding to the user question and answer information is generated based on the first question and answer subgraph.

[0072] Preferably, the system further comprises:

[0073] A subgraph updating module, configured to extract user update keywords from the user update question and answer information according to the similarity between the user question and answer information and the user update question and answer information after receiving the user update question and answer information;

[0074] Extracting a second question-and-answer subgraph corresponding to the user-updated question-and-answer information based on the user-updated keyword and the first question-and-answer subgraph;

[0075] Output the user question and answer output information corresponding to the user updated question and answer information based on the second question and answer subgraph.

[0076] Preferably, the information extraction module is further used to:

[0077] The knowledge graph corresponding to the user question and answer information is retrieved in a search enhancement manner to obtain question and answer extension information, and the question and answer extension information is used as the context corresponding to the user question and answer information.

[0078] In a fifth aspect, the present application further provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus;

[0079] The memory is used to store one or more programs;

[0080] When the one or more programs are executed by the at least one processor, the method of fusing the large model with the knowledge graph as described above and the method of applying the target reasoning model based on knowledge graph fusion are implemented.

[0081] In a sixth aspect, the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, it implements the method for fusing the large model with the knowledge graph as described above and the application method of the target reasoning model based on the knowledge graph fusion.

[0082] Compared with the prior art, the present invention has the following beneficial effects:

[0083] The present invention provides a method for fusing a large model with a knowledge graph, by embedding the acquired knowledge graph into a knowledge graph to generate knowledge graph embedding information, and then generating a natural language question based on the knowledge graph embedding information, and inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, outputting the entity representation and relationship representation corresponding to the natural language question, and then extracting the subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation, and then converting the subgraph into natural language, and then integrating the entity relationship learning model and the knowledge graph embedding information into the large model, generating a target reasoning model, and realizing the fusion of the knowledge graph and the large model. By combining the language generation capability of the large model with the structure and semantic information of the knowledge graph, the model can extract key information and conduct in-depth understanding and reasoning, enhancing the reasoning capability of the model when dealing with complex problems, and further improving the clarity and interpretability of the subsequent output content of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a flow chart of the method for fusing a large model with a knowledge graph of the present invention;

[0085] Figure 2 It is a TransR schematic diagram of the method for fusing a large model with a knowledge graph of the present invention;

[0086] Figure 3It is a schematic diagram of the structure of the entity relationship learning model of the large model and knowledge graph fusion method of the present invention;

[0087] Figure 4 It is a structural schematic diagram of the large model and knowledge graph fusion system of the present invention;

[0088] Figure 5 It is a flow chart of the application method of the target reasoning model based on knowledge graph fusion of the present invention;

[0089] Figure 6 It is a schematic diagram of the structure of the application system of the target reasoning model based on knowledge graph fusion of the present invention;

[0090] Figure 7 The figure is a schematic diagram of the structure of an electronic device of the present invention. DETAILED DESCRIPTION

[0091] In order to solve the technical problem of the lack of understanding and reasoning capabilities in the fusion of large models and knowledge graphs, the present invention combines the advanced language understanding and reasoning capabilities of the large language model (LLM) with the structured knowledge of the knowledge graph (KG) to significantly improve the intelligence level of enterprise knowledge services. The core principle is to use the deep semantic analysis and knowledge generation capabilities of LLM, supplemented by the rich structured information of KG, to achieve efficient and accurate intelligent reasoning, and enhance the interpretability of the KG-based question-answering system. By building advanced intelligent question-answering system tools, the present invention is not only committed to improving the efficiency and quality of enterprise knowledge services, but also through verification in actual application scenarios, further strengthening the innovation capabilities and knowledge management of enterprises. In addition, through data-driven decision support, this project also aims to promote enterprises to achieve more efficient knowledge utilization and more accurate business insights.

[0092] In order to better understand the present invention, the content of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0093] Embodiment 1:

[0094] A method for fusing a large model with a knowledge graph, such as Figure 1 As shown, including:

[0095] Step 1: Obtain a knowledge graph and an entity relationship learning model, and perform knowledge graph embedding on the knowledge graph to generate knowledge graph embedding information;

[0096] Step 2: Generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output the entity representation and relationship representation corresponding to the natural language question;

[0097] Step 3: extracting a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation;

[0098] Step 4: Convert the subgraph into natural language and integrate it into the big model by combining the entity relationship learning model and the knowledge graph embedding information to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the big model.

[0099] In this embodiment, in the process of embedding the acquired knowledge graph into the knowledge graph to generate knowledge graph embedding information in step 1, a relation embedding space is constructed by the TransR method, and the entities of the knowledge graph are mapped to the relation embedding space;

[0100] Establishing a projection matrix based on the relationship between the triples of the knowledge graph;

[0101] Calculating the distance between the head entity and the tail entity in the triplet by a scoring formula;

[0102] Based on the triples in the knowledge graph, replacing the original head entity or tail entity with a random entity, constructing a negative example set of the triples;

[0103] Knowledge graph embedding information is generated according to the relation embedding space, the projection matrix, the distance between the head entity and the tail entity, and the negative example set of the triples.

[0104] In one embodiment,

[0105] Among them, knowledge graph embedding is the product of knowledge graph representation learning. Its purpose is to convert text data in the knowledge graph into vectors that are easy to calculate and understand. These vectors store the structured relationships of knowledge in the knowledge graph and can be used in downstream tasks of the knowledge graph.

[0106] In the knowledge graph, an entity may have multiple attributes, and different relationships correspond to different attributes of the entity. For example, "Mr. A" is both a woman's husband and an American born in Hawaii. The relationship "husband" and the relationship "age" correspond to different attributes. Intuitively similar entities in the same semantic space should be closer in space, and in different semantic spaces, their distance should be farther according to the expression of different aspects of the entity.

[0107] like Figure 2 As shown, Figure 2 This is a TransR schematic diagram of the large model and knowledge graph fusion method of the present invention, based on Figure 2 An introduction to knowledge graph embedding.

[0108] Through TransR, an independent relational space is established and entities are mapped into it, so that the semantics of entities in each relation are interpreted. A triple includes the head entity embedding h, the tail entity embedding t and the relation embedding r, and a projection matrix Mr is established based on the existing relations. The mapping formula is as follows:

[0109] h r =hM r ,t r =tM r

[0110] In order to make the head entity and tail entity with the same relationship closer, the scoring formula is as follows:

[0111]

[0112] In addition, knowledge graph embedding also needs to construct negative examples during the training process. For a given triple in the knowledge graph, a random entity is used to replace the original head entity or tail entity to construct a negative example set of the triple; then the knowledge graph is embedded based on the above content.

[0113] In this embodiment, in step 2, the process of inputting the knowledge graph embedding information and the natural language question into the entity relationship learning model to output the entity representation and relationship representation corresponding to the natural language question includes:

[0114] Inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, and learning the entity relationship representation in the knowledge graph embedding information through the entity relationship learning model;

[0115] Based on the natural language question, the entity representation and relationship representation corresponding to the natural language question are determined from the knowledge graph embedding information through the entity relationship learning model, and output.

[0116] In one embodiment, the vector modeling-based method often performs sorting by calculating the score between the question vector and the triple vector. In order to give full play to the role of knowledge graph embedding, the proposed knowledge graph question answering method based on knowledge graph embedding inputs the question into the entity relationship learning model. After the model learns the information in the knowledge graph embedding space, it predicts the entity representation and relationship representation respectively, connects the question semantic space and the knowledge graph embedding space together, and bridges the semantic gap between the question and the knowledge graph.

[0117] The purpose of the entity relationship learning model is to find the head entity representation in the entity embedding space and the relationship representation in the relationship embedding space. Enter a natural language question. Assuming that the answer to the question exists in the knowledge graph, then the relationship and entities implied in the question must exist in the knowledge graph, that is, the representation of the relationship and entity must exist in the knowledge graph relationship embedding space. Therefore, an entity relationship learning model is designed to learn the entity representation and relationship representation by inputting natural language questions into the model. In addition, the entity relationship learning model uses the same model structure when learning entity and relationship representations, and completes the results of two sets of parameters for one model through different inputs and objective functions. This allows models with the same structure to predict two different representation outputs. The structure of the entity relationship learning model is as follows: Figure 3 As shown in the figure, BERT can be understood as a language representation model, Attention can be understood as the attention mechanism layer, and c 1 、c 2 、c 3 …c n It can be understood as the input layer of the model.

[0118] In this embodiment, in step 3, in the process of representing the subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation ellipsoid, the output information corresponding to the natural language question is extracted from the knowledge graph embedding information based on the context of the natural language question;

[0119] Determine the extended information corresponding to the output information based on the relationship representation and the entity representation;

[0120] A subgraph is generated based on the output information and the extended information.

[0121] In one embodiment, in the process of extracting a subgraph related to the input text, a graph traversal algorithm can be used to extract the subgraph related to the input text. For example, a depth-first search (DFS) or a breadth-first search (BFS) starts from the starting node in the knowledge graph and traverses according to certain rules and constraints. During the traversal process, entities, relationships, or attribute nodes related to the input text are selected according to the context and task requirements, and are added to the subgraph.

[0122] Based on the context information of the input text, similarity matching or keyword matching methods are used to filter out nodes and edges related to the text. For example, word vectors or text feature representations can be used to calculate the similarity between nodes and the input text, and nodes with high similarity are selected as part of the subgraph.

[0123] During the graph traversal, the scope of the subgraph can be expanded by defining path constraints and restrictions. For example, based on the relationships between entities in the input text, the subgraph can be expanded to include other entities and relationship nodes on these relationship paths. This increases the richness of the subgraph and provides a more comprehensive knowledge background.

[0124] In one embodiment, after knowledge embedding and subgraph extraction are completed, the information of the knowledge graph needs to be integrated into the input of the model to provide richer context and semantic knowledge, thereby improving the reasoning and generation capabilities of the model. The following methods can be used to integrate subgraph information and guide the large model to focus on specific knowledge points.

[0125] Depending on the entities and relationships in the subgraph, they can be described in the prompt in natural language. For example, if the subgraph involves character relationships, descriptions like "X is the parent of Y" or "X and Y are siblings" can be used to guide the large model to focus on family relationships.

[0126] In addition, special marking symbols can be used to represent subgraph information. For example, using entity marking symbols (such as [ENTITY]) and relationship marking symbols (such as [RELATION]) to mark entities and relationships in the subgraph in the prompt can clearly instruct the large model to pay attention to the important information in the subgraph; if the subgraph contains attribute information, these attributes can be mentioned in the prompt, or question-guided prompt statements can be designed to integrate the information in the subgraph into the question. For example, you can use question guidance similar to "Based on the information in the subgraph, what relationship do you think X is?" or "What is the relationship between entities X and Y in the subgraph?" to guide the large model to pay attention to the entities and relationships in the subgraph.

[0127] In one embodiment, since a lot of information in real life is changing, it is possible to separate the knowledge space and the language space and inject knowledge information into the reasoning stage. Since the acquisition, update and application of knowledge change according to the context and demand when processing information, dynamic knowledge fusion is required, such as using a dual-tower structure, one for processing input text and the other for processing the input of related knowledge graphs. In order to make the interaction between the two more sufficient, direct splicing or adding an attention mechanism module can also be used.

[0128] In another embodiment, retrieval-augmented knowledge fusion is a method for processing external knowledge, which combines non-parametric and parametric modules to better process external knowledge. Taking RAG (Retrieval-Augmented Generation) as an example, the knowledge graph related to the input text is first retrieved in the non-parametric module through maximum inner product search (MIPS) to obtain some documents. Then, using Seq2Seq LLMs, these documents are used as hidden variables z and provided to the output generator as additional contextual information. By using different retrieval documents as conditions in different generation steps, rather than just using a single document to guide the entire generation process. Retrieval-enhanced knowledge fusion can better utilize external knowledge and improve the quality and diversity of generated text.

[0129] The differences between the present invention and the prior art and the corresponding beneficial effects include:

[0130] (1) Entity / relation-based representation learning methods: Traditional knowledge graphs often rely on triple embedding representations when processing user queries, which may lead to incomplete matches with user inputs. User queries usually contain target entities and their related question expressions, not just the entities themselves. This difference may affect the accuracy and efficiency of queries. Therefore, this paper generates question embeddings by simulating user questions, and then fuses these embeddings with triple embeddings to narrow the semantic differences between user inputs and knowledge graph nodes.

[0131] (2) Subgraph-enhanced hint method: In the current knowledge graph application field, triples are directly input into the large language model as prior knowledge. However, the large language model has limited understanding of the logical relationship and semantic structure between triples, which directly affects the model's ability to generate accurate answers based on this prior knowledge. Therefore, this paper converts the retrieved triple information base into natural language text through methods such as triple parsing and natural language template design to improve its understanding and application of prior knowledge.

[0132] Embodiment 2:

[0133] The present invention based on the same inventive concept also provides a large model and knowledge graph fusion system, such as Figure 4 As shown, including:

[0134] An embedding module is configured to obtain a knowledge graph and an entity relationship learning model, and perform knowledge graph embedding on the knowledge graph to generate knowledge graph embedding information;

[0135] An output module is configured to generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output an entity representation and a relationship representation corresponding to the natural language question;

[0136] an extraction module, configured to extract a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation;

[0137] The fusion module is configured to convert the sub-graph into natural language and integrate it into the large model by combining the entity relationship learning model and the knowledge graph embedding information to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the large model.

[0138] The extraction module is further configured to:

[0139] Extracting output information corresponding to the natural language question from the knowledge graph embedding information based on the context of the natural language question;

[0140] Determine the extended information corresponding to the output information based on the relationship representation and the entity representation;

[0141] A subgraph is generated based on the output information and the extended information.

[0142] The embedded module is further configured as follows:

[0143] Constructing a relation embedding space by using the TransR method, and mapping entities of the knowledge graph to the relation embedding space;

[0144] Establishing a projection matrix based on the relationship between the triples of the knowledge graph;

[0145] Calculating the distance between the head entity and the tail entity in the triplet by a scoring formula;

[0146] Based on the triples in the knowledge graph, replacing the original head entity or tail entity with a random entity, constructing a negative example set of the triples;

[0147] Knowledge graph embedding information is generated according to the relation embedding space, the projection matrix, the distance between the head entity and the tail entity, and the negative example set of the triples.

[0148] The fusion module is further configured as follows:

[0149] Inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, and learning the entity relationship representation in the knowledge graph embedding information through the entity relationship learning model;

[0150] Based on the natural language question, the entity representation and relationship representation corresponding to the natural language question are determined from the knowledge graph embedding information through the entity relationship learning model, and output.

[0151] Embodiment 3:

[0152] The present invention based on the same inventive concept also provides an application method of a target reasoning model based on knowledge graph fusion, such as Figure 5 As shown, including:

[0153] Step 1: Obtain the context corresponding to the user question and answer information, and extract question and answer keywords from the user question and answer information;

[0154] Step 2: Substitute the question-and-answer keywords into the pre-built target reasoning model to obtain the user question-and-answer output information corresponding to the user question-and-answer information

[0155] Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any one of claims 1 to 4.

[0156] In this embodiment, the process of obtaining the context corresponding to the user question and answer information in step 1 includes:

[0157] The knowledge graph corresponding to the user question and answer information is retrieved in a search enhancement manner to obtain question and answer extension information, and the question and answer extension information is used as the context corresponding to the user question and answer information.

[0158] In this embodiment, in step 2, the process of substituting the question and answer keywords into the pre-built target reasoning model to obtain the user question and answer output information corresponding to the user question and answer information includes:

[0159] Outputting entity representation and relationship representation corresponding to the user question and answer information based on the question and answer keywords through an entity relationship learning model in a pre-built target reasoning model;

[0160] Extracting a first question-answer subgraph corresponding to the user question-answer information based on the entity representation, the relationship representation, and the context of the user question-answer information;

[0161] User question and answer output information corresponding to the user question and answer information is generated based on the first question and answer subgraph.

[0162] In this embodiment, after generating the user question and answer output information corresponding to the user question and answer information based on the first question and answer subgraph in step 2, the method further includes:

[0163] After receiving the user updated question and answer information, extracting the user updated keywords from the user updated question and answer information according to the similarity between the user question and answer information and the user updated question and answer information;

[0164] Extracting a second question-and-answer subgraph corresponding to the user-updated question-and-answer information based on the user-updated keyword and the first question-and-answer subgraph;

[0165] Output the user question and answer output information corresponding to the user updated question and answer information based on the second question and answer subgraph.

[0166] In one embodiment, the subgraph is dynamically updated according to the task requirements and the changes in the input text. When the input text changes, the graph traversal and subgraph extraction can be performed again to maintain the consistency and real-time performance of the subgraph and the input text, and in order to make the generation of the new subgraph closer to the output text corresponding to the user input text, the second question and answer subgraph can be extracted and generated based on the previous user question and answer information and the generated first question and answer subgraph.

[0167] The present invention combines the language description generated by the large model with the structural information of the knowledge graph, which improves the interpretability of the system, makes the reasoning process more transparent, and improves the accuracy of the reasoning results.

[0168] Embodiment 4:

[0169] The present invention based on the same inventive concept also provides an application system of a target reasoning model based on knowledge graph fusion, such as Figure 6 As shown, including:

[0170] An information extraction module, used to obtain the context corresponding to the user question and answer information, and extract question and answer keywords from the user question and answer information;

[0171] An output information module, used to substitute the question-and-answer keywords into a pre-built target reasoning model to obtain user question-and-answer output information corresponding to the user question-and-answer information;

[0172] Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any one of claims 1 to 4.

[0173] Preferably, the output information module is further used to:

[0174] Outputting entity representation and relationship representation corresponding to the user question and answer information based on the question and answer keywords through an entity relationship learning model in a pre-built target reasoning model;

[0175] Extracting a first question-answer subgraph corresponding to the user question-answer information based on the entity representation, the relationship representation, and the context of the user question-answer information;

[0176] User question and answer output information corresponding to the user question and answer information is generated based on the first question and answer subgraph.

[0177] Preferably, the system further comprises:

[0178] A subgraph updating module, configured to extract user update keywords from the user update question and answer information according to the similarity between the user question and answer information and the user update question and answer information after receiving the user update question and answer information;

[0179] Extracting a second question-and-answer subgraph corresponding to the user-updated question-and-answer information based on the user-updated keyword and the first question-and-answer subgraph;

[0180] Output the user question and answer output information corresponding to the user updated question and answer information based on the second question and answer subgraph.

[0181] Preferably, the information extraction module is further used to:

[0182] The knowledge graph corresponding to the user question and answer information is retrieved in a search enhancement manner to obtain question and answer extension information, and the question and answer extension information is used as the context corresponding to the user question and answer information.

[0183] Example 5

[0184] like Figure 7 As shown, the present invention also provides an electronic device, which may be a computer device, a single-chip device, an intelligent mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, the processor, and the transceiver component are connected via a bus; the memory may be used to store an execution program, and an exemplary execution program may include instructions; the processor is used to execute the instructions stored in the memory. The memory may also be used to store data, which may be called and / or modified when the instructions are executed.

[0185] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in a storage medium to implement the corresponding method flow or corresponding function, so as to realize a large model and knowledge graph fusion method and a target reasoning model application method based on knowledge graph fusion in the above-mentioned embodiment.

[0186] Example 6

[0187] Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the storage medium here can include both built-in storage media in electronic devices and, of course, extended storage media supported by electronic devices. The storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more execution programs (including program codes). It should be noted that the storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk storage. The processor loads and executes one or more instructions stored in the storage medium, which can implement the steps of a large model and knowledge graph fusion method and a target reasoning model application method based on knowledge graph fusion in the above-mentioned embodiment.

[0188] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0192] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are included in the scope of the claims of the present invention to be approved.

Claims

1. A method for fusing a large model with a knowledge graph, characterized in that: include: Obtain a knowledge graph and an entity relationship learning model, and perform knowledge graph embedding on the knowledge graph to generate knowledge graph embedding information; Generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output an entity representation and a relationship representation corresponding to the natural language question; Extracting a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation; The subgraph is converted into natural language and combined with the entity relationship learning model and the knowledge graph embedding information into the big model to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the big model.

2. The method according to claim 1, characterized in that The extracting a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation includes: Extracting output information corresponding to the natural language question from the knowledge graph embedding information based on the context of the natural language question; Determine the extended information corresponding to the output information based on the relationship representation and the entity representation; A subgraph is generated based on the output information and the extended information.

3. The method according to claim 1, characterized in that The step of inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model and outputting entity representation and relationship representation corresponding to the natural language question includes: Inputting the knowledge graph embedding information and the natural language question into an entity relationship learning model, and learning the entity relationship representation in the knowledge graph embedding information through the entity relationship learning model; Based on the natural language question, the entity representation and relationship representation corresponding to the natural language question are determined from the knowledge graph embedding information through the entity relationship learning model, and output.

4. A large model and knowledge graph fusion system, characterized in that: include: An embedding module, used to obtain a knowledge graph and an entity relationship learning model, and to embed the knowledge graph into the knowledge graph to generate knowledge graph embedding information; An output module, configured to generate a natural language question based on the knowledge graph embedding information, input the knowledge graph embedding information and the natural language question into an entity relationship learning model, and output an entity representation and a relationship representation corresponding to the natural language question; An extraction module, configured to extract a subgraph corresponding to the natural language question based on the natural language question, the entity representation, and the relationship representation; A fusion module is used to convert the subgraph into natural language and integrate it into the large model by combining the entity relationship learning model and the knowledge graph embedding information to generate a target reasoning model, thereby realizing the fusion of the knowledge graph and the large model.

5. An application method of a target reasoning model based on knowledge graph fusion, characterized in that: include: Obtaining a context corresponding to the user question and answer information, and extracting question and answer keywords from the user question and answer information; Substituting the question-and-answer keywords into a pre-built target reasoning model to obtain user question-and-answer output information corresponding to the user question-and-answer information; Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any one of claims 1 to 3.

6. The method according to claim 5, characterized in that Substituting the question and answer keywords into a pre-built target reasoning model to obtain user question and answer output information corresponding to the user question and answer information includes: Outputting entity representation and relationship representation corresponding to the user question and answer information based on the question and answer keywords through an entity relationship learning model in a pre-built target reasoning model; Extracting a first question-answer subgraph corresponding to the user question-answer information based on the entity representation, the relationship representation, and the context of the user question-answer information; User question and answer output information corresponding to the user question and answer information is generated based on the first question and answer subgraph.

7. The method according to claim 6, characterized in that After generating the user question and answer output information corresponding to the user question and answer information based on the first question and answer subgraph, the method further includes: After receiving the user updated question and answer information, extracting the user updated keywords from the user updated question and answer information according to the similarity between the user question and answer information and the user updated question and answer information; Extracting a second question-and-answer subgraph corresponding to the user-updated question-and-answer information based on the user-updated keyword and the first question-and-answer subgraph; Output the user question and answer output information corresponding to the user updated question and answer information based on the second question and answer subgraph.

8. An application system of a target reasoning model based on knowledge graph fusion, characterized in that: include: An information extraction module, used to obtain the context corresponding to the user question and answer information, and extract question and answer keywords from the user question and answer information; An output information module, used to substitute the question-and-answer keywords into a pre-built target reasoning model to obtain user question-and-answer output information corresponding to the user question-and-answer information; Wherein, the target reasoning model is constructed by a large model and knowledge graph fusion method as described in any one of claims 1 to 3.

9. An electronic device, characterized in that: include: at least one processor and memory; The memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the large model and knowledge graph fusion method described in any one of claims 1 to 3 and the application method of the target reasoning model based on knowledge graph fusion described in any one of claims 5 to 7 are implemented.

10. A readable storage medium, characterized in that: An execution program is stored thereon, and when the execution program is executed, it implements the large model and knowledge graph fusion method as described in any one of claims 1 to 3 and the application method of the target reasoning model based on knowledge graph fusion as described in any one of claims 5 to 7.

Images