Knowledge fusion method and device based on knowledge graph and large language model

Abstract

The invention relates to the technical field of data processing, and particularly discloses a knowledge fusion method and device based on a knowledge graph and a large language model. According to the method, the unknown knowledge of the large language model can be determined according to the preset knowledge graph, the to-be-learned knowledge is determined in the knowledge graph data according to the unknown knowledge, the large language model is trained according to the to-be-learned knowledge, and fusion of the unknown knowledge and the known knowledge of the large language model is realized. The knowledge graph can provide reliable factual knowledge, the knowledge to be learned in the knowledge graph data is automatically injected into the large language model through knowledge detection and knowledge injection, manual operation is not needed, and the processing efficiency of the large language model for processing a natural language processing task and the accuracy and credibility of a processing result are ensured.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a knowledge fusion method and device based on a knowledge graph and a large language model. Background Art

[0002] Large language models are natural language processing models based on deep learning. By training on large-scale text data, they can demonstrate strong understanding and generation capabilities in a variety of language tasks. Large language models, such as GPT-3 and GPT-4, predict the next word through autoregression to generate coherent text. These models have achieved remarkable results in tasks such as text generation, translation, summarization, and question-answering. However, since large language models mainly rely on statistical methods to predict the next word, there is no built-in mechanism to verify the authenticity of the generated content. This leads to the fact that in practical applications, large models may generate inaccurate or false information, misleading users.

[0003] Knowledge graph is a technology that stores and displays structured information of knowledge in the form of graphs. It represents entities and their relationships in the form of nodes and edges. Knowledge graphs are widely used in various fields, such as search engines, recommendation systems, and intelligent question-answering, to improve the accuracy and efficiency of information retrieval. The core of knowledge graph technology lies in its ability to efficiently store, manage, and query a large amount of factual knowledge and support complex logical reasoning. However, when dealing with natural language understanding and generation tasks, knowledge graphs cannot directly process unstructured data and require a lot of manual intervention to update and maintain the knowledge base, resulting in low efficiency in natural language processing tasks.

[0004] In summary, current natural language processing tasks have problems with inaccurate results or low processing efficiency. Therefore, how to improve the efficiency of natural language processing tasks and the accuracy of processing results has become an urgent problem to be solved. Summary of the invention

[0005] The present application provides a knowledge fusion method and device based on a knowledge graph and a large language model to improve the efficiency of natural language processing tasks and the accuracy of processing results.

[0006] In a first aspect, the present application provides a knowledge fusion method based on a knowledge graph and a large language model, the method comprising: Acquire knowledge graph data, and obtain a knowledge detection data set based on the knowledge graph data; Based on the knowledge detection data set, performing knowledge detection on a preset large language model to determine known knowledge and unknown knowledge of the large language model; Determine the knowledge to be learned in the knowledge graph data based on the unknown knowledge; The knowledge to be learned is injected into the large language model to integrate the unknown knowledge and the known knowledge.

[0007] In a second aspect, the present application further provides a knowledge fusion device based on a knowledge graph and a large language model, the device comprising: A detection data set acquisition module is used to acquire knowledge graph data and obtain a knowledge detection data set based on the knowledge graph data; An unknown knowledge determination module, used to perform knowledge detection on a preset large language model based on the knowledge detection data set, and determine known knowledge and unknown knowledge of the large language model; A module for determining knowledge to be learned, used for determining knowledge to be learned in the knowledge graph data based on the unknown knowledge; The to-be-learned knowledge fusion module is used to inject the to-be-learned knowledge into the large language model to fuse the unknown knowledge and the known knowledge.

[0008] In a third aspect, the present application also provides a computer device, comprising a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the knowledge fusion method based on the knowledge graph and the large language model as described above when executing the computer program.

[0009] In a fourth aspect, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the knowledge fusion method based on the knowledge graph and the large language model as described above.

[0010] The present application discloses a knowledge fusion method and device based on a knowledge graph and a large language model, which obtains knowledge graph data, and based on the knowledge graph data, obtains questions to be answered and the correct answers corresponding to the questions to be answered; processes the questions to be answered based on a preset large language model to obtain predicted answers to the questions to be answered; determines the unknown knowledge of the large language model based on the correct answers and the predicted answers, and determines the knowledge to be learned in the knowledge graph data based on the unknown knowledge; injects the knowledge to be learned into the large language model to fuse the unknown knowledge with the known knowledge of the large language model. The present application can determine the unknown knowledge of the large language model based on the preset knowledge graph, and determine the knowledge to be learned in the knowledge graph data based on the unknown knowledge, train the large language model based on the knowledge to be learned, and realize the fusion of unknown knowledge with the known knowledge of the large language model. The knowledge graph can provide reliable factual knowledge, and automatically inject the knowledge to be learned in the knowledge graph data into the large language model through knowledge detection and knowledge injection, without manual operation, ensuring the processing efficiency of the large language model in processing natural language processing tasks and the accuracy and credibility of the processing results. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] Figure 1 It is a structural diagram of a knowledge fusion system based on a knowledge graph and a large language model provided in an embodiment of the present application; Figure 2 This is a schematic flow chart of a first embodiment of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application; Figure 3 is a schematic flow chart of a second embodiment of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application; Figure 4 It is a schematic flow chart of a third embodiment of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application; Figure 5 A schematic block diagram of a knowledge fusion device based on a knowledge graph and a large language model provided in an embodiment of the present application; Figure 6 A schematic block diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0015] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0016] It should be further understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0017] The embodiment of the present application provides a knowledge fusion method and device based on a knowledge graph and a large language model. Among them, the knowledge fusion method based on a knowledge graph and a large language model can be applied to a server, and through knowledge detection and knowledge injection, the knowledge to be learned in the knowledge graph data is automatically injected into the large language model without manual operation, thereby ensuring the processing efficiency of the large language model in processing natural language processing tasks and the accuracy and credibility of the processing results. Among them, the server can be an independent server or a server cluster.

[0018] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0019] See also Figure 1 , Figure 1 1 is a schematic diagram of a knowledge fusion system based on a knowledge graph and a large language model provided in an embodiment of the present application. The knowledge fusion system based on a knowledge graph and a large language model is used to execute a knowledge fusion method based on a knowledge graph and a large language model. The knowledge fusion system based on a knowledge graph and a large language model can be configured on a server.

[0020] like Figure 1As shown in the figure, the knowledge fusion system based on knowledge graph and large language model includes knowledge detection module, knowledge injection module, knowledge adapter module and multimodal fusion module. Among them, the knowledge detection module identifies the knowledge blind spots of the large language model by generating multiple choice questions and initial knowledge state evaluation; the knowledge adapter module includes multiple parallel adapters for learning new knowledge and keeping the original model parameters unchanged; the knowledge injection module includes a knowledge injector model for intelligently injecting new knowledge and adjusting the amount of knowledge to avoid affecting the existing knowledge system; the multimodal fusion module is used to process multiple data modalities and improve the comprehensive processing capabilities of the system.

[0021] In one embodiment, the knowledge detection module converts the triples in the knowledge graph (e.g., <entity 1, relationship, entity 2>) into multiple-choice questions, thereby generating a knowledge detection dataset. The multiple-choice questions include questions to be answered, interference options, and correct answers. For example, given the triple<Paris, is the capital of, France> , generating the question "Paris is the capital of which country?" The answers include France, Germany, Italy, etc. The questions to be answered in the generated knowledge detection dataset are input into the large language model, and the initial knowledge state of the model is evaluated through its output to identify the known and unknown knowledge areas of the model.

[0022] In one embodiment, the knowledge injection module is used to identify the current knowledge state of the large language model and determine whether the large language model needs new knowledge based on the current knowledge state. If the model has already mastered a certain knowledge point, the impact of the knowledge injector is minimized; if the model lacks a certain knowledge point, more relevant knowledge is injected. According to the current knowledge state of the model, the amount of knowledge injected is dynamically adjusted to ensure that the integration of knowledge does not affect the original knowledge system of the model.

[0023] In one embodiment, a knowledge adapter module, including multiple parallel adapters, is designed in the last few layers of a large language model to be trained independently of the original model parameters to minimize the impact on the existing model parameters. New knowledge patterns are learned and stored, and the new knowledge is integrated with the original knowledge.

[0024] In one embodiment, the multimodal fusion module processes text data as well as non-text data such as images, audio, and video. By introducing a multimodal encoder, the non-text data is converted into virtual text tokens and aligned with the text input of the large language model. Combining data from different modalities improves the system's knowledge integration and generation capabilities.

[0025] See also Figure 2 , Figure 2This is a schematic flow chart of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application. The knowledge fusion method based on a knowledge graph and a large language model can be applied to a server to automatically inject the knowledge to be learned in the knowledge graph data into the large language model through knowledge detection and knowledge injection, without manual operation, thereby ensuring the processing efficiency of the large language model in processing natural language processing tasks and the accuracy and credibility of the processing results.

[0026] like Figure 2 As shown, the knowledge fusion method based on the knowledge graph and the large language model specifically includes steps S101 to S104.

[0027] S101, acquiring knowledge graph data, and obtaining a knowledge detection data set based on the knowledge graph data; In one embodiment, the knowledge graph stores and displays structured information of knowledge in the form of a graph, represents entities and their relationships in the form of nodes and edges, identifies the nodes and edges in the knowledge graph, and obtains at least one knowledge triple, such as <entity 1, relationship, entity 2>.

[0028] In one embodiment, the knowledge triples in the knowledge graph data are converted into multiple-choice questions, with each knowledge triple corresponding to one multiple-choice question. After all the knowledge triples in the knowledge graph data are converted, a knowledge detection data set is obtained.

[0029] S102: Based on the knowledge detection data set, perform knowledge detection on a preset large language model to determine known knowledge and unknown knowledge of the large language model; In one embodiment, the knowledge detection data set includes a plurality of multiple-choice questions, each of which includes a question to be answered, answer options, and a correct answer corresponding to the question to be answered.

[0030] In one embodiment, the questions to be answered in the knowledge detection data set are used as input parameters of the large language model, and the questions to be answered are processed by the large language model to obtain the predicted answers of the large language model.

[0031] In one embodiment, the predicted answer is compared with the correct answer to determine whether the answer predicted by the large language model is correct. If the predicted answer is correct, it indicates that the knowledge corresponding to the question to be answered is known knowledge of the large language model. If the predicted answer is wrong, it indicates that the knowledge corresponding to the question to be answered is unknown knowledge of the large language model.

[0032] S103, determining knowledge to be learned in the knowledge graph data based on the unknown knowledge; In one embodiment, the knowledge corresponding to the unknown knowledge in the knowledge graph data is used as the knowledge to be learned. For example, if the question to be answered is "Paris is the capital of which country?", and the predicted answer of the large language model is Germany, it means that the predicted answer is wrong, and "Paris is the capital of which country?" or "Which city is the capital of France?" is the unknown knowledge of the large language model. "Paris is the capital of France" in the knowledge graph data is used as the knowledge to be learned.

[0033] S104: injecting the knowledge to be learned into the large language model to integrate the unknown knowledge and the known knowledge.

[0034] In one embodiment, the knowledge to be learned is injected into the large language model. Specifically, through the knowledge adapter module of the large language model, model training is performed according to the knowledge to be learned to achieve learning of the knowledge to be learned.

[0035] In one embodiment, the newly learned knowledge is integrated with the known knowledge of the large language model to achieve the fusion of unknown knowledge and known knowledge.

[0036] See also Figure 3 , Figure 3 It is a schematic flow chart of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application.

[0037] like Figure 3 As shown, the knowledge fusion method based on the knowledge graph and the large language model specifically includes steps S201 to S202.

[0038] S201. Based on the knowledge graph data, obtain at least one knowledge triple; Furthermore, the obtaining of at least one knowledge triple based on the knowledge graph data includes: respectively obtaining non-text data and text data in the knowledge graph data; converting the non-text data into text tokens; and obtaining at least one knowledge triple based on the text data and the text tokens.

[0039] In one embodiment, the knowledge graph stores and displays structured information of knowledge in the form of a graph, represents entities and their relationships in the form of nodes and edges, identifies the nodes and edges in the knowledge graph data, and obtains at least one knowledge triple, such as <entity 1, relationship, entity 2>.

[0040] In one embodiment, the knowledge graph data may include text data and non-text data, such as images, audio, and video, etc. Through the multimodal fusion module, these non-text data are converted into virtual text tokens and aligned with the text input of the large language model, so that the system can play a role in more application scenarios and improve the comprehensive processing capabilities.

[0041] In a specific embodiment, different types of data stored in a knowledge graph are identified and distinguished. A knowledge graph generally contains text data (such as descriptions, tags, attribute values, etc.) and non-text data (such as multimedia content such as images, sounds, and videos).

[0042] In one embodiment, text data and non-text data are extracted separately. Text data can be used directly to construct questions and answers. Non-text data requires further processing. Specifically, non-text data is converted into text tokens through a multimodal fusion module. For example, if an image is converted into descriptive text, the converted text description will be decomposed into smaller units through natural language processing techniques such as word segmentation and standardization to obtain text tokens. These text tokens will serve as virtual representations of non-text data, so that non-text data that could not be directly used for text processing can participate in the construction and use of knowledge graphs in the form of text.

[0043] In one embodiment, knowledge triples are constructed using text data and text tokens converted from non-text data. For example, if the non-text data is a picture, the converted text tokens are descriptions of the picture, which can be combined with entities and relationships in the knowledge graph to form triples. Specifically, entities such as names, places, organizations, etc. are identified in the text data and text tokens. Then, the relationships between entities are determined, such as "belongs to", "is located in", "is", etc., to form a complete triple (entity 1, relationship, entity 2).

[0044] S202. Based on at least one of the knowledge triples, obtain at least one question to be answered and a correct answer corresponding to the question to be answered, and generate the knowledge detection data set.

[0045] In one embodiment, entities (such as names of people, places, etc.) and relationships between entities (such as "born in", "is the capital city", etc.) in knowledge triples are identified.

[0046] In one embodiment, questions to be answered are generated based on knowledge triples. Specifically, a predefined question template may be used. For example, if the triple is ("Albert Einstein", "born in", "Germany"), the template "In which country was [entity] born?" may be used to generate the question "In which country was Albert Einstein born?" In one embodiment, for each generated question to be answered, the correct answer is extracted from the corresponding knowledge triple. For example, in the above example, the correct answer is "Germany".

[0047] In one embodiment, in order to construct an effective knowledge detection dataset, several wrong options (distractors) are generated for each question. These options should be similar to the correct answer in logic or common sense, but are wrong in fact. Diverse wrong options are generated to fully test the discrimination ability of the large language model.

[0048] In one embodiment, the generated questions to be answered, correct answers, and incorrect options are integrated to form a knowledge detection data set, and the data set is verified to ensure the accuracy of the questions and answers, as well as the effectiveness of the interference items.

[0049] For example, according to the knowledge triple<Paris, is the capital of, France> , generating the question to be answered: "Paris is the capital of which country?" The answer options include France, Germany, Italy, etc. The correct answer is France, among which Germany and Italy are wrong options (distractor options).

[0050] Furthermore, based on the knowledge detection data set, knowledge detection is performed on a preset large language model to determine the known knowledge and unknown knowledge of the large language model, including: processing the question to be answered based on the large language model to obtain a predicted answer to the question to be answered; based on the correct answer and the predicted answer, determining the known knowledge and the unknown knowledge.

[0051] In one embodiment, the questions to be answered in the knowledge detection data set are input one by one into a preset large language model. The large language model receives the questions and processes and analyzes the questions using the knowledge and algorithms obtained through its training to generate predicted answers.

[0052] In one embodiment, the predicted answers generated by the large language model are matched and compared with the correct answers in the knowledge detection dataset. The accuracy of the predicted answers is evaluated to determine how well the model understands the question.

[0053] In one embodiment, if the predicted answer provided by the large language model is consistent with or very close to the correct answer, it is considered that the model has mastered the knowledge point, and the knowledge point is used as the known knowledge of the model.

[0054] In one embodiment, if the predicted answer does not match the correct answer, it indicates that the model is missing or insufficient in this knowledge point, and this knowledge point is regarded as unknown knowledge of the model.

[0055] See also Figure 4 , Figure 4 It is a schematic flow chart of a knowledge fusion method based on a knowledge graph and a large language model provided in an embodiment of the present application.

[0056] like Figure 4 As shown, before step S104, the knowledge fusion method based on the knowledge graph and the large language model specifically also includes steps S301 to S302.

[0057] S301, based on a preset knowledge injector, evaluating the current knowledge state of the large language model; In one embodiment, the knowledge injector is a pre-set evaluation tool for analyzing and evaluating the current knowledge state of a large language model. Specifically, by inputting a specific query or question, the knowledge injector evaluates the model's answering ability and identifies the knowledge coverage and depth.

[0058] In one embodiment, the model's answers are compared with the information in the knowledge graph to map out the model's knowledge structure. The model's performance in different knowledge areas is evaluated to identify its strengths and weaknesses and determine the current knowledge state of the large language model.

[0059] S302, based on the current knowledge state, determining the target learning knowledge of the large language model and the knowledge amount of the target learning knowledge in the knowledge to be learned; In one embodiment, it is determined whether the large language model needs new knowledge based on the current knowledge state. If the model has already mastered a certain knowledge point, the influence of the knowledge injector is minimized; if the model lacks a certain knowledge point, more relevant knowledge is injected.

[0060] In one embodiment, the amount of knowledge injected is dynamically adjusted according to the current knowledge state of the model, and a learning strategy is generated to ensure that the learning of unknown knowledge and the integration of unknown knowledge with known knowledge will not affect the original knowledge system of the model.

[0061] In one embodiment, the current knowledge level and learning ability of a large language model are evaluated by analyzing its performance on various tasks. Performance indicators such as accuracy, recall, and F1 score are used to measure the model's mastery of existing knowledge.

[0062] In one embodiment, based on the performance of the model on a specific task, it is determined whether the model needs to learn new knowledge to improve its performance. A performance threshold is set, and when the performance of the model is lower than the threshold, it is considered that the model needs to learn new knowledge.

[0063] In one embodiment, knowledge domains related to the current performance shortcomings of the model are obtained from the knowledge to be learned, and the knowledge domains most related to the model application scenario are selected as target learning knowledge.

[0064] In one embodiment, the amount of knowledge that the model needs to learn is determined. Specifically, the amount of knowledge can be determined based on the complexity of the unknown knowledge domain or the specific requirements of the model.

[0065] S303: Generate a learning strategy for the knowledge to be learned based on the target learning knowledge and the amount of knowledge, so as to inject the knowledge to be learned into the large language model based on the learning strategy.

[0066] In one embodiment, based on the target learning knowledge and knowledge amount, a specific learning strategy is designed, which may include learning sequence, emphasis and methods, and how to allocate computing resources and time to optimize the learning process.

[0067] In one embodiment, the learning strategy can be dynamically adjusted according to the model learning progress and effect.

[0068] Furthermore, injecting the knowledge to be learned into the large language model includes: based on the knowledge adapter module of the large language model, performing model training according to the learning strategy to learn the target learning knowledge, and integrating the target learning knowledge with the known knowledge.

[0069] In one embodiment, the target learning knowledge is gradually injected into the large language model according to the learning strategy.

[0070] In a specific embodiment, the determined target learning knowledge is input into the knowledge adapter module in an appropriate format. The input knowledge is cleaned, standardized and other preprocessing steps are performed to ensure that it is suitable for model training. The training process is initialized using the adapter module, including setting parameters such as training cycle and batch size.

[0071] In one embodiment, the knowledge adapter module includes a parallel adapter, which can independently learn new knowledge and effectively improve the efficiency of knowledge fusion.

[0072] In one embodiment, during the training process, the adapter module merges the newly learned knowledge with the existing knowledge base of the model to ensure that there is no conflict when the new and old knowledge are merged, and to maintain the consistency and accuracy of the knowledge base.

[0073] In one embodiment, the progress and performance indicators of model training are monitored in real time to ensure that the training proceeds as planned. The performance of the model on new knowledge and the integration effect with known knowledge are regularly evaluated. According to the monitoring results, the training strategy is adjusted to improve the learning efficiency and the quality of knowledge integration. In addition, the parameters of the adapter module are fine-tuned to better adapt to changes in the learning process. The new knowledge obtained in training and testing is updated to the knowledge base of the model.

[0074] In one embodiment, the knowledge adapter module is a parallel adapter added to the last few layers of the large language model, which can be trained independently of the original parameters of the large language model to minimize the impact on the existing model parameters. Through the knowledge adapter module, unknown knowledge can be learned and stored, and unknown knowledge can be integrated with known knowledge.

[0075] See also Figure 5 , Figure 5 The embodiment of the present application provides a schematic block diagram of a knowledge fusion device based on a knowledge graph and a large language model, which is used to execute the aforementioned knowledge fusion method based on a knowledge graph and a large language model. The knowledge fusion device based on a knowledge graph and a large language model can be configured on a server.

[0076] like Figure 5 As shown, the knowledge fusion device 400 based on the knowledge graph and the large language model includes: The detection data set acquisition module 401 is used to acquire knowledge graph data and obtain a knowledge detection data set based on the knowledge graph data; An unknown knowledge determination module 402 is used to perform knowledge detection on a preset large language model based on the knowledge detection data set to determine known knowledge and unknown knowledge of the large language model; A to-be-learned knowledge determination module 403 is used to determine the to-be-learned knowledge in the knowledge graph data based on the unknown knowledge; The to-be-learned knowledge fusion module 404 is used to inject the to-be-learned knowledge into the large language model to fuse the unknown knowledge and the known knowledge.

[0077] Furthermore, the detection data set acquisition module 401 includes: A knowledge triple acquisition unit, used for acquiring at least one knowledge triple based on the knowledge graph data; The detection data set generation unit is used to obtain at least one question to be answered and the correct answer corresponding to the question to be answered based on at least one of the knowledge triples, and generate the knowledge detection data set.

[0078] Furthermore, the knowledge triplet acquisition unit includes: A non-text data acquisition subunit, used to respectively acquire non-text data and text data in the knowledge graph data; a non-text data conversion subunit, for converting the non-text data into text tokens; The knowledge triplet obtaining subunit is used to obtain at least one of the knowledge triples based on the text data and the text tokens.

[0079] Furthermore, the unknown knowledge determination module 402 includes: A predicted answer obtaining unit, used for processing the question to be answered based on the large language model to obtain a predicted answer to the question to be answered; The knowledge determination unit is used to determine the known knowledge and the unknown knowledge based on the correct answer and the predicted answer.

[0080] Furthermore, the knowledge fusion device 400 based on the knowledge graph and the large language model further includes a learning strategy generation module, and the learning strategy generation module includes: A current knowledge state evaluation unit, used for evaluating the current knowledge state of the large language model based on a preset knowledge injector; A target knowledge and knowledge amount determination unit, used to determine the target learning knowledge of the large language model and the knowledge amount of the target learning knowledge in the knowledge to be learned based on the current knowledge state; A learning strategy generating unit is used to generate a learning strategy for the knowledge to be learned based on the target learning knowledge and the knowledge amount, so as to inject the knowledge to be learned into the large language model based on the learning strategy.

[0081] Furthermore, the to-be-learned knowledge fusion module includes: A model training unit is used to perform model training based on the knowledge adapter module of the large language model according to the learning strategy to learn the target learning knowledge and integrate the target learning knowledge with the known knowledge.

[0082] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and each module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0083] The above-mentioned device can be implemented in the form of a computer program. Figure 6 Runs on the computer device shown.

[0084] See also Figure 6 , Figure 6 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0085] See also Figure 6 The computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0086] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any knowledge fusion method based on the knowledge graph and the large language model.

[0087] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0088] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any knowledge fusion method based on the knowledge graph and the large language model.

[0089] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0090] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0091] In one embodiment, the processor is used to run a computer program stored in the memory to implement the following steps: Acquire knowledge graph data, and obtain a knowledge detection data set based on the knowledge graph data; Based on the knowledge detection data set, performing knowledge detection on a preset large language model to determine known knowledge and unknown knowledge of the large language model; Determine the knowledge to be learned in the knowledge graph data based on the unknown knowledge; The knowledge to be learned is injected into the large language model to integrate the unknown knowledge and the known knowledge.

[0092] In one embodiment, when obtaining a knowledge detection data set based on the knowledge graph data, the processor is used to implement: Based on the knowledge graph data, obtaining at least one knowledge triple; Based on at least one of the knowledge triples, at least one question to be answered and a correct answer corresponding to the question to be answered are obtained, and the knowledge detection data set is generated.

[0093] In one embodiment, when the processor acquires at least one knowledge triple based on the knowledge graph data, it is configured to implement: Respectively obtain non-text data and text data in the knowledge graph data; converting the non-text data into text tokens; At least one of the knowledge triples is obtained based on the text data and the text tokens.

[0094] In one embodiment, when the processor performs knowledge detection on a preset large language model based on the knowledge detection data set to determine known knowledge and unknown knowledge of the large language model, it is used to implement: Processing the question to be answered based on the large language model to obtain a predicted answer to the question to be answered; Based on the correct answer and the predicted answer, the known knowledge and the unknown knowledge are determined.

[0095] In one embodiment, before injecting the knowledge to be learned into the large language model to merge the unknown knowledge and the known knowledge, the processor is further configured to implement: Based on a preset knowledge injector, evaluating the current knowledge state of the large language model; Based on the current knowledge state, determining target learning knowledge of the large language model and the knowledge amount of the target learning knowledge in the knowledge to be learned; Based on the target learning knowledge and the amount of knowledge, a learning strategy for the knowledge to be learned is generated, so as to inject the knowledge to be learned into the large language model based on the learning strategy.

[0096] In one embodiment, when injecting the knowledge to be learned into the large language model, the processor is used to implement: The knowledge adapter module based on the large language model performs model training according to the learning strategy to learn the target learning knowledge and integrate the target learning knowledge with the known knowledge.

[0097] A computer-readable storage medium is also provided in an embodiment of the present application, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and the processor executes the program instructions to implement any one of the knowledge fusion methods based on the knowledge graph and the large language model provided in the embodiments of the present application.

[0098] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0099] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A knowledge fusion method based on knowledge graph and large language model, characterized in that: include: Acquire knowledge graph data, and obtain a knowledge detection data set based on the knowledge graph data; Based on the knowledge detection data set, performing knowledge detection on a preset large language model to determine known knowledge and unknown knowledge of the large language model; Determine the knowledge to be learned in the knowledge graph data based on the unknown knowledge; The knowledge to be learned is injected into the large language model to integrate the unknown knowledge and the known knowledge.

2. The knowledge fusion method based on knowledge graph and large language model according to claim 1 is characterized in that: The step of obtaining a knowledge detection data set based on the knowledge graph data includes: Based on the knowledge graph data, obtaining at least one knowledge triple; Based on at least one of the knowledge triples, at least one question to be answered and a correct answer corresponding to the question to be answered are obtained, and the knowledge detection data set is generated.

3. The knowledge fusion method based on knowledge graph and large language model according to claim 2 is characterized in that: The acquiring at least one knowledge triple based on the knowledge graph data includes: Respectively obtain non-text data and text data in the knowledge graph data; converting the non-text data into text tokens; At least one of the knowledge triples is obtained based on the text data and the text tokens.

4. The knowledge fusion method based on knowledge graph and large language model according to claim 2 is characterized in that: The step of performing knowledge detection on a preset large language model based on the knowledge detection data set to determine known knowledge and unknown knowledge of the large language model includes: Processing the question to be answered based on the large language model to obtain a predicted answer to the question to be answered; Based on the correct answer and the predicted answer, the known knowledge and the unknown knowledge are determined.

5. The knowledge fusion method based on knowledge graph and large language model according to claim 1 is characterized in that: Before injecting the knowledge to be learned into the large language model to integrate the unknown knowledge and the known knowledge, the method further includes: Based on a preset knowledge injector, evaluating the current knowledge state of the large language model; Based on the current knowledge state, determining target learning knowledge of the large language model and the knowledge amount of the target learning knowledge in the knowledge to be learned; Based on the target learning knowledge and the amount of knowledge, a learning strategy for the knowledge to be learned is generated, so as to inject the knowledge to be learned into the large language model based on the learning strategy.

6. The knowledge fusion method based on knowledge graph and large language model according to claim 5 is characterized in that: The injecting the knowledge to be learned into the large language model comprises: The knowledge adapter module based on the large language model performs model training according to the learning strategy to learn the target learning knowledge and integrate the target learning knowledge with the known knowledge.

7. A knowledge fusion device based on knowledge graph and large language model, characterized in that: include: A detection data set acquisition module is used to acquire knowledge graph data and obtain a knowledge detection data set based on the knowledge graph data; An unknown knowledge determination module, used to perform knowledge detection on a preset large language model based on the knowledge detection data set, and determine known knowledge and unknown knowledge of the large language model; A module for determining knowledge to be learned, used for determining knowledge to be learned in the knowledge graph data based on the unknown knowledge; The to-be-learned knowledge fusion module is used to inject the to-be-learned knowledge into the large language model to fuse the unknown knowledge and the known knowledge.

8. The knowledge fusion device based on knowledge graph and large language model according to claim 7 is characterized in that: The detection data set acquisition module includes: A knowledge triple acquisition unit, used to acquire at least one knowledge triple based on the knowledge graph data; The detection data set generation unit, based on at least one of the knowledge triples, the user obtains at least one question to be answered and the correct answer corresponding to the question to be answered, and generates the knowledge detection data set.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the knowledge fusion method based on the knowledge graph and the large language model as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the knowledge fusion method based on a knowledge graph and a large language model as described in any one of claims 1 to 7.

Images