Large model question answering method, device and equipment for multi-source heterogeneous knowledge base
Through the big model question-and-answer method for multi-source heterogeneous knowledge base, using key information extraction and entity linking models, the problem of insufficient accuracy and timeliness in knowledge question-and-answer is solved, and the accuracy of answers and domain knowledge is improved. Visibility.
Patent Information
- Application Number
- CN202410615911.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-17
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-05-17
AI Technical Summary
The big model has the limitation of accurately capturing factual knowledge in knowledge Q&A. Generalization leads to low correctness of answers, and may generate content that does not match the facts, and is insufficient timeliness.
Using a big-model question-and-answer approach for multi-source heterogeneous knowledge base, through key information extraction and entity linking models, low-rank adaptive method and SimCSE fine-tuning model, key information is extracted and entity information is matched from multi-source heterogeneous knowledge base. Enhance Q&A skills.
It improves the accuracy of big model question and answers, enhances the visibility and timeliness of domain knowledge, and avoids the problems of generalization errors and insufficient timeliness.
Smart Images

Figure CN118503494B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of knowledge question answering, and in particular to a large-model question answering method, device and equipment for multi-source heterogeneous knowledge bases. Background Art
[0002] At present, large models (i.e., large language models) such as ChatGPT and LLama have achieved good results in areas such as knowledge question answering due to their emergence ability and generalization. However, large models still have certain limitations for knowledge question answering. Specifically, since large models are black box models, they are usually unable to accurately capture and obtain factual knowledge, and large models are pre-trained with knowledge from multiple sources. The knowledge content they store is relatively generalized. For a question, it may not be possible to accurately obtain factual knowledge, resulting in a low accuracy rate of the answer generated by the final model.
[0003] Therefore, it is necessary to develop a large model question answering method, device and equipment for multi-source heterogeneous knowledge base to enhance the knowledge question answering ability of large models. Summary of the invention
[0004] In view of the above problems, the embodiments of the present application provide a large-model question-answering method, apparatus and device for a multi-source heterogeneous knowledge base, so as to overcome the above problems or at least partially solve the above problems.
[0005] In a first aspect of an embodiment of the present application, a large model question answering method for a multi-source heterogeneous knowledge base is provided, the method comprising:
[0006] Get the question information entered by the user;
[0007] Using a key information extraction model, extract key information from the question information, where the key information is an entity in the question information; the key information extraction model is a large model that uses supervised data and an entity extraction fine-tuning template and performs parameter fine-tuning through a low-rank adaptive method;
[0008] Using an entity linking model, the key information is matched with entities in a multi-source heterogeneous knowledge base to obtain incremental information; the multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method;
[0009] The incremental information and the question information are spliced using a knowledge question and answer template, and input into the original question and answer model to obtain an answer to the question information.
[0010] The second aspect of the embodiment of the present application further provides a large model question-answering device for a multi-source heterogeneous knowledge base, which is applied to the large model question-answering method for a multi-source heterogeneous knowledge base described in the first aspect of the embodiment of the present application, and the device includes:
[0011] A question information acquisition module is used to obtain question information input by the user;
[0012] A key information extraction module, used to extract key information from the question information using a key information extraction model, wherein the key information is an entity in the question information; the key information extraction model is a large model that uses supervised data and an entity extraction fine-tuning template and performs parameter fine-tuning through a low-rank adaptive method;
[0013] An entity matching module is used to use an entity linking model to perform entity matching between the key information and entities in a multi-source heterogeneous knowledge base to obtain incremental information; the multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method;
[0014] The answer generation module is used to use the knowledge question and answer template to splice the incremental information and the question information, input the original question and answer model, and obtain the answer to the question information.
[0015] The third aspect of the embodiment of the present application also provides an electronic device, including a memory, a processor and a computer program stored on the memory, and the processor executes the computer program to implement the steps in the large-model question-answering method for a multi-source heterogeneous knowledge base described in the first aspect of the embodiment of the present application.
[0016] The fourth aspect of the embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the large-model question-answering method for a multi-source heterogeneous knowledge base described in the first aspect of the embodiment of the present application are implemented.
[0017] The fifth aspect of the embodiments of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor implements the steps in the large-model question-answering method for a multi-source heterogeneous knowledge base as described in the first aspect of the embodiments of the present application.
[0018] The embodiment of the present application provides a large-model question-answering method for a multi-source heterogeneous knowledge base, the method comprising: obtaining question information input by a user; extracting key information from the question information using a key information extraction model, wherein the key information is an entity in the question information; the key information extraction model is a large model after parameter fine-tuning using a low-rank adaptive method using supervised data and an entity extraction fine-tuning template; matching the key information with entities in a multi-source heterogeneous knowledge base using an entity linking model to obtain incremental information; the multi-source heterogeneous knowledge base comprises at least a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of an original vector model using an unsupervised SimCSE fine-tuning method; concatenating the incremental information and the question information using a knowledge question-answering template, and inputting the result into the original question-answering model to obtain an answer to the question information.
[0019] The specific beneficial effects are:
[0020] The embodiment of the present application is to extract key information from the question information, and then match the corresponding entity information (i.e., incremental information) from the multi-source heterogeneous knowledge base based on the key information. On the one hand, the present application makes full use of the characteristics of local structured knowledge, such as visibility, confirmability, and easy transferability, and inputs the matched local knowledge (incremental information) together with the question information into the large model (original question-and-answer model) for knowledge question-and-answering, thereby effectively enhancing the ability of the large model to answer questions, achieving domain enhancement, and improving the accuracy of the model's answers. On the other hand, by utilizing the fact that local knowledge (multi-source heterogeneous knowledge base) is easier to update, it is easy for users to confirm and update the content of local knowledge, thereby avoiding the problem of insufficient timeliness of the large model. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0022] Figure 1 It is a flowchart of the steps of a large model question-answering method for a multi-source heterogeneous knowledge base provided in an embodiment of the present application;
[0023] Figure 2 It is a flowchart of a knowledge question and answer provided in an embodiment of the present application;
[0024] Figure 3 It is a schematic diagram of a parameter fine-tuning process of a low-rank adaptive method provided in an embodiment of the present application;
[0025] Figure 4 It is a structural schematic diagram of a large-model question-answering device for a multi-source heterogeneous knowledge base provided in an embodiment of the present application;
[0026] Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] The exemplary embodiments of the present application will be described in more detail below in conjunction with the accompanying drawings in the embodiments of the present application. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to enable the scope of the present application to be fully communicated to those skilled in the art.
[0028] Currently, large models (Large Language Model, LLM) such as ChatGPT and LLama have achieved good results in fields such as knowledge question answering due to their emergence ability and generalization. Therefore, large model question answering enhancement based on local knowledge may be the next research direction for large model related tasks.
[0029] Large models such as ChatGPT may have hundreds of billions or tens of billions of parameters, and their representation capabilities have also been greatly improved. And as the number of model parameters increases, the training cost required also increases, including more training resources, more unsupervised training data, and more sophisticated supervised training data. The initial cost enables the model to have better question-answering capabilities, as well as better downstream task migration capabilities. Generally, models with more than one billion parameters are called large models, and their model frameworks have not changed much. Most large models are improved on the basis of Transformer.
[0030] However, big models still have some limitations for knowledge question answering. First, because big models are black box models, they are usually unable to accurately capture and obtain factual knowledge. In addition, big models are pre-trained with knowledge from multiple sources, and the knowledge they store is relatively generalized. For a question, it may not be possible to accurately obtain factual knowledge.
[0031] In addition, large models may have hallucination problems. Large models may generate content that is inconsistent with user input, contradictory to previously generated content, or inconsistent with known world knowledge. The hallucination problem of large models may include the following reasons: insufficient amount of relevant data. When the amount of training data related to the large model field or problem is insufficient, no matter how much computing resources are invested, better results will not be achieved. Insufficient data may cause the model to learn false or wrong knowledge, thus generating hallucinations; insufficient data quality. When the amount of data increases, the quality of the data is difficult to guarantee. The model may learn wrong information and memorize it, and then may generate content that is inconsistent with the facts. At the same time, data inconsistency can also cause hallucinations; the reason for over-generalization of the model. During training, the large model may learn some rules or patterns that are inconsistent with actual knowledge, which is called over-generalization. When faced with new data, the rules or patterns learned during training may be incorrectly applied to the new data, thus generating hallucinations.
[0032] In addition, large models are generally pre-trained with corpus from a period of time, and the knowledge trained may be outdated and incomplete, so new knowledge cannot be used to answer questions. Constantly updating parameters with new knowledge to improve the timeliness of large models will incur high costs. When the knowledge content is constantly changing and may conflict with the original knowledge, there is uncertainty as to whether the new knowledge can be correctly acquired.
[0033] In view of the above problems, the embodiment of the present application proposes a large model question answering method for a multi-source heterogeneous knowledge base to enhance the knowledge question answering ability of the large model. The large model question answering method for a multi-source heterogeneous knowledge base provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings through some embodiments and their application scenarios.
[0034] The first aspect of the embodiment of the present application provides a large model question answering method for a multi-source heterogeneous knowledge base, referring to Figure 1 , Figure 1 A flowchart of the steps of a large model question answering method for a multi-source heterogeneous knowledge base provided in an embodiment of the present application is as follows: Figure 1 As shown, the method includes:
[0035] Step S101, obtaining question information input by the user.
[0036] Specifically, in an actual knowledge question-answering application scenario, question information is input by a user, which may be in the form of text input or voice input, etc., to convert the information input by the user into question information in text form.
[0037] Step S102, using a key information extraction model to extract key information from the question information, where the key information is the entities in the question information; the key information extraction model is a large model that uses supervised data and entity extraction fine-tuning templates and performs parameter fine-tuning through a low-rank adaptive method.
[0038] Specifically, through step S102, the key information in the question information is extracted. The form of the key information can be set according to the format of the multi-source heterogeneous knowledge base and the format of the expected question. In this embodiment, the form is set to the form of entity extraction. For the question information input by the user, which is often a question about one or more entities, the key information extraction model can be used to extract the most critical entity information from the question information. The key information is the entity in the question information, including person entity, place entity, time entity, etc. Figure 2 , Figure 2 A schematic diagram of a knowledge question-answering process is shown, such as Figure 2 As shown, the question information "What important contributions does Task A make in philosophy?" is first extracted through the key information extraction model to obtain the key information "Person A" (belonging to the person entity).
[0039] The key information extraction model is a large model that uses supervised data and entity extraction fine-tuning templates and performs parameter fine-tuning through a low-rank adaptive method. In this embodiment, a lightweight supervised fine-tuning based on a low-rank adaptive method (LoRA) is used to extract key information using a large model. The data fine-tuned by LoRA is supervised data labeled according to the form of key information. Figure 2 As shown, the Llama2 model is selected as the original question-answering model, and the LoRA fine-tuning method is adopted to train the Llama2 model with supervised data to obtain a key information extraction model. Exemplarily, the supervised data includes sample question information and key information (such as person entities, place entities, etc.) marked in the sample question information. In the training stage of the information extraction model, the original question-answering model (large model) is used for training relying on supervised data to obtain a domain-enhanced key information extraction model. In an embodiment of the present application, a large model refers to a large language model with more than one billion parameters.
[0040] Optionally, the large model used for training to extract key information in the embodiment of the present application can adopt the structure of the Llama (Large Language Model Meta AI) large model. Llama is an open and efficient large language model, whose data sets are all public data sets, and its structure is also a Decoder-only structure using the Transformer model, using a 32-layer Decoder. Llama2 is an improved model based on Llama, which uses more training data sets and has a larger context window capability. Its structure is the same as Llama. In the embodiment of the present application, the structure of the Chinese-Llama2-7b model can be selected as the basic model structure. Chinese-Llama2 refers to a series of Chinese models that are fine-tuned or use Llama2 based on Llama2. Due to the small proportion of pre-training corpus of Llama2, the Chinese ability of the original Llama2 is relatively weak. In order to improve the Chinese ability, two methods of fine-tuning or pre-training can be used. The embodiment of the present application selects the weight of the fine-tuned Flag Alpha's Llama2-Chinese-7b-Chat. Since it is fine-tuned based on Llama2, the model structure is the same as Llama.
[0041] Since large models have many parameters, overall adjustment requires less memory and is slower, so the embodiment of the present application proposes to adjust some parameters in the large model, that is, to adopt the method of efficient parameter fine tuning (PEFT).
[0042] In a possible implementation, the key information extraction model is trained by the following steps, that is, by using supervised data and entity extraction fine-tuning templates, and fine-tuning parameters by a low-rank adaptive method:
[0043] Step S201, based on the parameter matrix of the original large model, two low-rank matrices are generated. Specifically, the original large model may be a Llama (Large Language Model Meta AI) large model, and may further be Chinese-Llama2.
[0044] Step S202: Process the supervised data using the entity extraction fine-tuning template to obtain training data and corresponding labels.
[0045] Specifically, in the process of fine-tuning the parameters of the key information extraction model, it is necessary to add a template (entity extraction fine-tuning template) to perform simple processing on the input and output. First, design an entity extraction fine-tuning template based on the entity type to be extracted (i.e. the key information required). For example, the format of the entity extraction fine-tuning template based on Llama2 is as follows: Prompt=" <s> Human: Extract possible entities and entity types from the given text. The optional entity types are ['person', 'place']. Please answer in the format of (entity, entity type). Text: {instruction}\n< / s> <s>Assistant:".
[0046] In a possible implementation, the supervised data includes: sample question information, and key information in the sample question information; the step S202, using the entity extraction fine-tuning template to process the supervised data to obtain training data and corresponding labels, includes:
[0047] Step S2021: Use the entity extraction fine-tuning template to replace double quotation marks in the sample question information with single quotation marks.
[0048] Specifically, when the large model is fine-tuned with a small amount of data, the recognition and prediction effect of double quotation marks and other symbols in the input text (i.e., the sample question information input into the large model during the training process) is poor. Therefore, the embodiment of the present application proposes to convert double quotation marks and other symbols in the input text into single quotation marks before inputting into the large model.
[0049] Step S2022, using the entity extraction fine-tuning template, delete the double quotation marks at the boundaries of the list string in the supervised data, replace the double quotation marks of the key information in the sample question information with single quotation marks, delete the spaces in the list, and convert the boundaries of the sub-list into brackets.
[0050] Specifically, when the large model is fine-tuned with a small amount of data, the recognition and prediction effects of the double quotes at the boundaries of the list format, the double quotes in the output entity text, the spaces between the lists, and the square brackets at the boundaries of the sublists in the lists in the output results (i.e., the key information in the sample question information) are poor. Therefore, the embodiment of the present application proposes that before the output text (the key information in the sample question information) is input into the large model, the double quotes at the boundaries of the list string are deleted, the double quotes in the output entity text are replaced with single quotes, the spaces between the lists are deleted, and the boundaries of the sublists are converted into brackets.
[0051] Step S2023, using the entity extraction fine-tuning template, the sample question information, and the key information in the sample question information and the entity extraction fine-tuning template are spliced to obtain the training data and the corresponding labels.
[0052] Specifically, the entity extraction fine-tuning template, input (sample question information) and output (key information in sample question information) in the training data (supervised data) are concatenated as the input for the large model training. During concatenation, the input is replaced with the {instruction} part in the template. At the same time, since the large model is a decoder-only structure, the output (i.e. the corresponding label of the training data, i.e. the key information in the sample question information) should also be concatenated to the back of the input as the training data input. The output in the training data is used as the training output of the large model.
[0053] Step S203: input the training data into the large model, use the two low-rank matrices to perform forward propagation, and obtain the output result of the model.
[0054] Specifically, refer to Figure 3 , Figure 3 A schematic diagram of the parameter fine-tuning process of a low-rank adaptive method is shown, Figure 3 As shown, the training data (i.e. Figure 3 Input) in the large model (such as Figure 3 LLama2 model structure shown in the figure), forward propagation is performed in the large model (through the hidden layer hidde, attention layer atention, etc.), and the output result (such as Figure 3 Here, down, up, k, q, v, o, and gate are the original parameter matrices of the large model, and A and B represent two low-rank matrices generated based on the original parameter matrix.
[0055] Step S204, calculating a loss function value according to the output result and the label corresponding to the training data.
[0056] Step S205, according to the loss function value, the parameters in the two low-rank matrices are updated. Specifically, in this process, the model parameters of the large model itself are frozen, and only the parameters of the two low-rank matrices are updated. For multiple LoRA adjustable parameters in each layer of the model, the LoRA method will add two low-rank matrices for training and updating.
[0057] Step S206, using the new training data, conduct the next round of model training until a preset number of training times is reached or the loss function value converges.
[0058] Step S207, merging the two low-rank matrices after parameter update into the original model parameters through a parameter merging method to obtain a trained key information extraction model.
[0059] Specifically, the embodiment of the present application uses the low-rank adaptive method LoRA for efficient parameter fine-tuning. LoRA is a "plug and play" method that uses two low-rank matrices to replace a large matrix for fine-tuning. When fine-tuning the entire amount, all parameters of a matrix will be updated, and since fine-tuning is to update the parameters, it is better to directly simulate the update amount of the parameters. LoRA converts the update matrix into two low-rank matrices. When LoRA is fine-tuned, the original model parameters are frozen, and only the parameters of the two low-rank matrices are updated. After fine-tuning, they can be merged onto the large matrix, and the model fine-tuning is completed without increasing the number of model parameters. The rank of LoRA is an important hyperparameter. The larger the rank, the more trainable parameters and the stronger the expressive power.
[0060] Step S103, using the entity linking model, entity matching the key information with entities in the multi-source heterogeneous knowledge base to obtain incremental information. The multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base. The entity linking model is obtained by fine-tuning the parameters of the original vector model through the unsupervised SimCSE fine-tuning method.
[0061] like Figure 2 As shown, the entity linking model is used to match the extracted key information with local knowledge (entities in a multi-source heterogeneous knowledge base). The multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base and an unstructured knowledge base. Thus, the extracted key information can be matched with knowledge of multiple different structures and different data sources. The graph database refers to a database for storing graph data; the text database refers to a database that stores information in text form; the structured knowledge base refers to a local database that stores information of multiple structured entities, and for each entity stored, it includes multiple fields, including a text description of the entity. Local domain knowledge (entity information stored in a multi-source heterogeneous knowledge base) has rich information knowledge, and the local knowledge has the characteristics of visibility, confirmability, and easy transferability; the unstructured knowledge base refers to a knowledge base for storing unstructured data. Taking the character entity introduction of "Character A" as an example, the information of the entity "Character A" stored in the multi-source heterogeneous knowledge base includes a text description, such as personal information and main experience of character A. Therefore, the embodiment of the present application proposes to first extract the main key information (entity) from the question information, and then query the local knowledge related to the entity in the local multi-source heterogeneous knowledge base, combine the retrieved information (incremental information) with the question information, and input them into the large model for knowledge question and answering, thereby enhancing the ability to answer related questions based on the information in the local multi-source heterogeneous knowledge base.
[0062] In order to avoid the errors existing in entity linking using only the entity linking model, the embodiment of the present application proposes to use a method combining rules and entity linking models to perform entity linking. First, a part of entity linking candidates is determined by the rule-based entity linking method, and then entity linking is performed using the entity linking model method.
[0063] In a possible implementation, the step S103, using an entity linking model to perform entity matching on the key information and entities in a multi-source heterogeneous knowledge base to obtain incremental information, includes:
[0064] Step S1031, obtaining one or more candidate entities by matching from the multi-source heterogeneous knowledge base according to preset rules.
[0065] The entity itself includes some key attribute information. Based on the preset rules, the entity attributes are extracted and matched, and the key information is aligned with the corresponding entities in the multi-source heterogeneous knowledge base to find the candidate standard entity set (i.e., one or more candidate entities). Different rules may be used for different types of entity alignment. Taking the short entity of equipment type as an example, its corresponding attribute information mainly includes digital model, text model, equipment type, etc.
[0066] In a possible implementation, the step S1031, according to a preset rule, matches and obtains one or more candidate entities from the multi-source heterogeneous knowledge base, including:
[0067] Step S1031-a, extracting attributes from the key information according to preset rules to obtain attribute information of the key information; the attribute information at least includes: numerical model information, text model information, and equipment type information.
[0068] Specifically, the preset rules for extracting numerical model attributes are mainly used to extract numerical model attributes in entities. The extraction formula can be expressed as:
[0069] {Num-Type}<Type_Key> ;
[0070] Among them, {Num_Type} represents a numerical model consisting of symbols, letters and numbers.<Type_Key> The extracted model information is "C-17", where the attributes of the entity include symbols, numbers and letters.
[0071] The preset rules for text model attribute extraction are mainly used to extract text model attributes in entities. The extraction formula can be expressed as:
[0072] {Text-Type}<Type_Key> ;
[0073] Among them, {Text-Type} represents the text type, <type-key>The existing keywords are "model" and "number". Taking the entity "Reagan Yacht" as an example, the extracted text model information is "Reagan". The text model may also be marked with symbols, such as the double quotes in the "Reagan" yacht indicate the key field including the text model.
[0074] The preset rules for equipment type attribute extraction are mainly used to extract equipment type attributes from entities. The extraction formula can be expressed as:
[0075] {Type_Key};
[0076] Among them, {Type_Key} represents the equipment type in the entity. Taking the entity "C-17 transport aircraft" as an example, the extracted equipment type information is "transport aircraft".
[0077] Step S1031 - b , by calculating text similarity, matching the attribute information with the entities in the multi-source heterogeneous knowledge base to obtain the one or more candidate entities.
[0078] For each input entity (i.e., the key information extracted in step S102), its attributes (attribute information of the key information) are extracted, and then matched with the entities in the multi-source heterogeneous knowledge base through the attributes. Since one or more attribute information can be extracted from the key information in step S1031-a, in the process of attribute matching, the matching of the entire entity is judged by the matching of a single attribute, that is, as long as a single attribute information can be matched with a certain entity, the entity is determined as a candidate entity. Since the representations of different entities may be different for the same attribute, in order to retain a certain fault tolerance rate, a single attribute is matched using a similarity method. The relevant similarity calculation method can calculate the similarity of two paragraphs of text by converting the text into a set, and calculating the intersection and union of the two sets to calculate the similarity of the two sets. The embodiment of the present application takes into account that when aligning entities, the attributes of the input entity are more important than the standard entity (i.e., incremental information), and uses the following method to calculate the similarity of a single attribute, and the formula is as follows:
[0079] Similarity(A,B)=Intersection(A,B) / A;
[0080] Among them, A is the attribute information of the input entity (key information), and B is the attribute information of an entity in the multi-source heterogeneous knowledge base.
[0081] After matching all attributes of the input entity (key information), one or more candidate entities may be obtained. When there is only one candidate entity, the candidate entity is directly used as the result of entity alignment; when there are multiple candidate entities, these candidate entities are used as model candidates, and the entity linking model is used to select the most suitable entity as the result of entity alignment (and then the final matched entity to determine the incremental information).
[0082] Step S1032: utilizing the entity linking model to determine the incremental information according to the candidate entity.
[0083] like Figure 2 As shown, the embodiment of the present application can use the entity linking model to match key entities with entities in the knowledge base. In order to strengthen the entity linking capability in the field, the SimCSE (Simple Contrastive Learning of Sentence Embeddings) method is used to perform unsupervised fine-tuning of the original vector model based on the original vector model. Among them, SimCSE is a contrastive learning framework that is used to significantly improve the effect of vector representation through contrastive learning methods based on the original pre-trained model. The unsupervised data used for fine-tuning is the cleaned domain data.
[0084] In a possible implementation, the step S1032, using the entity linking model to determine the incremental information according to the candidate entity, includes:
[0085] Using the entity linking model, vector encoding is performed on the key information to obtain a key information vector;
[0086] Using the entity linking model, vector encoding is performed on the candidate entity to obtain a candidate entity vector;
[0087] Calculating the vector similarity between the key information vector and each of the candidate entity vectors;
[0088] The incremental information is determined according to the candidate entity corresponding to the highest vector similarity value. The candidate entity corresponding to the highest vector similarity value is determined as the standard entity, and then the information about the standard entity in the multi-source heterogeneous knowledge base is determined as the incremental information.
[0089] Specifically, in entity linking, the vector method can perform entity comparison by encoding entity vectors. Since the entities in entity alignment may include information about multiple words, similar to sentences, the embodiment of the present application uses the sentence vector method to represent entity vectors. In the sentence vector method, the related method mainly generates the vector of the sentence by summing the word vectors. This method does not use the interactive information between words. The Bert-based model can capture the information between words in sentences by stacking Transformer encoders, but its effect on the semantic representation of sentences is still limited.
[0090] In this embodiment, the original vector model used is the WWM model to better adapt to the vector encoding ability of Chinese. WWM (Pre-Training with Whole Word Masking for Chinese BERT) is a training method using whole word masking, which is applied to the pre-training model of Chinese and is an improvement based on BERT. This pre-training method has achieved better results than the BERT model on a number of Chinese natural language processing tasks. Compared with BERT, WWM uses the whole word masking method, which is not only to continuously mask entity words and phrases, but also to continuously mask all characters that can form Chinese words. The specific approach is that for Chinese, if part of the characters of a completed word are masked, the other parts of the word will also be masked, that is, all the Chinese characters that form the same word are masked, that is, the whole word masking. The purpose of doing this is that during the pre-training process, the model can learn the semantic information of Chinese words, and the encoding of each word obtained after training has the semantic information of the word, which is more friendly to different Chinese NLP tasks. The embodiment of the present application uses WWM as the original vector model to vector encode the entity. Based on this, the SimCSE method is used to perform unsupervised fine-tuning of the original vector model based on the original vector model.
[0091] SimCSE is divided into two categories, unsupervised SimCSE and supervised SimCSE. Supervised SimCSE puts positive pairs and negative pairs into a batch for contrastive learning. In a batch, a vector and its corresponding positive example constitute a positive pair, and all other vectors constitute a negative pair. While shortening the distance between positive pairs, it also shortens the distance between negative pairs. Unsupervised SimCSE optimizes the vector representation through contrastive learning, making it more uniform in spatial distribution and closer between positive pairs. The unsupervised method of SimCSE uses Dropout, an anti-overfitting technology in neural networks. Dropout randomly closes the connections of some neurons in the neural network. If the neurons closed are different, the final output results of the model will also be different. Therefore, SimCSE inputs a sentence into the same model twice, but uses two different dropouts. In this way, the vectors of the two sentences are different, but the semantics they represent are the same, and they can be used as positive pairs of the model. Then two different sentences are input into the same model respectively. The vectors of the two sentences are different, and the semantics they represent are different, and they are used as negative pairs of the model. In a batch, different dropouts representing the same entity are used as positive pairs, and the entity and other entities are used as negative pairs. The original vector model is trained through the SimCSE contrast learning method. While shortening the distance between positive pairs, the distance between negative pairs is increased, and the representation of data is better learned, so that its vector representation method has a better effect.
[0092] In the embodiment of the present application, the entity linking method based on SimCSE has better solved the anisotropy problem of vector representation in the pre-trained model (that is, the trained vector representation only occupies a narrow part of the vector space, limiting the representation ability of the vector). In related methods, the method to alleviate this problem is mainly post-processing and adding regularization during the training process, which cannot maintain good alignment while improving the uniformity of the vector. The embodiment of the present application adopts the contrastive learning framework of SimCSE and trains both positive and negative pairs at the same time, which better solves this problem and achieves good results in knowledge fusion.
[0093] Step S104, using the knowledge question and answer template to splice the incremental information and the question information, input the original question and answer model, and obtain the answer to the question information.
[0094] In actual knowledge question-answering applications, it is first necessary to determine whether key information has been obtained through information extraction in step S102, and to search for relevant knowledge information (incremental information) through entity linking in step S103. If no incremental information is found, a template needs to be created and combined with the question information to directly ask questions to the original question-answering model and give answers to the questions. If relevant knowledge information (incremental information) has been found, the knowledge information can be used to enhance the large model question-answering.
[0095] In this embodiment, the original question-answering model is used to answer questions that are spliced with knowledge. Specifically, the knowledge question-answering template is used to splice the knowledge and questions, and then they are input into the large model (original question-answering model) to generate answers. After obtaining the incremental information and question information, the entity extraction fine-tuning template is used to process the input text (incremental information and question information) according to steps S2021-S2023. For example, the double quotes at the boundaries of the list string are deleted, the double quotes for key information are replaced with single quotes, the spaces in the list are deleted, the boundaries of the sublists are converted to brackets, etc. Correspondingly, after obtaining the answer output by the original question-answering model, the predicted answer needs to be decoded. In actual prediction, since large models often have problems with missing or extra symbols, the symbols can be simply modified through scripts, and then the converted text format is converted to the type of list through the script to obtain the prediction result. As Figure 2 As shown in the figure, a template for question and answer based on relevant knowledge (knowledge question and answer template) is constructed, and the knowledge question and answer template, the question and the searched text knowledge (incremental information with structured information) are spliced (i.e. Figure 2 The original grammar model predicts and gives a large model answer based on the knowledge. For example, in this embodiment, the knowledge question answering template is " <s> Human: Answer the following questions based on the text below. Text: {instruction} Question: {question}\n< / s> <s>Assistant:". Instruction is the acquired knowledge (incremental information with structured information), and question is the question asked by the user.
[0096] In a possible implementation manner, before using the knowledge question and answer template to splice the incremental information and the question information, the method further includes:
[0097] Determine whether the information length of the incremental information exceeds the model context length limit;
[0098] When the information length of the incremental information exceeds the model context length limit:
[0099] Processing the incremental information by segmenting, constructing a segmentation template, or constructing an integration template; or
[0100] The incremental information is intercepted by a truncation method.
[0101] Specifically, for the searched relevant knowledge information (incremental information), it is necessary to determine the length of the relevant knowledge information. If its length exceeds the limit of the context length of a single model, if hardware conditions permit, it is necessary to perform segmentation, construct segmentation templates, construct integration templates and other methods to process the long text knowledge. Under general hardware conditions, the truncation method is used to extract the text knowledge.
[0102] In a possible implementation, the step of using a knowledge question and answer template to concatenate the incremental information and the question information, inputting the original question and answer model, and obtaining an answer to the question information includes:
[0103] The original question-answering model generates a first candidate answer based on the incremental information;
[0104] The original question-answering model performs knowledge retrieval based on the question information to generate a second candidate answer;
[0105] An answer to the question information is generated according to the first candidate answer and the second candidate answer.
[0106] The parameters used in the large model used in step S104 are different from those used in the key information extraction model of step S102. The key information extraction model of step S102 can be a large model based on the original question-answering model (such as the Chinese-Llama2 model) after fine-tuning the parameters through a low-rank adaptive method. The model used in step S104 is the original question-answering model (such as the Chinese-Llama2 model) whose parameters have not been fine-tuned. This allows the large model to retain its own knowledge and question-answering capabilities while adding new knowledge (the incremental information obtained in step S103), and can combine the knowledge of the knowledge base (incremental information) and its own knowledge to give answers to the domain questions input by the user.
[0107] In order to enhance the knowledge question-answering capability of the large model, the embodiment of the present application uses lightweight supervised fine-tuning based on LoRA to extract key information from the question information, and then uses the entity linking method based on SimCSE to match the corresponding entity information (i.e., incremental information) from the multi-source heterogeneous knowledge base according to the key information, and finally uses the structured local domain knowledge of the large model (incremental information matched in the multi-source heterogeneous knowledge base) to accurately answer questions in related fields. On the one hand, this application makes full use of the visibility, confirmability, and easy transferability of local structured knowledge, and inputs the matched local knowledge (incremental information) and question information into the large model (original question-answering model) for knowledge question-answering, thereby effectively enhancing the question-answering capability of the large model, achieving domain enhancement, and improving the accuracy of the model's answers. On the other hand, by utilizing the fact that local knowledge (multi-source heterogeneous knowledge base) is easier to update, it is convenient for users to confirm and update the content of local knowledge, thereby avoiding the problem of insufficient timeliness of the large model.
[0108] In order to verify the technical effect of the large model question answering method proposed in the embodiment of the present application, an experiment is conducted according to the above steps S101 to S104. The specific description is as follows:
[0109] In step S102, i.e., the information extraction step of LoRA fine-tuning, 176 domain entity extracted data are annotated, of which 153 are used for training and 23 are used for testing. In step S103, the unsupervised SimCSE method is used to fine-tune the entity linking model, without using a supervised training set, and 100 data are annotated for entity linking evaluation. In step S104, the original question-answering model is used for question-answering without training. Among them, the database used is a domain database, and the database is in the form of entity names and corresponding text descriptions. The number of entities in the database is 1,157. In order to evaluate the overall solution, 11 domain questions were created and annotated to evaluate the performance of the overall solution, and the effects of different solutions were compared by manually evaluating the answers of different solutions to these questions.
[0110] First, a LoRA fine-tuning experiment was conducted.
[0111] In step S102, i.e., the information extraction step of LoRA fine-tuning, the Chinese-Llama2 model was used for fine-tuning, and the Llama2 model was used as a comparative experiment. The experimental parameters of the two are shown in Table 1 below:
[0112] Parameter name Parameter Value Batch size 2 Lora rank (Lora rank parameter) 8 Lora dropout (Lora dropout parameter) 0.05 Lora alpha (Lora scaling parameter) 16 Num epochs (iteration number parameter) 5 Max source length 500 Max target length (maximum output length) 200 Warmup ratio (learning rate optimization parameter) 0.05 Learning rate 2e-5 Learning rate scheduler Constant with warmup Lora target modules All
[0113] Table 1
[0114] Regarding the maximum length of text input. Since the large model is a generative model, it will splice longer templates when inputting, and it also needs to convert the results into text and splice them to the input for training. Therefore, the maximum input length during actual training is 700. Since the model is only used for key information extraction, the result length is short, so the target length is set to 200. At the same time, the input length is only used to extract key information for the problem, and the problem length is set to 500. During the experiment, LoRA was used to fine-tune all target modules, including 7 types of matrices: down, up, k, q, v, o, and gate. Due to the large number of parameters in the large model, the warmup method was used, and the learning rate change method of Constant with warmup was used. When the learning rate reaches the maximum, it will not change, so as to learn more fully. Through experiments, it is found that the effect is better than the change method of using linear learning rate decrease. Due to time cost and computing resource cost, 8-bit quantization is used for large model training.
[0115] Table 2 is the experimental results table. It can be seen from the experimental results that when a large model is used in a new downstream task, it needs to learn its expression format and ability to handle tasks. In a very small number of cases, the format is inaccurate and it is impossible to automatically obtain an evaluable evaluation result. As can be seen from Table 2, Chinese-Llama2 has a better effect than Llama2 in some of the initial data, probably because it has some Chinese expression ability at the beginning, but when the number of training items increases, the training effect is basically the same.
[0116]
[0117] Table 2
[0118] It should be noted that although the structure of automatic evaluation is not ideal with very little data, this is a fine-tuning experiment on multiple entity types, including 20 entity types such as people, time, and place. If the local knowledge base or questions only focus on some important entities, the effect may become better. In addition, the experimental results here only represent the automatic evaluation results in a given format. For example, when the initial result is 0, it may be that the large model finds the entity, but has not learned the format that needs to be output, so it cannot be used as a positive example under the automatic evaluation method.
[0119] Then, a SimCSE fine-tuning experiment is performed. According to step S103, entity matching is performed using the entity linking model. In the experiment, WWM is used as the original vector model for unsupervised SimCSE fine-tuning to obtain the entity linking model. The experimental parameter settings are shown in Table 3 below:
[0120] Parameter name Parameter Value Batch size 16 Learning rate 1e-5 Temperature 0.05 Num epochs (iteration number parameter) 5 Optimizer AdamW Output Embedding Cls
[0121] Table 3
[0122] In this experiment, 6500 pre-processed domain sentences were selected for fine-tuning. After integrating the entity linking rules, 100 domain entities were evaluated for entity linking. The evaluation results are shown in Table 4 below:
[0123] Evaluation indicators Evaluation value Precision 0.7838 Recall 0.9355 F1 0.8530
[0124] Table 4
[0125] According to Table 4, we can see that the results of entity linking are relatively good, which may include the following reasons: the rule-based alignment method is used to align entities or find candidates for entity alignment based on entity attributes, which enhances the accuracy of entity alignment; the WWM-based model is used, and the full-word masking method of the model can better learn Chinese information during pre-training. SimCSE is used for unsupervised fine-tuning in the field, and the information in the field is better learned without supervised corpus.
[0126] Finally, according to step S104, the question-answering results are evaluated. In order to evaluate the overall solution, 11 domain questions are created and annotated to evaluate the performance of our overall solution. The effects of different solutions are compared by manually evaluating the answers to these questions. Since the generated sentences need to be manually defined, the severity of their errors may also vary. Five types of identifiers are set to evaluate the model's answers, and their definitions are shown in Table 5 below:
[0127]
[0128] Table 5
[0129] According to Table 5, the best result is Good, which is the correct answer. The second best result is Reject, which means the model did not find the contained knowledge and therefore refused to answer. Poor identifiers include Bad1 and Bad2, which means there are partial errors. The worst result is Bad-, which is a completely wrong answer. The evaluation results are shown in Table 6 below, where T indicates the type of the question according to the relevant header.
[0130]
[0131] Table 6
[0132] It should be noted that Article 11 is the result of the embodiment of the present application directly answering the question using the model without searching the key information in the knowledge base. Therefore, the answer result of this article is the same in the two cases. The purpose of Article 11 is only to verify the completeness of the embodiment of the present application and it is not counted in the effect comparison.
[0133] For the experimental results of the first 10 items, we can see the following advantages of the scheme proposed in the embodiment of the present application: when using the knowledge base information, 3 answers are completely correct, of which 7 knowledge bases contain relevant information. It can be seen that the complete successful utilization rate of the knowledge base information of this scheme is 3 / 7; in the comparison of direct answers, when the rejection of answers is better than the partial wrong answers, this scheme has a 90% improvement in effect compared with the direct use of the large model answer, including 2 partial information improvements (question 7 and question 8), 3 correct answers, and 4 rejection answers. The situation where the knowledge base has correct information but the answer is wrong still exists. There are 3 questions and answers with this situation, including question 7, question 8 and question 10, of which 2 groups gave partially correct information and 1 group was completely wrong. For the situation where there is no question-related knowledge in the knowledge base, this scheme can still improve the partial effect. At the same time, in order to verify the model's ability to judge ambiguous knowledge, we set up a test for question 4. The knowledge base of the key entity in this question includes ambiguous information related to the question, but it should not be used as the answer to the question. It is found that this scheme can also reject the answer well. For questions 2, 4, and 5, this solution gives a better answer of rejection.
[0134] In addition, it is found that the large model is also sensitive to the length of knowledge, as shown in Table 7 below:
[0135]
[0136] Table 7
[0137] The correct knowledge to answer question 1 is within 500 characters in length. When only the first 500 characters of knowledge are input into the model, the model can answer correctly. However, when the length is 4000 characters, the model cannot answer correctly. The possible reason is that too much knowledge has caused some interference to the model.
[0138] In summary, the large model question answering method for multi-source heterogeneous knowledge bases proposed in the embodiments of the present application can effectively use external knowledge to improve the question answering ability of the large model, solve some shortcomings of the large model itself having knowledge limitations, and solve the challenges of relying on unopened large models, entity link ambiguity, and key information ambiguity, including the following advantages:
[0139] This application only uses lightweight open source large models, does not need to rely on non-open source large models such as GPT4, can be deployed locally, and takes up less space. The embodiment of the present application adopts a step-by-step form, and a single step can be adjusted according to user needs. For example, the format of key information extraction in the first step can be set by yourself to meet user needs more specifically. The embodiment of the present application uses a more accurate key information extraction and entity linking method, which can more accurately find relevant knowledge. The logic of the embodiment of the present application is fixed, which avoids intermediate errors that may occur in the process, such as the query step provided by the large model itself.
[0140] The second aspect of the embodiment of the present application further provides a large model question-answering device for a multi-source heterogeneous knowledge base, which is applied to the large model question-answering method for a multi-source heterogeneous knowledge base described in the first aspect of the embodiment of the present application, referring to Figure 4 , Figure 4 A schematic diagram of the structure of a large model question-answering device for multi-source heterogeneous knowledge base is shown in FIG. Figure 4 As shown, the device comprises:
[0141] A question information acquisition module is used to obtain question information input by the user;
[0142] A key information extraction module, used to extract key information from the question information using a key information extraction model, wherein the key information is an entity in the question information; the key information extraction model is a large model that uses supervised data and an entity extraction fine-tuning template and performs parameter fine-tuning through a low-rank adaptive method;
[0143] An entity matching module is used to use an entity linking model to perform entity matching between the key information and entities in a multi-source heterogeneous knowledge base to obtain incremental information; the multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method;
[0144] The answer generation module is used to use the knowledge question and answer template to splice the incremental information and the question information, input the original question and answer model, and obtain the answer to the question information.
[0145] In a possible implementation, the key information extraction model is trained through the following steps:
[0146] Based on the parameter matrix of the original large model, two low-rank matrices are generated;
[0147] Using the entity extraction fine-tuning template, the supervised data is processed to obtain training data and corresponding labels;
[0148] Input the training data into the large model, use the two low-rank matrices to perform forward propagation, and obtain the output result of the model;
[0149] Calculate the loss function value according to the output result and the label corresponding to the training data;
[0150] According to the loss function value, updating the parameters in the two low-rank matrices;
[0151] Use the new training data to conduct the next round of model training until the preset number of training times is reached or the loss function value converges;
[0152] Through the parameter merging method, the two low-rank matrices after parameter update are merged into the original model parameters to obtain a trained key information extraction model.
[0153] In a possible implementation, the supervised data includes: sample question information, and key information in the sample question information; the method of using the entity extraction fine-tuning template to process the supervised data to obtain training data and corresponding labels includes:
[0154] Using the entity extraction fine-tuning template, the double quotation marks in the sample question information are replaced with single quotation marks;
[0155] Using the entity extraction fine-tuning template, delete the double quotation marks at the boundaries of the list string in the supervised data, replace the double quotation marks of the key information in the sample question information with single quotation marks, delete the spaces in the list, and convert the boundaries of the sublist into brackets;
[0156] The entity extraction fine-tuning template is used to splice the sample question information, key information in the sample question information and the entity extraction fine-tuning template to obtain the training data and corresponding labels.
[0157] In a possible implementation, the entity matching module includes:
[0158] A candidate entity matching submodule, used for matching and obtaining one or more candidate entities from the multi-source heterogeneous knowledge base according to preset rules;
[0159] The incremental information determination submodule is used to determine the incremental information from the candidate entity by using an entity linking model, wherein the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method.
[0160] In a possible implementation, the candidate entity matching submodule includes:
[0161] An attribute extraction unit, configured to extract attributes from the key information according to preset rules to obtain attribute information of the key information; the attribute information at least includes: numerical model information, text model information, and equipment type information;
[0162] The text similarity calculation unit is used to calculate text similarity and perform attribute matching between the attribute information and entities in the multi-source heterogeneous knowledge base to obtain the one or more candidate entities.
[0163] In a possible implementation manner, the incremental information determination submodule includes:
[0164] A first vector encoding unit, configured to perform vector encoding on the key information by using the entity linking model to obtain a key information vector;
[0165] A second vector encoding unit, configured to perform vector encoding on the candidate entity using the entity linking model to obtain a candidate entity vector;
[0166] A vector similarity calculation unit, used to calculate the vector similarity between the key information vector and each of the candidate entity vectors;
[0167] The incremental information determining unit is used to determine the incremental information according to the candidate entity corresponding to the highest vector similarity value.
[0168] In a possible implementation, the device further includes:
[0169] A length judgment module, used for judging whether the information length of the incremental information exceeds the model context length limit before splicing the incremental information and the question information using the knowledge question and answer template;
[0170] A processing module is configured to: when the information length of the incremental information exceeds the model context length limit:
[0171] Processing the incremental information by segmenting, constructing a segmentation template, or constructing an integration template; or
[0172] The incremental information is intercepted by a truncation method.
[0173] In a possible implementation, the answer generation module includes:
[0174] A first candidate answer generation submodule, configured for the original question-answering model to generate a first candidate answer based on the incremental information;
[0175] A second candidate answer generation submodule, used for the original question-answering model to perform knowledge retrieval based on the question information to generate a second candidate answer;
[0176] The answer generation submodule is used to generate an answer to the question information according to the first candidate answer and the second candidate answer.
[0177] The present application also provides an electronic device, referring to Figure 5 , Figure 5 Schematic diagram of an electronic device according to an embodiment of the present application. Figure 5 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory 110 and the processor 120 are connected via a bus communication. A computer program is stored in the memory 110. The computer program can be executed on the processor 120, thereby implementing the steps in the large-model question-answering method for a multi-source heterogeneous knowledge base disclosed in the embodiment of the present application.
[0178] An embodiment of the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps in the large-model question-answering method for a multi-source heterogeneous knowledge base as disclosed in the embodiment of the present application are implemented.
[0179] The embodiment of the present application also provides a computer program product. When the computer program product is run on an electronic device, the processor implements the steps of the large-model question-answering method for a multi-source heterogeneous knowledge base as disclosed in the embodiment of the present application.
[0180] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0181] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0182] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0183] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0184] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.
[0185] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.
[0186] The above is a detailed introduction to the large-model question-answering method, device and equipment for multi-source heterogeneous knowledge bases provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.< / s> < / s>
Claims
1. A large-model question-answering method for multi-source heterogeneous knowledge bases, characterized in that: The method comprises: Get the question information entered by the user; Using a key information extraction model, extract key information from the question information, where the key information is an entity in the question information; the key information extraction model is a large model that uses supervised data and an entity extraction fine-tuning template and performs parameter fine-tuning through a low-rank adaptive method; Using an entity linking model, the key information is matched with entities in a multi-source heterogeneous knowledge base to obtain incremental information; the multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method; The incremental information and the question information are spliced using a knowledge question and answer template, and input into the original question and answer model to obtain an answer to the question information.
2. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 1 is characterized in that: The key information extraction model is trained through the following steps: Based on the parameter matrix of the original large model, two low-rank matrices are generated; Using the entity extraction fine-tuning template, the supervised data is processed to obtain training data and corresponding labels; Input the training data into the large model, use the two low-rank matrices to perform forward propagation, and obtain the output result of the model; Calculate the loss function value according to the output result and the label corresponding to the training data; According to the loss function value, updating the parameters in the two low-rank matrices; Use the new training data to conduct the next round of model training until the preset number of training times is reached or the loss function value converges; Through the parameter merging method, the two low-rank matrices after parameter update are merged into the original model parameters to obtain a trained key information extraction model.
3. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 2 is characterized in that: The supervised data includes: sample question information, and key information in the sample question information; the supervised data is processed by using the entity extraction fine-tuning template to obtain training data and corresponding labels, including: Using the entity extraction fine-tuning template, the double quotation marks in the sample question information are replaced with single quotation marks; Using the entity extraction fine-tuning template, delete the double quotation marks at the boundaries of the list string in the supervised data, replace the double quotation marks of the key information in the sample question information with single quotation marks, delete the spaces in the list, and convert the boundaries of the sublist into brackets; The entity extraction fine-tuning template is used to splice the sample question information, key information in the sample question information and the entity extraction fine-tuning template to obtain the training data and corresponding labels.
4. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 1 is characterized in that: The entity linking model is used to match the key information with entities in a multi-source heterogeneous knowledge base to obtain incremental information, including: Obtaining one or more candidate entities by matching from the multi-source heterogeneous knowledge base according to preset rules; The incremental information is determined according to the candidate entities using the entity linking model.
5. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 4 is characterized in that: The one or more candidate entities are obtained by matching from the multi-source heterogeneous knowledge base according to the preset rules, including: Extracting attributes from the key information according to preset rules to obtain attribute information of the key information; the attribute information at least includes: numerical model information, text model information, and equipment type information; By calculating text similarity, the attribute information is matched with the entities in the multi-source heterogeneous knowledge base to obtain the one or more candidate entities.
6. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 4 is characterized in that: The determining the incremental information from the candidate entities by using the entity linking model includes: Using the entity linking model, vector encoding is performed on the key information to obtain a key information vector; Using the entity linking model, vector encoding is performed on the candidate entity to obtain a candidate entity vector; Calculating the vector similarity between the key information vector and each of the candidate entity vectors; The incremental information is determined according to the candidate entity corresponding to the highest vector similarity value.
7. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 1 is characterized in that: Before using the knowledge question and answer template to splice the incremental information and the question information, the method further includes: Determine whether the information length of the incremental information exceeds the model context length limit; When the information length of the incremental information exceeds the model context length limit: Processing the incremental information by segmenting, constructing a segmentation template, or constructing an integration template; or The incremental information is intercepted by a truncation method.
8. The large-model question-answering method for multi-source heterogeneous knowledge base according to claim 1 is characterized in that: The step of using the knowledge question and answer template to splice the incremental information and the question information, inputting the original question and answer model, and obtaining the answer to the question information includes: The original question-answering model generates a first candidate answer based on the incremental information; The original question-answering model performs knowledge retrieval based on the question information to generate a second candidate answer; An answer to the question information is generated according to the first candidate answer and the second candidate answer.
9. A large-model question-answering device for multi-source heterogeneous knowledge bases, characterized in that: A large model question answering method for a multi-source heterogeneous knowledge base applied to any one of claims 1 to 8, wherein the device comprises: A question information acquisition module is used to obtain question information input by the user; A key information extraction module, used to extract key information from the question information using a key information extraction model, wherein the key information is an entity in the question information; the key information extraction model is a large model that uses supervised data and an entity extraction fine-tuning template and performs parameter fine-tuning through a low-rank adaptive method; An entity matching module is used to use an entity linking model to perform entity matching between the key information and entities in a multi-source heterogeneous knowledge base to obtain incremental information; the multi-source heterogeneous knowledge base includes at least: a graph database, a text database, a structured knowledge base, and an unstructured knowledge base, and the entity linking model is obtained by fine-tuning the parameters of the original vector model through an unsupervised SimCSE fine-tuning method; The answer generation module is used to use the knowledge question and answer template to splice the incremental information and the question information, input the original question and answer model, and obtain the answer to the question information.
10. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor is executed, the steps in the large-model question-answering method for a multi-source heterogeneous knowledge base described in any one of claims 1 to 8 are implemented.
