Knowledge base query method and device, electronic equipment and storage medium
By introducing a learnable hint layer and similarity calculation into the large language model, the problem of insufficient understanding of the knowledge base by the large language model is solved, and more efficient and accurate query results are generated.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-06-20
- Publication Date
- 2026-07-31
AI Technical Summary
Large language models ignore the existing conditions of the knowledge base during the pre-training stage, resulting in insufficient understanding and utilization of professional domain knowledge and low accuracy of generated query results.
By introducing a learnable cue layer to fine-tune the large language model, and utilizing the existing conditions of the target knowledge base, candidate input sequences are generated and their similarity is calculated. Relevant text sequences are selected as contextual information for combination, and query results are output.
It improves the efficiency of large language models in utilizing the target knowledge base and the accuracy of query results, reduces system computation and storage overhead, and avoids catastrophic forgetting phenomena.
Smart Images
Figure CN116737879B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical data processing, and more particularly to a knowledge base query method, apparatus, electronic device, and storage medium. Background Technology
[0002] In the field of digital healthcare, Large Language Models (LLM) are combined with the knowledge bases (KB) of medical institutions to realize application scenarios such as online medical intelligent customer service and online consultation.
[0003] For example, a patient can initiate a medical consultation request to medical institution A from home via a smart terminal. Medical institution A can then respond and answer questions through a dedicated online medical consultation system (LLM+KB), including various formats such as text, voice, and images, to collect and record the patient's medical information for subsequent diagnosis and treatment.
[0004] A large language model (LLM) is a model that uses deep neural networks to learn the probability distribution of natural language. It can be used for various natural language processing tasks and can generate coherent and meaningful text based on given context or conditions. A knowledge base is a database that stores structured or semi-structured knowledge and can be used to provide factual, professional, or domain-specific knowledge.
[0005] In the process of realizing this invention, the inventors discovered that the prior art has at least the following problems:
[0006] During the pre-training phase, large language models primarily use unlabeled natural language text data and probability distribution-based sampling or greedy strategies to select the next word. This can easily overlook the existing conditions of the knowledge base (structured or semi-structured knowledge data and retrieval capabilities), resulting in insufficient understanding and utilization of the professional domain knowledge within the knowledge base. Consequently, the accuracy of the query results generated by the large language model is relatively low.
[0007] Therefore, how to quickly improve the large language model to make full use of the existing conditions of the target knowledge base in order to improve the accuracy of query results is an urgent problem to be solved. Summary of the Invention
[0008] In view of the above, it is necessary to provide a knowledge base query method, the purpose of which is to improve the accuracy of the query results of the large language model by making full use of the existing conditions of the target knowledge base.
[0009] The knowledge base query method provided by this invention includes:
[0010] Obtain at least one type of data from the initial knowledge base dataset, convert the type of data into the corresponding text sequence, and construct the target knowledge base based on all the text sequences;
[0011] When the preset large language model detects that it has received the initial input sequence for querying the target knowledge base, the first word of the initial input sequence is processed by the preset prompting layer to generate a candidate input sequence;
[0012] The first word of the initial input sequence is replaced with the second word of the candidate input sequence to update the initial input sequence into the target input sequence. The similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base is calculated to generate a similarity set.
[0013] Based on a preset similarity selection strategy, a preset number of text sequences are selected from the target knowledge base as context information obtained from the query based on the target input sequence. The target input sequence and the context information are combined to fine-tune the large language model and output the query results.
[0014] Optionally, the construction of the target knowledge base based on all text sequences includes:
[0015] Each text sequence is written into the file of the initial knowledge base;
[0016] A preset function is invoked to add newline characters to each of the text sequences in the file, thereby generating the target knowledge base.
[0017] Optionally, before processing the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence, the method further includes:
[0018] The prompting layer is added before the first connection layer of the large language model, and the learning parameters of the prompting layer are defined so that the prompting layer obtains the initial input sequence of the large language model.
[0019] Optionally, the step of performing feature processing on the first word of the initial input sequence through a preset prompt layer to generate a candidate input sequence includes:
[0020] The word that is first in the initial input sequence is taken as the first word;
[0021] The vector of the first word is input into the prompt layer for feature processing to generate the first feature vector;
[0022] The candidate input sequence is generated by fusing the first feature vector with the vector of the first word.
[0023] Optionally, replacing the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence to the target input sequence includes:
[0024] The first word in the candidate input sequence is taken as the second word;
[0025] The target input sequence is obtained by replacing the first word with the second word.
[0026] Optionally, the step of selecting a preset number of text sequences from the target knowledge base as context information obtained based on the target input sequence, according to a preset similarity selection strategy, includes:
[0027] The similarity values in the similarity set are sorted from largest to smallest, and the text sequences corresponding to the top K values in the sort are selected as the context information obtained from the query based on the target input sequence.
[0028] Optionally, combining the target input sequence and the context information to fine-tune the large language model and output query results includes:
[0029] The target input sequence and the context information are filled into a preset template;
[0030] The target input sequence and the context information are marked at the positions of the template, and the large language model is fine-tuned based on the template after the insertion marks to generate the query results.
[0031] To address the above problems, the present invention also provides a knowledge base query device, the device comprising:
[0032] The transformation and construction module is used to obtain at least one type of data from the initial knowledge base dataset, convert the type of data into the corresponding text sequence, and construct the target knowledge base based on all the text sequences;
[0033] The monitoring and processing module is used to process the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence when the preset large language model receives the initial input sequence for querying the target knowledge base.
[0034] The replacement and calculation module is used to replace the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence into a target input sequence, calculate the similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base, and generate a similarity set.
[0035] The selection and combination module is used to select a preset number of text sequences from the target knowledge base based on a preset similarity selection strategy as context information obtained from the query based on the target input sequence, and combine the target input sequence and the context information to fine-tune the large language model and output the query results.
[0036] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:
[0037] At least one processor; and,
[0038] A memory communicatively connected to the at least one processor; wherein,
[0039] The memory stores a knowledge base query program that can be executed by the at least one processor, and the knowledge base query program is executed by the at least one processor to enable the at least one processor to perform the above-described knowledge base query method.
[0040] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing a knowledge base query program, which can be executed by one or more processors to implement the aforementioned knowledge base query method.
[0041] Compared to existing technologies, this invention acquires at least one type of data from the initial knowledge base dataset, converts the type data into corresponding text sequences, and constructs a target knowledge base based on all text sequences. When the preset large language model receives the initial input sequence for querying the target knowledge base, the first word of the initial input sequence undergoes feature processing through a preset prompting layer to generate a candidate input sequence. The second word of the candidate input sequence replaces the first word of the initial input sequence to update the initial input sequence to the target input sequence. The similarity between the first vector of the target input sequence and the second vectors of each text sequence in the target knowledge base is calculated to generate a similarity set. Based on a preset similarity selection strategy, a preset number of text sequences are selected from the target knowledge base as context information obtained from the query based on the target input sequence. The target input sequence and context information are combined to fine-tune the large language model and output the query results. This invention converts all type data in the target knowledge base into a unified text sequence and uses an additional prompting layer to fine-tune the large language model, enabling the large language model to fully utilize the existing conditions of the target knowledge base and improving the accuracy of the query results.
[0042] This invention can be applied to fields such as medical data processing, enabling medical institutions' knowledge bases to be fully utilized by large language models, helping medical workers or patients to quickly find responses and answers. Attached Figure Description
[0043] Figure 1 This is a flowchart illustrating a knowledge base query method provided in an embodiment of the present invention;
[0044] Figure 2 This is a schematic diagram of a knowledge base query device provided in an embodiment of the present invention;
[0045] Figure 3 A schematic diagram of the structure of an electronic device for implementing a knowledge base query method according to an embodiment of the present invention;
[0046] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.
[0048] It should be noted that the descriptions involving "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.
[0049] This invention can be applied to fields such as medical data processing, enabling medical institutions' knowledge bases to be fully utilized by large language models, helping medical workers or patients to quickly find responses and answers.
[0050] In existing technologies, large speech models typically use small-scale labeled data related to the target task during the fine-tuning stage, while ignoring knowledge data related to the target domain or scenario, resulting in a lack of adaptation and optimization for specific domains or scenarios.
[0051] The core idea of this invention is to use the target knowledge base as an additional data source. Without changing the parameters of the original large speech model, a learnable prompt layer (Soft Prompt) is introduced to fine-tune the large speech model. This allows the large language model to make full use of the existing conditions of the target knowledge base, combining the knowledge of the large language model with the knowledge of the target knowledge base, thereby improving the accuracy of the query results of the large language model.
[0052] Reference Figure 1 The diagram shown is a flowchart illustrating a knowledge base query method according to an embodiment of the present invention. This method is executed by an electronic device.
[0053] In this embodiment, the knowledge base query method includes:
[0054] S1. Obtain at least one type of data from the initial knowledge base dataset, convert the type of data into a corresponding text sequence, and construct the target knowledge base based on all the text sequences;
[0055] In this embodiment, the dataset of the initial knowledge base refers to a data set in a certain domain, such as a data set in the field of medical institutions.
[0056] This invention will use the following example H as an example throughout:
[0057] For example, retrieve a predetermined number (e.g., 100,000) of historical outpatient data from the initial knowledge base (database) of medical institution A. This historical outpatient data includes the following:
[0058] 1. Patient basic information: including name, gender, age, ID number, contact information, etc.
[0059] 2. Medical record information: including medical history, diagnosis, treatment plan, medication, etc.
[0060] 3. Examination results: including the results of various examinations, such as complete blood count, urinalysis, blood glucose, electrocardiogram, CT scan, MRI, etc.
[0061] 4. Test results: This includes the results of various tests, such as blood biochemistry, immunology, and microbiology.
[0062] 5. Medical expenses: including registration fee, examination fee, treatment fee, medication fee, etc.
[0063] 6. Doctor Information: This includes the doctor's name, title, and license number.
[0064] 7. Medical equipment information: including the brand, model, and usage status of the medical equipment.
[0065] 8. Management information of medical institution A: including the name, address, legal representative, business license, etc. of medical institution A.
[0066] These historical outpatient data include triples, attribute graphs, and tables.
[0067] The dataset can also be a collection of data from other fields, and there is no limitation here.
[0068] In one embodiment, the type data includes triples, attribute graph nodes, and table rows, and the conversion of the type data into a corresponding text sequence includes:
[0069] If the data type is a triple, then the entities and relations in the triple are converted into a text sequence.
[0070] The dataset is D = {d1, d2, ..., d...} N}, where each d i It is either a document or a data item. Each d needs to be... i Convert to one or more text sequences Each s ij It is a string consisting of several words.
[0071] If d i Given a triple (e1, r, e2), where e1 and e2 are entities and r is a relation, convert the triple into a text sequence s. i ={The relationship between e1 and e2 is r}.
[0072] In one embodiment, the method further includes:
[0073] If the data type is an attribute graph node, then the entities, attributes, and values in the attribute graph node are converted into a text sequence.
[0074] If d i Given an attribute graph with nodes (e, a, v), where e is an entity, a is an attribute, and v is a value, convert the attribute graph into a text sequence s. i ={e's a is v}.
[0075] In one embodiment, the method further includes:
[0076] If the data type is a table row, then the cell values of each row in the table row are converted into a text sequence.
[0077] If d i It is a table row (c1, c2, ..., c K ), where c kIf the value is the cell value corresponding to column k, then the table row is converted into a text sequence s. i = {Column 1: c1, Column 2: c2, ..., Column K: c K}
[0078] To facilitate subsequent processing, this invention converts each element (such as entity, attribute, relation, etc.) of the initial knowledge base into its corresponding text representation, and converts each piece of information (such as triple, attribute graph node, table row, etc.) in the initial knowledge base into a text sequence.
[0079] In one embodiment, before constructing the target knowledge base based on all text sequences, the method further includes:
[0080] Add all text sequences to a preset list to obtain a list of text sequences.
[0081] In one embodiment, constructing the target knowledge base based on all text sequences includes:
[0082] Each text sequence is written into the file of the initial knowledge base;
[0083] A preset function is invoked to add newline characters to each of the text sequences in the file, thereby generating the target knowledge base.
[0084] Iterate through each text sequence in the text sequence list and write each text sequence into the file of the initial knowledge base. Each text sequence occupies one line in the file, and a newline character is added at the end of the line.
[0085] By calling a preset function, the initial knowledge base file is updated to generate the target knowledge base.
[0086] In other embodiments, the knowledge base can be represented in various forms, such as lists, dictionaries, and databases. For ease of subsequent processing, the target knowledge base can be stored in memory as a list or on disk as a file.
[0087] Continuing with example H above, a target knowledge base in the medical field is constructed by generating a text sequence from the historical outpatient data and medical data of the initial knowledge base of medical institution A.
[0088] In step S1, all types of data in the initial knowledge base are converted into corresponding text sequences, and the target knowledge base is constructed based on all text sequences. This allows each element and piece of information in the target knowledge base to be converted into a unified text sequence format, facilitating subsequent processing by large speech models and thus improving the utilization efficiency of the target knowledge base.
[0089] S2. When the preset large language model detects that it has received the initial input sequence for querying the target knowledge base, the first word of the initial input sequence is processed through the preset prompt layer to generate a candidate input sequence.
[0090] Large Language Models (LLMs) are models that use deep neural networks to learn the probability distribution of natural language. They can be used for various natural language processing tasks, such as text generation, text summarization, machine translation, and question answering.
[0091] Large language models are typically pre-trained using large-scale unlabeled text data (e.g., pre-trained by learning from data from various domains through a network) to capture general language knowledge and rules, and then fine-tuned for different tasks to adapt to specific domains and scenarios.
[0092] The soft prompt layer is a learnable parameter layer that generates a first feature vector (soft prompt vector) based on the initial input sequence received by the large language model, and adds the first feature vector to the vector of the initial input sequence, thereby changing the input representation of the large language model.
[0093] Soft suggestion vectors can be seen as an implicit text prefix that can guide large language models to generate appropriate query results based on specific domain tasks (such as the medical field mentioned above).
[0094] The cue layer can take many forms, such as linear transformation, multilayer perceptron, convolutional neural network, etc.
[0095] This invention does not require changing the parameters of the large language model (original LLM); it only requires introducing an additional cue layer to achieve fine-tuning of the large language model. This significantly reduces system computational and storage overhead while avoiding catastrophic forgetting and preserving the general knowledge of the large language model.
[0096] Continuing with example H above, medical institution A applies the pre-trained large language model to the target knowledge base of the medical institution for business processing. The initial input sequence of medical institution A to the large language model is - "What is leukemia";
[0097] By using a cue layer pre-set before the first connection layer (embedded glayer) of the large language model, the first word "what" in the initial input sequence is processed to generate a candidate input sequence.
[0098] In one embodiment, before performing feature processing on the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence, the method further includes:
[0099] The prompting layer is added before the first connection layer of the large language model, and the learning parameters of the prompting layer are defined so that the prompting layer obtains the initial input sequence of the large language model.
[0100] The first connection layer is typically an embedding layer, used to convert the discrete input words or characters into a continuous vector representation. The parameters of this embedding layer are usually learned during training.
[0101] Define the dimensions of the learning parameters for the prompting layer (e.g., 768 dimensions), and also create the word segmenter for the prompting layer and the model object for the large language model.
[0102] A tokenizer is used to segment the initial input sequence (text) from the user, facilitating subsequent processing and analysis. In the prompting layer, the tokenizer helps the system better understand the user's input, thereby providing more accurate and useful prompts.
[0103] In one embodiment, the step of performing feature processing on the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence includes:
[0104] The word that is first in the initial input sequence is taken as the first word;
[0105] The vector of the first word is input into the prompt layer for feature processing to generate the first feature vector;
[0106] The candidate input sequence is generated by fusing the first feature vector with the vector of the first word.
[0107] A word (Chinese word) is generally composed of one or more Chinese characters, such as: cold, medicine, human genome project, etc.
[0108] According to the order from front to back, read the first word in the initial input sequence (i.e. the first word in front) as the first word, add the first feature vector obtained after processing to the vector of the first word to generate the candidate input sequence.
[0109] In step S2, the first word of the initial input sequence undergoes feature processing through a preset prompt layer to generate a candidate input sequence. This improves the accuracy and reliability of the input sequence, thereby enhancing the efficiency and precision of knowledge base queries.
[0110] S3. Replace the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence to the target input sequence, calculate the similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base, and generate a similarity set;
[0111] In this embodiment, the first word of the initial input sequence is replaced by the first word (second word) in the candidate input sequence, so as to update the initial input sequence to the target input sequence;
[0112] Obtain each text sequence from the target knowledge base file, use the large language model as an encoder to encode the target input sequence and each text sequence to obtain the first vector of the target input sequence and the second vector of each text sequence, calculate the similarity between the first vector of the target input sequence and the second vector of each text sequence, add the calculated similarity to the similarity list, and generate a similarity set based on the similarity list.
[0113] Continuing with example H above, medical institution A inputs the initial input sequence "What is leukemia" into the large language model. By calculating the similarity between each text sequence in the target knowledge base and the target input sequence, a similarity set is obtained.
[0114] The target knowledge base may store the following text sequences: "Leukemia is a malignant tumor of the hematopoietic system", "The main symptoms of leukemia include fever, bleeding, anemia, and infection", and "Common treatments for leukemia include chemotherapy, radiotherapy, targeted therapy, and hematopoietic stem cell transplantation".
[0115] In one embodiment, replacing the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence to the target input sequence includes:
[0116] The first word in the candidate input sequence is taken as the second word;
[0117] The target input sequence is obtained by replacing the first word with the second word.
[0118] Read the first word in the input sequence (i.e., the first one in front) and use it as the second word.
[0119] Replace the first word with the second word in the initial input sequence, and use the resulting initial input sequence as the target input sequence.
[0120] This method replaces the first word in the initial input sequence with the second word from a candidate input sequence, thus updating the initial input sequence to the target input sequence. This improves the accuracy and coherence of the input sequence, providing users with a more convenient input experience. It can be widely applied in fields such as speech recognition and natural language processing.
[0121] In one embodiment, calculating the similarity between the first vector of the target input sequence and the second vectors of each text sequence in the target knowledge base, and generating a similarity set, includes:
[0122] The target input sequence is encoded using the large language model to obtain a first vector representation of the target input sequence;
[0123] The large language model is used to encode each of the text sequences to obtain a second vector representation corresponding to each of the text sequences;
[0124] Calculate the similarity between the first vector representation and each of the second vector representations, add the calculated similarity to the similarity list, and generate the similarity set.
[0125] In step S3, the similarity between the first vector of the target input sequence and the second vectors of each text sequence in the target knowledge base is calculated, generating a similarity set. This can effectively improve the accuracy and efficiency of text matching, providing a new solution for the field of text processing.
[0126] This invention does not require changing the parameters of the large language model (original LLM). It only requires introducing an additional cue layer to fine-tune the large language model and fully utilize the existing conditions of the target knowledge base. This significantly reduces system computational and storage overhead, while avoiding catastrophic forgetting and preserving the general knowledge of the large language model.
[0127] In existing technologies, large language models primarily use unlabeled natural language text data and probability distribution-based sampling or greedy strategies to select the next word during the pre-training stage. This approach tends to overlook the existing conditions of the knowledge base (structured or semi-structured knowledge data and retrieval capabilities), resulting in insufficient understanding and utilization of the domain knowledge within the knowledge base. Consequently, the accuracy of query results generated by large language models is relatively low.
[0128] S4. Based on a preset similarity selection strategy, a preset number of text sequences are selected from the target knowledge base as context information obtained from the query based on the target input sequence. The target input sequence and the context information are combined to fine-tune the large language model and output the query results.
[0129] In this embodiment, the present invention utilizes the large language model itself as an encoder to convert both the target input sequence and the text sequence in the target knowledge base into vector representations, and selects the most relevant information as context information based on cosine similarity. This leverages the semantic knowledge learned during the pre-training of the large language model to improve the accuracy and efficiency of retrieval.
[0130] Based on the similarity set selection strategy, a preset number of text sequences are selected as target input sequences for retrieval context information.
[0131] Continuing with example H above, medical institution A inputs the initial input sequence "What is leukemia" into the large language model, and selects the text sequence with the highest similarity to the target input sequence from the target knowledge base;
[0132] The text sequences with the highest similarity may be: "Leukemia is a malignant tumor of the hematopoietic system", "The main symptoms of leukemia include fever, bleeding, anemia, and infection", and "Common treatments for leukemia include chemotherapy, radiotherapy, targeted therapy, and hematopoietic stem cell transplantation". These text sequences with the highest similarity can be used as contextual information.
[0133] The context information and target input sequence are populated into a pre-defined template for combination to generate a complete prompt (template). Based on the template's calling function, the content of the template is used to generate query results for the large language model.
[0134] In one embodiment, selecting a preset number of text sequences from the target knowledge base as context information obtained from the query based on the target input sequence, based on a preset similarity selection strategy, includes:
[0135] The similarity values in the similarity set are sorted from largest to smallest, and the text sequences corresponding to the top K values in the sort are selected as the context information obtained from the query based on the target input sequence.
[0136] The selection strategy for similarity sets can be either from largest to smallest numerical value or from smallest to largest numerical value.
[0137] Sort the similarity values in the similarity set from highest to lowest, select the text sequences corresponding to the top K values (e.g., the top 10), remove the newline characters at the end of these selected text sequences and add them to the context information list. Use this context information list as the context information for retrieving the target input sequence.
[0138] It can quickly and effectively select the text sequence with the highest similarity to the target input sequence from the target knowledge base as context information, thereby improving the accuracy and efficiency of text retrieval. It can be widely used in text retrieval, information retrieval and other fields.
[0139] In one embodiment, the method further includes:
[0140] The text sequences with similarity values greater than a first threshold in the similarity set are selected as the context information for retrieval of the target input sequence.
[0141] In other embodiments, the similarity set selection strategy may further include: selecting text sequences in the similarity set whose similarity values are greater than a first threshold (e.g., 0.8) as context information for retrieval of target input sequences.
[0142] In one embodiment, combining the target input sequence and the context information to fine-tune the large language model and output query results includes:
[0143] The target input sequence and the context information are filled into a preset template;
[0144] The target input sequence and the context information are marked at the positions of the template, and the large language model is fine-tuned based on the template after the insertion marks to generate the query results.
[0145] This invention utilizes a predefined template to insert predefined markers between context information and the target input sequence to form a complete prompt. The complete prompt is then used as the input to a large speech model to output the query result. This is equivalent to performing multiple fine-tunings on the initial input sequence of the large speech model (candidate input sequence - target input sequence - complete prompt) before outputting the query result.
[0146] This allows for fine-tuning of the large speech model using the feature vectors generated by the cue layer, thereby improving the quality and accuracy of the large speech model's generation.
[0147] In one embodiment, the step of inserting markers at the positions of the target input sequence and the context information in the template, respectively, and fine-tuning the large language model based on the template with the inserted markers to generate the query results includes:
[0148] A first marker is inserted before the position of the template for the target input sequence;
[0149] A second marker is inserted before the position of the context information in the template;
[0150] Based on the function call of the template, a complete hint is generated from the template with the first and second tags inserted;
[0151] The complete prompt is converted into a tensor representation and used as input to the large language model. The large language model is then fine-tuned to generate the query result.
[0152] First, connect the various text sequences in the context information with semicolons and add a period at the end. Then, fill the context information and target input sequence into the preset template.
[0153] For example, using semicolons to connect and adding a period at the end, the format is: "Leukemia is a malignant tumor of the hematopoietic system," "The main symptoms of leukemia include fever, bleeding, anemia, and infection," and "Common treatments for leukemia include chemotherapy, radiotherapy, targeted therapy, and hematopoietic stem cell transplantation."
[0154] Insert markers before the positions of the target input sequence and context information in the template. For example, insert the markers as: - Known information: {context_text} Problem: {input_text};
[0155] The template generated after inserting the marker is as follows: - Known information: {"Leukemia is a malignant tumor of the hematopoietic system", "The main symptoms of leukemia include fever, bleeding, anemia, infection, etc.", "Common treatments for leukemia include chemotherapy, radiotherapy, targeted therapy, and hematopoietic stem cell transplantation, etc."} The question is: {What is leukemia?}
[0156] The template after inserting the markup is used as the returned complete prompt. The complete prompt is converted into a tensor representation according to the function call of the template. This tensor representation is used as the input of the large language model. The large language model is fine-tuned using this tensor representation, and the query results are output.
[0157] In step S4, contextual information and the target input sequence are populated into a preset template to generate a complete prompt. By inserting markers, the positions of the target input sequence and contextual information within the template can be dynamically adjusted. This technique can be applied to query result generation in large language models, improving the accuracy and fluency of the generated text.
[0158] like Figure 2 The diagram shown is a schematic diagram of a knowledge base query device provided in an embodiment of the present invention.
[0159] The knowledge base query device 100 of this invention can be installed in an electronic device. Depending on the functions implemented, the knowledge base query device 100 may include a conversion and construction module 110, a monitoring and processing module 120, a replacement and calculation module 130, and a selection and combination module 140. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.
[0160] In this embodiment, the functions of each module / unit are as follows:
[0161] The conversion and construction module 110 is used to obtain at least one type of data in the dataset of the initial knowledge base, convert the type of data into the corresponding text sequence, and construct the target knowledge base based on all the text sequences.
[0162] The monitoring and processing module 120 is used to generate a candidate input sequence by performing feature processing on the first word of the initial input sequence through a preset prompting layer when the preset large language model receives the initial input sequence for querying the target knowledge base.
[0163] The replacement and calculation module 130 is used to replace the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence into a target input sequence, calculate the similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base, and generate a similarity set.
[0164] The selection and combination module 140 is used to select a preset number of text sequences from the target knowledge base based on a preset similarity selection strategy as context information obtained from the query based on the target input sequence, and combine the target input sequence and the context information to fine-tune the large language model and output the query results.
[0165] In one embodiment, constructing the target knowledge base based on all text sequences includes:
[0166] Each text sequence is written into the file of the initial knowledge base;
[0167] A preset function is invoked to add newline characters to each of the text sequences in the file, thereby generating the target knowledge base.
[0168] In one embodiment, before performing feature processing on the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence, the method further includes:
[0169] The prompting layer is added before the first connection layer of the large language model, and the learning parameters of the prompting layer are defined so that the prompting layer obtains the initial input sequence of the large language model.
[0170] In one embodiment, the step of performing feature processing on the first word of the initial input sequence through a preset prompting layer to generate a candidate input sequence includes:
[0171] The word that is first in the initial input sequence is taken as the first word;
[0172] The vector of the first word is input into the prompt layer for feature processing to generate the first feature vector;
[0173] The candidate input sequence is generated by fusing the first feature vector with the vector of the first word.
[0174] In one embodiment, replacing the first word of the initial input sequence with the second word of the candidate input sequence to update the initial input sequence to the target input sequence includes:
[0175] The first word in the candidate input sequence is taken as the second word;
[0176] The target input sequence is obtained by replacing the first word with the second word.
[0177] In one embodiment, selecting a preset number of text sequences from the target knowledge base as context information obtained from the query based on the target input sequence, based on a preset similarity selection strategy, includes:
[0178] The similarity values in the similarity set are sorted from largest to smallest, and the text sequences corresponding to the top K values in the sort are selected as the context information obtained from the query based on the target input sequence.
[0179] In one embodiment, combining the target input sequence and the context information to fine-tune the large language model and output query results includes:
[0180] The target input sequence and the context information are filled into a preset template;
[0181] The target input sequence and the context information are marked at the positions of the template, and the large language model is fine-tuned based on the template after the insertion marks to generate the query results.
[0182] like Figure 3 The diagram shown is a structural schematic of an electronic device for implementing a knowledge base query method according to an embodiment of the present invention.
[0183] In this embodiment, the electronic device 1 includes, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can be interconnected via a system bus. The memory 11 stores a knowledge base query program 10, which can be executed by the processor 12. Figure 3 Only the electronic device 1, which includes components 11-13 and the knowledge base query program 10, is shown. Those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.
[0184] The memory 11 includes RAM and at least one type of readable storage medium. The RAM provides a cache for the operation of the electronic device 1; the readable storage medium can be a non-volatile storage medium such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the readable storage medium can be an internal storage unit of the electronic device 1; in other embodiments, the non-volatile storage medium can also be an external storage device of the electronic device 1, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 1. In this embodiment, the readable storage medium of the memory 11 is typically used to store the operating system and various application software installed on the electronic device 1, such as storing the code of the knowledge base query program 10 in one embodiment of the present invention. Furthermore, the memory 11 can also be used to temporarily store various types of data that have been output or will be output.
[0185] In some embodiments, processor 12 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 12 is typically used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication with other devices. In this embodiment, processor 12 is used to run program code stored in memory 11 or process data, such as running knowledge base query program 10.
[0186] The network interface 13 may include a wireless network interface or a wired network interface, which is used to establish a communication connection between the electronic device 1 and the terminal (not shown in the figure).
[0187] Optionally, the electronic device 1 may further include a user interface, which may include a display, an input unit such as a keyboard, and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.
[0188] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.
[0189] The knowledge base query program 10 stored in the memory 11 of the electronic device 1 is a combination of multiple instructions, which, when run in the processor 12, can achieve the following:
[0190] Obtain at least one type of data from the initial knowledge base dataset, convert the type of data into the corresponding text sequence, and construct the target knowledge base based on all the text sequences;
[0191] When the preset large language model detects that it has received the initial input sequence for querying the target knowledge base, the first word of the initial input sequence is processed by the preset prompting layer to generate a candidate input sequence;
[0192] The first word of the initial input sequence is replaced with the second word of the candidate input sequence to update the initial input sequence into the target input sequence. The similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base is calculated to generate a similarity set.
[0193] Based on a preset similarity selection strategy, a preset number of text sequences are selected from the target knowledge base as context information obtained from the query based on the target input sequence. The target input sequence and the context information are combined to fine-tune the large language model and output the query results.
[0194] Specifically, the processor 12's implementation method for the aforementioned knowledge base query program 10 can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.
[0195] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable medium can be non-volatile or non-combustible. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0196] The computer-readable storage medium stores a knowledge base query program 10, which can be executed by one or more processors. The specific implementation of the computer-readable storage medium of the present invention is basically the same as the embodiments of the knowledge base query method described above, and will not be repeated here.
[0197] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0198] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0199] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0200] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0201] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.
[0202] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.
[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method of querying a knowledge base, characterized by, The method includes: Obtain at least one type of data from the initial knowledge base dataset. The type of data includes triples, attribute graph nodes, and table rows. Convert the obtained type of data into corresponding text sequences. Construct the target knowledge base based on all the text sequences. When the preset large language model detects that it has received the initial input sequence for querying the target knowledge base, the first word in the first order of the initial input sequence is pre-set in the prompt layer before the first connection layer of the large language model. The prompt layer performs feature processing on the first word to generate a candidate input sequence. The prompt layer is a learnable parameter layer with predefined learning parameters. The first word in the initial input sequence is replaced by the second word in the first-ranked candidate input sequence to obtain the target input sequence. The similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base is calculated to generate a similarity set. The similarity values in the similarity set are sorted from largest to smallest. A preset number of text sequences corresponding to the values in the target knowledge base are selected according to the sorting order as context information obtained from the target input sequence. The target input sequence and the context information are combined to fine-tune the large language model and output the query results.
2. The knowledge base query method as described in claim 1, characterized in that, The construction of the target knowledge base based on all text sequences includes: Each text sequence is written into the file of the initial knowledge base; A preset function is invoked to add newline characters to each of the text sequences in the file, thereby generating the target knowledge base.
3. The knowledge base query method as described in claim 1, characterized in that, The step of processing the first word in the initial input sequence through a preset prompting layer to generate a candidate input sequence includes: The vector of the first word is input into the prompt layer for feature processing to generate the first feature vector; The candidate input sequence is generated by fusing the first feature vector with the vector of the first word.
4. The knowledge base query method as described in claim 1, characterized in that, The step of combining the target input sequence and the context information to fine-tune the large language model and output query results includes: The target input sequence and the context information are filled into a preset template; The target input sequence and the context information are marked at the positions of the template, and the large language model is fine-tuned based on the template after the insertion marks to generate the query results.
5. A knowledge base query device, characterized in that, The device includes: The transformation and construction module is used to obtain at least one type of data from the dataset of the initial knowledge base. The type of data includes triples, attribute graph nodes, and table rows. The obtained type of data is converted into corresponding text sequences, and the target knowledge base is constructed based on all the text sequences. The monitoring and processing module is used to, when the preset large language model receives the initial input sequence for querying the target knowledge base, input the first word ranked first in the initial input sequence into the prompt layer before the first connection layer of the large language model, and perform feature processing on the first word through the prompt layer to generate a candidate input sequence. The prompt layer is a learnable parameter layer with predefined learning parameters. The replacement and calculation module is used to replace the first word in the first position of the initial input sequence with the second word in the first position of the candidate input sequence to obtain the target input sequence, calculate the similarity between the first vector of the target input sequence and the second vector of each text sequence in the target knowledge base, and generate a similarity set. The selection and combination module is used to sort the similarity values in the similarity set from largest to smallest, select a preset number of text sequences corresponding to the values in the target knowledge base according to the sorting order, and use them as context information obtained from the query based on the target input sequence. The target input sequence and the context information are combined to fine-tune the large language model and output the query results.
6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a knowledge base query program that can be executed by the at least one processor, the knowledge base query program being executed by the at least one processor to enable the at least one processor to perform the knowledge base query method as described in any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a knowledge base query program, which can be executed by one or more processors to implement the knowledge base query method as described in any one of claims 1 to 4.