Method, device and equipment for realizing knowledge questions and answers

Abstract

The invention discloses a method for realizing knowledge questions and answers, which comprises the following steps of: aiming at a query statement input by a user, inputting the query statement into a trained language model, and predicting a lexical element set corresponding to a next lexical element output aiming at the query statement by utilizing the language model. And if the lexical element set comprises the first lexical element and the text corresponding to the first lexical element is the text really existing in the knowledge base corresponding to the language model, it is indicated that really related contents exist in the recall result, and an answer is output based on the text corresponding to the first lexical element included in the lexical element set. Wherein the language model is used for answering query statements for the knowledge field corresponding to the knowledge base. Visibly, through the language model provided by the application, it can be ensured that the output answer is the content really existing in the knowledge base and related to the query statement, and the accuracy of knowledge questions and answers is improved. For example, in the IT field, the reply accuracy of an IT intelligent assistant can be improved, and the efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and equipment for implementing knowledge question and answer. Background Art

[0002] Retrieval Augmented Generation (RAG) is a method to optimize the output of a large language model (LLM). RAG includes three main processes: retrieval, augmentation, and generation. Retrieval: Obtain relevant information from the knowledge base based on the user's query content. Specifically, the user's query is converted into a vector through the model so that it can be compared with the relevant knowledge stored in the vector database. Through semantic similarity search, the top m data that best matches the query are found. Augmentation: The user's query content and the retrieved relevant knowledge are embedded in a preset prompt word template. Generation: The prompt word content that has been enhanced by retrieval is input into the large language model to generate the desired output.

[0003] However, due to the training logic of semantic vectors, it is impossible to exhaustively enumerate irrelevant negative example vectors during training, so it is easy to recall irrelevant vectors, reducing the accuracy of knowledge question answering. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a method, device and equipment for implementing knowledge question and answering to improve the accuracy of question and answering.

[0005] To achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0006] In a first aspect of the present application, a method for implementing knowledge question answering is provided, the method comprising:

[0007] Get the input query statement;

[0008] Inputting a query statement into a language model to obtain a word unit set corresponding to the next word unit predicted by the language model for the query statement;

[0009] If the word-gram set includes a first word-gram, outputting an answer based on the text corresponding to the first word-gram;

[0010] The first word is an identifier corresponding to a text in a knowledge base corresponding to a language model, the language model is fine-tuned based on the knowledge base and the language model is used to answer query statements for a knowledge domain corresponding to the knowledge base.

[0011] In a second aspect of the present application, a device for implementing knowledge question answering is provided, the device comprising:

[0012] An acquisition unit, used for acquiring an input query statement;

[0013] A processing unit, configured to input the query statement into a language model, and obtain a word-unit set corresponding to a next word-unit predicted by the language model for the query statement;

[0014] an output unit, configured to output an answer based on a text corresponding to a first word-gram if the word-gram set includes the first word-gram;

[0015] The first word is an identifier corresponding to a text in a knowledge base corresponding to the language model, the language model is fine-tuned based on the knowledge base, and the language model is used to answer query statements for a knowledge domain corresponding to the knowledge base.

[0016] In the third aspect of the present application, an electronic device is provided, comprising: a processor and a memory; the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes the method described in the first aspect.

[0017] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on a device, the device executes the method described in the first aspect.

[0018] In a fifth aspect of the present application, a computer program product is provided, wherein the computer program product comprises a computer program / instructions, and when the computer program / instructions are executed by a processor, the method described in the first aspect is implemented.

[0019] It can be seen that the embodiments of the present application have the following beneficial effects:

[0020] In the present application, for a query statement input by a user, the query statement is input into a trained language model, and the language model is used to predict a token set corresponding to the next token output for the query statement. If the token set includes a first token, since the text corresponding to the first token is a text that actually exists in the knowledge base corresponding to the language model, indicating that there is real and relevant content in the result of this recall, an answer is output based on the text corresponding to the first token included in the token set. Among them, the language model is used to answer query statements for the knowledge field corresponding to the above-mentioned knowledge base. That is, through the language model provided by the present application, it can be ensured that the output answer is content that actually exists in the knowledge base and is related to the query statement, thereby improving the accuracy of knowledge questions and answers. For example, in the IT field, the accuracy of IT smart assistant responses can be improved, thereby improving efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 A schematic diagram of RAG model prediction provided in an embodiment of the present application;

[0023] Figure 2 A flow chart of a method for implementing knowledge question answering provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the structure of a device for implementing knowledge question answering provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] RAG is an artificial intelligence (AI) framework that combines the advantages of traditional information retrieval systems (such as databases) with the functions of generative large speech models (LLMs). Specifically, relevant knowledge fragments are obtained through the retrieval step, and then these fragments are used to guide the model's answers, thereby improving the model's answer quality and dialogue coherence. Although the training of large models uses a lot of knowledge, it still cannot cover all areas, so when faced with unknown knowledge questions, large models cannot give effective answers.

[0028] However, when faced with unknown questions, the large model will still give seemingly reasonable wrong answers, making it difficult for users to distinguish between true and false. This is the so-called hallucination problem of the large model. The main reason why the large model produces hallucinations lies in its training nature, that is, after a large amount of training, it predicts the next element (token) to be output. When faced with unknown questions, the large model does not know that "it does not know" and will still predict the next token with the highest probability. Specifically, (1) after receiving the retrieved relevant content, the LLM model performs word segmentation on the content and splits it into multiple tokens; (2) based on the transformer architecture, it understands the relationship between tokens and thus understands the overall meaning of the retrieved content; (3) based on the context, it predicts the next token. When predicting, there may be multiple results, each of which has a corresponding probability value. According to the probability value, the token with the highest probability is selected as the prediction result of the next word, and step (3) is repeated until the entire content is generated.

[0029] For example, Figure 1 As shown in the figure, the input is "weather", and the next token with the highest probability after the first prediction by LLM is "good"; taking "the weather is good" as input, the next token with the highest probability after the second prediction by LLM is ","; taking "the weather is good," as input, the next token with the highest probability after the third prediction by LLM is "okay"; taking "the weather is good, ok" as input, the next token with the highest probability after the fourth prediction by LLM is "go out".

[0030] Since knowledge retrieval focuses on the precise expression of objective knowledge, the hallucination problem of LLM makes it difficult to complete the knowledge generation task, resulting in inaccurate answers.

[0031] In order to solve the above problems, when training a language model (i.e., a knowledge question-answering model), the present application regards the documents existing in the knowledge base as a special word-gram, and trains the language model with the purpose of including the special word-gram in the word-gram set corresponding to the next word-gram predicted for the training sentence. That is, the language model is fine-tuned using the text existing in the knowledge base. Since the text corresponding to the predicted word-gram is real knowledge in the knowledge base, the hallucination problem can be effectively solved and the accuracy of the answer can be improved. Moreover, due to the definition of special word-grams, it is only necessary to predict the next word-gram once, without predicting the next word-gram multiple times, thereby improving the speed of knowledge answering.

[0032] To facilitate understanding of the specific implementation of this application, the training process of the knowledge question answering model will be explained below.

[0033] The training process of the knowledge question answering model includes pre-training and fine-tuning. Pre-training refers to the process of training the model on a large number of data sets. The pre-training data sets are usually large and diverse. After training, a large model with strong general capabilities is obtained. During the pre-training process, the retrieval module of RAG is designed as a probability distribution model to select the most relevant knowledge fragments from the knowledge base based on the given question. When relevant knowledge fragments are detected, the RAG model uses these fragments to guide the generation module to generate answers. The generation module is a pre-trained Transformer structure, and the input it receives not only includes the given question, but also the encoding vector of the knowledge fragment retrieved by the retrieval module. In this way, the generation module can make full use of the information in the knowledge base when generating answers.

[0034] Model fine-tuning: It means further training a large model on a specific task or a small data set to improve the model's performance in solving targeted problems. The difference from the pre-training stage is that the amount of data used in the fine-tuning stage is smaller, and mainly uses data from vertical fields. A vertical model and industry model is obtained through fine-tuning.

[0035] When fine-tuning the language model, this application adds each document in the knowledge base as a special word unit to the word unit list corresponding to the model, so that when predicting, only the next word unit needs to be predicted once, which not only improves the speed of knowledge question answering, but also effectively solves the hallucination problem. Specifically, the language model can be fine-tuned using fine-tuning methods such as LORA and DORA, which is not limited in this embodiment.

[0036] In addition, before obtaining the word element corresponding to each document, data cleaning can be performed in advance to remove duplicate documents, low-quality documents, and erroneous documents in the knowledge base to ensure the data quality of the knowledge base.

[0037] When fine-tuning the language model, a sample set is obtained, which is obtained based on the knowledge base, and the language model is fine-tuned based on the sample set. That is, the language model is fine-tuned using the content included in the knowledge base. Generally, in order to improve the accuracy of the language model's refusal to answer, the language model is also fine-tuned using a negative sample set, so that the language model refuses to answer questions not covered by the above knowledge base. Specifically, a negative sample set is obtained, and the language model is fine-tuned based on the negative sample set.

[0038] The process of obtaining a positive sample set includes: obtaining the second word unit corresponding to any text in the knowledge base; and determining the question and the second word unit corresponding to each text as a positive sample set. That is, using the text in the knowledge base, the question and the second word unit corresponding to the text are used as the positive sample set for fine-tuning the language model.

[0039] Wherein, obtaining the second word element corresponding to the text includes: obtaining an identifier of the text in the knowledge base, and determining the identifier as the second word element of the text; or obtaining a hash value corresponding to the text, and determining the hash value as the second word element of the text. In other words, the second word element corresponding to the text can be an identifier of the text in the knowledge base, or a hash value generated based on the content of the text. This embodiment does not limit the form of expression of the word element corresponding to the text, and only needs to ensure uniqueness.

[0040] Among them, the process of fine-tuning the language model using positive sample sets and negative sample sets includes: for the positive sample set, adjusting the loss weight of the positive sample set in the loss function corresponding to the language model, so that the vector corresponding to the question in the positive sample set is close to the vector corresponding to the second word in the positive sample set, thereby improving the relevance of the answer result.

[0041] For the negative sample set, the weight of the negative sample set in the loss function corresponding to the language model is adjusted so that the vector corresponding to the question in the negative sample set is far away from the vector corresponding to the second word, so that the model can correctly reject the answer. Furthermore, in order to facilitate the model to accurately reject the answer, the language model can be fine-tuned using question-answering data from other fields other than the current field.

[0042] In order to enhance the generalization ability of the language model, the questions in the positive sample set include not only the original questions corresponding to the text in the knowledge base, but also the enhanced questions after data enhancement processing of the original questions. The word units corresponding to the original questions and the enhanced questions are the same.

[0043] For example, if the original question is "How do I apply for a virtual desktop?", then data augmentation can generate enhanced questions such as "How should I apply for a virtual desktop?", "How do I apply for a virtual desktop?", "What can I do to apply for a virtual desktop?", "How can I apply for a virtual desktop?", etc. Specifically, the prompt words are used to request the large language model to generate more enhanced questions with semantics similar to the original question.

[0044] It should be noted that the texts mentioned in this embodiment can be of different semantic granularities, such as sentences, paragraphs or entire documents. For each text in the knowledge base, there is a unique second word-gram corresponding to it. It should be noted that in order to distinguish the special words involved in the fine-tuning stage and the application stage, the special words involved in the application stage are referred to as first words-grams and the special words involved in the fine-tuning stage are referred to as second words-grams in this embodiment. Among them, the methods for obtaining the first word-gram and the second word-gram are the same, and the relevant descriptions can be referred to above.

[0045] Specifically, this embodiment may use the following weighted cross entropy loss function. Of course, within the framework of this solution, other loss functions may also be used, such as a standard cross entropy loss function, a standard weighted cross entropy loss function, and the like.

[0046]

[0047] Among them, t i Indicates the label corresponding to the i-th sample, w i Indicates t i The weight of the word; x represents the input word sequence used to predict the next word; N represents the length of the word sequence, that is, the word sequence includes N words; w j Represents the jth word t in the word sequence j The weight of logit i It represents the i-th component of the logit formed by the input x after the forward operation and the hidden vector (hiddenstate), which is used to calculate t i logit of the probability of occurrence, logit j Same reason.

[0048] For the special word-gram involved in this embodiment, the corresponding weight value w satisfies w1>1. Of course, different w can be set for different special word-grams; for ordinary word-grams, the corresponding weight value w2 can be set to w2<1. The specific setting scheme can be diversified and is not subject to the above restrictions, as long as the weight of the special word-gram is greater than that of the ordinary word-gram.

[0049] After the above training, the following effects can be achieved: On the one hand, if the sample set is a negative sample set, that is, ordinary questions and answers whose labels are not special words, the above weighting makes the probability of predicting special words for ordinary questions and answers greatly reduced, thereby greatly improving the accuracy of refusing to answer.

[0050] On the other hand, if the training data is a knowledge question and answer pair (positive sample set) of the target domain (a knowledge domain corresponding to a certain knowledge base), that is, the label is a special word, which can be called T; then the above weighting makes it possible to greatly increase the probability of T being predicted under similar questions (that is, similar questions are constructed through the data enhancement scheme in this embodiment, so that the generalization ability of the model is enhanced, and more similar questions can be handled well), while the probability of non-special words being predicted under similar questions is greatly reduced, thereby greatly improving the relevance of domain knowledge questions and answers.

[0051] After the knowledge question answering model is trained, it can be used to provide question answering services for users. For ease of understanding, the following will be explained with reference to the accompanying drawings.

[0052] See also Figure 2 , which is a flow chart of a method for implementing knowledge question answering provided in an embodiment of the present application, such as Figure 2 As shown, the method includes:

[0053] S201: Obtain an input query statement.

[0054] S202: Input a query statement into a language model to obtain a word-unit set corresponding to the next word-unit predicted by the language model for the query statement.

[0055] In this embodiment, a user can enter a query statement on an electronic device deployed with a language model. After receiving the query statement entered by the user, the electronic device uses the language model to predict a word-gram set corresponding to the next word-gram corresponding to the query statement. The word-gram set includes at least one word-gram and a probability value corresponding to each word-gram, and the final output of the large language model is the word-gram with the highest probability value.

[0056] S203: If the word-gram set includes a first word-gram, output an answer based on the text corresponding to the first word-gram.

[0057] S204: If the word-gram set does not include the first word-gram, output second prompt information.

[0058] Among them, the text corresponding to the first word element is the real content in the knowledge base corresponding to the knowledge question and answer model. The first word element can be the word element corresponding to the document, or it can be the word element corresponding to a certain fragment or a certain sentence in the document. This embodiment does not limit this. The first word element is used to uniquely point to a text, that is, the first word element is the identifier of the text existing in the knowledge base corresponding to the language model. For example, the first word element is the identifier of the text in the knowledge base, or the first word element is a hash value determined based on the text. When the text is a document, the hash value can be calculated based on the title of the document and the summary of the document. When predicting the next word element based on the language model, it can be determined whether the word element set corresponding to the prediction of the next word element includes the first word element. If the first word element is included, it indicates that the predicted content is real content, and the answer is generated based on the text corresponding to the first word element; if the first word element is not included, it indicates that the predicted content is not real content, and there is an illusion problem, then the second prompt information is output. The second prompt information is used to prompt the user that the language model cannot answer the input query statement, so as to achieve accurate rejection.

[0059] Before outputting the answer based on the text corresponding to the first word, the predicted probabilities corresponding to the words in the word set can be sorted in descending order to determine the first k words. Different processing is performed depending on whether the first k words include the first word, including the following situations:

[0060] In one case, if the first k words include the first word, it is considered that a precise recall is performed, and the answer is output based on the text corresponding to each first word. Specifically, if the first k words include m first words, the answer is output based on the text corresponding to the m first words. Where m is greater than or equal to 1 and less than or equal to k.

[0061] In another case, if the first k words do not include the first word but the remaining words include the first word, it is considered that the recall is performed but the results are weakly related, and the answer is output based on the text corresponding to each of the included first words, and the first prompt information is output at the same time. The first prompt information is used to prompt the user that the answer output is weakly related to the query sentence and is for reference only.

[0062] Outputting answers based on the text corresponding to each first word includes: obtaining a link and / or a first summary of the text corresponding to each first word, and outputting a link and / or a first summary of the text corresponding to each first word. The output link corresponds to the text corresponding to the first word. If the text corresponding to the first word is a document, the link is a document link; if the text corresponding to the first word is a fragment in the document, the link is a link that can locate the fragment. This implementation does not require the use of a large language model and can be applied to on-call engineers or intelligent customer service scenarios.

[0063] For example, the first word units include word unit 1, word unit 2 and word unit 3, the first summary of word unit 1, the first summary of word unit 2 and the first summary of word unit 3 are obtained, and the above three first summaries are output as answers.

[0064] Specifically, if the text corresponding to the first word is a document, the first summary of the text corresponding to the first word can be obtained in the following manner: obtain the first semantic vector corresponding to the query statement, determine the semantic similarity between the first semantic vector and each second semantic vector, and use the segment corresponding to the second semantic vector whose semantic similarity meets the preset condition as the first summary. The text corresponding to the first word includes multiple segments, each segment corresponds to a second semantic vector, and the preset condition can be that the semantic similarity is greater than or equal to a threshold or the semantic similarity is the highest. The threshold can be set according to the actual application.

[0065] For example, the document corresponding to the first word contains three segments, and the semantic similarity between the second semantic vector corresponding to each segment and the first semantic vector corresponding to the query statement is calculated, and the segment corresponding to the second semantic vector with the highest semantic similarity is used as the first summary of the document. Alternatively, the threshold is 0.75, and the segment corresponding to the second semantic vector with a semantic similarity greater than or equal to 0.75 is used as the first summary of the document.

[0066] If the text corresponding to the first word is a segment in the document, the segment can be used as the first summary of the text corresponding to the first word.

[0067] Among them, the answer is output based on the text corresponding to the first word, and the second summary can also be generated for the large language model based on the query sentence and the text corresponding to all the first words included in the word set, and the second summary is output. That is, the large language model fuses the text corresponding to all the first words in the word set based on the query sentence to obtain the second summary. This implementation scheme can be applied to user-oriented question and answer scenarios.

[0068] It should be noted that, in this embodiment, if the word-unit set includes the first word-unit, the form in which the large taro ball model outputs the answer based on the text corresponding to the first word-unit is not subject to the above restrictions, and other methods can also be used to answer. If the word-unit set does not include the first word-unit, the large taro ball model can output the answer and the third prompt information based on the word-unit set, and the third prompt information is used to prompt the user that the answer output is not the knowledge in the knowledge base, and other methods can also be used to answer, which is not limited in this embodiment.

[0069] It can be seen that the language model provided by this application can ensure that the output answer is content that actually exists in the knowledge base and is related to the query statement, thereby improving the accuracy of knowledge questions and answers. Moreover, since the language model is targeted at knowledge questions and answers in a specific field, it only needs to predict the next word element once, which simplifies the knowledge question and answer process, takes less time, and improves the user experience.

[0070] Based on the above method embodiments, the present application provides a device and electronic device for implementing knowledge question and answer, which will be described below in conjunction with the accompanying drawings.

[0071] See also Figure 3 , which is a schematic diagram of a device structure for implementing knowledge question answering provided in an embodiment of the present application, such as Figure 3 As shown, the device 300 includes: an acquisition unit 301, a processing unit 302 and an output unit 303.

[0072] Specifically, the acquisition unit 301 is used to acquire an input query statement;

[0073] A processing unit 302 is configured to input the query statement into a language model to obtain a word-gram set corresponding to a next word-gram predicted by the language model for the query statement;

[0074] An output unit 303, configured to output an answer based on the text corresponding to the first word-gram if the word-gram set includes the first word-gram;

[0075] The first word is an identifier corresponding to a text in a knowledge base corresponding to the language model, the language model is fine-tuned based on the knowledge base, and the large language model is used to answer query statements for the knowledge field corresponding to the knowledge base.

[0076] In some embodiments, if the word-gram set includes the first word-gram, the output unit 303 is specifically used to determine the first k word-grams based on the predicted probabilities corresponding to each word-gram in the word-gram set, where k is not less than 1; if the first word-gram is included in the first k word-grams, an answer is output based on the texts corresponding to each of the first word-grams included; if the first word-gram is not included in the first k word-grams and the word-gram set includes the first word-gram in addition to the first k word-grams, an answer and a first prompt information are output based on the texts corresponding to each of the first word-grams included, and the first prompt information is used to prompt the user that the answer is weakly related to the query statement.

[0077] In some embodiments, the output unit 303 is specifically used to obtain a link and / or a first summary of the text corresponding to each first word; output a link and / or the first summary of the text corresponding to each first word; or, generate a second summary based on the query statement and the text corresponding to all the first words included in the query statement by the large language model, and output the second summary.

[0078] In some embodiments, the output unit 303 is specifically used to obtain a first semantic vector corresponding to the query statement if the first word element is a word element corresponding to a document in the knowledge base; determine the semantic similarity between the first semantic vector and each second semantic vector, the document includes multiple fragments, and each fragment corresponds to a second semantic vector; and use the fragment corresponding to the second semantic vector whose semantic similarity meets a preset condition as the first summary.

[0079] In some implementations, the output unit 303 is specifically configured to use the segment corresponding to the first word-gram as a first summary of the text corresponding to the first word-gram if the first word-gram is a word-gram corresponding to a segment in a document in the knowledge base.

[0080] In some implementations, the output unit 303 is further configured to output a second prompt message if the word-gram set does not include the first word-gram, wherein the second prompt message is configured to prompt the user that the query statement cannot be answered.

[0081] In some implementations, the fine-tuning process of the language model includes: obtaining a sample set, where the sample set is obtained based on the knowledge base; and fine-tuning the language model based on the sample set.

[0082] In some implementations, the sample set is a positive sample set; the fine-tuning process further includes: obtaining a negative sample set, and fine-tuning the language model based on the negative sample set, so that the language model refuses to answer questions not covered by the knowledge base.

[0083] In some implementations, obtaining the sample set includes: for any text in the knowledge base, obtaining a second word corresponding to the text; and determining the question and the second word corresponding to each text as a positive sample set.

[0084] In some embodiments, obtaining the second word element corresponding to the text includes: obtaining an identifier of the text in the knowledge base, and determining the identifier as the second word element of the text; or, obtaining a hash value corresponding to the text, and determining the hash value as the second word element of the text.

[0085] In some implementations, fine-tuning the language model based on the sample set includes: for the positive sample set, adjusting the weight of the positive sample set in the loss function corresponding to the language model so that the vector corresponding to the question in the positive sample set is close to the vector corresponding to the second word in the positive sample set;

[0086] Fine-tuning the language model based on the negative sample set includes: for the negative sample set, adjusting the weight of the negative sample set in the loss function corresponding to the language model, so that the vector corresponding to the problem in the negative sample set is far away from the vector corresponding to the second word.

[0087] It should be noted that the specific implementation of each unit in this embodiment can refer to the relevant description in the above method embodiment. The division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The functional units in the embodiment of the present application can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. For example, in the above embodiment, the processing unit and the sending unit can be the same unit or different units. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0088] See also Figure 4 , which shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0089] like Figure 4 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0090] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0091] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0092] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept, and the technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0093] The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method provided by the above embodiments is implemented.

[0094] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0095] In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0096] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0097] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can execute the method.

[0098] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0099] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0100] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit / module does not, in some cases, constitute a limitation on the unit itself.

[0101] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0102] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0103] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0104] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0105] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for implementing knowledge question answering, characterized in that: The method further comprises: Get the input query statement; Inputting the query statement into a language model to obtain a word unit set corresponding to the next word unit predicted by the language model for the query statement; If the word-gram set includes a first word-gram, outputting an answer based on the text corresponding to the first word-gram; The first word is an identifier corresponding to a text in a knowledge base corresponding to the language model, the language model is fine-tuned based on the knowledge base, and the language model is used to answer query statements for a knowledge domain corresponding to the knowledge base.

2. The method according to claim 1, characterized in that If the word-gram set includes a first word-gram, outputting an answer based on the text corresponding to the first word-gram includes: Determine the first k word-grams based on the prediction probability corresponding to each word-gram in the word-gram set, where k is not less than 1; If the first k word-grams include a first word-gram, output an answer based on the text corresponding to each of the first word-grams included; If the first word-gram is not included in the first k word-grams and the word-gram set includes the first word-gram in addition to the first k word-grams, an answer and a first prompt information are output based on the text corresponding to each of the first word-grams included, and the first prompt information is used to prompt the user that the answer is weakly related to the query statement.

3. The method according to claim 1 or 2, characterized in that: Outputting answers based on the texts corresponding to the first word-grams includes: Obtain a link and / or a first summary of the text corresponding to each of the first word-grams; Outputting a link to the text corresponding to each of the first word-grams and / or the first summary; or, The large language model generates a second summary based on the query statement and texts corresponding to all the first word-grams included therein, and outputs the second summary.

4. The method according to claim 3, characterized in that If the first word is a word corresponding to a document in the knowledge base, obtaining a first summary of the text corresponding to each of the first word includes: Obtaining a first semantic vector corresponding to the query statement; Determining a semantic similarity between the first semantic vector and each second semantic vector, the document comprising a plurality of segments, each segment corresponding to a second semantic vector; The segment corresponding to the second semantic vector whose semantic similarity meets a preset condition is used as the first summary.

5. The method according to claim 3, characterized in that: If the first word is a word corresponding to a fragment in a document in the knowledge base, obtaining a first summary of the text corresponding to each of the first word includes: The segment corresponding to the first word is used as a first summary of the text corresponding to the first word.

6. The method according to claim 1, characterized in that The method further comprises: If the word-gram set does not include the first word-gram, a second prompt message is output, where the second prompt message is used to prompt the user that the query statement cannot be answered.

7. The method according to claim 1, characterized in that The fine-tuning process of the language model includes: Acquire a sample set, where the sample set is obtained based on the knowledge base; The language model is fine-tuned based on the sample set.

8. The method according to claim 7, characterized in that The sample set is a positive sample set; The fine-tuning process also includes: obtaining a negative sample set, and fine-tuning the language model based on the negative sample set, so that the language model refuses to answer questions that are not covered by the knowledge base.

9. The method according to claim 8, characterized in that The obtaining of the sample set comprises: For any text in the knowledge base, obtain a second word corresponding to the text; The question and the second word corresponding to each text are determined as the positive sample set.

10. The method according to claim 9, characterized in that The obtaining of the second word corresponding to the text includes: Obtaining an identifier of the text in the knowledge base, and determining the identifier as a second word of the text; or, A hash value corresponding to the text is obtained, and the hash value is determined as a second word element of the text.

11. The method according to any one of claims 8 to 10, characterized in that: Fine-tuning the language model based on the sample set includes: For the positive sample set, adjusting the weight of the positive sample set in the loss function corresponding to the language model so that the vector corresponding to the question in the positive sample set is close to the vector corresponding to the second word in the positive sample set; Fine-tuning the language model based on the negative sample set includes: For the negative sample set, the weight of the negative sample set in the loss function corresponding to the language model is adjusted so that the vector corresponding to the question in the negative sample set is far away from the vector corresponding to the second word.

12. A device for implementing knowledge question answering, characterized in that: The device includes: An acquisition unit, used for acquiring an input query statement; A processing unit, configured to input the query statement into a language model, and obtain a word-unit set corresponding to a next word-unit predicted by the language model for the query statement; an output unit, configured to output an answer based on a text corresponding to a first word-gram if the word-gram set includes the first word-gram; The first word is an identifier corresponding to a text in a knowledge base corresponding to the language model, the language model is fine-tuned based on the knowledge base, and the large language model is used to answer query statements for the knowledge field corresponding to the knowledge base.

13. An electronic device, characterized in that: The device comprises: a processor and a memory; The memory is used to store instructions or computer programs; The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a device, the device executes the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images