Knowledge question answering method, system and apparatus, and storage medium

Abstract

Disclosed in the present application are a knowledge question answering method, system and apparatus, and a storage medium. The method comprises: acquiring input information and context information; inputting the input information and the context information into a large language model to obtain a first probability distribution, the first probability distribution being used for characterizing answer information and a corresponding probability which are predicted on the basis of the input information and the context information; inputting the input information into the large language model to obtain a second probability distribution; according to the context information, the first probability distribution and the second probability distribution, determining an adjustment factor; by means of the adjustment factor, updating the first probability distribution to obtain a third probability distribution; and sampling the third probability distribution to determine target answer information. The embodiments of the present application help to improve the accuracy of predicted answer information and increase the prediction speed, and can be widely applied to the technical field of artificial intelligence.

Description

Knowledge question answering method, system, device and storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on October 25, 2023, with application number 2023113938813 and application name “Knowledge Question Answering Method, System, Device and Storage Medium”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a knowledge question answering method, system, device and storage medium. Background Art

[0003] Language models have important applications in many fields, such as knowledge question answering, route navigation, and customer service systems. Language models are very effective in generating coherent answers or fluent continuations. During the generation process, they rely mainly on two sources of knowledge: (1) prior knowledge, which is implicit knowledge learned during pre-training and stored in the model parameters; and (2) contextual knowledge, which is passed to the language model as external context input. The language model in the related art can accurately predict the answer information based on the prior knowledge. However, the answer information is not well related to the context, and there may be situations where the answer information and the context are inconsistent. To address this issue, the related art improves the prediction accuracy by setting weights for the prior knowledge and context and then influencing the model training process. However, this solution requires manual adjustment of the weights and cannot distinguish the different levels of importance of the context; it relies on human experience and knowledge, which has limited improvement in the model's prediction accuracy; and it also requires adjustment of the model, which is time-consuming and labor-intensive.

[0004] Summary of the Invention

[0005] The purpose of this application is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0006] To this end, the purpose of this application is to provide a highly accurate knowledge question answering method, system, device and storage medium.

[0007] In order to achieve the above technical objectives, the technical solutions adopted in the embodiments of the present application include:

[0008] On the one hand, an embodiment of the present application provides a knowledge question and answer method, which includes: obtaining input information and context information; inputting the input information and the context information into a large language model to obtain a first probability distribution; the first probability distribution is used to characterize the answer information and corresponding probability predicted based on the input information and the context information; inputting the input information into the large language model to obtain a second probability distribution; determining an adjustment factor based on the context information, the first probability distribution and the second probability distribution; and updating the first probability distribution by the adjustment factor to obtain a third probability distribution; performing a sampling operation on the third probability distribution to determine the target answer information.

[0009] The embodiment of the present application generates a first probability distribution using input information and contextual information, and generates a second probability distribution using the input information. Based on the relationship between the first and second probability distributions, the first probability distribution is adjusted and then sampled to obtain the target answer information. By adjusting the probability distribution using an adjustment factor, the embodiment of the present application alleviates the problem of manual adjustment relying on human experience and knowledge, thereby improving the accuracy of the predicted answer information. Furthermore, the embodiment of the present application eliminates the need to adjust the large language model, thereby improving prediction speed.

[0010] In some embodiments, determining the adjustment factor based on the context information, the first probability distribution, and the second probability distribution includes:

[0011] determining a first value according to a relationship between the first probability distribution and the second probability distribution;

[0012] An adjustment factor is determined with the first value as a base and a first parameter as an exponent; the first parameter is used to adjust the degree of attention paid by the predicted answer information to the context information.

[0013] In some embodiments, the method further comprises:

[0014] When the first probability distribution is the same as the second probability distribution, determining the first parameter to be 0;

[0015] Alternatively, when the similarity between the first probability distribution and the second probability distribution is greater than a first threshold, determining that the first parameter is a second value;

[0016] Alternatively, when the similarity between the first probability distribution and the second probability distribution is less than or equal to the first threshold, the first parameter is determined to be a third value; and the third value is greater than the second value.

[0017] In some embodiments, the method further comprises:

[0018] determining a first divergence based on the first probability distribution and the second probability distribution, wherein the first divergence is related to a quotient of the first probability distribution and the second probability distribution;

[0019] determining a second divergence based on the first probability distribution and the second probability distribution, wherein the second divergence is related to a quotient of the second probability distribution and the first probability distribution;

[0020] The first divergence and the second divergence are averaged to determine a first parameter.

[0021] In some embodiments, updating the first probability distribution by the adjustment factor to obtain a third probability distribution includes:

[0022] Updating the first probability distribution by the adjustment factor and performing normalization processing to obtain a fourth probability distribution;

[0023] The fourth probability distribution is processed through a fully connected layer to obtain a third probability distribution.

[0024] In some embodiments, the large language model is trained by the following steps:

[0025] Acquire an input information sample; the input information sample includes a true result of the answer information corresponding to the input information;

[0026] Processing the input information sample through a large language model to obtain a sample prediction result;

[0027] The large language model is trained according to the error between the sample prediction result and the true result until the error is less than or equal to a preset threshold.

[0028] In some embodiments, the sampling operation on the third probability distribution to determine target answer information includes:

[0029] Determining a second threshold; determining a plurality of probability distributions from the third probability distribution based on the probability ranking, performing sampling operations on the plurality of probability distributions to determine target answer information; the number of the plurality of probability distributions being the second threshold;

[0030] Alternatively, a third threshold is determined, the third probability distribution is updated according to the third threshold, and a sampling operation is performed on the updated third probability distribution to determine target answer information.

[0031] On the other hand, the present application embodiment proposes a knowledge question answering system, including:

[0032] The first module is used to obtain input information and context information;

[0033] A second module is configured to input the input information and the context information into a large language model to obtain a first probability distribution; the first probability distribution is configured to represent the answer information predicted based on the input information and the context information and the corresponding probability;

[0034] A third module is configured to input the input information into the large language model to obtain a second probability distribution;

[0035] A fourth module is configured to determine an adjustment factor based on the context information, the first probability distribution, and the second probability distribution; and update the first probability distribution using the adjustment factor to obtain a third probability distribution;

[0036] The fifth module is used to perform a sampling operation on the third probability distribution to determine target answer information.

[0037] On the other hand, an embodiment of the present application provides a knowledge question-answering device, comprising:

[0038] at least one processor;

[0039] at least one memory for storing at least one program;

[0040] When the at least one program is executed by the at least one processor, the at least one processor implements the above-mentioned knowledge question and answer method.

[0041] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor. When the program is executed by the processor, it is used to implement the above-mentioned knowledge question and answer method.

[0042] The knowledge question-answering method of the embodiment of the present application includes: obtaining input information and context information; inputting the input information and the context information into a large language model to obtain a first probability distribution; the first probability distribution is used to characterize the answer information and the corresponding probability predicted based on the input information and the context information; inputting the input information into the large language model to obtain a second probability distribution; determining an adjustment factor based on the context information, the first probability distribution and the second probability distribution; and updating the first probability distribution by the adjustment factor to obtain a third probability distribution; sampling the third probability distribution to determine the target answer information. The embodiment of the present application generates a first probability distribution by input information and upper and lower information, and generates a second probability distribution by input information; according to the relationship between the first probability distribution and the second probability distribution, the first probability distribution is adjusted and then sampled to obtain the target answer information. The embodiment of the present application adjusts the probability distribution by the adjustment factor, which alleviates the problem that the manual adjustment process depends on human experience and knowledge, and is conducive to improving the accuracy of the predicted answer information; at the same time, the embodiment of the present application does not need to adjust the large language model, which improves the prediction speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present application or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0044] FIG1 is a flow chart of an embodiment of the knowledge question answering method provided by the present application;

[0045] FIG2 is a flow chart of another embodiment of the knowledge question answering method provided by the present application;

[0046] FIG3 is a flow chart of an embodiment of a process for determining an adjustment factor provided by the present application;

[0047] FIG4 is a schematic diagram of a flow chart of an embodiment of a process for determining a first parameter provided by the present application;

[0048] FIG5 is a flow chart of another embodiment of a process for determining a first parameter provided by the present application;

[0049] FIG6 is a flow chart of an embodiment of a large language model training process provided by the present application;

[0050] FIG7 is a schematic diagram showing the effect of an embodiment of the knowledge question answering method provided by the present application;

[0051] FIG8 is a schematic diagram of the structure of an embodiment of the knowledge question answering system provided by the present application;

[0052] FIG9 is a schematic structural diagram of an embodiment of a knowledge question-answering device provided by the present application;

[0053] FIG10 is a schematic structural diagram of another embodiment of the knowledge question-answering device provided in this application. DETAILED DESCRIPTION

[0054] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. For the step numbers in the following embodiments, they are provided only for the convenience of explanation and are not intended to limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0055] First, the terms involved in the embodiments of this application are explained:

[0056] Large language models are AI models designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more.

[0057] Hallucination: When the text generated by the model does not follow the original text or does not conform to the facts, the model is considered to have hallucination problems.

[0058] JS divergence (Jenson's Shannon divergence): measures the similarity between two probability distributions, and its value is between 0 and 1.

[0059] In the related art, large language models are very effective in generating coherent answers or fluent continuations. During the generation process, they rely mainly on two sources of knowledge: (1) prior knowledge, which is the implicit knowledge learned during pre-training and stored in the model parameters; and (2) contextual knowledge, which is passed to the language model as external context input.

[0060] The large language models used in related technologies are basically trained based on general knowledge and perform poorly in the field of private data. Therefore, the use of large language models plug-in private knowledge bases for question answering is widely used. However, previous studies have shown that current large language models only rely on trained parameters to answer questions and cannot fully pay attention to the new information introduced by contextual knowledge. This may lead to hallucinations in the answers, and the generated answers contain facts that do not exist in the context or contradict existing prior knowledge. For example, for a trained LLAMA large model, when the input context is "Argentina won the World Cup in 1978, 1986 and 2022" and the question is "How many World Cups has Argentina won?", the LLAMA large model still outputs "two". This is because the data used to train the large language model is now outdated. How to balance these two sources of knowledge in the generation process and reduce the generation hallucinations of large language models is an urgent problem that needs to be solved.

[0061] To alleviate the hallucination problem of large models, the following methods are used in related technologies:

[0062] 1. Starting with the large language model training data itself, text that is prone to hallucinations is downweighted and deterministic text is weighted. For example, deterministic knowledge such as textbooks and science textbooks is weighted more heavily, while online knowledge is downweighted. However, artificially weighting knowledge is a crude method and cannot effectively quantify its importance.

[0063] Related technologies also optimize large-model decoding generation by adopting a more deterministic sampling strategy, providing corresponding negative feedback when the model generates hallucinated results, and using reinforcement learning for optimization. However, this approach significantly reduces the diversity of large language model generation and is only suitable for specialized fields. Furthermore, research in this area requires retraining the model and cannot be easily embedded into other already trained large language models.

[0064] 3. OpenAI proposed a new method to mitigate hallucinations and achieve better alignment in large language models. This method, called "process supervision," improves the mathematical reasoning capabilities of large language models by rewarding each correct reasoning step. Unlike traditional "outcome supervision," this approach focuses not only on the correctness of the final answer but also on each step of the reasoning process. However, this method is not easily scalable to other large models.

[0065] In view of the problem that the above-mentioned related technical solutions all require re-fine-tuning of the large language model, the embodiment of the present application optimizes the output distribution of the large language model through context comparison decoding, without the need to re-fine-tune the model, and encourages the large model to pay full attention to contextual knowledge, which can be easily embedded in other large language model frameworks.

[0066] The knowledge question answering method and system proposed according to the embodiments of the present application will be described in detail below with reference to the accompanying drawings. First, the knowledge question answering method proposed according to the embodiments of the present application will be described with reference to the accompanying drawings.

[0067] 1 , a knowledge question and answer method is provided in an embodiment of the present application. The knowledge question and answer method in the embodiment of the present application can be applied to a terminal, a server, or software running in a terminal or a server. The terminal can be a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The knowledge question and answer method in the embodiment of the present application mainly includes the following steps:

[0068] S100: Acquire input information and context information;

[0069] S200: Inputting input information and context information into a large language model to obtain a first probability distribution; the first probability distribution is used to represent answer information predicted based on the input information and context information and the corresponding probability;

[0070] S300: Inputting input information into the large language model to obtain a second probability distribution;

[0071] S400: Determine an adjustment factor based on context information, the first probability distribution, and the second probability distribution; and update the first probability distribution using the adjustment factor to obtain a third probability distribution;

[0072] S500: Perform a sampling operation on the third probability distribution to determine target answer information.

[0073] In some embodiments, the present application proposes a context contrast decoding scheme that encourages large language models to pay full attention to their input context in the process of generating answers, follow a contrasting output distribution, amplify the difference between the output probabilities of the model using context decoding and not using context decoding, and effectively weaken the prior knowledge while providing more relevant context information. In some embodiments, referring to Figure 2, step 1: modeling the original prior probability distribution of the large model; step 2: introducing context and adjusting the model response to condition x, y <tThe point mutual information between the lower context and the input x is multiplied by the prior probability; Step three: Introducing the JS divergence model to model the weights between the context response and the prior response mutual information, which alleviates the problem of manual parameter adjustment relying on experience. Step four: Normalize the large model response to a probability distribution, and the large model samples and outputs from the probability distribution. It should be noted that the large language model in the embodiment of the present application can be any model in the field of artificial intelligence that can realize question and answer, and this application does not impose any specific restrictions.

[0074] The embodiments of the present application can alleviate the problem that large language models find it difficult to fully pay attention to the input context and generate text that is unfaithful or contains hallucinations. When the contextual knowledge provided contradicts the prior knowledge of the model, the context contrast decoding scheme can effectively alleviate this contradiction, bringing substantial improvements to tasks that are crucial for resolving knowledge conflicts. As the model size in knowledge conflict tasks grows, the benefits brought by context contrast decoding gradually increase. Secondly, context contrast decoding can be easily embedded in existing pre-trained large language models without any additional fine-tuning, thereby improving model processing speed.

[0075] The knowledge question-answering method of the embodiment of the present application includes: obtaining input information and context information; inputting the input information and context information into a large language model to obtain a first probability distribution; the first probability distribution is used to characterize the answer information and the corresponding probability predicted based on the input information and context information; inputting the input information into the large language model to obtain a second probability distribution; determining an adjustment factor based on the context information, the first probability distribution, and the second probability distribution; and updating the first probability distribution through the adjustment factor to obtain a third probability distribution; performing a sampling operation on the third probability distribution to determine the target answer information. The embodiment of the present application generates a first probability distribution through input information and context information, and generates a second probability distribution through input information; based on the relationship between the first probability distribution and the second probability distribution, the first probability distribution is adjusted and then sampled to obtain the target answer information. The embodiment of the present application adjusts the probability distribution through the adjustment factor, which alleviates the problem that the manual adjustment process depends on human experience and knowledge, and is conducive to improving the accuracy of the predicted answer information; at the same time, the embodiment of the present application does not need to adjust the large language model, which improves the prediction speed.

[0076] In some embodiments, as shown in FIG3 , determining the adjustment factor based on the context information, the first probability distribution, and the second probability distribution includes:

[0077] S410: Determine a first value according to a relationship between the first probability distribution and the second probability distribution;

[0078] S420: Determine an adjustment factor with the first value as a base and a first parameter as an exponent; the first parameter is used to adjust the degree of attention paid by the predicted answer information to the context information.

[0079] In the embodiment of the present application, the adjustment factor can be determined by an exponential function, and the specific expression of the exponential function is determined by the relationship between the first probability distribution and the second probability distribution. Of course, those skilled in the art can also set other forms of functions / relationships as the adjustment factor according to actual needs, and the embodiment of the present application does not limit the specific form of the adjustment factor.

[0080] In some embodiments, referring to FIG4 , the knowledge question answering method further includes:

[0081] S421: When the first probability distribution is the same as the second probability distribution, determining the first parameter to be 0;

[0082] S422: Alternatively, when the similarity between the first probability distribution and the second probability distribution is greater than a first threshold, determining the first parameter to be a second value;

[0083] S423: Alternatively, when the similarity between the first probability distribution and the second probability distribution is less than or equal to a first threshold, determining the first parameter to be a third value; the third value is greater than the second value.

[0084] In some possible implementations, the embodiments of the present application determine the value of the first parameter based on the different relationship forms between the first probability distribution and the second probability distribution. In some embodiments, the first parameter can be calculated using a mathematical formula or determined by a table lookup, etc., and the embodiments of the present application do not impose specific limitations.

[0085] In some embodiments, referring to FIG5 , the knowledge question answering method further includes:

[0086] S424: Determine a first divergence based on the first probability distribution and the second probability distribution; the first divergence is related to the quotient of the first probability distribution and the second probability distribution;

[0087] S425: Determine a second divergence based on the first probability distribution and the second probability distribution; the second divergence is related to the quotient of the second probability distribution and the first probability distribution;

[0088] S426: Average the first divergence and the second divergence to determine a first parameter.

[0089] In some possible implementations, the context contrast decoding scheme proposed in this application models the prior knowledge as a second probability distribution p θ (y t |x,y <t ) (i.e., determine the output of the tth time based on the prior knowledge of the output results and input information of the previous t-1 times), and use the context c and generate y tThe mutual information between them is used as an adjustment factor to encourage the large language model to fully pay attention to contextual knowledge. The large language model samples from the first probability distribution expressed by the following formula (1):

[0090] The large model response is the product of the original output probability and the mutual information, is the adjustment factor, and α is the first parameter, which is used to adjust the degree of attention to contextual knowledge.

[0091] It is understandable that, for parameter α, in order to avoid manual parameter adjustment, the embodiment of the present application can use JS divergence modeling to determine the first parameter. The JS divergence value range is [0,1]. When the distribution is the same, the JS divergence is 0. When the probability output of the model containing the context (i.e., the first probability distribution) is close to the output of the model without the context (i.e., the second probability distribution), the adjusted model response tends to be consistent with the original model response. When the distribution gap between the two is large, the large model response tends to output the response after focusing on the context c. Define q1 = p θ (y t |x,c,y <t ), q2=p θ (y t |x,y <t ), JS divergence is calculated as follows:

[0092] The embodiment of the present application determines the first parameter through the JS divergence formula, and then determines the adjustment factor. After adjusting the first probability distribution, sampling is performed to determine the target answer information, which alleviates the problem of manual adjustment relying on experience and improves the prediction accuracy of the model.

[0093] In some embodiments, updating the first probability distribution by the adjustment factor to obtain a third probability distribution includes:

[0094] Updating the first probability distribution by the adjustment factor and performing normalization processing to obtain a fourth probability distribution;

[0095] The fourth probability distribution is processed through a fully connected layer to obtain a third probability distribution.

[0096] In some embodiments, the large language model response after adding contextual contrast decoding in the present application needs to be renormalized into a probability distribution. That is, after updating the first probability distribution by adjusting the factor, the obtained probability distribution is normalized before the sampling operation is performed. In some embodiments, the output probability after normalization is shown in the following formula group (3):

[0097] From this we can see that even if the context comparison decoding module is added, the large model can still maintain the original different sampling strategies to sample the obtained probability distribution and obtain the target answer information.

[0098] In some embodiments, referring to FIG6 , the large language model is trained by the following steps:

[0099] S600: Obtain an input information sample; the input information sample includes a true result of the answer information corresponding to the input information;

[0100] S700: Processing the input information sample through the large language model to obtain a sample prediction result;

[0101] S800: Training the large language model according to the error between the sample prediction result and the true result until the error is less than or equal to a preset threshold.

[0102] In some possible implementations, a large language model is trained using prior knowledge to obtain a large language model with acceptable errors. The probability distribution obtained by the large language model is subsequently adjusted using the aforementioned knowledge question-answering method and then sampled to obtain answer information.

[0103] In some embodiments, performing a sampling operation on the third probability distribution to determine target answer information includes:

[0104] Determine a second threshold; determine a plurality of probability distributions from the third probability distribution based on the probability ranking, perform sampling operations on the plurality of probability distributions, and determine target answer information; the number of the plurality of probability distributions is the second threshold;

[0105] Alternatively, a third threshold is determined, the third probability distribution is updated according to the third threshold, and a sampling operation is performed on the updated third probability distribution to determine the target answer information.

[0106] In some possible implementations, the present application embodiment may adopt different sampling strategies to sample the updated third probability distribution to obtain target answer information. The specific process is as follows:

[0107] For the large language model θ, an input x and an external context c, the model θ is required to generate a response y based on the input x and the context. The original large model outputs the response y t It can be directly obtained by autoregressive sampling from the conditional probability distribution (i.e., the first probability distribution) of the given input x and context c, and is calculated as shown in the following formula (4): t ~p θ (y t |x,c,y <t ) Formula (4)

[0108] However, when this method contains knowledge in context c that is not related to the prior knowledge of the large model, it will be difficult for the model to effectively focus on context c and will over-rely on the prior knowledge of the large language model. Even when given context c and input x, the response result of the large model is still contrary to the existing facts. Therefore, the embodiment of the present application uses the above-mentioned knowledge question and answer method to adjust the first probability distribution, add the influence of the context, enhance the influence of the context on the prediction results, and improve the accuracy of the model. Referring to the comparison diagram shown in Figure 7, through the knowledge question and answer method provided by the present application, after fully analyzing the context information "Argentina won the World Cup in 1978, 1986 and 2022", the probability distribution of the model output can be adjusted to obtain the correct answer information of winning the world three times.

[0109] The present application proposes a contextual comparative decoding scheme that uses mutual information (i.e., an adjustment factor) to encourage large language models to fully focus on contextual knowledge, alleviating the large model hallucination problem. This approach eliminates the need for fine-tuning and allows for direct embedding of different large language models. Furthermore, the present application proposes using JS divergence modeling to automatically optimize parameters when decoding with and without context, without requiring human intervention and improving model processing speed.

[0110] The knowledge question-answering method of the embodiment of the present application includes: obtaining input information and context information; inputting the input information and context information into a large language model to obtain a first probability distribution; the first probability distribution is used to characterize the answer information and the corresponding probability predicted based on the input information and context information; inputting the input information into the large language model to obtain a second probability distribution; determining an adjustment factor based on the context information, the first probability distribution, and the second probability distribution; and updating the first probability distribution through the adjustment factor to obtain a third probability distribution; performing a sampling operation on the third probability distribution to determine the target answer information. The embodiment of the present application generates a first probability distribution through input information and context information, and generates a second probability distribution through input information; based on the relationship between the first probability distribution and the second probability distribution, the first probability distribution is adjusted and then sampled to obtain the target answer information. The embodiment of the present application adjusts the probability distribution through the adjustment factor, which alleviates the problem that the manual adjustment process depends on human experience and knowledge, and is conducive to improving the accuracy of the predicted answer information; at the same time, the embodiment of the present application does not need to adjust the large language model, which improves the prediction speed.

[0111] Next, a knowledge question-answering system proposed according to an embodiment of the present application will be described with reference to FIG8 .

[0112] FIG8 is a schematic diagram of the structure of a knowledge question answering system according to an embodiment of the present application. The system specifically includes:

[0113] The first module 810 is used to obtain input information and context information;

[0114] The second module 820 is configured to input the input information and the context information into the large language model to obtain a first probability distribution; the first probability distribution is configured to represent the predicted answer information based on the input information and the context information and the corresponding probability;

[0115] The third module 830 is used to input the input information into the large language model to obtain a second probability distribution;

[0116] The fourth module 840 is configured to determine an adjustment factor based on the context information, the first probability distribution, and the second probability distribution; and update the first probability distribution using the adjustment factor to obtain a third probability distribution;

[0117] The fifth module 850 is used to perform a sampling operation on the third probability distribution to determine target answer information.

[0118] In some embodiments, the fourth module is further configured to determine the first value based on a relationship between the first probability distribution and the second probability distribution;

[0119] An adjustment factor is determined with the first value as a base and a first parameter as an exponent; the first parameter is used to adjust the degree of attention paid by the predicted answer information to the context information.

[0120] In some embodiments, the fourth module is further configured to:

[0121] When the first probability distribution is the same as the second probability distribution, determining the first parameter to be 0;

[0122] Alternatively, when the similarity between the first probability distribution and the second probability distribution is greater than a first threshold, determining the first parameter to be the second value;

[0123] Alternatively, when the similarity between the first probability distribution and the second probability distribution is less than or equal to a first threshold, the first parameter is determined to be a third value; and the third value is greater than the second value.

[0124] In some embodiments, the fourth module is further configured to:

[0125] determining a first divergence based on the first probability distribution and the second probability distribution, wherein the first divergence is related to a quotient of the first probability distribution and the second probability distribution;

[0126] determining a second divergence based on the first probability distribution and the second probability distribution, wherein the second divergence is related to a quotient of the second probability distribution and the first probability distribution;

[0127] The first divergence and the second divergence are averaged to determine a first parameter.

[0128] In some embodiments, the fifth module of the system is further configured to update the first probability distribution using the adjustment factor and perform normalization processing to obtain a fourth probability distribution;

[0129] The fourth probability distribution is processed through a fully connected layer to obtain a third probability distribution.

[0130] In some embodiments, the system further includes a seventh module for training the large language model. Specifically, the seventh module is used to obtain input information samples; the input information samples include real results of answer information corresponding to the input information;

[0131] Process the input information sample through the large language model to obtain the sample prediction result;

[0132] Based on the error between the sample prediction result and the actual result, the large language model is trained until the error is less than or equal to the preset threshold.

[0133] In some embodiments, the fifth module is configured to determine a second threshold; determine a plurality of probability distributions from the third probability distribution based on the probability ranking, perform sampling operations on the plurality of probability distributions, and determine target answer information; the number of the plurality of probability distributions is the second threshold;

[0134] Alternatively, a third threshold is determined, the third probability distribution is updated according to the third threshold, and a sampling operation is performed on the updated third probability distribution to determine the target answer information.

[0135] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] The present invention provides a knowledge question answering device, including:

[0137] at least one processor;

[0138] at least one memory for storing at least one program;

[0139] When at least one program is executed by at least one processor, the at least one processor implements a knowledge question answering method.

[0140] Similarly, the contents of the above method embodiments are applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0141] The knowledge question answering device for executing the above-mentioned knowledge question answering method provided in the embodiment of the present application can be a terminal. Referring to FIG9 , FIG9 is a partial structural block diagram of the terminal provided in the embodiment of the present application, and the terminal includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will understand that the terminal structure shown in FIG9 does not constitute a limitation on mobile phones, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0142] The RF circuit can be used to receive and send signals during information transmission or calls. In particular, it receives downlink information from the base station and sends it to the processor for processing; in addition, it sends the designed uplink data to the base station.

[0143] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory.

[0144] The input unit can be used to receive input digital or character information and generate key signal input related to the settings and function control of the mobile phone. Specifically, the input unit can include a touch panel 1031 and other input devices 1032.

[0145] The display unit 1040 may include a display panel 1041 .

[0146] The audio circuit 1060 , the speaker 1061 , and the microphone 1062 may provide an audio interface.

[0147] In this embodiment, the processor included in the terminal can execute the knowledge question and answer method of the previous embodiment.

[0148] The knowledge question answering device for executing the above-mentioned knowledge question answering method provided in the embodiment of the present application can also be a server. Referring to Figure 10, Figure 10 is a partial structural block diagram of the server provided in the embodiment of the present application. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (abbreviated as CPU, i.e., Central Processing Units), 1122 (for example, one or more processors) and memory 1132, and one or more storage media 1130 (for example, one or more mass storage devices) for storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 can be temporary storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server 1100. Furthermore, the central processing unit 1122 can be configured to communicate with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0149] The server may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0150] The processor in the server can be used to execute the above knowledge question answering method.

[0151] Similarly, the contents of the above method embodiments are applicable to the present device embodiments. The functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0152] An embodiment of the present application also provides a computer-readable storage medium, which stores a program executable by a processor. When the program executable by the processor is executed by the processor, it is used to perform the above-mentioned knowledge question and answer method.

[0153] Similarly, the contents of the above method embodiments are applicable to the present storage medium embodiment. The functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0154] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiments presented and described in the flow chart of the present application are provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operations and logical flows presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0155] In addition, although the present application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present application. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the routine skills of an engineer. Therefore, a person skilled in the art can implement the present application as set forth in the claims using ordinary techniques without undue experimentation. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present application, which is determined by the full scope of the appended claims and their equivalents.

[0156] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several programs for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0157] The logic and / or steps represented in a flowchart or otherwise described herein, for example, may be considered as an ordered list of executable programs for implementing the logical functions, and may be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can retrieve and execute a program from a program execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, a program execution system, apparatus, or device.

[0158] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0159] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable program execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0160] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0161] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and intent of the present application, and that the scope of the present application is defined by the claims and their equivalents.

[0162] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A knowledge question answering method, comprising: Get input information and context information; Inputting the input information and the context information into a large language model to obtain a first probability distribution; The first probability distribution is used to characterize the answer information predicted based on the input information and the context information and the corresponding probability; Inputting the input information into the large language model to obtain a second probability distribution; Determining an adjustment factor according to the context information, the first probability distribution, and the second probability distribution; and updating the first probability distribution by the adjustment factor to obtain a third probability distribution; A sampling operation is performed on the third probability distribution to determine target answer information.

2. The knowledge question answering method according to claim 1, wherein: The determining the adjustment factor according to the context information, the first probability distribution, and the second probability distribution includes: Determining a first value according to a relationship between the first probability distribution and the second probability distribution; An adjustment factor is determined with the first value as a base and a first parameter as an exponent; the first parameter is used to adjust the degree of attention paid by the predicted answer information to the context information.

3. The knowledge question answering method according to claim 2, further comprising: When the first probability distribution is the same as the second probability distribution, determining the first parameter to be 0; Alternatively, when the similarity between the first probability distribution and the second probability distribution is greater than a first threshold, determining that the first parameter is a second value; Alternatively, when the similarity between the first probability distribution and the second probability distribution is less than or equal to the first threshold, the first parameter is determined to be a third value; and the third value is greater than the second value.

4. The knowledge question answering method according to claim 2, further comprising: Determining a first divergence according to the first probability distribution and the second probability distribution; The first divergence is associated with a quotient of the first probability distribution and the second probability distribution; Determining a second divergence according to the first probability distribution and the second probability distribution, wherein the second divergence is related to a quotient of the second probability distribution and the first probability distribution; The first divergence and the second divergence are averaged to determine a first parameter.

5. The knowledge question answering method according to claim 1, wherein: The updating the first probability distribution by the adjustment factor to obtain a third probability distribution includes: Updating the first probability distribution by the adjustment factor and performing normalization processing to obtain a fourth probability distribution; The fourth probability distribution is processed through a fully connected layer to obtain a third probability distribution.

6. The knowledge question answering method according to claim 1, wherein: The large language model is trained by the following steps: Acquire an input information sample; the input information sample includes a true result of the answer information corresponding to the input information; Processing the input information sample through a large language model to obtain a sample prediction result; The large language model is trained according to the error between the sample prediction result and the true result until the error is less than or equal to a preset threshold.

7. The knowledge question answering method according to claim 1, wherein: The performing a sampling operation on the third probability distribution to determine target answer information includes: Determine a second threshold; determine a plurality of probability distributions from the third probability distribution according to the probability ranking, perform sampling operations on the plurality of probability distributions, and determine target answer information; the number of the plurality of probability distributions is the second threshold; Alternatively, a third threshold is determined, the third probability distribution is updated according to the third threshold, a sampling operation is performed on the updated third probability distribution, and target answer information is determined.

8. The knowledge question answering method according to claim 2, wherein: The value of the first parameter is determined according to different relationship forms between the first probability distribution and the second probability distribution, wherein the first parameter is calculated by a mathematical formula or determined by a table lookup or the like.

9. A knowledge question answering system, comprising: The first module is used to obtain input information and context information; A second module is used to input the input information and the context information into a large language model to obtain a first probability distribution; The first probability distribution is used to characterize the answer information predicted based on the input information and the context information and the corresponding probability; A third module is used to input the input information into the large language model to obtain a second probability distribution; A fourth module is configured to determine an adjustment factor according to the context information, the first probability distribution, and the second probability distribution; and to update the first probability distribution by the adjustment factor to obtain a third probability distribution; The fifth module is used to perform a sampling operation on the third probability distribution to determine target answer information.

10. A knowledge question-answering device, comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the knowledge question and answer method as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a program executable by a processor, wherein: The program executable by the processor is used to implement the knowledge question and answer method according to any one of claims 1 to 8 when executed by the processor.

Images