Answer generation method and device based on context traceability, equipment and medium
Generating markers through large language model prediction and context sensitivity detection solves the problem of language model hallucination, improves the reliability and quality of answers, and ensures that the answer is consistent with the target text.
Patent Information
- Application Number
- CN202411152877.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-27
AI Technical Summary
Language models are prone to hallucinations when generating answers, causing the answer to be separated from the specified text content, lack of semantic hierarchy and diversity, and difficult to evaluate authenticity, affecting the quality of answers.
The large language model predicts the generation of marks and performs context sensitivity detection, determines the marks based on the target text, and generates answers based on the mark correspondence relationship to ensure the context traceability of the answers.
It effectively avoids illusions, improves the reliability and quality of answers, and ensures that the content of the answer is consistent with the target text.
Smart Images
Figure CN119202154A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of language model application, and in particular to a method, device, equipment and medium for generating an answer based on context tracing. Background Art
[0002] In the field of language model application, language models can be used in question-answering scenarios. Among them, language models can generate corresponding answers to questions based on specified texts, but language model hallucinations may occur in the process of generating answers, resulting in errors in the generated answers that deviate from the specified text content.
[0003] For language models, if character-level matching is used to generate answers for a given text, the generated answers may lack semantic level comparison, resulting in poor diversity and inclusiveness of the answers, and poor answer quality. Moreover, language models have a "black box" feature, and it is difficult to evaluate the authenticity of the given text references during the answer generation process, making it difficult to avoid the defects of language model hallucinations.
[0004] How to reduce language model hallucination to improve the quality of generated answers is the technical problem to be solved by this application. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide a context-based answer generation method, apparatus, device and medium to reduce language model hallucinations to improve the quality of generated answers.
[0006] In a first aspect, a method for generating an answer based on context tracing is provided, comprising:
[0007] Predicting a plurality of generated tokens for a target text question based on the target text based on the large language model, wherein the plurality of generated tokens are used to generate a text answer corresponding to the text question;
[0008] Based on the target text, the multiple generated tokens are respectively subjected to context sensitivity index detection, where the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the multiple generated tokens;
[0009] Determining, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard;
[0010] The text answer is generated by the large language model based on tag correspondences, wherein the tag correspondences include correspondences between the target generation tags and the context tags.
[0011] In a second aspect, a context tracing-based answer generation device is provided, comprising:
[0012] A prediction module, which predicts a plurality of generated tokens for a target text question according to the target text based on a large language model, wherein the plurality of generated tokens are used to generate a text answer corresponding to the text question;
[0013] A detection module, performing context sensitivity index detection on each of the plurality of generated tokens based on the target text, wherein the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the plurality of generated tokens;
[0014] A determination module determines, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard;
[0015] A generation module generates the text answer through the large language model based on the tag correspondence, wherein the tag correspondence includes the correspondence between the target generation tag and the context tag.
[0016] According to a third aspect, an electronic device is provided. The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the method according to the first aspect are implemented.
[0017] According to a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method according to the first aspect are implemented.
[0018] In a fifth aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to execute part or all of the steps of the method of the first aspect.
[0019] In an embodiment of the present application, a large language model is used to predict a generation tag for generating a text answer to a target text question based on the target text, and then a context sensitivity index test is performed on multiple generation tags to determine the target generation tag that is affected by the target text during prediction. Then, the context tag based on which the large language model predicts the above-mentioned target generation tag is determined from the target text. Subsequently, based on the correspondence between the target generation tag and the context tag based on which it is based, a text answer is generated by the large language model. The text answer generated by the solution provided by the embodiment of the present application has the characteristic of context tracing, which avoids the illusion of the large language model in the process of generating text answers, thereby avoiding errors in the text answers that are out of touch with the content of the target text, thereby improving the reliability of the generated answers and improving the quality of the text answers. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0021] Figure 1a This is one of the flowcharts of a method for generating an answer based on context tracing in one embodiment of the present application;
[0022] Figure 1b This is a second flow chart of a method for generating an answer based on context tracing in an embodiment of the present application;
[0023] Figure 2 This is a flowchart of a method for generating an answer based on context tracing according to an embodiment of the present application;
[0024] Figure 3 This is a fourth flowchart of a method for generating an answer based on context tracing in an embodiment of the present application;
[0025] Figure 4a This is a flowchart of a method for generating an answer based on context tracing according to an embodiment of the present application;
[0026] Figure 4b This is a flowchart of a method for generating an answer based on context tracing according to an embodiment of the present application;
[0027] Figure 5 This is a flowchart of a method for generating an answer based on context tracing according to an embodiment of the present application, No. 7;
[0028] Figure 6 This is a flowchart of an answer generation method based on context tracing in one embodiment of the present application.
[0029] Figure 7a This is a ninth flowchart of a method for generating an answer based on context tracing in an embodiment of the present application;
[0030] Figure 7b This is a flowchart of a method for generating an answer based on context tracing according to an embodiment of the present application;
[0031] Figure 8 It is a structural diagram of an answer generation device based on context tracing in one embodiment of the present application. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. The numbering of the drawings in this application is only used to distinguish the various steps in the scheme, and is not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.
[0033] In order to solve the problems existing in the related art, the embodiment of the present application provides a method for generating answers based on context tracing. This solution involves natural language inference (NLI) and can be applied to question-answering application scenarios based on large language models. Figure 1a As shown, the following steps are included:
[0034] S11: Predicting a plurality of generated tags for a target text question according to the target text based on a large language model, wherein the plurality of generated tags are used to generate a text answer corresponding to the text question.
[0035] Large Language Model (LLM) can achieve multiple tasks such as text parsing and recognition, retrieval and screening, and text processing based on machine learning technology. In question-and-answer application scenarios, LLM can predict the corresponding text answer based on the target question. For example, the large language model can be instructed to predict the corresponding text answer to the target text question for a specific target text. In actual applications, LLM may fabricate non-existent information, causing the generated text answer to deviate from the content of the target text, resulting in errors in the text answer.
[0036] In this step, LLM is used to predict multiple generated tags for the target text question based on the target text, and these generated tags are used to generate text answers in subsequent steps. In the embodiment of the present application, the above-mentioned multiple generated tags are not directly used to generate text answers, so as to avoid including generated tags due to LLM hallucination in these generated tags, thereby avoiding errors in the text answers generated by this solution.
[0037] S12: Performing context sensitivity index detection on each of the multiple generated tokens based on the target text, where the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the multiple generated tokens.
[0038] In this step, the context sensitivity index of each target text is tested, and the context sensitivity index obtained by the test is used to indicate whether the corresponding generated tag has the characteristic of context sensitivity. Among them, the basis for the large language model to predict each generated tag is often different. Some generated tags are generated according to the original content of the target text. These generated tags are highly influenced by the target text and have strong authenticity. Another part of the generated tags may be generated by LLM hallucination. This part of the generated tags is separated from the real content of the target text and may contain errors.
[0039] Through this step, the context sensitivity index detected can indicate the degree to which the generated mark is influenced by the target text, so that the generated mark that can be traced back to the original text and the generated mark that cannot be traced back to the original text can be distinguished through the context sensitivity index.
[0040] Optionally, the context sensitivity index may specifically include one or more parameter values to indicate the context sensitivity of the corresponding generated token. For example, the context sensitivity index includes a context sensitivity value, and the larger the value, the stronger the context sensitivity of the corresponding generated token, that is, the stronger the dependence of the generated token on the target text predicted by the LLM.
[0041] Optionally, in this step, the features of the generated mark can be compared with the features of the target text to detect the context sensitivity index of the generated mark. In practical applications, various methods such as keyword retrieval and feature comparison can be combined according to actual needs.
[0042] S13: Determine, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard.
[0043] In this step, the generated tags that meet the preset context sensitivity standard can be first selected from multiple generated tags. For any target generated tag among these generated tags that meet the standard, it is determined which part of the context tags in the target text is specifically based on which the large language model predicts the target generated tag.
[0044] Through this step, the corresponding context tag can be determined for the target generated tag. Among them, LLM is implemented based on the corresponding context tag when predicting the target generated tag, so that the tag correspondence relationship between the target generated tag and the based context tag can be constructed.
[0045] In practical applications, the context tag corresponding to the target generation tag may be one or more.
[0046] S14: Generate the text answer through the large language model based on the tag correspondence, where the tag correspondence includes the correspondence between the target generation tag and the context tag.
[0047] In this step, based on the correspondence between the target generation tags and the context tags determined in the above steps, a text answer is generated accordingly.
[0048] Optionally, at least some of the target generation tags are selected from the target generation tags having a tag correspondence relationship to construct a text answer, so that the target generation tags used in the text answer all have the characteristic of traceable context, ensuring that the content of the text answer is supported by the original content of the target text and ensuring authenticity.
[0049] In actual application scenarios, due to the small number of target generation tokens or other reasons, there may be a situation where the target generation tokens are not enough to generate text answers. In this regard, we can try to increase the proportion of target generation tokens used in the process of generating text answers, and ensure that as much content as possible in the generated text answers has traceable context features, thereby improving the authenticity of the text answers and reducing the defects of large language model hallucinations.
[0050] The solution provided in the embodiment of the present application can be applied to the Retrieval Augmented Generation (RAG) scenario. Based on the specified target text, the user can input the target question text according to actual needs. The solution provided in the embodiment of the present application can realize the answer traceability at the sentence level or even the semantic entity level, improve the accuracy of answer attribution, and make the answer generated by LLM more reliable, avoiding problems such as LLM illusion.
[0051] Next, an example is used to further illustrate this solution. Figure 1b .
[0052] First, the user can input questions to the large language model, and the large language model generates generation tags for constructing answers based on the context blocks of the specified target text. Then, context-sensitive detection is performed on each generation tag to identify target generation tags with context-sensitive characteristics. Subsequently, context attribution is performed on these target generation tags to determine which context tags in the target text the large language model relied on when generating the target generation tags. Furthermore, an attribution pair can be generated based on the correspondence between the target generation tag and the context tag. The attribution pair can be used to improve the quality of the generated text answer, make the content in the text answer traceable, and avoid errors in the text answer caused by the large language model hallucination.
[0053] In an embodiment of the present application, a large language model is used to predict a generation tag for generating a text answer to a target text question based on the target text, and then a context sensitivity index test is performed on multiple generation tags to determine the target generation tag that is affected by the target text during prediction. Then, the context tag based on which the large language model predicts the above-mentioned target generation tag is determined from the target text. Subsequently, based on the correspondence between the target generation tag and the context tag based on which the text answer is generated, the text answer is generated by the large language model. The text answer generated by the solution provided in the embodiment of the present application has the characteristics of context tracing, which avoids the illusion of the large language model in the process of generating text answers, thereby avoiding errors in the text answers that are out of touch with the content of the target text, thereby improving the reliability of the generated answers and improving the quality of the text answers. This solution can be implemented directly inside the LLM without the need for additional natural language inference model evaluation outside the model, which can ensure the efficiency of answer generation and reduce computational costs.
[0054] Based on the solution provided in the above embodiment, optionally, Figure 2 As shown, in the above step S12, the context sensitivity index detection is performed on the multiple generated tags based on the target text, including:
[0055] S21: Determine, by contrast, a first prediction probability distribution of each generated token predicted by the large language model based on the target text, and a second prediction probability distribution of each generated token predicted by the large language model not based on the target text.
[0056] Assuming that the number of generated tokens predicted by the large language model is n, in this step, the corresponding prediction probabilities of the n generated tokens with and without context are determined one by one. The presence of context means that the large language model performs prediction based on the target text, and the absence of context means that the large language model does not perform prediction based on the target text. For the i-th generated token, the corresponding first prediction probability distribution can be expressed as The corresponding second predicted probability distribution can be expressed as Among them, ctx refers to the target text (Context), and no-ctx refers to not based on the target text.
[0057] Optionally, for the first predicted probability distributions corresponding to the multiple generated tags, a predicted probability distribution set in the presence of context can be constructed, which is expressed as For the second predicted probability distribution corresponding to multiple generated tags, a set of predicted probability distributions without context can be constructed, expressed as
[0058] S22: Determine a context sensitivity index of the generated mark to be detected based on the first predicted probability distribution and the second predicted probability distribution corresponding to the generated mark to be detected.
[0059] The generated mark to be detected refers to the generated mark for which the context sensitivity index has not been determined among the multiple generated marks. In this step, the context sensitivity index detection is performed on the multiple generated marks respectively to quantify the sensitivity of each generated mark to the context. The obtained context sensitivity index can be used to identify the target generated mark that is significantly affected by the context during the generation process.
[0060] For example, for the i-th generated tag, if the corresponding first predicted probability distribution With the second predicted probability distribution If the difference is greater than the preset standard, it indicates that the generated marker is greatly influenced by the target text during the generation process, that is, the generated marker has the characteristic of context sensitivity, and thus the generated marker can be identified as the target generated marker.
[0061] Based on the solution provided in the above embodiment, optionally, Figure 3 As shown, in the above step S22, based on the first predicted probability distribution and the second predicted probability distribution corresponding to the generated mark to be detected, determining the context sensitivity index of the generated mark to be detected includes:
[0062] S31: Based on the first predicted probability distribution and the second predicted probability distribution corresponding to the i-th generated marker to be detected among the multiple generated markers, determine the context sensitivity index of the generated marker to be detected by relative entropy.
[0063] In this application example, the first predicted probability distribution is expressed as The second prediction probability distribution is expressed as The context sensitivity index is denoted as m i ,in,
[0064] In this solution, the context sensitivity index of the generated mark to be detected is determined by relative entropy. Relative entropy can also be called KL divergence (Kullback-Leibler divergence), which is used to measure the degree of difference between two probability distributions. In this step, by quantifying the sensitivity of each generated mark to the context, those target generated marks that are significantly affected by the context during the generation process can be effectively identified.
[0065] Through this step, the context sensitivity index for each generated tag detection can be expressed in the form of a set as m= <m1,…,m n >.
[0066] The solution provided by the embodiment of the present application objectively determines the context sensitivity of each generated mark through relative entropy, and effectively realizes the quantification of context sensitivity. The detected context sensitivity index can effectively indicate whether the corresponding generated mark has the characteristic of context sensitivity, which is conducive to objectively and efficiently determining the target generated mark.
[0067] Based on the solution provided in the above embodiment, optionally, Figure 4a As shown, before the above step S13, that is, before determining the context tag based on which the large language model predicts the target generated tag from the target text, it also includes:
[0068] S41: Performing a binarization conversion on the context sensitivity index of the i-th generated marker among the multiple generated markers through a first selector function to obtain a generated marker binarization label, wherein the generated marker binarization label is used to characterize the i-th generated marker predicted by the large language model. The degree to which the target text affects the
[0069] In the present application example, the first selector function is represented by S c , the i-th generated tag is represented as The context sensitivity index of the i-th generated token is denoted as m i , the generated binary label obtained by conversion is represented as C, where
[0070] Among them, the first selector function can flexibly select a binary classification function according to actual needs, for example, a logistic regression function can be selected to implement it.
[0071] In this step, by performing a binary conversion through the first and selector functions, each generated mark can be classified into two categories according to the context sensitivity index, highlighting the difference between the two types of generated marks, and realizing efficient classification marking.
[0072] Among them, the converted binary label can be expressed as Multiple generation tags can be represented as a set:
[0073] S42: Determine the target generation mark from the multiple generation marks based on the generation mark binary label.
[0074] Since the binary label can significantly indicate whether the generated label has the characteristic of being context-sensitive, in this step, the target generated label can be efficiently determined from multiple generated labels based on the binary label.
[0075] In this step, a target generation mark can be selected from multiple generation marks by setting a threshold. The target generation mark satisfies: c (m i )=m i ≥m * Among them, m * Indicates the threshold value.
[0076] The above thresholds can be set by calibrating the threshold. Specifically, the model prediction and the answer attribution annotation set are combined to maximize the consistency. Specifically, by adjusting the threshold, the threshold with the highest matching degree between the model prediction answer attribution and the annotation is used as the calibration threshold m. * .
[0077] In the actual implementation process, the threshold interval (0,1) and the granularity value [0.01,0.1] can be designed to generate a candidate threshold set based on this. Each threshold corresponds to a model prediction result, which is compared with the annotation set. If the gap is large, it needs to be adjusted to other candidate thresholds until the prediction result is most similar to the annotation set gap. The candidate threshold set can be adjusted from small to large, and the current matching result is compared with the previous matching result to decide whether to increase or decrease. The high and low thresholds affect the effect of predicting the answer attribution.
[0078] Alternatively, the above thresholds can be set by example-level thresholding. Specifically, the discriminative calibration threshold method is adopted, which does not require the entire annotation set. The contrast score is calculated by a small sample set and its mean is calculated. With standard deviation σ m Calculate the threshold and set the threshold can be expressed as:
[0079] See also Figure 4b In the process shown in the embodiment of the present application, the context sensitivity of multiple generated tags is first quantified by KL divergence, and then the context sensitivity index is binarized, so as to effectively distinguish whether the generated tags are context sensitive. Then, the target generated tags with context sensitive characteristics can be efficiently and accurately identified based on the binarized labels.
[0080] Based on the solution provided in the above embodiment, optionally, Figure 5 As shown, in the above step S13, determining from the target text the context tag based on which the large language model predicts the target generated tag includes:
[0081] S51: Determine attribution scores for the context tags in the target text respectively, where the attribution scores are used to indicate the influence of the corresponding context tags on the target generated tags predicted by the large language model.
[0082] The target generation mark determined in the above embodiment is greatly affected by the context during the generation process. Based on this, in this step, an attribution score is determined for each context mark in the target text to evaluate which context marks specifically affect the target generation mark during the prediction generation process.
[0083] S52: Determine the context tag whose attribution score is greater than the preset attribution score as the context tag based on which the large language model predicts the target generated tag.
[0084] The attribution score can quantify the influence of the corresponding context tag on the target generated tag predicted by the large language model. In this step, the context tags are screened based on the preset attribution scores to determine the context tags with the greatest influence on the predicted target generated tag.
[0085] Through the solution provided in the embodiment of the present application, the influence of context tags on target generation tags can be quantified based on the attribution score, thereby objectively and effectively identifying context tags that are closely related to the target generation tags.
[0086] Based on the solution provided in the above embodiment, optionally, Figure 6 As shown, in the above step S51, attribution scores are determined for the context tags in the target text respectively, including:
[0087] S61: Determine a third prediction probability of the target generated tag obtained by the large language model based on the text prediction containing the target context tag and a fourth prediction probability of the target generated tag obtained by the large language model based on the text prediction not containing the target context tag, wherein the target context tag is any context tag in the target text.
[0088] The solution provided in the embodiment of the present application is used to generate tags for identified targets and determine which specific context tags have a significant impact on predicting the generated tags for these targets.
[0089] In this step, for a target generation tag, the probability of LLM predicting the target generation tag based on two texts is determined respectively. Among them, the first text is the text containing the target context tag, and the third prediction probability is obtained. The second text is the text that does not contain the target context tag, and the fourth prediction probability is obtained. The difference between the two texts is whether they contain the target context tag. If the difference between the two corresponding prediction probabilities is large, it indicates that the target context tag has a large influence on the prediction of the target generation tag. On the contrary, if the difference between the two corresponding prediction probabilities is small, it indicates that the target context tag has a small influence on the prediction of the target generation tag.
[0090] Optionally, for multiple target generated tokens with context-sensitive characteristics, an alternative version without the original context is generated. The contrast tokens generated by the large language model based on the alternative version without the original context can be represented as a set: Based on this, for the i-th target generated text, the corresponding third prediction probability is expressed as The fourth predicted probability is expressed as
[0091] S62: Determine an attribution score of the target contextual tag relative to the target generated tag based on the third predicted probability and the fourth predicted probability.
[0092] In this step, contrast feature attribution is used to calculate the attribution score of each context tag in the context. The attribution score is quantified by calculating the L2 norm of the difference gradient vector between the predicted probabilities. The attribution score can be expressed as in,
[0093] in, For context ctx j The gradient L2 norm of the difference in predicted probabilities, For the model based on the context ctx j The predicted probability, For models not based on context ctx j The predicted probability.
[0094] Through the solution provided by the embodiment of the present application, an attribution score can be determined for each context tag for the target generation tag, thereby quantifying the degree to which the context tag affects the prediction of the target generation tag. The attribution score generated by this solution can quantitatively represent the correspondence between the target generation tag and the target context tag.
[0095] Based on the solution provided in the above embodiment, optionally, as shown in FIG7 , in the above step S52, determining the context tag whose attribution score is greater than the preset attribution score as the context tag based on which the large language model predicts to obtain the target generated tag includes:
[0096] S71: Perform a binarization conversion on the attribution score of the j-th context marker among the multiple context markers relative to the i-th generated marker through a second selector function to obtain a context marker binarization label, wherein the context marker binarization label is used to characterize the degree of influence of the j-th context marker when the large language model predicts the i-th generated marker.
[0097] In the present application example, the second selector function is represented by S I , the jth context token is denoted as ctx j , the attribution score of the i-th generated tag is expressed as The converted contextual label binary label is expressed as
[0098] Among them, the second selector function can flexibly select a binary classification function according to actual needs, for example, a logistic regression function can be selected to implement it.
[0099] In this step, the attribution score is converted into a binary value through the selector function, and each context tag can be classified into two categories according to the attribution score, highlighting the difference between the two types of context tags, and clearly indicating whether the context tags have an important impact on the prediction of the target generated tag.
[0100] Among them, the converted contextual label binary label can be expressed as
[0101] S72: Determine, from the multiple context tags based on the context tag binarization label, the context tag on which the large language model predicts the target generated tag.
[0102] In this step, feature attribution can be selected by setting a threshold. The selected domain target generated tag has a corresponding context tag which can be expressed as Among them, a i* Indicates the threshold value, which can be set flexibly according to actual needs.
[0103] Optionally, in an actual application, the binary labels corresponding to multiple context tags may be sorted in descending order, so as to select the Top-K context tags and determine the context tags based on which the large language model predicts the target generated tags.
[0104] Optionally, the target generation tag and the corresponding context tag are stored correspondingly to form an attribution pairing set, which can be expressed as:
[0105] See also Figure 7b The process shown in the figure, the scheme provided by the embodiment of the present application can be used to implement feature attribution. Among them, contrast features are generated for multiple context tags to highlight the impact of each context tag on the target generated tag. Then, the attribution score is calculated for each context tag to quantify the impact of each context tag on the target generated tag. Furthermore, efficient binary classification is achieved for each context tag through binary labels, so as to select feature attribution and realize the corresponding association between the target generated tag and the context tag.
[0106] The solution provided in the embodiment of the present application is implemented based on the gradient-based feature attribution technology, which ensures the authenticity of the attribution, reflects the contextual information actually used by the model in the generation process, and can effectively identify the contextual tags that truly affect the generation of the answer, thereby providing more realistic attribution.
[0107] In order to solve the problems existing in the related art, the embodiment of the present application also provides an answer generation device 80 based on context tracing, such as Figure 8 As shown, including:
[0108] A prediction module 81 predicts a plurality of generated tokens for a target text question based on a large language model according to the target text, wherein the plurality of generated tokens are used to generate a text answer corresponding to the text question;
[0109] A detection module 82 performs context sensitivity index detection on each of the plurality of generated tokens based on the target text, wherein the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the plurality of generated tokens;
[0110] A determination module 83 determines, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard;
[0111] The generating module 84 generates the text answer through the large language model based on the tag correspondence relationship, wherein the tag correspondence relationship includes the correspondence relationship between the target generation tag and the context tag.
[0112] The device provided in the embodiment of the present application predicts generation tags for generating text answers to target text questions based on the target text through a large language model, and then performs context sensitivity index detection on multiple generation tags to determine the target generation tags that are affected by the target text during prediction. Then, the context tags based on which the large language model predicts the above-mentioned target generation tags are determined from the target text. Subsequently, based on the correspondence between the target generation tags and the context tags based on which they are based, a text answer is generated through the large language model. The text answer generated by the solution provided in the embodiment of the present application has the characteristic of context tracing, which avoids the illusion of the large language model in the process of generating text answers, thereby avoiding errors in the text answers that are out of touch with the content of the target text, thereby improving the reliability of the generated answers and improving the quality of the text answers.
[0113] Among them, the above modules in the device provided by the embodiment of the present application can also implement the method steps provided by the above method embodiment. Alternatively, the device provided by the embodiment of the present application can also include other modules in addition to the above modules to implement the method steps provided by the above method embodiment. And the device provided by the embodiment of the present application can achieve the technical effects that can be achieved by the above method embodiment.
[0114] Preferably, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned embodiment of the answer generation method based on context tracing are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0115] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned embodiment of the answer generation method based on context tracing is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0116] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program can be operated to enable a computer to execute part or all of the steps of the above-mentioned embodiment of the answer generation method based on context tracing, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0118] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.
[0119] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0122] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0123] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0124] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0125] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0126] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for generating an answer based on context tracing, characterized in that: include: Predicting a plurality of generated tokens for a target text question based on the target text based on the large language model, wherein the plurality of generated tokens are used to generate a text answer corresponding to the text question; Based on the target text, the multiple generated tokens are respectively subjected to context sensitivity index detection, where the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the multiple generated tokens; Determining, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard; The text answer is generated by the large language model based on tag correspondences, wherein the tag correspondences include correspondences between the target generation tags and the context tags.
2. The method according to claim 1, characterized in that Performing context sensitivity index detection on the multiple generated tags based on the target text respectively includes: Determine, by contrast, a first prediction probability distribution of each generated token predicted by the large language model based on the target text, and a second prediction probability distribution of each generated token predicted by the large language model without being based on the target text; Based on the first predicted probability distribution and the second predicted probability distribution corresponding to the generated mark to be detected, a context sensitivity index of the generated mark to be detected is determined.
3. The method according to claim 2, characterized in that Determining a context sensitivity index of the generated mark to be detected based on a first predicted probability distribution and a second predicted probability distribution corresponding to the generated mark to be detected includes: Based on the first predicted probability distribution and the second predicted probability distribution corresponding to the i-th generated marker to be detected among the multiple generated markers, a context sensitivity index of the generated marker to be detected is determined by relative entropy.
4. The method according to claim 3, characterized in that Before determining from the target text the context tag based on which the large language model predicts the target generated tag, the method further includes: Performing a binarization conversion on the context sensitivity index of the i-th generated token among the multiple generated tokens through a first selector function to obtain a generated token binarization label, wherein the generated token binarization label is used to represent the degree of influence of the target text when the large language model predicts the i-th generated token; The target generation mark is determined from the plurality of generation marks based on the generation mark binarization label.
5. The method according to any one of claims 1 to 4, characterized in that: Determining, from the target text, a context tag based on which the large language model predicts a target generated tag, comprising: Determining attribution scores for the context tags in the target text respectively, wherein the attribution scores are used to indicate the influence of the corresponding context tags on the target generated tags predicted by the large language model; The context tag whose attribution score is greater than the preset attribution score is determined as the context tag based on which the large language model predicts the target generated tag.
6. The method according to claim 5, characterized in that Determining attribution scores for the context tags in the target text respectively includes: Determine a third prediction probability of the target generated tag obtained by the large language model based on the text prediction containing the target context tag and a fourth prediction probability of the target generated tag obtained by the large language model based on the text prediction not containing the target context tag, wherein the target context tag is any context tag in the target text; An attribution score for the target contextual tag relative to the target generated tag is determined based on the third predicted probability and the fourth predicted probability.
7. The method according to claim 6, characterized in that Determining the context tag whose attribution score is greater than the preset attribution score as the context tag based on which the large language model predicts to obtain the target generated tag includes: Performing a binarization conversion on the attribution score of the j-th context marker among the multiple context markers relative to the i-th generated marker by a second selector function to obtain a context marker binarization label, wherein the context marker binarization label is used to characterize the degree of influence of the j-th context marker when the large language model predicts the i-th generated marker; The context tag based on which the large language model predicts the target generated tag is determined from the multiple context tags based on the context tag binarization label.
8. A context-based answer generation device, characterized in that: include: A prediction module, which predicts a plurality of generated tokens for a target text question according to the target text based on a large language model, wherein the plurality of generated tokens are used to generate a text answer corresponding to the text question; A detection module, performing context sensitivity index detection on each of the plurality of generated tokens based on the target text, wherein the context sensitivity index is used to indicate the degree to which the large language model is influenced by the target text when predicting the plurality of generated tokens; A determination module determines, from the target text, a context tag based on which the large language model predicts a target generated tag, wherein the target generated tag is any generated tag whose context sensitivity index satisfies a preset context sensitivity standard; A generation module generates the text answer through the large language model based on the tag correspondence, wherein the tag correspondence includes the correspondence between the target generation tag and the context tag.
9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.