Knowledge boundary determination method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a knowledge boundary determination method and device, a storage medium and electronic device.The method comprises the steps that when a first model outputs second information based on input first information, a first uncertainty quantization parameter of the first model is calculated; obtaining a second uncertainty quantization parameter output by the second model; calculating a target uncertainty quantization parameter of the first model by using the first uncertainty quantization parameter and the second uncertainty quantization parameter; and determining a knowledge boundary of the first model according to the target uncertainty parameter. By means of the method and device, the problem that the knowledge boundary cannot be accurately determined in the related technology is solved, and then the effect of accurately determining the knowledge boundary of the model is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the computer field, and specifically, to a method and device for determining a knowledge boundary, a storage medium, and an electronic device. Background Art

[0002] With the rapid development of deep learning and natural language processing technology, large language models (LLMs) have become the cornerstone of applications in many fields, from text generation, question-answering systems to translation and creation. These models can understand and generate human-level language by training massive amounts of text data.

[0003] When large language models generate responses, the credibility and uncertainty of their output is a key issue. Models may confidently generate wrong, inaccurate, or out-of-distribution (OOD) information, which is more serious outside the model's knowledge boundary. Therefore, quantifying the uncertainty of the model, that is, accurately assessing the model's confidence level in a certain prediction or generated content, is the basis for ensuring the reliability of the model output.

[0004] In practical applications, especially in high-risk areas such as finance, medical and legal decision-making, the output of the model needs to be not only accurate, but also consistent and robust when facing new data or edge cases. The model needs to be able to self-assess the reliability of its output and express this uncertainty when it is uncertain. However, the methods in related technologies find it difficult to accurately assess the uncertainty of the model. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining a knowledge boundary, a storage medium, and an electronic device, so as to at least solve the problem that the knowledge boundary cannot be accurately determined in the related art.

[0006] According to an embodiment of the present application, a method for determining a knowledge boundary is provided, comprising: when the first model outputs second information based on input first information, calculating a first uncertainty quantification parameter of the first model; obtaining a second uncertainty quantification parameter output by the second model, wherein the second uncertainty quantification parameter is an uncertainty quantification parameter output by the second model based on input first model feature information, and the first model feature information is model feature information when the first model outputs the second information; using the first uncertainty quantification parameter and the second uncertainty quantification parameter, calculating a target uncertainty quantification parameter of the first model; determining the knowledge boundary of the first model according to the target uncertainty parameter; wherein the first uncertainty quantification parameter and the second uncertainty parameter are both used to represent indicators of the degree of confidence in the output of the second information by the first model based on the input first information.

[0007] In an exemplary embodiment, when the first model outputs second information based on the input first information, the first uncertainty quantification parameter of the first model is calculated, including: respectively calculating the third uncertainty quantification parameters of N first word-grams, wherein the N first word-grams are all word-grams generated by the first model based on the first information, the N word-grams are all used to construct the second information, the N is a natural number greater than or equal to 1, and the third uncertainty quantification parameter is used to represent an indicator of the degree of confidence in the generation of the first word-gram by the first model; and calculating the first uncertainty quantification parameter based on the third uncertainty quantification parameters of the N first word-grams.

[0008] In an exemplary embodiment, the third uncertainty quantization parameters of the N first word-grams are calculated respectively, including: determining M candidate word-grams of a target word-gram, wherein the target word-gram is any word-gram among the N first word-grams, the M candidate word-grams are all word-grams whose semantic similarity with the target word-gram satisfies a first threshold, and the M is a natural number greater than or equal to 1; calculating the probability of the target word-gram and the M candidate word-grams appearing in the second information to obtain M+1 first probabilities; and calculating the third uncertainty quantization parameter based on the M+1 first probabilities.

[0009] In an exemplary embodiment, the first uncertainty quantization parameter is calculated based on the third uncertainty quantization parameters of the N first word elements, including: performing weighted calculation on the weight coefficients of the N first word elements and the third uncertainty quantization parameters of the N first word elements to obtain the first uncertainty quantization parameter, wherein the weight coefficient represents the semantic contribution of the first word element in the process of generating the second information.

[0010] In an exemplary embodiment, the second model is a model obtained by iteratively training the second initial model using a sample data set until the number of times the second initial model is trained reaches a target number of times, or the loss value output by the target loss function of the second initial model meets a preset training end condition. The iterative training process includes: training the second initial model for the kth time through the following steps, wherein k is a positive integer: initializing the sample parameter group of the second initial model trained for the k-1th time, wherein the sample parameter group includes the leaf node threshold of the sample decision tree to be constructed and the depth of the sample decision tree to be constructed; obtaining P sub-data sets from the sample data set for the kth training of the second initial model, wherein the P sub-data sets each include first sample information, second sample information, third sample information and first parameter, wherein the first sample information is input information, the second sample information is information output by the first model based on the first sample information, and the The three sample information is the true response information of the above-mentioned first information, the above-mentioned first parameter is used to represent the difference between the above-mentioned second sample information and the above-mentioned third sample information, and the above-mentioned P is a natural number greater than or equal to 1; the above-mentioned P sub-data sets are input into the above-mentioned second initial model of the above-mentioned k-1th training to obtain the uncertainty quantification parameter of the kth training, wherein the above-mentioned uncertainty quantification parameter of the above-mentioned k-1th training is used to represent the index of the confidence degree of the above-mentioned second initial model of the above-mentioned k-1th training in outputting the above-mentioned second sample information for the above-mentioned first sample information in the P above-mentioned sub-data sets; according to the above-mentioned P sub-data sets, the uncertainty quantification parameter of the above-mentioned k-th training and the above-mentioned third sample information, determine the target loss value output by the loss function of the above-mentioned k-th training; when the above-mentioned k is less than the above-mentioned target number, or when the above-mentioned target loss value does not meet the above-mentioned training end condition, adjust the value of the above-mentioned sample parameter group in the above-mentioned second initial model of the above-mentioned k-1th training to obtain the above-mentioned second initial model of the above-mentioned k-1th training.

[0011] In an exemplary embodiment, the second initial model of the k-1th training outputs the uncertainty quantification parameters of the kth training through the following steps: constructing P sample decision trees based on the P sub-data sets and the sample parameter group; determining P second parameters output by the P sample decision trees, wherein the second parameters are used to represent the numerical value output by the sample decision trees; and outputting the uncertainty quantification parameters of the kth training based on the P second parameters.

[0012] In an exemplary embodiment, determining the knowledge boundary of the above-mentioned first model according to the above-mentioned target uncertainty parameters includes: obtaining a target uncertainty parameter set, wherein the above-mentioned target uncertainty parameter set includes multiple historical uncertainty parameters, and the above-mentioned historical uncertainty parameters are used to represent the uncertainty quantification parameters determined by the above-mentioned first model when outputting information; analyzing the distribution between the above-mentioned target uncertainty parameters and the multiple historical uncertainty parameters to obtain the target distribution; determining a candidate threshold set based on the above-mentioned target distribution; determining the target threshold from the above-mentioned candidate threshold set to obtain the knowledge boundary of the above-mentioned first model.

[0013] In an exemplary embodiment, determining a target threshold from the candidate threshold set to obtain the knowledge boundary of the first model includes: obtaining a verification data set, wherein the verification data set includes K input information, and K is a natural number greater than or equal to 1; using the verification data set to evaluate the fitness of the candidate thresholds in the candidate threshold set, wherein the following operations are performed for each candidate threshold in the candidate threshold set to determine the fitness of the candidate threshold: when the candidate threshold is determined as the knowledge boundary of the first model, in the i-th round of evaluation, the first model outputs K fourth information based on the verification data set, and the candidate threshold is any threshold in the candidate threshold set; calculating uncertainty quantification parameters of the K fourth information; calculating the probability that the first model correctly outputs the fourth information based on the K uncertainty quantification parameters of the fourth information to determine the fitness of the candidate threshold; determining the target threshold based on the K fitnesses to obtain the knowledge boundary of the first model.

[0014] In an exemplary embodiment, after determining the knowledge boundary of the first model according to the target uncertainty parameter, the method further includes: obtaining a fifth uncertainty quantification parameter calculated by the first model when outputting the sixth information based on the input fifth information, wherein the fifth uncertainty quantification parameter is used to represent an indicator of the degree of confidence of the first model in outputting the sixth information based on the input fifth information; comparing the fifth uncertainty quantification parameter with the knowledge boundary to obtain a comparison result; if the comparison result is a first preset result, instructing the first model to output the sixth information, wherein the first preset result is used to indicate that the fifth information is within the knowledge boundary; if the comparison result is a second preset result, instructing the first model to stop outputting the sixth information and sending a prompt message, wherein the second preset result is used to indicate that the fifth information exceeds the knowledge boundary, and the prompt message is used to prompt the user to adjust the fifth information.

[0015] According to another embodiment of the present application, a device for determining a knowledge boundary is provided, including: a first calculation module, used to calculate a first uncertainty quantification parameter of the first model when the first model outputs second information based on input first information; an acquisition module, used to obtain a second uncertainty quantification parameter output by the second model, wherein the second uncertainty quantification parameter is an uncertainty quantification parameter output by the second model based on input first model feature information, and the first model feature information is model feature information when the first model outputs the second information; a second calculation module, used to calculate a target uncertainty quantification parameter of the first model using the first uncertainty quantification parameter and the second uncertainty quantification parameter; a determination module, used to determine the knowledge boundary of the first model according to the target uncertainty parameter, wherein the first uncertainty quantification parameter and the second uncertainty parameter are both indicators used to represent the degree of confidence that the first model outputs the second information based on the input first information.

[0016] According to another embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0017] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0018] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0019] Through the present application, when the first model outputs the second information based on the input first information, the uncertainty quantization parameter of the first model is calculated, the second uncertainty quantization parameter is obtained from the second model, and the target uncertainty quantization parameter of the first model is calculated by combining the first and second uncertainty quantization parameters, and then the knowledge boundary of the first model is determined based on the value of the target uncertainty quantization parameter. Therefore, the problem that the knowledge boundary cannot be accurately determined in the related art can be solved, thereby achieving the effect of accurately determining the knowledge boundary of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a hardware environment schematic diagram of a method for determining a knowledge boundary according to an embodiment of the present application;

[0021] Figure 2is a flow chart of a method for determining a knowledge boundary according to an embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of large model knowledge boundary identification and expression based on multi-source uncertainty quantization score fusion according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of large model knowledge boundary identification and expression according to an embodiment of the present application;

[0024] Figure 5 is a flow chart of a knowledge boundary according to an embodiment of the present application;

[0025] Figure 6 It is a structural block diagram of a device for determining a knowledge boundary according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 FIG. 1 is a schematic diagram of a hardware environment of a method for determining a knowledge boundary in an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for determining a knowledge boundary in an embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to a server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a method for determining a knowledge boundary is provided. Figure 2 is a flow chart of a method for determining a knowledge boundary according to an embodiment of the present application, such as Figure 2 As shown, the process includes the following steps:

[0032] Step S202, when the first model outputs second information based on the input first information, calculate a first uncertainty quantization parameter of the first model;

[0033] Optionally, this embodiment can be applied in decision support fields that require high accuracy, including but not limited to scientific research, intelligent customer service usage scenarios, scenarios for providing financial investment advice, or the medical field for diagnosing patient diseases.

[0034] Optionally, the first model in this embodiment can be a large model, which is used to output corresponding text or answer questions based on the information input by the user. It is a pre-trained language model with large-scale parameters, for example, a generative pre-trained model (Generative Pre-trained Transformer, abbreviated as GPT), a bidirectional encoder representation converter (Bidirectional Encoder Representations from Transformers, abbreviated as BERT), etc.

[0035] Optionally, the first uncertainty quantification parameter in this embodiment can be a gray box uncertainty quantification score, that is, a probability evaluation model based on the information output by the model in its generation or prediction process, focusing on the output results of the model without the need for in-depth understanding of the internal operating mechanism of the model.

[0036] Step S204, obtaining a second uncertainty quantification parameter output by the second model, wherein the second uncertainty quantification parameter is an uncertainty quantification parameter output by the second model based on the input first model feature information, and the first model feature information is model feature information when the first model outputs the second information;

[0037] Optionally, the second model in this embodiment can be a machine learning model, which is a model for uncertainty quantification, such as a random forest model, which outputs uncertainty quantification parameters based on the internal state (white box features) of the first model.

[0038] Optionally, the first model feature information in this embodiment refers to the internal state information of the first model when it outputs the second information, such as the hidden activation state of each layer, attention weight, etc., which reflects the internal working state of the large model when processing input information.

[0039] Optionally, the second uncertainty quantization parameter in this embodiment can be a white-box uncertainty quantization score, that is, estimating the uncertainty of the model in the generation or prediction process by analyzing the internal state or behavior of the model, going deep into the model, and using its hidden layer activation, attention mechanism and other internal parameters to determine whether the model's response to specific input is reliable.

[0040] Step S206, calculating a target uncertainty quantification parameter of the first model using the first uncertainty quantification parameter and the second uncertainty quantification parameter;

[0041] Optionally, the target uncertainty quantification parameter in this embodiment is the result of combining the first uncertainty quantification parameter and the second uncertainty quantification parameter, which comprehensively reflects the uncertainty level when the large model (i.e., the first model mentioned above) outputs the second information, and can be calculated by weighted averaging or other comprehensive methods.

[0042] Optionally, the target uncertainty quantization parameter in this embodiment may be called a comprehensive uncertainty quantization score, that is, a score obtained by combining the above-mentioned gray box uncertainty quantization score and the white box uncertainty quantization score.

[0043] Optionally, in this embodiment, the target uncertainty quantization parameter may be obtained by using the first uncertainty quantization parameter and the second uncertainty quantization parameter in a weighted calculation manner.

[0044] Step S208, determining the knowledge boundary of the first model according to the target uncertainty parameter; wherein the first uncertainty quantification parameter and the second uncertainty parameter are both indicators for indicating the confidence level of the first model in outputting the second information based on the first information input.

[0045] Optionally, the knowledge boundary in this embodiment is the limit range of the output capability of the large model (i.e., the first model mentioned above), beyond which the output of the model may become unreliable or produce hallucinations, helping the model to identify which inputs can safely generate outputs and which inputs may exceed its capabilities.

[0046] The execution order of step S202 and step S204 is interchangeable, that is, step S204 may be executed first, and then step S202.

[0047] The steps in this embodiment are as follows: Figure 3 As shown, the knowledge boundary identification and expression of the large model based on the fusion of multi-source uncertainty quantization scores:

[0048] Step S302, calculating the gray box uncertainty quantization score;

[0049] Step S304, calculating a white box uncertainty quantization score;

[0050] Step S306, using the gray box uncertainty quantification score and the white box uncertainty quantification score, calculate a comprehensive uncertainty quantification score;

[0051] Step S308, obtaining the knowledge boundary based on the comprehensive uncertainty quantification score, and expressing the knowledge boundary.

[0052] The execution order of step S302 and step S304 is interchangeable, that is, step S304 may be executed first, and then step S302.

[0053] Through the above steps, when the first model outputs the second information based on the input first information, the uncertainty quantification parameter of the first model is calculated, the second uncertainty quantification parameter is obtained from the second model, and the target uncertainty quantification parameter of the first model is calculated by combining the first and second uncertainty quantification parameters, and then the knowledge boundary of the first model is determined based on the value of the target uncertainty quantification parameter. Therefore, the problem that the knowledge boundary cannot be accurately determined in the related art can be solved, thereby achieving the effect of accurately determining the knowledge boundary of the model.

[0054] In an exemplary embodiment, when the first model outputs second information based on the input first information, the first uncertainty quantification parameter of the first model is calculated, including: respectively calculating the third uncertainty quantification parameters of N first word-grams, wherein the N first word-grams are all word-grams generated by the first model based on the first information, the N word-grams are all used to construct the second information, the N is a natural number greater than or equal to 1, and the third uncertainty quantification parameter is used to represent an indicator of the degree of confidence in the generation of the first word-gram by the first model; and calculating the first uncertainty quantification parameter based on the third uncertainty quantification parameters of the N first word-grams.

[0055] Optionally, the first word-grams in this embodiment are word-grams generated by the first model based on the input first information (for example, the user's question) in the process of generating the second information (such as an answer, text), and these word-grams are the basic units that constitute the answer or text.

[0056] Optionally, in this embodiment, the third uncertainty quantification parameter is an indicator used to represent the degree of confidence or uncertainty when the first model generates each word unit, that is, the uncertainty quantification of each word unit can be obtained by calculating the probability distribution entropy of the word unit or using other statistical methods.

[0057] Optionally, the large language model in this embodiment generates text in an autoregressive manner, and the model gradually predicts the next word based on the input first information x and the previously generated sequence s <i ={z 1 , z 2 , …, z i-1}, the i-th word z i The generation probability can be expressed as p(z i |s <i , x), Predictive Entorpy (PE) is used to measure the uncertainty of the model for the entire sentence s during the generation process and can be expressed as Among them, 1≤i≤N, s is a complete sentence consisting of N words, -logp(z i |s <i , x) is the word zi Raw prediction entropy, prediction entropy is the accumulation of uncertainty of each word in the entire sentence. A higher prediction entropy indicates a greater uncertainty in the generation process. In this embodiment, prediction entropy is used as a benchmark method for quantifying uncertainty.

[0058] For example, suppose a large model is deployed in an intelligent customer service system to answer users' inquiries about the weather. The first information input by the user is: "What will the weather be like in Beijing tomorrow?" The answer generated by the model (i.e., the second information mentioned above) is "The weather in Beijing tomorrow is predicted to be sunny, with a temperature between 16 degrees Celsius and 25 degrees Celsius." The N first word units may include "tomorrow," "Beijing," "of," "weather," "forecast," "for," "sunny," "temperature," "at," "16 degrees Celsius," "to," "25 degrees Celsius," and "between." The third uncertainty quantification parameter may be calculated for each word unit; for example, for the word unit "sunny," the model may have sufficient data and information to determine the reliability of its generation, so its third uncertainty quantification parameter is low; while "16 degrees Celsius" and "25 degrees Celsius" may have higher uncertainty when the model generates them due to sparse data or rapid changes in the external environment, and their third uncertainty quantification parameters are higher. By comprehensively calculating the third uncertainty quantification parameter of each word unit, the first uncertainty quantification parameter of the overall answer can be obtained.

[0059] Through the above steps, the third uncertainty quantization parameters of the N generated word-grams are calculated, and then the first uncertainty quantization parameters are calculated based on the third uncertainty quantification parameters of the N word-grams, so that the model is more objective when evaluating uncertainty, thereby improving the uncertainty quantification accuracy.

[0060] In an exemplary embodiment, the third uncertainty quantization parameters of the N first word-grams are calculated respectively, including: determining M candidate word-grams of a target word-gram, wherein the target word-gram is any word-gram among the N first word-grams, the M candidate word-grams are all word-grams whose semantic similarity with the target word-gram satisfies a first threshold, and the M is a natural number greater than or equal to 1; calculating the probability of the target word-gram and the M candidate word-grams appearing in the second information to obtain M+1 first probabilities; and calculating the third uncertainty quantization parameter based on the M+1 first probabilities.

[0061] Optionally, the candidate word in this embodiment is a word that is semantically similar to the target word, and its similarity exceeds a set first threshold. For example, candidate word elements similar to "sunny" may include "sunshine", "bright", "clear", etc.

[0062] Optionally, the M+1 first probabilities in this embodiment include the probabilities of the target word and its M candidate word elements appearing in the generated second information (text). For example, in the example of generating a weather forecast, the large model may calculate that the probability of "clear" appearing is 0.4, while the probability of "sunny" is 0.3, the probability of "bright" is 0.2, and the probability of "sunny" is 0.1. These probabilities constitute the M+1 first probabilities.

[0063] Optionally, in this embodiment, candidate words with similar semantic relationships are clustered and their probability distributions are merged. Semantic relationships refer to the association in meaning between words, phrases or sentences, which can usually be divided into three types: entailment, contradiction and neutrality.

[0064] Optionally, in this embodiment, for English words, microsoft-deberta-large-mnli can be used to perform semantic relationship judgment, and for Chinese, other languages, and cross-language words, the MoritzLaurer-MoritzLaurermDeBERTa-v3-base-xnli-multilingual-nli-2mil7 model can be used to perform semantic relationship judgment.

[0065] Optionally, in this embodiment, the third uncertainty quantization parameter based on the N first word-grams may be a first uncertainty quantization parameter calculated in a weighted manner.

[0066] Optionally, in this embodiment, if the first information x and the sentence For the generated single word z i (1≤i≤N j ), classify the other candidate word-grams of the word-gram (i.e., the original word-gram below), and merge the word-grams that have an implied relationship with the original generated word-gram by probability. The probability of the original generated word-gram can be updated as in, represents the sum of the probabilities of all candidate words that are judged to have an implied relationship with the generated word, including the original word. The original word z i (1≤i≤N j ) is PE(z i ,s j , x) = -logp e (z i ).

[0067] Through the above steps, M semantically similar candidate words are determined for each generated word, and the probabilities of the target word and the M candidate words appearing in the second information are calculated to obtain M+1 first probabilities. Then, based on the M+1 first probabilities, the third uncertainty quantification parameter of the word is calculated. By merging the probability distribution of words that are semantically similar to the target word, the uncertainty caused by synonyms and generation diversity is alleviated, making the uncertainty quantification at the word level more accurate.

[0068] In an exemplary embodiment, the first uncertainty quantization parameter is calculated based on the third uncertainty quantization parameters of the N first word elements, including: performing weighted calculation on the weight coefficients of the N first word elements and the third uncertainty quantization parameters of the N first word elements to obtain the first uncertainty quantization parameter, wherein the weight coefficient represents the semantic contribution of the first word element in the process of generating the second information.

[0069] Optionally, the weight coefficient in this embodiment represents the semantic contribution of the first word in the process of generating the second information, and is used to measure the semantic contribution of a word z. i The importance of reflecting the semantics of the second information s.

[0070] Optionally, in this embodiment, the weight coefficient of each first word can be determined by the following method: comparing the degree of change in the semantics of the sentence before and after the word is removed, expressed as Among them, g(·,·) is a function used to measure the semantic similarity of two sentences, with a value range of 0 to 1. The Cross-Encoder-RoBERTa-large model can be used as a method for calculating semantic similarity. x∪s represents the combination of the first input information x and the second input information s. Indicates that z has been removed i Sentences, SC(z i , s, x) is larger, indicating that z is removed i The semantic change caused by i Has more semantic information.

[0071] Optionally, the first uncertainty quantization parameter UQ in this embodiment is gray-box (s j , x) is the sum of the third uncertainty quantization parameters of the N first word units after reweighting: Assume that the first input information is x and the second generated information is Word z i (1≤i≤N j ) has a semantic contribution ratio of SC(z i , s, x) represents word z i The original semantic contribution of represents the total original semantic contribution of N words in the second information, then the first uncertainty quantization parameter in, PE(z i ,s j , x) is the word z i The third uncertainty quantification parameter, is the semantic contribution ratio of the word.

[0072] In an exemplary embodiment, the second model is a model obtained by iteratively training the second initial model using a sample data set until the number of times the second initial model is trained reaches a target number of times, or the loss value output by the target loss function of the second initial model meets a preset training end condition. The iterative training process includes: training the second initial model for the kth time through the following steps, wherein k is a positive integer: initializing the sample parameter group of the second initial model trained for the k-1th time, wherein the sample parameter group includes the leaf node threshold of the sample decision tree to be constructed and the depth of the sample decision tree to be constructed; obtaining P sub-data sets from the sample data set for the kth training of the second initial model, wherein the P sub-data sets each include first sample information, second sample information, third sample information and first parameter, wherein the first sample information is input information, the second sample information is information output by the first model based on the first sample information, and the The three sample information is the true response information of the above-mentioned first information, the above-mentioned first parameter is used to represent the difference between the above-mentioned second sample information and the above-mentioned third sample information, and the above-mentioned P is a natural number greater than or equal to 1; the above-mentioned P sub-data sets are input into the above-mentioned second initial model of the above-mentioned k-1th training to obtain the uncertainty quantification parameter of the kth training, wherein the above-mentioned uncertainty quantification parameter of the above-mentioned k-1th training is used to represent the index of the confidence degree of the above-mentioned second initial model of the above-mentioned k-1th training in outputting the above-mentioned second sample information for the above-mentioned first sample information in the P above-mentioned sub-data sets; according to the above-mentioned P sub-data sets, the uncertainty quantification parameter of the above-mentioned k-th training and the above-mentioned third sample information, determine the target loss value output by the loss function of the above-mentioned k-th training; when the above-mentioned k is less than the above-mentioned target number, or when the above-mentioned target loss value does not meet the above-mentioned training end condition, adjust the value of the above-mentioned sample parameter group in the above-mentioned second initial model of the above-mentioned k-1th training to obtain the above-mentioned second initial model of the above-mentioned k-1th training.

[0073] Optionally, in this embodiment, the first parameter is used to represent the difference between the second sample information and the third sample information. The first parameter Z can be calculated by a scoring function. If Z=0, it means that the response result of the large model completely deviates from the input information, and Z=1 indicates a perfect answer.

[0074] Optionally, the sub-dataset in this embodiment includes input information (i.e., the first sample information), output information (i.e., the second sample information), true response information (i.e., information that should actually be output based on the input information), and parameters assigned to the output information based on the true response. For example, a data set containing n samples is constructed. Among them, D raw is generated based on the first sample information and the real response information, x i =(x i，1 , …, x i，ki ),y i =(y i，1 , …, y i，mi ) represent the first sample information and the response result corresponding to the large model (i.e. the second sample information mentioned above), y i，ture Represents x i The real response information, s(y i ,y i，ture ) is based on the real corresponding information y i，ture The response result y i A score assigned.

[0075] Optionally, in this embodiment, based on D raw Construct a supervised learning task, that is, construct in, z i Represents the first parameter above, the vector v i Including based on (x i ,y i ), which comes from white-box features, i.e., the hidden layer activation information of each layer of the large model.

[0076] In actual use, a model is trained using the random forest algorithm It can be based on the feature vector v iDetermine the second uncertainty quantification parameter of the large model, use Dun as the training basis during the training process, and the steps of training the second model are as follows: first initialize various parameters, that is, set the number n of decision trees in the second initial model, as well as the maximum depth and minimum number of leaf node samples of each tree; for each decision tree, randomly extract n samples from the original data set using the bootstrap method; for each split node, randomly select a feature subset, and select the best split point, that is, the feature and threshold that maximize the information gain; recursively build the tree until the maximum depth is reached or the number of samples in the node of the decision subtree exceeds the minimum number of leaf node samples; when the number of training times is less than the target number of times, or the target loss value does not meet the training end condition, adjust the value of the sample parameter group in the second initial model of the k-1th training to obtain the second initial model of the kth training.

[0077] Through the above steps, the second initial model is iteratively trained using the sample data set until the number of training times reaches the target number of times, or the model loss value meets the preset training end condition: in the kth training, the parameter group of the k-1 training results is initialized, P sub-datasets are extracted from the sample data set for training, and based on the input P sub-datasets, the uncertainty quantification parameters of the kth training are obtained. According to the sub-datasets, uncertainty quantification parameters and true response information, the loss value output by the loss function of the kth training is determined, and the model is optimized. Through supervised learning, the second model can learn how to evaluate its uncertainty based on the internal state of the large model, which improves the robustness and accuracy of knowledge boundary identification.

[0078] In an exemplary embodiment, the second initial model of the k-1th training outputs the uncertainty quantification parameters of the kth training through the following steps: constructing P sample decision trees based on the P sub-data sets and the sample parameter group; determining P second parameters output by the P sample decision trees, wherein the second parameters are used to represent the numerical value output by the sample decision trees; and outputting the uncertainty quantification parameters of the kth training based on the P second parameters.

[0079] Optionally, in this embodiment, the uncertainty quantization parameter of the k-th training may be output based on the P second parameters by taking an average value.

[0080] Through the above steps, P decision trees are constructed based on the sub-datasets and parameter groups, P second parameters are predicted by the decision trees, and the uncertainty quantification parameters of the kth training are obtained by combining the P second parameters. This can capture the complexity of the internal state of the large model and help to more accurately predict the uncertainty quantification parameters.

[0081] In an exemplary embodiment, determining the knowledge boundary of the above-mentioned first model according to the above-mentioned target uncertainty parameters includes: obtaining a target uncertainty parameter set, wherein the above-mentioned target uncertainty parameter set includes multiple historical uncertainty parameters, and the above-mentioned historical uncertainty parameters are used to represent the uncertainty quantification parameters determined by the above-mentioned first model when outputting information; analyzing the distribution between the above-mentioned target uncertainty parameters and the multiple historical uncertainty parameters to obtain the target distribution; determining a candidate threshold set based on the above-mentioned target distribution; determining the target threshold from the above-mentioned candidate threshold set to obtain the knowledge boundary of the above-mentioned first model.

[0082] Optionally, the target uncertainty parameter in this embodiment can be calculated by weighted averaging. For example, the gray box uncertainty quantization score (i.e., the first uncertainty quantization parameter) and the white box uncertainty quantization score (i.e., the second uncertainty quantization parameter) are combined by weighted averaging to obtain a comprehensive uncertainty quantization score, UQ combined =ω 1 ·UQ gray-box +ω 2 ·UQ white-box , where ω 1 ,ω 2 is the weight parameter, ω 1 +ω 2 =1, the weight parameter can be determined by experimental optimization or domain knowledge. In general, ω 1 =ω 2 =0.5, UQ gray-box represents the first uncertainty quantification parameter, UQ white-box represents the second uncertainty quantification parameter mentioned above.

[0083] Optionally, the target uncertainty parameter set in this embodiment is a set including a plurality of previously calculated uncertainty quantification parameters, which reflect the uncertainty level when the first model outputs information under different inputs.

[0084] Optionally, the target distribution in this embodiment refers to the distribution characteristics of uncertainty quantification parameters obtained by performing statistical analysis (such as calculating probability distribution, mean, standard deviation, etc.) on multiple historical uncertainty parameters in the target uncertainty parameter set.

[0085] Optionally, the knowledge boundary in this embodiment refers to the limit of the output capacity and knowledge reserve scope of the first model. For example, in the scenario of financial forecasting, if the first model is a large model used to predict stock market trends, then the knowledge boundary may refer to the range of stock types and market conditions that the model can accurately predict. When a user asks about the future trend of a small business stock that the model has never been exposed to, based on the judgment of the target threshold, the model can recognize that this question exceeds its knowledge boundary and give an appropriate prompt or refuse to answer.

[0086] Through the above steps, the target uncertainty parameter set is collected, including multiple uncertainty quantification parameters in the past, the distribution of these historical parameters is analyzed, and the possible candidate threshold set is determined. Then, the best threshold is selected from the candidate threshold set as the knowledge boundary, which can ensure that the determined knowledge boundary not only reflects the model capability but also has the effectiveness of practical application.

[0087] In an exemplary embodiment, determining a target threshold from the candidate threshold set to obtain the knowledge boundary of the first model includes: obtaining a verification data set, wherein the verification data set includes K input information, and K is a natural number greater than or equal to 1; using the verification data set to evaluate the fitness of the candidate thresholds in the candidate threshold set, wherein the following operations are performed for each candidate threshold in the candidate threshold set to determine the fitness of the candidate threshold: when the candidate threshold is determined as the knowledge boundary of the first model, in the i-th round of evaluation, the first model outputs K fourth information based on the verification data set, and the candidate threshold is any threshold in the candidate threshold set; calculating uncertainty quantification parameters of the K fourth information; calculating the probability that the first model correctly outputs the fourth information based on the K uncertainty quantification parameters of the fourth information to determine the fitness of the candidate threshold; determining the target threshold based on the K fitnesses to obtain the knowledge boundary of the first model.

[0088] Optionally, in this embodiment, the F1 score may be used to calculate the probability that the first model correctly outputs the fourth information.

[0089] For example, a large model based on machine learning is used to output information in the field of literature. A validation data set containing K input questions is collected, where K = 100. Each question is about different book information, such as "querying Lu Xun's representative articles or novels" and "querying books related to autumn". These questions have known answers and can be used to test the performance of the model on different questions. The validation data set is used to evaluate the suitability of the candidate thresholds in the candidate threshold set. A set of candidate threshold sets, such as {0.3, 0.4, 0.5, 0.6, 0.7}, is determined from historical data to evaluate the model performance under different uncertainty levels. For each candidate threshold, the following operations are performed: the candidate threshold (such as 0.4) is used as the candidate threshold of the knowledge boundary. In the i-th round of evaluation, the model is allowed to output a prediction (the fourth information) based on each question in the validation data set, and 100 prediction results are obtained. For each prediction, its uncertainty quantification parameter is calculated. Based on the uncertainty quantification parameters of these 100 predictions, the probability that the model correctly outputs the prediction under the current candidate threshold is calculated. When the uncertainty quantification parameter is lower than 0.4, the prediction accuracy of the model is 90%. Repeat the above process and evaluate the suitability of each candidate threshold. For example, for a threshold of 0.5, it is found that the model's prediction accuracy is 88%; while for a threshold of 0.6, the model's prediction accuracy drops to 84%. According to the evaluation of each candidate threshold, the final target threshold is determined based on K fitnesses (i.e., the probability of the model's correct prediction under each candidate threshold). In this example, 0.4 is selected as the target threshold because it ensures a higher prediction accuracy while allowing the model to give an answer within a certain uncertainty range.

[0090] Through the above steps, the validation data set is used to evaluate each threshold in the candidate threshold set to determine its suitability: for each threshold, the probability of the model's correct output is calculated to evaluate whether it can effectively distinguish between the reliable and unreliable outputs of the model, and the threshold with the highest suitability is selected as the knowledge boundary. Through the evaluation of the validation data set, the threshold that best reflects the actual uncertainty of the model can be selected, further enhancing the accuracy of knowledge boundary identification.

[0091] In an exemplary embodiment, after determining the knowledge boundary of the first model according to the target uncertainty parameter, the method further includes: obtaining a fifth uncertainty quantification parameter calculated by the first model when outputting the sixth information based on the input fifth information, wherein the fifth uncertainty quantification parameter is used to represent an indicator of the degree of confidence of the first model in outputting the sixth information based on the input fifth information; comparing the fifth uncertainty quantification parameter with the knowledge boundary to obtain a comparison result; if the comparison result is a first preset result, instructing the first model to output the sixth information, wherein the first preset result is used to indicate that the fifth information is within the knowledge boundary; if the comparison result is a second preset result, instructing the first model to stop outputting the sixth information and sending a prompt message, wherein the second preset result is used to indicate that the fifth information exceeds the knowledge boundary, and the prompt message is used to prompt the user to adjust the fifth information.

[0092] Optionally, the step of determining the fifth uncertainty quantization parameter in this embodiment is similar to the step of determining the above-mentioned target uncertainty parameter, and will not be repeated here.

[0093] Optionally, the knowledge boundary of the large model is identified and expressed as Figure 4 As shown:

[0094] Step S402, calculating the comprehensive uncertainty quantization score UQ combined (i.e. the fifth uncertainty quantification parameter mentioned above);

[0095] Step S404, comparing the comprehensive uncertainty quantization score and the threshold θ (i.e. the above-mentioned knowledge boundary) to obtain a comparison result;

[0096] Step S406: Determine whether to output the generated content and the prompt information given based on the comparison result: combined <θ, it means that the knowledge boundary has not been exceeded, and the large model outputs the original generated content normally, and at the same time outputs a prompt "the knowledge boundary has not been exceeded, and the output result is highly reliable"; UQ combined >θ, it means that the knowledge boundary is exceeded, and the large model refuses to output the original generated content. At the same time, it prompts "The knowledge boundary of the large model is exceeded, and the output result is unreliable. Please refine the question or add background information to help the model understand your needs more accurately."

[0097] Through the above steps, when the first model outputs an answer based on new input information, the uncertainty quantification parameters of the output are calculated, and the uncertainty quantification parameters of the model output are compared with the knowledge boundary to determine whether it exceeds the range of reliable output. Information is output based on the judgment result. By real-time monitoring of the uncertainty of the model output, the model behavior can be adjusted in time to avoid outputting unreliable information under high uncertainty conditions, thereby improving the safety, reliability and user experience of the model in practical applications.

[0098] It should be noted that, through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0099] The above method is explained below with a specific example:

[0100] Figure 5 is a flow chart of a knowledge boundary according to an embodiment of the present application. In this example, the large model is used to predict the financial performance of a given company, such as Figure 5 As shown, the process includes the following steps:

[0101] Step S502, when the large model outputs the second information based on the input first information, the first uncertainty quantification parameter of the first model is calculated. When the user asks the intelligent customer service system a question: "How will Tesla's stock trend in the next week?", the first model in the system (a large language model based on deep learning) generates an answer (the second information) based on this input (i.e., the first information): "Tesla's stock may rise in the next week, and the increase is expected to be between 2% and 5%." At the same time, the first model uses its output probability distribution to calculate a prediction entropy, which indicates the model's confidence in the answer. For example, the calculated first uncertainty quantification parameter is 0.3, indicating that the model has low uncertainty in the answer.

[0102] Step S504, obtaining a second uncertainty quantification parameter output by the second model. The second model (supervised model, such as a random forest model) outputs a second uncertainty quantification parameter based on the internal feature information (first model feature information) when the large model generates an answer, such as hidden layer activation, attention weight, etc. The parameter reflects the uncertainty of the large model when generating the above answer as considered by the second model. Assuming that the second uncertainty quantification parameter output by the second model is 0.4, this indicates that the second model believes that the large model has a medium level of uncertainty when predicting Tesla's stock trend.

[0103] Step S506, using the first uncertainty quantization parameter and the second uncertainty quantization parameter, calculate the target uncertainty quantization parameter of the large model, and calculate the target uncertainty quantization parameter by combining the first uncertainty quantization parameter 0.3 of the first model and the second uncertainty quantization parameter 0.4 of the second model through a weighted average method. Assuming that the weights are 0.6 and 0.4 respectively, the target uncertainty quantization parameter can be calculated as 0.35;

[0104] Step S508, determine the knowledge boundary of the large model according to the target uncertainty parameter, and finally set the threshold of the target uncertainty quantization parameter to 0.3 based on historical data and model performance as the identifier of the knowledge boundary.

[0105] Through the above steps, the knowledge boundaries of large models can be identified more accurately, avoiding giving potentially inaccurate answers in areas where the model is uncertain or not good at, improving the reliability of model output, and enhancing the security and user experience of the system, especially in high-risk application areas such as finance, medical care, and legal consulting. It ensures that the output of the model within the boundary is verified and credible, and the output outside the boundary will receive appropriate warnings or rejections, thereby avoiding potential risks and errors.

[0106] In this embodiment, a device for determining a knowledge boundary is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0107] Figure 6 is a structural block diagram of a device for determining a knowledge boundary according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0108] A first calculation module 602, configured to calculate a first uncertainty quantization parameter of the first model when the first model outputs second information based on the input first information;

[0109] An acquisition module 604 is used to acquire a second uncertainty quantization parameter output by the second model, wherein the second uncertainty quantization parameter is an uncertainty quantization parameter output by the second model based on the input first model feature information, and the first model feature information is model feature information when the first model outputs the second information;

[0110] A second calculation module 606 is used to calculate a target uncertainty quantification parameter of the first model using the first uncertainty quantification parameter and the second uncertainty quantification parameter;

[0111] Determination module 608 is used to determine the knowledge boundary of the above-mentioned first model based on the above-mentioned target uncertainty parameter, wherein the above-mentioned first uncertainty quantification parameter and the above-mentioned second uncertainty parameter are indicators used to represent the degree of confidence of the above-mentioned first model in outputting the above-mentioned second information based on the above-mentioned first information input.

[0112] In an exemplary embodiment, the first calculation module 602 includes: a first calculation unit, used to respectively calculate the third uncertainty quantification parameters of N first word-grams, wherein the N first word-grams are all word-grams generated by the first model based on the first information, the N word-grams are all used to construct the second information, the N is a natural number greater than or equal to 1, and the third uncertainty quantification parameter is an indicator of the degree of confidence in the generation of the first word-grams by the first model; a second calculation unit, used to calculate the first uncertainty quantification parameter based on the third uncertainty quantification parameters of the N first word-grams.

[0113] In an exemplary embodiment, the first calculation module 602 includes: a first determination unit, used to determine M candidate word-grams of a target word-gram, wherein the target word-gram is any one of the N first word-grams, the M candidate word-grams are all word-grams whose semantic similarity with the target word-gram satisfies a first threshold, and the M is a natural number greater than or equal to 1; a third calculation unit, used to calculate the probability of the target word-gram and the M candidate word-grams appearing in the second information to obtain M+1 first probabilities; and a fourth calculation unit, used to calculate the third uncertainty quantization parameter based on the M+1 first probabilities.

[0114] In an exemplary embodiment, the first calculation module 602 includes: a fifth calculation unit, used to perform weighted calculation on the weight coefficients of the N first word elements and the third uncertainty quantization parameters of the N first word elements to obtain the first uncertainty quantization parameter, wherein the weight coefficient represents the semantic contribution of the first word element in the process of generating the second information.

[0115] In an exemplary embodiment, the acquisition module 604 performs the k-th training on the second initial model through the following steps, wherein k is a positive integer: a first initialization unit is used to initialize the sample parameter group of the second initial model for the k-1-th training, wherein the sample parameter group includes the leaf node threshold of the sample decision tree to be constructed and the depth of the sample decision tree to be constructed; a first acquisition unit is used to obtain P sub-data sets used for the k-th training of the second initial model from the sample data set, wherein the P sub-data sets each include first sample information, second sample information, third sample information and a first parameter, wherein the first sample information is input information, the second sample information is information output by the first model based on the first sample information, the third sample information is true response information of the first information, and the first parameter is used to represent the difference between the second sample information and the third sample information. The above-mentioned P is a natural number greater than or equal to 1; the first input unit is used to input the P above-mentioned sub-data sets into the above-mentioned second initial model of the above-mentioned k-1th training to obtain the uncertainty quantification parameter of the k-th training, wherein the above-mentioned uncertainty quantification parameter of the k-th training is used to represent the index of the confidence degree of the above-mentioned second initial model of the above-mentioned k-1th training in outputting the above-mentioned second sample information for the above-mentioned first sample information in the P above-mentioned sub-data sets; the second determination unit is used to determine the target loss value output by the loss function of the above-mentioned k-th training according to the above-mentioned P sub-data sets, the uncertainty quantification parameter of the above-mentioned k-th training and the above-mentioned third sample information; the first adjustment unit is used to adjust the value of the above-mentioned sample parameter group in the above-mentioned second initial model of the above-mentioned k-1th training when the above-mentioned k is less than the above-mentioned target number, or when the above-mentioned target loss value does not meet the above-mentioned training end condition, to obtain the above-mentioned second initial model of the above-mentioned k-th training.

[0116] In an exemplary embodiment, the acquisition module 604 includes: a first construction unit, used to construct P sample decision trees based on the P above-mentioned sub-data sets and the above-mentioned sample parameter group; a third determination unit, used to determine the P second parameters output by the P above-mentioned sample decision trees, wherein the above-mentioned second parameters are used to represent the numerical value output by the sample decision tree; a first output unit, used to output the uncertainty quantization parameter of the above-mentioned k-th training based on the P above-mentioned second parameters.

[0117] In an exemplary embodiment, the determination module 608 includes: a second acquisition unit, used to acquire a target uncertainty parameter set, wherein the target uncertainty parameter set includes multiple historical uncertainty parameters, and the historical uncertainty parameters are used to represent the uncertainty quantification parameters determined by the first model when outputting information; a first analysis unit, used to analyze the distribution between the target uncertainty parameter and the multiple historical uncertainty parameters to obtain the target distribution; a fourth determination unit, used to determine a candidate threshold set based on the target distribution; and a fifth determination unit, used to determine the target threshold from the candidate threshold set to obtain the knowledge boundary of the first model.

[0118] In an exemplary embodiment, the determination module 608 includes: a third acquisition unit, used to acquire a verification data set, wherein the verification data set includes K input information, and the K is a natural number greater than or equal to 1; a first evaluation unit, used to evaluate the suitability of the candidate threshold in the candidate threshold set using the verification data set, wherein the following operations are performed on each candidate threshold in the candidate threshold set to determine the suitability of the candidate threshold: when the candidate threshold is determined as the knowledge boundary of the first model, in the i-th round of evaluation, the first model outputs K fourth information based on the verification data set, and the candidate threshold is any threshold in the candidate threshold set; calculates the uncertainty quantification parameters of the K fourth information; calculates the probability that the first model correctly outputs the fourth information based on the K uncertainty quantification parameters of the fourth information to determine the suitability of the candidate threshold; a sixth determination unit, used to determine the target threshold based on the K above suitabilities to obtain the knowledge boundary of the first model.

[0119] In an exemplary embodiment, the determination module 608 includes: a fourth acquisition unit, used to obtain the fifth uncertainty quantification parameter calculated by the above-mentioned first model when the above-mentioned first model outputs the sixth information based on the input fifth information, wherein the above-mentioned fifth uncertainty quantification parameter is used to represent an indicator of the degree of confidence of the above-mentioned first model in outputting the above-mentioned sixth information based on the above-mentioned fifth information input; a first comparison unit, used to compare the above-mentioned fifth uncertainty quantification parameter with the above-mentioned knowledge boundary to obtain a comparison result; a first indication unit, used to indicate the above-mentioned first model to output the above-mentioned sixth information when the above-mentioned comparison result is a first preset result, wherein the above-mentioned first preset result is used to indicate that the above-mentioned fifth information is within the above-mentioned knowledge boundary; a second indication unit, used to indicate the above-mentioned first model to stop outputting the above-mentioned sixth information and send a prompt message when the above-mentioned comparison result is a second preset result, wherein the above-mentioned second preset result is used to indicate that the above-mentioned fifth information exceeds the above-mentioned knowledge boundary, and the above-mentioned prompt message is used to prompt the user to adjust the above-mentioned fifth information.

[0120] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0121] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0122] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0123] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0124] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0125] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above method embodiments are implemented.

[0126] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0127] The embodiments of the present application also provide a computer program, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the above method embodiments.

[0128] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0129] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0130] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a knowledge boundary, characterized in that: The method comprises: When the first model outputs second information based on the input first information, calculating a first uncertainty quantification parameter of the first model; Acquire a second uncertainty quantification parameter output by the second model, wherein the second uncertainty quantification parameter is an uncertainty quantification parameter output by the second model based on input first model feature information, and the first model feature information is model feature information when the first model outputs the second information; Calculating a target uncertainty quantification parameter of the first model using the first uncertainty quantification parameter and the second uncertainty quantification parameter; determining a knowledge boundary of the first model according to the target uncertainty parameter; The first uncertainty quantification parameter and the second uncertainty parameter are both used to represent an indicator of the confidence level of the first model in outputting the second information based on the first input information.

2. The method according to claim 1, characterized in that When the first model outputs second information based on the input first information, calculating a first uncertainty quantification parameter of the first model includes: Calculating third uncertainty quantification parameters of N first word-grams respectively, wherein the N first word-grams are all word-grams generated by the first model based on the first information, the N word-grams are all used to construct the second information, N is a natural number greater than or equal to 1, and the third uncertainty quantification parameter is an indicator of the confidence level of the first model in generating the first word-gram; The first uncertainty quantization parameter is calculated based on the third uncertainty quantization parameters of the N first word-grams.

3. The method according to claim 2, characterized in that Calculating the third uncertainty quantization parameters of the N first word-units respectively includes: Determine M candidate word-grams of a target word-gram, wherein the target word-gram is any word-gram among the N first word-grams, the M candidate word-grams are all word-grams whose semantic similarity with the target word-gram satisfies a first threshold, and M is a natural number greater than or equal to 1; Calculate the probability that the target word and the M candidate word appear in the second information to obtain M+1 first probabilities; The third uncertainty quantization parameter is calculated based on M+1 of the first probabilities.

4. The method according to claim 2 or 3, characterized in that: Calculating the first uncertainty quantization parameter based on the third uncertainty quantization parameters of the N first word-grams includes: The weight coefficients of the N first word-grams and the third uncertainty quantization parameters of the N first word-grams are weightedly calculated to obtain the first uncertainty quantization parameter, wherein the weight coefficient is used to represent the semantic contribution of the first word-gram in the process of generating the second information.

5. The method according to claim 1, characterized in that The second model is a model obtained by iteratively training the second initial model using the sample data set until the number of times the second initial model is trained reaches a target number of times, or the loss value output by the target loss function of the second initial model meets a preset training end condition, and the iterative training process includes: The second initial model is trained for the kth time by the following steps, where k is a positive integer: Initializing a sample parameter group of the second initial model trained for the k-1th time, wherein the sample parameter group includes a leaf node threshold of a sample decision tree to be constructed and a depth of the sample decision tree to be constructed; Acquire P sub-datasets for performing the k-th training on the second initial model from the sample data set, wherein each of the P sub-datasets includes first sample information, second sample information, third sample information, and a first parameter, the first sample information is input information, the second sample information is information output by the first model based on the first sample information, the third sample information is true response information of the first information, the first parameter is used to represent the difference between the second sample information and the third sample information, and P is a natural number greater than or equal to 1; Inputting the P sub-data sets into the second initial model of the k-1th training to obtain an uncertainty quantification parameter of the k-th training, wherein the uncertainty quantification parameter of the k-th training is used to represent an indicator of the confidence level of the second initial model of the k-1th training in outputting the second sample information for the first sample information in the P sub-data sets; Determine a target loss value output by a loss function of the k-th training according to the P sub-datasets, the uncertainty quantization parameter of the k-th training, and the third sample information; When k is less than the target number of times, or when the target loss value does not satisfy the training end condition, the value of the sample parameter group in the second initial model of the k-1th training is adjusted to obtain the second initial model of the kth training.

6. The method according to claim 5, characterized in that The second initial model of the k-1th training outputs the uncertainty quantization parameter of the kth training through the following steps: Constructing P sample decision trees based on the P sub-datasets and the sample parameter groups; Determine P second parameters output by the P sample decision trees, wherein the second parameters are used to represent the values ​​output by the sample decision trees; Output the uncertainty quantization parameter of the k-th training based on the P second parameters.

7. The method according to claim 1, characterized in that Determining the knowledge boundary of the first model according to the target uncertainty parameter includes: Acquire a target uncertainty parameter set, wherein the target uncertainty parameter set includes a plurality of historical uncertainty parameters, and the historical uncertainty parameters are used to represent uncertainty quantification parameters determined by the first model when outputting information; Analyze the distribution between the target uncertainty parameter and the plurality of historical uncertainty parameters to obtain target distribution; Based on the target distribution, determining a candidate threshold set; A target threshold is determined from the candidate threshold set to obtain a knowledge boundary of the first model.

8. The method according to claim 7, characterized in that Determining a target threshold from the candidate threshold set to obtain a knowledge boundary of the first model includes: Acquire a verification data set, wherein the verification data set includes K input information, and K is a natural number greater than or equal to 1; The fitness of the candidate thresholds in the candidate threshold set is evaluated using the verification data set, wherein the following operations are performed for each of the candidate thresholds in the candidate threshold set to determine the fitness of the candidate threshold: when the candidate threshold is determined as the knowledge boundary of the first model, in the i-th round of evaluation, the first model outputs K fourth information based on the verification data set, and the candidate threshold is any threshold in the candidate threshold set; the uncertainty quantification parameters of the K fourth information are calculated; the probability that the first model correctly outputs the fourth information is calculated based on the K uncertainty quantification parameters of the fourth information to determine the fitness of the candidate threshold; The target threshold is determined based on the K fitness levels to obtain the knowledge boundary of the first model.

9. The method according to claim 1, characterized in that: After determining the knowledge boundary of the first model according to the target uncertainty parameter, the method further includes: Obtaining a fifth uncertainty quantification parameter calculated by the first model when outputting sixth information based on the input fifth information, wherein the fifth uncertainty quantification parameter is used to represent an indicator of confidence level of the first model outputting the sixth information based on the input fifth information; comparing the fifth uncertainty quantification parameter with the knowledge boundary to obtain a comparison result; In a case where the comparison result is a first preset result, instructing the first model to output the sixth information, wherein the first preset result is used to indicate that the fifth information is within the knowledge boundary; When the comparison result is a second preset result, the first model is instructed to stop outputting the sixth information and send a prompt message, wherein the second preset result is used to indicate that the fifth information exceeds the knowledge boundary, and the prompt message is used to prompt the user to adjust the fifth information.

10. A device for determining a knowledge boundary, characterized in that: include: A first calculation module, configured to calculate a first uncertainty quantification parameter of the first model when the first model outputs second information based on input first information; An acquisition module, used for acquiring a second uncertainty quantization parameter output by the second model, wherein the second uncertainty quantization parameter is an uncertainty quantization parameter output by the second model based on input first model feature information, and the first model feature information is model feature information when the first model outputs the second information; A second calculation module, configured to calculate a target uncertainty quantification parameter of the first model by using the first uncertainty quantification parameter and the second uncertainty quantification parameter; A determination module is used to determine the knowledge boundary of the first model according to the target uncertainty parameter, wherein the first uncertainty quantification parameter and the second uncertainty parameter are both indicators used to represent the confidence level of the first model in outputting the second information based on the first input information.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 9 when executed by a processor.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 9 are implemented.

Images