A Method for Exploring the Knowledge Boundaries of a Subject Education Large Model

Abstract

The present invention belongs to the field of natural language processing, specifically relates to the field of large language models, and particularly relates to a method for exploring the knowledge boundary of a large model for subject education. Aiming at the problems of decreased ability and hallucination of large models in special fields, this method first utilizes the principle of large language models to obtain the token generation probability based on the softmax function, then designs an algorithm to calculate the probability volatility to measure the reply confidence, and then sets a dataset and manually annotates it to statistically obtain the volatility threshold within the knowledge boundary. Finally, specific instruction fine-tuning is performed, the annotated data is marked and the original reply outside the boundary is replaced as the fine-tuning dataset to correct the model's understanding of the knowledge boundary. This method effectively extracts the knowledge outside the boundary and fine-tunes the model through the evaluation of the model's reply and the exploration of the knowledge boundary, reduces the hallucination caused by lack of knowledge, and improves the practicality and accuracy of the model in the field of subject education.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, specifically relates to the field of large language models, and particularly relates to a method for exploring the knowledge boundary of a subject education large model. Background Art

[0002] Large Language Models (LLMs) are a type of natural language processing models trained through deep learning techniques, aiming to understand and generate human language. At the same time, they have broad application prospects in the field of education. However, currently, the vast majority of large language models, in order to improve the language understanding ability of the model, complete pre-training under a large amount of public domain data, and then perform alignment training such as instruction fine-tuning and reinforcement learning. Although this method improves the dialogue ability of the model and expands the knowledge coverage of the model, it inevitably leads to the lack of information in specific fields in the model, thereby causing a decline in the ability in special fields and even leading to the emergence of "hallucinations". Therefore, it is very important to improve the ability of the model in special fields and reduce hallucinations. Summary of the Invention

[0003] The present invention aims at the technical problem of the decline in the ability of the large model language in special fields, and proposes a method for exploring the knowledge boundary of a subject education large model.

[0004] To achieve the above object, the technical solution adopted by the present invention is as follows, including the following steps:

[0005] S1. First, extract the knowledge boundary of the subject large model. Utilize the principle that each character token in the text generated by the large language model has a probability value. Based on the softmax function, convert the logits into a probability distribution to obtain the generation probability of each token. The generation probability of each token where z i represents the logits value corresponding to the token t i which represents the unnormalized score of the model for each possible token in the vocabulary when generating the next token, V is the size of the vocabulary, representing the total number of tokens generated by the large language model;

[0006] S2. Then, design an algorithm for calculating the probability volatility of each token to measure the confidence of the model in generating responses. The greater the volatility δ, the less stable the confidence of the model in this response. The calculation method is as follows: where n represents the number of tokens in the generated sequence;

[0007] S3. Then, set the question dataset θ, obtain the responses of the subject education large model, and manually label whether the response content is within the knowledge boundary or outside the knowledge boundary. Then, by counting all δ, obtain a volatility threshold ρ within the knowledge boundary;

[0008] S4. Finally, perform specific instruction fine-tuning. After calculating how to evaluate whether the answer content of the large model is an answer within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, label the data in θ, define the data with δ exceeding ρ as data outside the knowledge boundary, and replace the original large model response content of these data. Use this as the instruction fine-tuning dataset to correct the large model's understanding of the knowledge boundary.

[0009] Preferably, in step S1, the extraction of the knowledge boundary of the subject education large model is to determine the knowledge boundary of the large model by calculating the volatility of the confidence curve of the model-generated response.

[0010] Compared with the prior art, the advantages and positive effects of the present invention are that by judging the accuracy of the answers of the model responses and exploring the boundary of the knowledge required to generate the answers, the knowledge outside the boundary of the model is effectively extracted, and this part of the knowledge is enriched to perform instruction fine-tuning on the large model, effectively reducing the hallucinations caused by the lack of knowledge of the model. Detailed implementation mode

[0011] In order to more clearly understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be further described below in conjunction with embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0012] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the present invention is not limited by the specific embodiments disclosed in the following specification.

[0013] Embodiment. Starting from the training principle of the large language model, each character token in the text generated by the model will have a probability value. Therefore, to a certain extent, tokens with low probability values are often likely to be answers outside the knowledge boundary of the model, that is, the hallucinations of the model. Some researchers obtain an average token probability threshold by counting the probability values of different answers of the large model within and outside the knowledge boundary in the closed-set data, and define the answers exceeding this threshold as answers within the knowledge boundary, and vice versa as answers outside the knowledge boundary. Although this method has a certain basis, it is often difficult to obtain the knowledge boundary value of the large model outside the closed-set data with an absolute threshold.

[0014] In view of the above situation, this method proposes the concept of calculating the volatility of the confidence curve of the large model's generation results to determine the knowledge boundary of the large model. First, extract the knowledge boundary of the subject large model. Using the principle that each character token in the text generated by the large language model has a probability value, based on the softmax function, convert the logits into a probability distribution to obtain the generation probability of each token. The generation probability of each token where z i represents the logits value corresponding to the token t i which represents the unnormalized score of the model for each possible token in the vocabulary when generating the next token. V is the size of the vocabulary, representing the total number of tokens generated by the large language model;

[0015] To measure the change in confidence of the entire generated sequence, then design an algorithm to calculate the volatility of the probability of each token to measure the confidence of the model's generated response. The larger the volatility δ, the less stable the confidence of the model in this response. The calculation method is as follows: where n represents the number of tokens in the generated sequence. The present invention proposes the concept of calculating the volatility of the confidence curve, which can break through the limitation that it is difficult to obtain the outer boundary value of the closed set with traditional absolute thresholds, and provides a new perspective and a more scientific method for determining the knowledge boundary. In addition, the algorithm for calculating the volatility of the probability of each token can effectively measure the change in confidence of the generated sequence and accurately judge the stability of the model's response confidence, providing a key quantitative indicator for knowledge boundary judgment.

[0016] Then set the question dataset θ, obtain the responses of the subject education large model, and manually annotate whether the response content is within or outside the knowledge boundary. Then, by counting all δ, obtain a volatility threshold ρ within the knowledge boundary; by obtaining the model responses and manually annotating, the situations inside and outside the knowledge boundary can be intuitively distinguished, laying a solid foundation for accurately calculating the volatility threshold later.

[0017] Finally, perform specific instruction fine-tuning. After completing the calculation of how to evaluate whether the answer content of the large model is within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, annotate the data in θ, define the data with δ exceeding ρ as data outside the knowledge boundary, and replace the original large model response content of these data. Use this as the instruction fine-tuning dataset to correct the knowledge boundary awareness of the large model. Through the above operations, not only can the knowledge outside the model boundary be accurately extracted, but also the model knowledge can be enriched through instruction fine-tuning, significantly reducing the hallucinations caused by lack of knowledge, improving the reliability and accuracy of the model in a specific field, and providing strong support for the performance improvement of the subject education large model in practical applications, having important practical significance and application value.

[0018] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention in any other form. Any person skilled in the relevant art may use the technical content disclosed above to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for exploring the knowledge boundary of a large model for subject education, characterized in that, Including the following steps: S1. First, extract the knowledge boundary of the subject large model. Using the principle that each character token in the text generated by the large language model has a probability value, based on the softmax function, convert the logits into a probability distribution to obtain the generation probability of each token. The generation probability of each token where z i represents the logits value corresponding to the token t i indicating the unnormalized score of the model for each possible token in the vocabulary when generating the next token. V is the size of the vocabulary, representing the total number of tokens generated by the large language model; S2. Next, design an algorithm to calculate the probability volatility of each token to measure the confidence of the model in generating responses. The greater the volatility δ, the less stable the confidence of the model in this response. The calculation method is as follows: where n represents the number of tokens in the generated sequence; S3. Then set the question dataset a, obtain the responses of the subject education large model, and manually label whether the response content is within the knowledge boundary or outside the knowledge boundary. Then, by statistically analyzing all δ, obtain a volatility threshold ρ within the knowledge boundary; S4. Finally, perform specific instruction fine-tuning. After calculating how to evaluate whether the answer content of the large model is within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, label the data within θ, define the data with δ exceeding ρ as data outside the knowledge boundary, and replace the original large model response content of these data. Use this as the instruction fine-tuning dataset to correct the large model's understanding of the knowledge boundary.

2. The method for exploring the knowledge boundary of a large model for subject education according to claim 1, wherein In the knowledge boundary extraction of the subject education large model in step S1, the knowledge boundary of the large model is determined by calculating the confidence volatility of the model-generated responses.

Images