Subject education large model knowledge boundary exploration method

Abstract

The invention belongs to the field of natural language processing, and particularly relates to the field of large language models, in particular to a subject education large model knowledge boundary exploration method. In order to solve the problems of capability decline and illusion of a large model in a special field, the method comprises the following steps: firstly, acquiring a token generation probability based on a softmax function by utilizing a large language model principle, then designing an algorithm to calculate probability volatility and measure reply confidence, and then setting a data set and carrying out manual annotation and statistics to obtain a volatility threshold in a knowledge boundary. And finally, performing specific instruction fine tuning, labeling data, replacing original replies of data outside the boundary to serve as a fine tuning data set, and correcting model knowledge boundary cognition. According to the method, through evaluation of model reply and knowledge boundary exploration, knowledge outside the boundary is effectively extracted, the model is finely adjusted, illusion caused by knowledge deficiency is reduced, and the practicability and accuracy of the model in the field of subject education are improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, specifically to the field of large language models, and in particular to a method for exploring knowledge boundaries of a subject education large model. Background Art

[0002] Large Language Models (LLMs) are a type of natural language processing model trained through deep learning technology, which aims to understand and generate human language. At the same time, it has broad application prospects in the field of education. However, in order to improve the language understanding ability of the model, most large language models are currently pre-trained with massive public domain data, and then carry out alignment training such as instruction fine-tuning and reinforcement learning. Although this method improves the model's conversation ability and expands the model's knowledge coverage, it also inevitably leads to the model's lack of information in specific fields, which in turn reduces its ability in special fields and leads to the emergence of "hallucinations". Therefore, it is very important to improve the model's ability in special fields and reduce hallucinations. Summary of the invention

[0003] In response to the technical problem that the ability of the above-mentioned big model language decreases in specific fields, the present invention proposes a method for exploring the knowledge boundaries of a subject education big model.

[0004] In order to achieve the above object, the technical solution adopted by the present invention comprises the following steps:

[0005] S1. First, extract the knowledge boundary of the subject model. Use the principle that each character token in the text generated by the large language model has a probability value. Based on the softmax function, convert logits into probability distribution to obtain the generation probability of each token. where z i Represents token i The corresponding logits value represents the unstandardized score of each possible token in the vocabulary when the model generates the next token. V is the vocabulary size, which represents the number of all tokens generated by the large language model.

[0006] S2. Then design an algorithm to calculate the volatility of each token probability to measure the confidence of the model-generated response. The larger the volatility δ, the more unstable the model's confidence in the response. The calculation method is as follows: Where n represents the number of tokens in the generated sequence;

[0007] S3. Then set the problem data set θ, obtain the responses of the subject education model, and manually mark whether the response content belongs to the knowledge boundary or outside the knowledge boundary. Then, by counting all δ, a volatility threshold ρ within the knowledge boundary is obtained;

[0008] S4. Finally, perform specific instruction fine-tuning. After completing the calculation of how to evaluate whether the answer content of the big model is within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, mark the data within θ, define the ρ data that exceeds δ as data outside the knowledge boundary, and replace the original big model response content of these data. This is used as the instruction fine-tuning data set to correct the knowledge boundary cognition of the big model.

[0009] Preferably, the knowledge boundary extraction of the subject education big model in step S1 is to determine the knowledge boundary of the big model by calculating the volatility of the confidence curve of the response generated by the model.

[0010] Compared with the prior art, the advantages and positive effects of the present invention are that, by judging the accuracy of the answers given by the model and exploring the boundaries of the knowledge required to generate the answers, the knowledge outside the model's boundaries is effectively extracted, and this part of knowledge is enriched to fine-tune the instructions of the large model, effectively reducing the illusion caused by the lack of knowledge in the model. DETAILED DESCRIPTION

[0011] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described below in conjunction with embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0012] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments of the following disclosure.

[0013] Embodiment, starting from the training principle of the large language model, each character token in the text generated by the model will have a probability value. Therefore, to a certain extent, tokens with low probability values ​​are often likely to be answers outside the knowledge boundary of the model, that is, the model's hallucination. Some researchers obtain an average token probability threshold by counting the probability values ​​of different answers inside and outside the knowledge boundary of the large model in the closed set data, and define answers exceeding the threshold as answers within the knowledge boundary, and vice versa as answers outside the knowledge boundary. Although this method has a certain basis, it is often difficult to obtain the knowledge boundary value of the large model outside the closed set data with an absolute threshold.

[0014] In view of the above situation, this method proposes the concept of calculating the volatility of the confidence curve of the large model generation results, and uses it to determine the knowledge boundary of the large model. First, the knowledge boundary of the subject large model is extracted, and the principle that each character token in the text generated by the large language model has a probability value is used. Based on the softmax function, logits are converted into probability distribution to obtain the generation probability of each token. The generation probability of each token is where z i Represents token i The corresponding logits value represents the unstandardized score of each possible token in the vocabulary when the model generates the next token. V is the vocabulary size, which represents the number of all tokens generated by the large language model.

[0015] In order to measure the change in confidence of the entire generated sequence, we then designed an algorithm to calculate the probability volatility of each token to measure the confidence of the model-generated response. The larger the volatility δ, the more unstable the confidence of the model in the response. The calculation method is as follows: Where n represents the number of tokens in the generated sequence; the present invention proposes the concept of calculating the volatility of the confidence curve, which can break through the limitation that the traditional absolute threshold is difficult to obtain the outer boundary value of the closed set, and provide a new perspective and a more scientific method for determining the knowledge boundary. In addition, the algorithm for calculating the probability volatility of each token can effectively measure the change in the confidence of the generated sequence, accurately judge the stability of the confidence of the model response, and provide key quantitative indicators for the judgment of the knowledge boundary.

[0016] Then, the problem data set θ is set, the responses of the subject education model are obtained, and the responses are manually marked to determine whether they are within or outside the knowledge boundary. Then, by counting all δ, a volatility threshold ρ within the knowledge boundary is obtained. By obtaining the model responses and manually labeling them, the situations inside and outside the knowledge boundary can be intuitively distinguished, laying a solid foundation for the subsequent accurate calculation of the volatility threshold.

[0017] Finally, specific instruction fine-tuning is performed. After completing the calculation of how to evaluate whether the answer content of the large model is within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, the data within θ is labeled, and the ρ data that exceeds δ is defined as data outside the knowledge boundary. The original large model reply content of these data is replaced, and this is used as the instruction fine-tuning data set to correct the knowledge boundary cognition of the large model. Through the above operations, not only can the knowledge outside the model boundary be accurately extracted, but also the model knowledge can be enriched through instruction fine-tuning, which significantly reduces the illusion caused by lack of knowledge, improves the reliability and accuracy of the model in specific fields, and provides strong support for the performance improvement of the subject education large model in practical applications, which has important practical significance and application value.

[0018] The above description is only a preferred embodiment of the present invention and does not limit the present invention in other forms. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A knowledge boundary exploration method for a subject education large model, characterized in that: The following steps are involved: S1. First, extract the knowledge boundary of the subject model. Use the principle that each character token in the text generated by the large language model has a probability value. Based on the softmax function, convert logits into probability distribution to obtain the generation probability of each token. where z i Represents token i The corresponding logits value represents the unstandardized score of each possible token in the vocabulary when the model generates the next token. V is the vocabulary size, which represents the number of all tokens generated by the large language model. S2. Then design an algorithm to calculate the volatility of each token probability to measure the confidence of the model-generated response. The larger the volatility δ, the more unstable the model's confidence in the response. The calculation method is as follows: Where n represents the number of tokens in the generated sequence; S3. Then set the problem data set θ, obtain the responses of the subject education model, and manually mark whether the response content belongs to the knowledge boundary or outside the knowledge boundary. Then, by counting all δ, a volatility threshold ρ within the knowledge boundary is obtained; S4. Finally, perform specific instruction fine-tuning. After completing the calculation of how to evaluate whether the answer content of the big model is within the knowledge boundary and obtaining the volatility threshold within the knowledge boundary, mark the data within θ, define the ρ data that exceeds δ as data outside the knowledge boundary, and replace the original big model response content of these data. This is used as the instruction fine-tuning data set to correct the knowledge boundary cognition of the big model.

2. According to claim 1, the knowledge boundary exploration method of a subject education large model is characterized in that: The knowledge boundary extraction of the subject education big model in step S1 is to determine the knowledge boundary of the big model by calculating the volatility of the confidence curve of the response generated by the model.

Images