Method and device for enhancing knowledge boundary perception ability of large language model based on dynamic self-adaption
By introducing dynamic adaptive retrieval enhancement methods and prompt word optimization strategies in large language models, the problem of question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-answer question-and-ans
Patent Information
- Application Number
- CN202510292628.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-12
- Publication Date
- 2025-06-27
AI Technical Summary
When large language models are used in different scenarios and industry fields, the Q&A performance is not always reliable, and there are problems such as the illusion of generating incorrect information, the quality of knowledge base content, opaque reasoning process, insufficient context understanding of complex dialogues, and the output of biased or discriminatory content.
The dynamic adaptive retrieval enhancement method is adopted to introduce a confidence measurement mechanism of open domain question-and-answer in large language models. The model's perception of knowledge boundaries is determined based on the confidence of the model, and the model's perception of knowledge boundaries is improved through five prompt word optimization strategies (reward punishment, step-by-step thinking, explanation of answers, self-reflection comparison, and verification chain).
It effectively improves the perception of knowledge boundaries by large language models, reduces the overconfidence of the model, improves the accuracy and interpretability of questions and answers, and at the same time achieves a dynamic balance between resource consumption and answer correctness.
Smart Images
Figure CN120216645A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of collaborative prediction, and specifically to a method and device based on a dynamically adaptive retrieval enhanced large language model. Background Art
[0002] In recent years, the emergence of large language models has greatly promoted the development of the natural language processing field and become an important milestone in artificial intelligence technology. Exploring the mysteries inside large language models and continuously enhancing the ability of large language models to learn and understand knowledge in various aspects have become the key points that artificial intelligence researchers have been continuously striving to explore. With the continuous progress of large language models, computers have made remarkable breakthroughs in understanding and processing complex language information. By deeply mining semantic clues, context information, and implicit knowledge structures in text data, large language models can understand and generate natural language more accurately and efficiently, enabling computers to have stronger reasoning and decision-making capabilities.
[0003] Although large language models have demonstrated excellent semantic understanding capabilities in the natural language processing field and achieved remarkable results in multiple benchmark tests, with their more extensive use in different scenarios and industry fields, researchers and users have gradually found that the question-and-answer performance of large language models is not always reliable, and their internal problems have gradually emerged. These problems not only affect the effectiveness and practical application effects of the models but also pose challenges to further development. Specifically, the main internal problems include the hallucination phenomenon of generating incorrect information, the quality problem of knowledge base content, or over time, the existing information has not been updated in a timely manner and has become outdated, the opacity and lack of interpretability of the reasoning process, insufficient context understanding of complex conversations, and the output of biased or discriminatory content, etc.
[0004] Currently, a common approach is to introduce a retrieval enhanced generation tool, that is, to retrieve the knowledge of a reliable external database and generate the answer corresponding to the question based on the retrieved content. However, if retrieval enhancement is performed on all questions raised by users, it can improve the accuracy of the answers to a certain extent, but this method is not optimal because it consumes a large amount of device resources.
[0005] In view of this, a dynamically adaptive retrieval enhancement method is introduced, that is, before introducing external knowledge, first give full play to the powerful reasoning ability of the large language model; if the reasoning ability of the model cannot effectively solve the problem, then consider introducing a retrieval enhanced generation tool. The core of this method is to require the model to accurately perceive the knowledge boundary perception situation of its own internal knowledge base. How to improve the model's perception of the knowledge boundary, reduce the model's overconfidence, and achieve a dynamic balance in the resources consumed by the model during reasoning and retrieval enhancement is the technical problem that needs to be solved currently. Summary of the Invention
[0006] To overcome the deficiencies in existing solutions, the objective of the present invention is to optimize the question-and-answer strategy in prompt words, break through the limitations existing in traditional question-and-answer methods, and a method for enhancing the knowledge boundary perception ability of a large language model through dynamic adaptive retrieval.
[0007] 1. A method for enhancing the knowledge boundary perception ability of a large language model through dynamic adaptive retrieval, characterized by comprising the following steps:
[0008] Measure the confidence level of the large language model using questions in open-domain question answering. By measurement, it means that when the large language model answers a question, it is required to output the certainty of the answer to the question.
[0009] When conducting question-and-answer, the large language model screens out content related to the question from the internal parameterized knowledge base K according to the prompt word content p and the question q, and finally outputs the answer a to the question and evaluates the certainty c of the question answer in the form of natural language. The formula is as follows.
[0010] (a, c) = f LLM (q, p)
[0011] Where c = 1 indicates that the model is certain about the given answer, and c = 0 indicates that the model is uncertain or has doubts about the given answer.
[0012] 2. The method for measuring the confidence level of the large language model according to 1, characterized in that the results and certainty of the question-and-answer are summarized. If the large language model is certain about the answer to the question, the answer is directly output; if the large language model is uncertain about the answer to the question, a retrieval-enhanced generation tool is tried to be introduced.
[0013] The knowledge base of the large language model is limited and it is impossible to know all the knowledge in the real world. At this time, a retrieval-enhanced generation tool is introduced to retrieve relevant documents D for the question q from the external knowledge base K, and then these retrieved documents are used to expand the knowledge of the large language model. The formula is as follows.
[0014] (a, c) = f LLM (q, p, D)
[0015] However, introducing the retrieval tool incurs additional overhead. If the quality of the retrieved documents cannot be guaranteed, it may mislead the large language model. At this time, the above method is improved, and confidence is used to guide the large language model when to retrieve. The formula is as follows.
[0016]
[0017] 3. Induction of the Q&A results and answer certainty of the large language model according to item 2, characterized in that when there is no improvement in the prompt words, there is still a high degree of overconfidence in the model's answers, that is, when answering a question, the answer given is wrong, but still the result of the answer is considered certain. At this time, in order to enhance the model's ability to perceive the knowledge boundary, it is necessary to optimize the content of the prompt words.
[0018] 4. The strategy for optimizing the content of the prompt words according to item 3, characterized in that it includes the following five prompt word optimization strategies.
[0019] (1) Reward and punishment strategy. The reward and punishment strategy mainly uses the reward and punishment mechanism to constrain the large language model to be cautious about the questions to be answered. When answering questions, the large language model is required to output the certainty of the Q&A results. If you don't know the answer to the question you are about to answer, don't fabricate an answer and just say you don't know. After answering the question, if the answer is correct on the premise of being certain about the answer given, you will receive a reward; but if the answer is wrong, then you will receive a punishment.
[0020] (2) Step-by-step thinking strategy. The step-by-step thinking strategy mainly requires the large language model to analyze the content of the question in detail when answering the question, formulate the problem-solving steps and results for each step according to the question content and requirements, and deduce the correct answer based on the problem-solving steps and corresponding results. This strategy is similar to the answer template for solving math application problems or proof questions.
[0021] (3) Explain the answer strategy. The explain the answer strategy requires the large language model to give detailed explanatory content of the corresponding answer when answering the question.
[0022] (4) Self-reflection and comparison strategy. The self-reflection and comparison strategy requires the model to actively explore different perspectives on the problem solution when answering the question, and then collect the problem-solving answers from different perspectives. After answering, the model will actively compare the differences between the answers based on the question content and scenario, and actively review and eliminate these differences to deduce the correct answer.
[0023] (5) Verification chain strategy. The verification chain strategy requires the large language model to generate multiple results when answering the question. Based on the question raised and multiple answers, the model is required to generate a series of verification questions. Use the verification questions to detect whether the answer matches the content of the question asked by the user.
[0024] 5. Based on the five prompting optimization strategies described in 4, summarize and conclude the results of the question and answer. The feature is that when the model is still uncertain about the results of the question and answer, it tries to introduce an external knowledge base and derives the final answer of the document based on the content obtained from retrieving the external knowledge base; when the model is certain about the results of the question and answer, it directly outputs the question and answer results.
[0025] 6. Summarize, statistically analyze, and evaluate the question and answer results obtained in 5 and the answers without any improvement to the prompting words. The feature is that the evaluation content includes the performance of the model's question and answer, the confidence level and overconfidence, conservativeness, and synchronization of the model, so as to derive the knowledge boundary perception ability of the model.
[0026] The evaluation metrics are as follows
[0027] Use A = A cc +A cu +A ic +A iu to represent the total number of test samples.
[0028] Use accuracy to measure the performance of the model's question and answer. If the correct answer is included in the question and answer result, then it is considered that the result answered by the large language model is correct. The formula is as follows.
[0029]
[0030] Use certainty rate to measure the confidence level of the model. The certainty rate refers to the proportion of the large language model's response indicating a certain answer to the question and answer. The higher the certainty rate, the higher the confidence level of the large language model.
[0031]
[0032] To understand the knowledge boundary perception ability of the large language model, this paper evaluates the overconfidence, conservativeness, and synchronization of the large language model respectively. Overconfidence refers to the proportion of the large language model's answer being wrong but affirming the answer given by itself when answering; while conservativeness measures the proportion of the large language model's answer being correct but being uncertain about the answer given by itself. The formula is as follows.
[0033]
[0034] 7. According to the method for enhancing the knowledge boundary perception ability of the large language model by dynamic adaptive retrieval enhancement described in 1, the feature is that the adaptive retrieval enhancement method is an improvement on the existing static retrieval enhancement method, and the latter refers to retrieving the external knowledge base for each question asked by the user.
[0035] 8. The method for enhancing the knowledge boundary perception ability of a large language model through dynamic adaptive retrieval enhancement according to claim 1, wherein the adaptive retrieval enhancement method can maintain a certain dynamic balance between the correctness of the question answer and the performance consumed by querying the external knowledge base during retrieval enhancement.
[0036] 9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for enhancing the knowledge boundary perception ability of a large language model through dynamic adaptive retrieval enhancement as described in any one of claims 1 to 8 are implemented.
[0037] 10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the method for enhancing the knowledge boundary perception ability of a large language model through dynamic adaptive retrieval enhancement as described in any one of claims 1 to 8 are implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings required for use in the description of the embodiments or the prior art will be briefly introduced below.
[0039] Figure 1 It is a flowchart of a method for enhancing the knowledge boundary perception ability of a large language model based on dynamic adaptive retrieval enhancement;
[0040] Figure 2 It is a schematic diagram of an adaptive retrieval enhancement architecture;
[0041] Figure 3 It is a working principle diagram of the self-reflection strategy of the present invention;
[0042] Figure 4 It is a sample question in the prompting word enhancement strategy provided in step 4 of the specific implementation manner of the present invention;
[0043] Figure 5 It is a structural block diagram of the electronic device in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0044] In order to more clearly understand the purpose, technical solution, and advantages of the present application, the present application will be described and explained below with reference to the drawings and embodiments.
[0045] Unless otherwise defined, technical terms or scientific terms involved in this application shall have the ordinary meanings understood by those with ordinary skills in the technical field to which this application belongs. In this application, words such as "a", "an", "one kind", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "containing", "having" and any variants thereof involved in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "linked", "coupled", etc. involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly connected. The term "a plurality of" involved in this application means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific order for the objects.
[0046] As Figure 1 , Figure 2 , Figure 3 shown, a method for enhancing the knowledge boundary perception ability of a large language model based on dynamic adaptive retrieval enhancement includes the following steps:
[0047] 1. A method for enhancing the knowledge boundary perception ability of a large language model based on dynamic adaptive retrieval enhancement, characterized by including the following steps:
[0048] Measure the confidence level of the large language model using the questions in open-domain question answering. By measurement, it means that when the large language model answers questions, it is required to output the certainty of the answer to the question;
[0049] When conducting question answering, the large language model screens out the content related to the question from the internal parameterized knowledge base K according to the prompt content p and the question q, and finally outputs the answer a to the question and evaluates the certainty c of the question answer in the form of natural language. The formula is as follows.
[0050] (a, c) = f ILM (q, p)
[0051] Among them, c = 1 indicates that the model affirms the given answer, and c = 0 indicates that the model is uncertain or has doubts about the given answer.
[0052] 2. According to the method for measuring the confidence level of the large language model described in 1, it is characterized in that the results and certainty of the question and answer are summarized. If the large language model is certain about the answer to the question, the answer is directly output; if the large language model is uncertain about the answer to the question, a retrieval-augmented generation tool is tried to be introduced.
[0053] The knowledge base of the large language model is limited and it is impossible to know all the knowledge in the real world. At this time, a retrieval-augmented generation tool is introduced to retrieve relevant documents D for the question q from the external knowledge base K, and then these retrieved documents are used to expand the knowledge of the large language model. The formula is as follows.
[0054] (o, c) = f ILM (q, p, D)
[0055] However, introducing the retrieval tool incurs additional overhead. If the quality of the retrieved documents cannot be guaranteed, it may mislead the large language model. At this time, the above method is improved, and confidence is used to guide when the large language model retrieves. The formula is as follows.
[0056]
[0057] 3. According to the summary of the question and answer results and answer certainty of the large language model described in 2, it is characterized in that when there is no improvement in the prompt words, the model still has a relatively high overconfidence when answering, that is, when answering the question, the answer is wrong, but still certain about the result of the answer. At this time, in order to enhance the model's ability to perceive the knowledge boundary, the content of the prompt words needs to be optimized.
[0058] 4. According to the strategy of optimizing the content of the prompt words described in 3, it is characterized in that
[0059] The following five prompt word optimization strategies are included.
[0060] (1) Reward and punishment strategy. The reward and punishment strategy mainly uses the reward and punishment mechanism to constrain the large language model to be cautious about the questions to be answered. When answering questions, the large language model is required to output the certainty of the question and answer results. If you don't know the answer to the question you are about to answer, don't fabricate an answer and just say you don't know. After answering the question, on the premise of being certain about the answer, if the answer is correct, you will get a reward; but if the answer is wrong, then you will receive a punishment.
[0061] (2) Step-by-step thinking strategy. The step-by-step thinking strategy mainly requires the large language model to analyze the content of the question in detail when answering the question, formulate the solution steps and results of each step according to the content and requirements of the question, and derive the correct answer based on the solution steps and corresponding results. This strategy is similar to the answer template for solving math application problems or proof questions.
[0062] (3) Explain the answer strategy. The explain the answer strategy requires the large language model to give the detailed explanation content of the corresponding answer when answering the question.
[0063] (4) Self-reflection and comparison strategy. The self-reflection and comparison strategy requires the model to actively explore different perspectives for answering the question when answering the question, and then collect the answers to the question from different perspectives. After answering, the model will actively compare the differences between the answers based on the content and scenario of the question, and actively review and eliminate these differences to derive the correct answer.
[0064] (5) Verification chain strategy. The verification chain strategy requires the large language model to generate multiple results when answering the question. Based on the question and multiple answers proposed, the model is required to generate a series of verification questions. The verification questions are used to detect whether the answers match the content of the question asked by the user.
[0065] 5. According to the five prompting word optimization strategies described in 4, summarize the results of the question and answer. The feature is that when the model is still uncertain about the results of the question and answer, it tries to introduce an external knowledge base and derives the final answer of the document based on the content retrieved from the external knowledge base; when it is certain about the results of the question and answer, it directly outputs the results of the question and answer.
[0066] 6. Summarize, statistically analyze and evaluate the question and answer results obtained in 5 and the answers without any improvement to the prompting words. The feature is that the evaluation content includes the performance of the model's question and answer, the confidence and overconfidence of the model, and the conservatism and synchronization, so as to derive the knowledge boundary perception ability of the model.
[0067] The evaluation metrics are as follows
[0068] Use to represent the total number of test samples.
[0069] Use accuracy to measure the performance of the model's question and answer. If the answer to the question and answer contains the correct answer, then it is considered that the result of the large language model's answer is correct. The formula is as follows.
[0070]
[0071] The confidence of the model is measured by the certainty rate, which refers to the proportion of whether the large language model's answer to a question represents a definite response. The higher the certainty rate, the higher the confidence of the large language model.
[0072]
[0073] To understand the large language model's ability to perceive the knowledge boundary, this paper evaluates the overconfidence, conservativeness, and synchrony of the large language model respectively. Overconfidence means that when the large language model answers a question, although the answer given is wrong, the proportion of affirming the answer given by itself; while conservativeness measures the proportion that the answer given by the large language model is correct, but it is uncertain about the answer given by itself. The formulas are as follows.
[0074]
[0075] 7. The method for enhancing the knowledge boundary perception ability of the large language model by the dynamic adaptive retrieval enhancement according to 1, wherein the adaptive retrieval enhancement method is an improvement on the existing static retrieval enhancement method, and the latter refers to retrieving from the external knowledge base for each question asked by the user.
[0076] 8. The method for enhancing the knowledge boundary perception ability of the large language model by the dynamic adaptive retrieval enhancement according to 1, wherein the adaptive retrieval enhancement method can maintain a certain dynamic balance between the correctness of the question answer and the performance consumed by querying the external knowledge base during the retrieval enhancement.
[0077] 9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method for enhancing the knowledge boundary perception ability of the large language model by the dynamic adaptive retrieval enhancement as described in any one of claims 1 to 8.
[0078] 10. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the steps of the method for enhancing the knowledge boundary perception ability of the large language model by the dynamic adaptive retrieval enhancement as described in any one of claims 1 to 8.
Claims
1. A method for enhancing the knowledge boundary perception ability of a large language model based on dynamic adaptive retrieval, characterized in that: The following steps are involved: The confidence of the large language model is measured using questions from open domain question answering. The so-called measurement is to require the large language model to output the certainty of the answer to the question when answering the question; When answering questions, the large language model filters out content related to the question from the internal parameterized knowledge base K based on the prompt word content p and the question q, and finally outputs the answer a and the certainty c of the answer to the question in the form of natural language. The formula is shown below. (a,c)=f LLM (q,p) Where c=1 means that the model is positive about the given answer, and c=0 means that the model is uncertain or has doubts about the given answer.
2. The method for measuring the confidence level of a large language model according to claim 1, characterized in that: The results and certainty of the question and answer are summarized. If the large language model is certain about the answer, the answer is directly output; if the large language model is uncertain about the answer, try to introduce a retrieval enhancement generation tool. The knowledge base of a large language model is limited, and it is impossible to know all the knowledge in the real world. At this time, the retrieval enhancement generation tool is introduced to retrieve relevant documents D for question q from the external knowledge base K, and then use these retrieved documents to expand the knowledge of the large language model. The formula is as follows. (a,c)=f LLM (q,p,D) However, the introduction of retrieval tools generates additional overhead, and if the quality of the retrieved documents cannot be guaranteed, it may mislead the large language model. At this time, the above method is improved, and confidence is used to guide the large language model when to retrieve. The formula is shown below.
3. The method of summarizing the question-answering results and answer certainty of a large language model according to claim 2, wherein: When there is no improvement in the prompt words, the model still has a high degree of overconfidence when answering, that is, when answering the question, the answer is wrong, but the answer is still certain. In order to enhance the model's ability to perceive knowledge boundaries, the prompt word content needs to be optimized.
4. The strategy for optimizing the prompt word content according to claim 3 is characterized in that: It includes the following five prompt word optimization strategies. (1) Reward and Penalty Strategy. The reward and penalty strategy mainly uses the reward and penalty mechanism to constrain the large language model to be cautious in dealing with the questions it is about to answer. When answering questions, the large language model is required to output the certainty of the question and answer results. If you don’t know the answer to the question you are about to answer, please do not make up an answer and just say you don’t know. After answering the question, if you are sure of the answer, if the answer is correct, you will be rewarded; but if the answer is wrong, you will be punished. (2) Step-by-step thinking strategy. The step-by-step thinking strategy mainly requires the large language model to analyze the content of the question in detail when answering the question, formulate each step of the problem-solving steps and results according to the question content and requirements, and deduce the correct answer based on the problem-solving steps and corresponding results. This strategy is similar to the answer template for solving math word problems or proof problems. (3) Explanation of answer strategy: The explanation of answer strategy requires the large language model to provide a detailed explanation of the corresponding answer when answering the question. (4) Self-reflection and comparison strategy. The self-reflection and comparison strategy requires the model to actively explore different perspectives on the problem when answering the question, and then collect answers from different perspectives. After the answer is completed, the model will actively compare the differences between the answers based on the content and scenario of the question, and actively review and eliminate these differences to derive the correct answer. (5) Verification chain strategy. The verification chain strategy requires the large language model to generate multiple results when answering questions. Based on the questions asked and multiple answers, the model is required to generate a series of verification questions. Verification questions are used to detect whether the answer matches the content of the question asked by the user.
5. According to the five prompt word optimization strategies described in claim 4, the results of the question and answer are summarized and concluded, characterized in that: When the model is still uncertain about the result of the question and answer, it tries to introduce an external knowledge base and deduce the final answer to the document based on the content obtained by retrieving the external knowledge base; when the model is certain about the result of the question and answer, it directly outputs the question and answer result.
6. The question-answering results obtained according to claim 5 and the answers without any improvement to the prompt words are summarized, counted and evaluated, characterized in that: The evaluation includes the performance of the model's question answering, the model's confidence and overconfidence, conservatism and synchronization, and thus the model's ability to perceive knowledge boundaries. The evaluation indicators are as follows Use A=A cc +A cu +A ic +A iu To represent the total number of test samples. The accuracy is used to measure the performance of the model question and answer. If the question and answer result contains the correct answer, then the result answered by the large language model is considered correct. The formula is as follows. The confidence of the model is measured by the determination rate, which refers to the proportion of whether the large language model represents a deterministic response to the question and answer results. The higher the determination rate, the higher the confidence of the large language model. In order to understand the perception of knowledge boundaries of the large language model, this paper evaluates the overconfidence, conservatism and synchronization of the large language model. Overconfidence refers to the proportion of the large language model that is positive about its own answer even though the answer it gives is wrong; while conservatism measures the proportion of the large language model that is correct but uncertain about its own answer. The formula is as follows.
7. The method for dynamically adaptively retrieving and enhancing a large language model to improve the model's knowledge boundary perception capability according to claim 1, characterized in that: The adaptive retrieval enhancement method is an improvement on the existing static retrieval enhancement method, which refers to performing an external knowledge base search for each question asked by the user.
8. The method for dynamically adaptively retrieving and enhancing a large language model to improve the model's knowledge boundary perception capability according to claim 1, characterized in that: The adaptive retrieval enhancement method can maintain a certain dynamic balance between the correctness of the answer to the question and the performance consumed in querying the external knowledge base when performing retrieval enhancement.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of dynamically adaptively retrieving and enhancing the large language model to improve the knowledge boundary perception ability of the model are implemented as described in any one of claims 1 to 8.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of dynamically adaptively retrieving and enhancing the knowledge boundary perception capability of a large language model according to any one of claims 1 to 8 are implemented.