Chain-of-thought-pool-based method and system for implementing stable outputs of large models

Abstract

The present invention belongs to the technical field of generative artificial intelligence. Disclosed are a chain-of-thought-pool-based method and system for implementing stable outputs of large models. In a chain-of-thought-pool-based consistency method provided in the present invention, language models of different capability levels are connected by means of a chain-of-thought pool on the basis of the characteristic of the capabilities of the language models being hierarchical and the caching characteristics of the fixed-capacity chain-of-thought pool. Therefore, the performance of small-parameter language models is fully exerted, thereby reducing the time needed for responding to some questions, and the costs for large-parameter language models to generate a chain of thought are also significantly reduced while ensuring accuracy.

Description

A Stable Output Method and System for Large Models Based on Think Pools Technical Field

[0001] This invention belongs to the field of generative artificial intelligence technology, specifically relating to a method for stable output of large models based on a thought chain pool. Background Technology

[0002] With the rapid development of artificial intelligence technology, question-answering systems based on large language models have attracted widespread attention due to their powerful natural language understanding and generation capabilities. However, these systems face a key challenge in practical applications: the inconsistency of language model output. This inconsistency can lead to instability in the output results, thereby affecting the system's security, robustness, and credibility, and ultimately negatively impacting the user experience.

[0003] Existing methods for improving the consistency of large models include self-consistent boosting (SC), universal consistency boosting (USC), and fine-grained consistency boosting (FSC). Self-consistent boosting (SC), building upon CoT, samples the model results multiple times to generate multiple candidate answers, then uses majority voting to generate the final answer. Universal consistency boosting (USC) abandons majority voting in the final result acquisition process, instead using prompts to allow the LLM to automatically select the best answer from multiple candidate answers. Fine-grained consistency boosting (FSC) abandons directly selecting the final answer, allowing the LLM to actively select consistent segments from multiple candidate answers and then automatically assemble them into the final answer. All the methods mentioned above are based on sampling a set of different thought chains. Although these consistency-improving methods are effective, their computational cost is high because they require generating a large number of thought chains to increase the chances of the correct answer appearing as the most common answer. Summary of the Invention

[0004] The purpose of this invention is to solve the above-mentioned technical problems and to provide a method and system for stable output of large models based on a thought chain pool.

[0005] The specific technical solution adopted in this invention is as follows:

[0006] In a first aspect, the present invention provides a method for stable output of a large model based on a thought chain pool, comprising:

[0007] S1. Based on the user's input question, select a large model from the large model queue as the large model to be executed;

[0008] S2. According to the preset capacity of the thought chain pool, input the question and the preset thought chain sampling prompt words into the current execution model, and drive the execution model to generate a series of thought chains for the question and fill the thought chain pool.

[0009] S3. After the thought chain pool is filled, input each thought chain in the thought chain pool, the preset confidence assessment prompt words and the question into the current execution big model to obtain the confidence score of each thought chain, and then convert it to obtain the uncertainty score of the thought chain.

[0010] S4. After obtaining the uncertainty score of each thought chain in the thought chain pool, the thought chains in the thought chain pool are filtered by a threshold method. Thought chains with uncertainty scores lower than a preset filtering threshold are considered valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thought chains in the thought chain pool with uncertainty scores exceeding the filtering threshold are removed, and the next large model is selected from the current large model queue as the execution large model. The loop returns to S2 and continues. Otherwise, the final answer is generated based on the thought chain pool and using the consistency improvement method, and returned to the user.

[0011] As a preferred embodiment of the first aspect above, in each round of executing S2, the number of thought chains generated by the current execution model for the problem is the difference between the thought chain capacity of the thought chain pool and the number of currently existing valid thought chains in the thought chain pool.

[0012] As a preferred embodiment of the first aspect above, the uncertainty score of the thought chain is generated as follows: first, the confidence score p∈[0,1] of the large model for each thought chain is obtained, and then the uncertainty score p'=1-p of the thought chain is calculated.

[0013] As a preferred embodiment of the first aspect above, the range of the filtering threshold is [0.5, 0.9], and the range of the iteration threshold is [0.05, 0.5].

[0014] As a preferred embodiment of the first aspect above, the comprehensive uncertainty score is a first uncertainty score, or a second uncertainty score, or a weighted score of the first uncertainty score and the second uncertainty score;

[0015] The first uncertainty score is the average score of the uncertainty scores of all valid thought chains in the thought chain pool;

[0016] The second uncertainty score is calculated as follows: First, cluster all valid thought chains in the thought chain pool according to the answer corresponding to each valid thought chain, and count the ratio b of the number of valid thought chains in the largest cluster to the total number of valid thought chains. Use 1-b as the second uncertainty score.

[0017] As a preferred embodiment of the first aspect above, the consistency enhancement method is a fine-grained self-consistency (FSC), a universal self-consistency (USC), or a self-consistency enhancement method (SC).

[0018] Secondly, the present invention provides a large model stable output system based on a thought chain pool, which includes a large model scheduler, a thought chain sampler, an uncertainty analyzer, an iterative discriminator, an answer generator, and a filter;

[0019] The large model scheduler is used to receive the user input question or the iteration instruction sent by the filter, and after receiving the question or iteration instruction, it extracts the large model at the head of the large model queue as the execution large model, and at the same time sends the first execution instruction to the mind chain sampler.

[0020] The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled.

[0021] The uncertainty analyzer is used to receive the second execution instruction sent by the thought chain sampler, and input each thought chain in the thought chain pool, the preset confidence evaluation prompt words and the question into the current execution big model to obtain the confidence score of each thought chain, and then convert it to obtain the uncertainty score of the thought chain. After generating the uncertainty scores of all thought chains, it sends the third execution instruction to the iterative discriminator.

[0022] The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and to filter the thought chains in the thought chain pool using a threshold method. The thought chains with uncertainty scores lower than a preset filtering threshold are considered as valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. Then, it is determined whether the comprehensive uncertainty score exceeds the iterative threshold. If it does not exceed the threshold, a fourth execution instruction is sent to the answer generator; otherwise, a fifth execution instruction is sent to the filter.

[0023] The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, generate the final answer using the consistency improvement method and return it to the user;

[0024] The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thought chains in the thought chain pool whose uncertainty scores exceed the filtering threshold, it retains only the valid thought chains and sends the iteration instruction to the large model scheduler.

[0025] Thirdly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the stable output method of a large model based on a thought chain pool as described in any of the first aspects above.

[0026] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, enables a stable output method for large models based on a thought pool as described in any of the first aspects above.

[0027] Fifthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0028] The memory is used to store computer programs;

[0029] The processor is configured to, when executing the computer program, implement the stable output method for large models based on the thought pool as described in any of the first aspects above.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] The consistency method based on the thought chain pool proposed in this invention leverages the hierarchical characteristics of language model capabilities and the caching characteristics of a fixed-capacity thought chain pool to connect language models with different capability levels. This not only fully utilizes the performance of language models with a small number of parameters and reduces the response time of some problems, but also significantly reduces the cost of generating thought chains for language models with a large number of parameters while ensuring accuracy. Attached Figure Description

[0032] Figure 1 is a flowchart of the steps of the stable output method for large models based on the thought chain pool;

[0033] Figure 2 is a schematic diagram of the internal composition of a large model stable output system based on the thought chain pool;

[0034] Figure 3 is a flowchart of the data processing of each module in the large model stable output system based on the thinking chain pool;

[0035] Figure 4 is a schematic diagram of the structure of a computer electronic device;

[0036] Figure 5 shows the accuracy improvement rate results in the embodiments of the present invention;

[0037] Figure 6 shows the reduction rate of the number of thought chains generated in the embodiments of the present invention. Detailed Implementation

[0038] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0039] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0040] As shown in Figure 1, in a preferred embodiment of the present invention, a method for stable output of a large model based on a thought chain pool is provided, which includes sequentially executed steps S1 to S4:

[0041] S1. For the question input by the user, select a large model (LLM) from the large model queue as the large model to be executed.

[0042] It should be noted that the "large model queue" in this invention can be pre-designed according to actual needs. It can be selected by evaluating the capability differences of different large models and adding those that meet the requirements to the LLM queue. For example, in an embodiment of this invention, the large model queue is a combination of Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. During each iteration, the large model at the head of the large model queue can be removed as the execution large model according to the conventional dequeue operation. That is, when the LLM scheduler receives a question input by the user, the LLM scheduler selects an LLM as the execution LLM dequeue for the subsequent steps. When it receives an iteration instruction sent by the filter, the LLM scheduler selects the next LLM as the execution LLM dequeue for the subsequent steps.

[0043] S2. According to the preset capacity of the thought chain pool, input the question and the preset thought chain sampling prompts into the current execution model, and drive the execution model to generate a series of thought chains for the question and fill the thought chain pool.

[0044] It should be noted that the capacity of the thought chain pool refers to the number of thought chains that the pool can hold. The specific value can be designed according to actual needs, preferably between 10 and 30. Each time step S2 is executed, the execution model needs to generate a corresponding number of thought chains to fill the pool based on the remaining space of the fixed-capacity thought chain pool. Except for the first filling, in subsequent filling rounds, since the pool may already contain thought chains retained from the previous round, the number of thought chains to be generated is calculated as follows: if the fixed capacity of the thought chain pool is preset to n, and there are already m thought chains in the pool, then this execution LLM only needs to generate nm thought chains. The execution LLM generates thought chains by inputting the question and preset thought chain sampling prompts into the current execution model. Each input to the execution model returns the thought chains corresponding to the question. A total of nm inputs yield nm thought chains to fill the thought chain pool. The aforementioned thought chain sampling prompts can be designed through prompt engineering according to actual needs, ensuring that the execution model can save the corresponding thought chains in addition to generating the answer. Each thought chain (r i ,a i ) is the reasoning part r i and answer part a i Composition, where i represents the i-th thought chain.

[0045] S3. After the thought chain pool is filled, input each thought chain in the thought chain pool, the preset confidence assessment prompt words and the question into the current execution big model to obtain the confidence score p∈[0,1] of each thought chain, and then calculate the uncertainty score p'=1-p of the thought chain.

[0046] It should be noted that the confidence assessment of the thought chain by the large model can be directly driven by prompt words. The thought chain to be assessed and the preset confidence assessment prompt words are simply input into the current executing large model. Similarly, the confidence assessment prompt words can be designed through prompt word engineering according to actual needs. They should be able to drive the executing large model to generate a confidence score for the thought chain, informing the large model to score the confidence of the thought chain based on the currently input question and thought chain, and output a quantified score value. Of course, in practical applications, the specific measurement method of the confidence score can be specified as needed. For example, in the embodiment of this invention, the measurement method for the confidence score of the thought chain is the Verb.2S top-1 measurement method. This means that the prompt words specify that the large model first needs to obtain the language model's confidence score p for the thought chain through a two-stage Verb.2S top-1 measurement method, with the confidence score ranging from [0,1]. Then, 1-p is used to obtain the language model's uncertainty score p' for the thought chain.

[0047] Of course, the generation of confidence scores for the thought chain is not unique when performing uncertainty analysis. Methods for generating confidence scores include P(True) (Kadavath et al., 2022), Verbalized 2S (Tian et al., 2023), and Response Probability (Wang et al., 2022), etc. The implementation methods for each can be found in the following literature:

[0048] Kadavath S,Conerly T,Askell A,et al.Language models(mostly)know what they know[J].arXiv preprint arXiv:2207.05221,2022.

[0049] Tian K, Mitchell E, Zhou A, et al. Just Ask for Calibration:Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback[C] / / Proceedings of the 2023Conference on Empirical Methods in Natural Language Processing.2023:5433-5442.

[0050] Wang X, Wei J, Schuurmans D, et al.Self-consistency improves chain of thought reasoning in language models[J].arXiv preprint arXiv:2203.11171,2022.

[0051] S4. After obtaining the uncertainty score of each thought chain in the thought chain pool, the thought chains in the thought chain pool are filtered by a threshold method. Thought chains with uncertainty scores lower than a preset filtering threshold are considered valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thought chains in the thought chain pool with uncertainty scores exceeding the filtering threshold are removed, and the next large model is selected from the current large model queue as the execution large model. The loop returns to S2 and continues. Otherwise, the final answer is generated based on the thought chain pool and using the consistency improvement method, and returned to the user.

[0052] It should be noted that the aforementioned filtering threshold is a hyperparameter that can be optimized based on actual performance. After each thought chain in the thought chain pool obtains an uncertainty score p', it can be determined whether it exceeds the filtering threshold. If it exceeds the threshold, it indicates that the uncertainty of the thought chain is too high and cannot be used as a valid thought chain. If it does not exceed the threshold, it indicates that the thought chain is relatively certain and can be retained in the thought chain pool as a valid thought chain for final consistency improvement. Similarly, the aforementioned iteration threshold is also a hyperparameter that can be optimized based on actual performance. In the embodiments of the present invention, the preferred value range of the filtering threshold is [0.5, 0.9], and the preferred value range of the iteration threshold is [0.05, 0.5].

[0053] Based on the uncertainty score p' of each thought chain in the thought chain pool and the filtering threshold, extract all valid thought chains whose uncertainty score p' does not exceed the filtering threshold (fr i ,fa i Once this is achieved, the comprehensive uncertainty score can be calculated. The comprehensive uncertainty score can be calculated in several different ways; three different calculation methods are given below.

[0054] The comprehensive uncertainty score calculated in the first method is denoted as the first uncertainty score, Score1, and is calculated as follows:

[0055] Calculate the average uncertainty score of all valid thought chains in the thought chain pool whose uncertainty scores p' are lower than the filtering threshold, and use this average score as the overall uncertainty score of the thought chain pool. Where m is the total number of effective thought chains, Si Let be the uncertainty score of the i-th thought chain.

[0056] The comprehensive uncertainty score calculated in the second form is denoted as the second uncertainty score, Score2, and is calculated as follows:

[0057] First, cluster all valid thought chains in the thought chain pool based on the answer corresponding to each valid thought chain. Calculate the ratio *b* of the number of valid thought chains in the largest cluster to the total number of valid thought chains, and use 1-*b* as the second uncertainty score. Specifically, the clustering method described above needs to be selected based on the answer. For common mathematical calculation or classification tasks, the answer portion of the valid thought chain is *fa*. i It can often be directly used as a category label, based on the answer part of the effective thinking chain. i The thought chains are categorized, and the number of valid thought chains in each category is counted. Based on the categorization results, the category with the most thought chains is obtained. Where m is the total number of effective thought chains, Ι(fa i =c) is an indicator function, when the answer part fa i The value is 1 for category c, and 0 otherwise. Therefore, the overall uncertainty score of the thought chain pool is obtained as follows:

[0058] Of course, if the answer part of the effective thinking chain is fa i If a cluster cannot be directly used as a label, then other models or algorithms are needed to distinguish the similarity between different answers and obtain their cluster labels.

[0059] The comprehensive uncertainty score calculated in the third form is denoted as Score3. It uses the weighted score of the first uncertainty score and the second uncertainty score, i.e., Score3 = α*Score1 + β*Score2, where α and β are the adjustable preset weights of Score1 and Score2, respectively.

[0060] The above three methods for calculating the comprehensive uncertainty fraction can be selected according to actual needs.

[0061] Furthermore, the consistency improvement method in this invention can be selected according to actual needs. For example, the fine-grained self-consistency (FSC), universal self-consistency (USC), or self-consistency (SC) methods can all be used to improve the consistency of the answer parts of all valid thought chains to obtain the final answer output. This is the prior art of each method and is not the focus of the improvement of this invention, so it will not be elaborated further.

[0062] Similarly, based on the same inventive concept, this invention also provides a large model stable output system based on a thought chain pool to implement the above-mentioned method for stable output of large models based on thought chain pools, as shown in Figure 2. It includes a large model scheduler, a thought chain sampler, an uncertainty analyzer, an iterative discriminator, an answer generator, and a filter, each with the following functions:

[0063] The large model scheduler is used to receive the user input question or the iteration instruction sent by the filter, and after receiving the question or iteration instruction, it extracts the large model at the head of the large model queue as the execution large model, and at the same time sends the first execution instruction to the mind chain sampler.

[0064] The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled.

[0065] The uncertainty analyzer is used to receive the second execution instruction sent by the thought chain sampler, and input each thought chain in the thought chain pool, the preset confidence evaluation prompt words and the question into the current execution big model to obtain the confidence score p∈[0,1] of each thought chain, and then convert it to obtain the uncertainty score p'=1-p of the thought chain. After completing the generation of the uncertainty scores of all thought chains, the third execution instruction is sent to the iterative discriminator.

[0066] The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and to filter the thought chains in the thought chain pool using a threshold method. The thought chains with uncertainty scores lower than a preset filtering threshold are considered as valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. Then, it is determined whether the comprehensive uncertainty score exceeds the iterative threshold. If it does not exceed the threshold, a fourth execution instruction is sent to the answer generator; otherwise, a fifth execution instruction is sent to the filter.

[0067] The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, generate the final answer using the consistency improvement method and return it to the user;

[0068] The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thought chains in the thought chain pool whose uncertainty scores exceed the filtering threshold, it retains only the valid thought chains and sends the iteration instruction to the large model scheduler.

[0069] In the large model stable output system based on the thought chain pool described above, the cooperation relationship and data flow between the large model scheduler, thought chain sampler, uncertainty analyzer, iterative discriminator, answer generator, and filter are shown in Figure 3. The scheduling process is as follows:

[0070] Step 1: When the LLM scheduler receives an input question or an iteration instruction sent by a filter, it selects an LLM from the pre-set LLM queue as the LLM to be executed in subsequent steps and dequeues it.

[0071] Step 2: When the thought chain sampler receives the execution instruction from the LLM scheduler, it uses the execution LLM to generate the corresponding number of thought chains to fill the thought chain pool based on the remaining space of the fixed-capacity thought chain pool.

[0072] Step 3: When the uncertainty analyzer receives the signal that the thought chain sampler is full of thought chain pool, it performs uncertainty analysis on the thought chains in the thought chain pool and generates an uncertainty score for each thought chain in the thought chain pool for performing LLM.

[0073] Step 4: After the iterative discriminator receives the signal that the uncertainty analyzer has generated the uncertainty score, it combines the execution of LLM, all the thought chains in the thought chain pool and their corresponding uncertainty scores to perform a comprehensive uncertainty analysis on the thought chain pool and calculate the comprehensive uncertainty score of the thought chain pool. Then, if the comprehensive uncertainty score does not reach the preset iteration threshold or the execution of LLM is the last LLM in the LLM queue, an answer generation instruction is sent to the answer generator, and the process proceeds to step 5; otherwise, a filtering instruction is sent to the filter, and the process proceeds to step 6.

[0074] Step 5: After receiving the answer generation instruction, the answer generator generates the final answer based on the thought chain pool and uses the consistency improvement method, and returns it to the user.

[0075] Step 6: After receiving the filtering instruction, the filter determines if the uncertainty score of a thought chain in the thought chain pool exceeds the preset filtering threshold, and then removes that thought chain from the thought chain pool. After filtering, an iteration instruction is sent to the LLM scheduler, and the process returns to Step 1.

[0076] In addition, based on the same inventive concept, as shown in Figure 4, the present invention also provides a computer electronic device corresponding to the stable output method of a large model based on a thought chain pool provided in the above embodiments, which includes a memory and a processor.

[0077] The memory is used to store computer programs;

[0078] The processor is used to implement the large model stable output method based on the thought chain pool as described above when executing the computer program.

[0079] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0080] Therefore, based on the same inventive concept, this invention provides a computer-readable storage medium corresponding to the method for stable output of large models based on a thought chain pool. The storage medium stores a computer program, which, when executed by a processor, can realize the method for stable output of large models based on a thought chain pool as described above.

[0081] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the stable output method of a large model based on a thought chain pool as described above.

[0082] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S4.

[0083] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0084] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0085] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0086] To demonstrate the advantages of the stable output method for large models based on the thought chain pool as described in S1 to S4 of the present invention, it is applied to a specific example below to show its technical effect.

[0087] Example

[0088] The steps in this embodiment are as described in S1 to S4 above, and will not be repeated here. The following section demonstrates some of the implementation processes and results.

[0089] This embodiment was tested on four publicly available mathematical reasoning datasets: GSM8K, SVAMP, ASDIV, and MultiArith; one commonsense reasoning dataset: StrategyQA; and one symbolic reasoning dataset: CoinFlip.

[0090] The number of samples for each dataset is shown in Table 1.

[0091] Table 1: Dataset Statistics

[0092] In this embodiment, the capacity of the thought pool used in the experiment is fixed at 20. The LLM queue used in the thought pool method is a combination of Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. The sampling parameter configuration for thought pool sampling in this embodiment is shown in Table 2.

[0093] Table 2: Sampling Parameter Configuration

[0094] In this embodiment, the confidence score of the thought chain in the uncertainty analyzer of the thought chain pool method used in the experiment is measured by the Verb.2S top-1 measurement method. First, the confidence score p∈[0,1] of the language model for the thought chain is obtained through the two-stage Verb.2Stop-1 measurement method, and then the uncertainty score p'=1-p of the thought chain is calculated.

[0095] Because the dataset used in the experiment has the characteristic of having a definite answer, and because of the thought chain (r i ,a i ) is the reasoning part r i and answer part a i Composition, where i represents the i-th thought chain. Utilizing this characteristic, this embodiment employs the second method described above for calculating the comprehensive uncertainty score, namely, based on the effective thought chains (fr) in the thought chain pool whose uncertainty scores are below the filtering threshold. i ,fa i According to the answer part fa of the effective thinking chain i The thought chains are directly categorized, with identical answers grouped together. The category with the most thought chains is determined based on this categorization. Where m is the total number of effective thought chains, Ι(fa i =c) is an indicator function, when the answer part fa i The value is 1 for category c and 0 otherwise, thus the calculation method of the comprehensive uncertainty score in the thinking pool method is as follows:

[0096] In this embodiment, the consistency enhancement method selected in the answer generator of the thought chain pool method is the self-consistent enhancement method (SC). The self-consistent enhancement method (SC) calculates the category with the most occurrences after the thought chain is collected as the answer. In the experiment, the number of thought chains collected by the self-consistent enhancement method (SC) is fixed at 20. The consistency enhancement methods used for comparison in the experiment are the general consistency method (USC) and the fine-grained consistency method (FSC). These two methods require the construction of prompt words.

[0097] In this embodiment, the prompt word template used in the experiment is as follows:

[0098] The evaluation metrics used in the experiment of this embodiment are as follows:

[0099] Indicator 1, Accuracy: The percentage of questions correctly solved using the Mind Pool method out of the total number of questions.

[0100] Indicator 2, Reuse Rate: The proportion of mind chains that are reused multiple times out of the total number of generated mind chains, representing the percentage of mind chains that are repeatedly used among all generated mind chains.

[0101] Indicator 3, Number of Problems Solved Ratio: The ratio of the number of problems solved correctly by the three LLM models. LLM1_QGen, LLM2_QGen, and LLM3_QGen represent the number of problems solved correctly by the LLM models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, respectively.

[0102] Solution ratio = #LLM1_QGen:#LLM2_QGen:#LLM3_QGen

[0103] Indicator 4, Mind Chain Ratio: The ratio of the number of mind chains generated by the three LLM models. LLM1_TGen, LLM2_TGen, and LLM3_TGen represent the number of mind chains generated by LLM models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, respectively.

[0104] Mind chain ratio = #LLM1_TGen:#LLM2_TGen:#LLM3_TGen

[0105] Indicator 5, Reuse Ratio: The ratio of the number of thought chains reused once to the number of thought chains reused twice. In the experiment, this is the ratio of the number of thought chains used by both Qwen2-1.5B and internlm2.5-7B to the number of thought chains used by Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. Reuse1 and Reuse2 represent the number of thought chains reused once and twice, respectively.

[0106] Reuse ratio = #Reuse1:#Reuse2

[0107] Metric 6, Accuracy Improvement Rate: Under the same dataset and problem conditions, the improvement in accuracy achieved using the Mind Pool method relative to the accuracy achieved using the Self-Consistent Boosting (SC) method is the highest accuracy achieved using the Mind Pool method, and SC_ACC represents the accuracy achieved using the Self-Consistent Boosting (SC) method.

[0108] Accuracy improvement rate = TP_ACC - SC_ACC

[0109] Indicator 7, Cost Reduction Rate: The cost reduction rate can be expressed in two ways. First, it's calculated by determining the reduction rate of the number of thought chains generated by the last LLM model using the thought chain pooling method compared to the number generated using the self-consistent boosting method (SC), assuming comparable accuracy between the thought chain pooling method and the self-consistent boosting method (SC). Only the last LLM model is calculated because it's intuitively known that the last LLM model has the largest number of parameters and the highest computational cost. Similarly, the second method is calculated by determining the reduction rate of the total number of thought chains generated by the last two LLM models using the thought chain pooling method compared to the number generated using the self-consistent boosting method (SC), assuming comparable accuracy between the thought chain pooling method and the self-consistent boosting method (SC). TP_LLM3_TNUM represents the number of thought chains generated by the LLM model Qwen2.5-14B at the highest accuracy using the thought chain pooling method; TP_LLM2_TNUM represents the number of thought chains generated by the LLM model internlm2.5-7B at the highest accuracy using the thought chain pooling method; and SC_LLM3_TNUM represents the number of thought chains generated by the LLM model Qwen2.5-14B when using the self-consistency boosting method (SC).

[0110] Finally, in this embodiment, three language models, Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, were used to apply consistency enhancement methods on various datasets: the Chain of Thought (COT) method, the Universal Consistency (USC) method, the Fine-Grained Consistency (FSC) method, and the Self-Consistency Enhancement (SC) method. The experimental results of the baseline accuracy are shown in Table 3.

[0111] Table 3 shows the accuracy of different LLM models using consistency enhancement methods on six datasets.

[0112] The experiment used the thought pool method with a fixed thought pool capacity of 20 and a fixed filtering threshold of 0.9. The accuracy, reuse rate, number of solutions, number of thought chains, and number of reuses of the thought pool method under different iteration thresholds were obtained by using different iteration thresholds. The experimental results are shown in Table 4.

[0113] Table 4 Performance metrics under different iteration thresholds

[0114] The experiment in this embodiment used the thought pool method with a fixed thought pool capacity of 20 and a fixed iteration threshold of 0.5. The accuracy, reuse rate, number of solutions, number of thought chains, and number of reuses of the thought pool method under different filtering thresholds were obtained. The experimental results are shown in Table 5.

[0115] Table 5 Performance indicators under different filtration thresholds

[0116] Based on the information provided in Tables 4 and 5, Figure 5 shows the improvement in accuracy across six datasets achieved using the thought pooling method relative to the accuracy achieved by the three language models using the self-consistent boosting method (SC), provided appropriate iteration and filtering thresholds are selected. The value in each cell of Figure 5 represents the difference between the highest accuracy achieved using the thought pooling method and the accuracy achieved by the three models using the self-consistent boosting method (SC) on the same dataset. Experimental results show that using the thought pooling method improves the accuracy of language models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B on six datasets by an average of approximately 20%, 7%, and 7%, respectively, indicating that the thought pooling method can effectively maintain accuracy at a level comparable to the highest-level language model, Qwen2.5-14B.

[0117] Based on the information provided in Tables 4 and 5, under the condition of selecting appropriate iteration thresholds and filtering thresholds, the accuracy achieved by using the thought pooling method is comparable to that of using the self-consistent boosting method (SC) alone. At this point, the accuracy of the thought pooling method is the same as the accuracy obtained in Figure 5, representing the highest accuracy. The reduction rate in the number of thought chains generated by the thought pooling method relative to the self-consistent boosting method (SC) under these conditions is shown in Figure 6. In Figure 6, the value of each cell in the Qwen2.5-14B column represents the reduction rate (i.e., cost reduction rate 1) of the number of thought chains generated by Qwen2.5-14B at the highest accuracy of the thought chain pooling method compared to the number of thought chains generated by Qwen2.5-14B at the self-consistency boosting method (SC); the value of each cell in the internlm2.5-7B+Qwen2.5-14B column represents the reduction rate (i.e., cost reduction rate 2) of the number of thought chains generated by internlm2.5-7B+Qwen2.5-14B at the highest accuracy of the thought chain pooling method compared to the number of thought chains generated by Qwen2.5-14B at the self-consistency boosting method (SC). This shows that the thought chain pooling method reduces the number of thought chains generated by the highest-level language model Qwen2.5-14B by an average of more than 80%, and reduces the number of thought chains generated by the highest-level language model Qwen2.5-14B and the second-highest-level language model internlm2.5-7B by an average of more than 62%, resulting in a significant reduction in cost.

[0118] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for stable output of a large model based on a thought chain pool, characterized in that, include: S1. Based on the user's input question, select a large model from the large model queue as the large model to be executed; S2. According to the preset capacity of the thought chain pool, input the question and the preset thought chain sampling prompt words into the current execution model, and drive the execution model to generate a series of thought chains for the question and fill the thought chain pool. S3. After the thought chain pool is filled, input each thought chain in the thought chain pool, the preset confidence assessment prompt words and the question into the current execution big model to obtain the confidence score of each thought chain, and then convert it to obtain the uncertainty score of the thought chain. S4. After obtaining the uncertainty score of each thought chain in the thought chain pool, the thought chains in the thought chain pool are filtered by a threshold method. Thought chains with uncertainty scores lower than a preset filtering threshold are considered valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thought chains in the thought chain pool with uncertainty scores exceeding the filtering threshold are removed, and the next large model is selected from the current large model queue as the execution large model. The loop returns to S2 and continues. Otherwise, the final answer is generated based on the thought chain pool and using the consistency improvement method, and returned to the user.

2. The method for stable output of large models based on a thought chain pool as described in claim 2, characterized in that, In each round of executing S2, the number of thought chains generated by the current execution model for the problem is the difference between the thought chain capacity of the thought chain pool and the number of currently existing valid thought chains in the thought chain pool.

3. The method for stable output of a large model based on a thought chain pool as described in claim 3, characterized in that, The uncertainty score of the thought chain is generated as follows: first, obtain the confidence score p∈[0,1] of the large model for each thought chain, and then convert it to obtain the uncertainty score p'=1-p of the thought chain.

4. The method for stable output of a large model based on a thought chain pool as described in claim 1, characterized in that, The filtering threshold ranges from [0.5, 0.9], and the iteration threshold ranges from [0.05, 0.5].

5. The method for stable output of a large model based on a thought chain pool as described in claim 5, characterized in that, The comprehensive uncertainty score is either the first uncertainty score, or the second uncertainty score, or a weighted score of the first uncertainty score and the second uncertainty score. The first uncertainty score is the average score of the uncertainty scores of all valid thought chains in the thought chain pool; The second uncertainty score is calculated as follows: First, cluster all valid thought chains in the thought chain pool according to the answer corresponding to each valid thought chain, and count the ratio b of the number of valid thought chains in the largest cluster to the total number of valid thought chains. Use 1-b as the second uncertainty score.

6. The method for stable output of a large model based on a thought chain pool as described in claim 1, characterized in that, The consistency improvement method is a fine-grained consistency method, a general consistency method, or a self-consistency improvement method.

7. A stable output system for large models based on a thought chain pool, characterized in that, It includes a large model scheduler, a thought chain sampler, an uncertainty analyzer, an iterative discriminator, an answer generator, and a filter; The large model scheduler is used to receive the user input question or the iteration instruction sent by the filter, and after receiving the question or iteration instruction, it extracts the large model at the head of the large model queue as the execution large model, and at the same time sends the first execution instruction to the mind chain sampler. The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled. The uncertainty analyzer is used to receive the second execution instruction sent by the thought chain sampler, and input each thought chain in the thought chain pool, the preset confidence evaluation prompt words and the question into the current execution big model to obtain the confidence score of each thought chain, and then convert it to obtain the uncertainty score of the thought chain. After generating the uncertainty scores of all thought chains, it sends the third execution instruction to the iterative discriminator. The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and to filter the thought chains in the thought chain pool using a threshold method. The thought chains with uncertainty scores lower than a preset filtering threshold are considered as valid thought chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thought chains. Then, it is determined whether the comprehensive uncertainty score exceeds the iterative threshold. If it does not exceed the threshold, a fourth execution instruction is sent to the answer generator; otherwise, a fifth execution instruction is sent to the filter. The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, generate the final answer using the consistency improvement method and return it to the user; The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thought chains in the thought chain pool whose uncertainty scores exceed the filtering threshold, it retains only the valid thought chains and sends the iteration instruction to the large model scheduler.

8. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can realize the stable output method of large models based on the thought chain pool as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the stable output method for large models based on a thought pool as described in any one of claims 1 to 6.

10. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the stable output method for large models based on the thought pool as described in any one of claims 1 to 6.

Images