Large model stable output method and system based on thinking chain pool

Abstract

The invention discloses a thinking chain pool-based large model stable output method and system, and belongs to the technical field of generative artificial intelligence. According to the thinking chain pool-based consistency method provided by the invention, the language models with different ability levels are connected through the thinking chain pool based on the hierarchical characteristics of the ability of the language models and the cache characteristics of the thinking chain pool with a fixed capacity, so that the performance of the small-parameter language model is fully exerted; the response time of part of problems is shortened, and the cost of generating the thinking chain by the large-parameter language model is remarkably reduced under the condition that the accuracy rate is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of generative artificial intelligence technology, and specifically relates to a large model stable output method based on a thought chain pool. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, question-answering systems based on large language models have attracted widespread attention due to their powerful natural language understanding and generation capabilities. However, these systems face a key challenge in practical applications: inconsistency in language model output. This inconsistency can lead to unstable output results, affecting the security, robustness, and trustworthiness of the system, and negatively impacting the user experience.

[0003] Existing methods for improving the consistency of large models include self-consistency improvement (SC), universal consistency improvement (USC), and fine-grained consistency improvement (FSC). Based on the CoT, self-consistency improvement (SC) samples the model results multiple times to generate multiple candidate answers, which are then determined by majority voting. Universal consistency improvement (USC) abandons the majority voting method in obtaining the final answer and instead uses prompts to allow the LLM to automatically select the best answer from multiple candidate answers. Fine-grained consistency improvement (FSC) abandons the direct selection of the final answer and instead allows the LLM to actively select consistent fragments from multiple candidate answers and then automatically assemble them into the final answer. All of the methods mentioned above are based on the step of sampling a set of different thought chains. Although these consistency improvement methods are effective, their implementation is computationally expensive because a large number of thought chains must be generated to increase the chances of the correct answer appearing as the most common answer. Summary of the Invention

[0004] The purpose of the present invention is to solve the above technical problems and provide a large model stable output method and system based on the thinking chain pool.

[0005] The specific technical solutions adopted in the present invention are as follows:

[0006] In a first aspect, the present invention provides a large model stable output method based on a thought chain pool, which includes:

[0007] S1. Based on the question input by the user, a large model is selected from the large model queue as the execution large model;

[0008] S2. Input the question and the preset thought chain sampling prompt words into the current execution model according to the thought chain capacity of the preset thought chain pool, and drive the execution model to generate a series of thought chains for the question and fill the thought chain pool;

[0009] S3. After the thought chain pool is filled, each thought chain in the thought chain pool, along with the preset confidence assessment prompt and the question, is input into the current execution model to obtain the confidence score of each thought chain, which is then converted into an uncertainty score for the thought chain;

[0010] S4. After obtaining the uncertainty score of each thinking chain in the thinking chain pool, the thinking chains in the thinking chain pool are screened by the threshold method, and the thinking chains with uncertainty scores lower than the preset filtering threshold are regarded as valid thinking chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thinking chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thinking chains with uncertainty scores exceeding the filtering threshold in the thinking chain pool are cleared, and the next large model is selected from the current large model queue as the execution large model, and the cycle is continued back to S2. Otherwise, the final answer is generated based on the thinking chain pool and the consistency improvement method is used, and returned to the user.

[0011] As a preferred embodiment of the above-mentioned first aspect, when executing S2 in each round, the number of thought chains generated by the current execution large model for the problem is the difference between the thought chain capacity of the thought chain pool and the number of valid thought chains currently existing in the thought chain pool.

[0012] As a preferred embodiment of the first aspect, the uncertainty score of the thought chain is generated by first obtaining the confidence score p∈[0,1] of the large model for each thought chain, and then converting it to obtain the uncertainty score p'=1-p of the thought chain.

[0013] As a preference for the first aspect above, the filtering threshold has a value interval of [0.5, 0.9], and the iteration threshold has a value interval of [0.05, 0.5].

[0014] As a preference of the first aspect above, the comprehensive uncertainty score is the first uncertainty score, or the second uncertainty score, or a weighted score of the first uncertainty score and the second uncertainty score;

[0015] The first uncertainty score is the average uncertainty score of all valid thinking chains in the thinking chain pool;

[0016] The second uncertainty score is calculated as follows: first cluster all valid thinking chains in the thinking chain pool according to the answer corresponding to each valid thinking chain, and count the ratio b of the number of valid thinking chains in the largest cluster to the total number of valid thinking chains, and use 1-b as the second uncertainty score.

[0017] As a preferred embodiment of the above-mentioned first aspect, the consistency improvement method is a fine-grained consistency method (Fine-Grained Self-Consistency, FSC), a universal consistency method (Universal Self-Consistency, USC) or a self-consistency improvement method (Self-Consistency, SC).

[0018] In a second aspect, the present invention provides a large model stable output system based on a thinking chain pool, which includes a large model scheduler, a thinking chain sampler, an uncertainty analyzer, an iterative discriminator, an answer generator, and a filter;

[0019] The large model scheduler is used to receive questions input by users or iteration instructions sent by filters, and after receiving the questions or iteration instructions, extract a large model at the head of the queue from the large model queue as the execution large model, and send the first execution instruction to the thinking chain sampler at the same time;

[0020] The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled;

[0021] The uncertainty analyzer is used to receive the second execution instruction sent by the thinking chain sampler, and input each thinking chain in the thinking chain pool, the preset confidence assessment prompt word and the question into the current execution model to obtain the confidence score of each thinking chain, and then convert it to obtain the uncertainty score of the thinking chain. After completing the generation of the uncertainty scores of all thinking chains, a third execution instruction is sent to the iterative discriminator;

[0022] The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and screen the thinking chains in the thinking chain pool by using a threshold method, taking the thinking chains with uncertainty scores lower than a preset filtering threshold as valid thinking chains, and then calculating a comprehensive uncertainty score based on the uncertainty scores of all valid thinking chains, and then judging whether the comprehensive uncertainty score exceeds the iterative threshold, if not, sending a fourth execution instruction to the answer generator, otherwise sending a fifth execution instruction to the filter;

[0023] The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, use the consistency improvement method to generate the final answer and return it to the user;

[0024] The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thinking chains in the thinking chain pool whose uncertainty scores exceed the filtering threshold, only retain the valid thinking chains and send an iterative instruction to the large model scheduler.

[0025] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the large model stable output method based on the thinking chain pool as described in any one of the first aspects above.

[0026] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the large model stable output method based on the thinking chain pool as described in any one of the first aspects above can be implemented.

[0027] In a fifth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0028] The memory is used to store computer programs;

[0029] The processor is used to implement the large model stable output method based on the thinking chain pool as described in any one of the first aspects above when executing the computer program.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The consistency method based on the thought chain pool proposed in the present invention is based on the hierarchical characteristics of the language model's capabilities and the caching characteristics of the fixed-capacity thought chain pool. Language models of different capability levels are connected through the thought chain pool, which not only fully utilizes the performance of the language model with a small number of parameters and reduces the response time of some problems, but also significantly reduces the cost of generating thought chains for the language model with a large number of parameters while ensuring the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A flowchart of the steps for the stable output method of the large model based on the thinking chain pool;

[0033] Figure 2 This is a schematic diagram of the internal composition of the large model stable output system based on the thinking chain pool;

[0034] Figure 3 To provide a data processing flow chart for each module in the stable output system of the large model based on the thinking chain pool;

[0035] Figure 4 It is a schematic diagram of the structure of computer electronic equipment;

[0036] Figure 5 This is the accuracy improvement result in the embodiment of the present invention;

[0037] Figure 6 It is the reduction rate of the number of thought chains generated in the embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0039] In the description of the present invention, it should be understood that the terms "first" and "second" are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly specifying the number of technical features being described. Therefore, features defined as "first" or "second" may explicitly or implicitly include at least one of such features.

[0040] like Figure 1 As shown, in a preferred embodiment of the present invention, a large model stable output method based on a thought chain pool is provided, which includes S1 to S4 executed in sequence:

[0041] S1. For the question input by the user, a large model (LLM) is selected from the large model queue as the execution large model.

[0042] It should be noted that the "large model queue" in the present invention can be pre-designed according to actual needs, and the ability differences of different large models can be evaluated to screen the large models that meet the requirements and add them to the LLM queue. For example, in an embodiment of the present invention, the large model queue is a combination of Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. In each iteration, the large model at the head of the queue can be taken out from the large model queue as the execution large model according to the conventional dequeue operation. That is, when the LLM scheduler receives the question input by the user, the LLM scheduler selects the LLM as the execution LLM dequeue in the subsequent steps. When receiving the iteration instruction sent by the filter, the LLM scheduler selects the next LLM as the execution LLM dequeue in the subsequent steps.

[0043] S2. According to the preset thinking chain capacity of the thinking chain pool, the question and the preset thinking chain sampling prompt words are input into the current execution model, driving the execution model to generate a series of thinking chains for the question and fill the thinking chain pool.

[0044] It should be noted that the thinking chain capacity of the thinking chain pool refers to the number of thinking chains that can be accommodated in the thinking chain pool. The specific value can be designed according to actual needs, and is preferably 10 to 30. Each time step S2 is executed, it is necessary to use the execution large model to generate a corresponding number of thinking chains based on the remaining space of the fixed-capacity thinking chain pool to fill the thinking chain pool. Except for the first filling, in the remaining filling rounds, since the thinking chain pool may already have thinking chains retained from the previous round, the number of thinking chains that need to be generated is calculated as follows: if the fixed capacity of the thinking chain pool is preset to n, and there are m thinking chains in the current thinking chain pool, then this execution LLM only needs to generate nm thinking chains. The way to execute LLM to generate thinking chains is to input the problem and the preset thinking chain sampling prompt words into the current execution large model. Each input to the execution large model will return the thinking chain of the corresponding problem. A total of nm inputs can be made, and nm thinking chains can be obtained to fill the thinking chain pool. The above-mentioned thinking chain sampling prompt words can be designed according to actual needs through the prompt word engineering, which can drive the execution large model to save the corresponding thinking chain in addition to generating the answer. Each thinking chain (r i ,a i ) is derived from the reasoning part r i and answer part a i where i represents the i-th thinking chain.

[0045] S3. After the thinking chain pool is filled, each thinking chain in the thinking chain pool, the preset confidence assessment prompt words and the question are input into the current execution model to obtain the confidence score p∈[0,1] of each thinking chain, and then convert it to obtain the uncertainty score p'=1-p of the thinking chain.

[0046] It should be noted that the confidence assessment of the thinking chain by the large model can be driven directly by prompt words. The thinking chain to be evaluated and the preset confidence assessment prompt words can be input into the current execution large model. Similarly, the confidence assessment prompt words can be designed according to actual needs through prompt word engineering, and can drive the execution large model to generate a confidence score for the thinking chain. The large model is informed to score the confidence of the thinking chain based on the currently input question and thinking chain, and output a quantitative score value. Of course, in actual applications, the measurement method of the confidence score can be specifically specified as needed. For example, in an embodiment of the present invention, the measurement method of the confidence score of the thinking chain is the Verb.2S top-1 measurement method. That is, the prompt word can specify that the large model needs to first obtain the language model's confidence score p for the thinking chain through the two-stage Verb.2S top-1 measurement method. The value range of its confidence score is [0,1], and then use 1-p to obtain the language model's uncertainty score p' for the thinking chain.

[0047] Of course, there is no single way to generate confidence scores using this chain of thought when performing uncertainty analysis. For example, confidence score generation methods include P(True) (Kadavath et al., 2022), Verbalized 2S (Tian et al., 2023), and Response Probability (Wang et al., 2022). For implementation of these methods, please refer to the following references:

[0048] Kadavath S,Conerly T,Askell A,et al.Language models(mostly)know whatthey know[J].arXiv preprint arXiv:2207.05221,2022.

[0049] Tian K, Mitchell E, Zhou A, et al. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tunedwith Human Feedback[C] / / Proceedings of the 2023Conference on EmpiricalMethods in Natural Language Processing.2023:5433-5442.

[0050] Wang

[0051] S4. After obtaining the uncertainty score of each thinking chain in the thinking chain pool, the thinking chains in the thinking chain pool are screened by the threshold method, and the thinking chains with uncertainty scores lower than the preset filtering threshold are regarded as valid thinking chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thinking chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thinking chains with uncertainty scores exceeding the filtering threshold in the thinking chain pool are cleared, and the next large model is selected from the current large model queue as the execution large model, and the cycle is continued back to S2. Otherwise, the final answer is generated based on the thinking chain pool and the consistency improvement method is used, and returned to the user.

[0052] It should be noted that the above-mentioned filtering threshold is a hyperparameter that can be optimized according to actual performance. After each thinking chain in the thinking chain pool obtains the uncertainty score p', it can be judged whether it exceeds the filtering threshold. If it exceeds, it means that the uncertainty of the thinking chain is too high and cannot be used as a valid thinking chain. If it does not exceed, it means that the thinking chain is relatively certain and can be retained in the thinking chain pool as a valid thinking chain for final consistency improvement. Similarly, the above-mentioned iteration threshold is also a hyperparameter that can be optimized according to actual performance. In an embodiment of the present invention, the value interval of the filtering threshold is preferably [0.5, 0.9], and the value interval of the iteration threshold is preferably [0.05, 0.5].

[0053] Based on the uncertainty score p' of each thinking chain in the thinking chain pool and the filtering threshold, all valid thinking chains (fr i ,fa i ), the subsequent calculation of the comprehensive uncertainty score can be performed. The comprehensive uncertainty score can be calculated in a variety of different ways. Three different calculation methods are given below.

[0054] The comprehensive uncertainty score calculated in the first form is recorded as the first uncertainty score Score1, which is calculated as follows:

[0055] Calculate the average uncertainty score of all valid thinking chains in the thinking chain pool whose uncertainty score p' is lower than the filtering threshold as the comprehensive uncertainty score of the thinking chain pool Where m is the total number of effective thinking chains, Si is the uncertainty score of the i-th thinking chain.

[0056] The comprehensive uncertainty score calculated in the second form is recorded as the second uncertainty score Score2, which is calculated as follows:

[0057] First, cluster all valid thought chains in the thought chain pool according to the answer corresponding to each valid thought chain, and calculate the ratio b of the number of valid thought chains in the largest cluster to the total number of valid thought chains, and use 1-b as the second uncertainty score. Specifically, the above clustering method needs to be selected according to the situation of the answer. For common math calculation or classification tasks, the answer part of the valid thought chain is fa i Often it can be used as a category label directly, according to the answer part of the effective thinking chain i Classify the thought chains, and then count the number of valid thought chains contained in each thought chain classification, and obtain the category with the largest number of thought chains based on the results of the thought chain classification. Where m is the total number of effective thinking chains, Ι(fa i =c) is the indicator function, when the answer part fa i The value is 1 when it is category c, otherwise it is 0. Then the comprehensive uncertainty score of the thinking chain pool is obtained as

[0058] Of course, if the answer part of the effective thinking chain is fa i If it cannot be used directly as a label, other models or algorithms are needed to distinguish the similarities between different answers and obtain their cluster labels.

[0059] The comprehensive uncertainty score calculated in the third form is recorded as Score3, which uses the weighted score of the first uncertainty score and the second uncertainty score, that is, Score3 = α*Score1+β*Score2, where α and β are the adjustable preset weights of Score1 and Score2 respectively.

[0060] The above three calculation forms of comprehensive uncertainty scores can be selected according to actual needs.

[0061] In addition, the consistency improvement method in the present invention can be selected according to actual needs. For example, the fine-grained consistency method (Fine-Grained Self-Consistency, FSC), the universal consistency method (Universal Self-Consistency, USC) or the self-consistency improvement method (Self-Consistency, SC) can all be adopted. These consistency improvement methods are used to improve the consistency of the answer parts of all valid thinking chains to obtain the final answer output. This belongs to the existing technology of each method and is not the improvement focus of the present invention. It will not be repeated here.

[0062] Similarly, based on the same inventive concept, the present invention also provides a large model stable output system based on a thinking chain pool to implement the above large model stable output method based on a thinking chain pool, such as Figure 2 As shown in the figure, it includes a large model scheduler, a thought chain sampler, an uncertainty analyzer, an iterative discriminator, an answer generator, and a filter. Their respective functions are as follows:

[0063] The large model scheduler is used to receive questions input by users or iteration instructions sent by filters, and after receiving the questions or iteration instructions, extract a large model at the head of the queue from the large model queue as the execution large model, and send the first execution instruction to the thinking chain sampler at the same time;

[0064] The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled;

[0065] The uncertainty analyzer is used to receive the second execution instruction sent by the thinking chain sampler, and input each thinking chain in the thinking chain pool, the preset confidence assessment prompt word and the question into the current execution model to obtain the confidence score p∈[0,1] of each thinking chain, and then convert it to obtain the uncertainty score p'=1-p of the thinking chain. After completing the generation of the uncertainty scores of all thinking chains, a third execution instruction is sent to the iterative discriminator;

[0066] The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and screen the thinking chains in the thinking chain pool by using a threshold method, taking the thinking chains with uncertainty scores lower than a preset filtering threshold as valid thinking chains, and then calculating a comprehensive uncertainty score based on the uncertainty scores of all valid thinking chains, and then judging whether the comprehensive uncertainty score exceeds the iterative threshold, if not, sending a fourth execution instruction to the answer generator, otherwise sending a fifth execution instruction to the filter;

[0067] The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, use the consistency improvement method to generate the final answer and return it to the user;

[0068] The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thinking chains in the thinking chain pool whose uncertainty scores exceed the filtering threshold, only retain the valid thinking chains and send an iterative instruction to the large model scheduler.

[0069] In the above-mentioned large model stable output system based on the thinking chain pool, the coordination relationship and data flow between the large model scheduler, thinking chain sampler, uncertainty analyzer, iterative discriminator, answer generator, and filter are as follows: Figure 3 As shown, the scheduling process is as follows:

[0070] Step 1: When the LLM scheduler receives the input question or the iteration instruction sent by the filter, it selects an LLM from the pre-set LLM queue as the execution LLM in the subsequent steps and dequeues it;

[0071] Step 2: After the thought chain sampler receives the execution instruction from the LLM scheduler, it uses the execution LLM to generate a corresponding number of thought chains according to the remaining space of the thought chain pool with a fixed capacity to fill the thought chain pool;

[0072] Step 3: When the uncertainty analyzer receives a signal from the thinking chain sampler that the thinking chain pool is full, it performs uncertainty analysis on the thinking chains in the thinking chain pool and generates an uncertainty score for executing the LLM for each thinking chain in the thinking chain pool;

[0073] Step 4. After the iterative discriminator receives the signal that the uncertainty analyzer has generated the uncertainty score, it performs a comprehensive uncertainty analysis on the thinking chain pool by combining the execution LLM, all thinking chains in the thinking chain pool, and the uncertainty scores corresponding to the thinking chains, and calculates the comprehensive uncertainty score of the thinking chain pool. Then, if the comprehensive uncertainty score does not reach the preset iteration threshold or the execution LLM is the last LLM in the LLM queue, it sends an answer generation instruction to the answer generator, and the process goes to step 5; otherwise, it sends a filtering instruction to the filter, and the process goes to step 6;

[0074] Step 5: After receiving the answer generation instruction, the answer generator generates the final answer based on the thought chain pool and uses the consistency improvement method to return it to the user;

[0075] Step 6: After receiving the filter instruction, the filter determines if the uncertainty score of a thought chain in the thought chain pool exceeds the preset filter threshold, and then removes the thought chain from the thought chain pool. After the filtering is completed, an iteration instruction is sent to the LLM scheduler, and the process returns to step 1.

[0076] In addition, based on the same inventive concept, Figure 4 As shown, the present invention also provides a computer electronic device corresponding to the large model stable output method based on the thinking chain pool provided in the above embodiment, which includes a memory and a processor;

[0077] The memory is used to store computer programs;

[0078] The processor is configured to implement the aforementioned large model stable output method based on the thinking chain pool when executing the computer program;

[0079] Furthermore, the logic instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention.

[0080] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium corresponding to a large model stable output method based on a thinking chain pool, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it can realize the large model stable output method based on a thinking chain pool as described above.

[0081] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the large model stable output method based on the thinking chain pool as described above.

[0082] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by the processor to perform the above steps S1 to S4.

[0083] It is understood that the storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage medium may be any medium capable of storing program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0084] It is understandable that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0085] It should also be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division. In actual implementation, there may be other division methods, for example, multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0086] In order to demonstrate the advantages of the large model stable output method based on the thought chain pool shown in S1 to S4 above of the present invention, it is applied to a specific example below to demonstrate its technical effect.

[0087] Example

[0088] The steps of this embodiment are shown in S1 to S4 above, which will not be described in detail here. The following is a demonstration of some implementation processes and implementation results.

[0089] This embodiment conducts experiments on four public mathematical reasoning datasets: GSM8K, SVAMP, ASDIV, and MultiArith; one common sense reasoning dataset: StrategyQA; and one symbolic reasoning dataset: CoinFlip.

[0090] The number of samples for each dataset is shown in Table 1.

[0091] Table 1: Dataset statistics

[0092]

[0093]

[0094] The experiment in this embodiment uses a thinking chain pool method with a fixed thinking chain pool capacity of 20. The LLM queues used in the thinking chain pool method are a combination of Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. The sampling parameter configuration for thinking chain sampling in this embodiment is shown in Table 2.

[0095] Table 2: Sampling parameter configuration

[0096]

[0097] The experiment in this example used the Verb.2S top-1 method to measure the confidence score of the thought chain in the uncertainty analyzer of the thought chain pool method. First, the language model's confidence score for the thought chain was obtained through the two-stage Verb.2Stop-1 method, p∈[0,1]. Then, the uncertainty score of the thought chain was converted to p'=1-p.

[0098] Because the data set used in the experiment has the characteristic of determining the answer, and because the thinking chain (r i ,a i ) is derived from the reasoning part r i and answer part a i , where i represents the i-th thinking chain. Taking advantage of this feature, this embodiment adopts the second comprehensive uncertainty score calculation method mentioned above, that is, based on the effective thinking chains (fr i ,fa i ), according to the answer part of the effective thinking chain i Classify the thought chains directly, group the same answers into one category, and obtain the category with the largest number of thought chains based on the results of thought chain classification. Where m is the total number of effective thinking chains, Ι(fa i =c) is the indicator function, when the answer part fa i The value is 1 when it is category c, otherwise it is 0, and the calculation method of the comprehensive uncertainty score in the thinking chain pool method is

[0099] The consistency improvement method used in the answer generator of the thought chain pool method used in the experiment of this embodiment is the self-consistency improvement method (SC). The self-consistency improvement method (SC) adopts a method of calculating the category with the largest number as the answer after the thought chain is collected. In the experiment, the number of thought chains collected by the self-consistency improvement method (SC) is fixed at 20, and the comparative consistency improvement methods used in the experiment are the universal consistency method (USC) and the fine-grained consistency method (FSC). These two methods require the construction of prompt words.

[0100] In this embodiment, the prompt word template used in the experiment is as follows:

[0101]

[0102] The evaluation indicators used in the experiment of this embodiment are as follows:

[0103] Indicator 1. Accuracy: The ratio of the number of problems correctly solved by the thinking chain pool method to the total number of problems

[0104]

[0105] Indicator 2, reuse rate: the ratio of the number of thought chains used multiple times to the total number of thought chains generated, indicating the proportion of thought chains used repeatedly in all generated thought chains.

[0106]

[0107] Indicator 3, number of solutions: the ratio of the number of problems correctly solved by the three LLM models. LLM1_QGen, LLM2_QGen, and LLM3_QGen represent the number of problems correctly solved by the LLM models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, respectively.

[0108] Solution ratio = #LLM1_QGen:#LLM2_QGen:#LLM3_QGen

[0109] Indicator 4, Thinking Chain Ratio: The ratio of the number of thinking chains generated by the three LLM models. LLM1_TGen, LLM2_TGen, and LLM3_TGen represent the number of thinking chains generated by the LLM models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, respectively.

[0110] Thought chain ratio = #LLM1_TGen:#LLM2_TGen:#LLM3_TGen

[0111] Indicator 5, Reuse ratio: The ratio of the number of thought chains reused once to the number of thought chains reused twice. In the experiment, the number of thought chains used by both Qwen2-1.5B and internlm2.5-7B was divided by the number of thought chains used by Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B. Reuse1 and Reuse2 represent the number of thought chains reused once and twice, respectively.

[0112] Reuse ratio = #Reuse1:#Reuse2

[0113] Indicator 6, Accuracy Improvement Rate: The improvement in the highest accuracy obtained by using the Thinking Chain Pool method relative to the accuracy obtained by using the Self-Consistency Improvement method (SC) under the same dataset and the same problem conditions. TP_ACC represents the highest accuracy obtained by using the Thinking Chain Pool method, and SC_ACC represents the accuracy obtained by using the Self-Consistency Improvement method (SC).

[0114] Accuracy improvement rate = TP_ACC - SC_ACC

[0115] Indicator 7, Cost Reduction Rate: The cost reduction rate can be measured in two ways. The first is by calculating the reduction rate of the number of thought chains generated by the last LLM model using the thought chain pool method relative to the number of thought chains generated using the self-consistency improvement method (SC), assuming the accuracy of the thought chain pool method and the self-consistency improvement method (SC) are comparable. Only the last LLM is calculated because it intuitively has the largest number of parameters and the highest computational cost. Similarly, the second is by calculating the reduction rate of the sum of the number of thought chains generated by the last two LLM models using the thought chain pool method relative to the number of thought chains generated using the self-consistency improvement method (SC), assuming the accuracy of the thought chain pool method and the self-consistency improvement method (SC) are comparable. TP_LLM3_TNUM represents the number of thinking chains generated by the LLM model Qwen2.5-14B using the thinking chain pool method at the highest accuracy, TP_LLM2_TNUM represents the number of thinking chains generated by the LLM model internlm2.5-7B using the thinking chain pool method at the highest accuracy, and SC_LLM3_TNUM represents the number of thinking chains generated by the LLM model Qwen2.5-14B when using the self-consistency improvement method (SC).

[0116]

[0117] Finally, the experiment in this embodiment used three language models, Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B, and applied consistency improvement methods on various data sets: the thinking chain method (COT), the universal consistency method (USC), the fine-grained consistency method (FSC), and the self-consistency improvement method (SC). The experimental results of the baseline accuracy are shown in Table 3.

[0118] Table 3 Accuracy of different LLM models using consistency improvement methods on 6 datasets

[0119]

[0120]

[0121] The experiment used the thinking chain pool method to obtain the accuracy, reuse rate, solution ratio, thinking chain ratio, and reuse ratio of the thinking chain pool method under different iteration thresholds under the conditions of a fixed thinking chain pool capacity of 20 and a fixed filtering threshold of 0.9. The experimental results are shown in Table 4.

[0122] Table 4 Performance indicators under different iteration thresholds

[0123]

[0124]

[0125]

[0126] The experiment of this embodiment uses the thinking chain pool method to obtain the accuracy, reuse rate, solution ratio, thinking chain ratio, and reuse ratio of the thinking chain pool method under different filtering thresholds under the conditions of a fixed thinking chain pool capacity of 20 and a fixed iteration threshold of 0.5. The experimental results are shown in Table 5.

[0127] Table 5 Performance indicators under different filtering thresholds

[0128]

[0129]

[0130]

[0131] According to the information provided in Tables 4 and 5, the highest accuracy rate obtained by using the thinking chain pool method under the condition of selecting appropriate iteration thresholds and filtering thresholds is compared with the accuracy rate obtained by the self-consistency improvement method (SC) of the three language models on the six data sets. Figure 5 shown. Figure 5The value in each box represents the difference between the highest accuracy achieved using the thought chain pooling method and the accuracy of the three models using the self-consistency improvement method (SC) on the same dataset. Experimental results show that the thought chain pooling method improves the accuracy of the language models Qwen2-1.5B, internlm2.5-7B, and Qwen2.5-14B using the self-consistency improvement method (SC) on the six datasets by an average of approximately 20%, 7%, and 7%, respectively. This demonstrates that the thought chain pooling method can effectively maintain accuracy comparable to the highest-level language model, Qwen2.5-14B.

[0132] According to the information provided in Table 4 and Table 5, under the condition of selecting appropriate iteration threshold and filtering threshold, the accuracy achieved by using the thinking chain pool method is equivalent to the accuracy of using the self-consistency improvement method (SC) alone. Figure 5 The obtained accuracy is the same as the highest accuracy. Under these conditions, the reduction rate of the number of thought chains generated by the thought chain pool method compared with the self-consistency improvement method (SC) is as follows: Figure 6 shown. Figure 6 In the figure, the value of each grid in the Qwen2.5-14B column represents the reduction rate of the number of thought chains generated by Qwen2.5-14B under the highest accuracy of the thought chain pool method compared with the reduction rate of the thought chains generated by Qwen2.5-14B under the self-consistency improvement method (SC), that is, the cost reduction rate 1; the value of each grid in the internlm2.5-7B+Qwen2.5-14B column represents the reduction rate of the number of thought chains generated by internlm2.5-7B+Qwen2.5-14B under the highest accuracy of the thought chain pool method compared with the reduction rate of the thought chains generated by Qwen2.5-14B under the self-consistency improvement method (SC), that is, the cost reduction rate 2. From this, it can be seen that the thought chain pool method reduces the number of thought chains generated using the highest ability level language model Qwen2.5-14B by an average of more than 80%, and the thought chain pool method reduces the number of thought chains generated using the highest ability level language model Qwen2.5-14B and the second highest ability level language model internlm2.5-7B by an average of more than 62%, with a significant cost reduction.

[0133] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A large model stable output method based on thought chain pool, characterized in that: include: S1. Based on the question input by the user, a large model is selected from the large model queue as the execution large model; S2. Input the question and the preset thought chain sampling prompt words into the current execution model according to the thought chain capacity of the preset thought chain pool, and drive the execution model to generate a series of thought chains for the question and fill the thought chain pool; S3. After the thought chain pool is filled, each thought chain in the thought chain pool, along with the preset confidence assessment prompt and the question, is input into the current execution model to obtain the confidence score of each thought chain, which is then converted into an uncertainty score for the thought chain; S4. After obtaining the uncertainty score of each thinking chain in the thinking chain pool, the thinking chains in the thinking chain pool are screened by the threshold method, and the thinking chains with uncertainty scores lower than the preset filtering threshold are regarded as valid thinking chains. Then, a comprehensive uncertainty score is calculated based on the uncertainty scores of all valid thinking chains. If the comprehensive uncertainty score exceeds the iteration threshold, the thinking chains with uncertainty scores exceeding the filtering threshold in the thinking chain pool are cleared, and the next large model is selected from the current large model queue as the execution large model, and the cycle is continued back to S2. Otherwise, the final answer is generated based on the thinking chain pool and the consistency improvement method is used, and returned to the user.

2. The large model stable output method based on the thought chain pool as claimed in claim 1, characterized in that: When executing S2 in each round, the number of thought chains generated by the current execution large model for the problem is the difference between the thought chain capacity of the thought chain pool and the number of valid thought chains currently existing in the thought chain pool.

3. The large model stable output method based on the thought chain pool as claimed in claim 1, characterized in that: The uncertainty score of the thought chain is generated as follows: first obtain the confidence score p∈[0,1] of the large model for each thought chain, and then convert it to obtain the uncertainty score p'=1-p of the thought chain.

4. The large model stable output method based on the thought chain pool as claimed in claim 1, characterized in that: The filtering threshold has a value interval of [0.5, 0.9], and the iteration threshold has a value interval of [0.05, 0.5].

5. The large model stable output method based on the thought chain pool as claimed in claim 1, characterized in that: The comprehensive uncertainty score is the first uncertainty score, or the second uncertainty score, or a weighted score of the first uncertainty score and the second uncertainty score; The first uncertainty score is the average uncertainty score of all valid thinking chains in the thinking chain pool; The second uncertainty score is calculated as follows: first cluster all valid thinking chains in the thinking chain pool according to the answer corresponding to each valid thinking chain, and count the ratio b of the number of valid thinking chains in the largest cluster to the total number of valid thinking chains, and use 1-b as the second uncertainty score.

6. The large model stable output method based on the thought chain pool as claimed in claim 1, characterized in that: The consistency improvement method is a fine-grained consistency method (Fine-Grained Self-Consistency, FSC), a universal consistency method (Universal Self-Consistency, USC) or a self-consistency improvement method (Self-Consistency, SC).

7. A large model stable output system based on the thinking chain pool, characterized by: Including large model scheduler, thought chain sampler, uncertainty analyzer, iterative discriminator, answer generator, filter; The large model scheduler is used to receive questions input by users or iteration instructions sent by filters, and after receiving the questions or iteration instructions, extract a large model at the head of the queue from the large model queue as the execution large model, and send the first execution instruction to the thinking chain sampler at the same time; The thought chain sampler is used to receive the first execution instruction sent by the large model scheduler, and input the problem and the preset thought chain sampling prompt words into the current execution large model according to the thought chain capacity of the preset thought chain pool, drive the execution large model to generate a series of thought chains for the problem and fill the thought chain pool, and send the second execution instruction to the uncertainty analyzer after the thought chain pool is filled; The uncertainty analyzer is used to receive the second execution instruction sent by the thinking chain sampler, and input each thinking chain in the thinking chain pool, the preset confidence assessment prompt word and the question into the current execution model to obtain the confidence score of each thinking chain, and then convert it to obtain the uncertainty score of the thinking chain. After completing the generation of the uncertainty scores of all thinking chains, a third execution instruction is sent to the iterative discriminator; The iterative discriminator is used to receive the third execution instruction sent by the uncertainty analyzer, and screen the thinking chains in the thinking chain pool by using a threshold method, taking the thinking chains with uncertainty scores lower than a preset filtering threshold as valid thinking chains, and then calculating a comprehensive uncertainty score based on the uncertainty scores of all valid thinking chains, and then judging whether the comprehensive uncertainty score exceeds the iterative threshold, if not, sending a fourth execution instruction to the answer generator, otherwise sending a fifth execution instruction to the filter; The answer generator is used to receive the fourth execution instruction sent by the iterative discriminator, and based on the thought chain pool, use the consistency improvement method to generate the final answer and return it to the user; The filter is used to receive the fifth execution instruction sent by the iterative discriminator, and after clearing the thinking chains in the thinking chain pool whose uncertainty scores exceed the filtering threshold, only retain the valid thinking chains and send an iterative instruction to the large model scheduler.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it can implement the large model stable output method based on the thinking chain pool as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the large model stable output method based on the thinking chain pool as described in any one of claims 1 to 6 is implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the large model stable output method based on the thinking chain pool as described in any one of claims 1 to 6 when executing the computer program.

Images