A mathematical reasoning method for large language models based on retelling quality optimization

By integrating multi-index scoring and pairwise comparison methods, combined with XGBoost classifier to screen the best retelling as Prompt, the problem of large language models performing poorly in mathematical reasoning, significantly improving its accuracy in mathematical Q&A tasks .

CN118674053BActive Publication Date: 2024-11-22HANGZHOU DIANZI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411170996.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-22
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Large language models perform weakly in mathematical reasoning and logical reasoning, and have different understandings of human cues, which affects their performance in mathematical Q&A tasks.

Method used

A mathematical problem retelling quality evaluation method that combines multi-index scores and pairwise comparisons is adopted, and combined with XGBoost classifiers, the best retelling is selected to improve the mathematical reasoning ability of the model. This method sets evaluation indicators such as coherence, consistency, fluency, correlation, matching and logic, and forms a partial order relationship through multiple comparisons to ensure that the relative rankings between retelling are clear, and finally select the best using the decision module Repeat as Prompt input to LLM.

Benefits of technology

The performance of LLM in mathematical Q&A tasks has been effectively improved. The experimental results show that the accuracy rate on the GSM8K test set has been improved by 2.28%, proving the effectiveness and expansion of this method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118674053B_ABST
    Figure CN118674053B_ABST
Patent Text Reader

Abstract

The present invention relates to a large language model mathematical reasoning method based on repetition quality optimization, which improves the mathematical reasoning ability of the large language model through high-quality problem repetition. This scheme integrates two modules: multi-index scoring and pairwise comparison: the former starts from six perspectives, determines the best prompting method through experiments, and uses the large language model to score the repetition quality with a single index; the latter uses the large language model to perform relative ranking between repetitions based on the comprehensive quality of the repetition, and finally trains a gradient boosting decision tree classification model, takes the scores of the multi-index scoring module and the relative relationship of the pairwise comparison module as feature inputs, and makes decisions and selects the best repetition. The present invention provides a general evaluation framework for repetition quality assessment using a large language model, which effectively improves the performance of the model on mathematical problems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a large language model mathematical reasoning method based on repetition quality optimization, which can effectively evaluate the quality of the repetition of mathematical problems and improve the performance of the model on mathematical reasoning data sets, and belongs to the technical field of natural language processing. Background Art

[0002] Large language models (LLMs) have excellent performance in many downstream tasks in the field of natural language processing (NLP), but not all tasks can be easily handled: LLMs usually perform well in downstream tasks such as text summarization, sentiment analysis, question-answering systems, and language translation, but perform slightly worse in logical reasoning and mathematical question-answering. The former requires LLM to understand grammar and semantics, extract information, and generate coherent text; the latter has higher requirements on the model's logical reasoning ability.

[0003] Prompts given by humans will greatly affect the response of the language model: Zero-shot CoT allows the model to generate its own reasoning path by adding special prompts; the introduction of In-Context Learning (ICL) also confirms that adding examples to prompts will affect the output of the model; different demonstration choices will also affect the performance of ICL. It can be seen that prompts have a significant impact on LLM, and whether LLM can understand prompts more accurately and comprehensively is directly related to the quality of the model's response.

[0004] Relevant research shows that there is a difference between LLM’s understanding of Prompt and the true intention of human input. Therefore, some work is dedicated to guiding the language model to fit human preferences, such as reinforcement learning from human feedback (RLHF), reinforcement learning from AI feedback (RLAIF) and direct preference optimization (DPO). Some work also goes the other way and optimizes Prompt to adapt to LLM’s understanding, such as using language model as prompt optimizer (OPRO), iteratively optimizing Prompt using Actor-Critic model (PACE) and training small model to optimize prompt (BPO).

[0005] The introduction of Rephrase+Respond (RaR) allows the large language model to restate the input question and then answer it, which can improve the performance of LLM in downstream tasks to a certain extent: the language model may have misunderstandings about the original input, and the restated questions will tend to enhance semantic clarity and add additional details, which is conducive to solving the original problem.

[0006] Effective evaluation methods are instructive for improving model performance. Related research shows that LLM can be used as an evaluator. For example, GPT-4 is used to evaluate the answers generated by small models. These models are usually trained with RLHF and show strong human alignment, which is sufficient to serve as an alternative to human evaluation. Taking the text summarization task as an example, G-Eval proposed an excellent evaluation framework, which, combined with the carefully designed prompts, showed good human alignment.

[0007] LLM's effective evaluation of text generation tasks has greatly promoted the development of this task and proved its outstanding ability in the field of text generation. In contrast, due to the relatively weak reasoning ability of the language model, it performs poorly in solving mathematical problems. Summary of the invention

[0008] In order to overcome the shortcomings of existing research, this paper provides a large language model mathematical reasoning method based on retelling quality optimization, aiming to make full use of the advantages of LLM in the field of text generation and improve its performance in weak fields such as mathematical question answering. By retelling mathematical problems, a connection with text generation tasks can be established: a high-quality retelling can enhance LLM's understanding of input prompts, and at the same time, as a supplement and emphasis on the original problem, indirectly improve the model's performance in mathematical question answering tasks.

[0009] The innovation of the present invention lies in proposing a method for evaluating the quality of mathematical problem repetition that integrates multi-index scoring and pairwise comparison, aiming to optimize the selection of the best repetition, thereby improving the mathematical reasoning ability of the large language model through high-quality problem repetition. Multi-index scoring can comprehensively consider multiple evaluation indicators, and the evaluation angle is more comprehensive, but there is also the possibility of score fluctuations; pairwise comparison can intuitively compare the comprehensive quality of the repetition and perform relative ranking, but it is impossible to achieve a comprehensive evaluation. Therefore, the present invention adopts a fusion strategy, comprehensively uses multi-index scoring and pairwise comparison, and combines the XGBoost classifier to make the final decision to select the best repetition.

[0010] In the multi-index scoring module, considering that the paraphrase and text summarization tasks are similar in a certain sense, the present invention refers to the evaluation index proposed by G-Eval in the text summarization task, optimizes and adapts it, and expands some evaluation indexes that are more suitable for mathematical problems on this basis. In the paired comparison module, the present invention adopts a multiple comparison and integration method to compare the paraphrases in pairs to determine their relative rankings and form a partial order relationship. Finally, the outputs of the above two modules are used as the input of the decision module to screen out the best paraphrase.

[0011] The specific steps of a mathematical reasoning method for a large language model based on retelling quality optimization are as follows:

[0012] Step 1: Select a math problem from the dataset, construct a prompt, and generate multiple paraphrases using LLM;

[0013] Step 2: Set six evaluation indicators, namely, coherence, consistency, fluency, relevance, matching and logic, for retelling quality evaluation; set different prompts for each indicator through experiments, so as to more effectively exert the evaluation performance of each indicator;

[0014] Step 3: Combine the six indicators in step 2 and their corresponding prompts, use LLM to perform multi-indicator scoring on each repetition, and use the probability mean of the output token as the score to ensure its discrimination;

[0015] Step 4: Use LLM to compare all repetitions in pairs, and obtain the relative relationship between repetitions by accumulating scores through multiple comparisons. To this end, the results of the pairwise comparisons should satisfy the partial order relationship to ensure that the relative ranking between repetitions is clear.

[0016] Step 5: Train a classifier as a decision model, which includes the following processes: preprocess the training set and select the samples with the best repetition; use the above samples to execute steps 1 to 4 to obtain the input features and labels of the model; select the gradient boosting tree algorithm to implement the classification task, and use XGBoost to train the classifier for the final decision;

[0017] Step 6: Input the scores of the multi-index scoring module in step 3 and the integral of the pairwise comparison module in step 4 into the trained decision module to obtain the decision result, which is selected as the best repetition;

[0018] Step 7: Use the best selected restatement combined with the original question as the prompt, and input it to GPT-3.5 in the form of zero-shot thinking chain (Zero-shot CoT) combined with self-consistency check (SC) to obtain the answer to the question.

[0019] The step 1 specifically includes: selecting a mathematical problem q from the data set and constructing a prompt for the problem , use LLM to generate n paraphrases , the generated i-th retelling can be expressed as:

[0020] .

[0021] The evaluation indicators in the step 2 are: referring to the evaluation method of the text summarization task, four indicators, namely coherence (Coh), consistency (Con), fluency (Flu) and relevance (Rel), are set as the evaluation indicators of the retelling quality; considering the characteristics of mathematical problems, two additional evaluation indicators that are more in line with mathematical tasks are added: matching (Mat) and logic (Log).

[0022] The prompt modes are: not using the indicator, using standard prompt (Standard), standard prompt + evaluation criteria (Standard+Criteria), standard prompt + evaluation criteria + evaluation steps (Standard+Criteria+Steps). The standard prompt includes the current task goal, constraints, the original math problem, the restatement to be evaluated, and the output template; the evaluation steps are automatically generated by GPT-4.

[0023] The four prompting methods are applied to the six evaluation indicators respectively, and their effects on the accuracy of the data set are observed to determine the best prompting method for each indicator. For different data sets of math question-answering tasks, the prompting method selected for each indicator may be different, and the specific selection should be determined based on the data set through experiments. The prompting method corresponding to each evaluation indicator can be expressed as:

[0024] .

[0025] The step three specifically includes:

[0026] Combined with the six indicators in step 2 and their corresponding prompts, LLM is used to score each retelling with multiple indicators, ranging from 1 to 5 points. A single score under this indicator can be expressed as .

[0027] Directly using the model output as the scoring result may result in a small degree of discrimination in the scoring. This solution uses the probability mean of the model output token as the single indicator score of the LLM for the current retelling. Specifically, the language model output score is The probability of , then repeat The final score of indicator Coh can be expressed as:

[0028] .

[0029] For LLMs such as GPT-4 that do not provide token output probabilities, we can use the average of multiple scoring to simulate. Taking the score of indicator Coh as an example, LLM is used to perform K scorings to obtain the score set ,in and , represents the output score of the model for the kth time. The number of times the output score is t in the score set S is , you can use To simulate the probability that the LLM output score is t Therefore, retell The final score of indicator Coh can be expressed as:

[0030] .

[0031] The step 4 specifically includes:

[0032] Use LLM to compare all paraphrases in pairs, using a circular comparison method, that is, each paraphrase needs to be compared with other alternative paraphrases once, so n paraphrases will form a total of A comparison group. and retelling The comparison can be expressed as ,in Indicates specific prompts for pairwise comparisons.

[0033] Since there is a position bias in paired comparisons, that is, LLM will favor the repetition that comes first in the prompt, this scheme will compare the comparison groups 5 times in sequence, swap the order of the two, and then compare them 5 times to reduce the impact of position bias on the results. For each comparison, LLM will choose the repetition with better overall performance, and the repetition will receive 2 points; if the two are evenly matched, each will receive 1 point. and The integrals obtained after comparison are expressed as and .

[0034] After a round of effective pairwise comparison, a clear partial order relationship should be formed between the repetitions, that is, the following constraints are met, and the superiority relationship between the repetitions can form a directed acyclic Hasse graph;

[0035]

[0036] If the above constraints are not met between the repetitions, the comparison should be considered self-contradictory, and the contradictory parts should be re-compared. Specifically, the superiority relationship between the repetitions is converted into a directed graph. The parts that do not meet the above constraints will form a loop in the graph. Use topological sorting to remove all points with in-degree 0 and out-degree 0. The remaining parts that cannot be removed are all the repetitions that need to be re-compared.

[0037] The step five specifically includes:

[0038] The decision module evaluates and makes decisions based on the output of the multi-index scoring module and the results of the pairwise comparison module, and selects one of the n retellings as the best retelling. Its essence is a classification model, and the training process is as follows:

[0039] 5.1 Sample selection: Use LLM to generate n paraphrases for each sample in the training set , respectively, combined with the original question, using the Zero-shot CoT+SC method to respond T times, generate T answers, and record the number of correct answers as ,but Corresponding to .

[0040] This plan defines the restatement It is an optimal restatement if and only if it satisfies the following conditions:

[0041]

[0042] Based on the above definition, samples with the best repetition are selected from the training set as training samples for the decision module.

[0043] 5.2 Obtain input features and labels: Perform steps 1 to 4 on the above-screened training samples in sequence, and obtain the scoring results from the multi-index scoring module , get the integral results from the pairwise comparison module , taking the above two as the input features of the model; taking the index of the best repetition as the label, it can be expressed as: , used to train the decision module.

[0044] 5.3 Model selection: The decision module of the scheme uses the XGBoost algorithm to implement the classifier. Scoring results and The essence of this model is a classification model. Given that the model's input feature dimensions are large, the gradient boosting tree (GBDT) algorithm is used to complete the classification task, which can effectively handle high-dimensional data and complex feature relationships. Furthermore, XGBoost is an implementation and optimization of the GBDT algorithm, which introduces regularization terms and uses a more complex tree structure to improve the performance and training speed of the model. Therefore, this solution selects XGBoost as the final model and trains the decision module.

[0045] 5.4 Optimize input features: The difference between the multi-index scores between repetitions and the difference between the integrals of the pairwise comparisons are used as the input features of the model, which reduces the feature dimension while increasing the interpretability of the feature. and As input features, the feature dimension is effectively reduced to .

[0046] The step six specifically includes:

[0047] The trained decision module can be obtained from step five, and the score of the multi-index scoring module in step three is After feature optimization, it can be transformed into , the integral of the pairwise comparison module in step 4 After feature optimization, it can also be transformed into , input the decision module trained in step 5, and get the prediction with the best repetition .

[0048] The step seven specifically includes:

[0049] Use the best selected paraphrase Combined with the original question q as a hint, the zero-shot thinking chain (Zero-shot CoT) combined with the self-consistency check (SC) is input to the LLM to get the answer to the question. Indicates a specific prompt: .

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] This paper proposes a mathematical question-answering automatic evaluation decision-making scheme based on a large language model, which provides a general evaluation framework for the quality evaluation of mathematical question retelling using LLM, with good scalability and adaptability. The scheme integrates two modules, multi-index scoring and pairwise comparison, taking into account comprehensiveness and comprehensiveness, and trains a decision model on this basis, which can make full use of the advantages of both, effectively screen out the best retelling, and thus improve the quality of LLM's response to the original question.

[0052] Experimental results show that this method can effectively deepen the language model's understanding of the original problem: on the GSM8K test set, the accuracy of GPT-3.5 combined with this method reached 86.81%, which is 2.28% higher than the Zero-shot CoT+SC benchmark. Among the methods that use GPT-3.5 as a base model, this solution is at the leading level. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A flowchart of a large language model mathematical reasoning method based on repetition quality optimization of the present invention;

[0055] Figure 2 is a schematic diagram of a multi-index scoring module of the present invention;

[0056] Figure 3 is a schematic diagram of a pairwise comparison module of the present invention;

[0057] Figure 4 is a schematic diagram of a decision module of the present invention;

[0058] Figure 5 Prompt graph used when generating retelling and generating answers for the present invention;

[0059] Figure 6 This is a prompt diagram used in the multi-index scoring and pairwise comparison of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] like Figure 1 As shown, the specific steps of a large language model mathematical reasoning method based on retelling quality optimization are as follows:

[0062] Step 1: Select a math problem, construct a prompt, and generate multiple paraphrases using LLM.

[0063] Step 2: Set six evaluation indicators, including coherence, consistency, fluency, relevance, matching and logic, for retelling quality assessment; set different prompts for each indicator through experiments to more effectively exert the evaluation performance of each indicator.

[0064] Step 3: Combine the six indicators in step 2 and their corresponding prompts, use LLM to perform multi-indicator scoring on each repetition, and use the probability mean of the output token as the score to ensure its discrimination. The schematic diagram of the multi-indicator scoring module is shown below: Figure 2 shown.

[0065] Step 4: Use LLM to compare all repetitions in pairs, and obtain the relative relationship between repetitions by accumulating scores through multiple comparisons; for this purpose, the results of the pairwise comparisons should satisfy the partial order relationship to ensure that the relative rankings between repetitions are clear. Figure 3 shown.

[0066] Step 5: Train a classifier as a decision model, which includes the following process: preprocess the training set and select the sample with the best repetition; use the above sample to execute steps 1 to 4 to obtain the input features and labels of the model; select the gradient boosting tree algorithm to implement the classification task, and use XGBoost to train the classifier to achieve the final decision. The decision module is as follows: Figure 4 shown.

[0067] Step 6: Input the scores of the multi-index scoring module in step 3 and the integral of the pairwise comparison module in step 4 into the trained decision module to obtain the decision result, which is selected as the best repetition.

[0068] Step 7: Use the best selected restatement combined with the original question as the prompt, and input it into the LLM in the form of zero-shot thinking chain (Zero-shot CoT) combined with self-consistency check (SC) to obtain the answer to the question.

[0069] The specific implementation process of step 1 is as follows:

[0070] Pick a math problem q from the dataset and construct a prompt for that problem Specific tips such as Figure 5 As shown, use LLM to generate n paraphrases , the generated i-th retelling can be expressed as:

[0071]

[0072] The specific implementation process of step 2 is as follows:

[0073] The present invention refers to the evaluation method of text summarization tasks and sets four indicators, namely coherence (Coh), consistency (Con), fluency (Flu) and relevance (Rel), as evaluation indicators of retelling quality. Considering the characteristics of mathematical problems, this solution adds two additional evaluation indicators that are more in line with mathematical tasks: matching (Mat) and logic (Log).

[0074] Recent related research has explored the impact of LLM evaluation prompts on text summarization tasks, pointing out that the evaluation quality of LLM under different prompts fluctuates. Based on this, this paper proposes a standard prompt (Standard), which includes the current task goal, constraints, original mathematical problems, retellings to be evaluated, and output templates; in addition, evaluation criteria (Criteria) and evaluation steps (Steps) are designed for the six evaluation indicators. The evaluation steps follow G-Eval and are automatically generated by GPT-4. The prompt details are as follows: Figure 6 shown.

[0075] On this basis, this scheme proposes four prompting methods, namely: not using this indicator, using standard prompt (Standard), standard prompt + evaluation criteria (Standard+Criteria), standard prompt + evaluation criteria + evaluation steps (Standard+Criteria+Steps); these four prompting methods are applied to the six evaluation indicators respectively, and their impact on the accuracy of the data set is observed to determine the best prompting method for each indicator.

[0076] For different data sets of math question-answering tasks, the prompting methods for each indicator may be different, and the specific selection should be determined based on the data set through experiments. Taking the GSM8K data set as an example, the three indicators of coherence, matching and logic use the prompting method of Standard+Criteria+Steps, the consistency indicator uses Standard+Criteria, and the fluency indicator only uses the Standard prompting method, and abandons the use of the correlation indicator; therefore, the prompting methods corresponding to each evaluation indicator can be expressed as follows:

[0077]

[0078] The specific implementation process of step three is as follows:

[0079] Combined with the six indicators in step 2 and their corresponding prompts, LLM is used to score each retelling with multiple indicators, ranging from 1 to 5 points. A single score under this indicator can be expressed as .

[0080] Directly using the model output as the scoring result may result in a small degree of discrimination in the scoring. This solution uses the probability mean of the model output token as the single indicator score of the LLM for the current retelling. Specifically, the language model output score is The probability of , then repeat The final score of indicator Coh can be expressed as:

[0081]

[0082] For LLMs such as GPT-4 that do not provide token output probabilities, we can use the average of multiple scoring to simulate. Taking the score of indicator Coh as an example, LLM is used to perform K scorings to obtain the score set ,in and , represents the output score of the model for the kth time. The number of times the output score is t in the score set S is , you can use To simulate the probability that the LLM output score is t Therefore, retell The final score of indicator Coh can be expressed as:

[0083]

[0084] The specific implementation process of step 4 is as follows:

[0085] Use LLM to compare all paraphrases in pairs, using a circular comparison method, that is, each paraphrase needs to be compared with other alternative paraphrases once, so n paraphrases will form a total of Specific tips for pairwise comparisons like Figure 6 Shown, repeat and retelling The comparison can be expressed as .

[0086] Since there is a position bias in paired comparisons, that is, LLM will favor the repetition that comes first in the prompt, this scheme will compare the comparison groups 5 times in sequence, swap the order of the two, and then compare them 5 times to reduce the impact of position bias on the results. For each comparison, LLM will choose the repetition with better overall performance, and the repetition will receive 2 points; if the two are evenly matched, each will receive 1 point. and The integrals obtained after comparison are expressed as and .

[0087] After a round of effective pairwise comparison, a clear partial order relationship should be formed between the repetitions, that is, the following constraints are met, and the superiority and inferiority relationship between the repetitions can form a directed acyclic Hasse graph.

[0088]

[0089] If the above constraints are not met between the repetitions, the comparison should be considered self-contradictory, and the contradictory parts should be re-compared. Specifically, the superiority relationship between the repetitions is converted into a directed graph. The parts that do not meet the above constraints will form a loop in the graph. Use topological sorting to remove all points with in-degree 0 and out-degree 0. The remaining parts that cannot be removed are all the repetitions that need to be re-compared.

[0090] The specific implementation process of step 5 is as follows:

[0091] The decision module is an n-choose-one decision classification model. It evaluates and makes decisions based on the output of the multi-index scoring module and the results of the pairwise comparison module, and selects one of the n retellings as the best retelling. It is essentially a classification model, and the training process is as follows:

[0092] 5.1 Sample selection. Use LLM to generate n paraphrases for each sample in the training set. , respectively, combined with the original question, using the Zero-shot CoT+SC method to respond T times, generate T answers, and record the number of correct answers as ,but Corresponding to .

[0093] This plan defines the restatement It is an optimal restatement if and only if it satisfies the following conditions:

[0094]

[0095] Based on the above definition, samples with the best repetition are selected from the training set as training samples for the decision module.

[0096] 5.2 Obtain input features and labels. Perform steps 1 to 4 on the above-screened training samples in sequence and obtain the scoring results from the multi-index scoring module. , get the integral results from the pairwise comparison module , taking the above two as the input features of the model; taking the index of the best repetition as the label, it can be expressed as: , used to train the decision module.

[0097] 5.3 Model selection. The decision module of this solution uses the XGBoost algorithm to implement the classifier. Scoring results and The essence of this model is a classification model. Given that the model's input feature dimensions are large, the gradient boosting tree (GBDT) algorithm is used to complete the classification task, which can effectively handle high-dimensional data and complex feature relationships. Furthermore, XGBoost is an implementation and optimization of the GBDT algorithm, which introduces regularization terms and uses a more complex tree structure to improve the performance and training speed of the model. Therefore, this solution selects XGBoost as the final model and trains the decision module.

[0098] 5.4 Optimize input features. In terms of input features, the difference between the multi-index scores between repetitions and the difference between the integrals of the pairwise comparisons are used as the input features of the model, which reduces the feature dimension while increasing the interpretability of the feature. and As input features, the feature dimension is reduced to .

[0099] The specific implementation process of step six is ​​as follows:

[0100] The trained decision module can be obtained from step five, and the score of the multi-index scoring module in step three is After feature optimization, it can be transformed into , the integral of the pairwise comparison module in step 4 After feature optimization, it can also be transformed into , input the decision module trained in step 5, and get the prediction with the best repetition .

[0101] The specific implementation process of step seven is as follows:

[0102] Use the best selected paraphrase Combined with the original question q as a hint, the zero-shot thinking chain (Zero-shot CoT) combined with the self-consistency check (SC) is input to the LLM to get the answer to the question. The specific hint is like Figure 5 shown.

[0103]

[0104] Experimental results:

[0105] The present invention uses GPT-3.5 as the basic model and tests it on the mathematical problem dataset GSM8K. With Zero-shot CoT+SC as the benchmark solution, the experimental accuracy of the GSM8K test set is 84.53%. On this basis, 3 repetitions are generated for each question, and combined with the mathematical question-answering automatic evaluation decision-making solution proposed in the present invention, the best repetition is screened out and then the answer is generated using Zero-shot CoT+SC. The accuracy of GPT-3.5 on the test set is improved to 86.81%, an increase of 2.28% compared to before.

[0106] Experiments show that generating paraphrases based on math problems and selecting the best paraphrases through multi-index scoring + pairwise comparison + decision-making module can effectively improve the performance of LLM on math problem datasets.

[0107] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. A mathematical reasoning method for large language models based on repetition quality optimization, Features: The following steps are involved: Step 1: Select a math problem from the dataset, construct a prompt, and generate multiple paraphrases using LLM; Step 2: Set six evaluation indicators, namely, coherence, consistency, fluency, relevance, matching and logic, for retelling quality evaluation; set different prompts for each indicator through experiments to give full play to the evaluation performance of each indicator; Step 3: Combine the six indicators in step 2 and their corresponding prompts, use LLM to perform multi-indicator scoring on each repetition, and use the average probability of the output token as the score; Step 4: Use LLM to compare all repetitions in pairs, and obtain the relative relationship between repetitions by accumulating scores through multiple comparisons; The step 4 specifically includes: Use LLM to compare all paraphrases in pairs, using a circular comparison method, that is, each paraphrase needs to be compared with other alternative paraphrases once, so n paraphrases will form a total of A comparison group, repeat r i and retell j The comparison can be expressed as LLM (prompt CMP ,q,r i ,r j ), where prompt CMP Specific prompts to indicate pairwise comparisons; When comparing the comparison groups, they will be compared 5 times in sequence, and then the order of the two will be exchanged and compared 5 times. For each comparison, LLM will choose the repetition with better overall performance, which will receive 2 points. If the two repetitions are evenly matched, each will receive 1 point and record r i With r j The scores obtained after comparison are and After a round of effective pairwise comparison, a clear partial order relationship should be formed between the repetitions, that is, the following constraints are met, and the superiority relationship between the repetitions can form a directed acyclic Hasse graph; If the above constraints are not met between the repetitions, the comparison should be considered self-contradictory. The contradictory parts should be re-compared, and the superiority relationship between the repetitions should be converted into a directed graph. The parts that do not meet the above constraints will form a loop in the graph. Use topological sorting to remove all points with in-degree 0 and out-degree 0. The remaining parts that cannot be removed are all the repetitions that need to be re-compared. Step 5: Train a classifier as a decision model, which includes the following processes: preprocess the training set and select the samples with the best repetition; use the above samples to execute steps 1 to 4 to obtain the input features and labels of the model; select the gradient boosting tree algorithm to implement the classification task, and use XGBoost to train the classifier to achieve the final decision; Step 6: Input the scores after multi-index scoring in step 3 and the scores after pairwise comparison in step 4 into the trained decision module to obtain the decision result, which is selected as the best repetition; Step 7: Use the best selected restatement combined with the original question as the prompt, and input it into GPT-3.5 in the form of zero-sample thinking chain combined with self-consistency check to obtain the answer to the question.

2. According to claim 1, a large language model mathematical reasoning method based on repetition quality optimization, Features: The step 1 specifically includes: selecting a math problem q from the data set, and constructing a prompt for the problem rephrase , use LLM to generate n repetitions R = {r 1 ,r 2 ,…,r n }, the generated i-th retelling can be expressed as: r i =LLM(prompt rephrase ,q)。 3. According to claim 2, a large language model mathematical reasoning method based on repetition quality optimization, Features: The evaluation indicators in step 2 are: referring to the evaluation method of the text summarization task, four general indicators are set: coherence, consistency, fluency and relevance, and matching and logic indicators are added for mathematical problems; The prompt modes are: not using the indicator, using standard prompts, standard prompts + evaluation standards, standard prompts + evaluation standards + evaluation steps, where the standard prompts include the current task objectives, constraints, the original mathematical problem, the restatement to be evaluated, and the output template; The evaluation step is automatically generated by GPT-4; The four prompting methods are applied to the six evaluation indicators respectively, and their effects on the accuracy of the data set are observed to determine the best prompting method for each indicator. The prompting methods corresponding to each evaluation indicator can be expressed as follows:

4. According to claim 3, a large language model mathematical reasoning method based on repetition quality optimization, Features: The step three specifically includes: Combined with the six indicators and their corresponding prompts in step 2, LLM was used to score each retelling with multiple indicators, ranging from 1 to 5 points. i A single score under the coherence index can be expressed as The probability mean of the model output token is used as the single indicator score of the LLM for the current retelling, and the probability of the language model output score t∈{1,2,3,4,5} is recorded as P(t), then the retelling r i The final score of the coherence index can be expressed as: For LLMs where GPT-4 does not provide token output probabilities, multiple scoring is averaged to simulate the repetition of r i In the scoring of the coherence index, LLM is used to score K times to obtain the scoring set S = {s 1 ,s 2 ,…,s K }, where s k ∈{1,2,3,4,5} and represents the output score of the model for the kth time, and the number of times the output score is t in the score set S is num t ,use To simulate the probability P(t) of LLM outputting a score of t, repeat r i The final score of the coherence index can be expressed as:

5. According to claim 1, a large language model mathematical reasoning method based on repetition quality optimization, Features: The step five specifically includes: The decision module is an n-choose-one decision classification model. It evaluates and makes decisions based on the output of the multi-index scoring module and the results of the pairwise comparison module, and selects one of the n retellings as the best retelling. The training process is as follows: 5.1 Sample selection: Use LLM to generate n repetitions R = {r 1 ,r 2 ,…,r n }, respectively, combined with the original question, use the zero-sample thinking chain combined with the self-consistency check method to respond T times, generate T answers, and record the number of correct answers as Cnt correct , then {r 1 ,r 2 ,…,r n } respectively correspond to Definition paraphrase i It is an optimal restatement if and only if it satisfies the following conditions: And j≠i, Select the best retelling samples from the training set as training samples for the decision module; 5.2 Obtain input features and labels: Perform steps 1 to 4 on the above-screened training samples in sequence, and obtain the scoring results from the multi-index scoring module Get the integration results from the pairwise comparison module The above two are used as the input features of the model; the index of the best repetition is used as the label, which can be expressed as: Used to train decision modules; 5.3 Model selection: The decision module of the scheme uses the XGBoost algorithm to implement the classifier, taking 6×n scoring results and n×(n-1) integral results as input features, and selecting one of the n paraphrases as the best paraphrase; 5.4 Optimize input features: The difference between the multi-index scores between repetitions and the difference between the scores of the pairwise comparisons are used as the input features of the model. and As input features, the feature dimension is effectively reduced to 6. The large language model mathematical reasoning method based on repetition quality optimization according to claim 1, Features: The step six specifically includes: Step 3: Scoring of the multi-index scoring module After feature optimization, it can be transformed into Step 4: Pairwise comparison module integration After feature optimization, it can also be transformed into Input the decision module trained in step 5 to obtain the prediction r with the best repetition pred .

7. The large language model mathematical reasoning method based on repetition quality optimization according to claim 1, Features: The step seven specifically includes: Use the best selected paraphrase pred Combine the original question q as a prompt, input it into LLM in the form of zero-sample thinking chain combined with self-consistency check to get the answer to the question, and use prompt answer Indicates a specific prompt, and gets: Answer = LLM (prompt answer ,q,r pred ).