A method and system for enhancing the logical reasoning ability of large language models

Abstract

This invention relates to a method and system for enhancing the logical reasoning ability of large language models, belonging to the field of artificial intelligence technology. It solves the problem of low accuracy in single-stage fine-tuning training of large models in existing technologies. The method includes: for each training sample in the training set, converting the conclusion and each premise of the training sample into symbolic representations to obtain the symbolic language of the training sample; for each training sample, calculating the complexity of the training sample based on the logical form and reasoning difficulty of the symbolic language; dividing the training set into training subsets of different complexity levels based on the complexity of each training sample; and training the large language model in stages based on the training subsets in order of increasing complexity to obtain a trained large language model. This improves the reasoning accuracy of the large language model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for enhancing the logical reasoning ability of large language models. Background Technology

[0002] Large language models have demonstrated superior performance in a wide range of natural language processing tasks. However, recent research has revealed that their logical reasoning capabilities still face significant challenges. Logical reasoning tasks require large models to perform complex deductive, inductive, or abductive reasoning based on a set of premises and logical rules. Specifically, the model must determine whether a given conclusion can be derived from the provided premises according to logical rules. For example, given the premises "metals conduct electricity; insulators do not conduct electricity; if something is made of iron, it is a metal; nails are made of iron," the large model must determine the correctness of the statement "nails do not conduct electricity," classifying it as "true, false, or indeterminate." To answer this question correctly, the large model needs to construct a logical reasoning chain: "nails → made of iron → metal → conduct electricity," thus inferring that the statement "nails do not conduct electricity" is "false." Surprisingly, the LLaMA-13B model only achieved 33.63% accuracy on the FOLIO dataset with 8-shot hints, only slightly better than random guessing (33.33%), indicating that pre-trained large models still face challenges in logical reasoning tasks.

[0003] Methods for enhancing the logical reasoning capabilities of large-scale models can be broadly categorized into three types: those based on external solvers, those based on cue design, and those based on fine-tuning training. The main idea behind external solver-based methods is to prompt the large model to parse the natural language question into symbolic language expressions, then call the corresponding logic solver for reasoning and solve the problem, and finally generate the final answer based on ensemble algorithms such as majority voting. Cue design-based methods have two main approaches. The first type explicitly models the logical chain when answering the question, such as CoT (Chain-of-Thought), ToT (Tree of Thoughts), and DoT (Diagram of Thought). The second type guides the large model step-by-step through symbolic natural language representation, task decomposition, reasoning steps, and result verification through carefully designed cue prompts to accurately answer logical questions. The limitations of large models in reasoning ability can be attributed to the lack of high-quality reasoning samples (especially logical multi-step deductions or proofs) in the pre-training corpus. Fine-tuning training methods mainly construct synthetic corpora containing deductive proofs and natural language examples that explicitly demonstrate the logical reasoning process in various ways, or collect a large number of reasoning steps to enhance the training data, thereby fine-tuning the training of large models to improve the accuracy and interpretability of solutions.

[0004] However, methods based on external solvers are easily affected by translation errors from natural language to logical symbolic language, and are prone to search explosion problems for solving complex logical reasoning problems. Hint-based methods rely on the initial logical reasoning performance of pre-trained large models, but experience shows that hint-based models like LLaMA-13B only slightly outperform random guessing in logical question answering tasks, due to the poor initial logical reasoning performance of these pre-trained large models.

[0005] Fine-tuning-based methods effectively avoid the aforementioned problems: on the one hand, the fine-tuned model can process natural language input end-to-end, avoiding errors caused by translation; on the other hand, fine-tuning training reduces the dependence on the initial logical reasoning performance of the pre-trained large language model. However, existing fine-tuning-based methods mainly adopt a single-stage fine-tuning strategy, inputting the relevant fine-tuning dataset into the pre-trained large model indiscriminately for traditional supervised training, without considering the difficulty differences between different logical question-and-answer samples. Ignoring the differences in the complexity of different logical question-and-answer samples and their implicit logical rules not only limits training efficiency but also results in very limited training improvement, especially for complex logical question-and-answer samples with high difficulty, where single-stage fine-tuning is difficult to improve the answer accuracy of the large model. Summary of the Invention

[0006] Based on the above analysis, the embodiments of the present invention aim to provide a method and system for enhancing the logical reasoning ability of large language models, in order to solve the problem of low answer accuracy of existing large language models trained by single-stage fine-tuning.

[0007] On one hand, embodiments of the present invention provide a method for enhancing the logical reasoning ability of large language models, comprising the following steps:

[0008] For each training sample in the training set, the conclusion and each premise of the training sample are converted into symbolic representations to obtain the symbolic language of the training sample.

[0009] For each training sample, the complexity of the training sample is calculated based on the logical form of the symbolic language and the reasoning difficulty.

[0010] The training set is divided into training subsets of different complexity levels based on the complexity of each training sample. The large language model is then trained in stages based on the training subsets in order of increasing complexity to obtain a well-trained large language model.

[0011] Based on the above method, the conclusions and each premise of the training sample are converted into symbolic representations in the following way, thus obtaining the symbolic language of the training sample:

[0012] For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols;

[0013] Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion;

[0014] The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

[0015] Based on further improvements to the above method, for each training sample, the complexity of the training sample is calculated based on the logical form of the symbolic language and the reasoning difficulty, including:

[0016] Convert the symbolic language of the training sample into disjunctive paradigm;

[0017] The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm.

[0018] The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm.

[0019] The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity.

[0020] Based on the above method, a further improvement is made, and the symbolic logic complexity of the training sample is calculated using the following formula based on the disjunctive paradigm:

[0021] ;

[0022] in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. The symbolic logic complexity of the sample.

[0023] Based on a further improvement to the above method, the logical reasoning complexity of the training sample is calculated using the disjunctive paradigm in the following manner:

[0024] ;

[0025] in, This represents the logical reasoning complexity of the training samples. This represents the probability that the conclusion of the training sample is true, calculated based on the disjunctive normal form. It indicates that it is true.

[0026] Based on the above method, a further improvement is made, and the probability that the conclusion of the training sample is true is calculated using the following formula based on the disjunctive normal form:

[0027] ;

[0028] in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. Indicates the given model parameters At that time, the first in the disjunction paradigm of the large language model decision The first disjunction in the disjunction The probability that the sum of its components is true. It indicates that it is true.

[0029] Based on a further improvement of the above method, the complexity of the training sample is obtained using the following method, based on the symbolic logic form complexity and the logical reasoning complexity:

[0030] The symbolic logic complexity and logical reasoning complexity of each training sample are normalized respectively.

[0031] Based on formula The complexity of obtaining this training sample is, where, This represents the complexity of the symbolic logic form after normalization. This represents the complexity of logical reasoning after normalization. This represents the weighting coefficient.

[0032] On the other hand, embodiments of the present invention provide a system for enhancing the logical reasoning capabilities of large language models, comprising:

[0033] The conversion module is used to convert the conclusion and each premise of each training sample in the training set into a symbolic representation, thereby obtaining the symbolic language of the training sample.

[0034] The complexity calculation module is used to calculate the complexity of each training sample based on the logical form of the symbolic language and the reasoning difficulty.

[0035] The model training module is used to divide the training set into training subsets of different complexity levels based on the complexity of each training sample; and to train the large language model in stages based on the training subsets in order of increasing complexity level to obtain the trained large language model.

[0036] Based on further improvements to the above system, the conclusions and each premise of the training sample are converted into symbolic representations in the following way, resulting in the symbolic language of the training sample:

[0037] For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols;

[0038] Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion;

[0039] The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

[0040] Based on further improvements to the above system, for each training sample, the complexity of the training sample is calculated based on the logical form of the symbolic language and the reasoning difficulty, including:

[0041] Convert the symbolic language of the training sample into disjunctive paradigm;

[0042] The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm.

[0043] The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm.

[0044] The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity.

[0045] Compared with existing technologies, the method for enhancing the logical reasoning ability of large language models provided in this embodiment converts natural language samples into logical symbolic language, thereby obtaining the complexity of each sample based on the logical form and reasoning difficulty of the symbolic language. The training set samples are stratified according to the complexity of the samples, and the large language model is fine-tuned and trained in stages from easy to difficult according to the complexity level from low to high, thereby enhancing the logical reasoning ability of the large language model and improving the reasoning accuracy of the large language model.

[0046] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0047] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0048] Figure 1 A flowchart illustrating a method for enhancing the logical reasoning capabilities of a large language model according to an embodiment of the present invention;

[0049] Figure 2 This is a block diagram of a system for enhancing the logical reasoning capabilities of large language models, as described in an embodiment of the present invention. Detailed Implementation

[0050] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0051] Large language models have made significant progress not only in natural language processing tasks but also demonstrated excellent performance in applications such as intelligent customer service. However, large language models still have significant shortcomings in complex logical reasoning. As one of the core elements of human intelligence, logical reasoning plays a crucial role in various tasks such as natural language understanding, intelligent question answering, and knowledge graph construction. Existing methods for improving the logical reasoning ability of large models based on solvers and hints are prone to problems such as being greatly affected by translation errors, solver search explosion, and dependence on the initial performance of pre-trained large language models. Fine-tuning training methods can effectively avoid these problems. However, existing fine-tuning methods ignore the difficulty differences between different logical samples, limiting training efficiency and improvement effects. Therefore, this invention proposes a novel staged fine-tuning strategy for propositional logic question answering tasks in logical reasoning. Inspired by the principles of curriculum learning, it gradually trains large models from easy to difficult to enhance their logical reasoning ability.

[0052] Based on this, a specific embodiment of the present invention discloses a method for enhancing the logical reasoning ability of large language models, such as... Figure 1 As shown, it includes:

[0053] S1. For each training sample in the training set, convert the conclusion and each premise of the training sample into symbolic representations to obtain the symbolic language of the training sample.

[0054] S2. For each training sample, calculate the complexity of the training sample based on the logical form of the symbolic language and the reasoning difficulty.

[0055] S3. The training set is divided into training subsets of different complexity levels based on the complexity of each training sample. The large language model is trained in stages based on the training subsets in order of increasing complexity level to obtain a trained large language model.

[0056] Compared with existing technologies, the method for enhancing the logical reasoning ability of large language models provided in this embodiment converts natural language samples into logical symbolic language, thereby obtaining the complexity of each sample based on the logical form and reasoning difficulty of the symbolic language. The training set samples are stratified according to the complexity of the samples, and the large language model is fine-tuned and trained in stages from easy to difficult according to the complexity level from low to high, thereby enhancing the logical reasoning ability of the large language model and improving the reasoning accuracy of the large language model.

[0057] In the evaluation of logical reasoning in large language models, logical reasoning question-answering tasks are typically described in natural language, including a set of propositions or constraints related to certain objects as the "premises" of the reasoning, and a corresponding statement about these objects as the "conclusion" of the reasoning. The goal of the task is to determine whether the conclusion can be derived from the given set of premises. If the conclusion can be derived from the premises, the conclusion is output as "true"; if the conclusion is derived as "false" from the premises, the conclusion is output as "false"; if the truth value of the conclusion cannot be determined from the premises, "cannot determine / don't know" is output. Below is an example of a logical question-answering task.

[0058] First, the large model is given the premises: "Premise 1: If Li Ming likes reading, then Li Ming will acquire knowledge," "Premise 2: If Li Ming acquires knowledge, then Li Ming is smarter than Zhang San," and "Premise 3: Li Ming likes reading." The next question is, "Is the following assertion true: Zhang San is smarter than Li Ming?" That is, the conclusion is "Zhang San is smarter than Li Ming." The correct answer is "false," because from the three premises and the inference rules, we can deduce "Li Ming is smarter than Zhang San," which contradicts the assertion. Therefore, the truth value of this conclusion is "false."

[0059] The sample corresponding to this example includes three premises, and the question is whether the conclusion "Zhang San is smarter than Li Ming" is true.

[0060] For a given sample, each premise and conclusion may consist of multiple atomic propositions, therefore its complexity cannot be simply measured by the number of premises. To assess the difficulty of propositional logic question answering tasks, we need to convert natural language into symbolic language.

[0061] Specifically, for each training sample in the training set, the conclusion and each premise of the training sample are converted into symbolic representations in the following way to obtain the symbolic language of the training sample:

[0062] For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols;

[0063] Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion;

[0064] The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

[0065] In practice, atomic propositions are extracted from the premise set of the training samples and assigned symbolic representations, thus forming a set of symbolic expressions. With natural language sentence sets mapping relationship Furthermore, based on the mapping It can output the original premises and conclusions of natural language representations as corresponding symbolic language representations.

[0066] In logic, a proposition is a statement that can be judged as true or false. Propositional logic uses symbolic representation to depict propositions in natural language and the logical relationships between them. Each proposition is recursively constructed from atomic propositions and logical connectives. An atomic proposition is a statement that cannot be further divided into sub-propositions, usually represented by letters, such as... Logical connectives can express logical relationships between propositions. The table below lists these logical connectives and their corresponding meanings in natural language.

[0067] Table 1 Logical Connectives

[0068]

[0069] These logical connectives cover the main relationships between propositions or sentences, among which `+` is a unary connective, while the others are binary connectives. To determine which inferences are valid, logic introduces truth-value semantics, assigning a truth value to each atomic proposition through a truth-value function. The truth-value function is defined as follows: ,in Denotes the set of all atomic propositions. Indicates the range of true values. and These represent the truth values ​​of propositions as "true" and "false," respectively. Based on this, the truth values ​​of compound propositions connected by propositional connectives can be summarized in the following table.

[0070] Table 2 Truth Table of Logical Connectives

[0071]

[0072] Let's continue with the example sample mentioned above. Based on the above definitions,

[0073] The conclusion contains the atomic proposition "Zhang San is smarter than Li Ming," which is used... To express.

[0074] Premise 1 contains two atomic propositions: "Li Ming likes reading" and "Li Ming will acquire knowledge." We can use... To express the sentence "Li Ming likes reading", use... The sentence "Li Ming will acquire knowledge" uses logical connectors. Connecting these two atomic propositions yields the symbolic representation of the premise: ".

[0075] Premise 2 contains two atomic propositions: "Li Ming acquires knowledge" and "Li Ming is smarter than Zhang San." The atomic proposition "Li Ming acquires knowledge" has already been expressed in Premise 1. The atomic proposition "Li Ming is smarter than Zhang San" is the negation of the atomic proposition "Zhang San is smarter than Li Ming" in the conclusion. This indicates the use of logical words. Connecting these two atomic propositions yields the symbolic representation of the premise: ".

[0076] Premise 3 contains an atomic proposition, "Li Ming likes reading," which is symbolically represented as "". ".

[0077] Therefore, logical samples can be written in symbolic language, represented as follows: ,in, This indicates the sample One premise, This represents the conclusion to be inferred. Furthermore, the complexity of the sample can be measured using symbolic language. The symbolic language representation of this example sample is { , , , }

[0078] Specifically, for each training sample, the complexity of the sample is calculated based on the logical form of the symbolic language and the difficulty of reasoning, including:

[0079] Convert the symbolic language of the training sample into disjunctive paradigm;

[0080] The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm.

[0081] The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm.

[0082] The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity.

[0083] The reasoning task of the above logic question-answering sample is equivalent to determining the truth value of the following formula:

[0084] ;

[0085] in, Indicates the number of premises. Indicates the first One premise.

[0086] Although the sample has been represented in symbolic form, due to the equivalence of logical formulas, the same semantic content can be expressed using different syntactic structures. For example, formulas Logically equivalent, since both only exist in For true and It is not true when it is false. Similarly, from the perspective of truth value, more complex examples include... They are also logically equivalent. Obviously, the forms of equivalent formulas may differ significantly. Therefore, the complexity of symbolic languages ​​cannot be reliably measured directly by translating the original formulas. Instead, all original formulas must be converted into a unified and comparable form.

[0087] In propositional logic, canonical forms such as disjunctive normal form (DNF) have well-defined and consistent syntactic structures, as defined below.

[0088] Demographic NF is the disjunction of conjunctions, where each disjunction consists of an atomic proposition or its negation. The formal DNF representation of a propositional logic is as follows: :

[0089] ;

[0090] in, This indicates the number of disjunctive branches in the disjunctive normal form. For the first The number of conjuncts in a disjunct. Indicates the first The first dissociation branch A combined withdrawal, For atomic propositions or their negations, Indicates the disjunction symbol, Represents the conjunction symbol.

[0091] The disjunctive normal form corresponding to the aforementioned example sample is: .

[0092] The number of disjunctions is 4, and the first disjunction is... The disjunction has two conjuncts. The first conjunct is... The second conjunct is .

[0093] The classic conclusion in logic, the DNF theorem, shows that every propositional logic formula can be equivalently converted into DNF, which provides a basis for defining a unified measure of the symbolic language complexity of propositional logic formulas.

[0094] In practice, the SymPy code library can be used to convert the logical symbolic language into disjunctive normal form.

[0095] Therefore, in practice, the symbolic language of the training samples is first converted into the disjunctive paradigm, and then the symbolic logic complexity and logical reasoning complexity of the training samples are calculated based on the disjunctive paradigm to obtain the complexity of the training samples.

[0096] The measurement of difficulty in logical question-answering tasks has not been previously studied. Based on the characteristics of logical question-answering tasks, this invention defines the difficulty of solving such tasks as the complexity of determining the truth value of propositions. This complexity depends on two factors, considering both the formal complexity and inference complexity of the logical sample: (a) the formal complexity of symbolic logic based on partial order relations; and (b) the logical inference complexity based on the uncertainty of inferring the truth value of atomic propositions from a large model. This invention combines the formal complexity and inference complexity of propositional logic. The former reflects the complexity of the logical form length of the logical question-answering sample, while the latter reflects the difficulty of the pre-trained large model in determining the truth value of the proposition. This approach aligns with the principle in logic that "the validity of a conclusion depends on the truth of its premises and the validity of the argumentation process."

[0097] Specifically, the symbolic logic complexity of the training sample is calculated using the following formula based on the disjunctive normal form:

[0098] ;

[0099] in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. The symbolic logic complexity of the training samples.

[0100] During implementation, the total length of the DNF formula will be... As a measure of its complexity, the basis for this measure is detailed below.

[0101] When measuring formal complexity, since all formulas have been converted to DNF form, we can ignore the semantic differences in the natural language corresponding to the atomic propositions and only focus on the differences in the number of conjunctions and disjunctions of the atomic propositions. Each DNF formula... The formal complexity is entirely determined by the integer sequence Decision. Without loss of generality, assuming Otherwise, the DNF formulas are reordered using the commutative law. Given two DNF formulas... and The corresponding sequence is and We define a binary relation between these two sequences. as follows:

[0102] ;

[0103] In fact, This indicates that the DNF formula corresponding to the former is formally more complex than the latter, because it represents the disjunction of more conjunction clauses, and the conjunction clauses contain more atomic propositions. If and If neither holds true, then the two formulas are considered incomparable in difficulty. Let... Let be the set of integer sequences corresponding to all transformed DNF formulas in the dataset, then constitute The partial order relation on.

[0104] Our aim is to define a complexity metric function. , Let represent the set of positive real numbers, such that samples with strictly higher difficulty receive higher complexity scores. This requires the principle of partial order of difficulty, defined as follows:

[0105] Definition: For any ,have ,

[0106] This is called the complexity metric function. It satisfies the principle of partial order of difficulty.

[0107] We take the length of DNF as the formal complexity of the symbolic language, that is... It can be proven that it satisfies the above-mentioned difficulty partial order principle.

[0108] theorem: It satisfies the principle of partial order of difficulty.

[0109] Proof: If Then there is and , .

[0110] Therefore we have:

[0111] .

[0112] Therefore, the total length of the DNF formula is used as the symbolic logic complexity.

[0113] Specifically, the logical reasoning complexity of the training sample is calculated using the disjunctive paradigm in the following way:

[0114] ;

[0115] in, This represents the logical reasoning complexity of the training samples. This represents the probability that the conclusion of the training sample is true, calculated based on the disjunctive normal form. It indicates that it is true.

[0116] Specifically, the probability that the conclusion of this training sample is true is calculated using the following formula based on the disjunctive normal form:

[0117] ;

[0118] in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. Indicates the given model parameters At that time, the first in the disjunction paradigm of the large language model decision The first disjunction in the disjunction The probability that the sum of its components is true. It indicates that it is true.

[0119] Based on the converted DNF, let... The proposition given for the large model to be fine-tuned The probability of it being true. Since we have extracted the smallest unit of atomic propositions from the premises, following recent Neuro-Symbolic AI research, we make the following independence assumption:

[0120] Given the parameters of the large model, the atomic propositions are independent and formalized as follows: .in, Represents the parameters of the large model.

[0121] Note that we rely only on the independence between atomic propositions (not the independence between premises), which is a weaker assumption. Therefore, we can calculate the probability that the conclusion of a logical question-and-answer exercise is true:

[0122] .

[0123] Let's continue with the example sample above. If the probability of the atomic proposition "Li Ming likes reading" being true is 0.2, that is... The probability that the atom "Li Ming will acquire knowledge" is true is 0.4, that is... The probability of the atomic proposition "Zhang San is smarter than Li Ming" is 0.7. The probability that the logical question-and-answer conclusion of this sample is true is calculated as follows:

[0124] ;

[0125] in, Indicates false. It indicates that it is true.

[0126] In practice, the probability of an atomic proposition being true can be calculated by inputting the atomic proposition into a large model multiple times and counting the number of times the large model answers "true". For example, if the atomic proposition "Li Ming likes reading" is input into the large model 10 times, and the large model answers "true" only twice, then the probability of the atomic proposition being true is 0.2.

[0127] After calculating the probabilities, we define the logical reasoning complexity based on information entropy:

[0128] .

[0129] For example, in the previous example,

[0130] .

[0131] For a training sample of a logic question answering problem, we weight and fuse the two complexities—symbolic logic form complexity and logical reasoning complexity of the large language model—into an overall difficulty measure for the sample, which is then used for training set stratification.

[0132] The complexity of the training sample is obtained based on the symbolic logic formal complexity and the logical reasoning complexity, in the following ways:

[0133] The symbolic logic complexity and logical reasoning complexity of each training sample are normalized respectively.

[0134] Based on formula The complexity of obtaining this training sample is, where, This represents the complexity of the symbolic logic form after normalization. This represents the complexity of logical reasoning after normalization. This represents the weighting coefficient.

[0135] During implementation, the complexity of the symbolic logic form is first normalized.

[0136] For the The symbolic logic complexity of the training samples is... The normalized value is:

[0137] ;

[0138] in, This represents the minimum value of the symbolic logic complexity in the training set. This represents the maximum value of the symbolic logic complexity in the training set.

[0139] The normalized value of the logical reasoning complexity of the i-th training sample is obtained in a similar manner.

[0140] In practice, based on the complexity of the training samples, the training set used for fine-tuning the large language model is divided into multiple subsets, each corresponding to a different complexity level. The large language model is then trained sequentially using the corresponding training subsets in order of increasing complexity, resulting in a well-trained large language model.

[0141] To illustrate the effectiveness of this invention, 31,420 propositional logic question-and-answer samples from the publicly available dataset FLDx2 were selected as the training dataset in the experimental verification. We divided the training set according to difficulty levels... The three training subsets, with data sizes of 15786, 9226, and 6408 respectively, were obtained from three levels. These three subsets were used sequentially to fine-tune the large model in stages. Fine-tuning was performed using LLaMA-Factory and DeepSpeed's Zero3 algorithm for parallel full-parameter fine-tuning. All experiments used a learning rate of 5e-6 and one training epoch. The batch size per device was set to 8, the gradient accumulation step to 1, the weight decay to 0.1, and the warmup ratio to 0.1. A cosine annealing learning rate scheduler was used, following the standard fine-tuning paradigm for large language models.

[0142] The effectiveness of the proposed method can be verified by comparing the performance of traditional single-stage fine-tuning and the phased fine-tuning of this invention. All fine-tuning experiments were based on the open-source models LLaMA-3.1 8B-Instruct and Ministral-8B-Instruct-2410. As shown in Table 1, the experiments compared the test results of the one-time instruction fine-tuning strategy (LLaMA column, Ministral column) and the phased instruction fine-tuning strategy proposed in this invention (LLaMA-PFT column, Ministral-PFT column). Here, acc represents the accuracy of the model's response, while acc_norm is the accuracy after standardization processing such as removing extra spaces and ignoring case. exact_match represents the proportion of generated text that completely matches the label. RI (Relative Improvement) represents the relative improvement compared to the baseline achieved by one-time instruction fine-tuning.

[0143] Table 3. Comparison of baseline test results for phased instruction fine-tuning and one-time instruction fine-tuning of the present invention on different inference benchmark datasets.

[0144]

[0145] The proposed method in this invention, based on hierarchical logical reasoning complexity, achieves data hierarchical and phased training by combining the complexity of symbolic logic form with the logical reasoning complexity of large models. This significantly outperforms single-stage fine-tuning baselines on multiple logical reasoning benchmark datasets such as FOLIO and LogiQA 2.0, effectively improving the performance of large models in complex logical reasoning tasks. Furthermore, experiments show that the large models trained using this method exhibit good generalization ability, demonstrating significant performance improvements on benchmark datasets such as BIG-bench, MathQA, and SNLI.

[0146] One specific embodiment of the present invention discloses a system for enhancing the logical reasoning ability of large language models, such as... Figure 2 As shown, it includes:

[0147] The conversion module is used to convert the conclusion and each premise of each training sample in the training set into a symbolic representation, thereby obtaining the symbolic language of the training sample.

[0148] The complexity calculation module is used to calculate the complexity of each training sample based on the logical form of the symbolic language and the reasoning difficulty.

[0149] The model training module is used to divide the training set into training subsets of different complexity levels based on the complexity of each training sample; and to train the large language model in stages based on the training subsets in order of increasing complexity level to obtain the trained large language model.

[0150] Preferably, the conclusions and each premise of the training sample are converted into symbolic representations in the following manner to obtain the symbolic language of the training sample:

[0151] For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols;

[0152] Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion;

[0153] The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

[0154] Preferably, for each training sample, the complexity of the training sample is calculated based on the logical form of the symbolic language and the reasoning difficulty, including:

[0155] Convert the symbolic language of the training sample into disjunctive paradigm;

[0156] The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm.

[0157] The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm.

[0158] The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity.

[0159] The above-described method and system embodiments are based on the same principles, and their related aspects can be referenced from each other to achieve the same technical effects. For specific implementation processes, please refer to the foregoing embodiments, which will not be repeated here.

[0160] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0161] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for enhancing the logical reasoning ability of large language models, characterized in that, Includes the following steps: For each training sample in the training set, the conclusion and each premise of the training sample are converted into symbolic representations to obtain the symbolic language of the training sample. For each training sample, the complexity of the training sample is calculated based on the logical form of the symbolic language and the reasoning difficulty. The training set is divided into training subsets of different complexity levels based on the complexity of each training sample. The large language model is trained in stages based on the training subset in order of increasing complexity to obtain a well-trained large language model. For each training sample, the complexity of that training sample is calculated based on the logical form of the symbolic language and the difficulty of reasoning, including: Convert the symbolic language of the training sample into disjunctive paradigm; The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm. The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm. The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity. The symbolic logic complexity of the training sample is calculated using the following formula based on the disjunctive normal form: ； in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. The symbolic logic complexity of the sample.

2. The method for enhancing the logical reasoning ability of large language models according to claim 1, characterized in that, The conclusions and each premise of the training sample are converted into symbolic representations in the following way to obtain the symbolic language of the training sample: For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols; Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion; The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

3. The method for enhancing the logical reasoning ability of large language models according to claim 1, characterized in that, The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm in the following manner: ； in, This represents the logical reasoning complexity of the training samples. This represents the probability that the conclusion of the training sample is true, calculated based on the disjunctive normal form. It indicates that it is true.

4. The method for enhancing the logical reasoning ability of large language models according to claim 3, characterized in that, The probability that the conclusion of this training sample is true is calculated using the following formula based on the disjunctive normal form: ； in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. Indicates the given model parameters At that time, the first in the disjunction paradigm of the large language model decision The first disjunction in the disjunction The probability that the sum of its components is true. It indicates that it is true.

5. The method for enhancing the logical reasoning ability of large language models according to claim 1, characterized in that, The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity as follows: The symbolic logic complexity and logical reasoning complexity of each training sample are normalized respectively. Based on formula The complexity of obtaining this training sample is, where, This represents the complexity of the symbolic logic form after normalization. This represents the complexity of logical reasoning after normalization. This represents the weighting coefficient.

6. A system for enhancing the logical reasoning ability of large language models, characterized in that, include: The conversion module is used to convert the conclusion and each premise of each training sample in the training set into a symbolic representation, thereby obtaining the symbolic language of the training sample. The complexity calculation module is used to calculate the complexity of each training sample based on the logical form of the symbolic language and the reasoning difficulty. The model training module is used to divide the training set into training subsets of different complexity levels based on the complexity of each training sample. The large language model is trained in stages based on the training subset in order of increasing complexity to obtain a well-trained large language model. For each training sample, the complexity of that training sample is calculated based on the logical form of the symbolic language and the difficulty of reasoning, including: Convert the symbolic language of the training sample into disjunctive paradigm; The symbolic logic complexity of the training sample is calculated based on the disjunctive paradigm. The logical reasoning complexity of the training sample is calculated based on the disjunctive paradigm. The complexity of the training sample is obtained based on the symbolic logic form complexity and the logical reasoning complexity. The symbolic logic complexity of the training sample is calculated using the following formula based on the disjunctive normal form: ； in, This indicates the number of disjunctive branches in the disjunctive normal form. Indicates the first The number of conjuncts in a disjunct. The symbolic logic complexity of the sample.

7. The system for enhancing the logical reasoning ability of large language models according to claim 6, characterized in that, The conclusions and each premise of the training sample are converted into symbolic representations in the following way to obtain the symbolic language of the training sample: For the conclusions and each premise of the training sample, extract each atomic proposition and represent it with symbols; Logical connectives are used to link symbolically represented atomic propositions to obtain a symbolic representation of the premise or conclusion; The symbolic language of the training sample is obtained based on the conclusions and the symbolic representation of each premise.

Images