A sequence knowledge fusion enhanced large model fine-tuning method
By constructing an initial sequence knowledge set, designing a SeqKQA dataset and a multi-dimensional evaluation index system, and fine-tuning with semantic pairing instructions, the accuracy of the large language model in sequence knowledge question answering was improved, the shortcomings of the model in sequence knowledge understanding and reasoning were solved, and higher question answering accuracy and stability were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EAST CHINA UNIV OF SCI & TECH
- Filing Date
- 2026-03-06
- Publication Date
- 2026-06-05
Smart Images
Figure CN122154837A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and natural language processing, specifically to the field of generative question answering using large language models, and more specifically, to provide a method for fine-tuning large models that incorporates sequence knowledge enhancement. Background Technology
[0002] With the continuous improvement of Large Language Models (LLMs), natural language question answering has gradually evolved from early factual information retrieval to an intelligent interaction mode capable of understanding complex semantics and contextual relationships. However, existing models still have significant shortcomings in the basic cognitive ability of sequence knowledge—common sense information with temporal order or simple logical sequence relationships. These types of questions are often intuitive for humans and require no complex reasoning to arrive at the correct answer, but models may make obvious errors in actual question answering. For example, for the question "Which season comes before autumn?", the correct answer is "winter," but existing models may give the obviously incorrect result of "summer," reflecting their deficiencies in sequence relationship modeling and reasoning. This problem indicates that the reliability of current large language models at the basic cognitive level still has room for improvement, which to some extent restricts their stability and credibility in practical applications. Therefore, it is necessary to conduct further research and improvement on the ability to understand and reason about sequence knowledge.
[0003] Currently, existing natural language question answering systems (NLE) still suffer from two typical defects in sequence knowledge understanding: 1) the curse of inversion, where the model, after learning sentences in the form "A is B," cannot automatically understand or generate the reverse logic "B is A." 2) the curse of multi-hops, where when the model learns two independent facts (A→B and B→C) separately, if it has not encountered co-occurring data samples during training, it will be unable to autonomously infer the indirect relationship between A and C. To address these defects, existing research has proposed several solutions, broadly categorized into three types: 1) Enhanced attention mechanisms. Optimizing the attention mechanism at the model architecture level allows the model to capture more comprehensive contextual information, strengthening its ability to perceive relationships between entities. 2) Optimized data construction. Starting from the training data level, through structured modification and high-definition annotation of the data, the model can learn relationships between entities more clearly during training, improving the accuracy of relationship reasoning. 3) Sample permutation training. Adjusting the training strategy by aggressively permuting and replacing training samples, allowing the model to learn relationships between entities from diverse sample formats.
[0004] While existing research has laid a foundation for sequence knowledge question answering tasks, it still has shortcomings at the fine-grained level: 1) It has not fully explored the potential weaknesses of models in basic sequence knowledge and has ignored the linguistic specificity of word order-driven semantics. 2) The constructed datasets are limited to a single scenario, containing only simple question-answer pairs and lacking systematic coverage of multiple types of sequence knowledge and multiple word order expressions. 3) The evaluation methods still rely on general metrics and lack specific evaluation criteria for understanding word order semantic differences and sequence relationships. 4) Fine-tuning strategies are not sufficiently targeted for Chinese sequence knowledge question answering tasks and lack customized training designs.
[0005] In summary, there is an urgent need in this field for a large-scale model fine-tuning method that integrates sequence knowledge enhancement. First, an initial sequence knowledge set is constructed and iteratively expanded by combining it with the test results of the large model. Then, a Sequential Knowledge Question Answering (SeqKQA) dataset is built using the sequence knowledge. Next, a dedicated evaluation index system is designed to quantify the difficulty of the questions. Finally, a targeted semantic pairing instruction fine-tuning method is proposed, which significantly improves the accuracy of large-scale model sequence relative positioning question answering. Summary of the Invention
[0006] The purpose of this invention is to provide a large model fine-tuning method that integrates sequence knowledge enhancement, which effectively alleviates the problems of insufficient sequence knowledge reserves, lack of dedicated sequence knowledge question answering datasets, imperfect evaluation system, and lack of targeted fine-tuning strategies in existing LLM optimization schemes, thereby improving the model's semantic understanding of word order changes and the accuracy of sequence relative positioning question answering.
[0007] To achieve the above objectives, the present invention provides the following solution, including:
[0008] S1: Iterative expansion of the sequence knowledge set through large-scale model testing. First, through LLM testing and error type analysis, we identified cognitive biases in the model's understanding of relative positions involving sequence. We abstracted this type of knowledge with a clear sequential structure into sequence knowledge and formed an initial sequence knowledge set based on this. Second, based on this initial set, we used GPT-4o to expand and generalize knowledge generation, and designed exclusive filtering prompts to remove knowledge that the model did not master, iteratively forming an expanded sequence knowledge set.
[0009] S2: Constructing the Sequence Knowledge Question Answering (SeqKQA) dataset. First, we define subsequence knowledge as a sequence of elements that maintains its original order, and define the ordered structure of interrogative words, relational words, and subjects as sentence patterns to support dataset construction. Second, we design questions from both positive and negative question directions, combining different sentence patterns, and use GPT-4o to complete the questions. Finally, we employ a human-machine collaborative annotation method, performing data annotation in three stages to obtain and partition the SeqKQA dataset, providing dedicated data support for question analysis and model optimization.
[0010] S3: Constructing a multi-dimensional evaluation index system to quantitatively characterize the bias in understanding the relative position of sequences. First, based on conventional evaluation indexes, we added the NotAttempted (NA) rate and Correct Given Attempted (CGA) rate to suit the task characteristics of sequence knowledge location question answering. The former measures the proportion of model-generated content that is irrelevant to the reference answer, while the latter counts the proportion of correct answers among samples that have made effective attempts to answer the question, in order to more accurately reflect the model's true understanding of the question's semantics. Second, we conducted experiments on open-source and closed-source models and used the evaluation indexes to quantitatively analyze the results, revealing the model's bias in understanding relative position.
[0011] S4: A semantic pairing instruction fine-tuning method is proposed to alleviate the model's understanding bias in relative sequence positioning. First, we design a semantic pairing strategy based on the SeqKQA training set to obtain training data. Second, we design a Chain of Thought (CoT) instruction to assist in the semantic pairing task. Finally, we fine-tune the open-source LLM using the training data, and evaluate the model's learning effect under known knowledge patterns and its adaptability under unseen sequence patterns using in-distribution and out-of-distribution test sets, respectively, thereby verifying the effectiveness of the proposed method and improving the model's accuracy in Chinese sequence relative position question answering tasks.
[0012] S1 aims to identify cognitive biases in LLM’s understanding of the relative positions of sequences through model testing, build and iteratively expand the initial sequence knowledge set, and lay the knowledge foundation for the construction of subsequent datasets.
[0013] S11: Identifying Model Inadequacies in Constructing Initial Sequence Knowledge. First, we tested the LLM by constructing a basic question-and-answer task focusing on relative order, revealing comprehension biases in this type of question. Second, we abstracted this type of content with a clear sequential structure into sequence knowledge and collected and summarized related error types, forming an initial sequence knowledge set encompassing five dimensions: numerical, numerical, chronological, hierarchical, and semantic logic. This type of knowledge can be formally represented as:
[0014]
[0015] in and The first and second of the sequence are respectively The and the first One element, Let be the total length of the sequence, if ,but exist Previously, if ,but exist after.
[0016] S12: Expanding and validating sequence knowledge to construct a set of effective sequence knowledge. First, we rewrite and expand the initial sequence knowledge using GPT-4o to generate a set of candidate sequence knowledge. Second, we design prompt words to filter candidate sequence knowledge to eliminate content that the model does not know. We conduct filtering experiments on closed-source and open-source models and finally obtain 100 effective sequence knowledge.
[0017] S2 aims to construct a high-quality SeqKQA dataset that adapts to word order features and targets the task of relative positioning of sequence knowledge, providing dedicated and accurate Chinese experimental data support for the analysis of the causes of the problem of bias in understanding the relative position of sequences, quantitative evaluation, and subsequent model optimization.
[0018] S21: Define subsequence knowledge and sentence structure to establish data construction specifications. First, we define subsequence knowledge as a new sequence formed by selecting some elements from a given sequence of knowledge while retaining their original order. Its formal definition is:
[0019]
[0020] in ,and for The subsequence; secondly, we define the question sentence as consisting of interrogative words. Relationship words ,subject The constructed ordered sequence is formally represented as:
[0021]
[0022] This definition, by constraining the order of the core components, characterizes the structural form of the question, supporting the subsequent construction of the dataset.
[0023] S22: Design the question generation rules. First, we divide questions into two categories based on different relational terms: forward and reverse. Forward questions use the relational term "before...", meaning they search forward within the sequence; reverse questions use the relational term "after...", meaning they search backward within the sequence. Second, we will... , , The three main components are organized in different orders to produce six sentence patterns.
[0024] S23: Human-computer collaborative annotation and segmentation of high-quality question-answer samples. First, we sampled triples from each filtered sequence of knowledge to generate a subsequence knowledge set. Second, we defined interrogative words and used the middle element of the subsequence triples as the subject of the question. We generated question templates based on six sentence structures and determined answer labeling rules according to the relation words and their relative positions to the subject: when the relation word is "before" or "before" and is immediately adjacent to the subject, the reference answer is the first element of the subsequence triple; otherwise, it is the last element. When the relation word is "after" or "after" and is immediately adjacent to the subject, the reference answer is the last element; otherwise, it is the first element. Then, we used GPT-4o to generate and optimize the question templates using natural language, forming semantically complete and naturally expressed questions, thus constructing a complete dataset. Finally, we invited two university students to perform data quality verification to ensure the accuracy of the dataset, resulting in a SeqKQA dataset containing 16,067 high-quality samples. We divided the training and test sets in a 3:1 ratio and further divided the test set into in-distribution and out-of-distribution test sets based on whether the sequence knowledge types involved in the test samples appeared in the training data.
[0025] S3 aims to design a multi-dimensional evaluation index system to quantitatively determine the bias in understanding the relative positions of sequences, and to provide a scientific evaluation basis for verifying the subsequent model optimization effect.
[0026] S31: Constructing evaluation metrics to quantify experimental results. First, common metrics are used in the evaluation system, including the accuracy rate (CO), which measures the proportion of predicted answers that are completely consistent with the reference answer, and the error rate (IN), which reflects the semantic contradiction between the predicted results and the reference answer. Second, NA and CGA are designed to distinguish between cases where the model does not respond effectively to the semantics of the question and cases where the model answers accurately in samples that have made valid attempts. Finally, the F1 score is obtained by calculating the harmonic mean of CO and CGA, forming an evaluation system containing five metrics to quantify the experimental results.
[0027] S32: Using metrics to assess the severity of the relative positional perception bias problem. First, we found that the F1 scores of all model experimental results were far below 100%, proving that positional perception blind spots are prevalent in both closed-source and open-source LLMs of different scales. Second, we categorized the experimental results of open-source models according to their number of parameters, finding that the smaller the number of model parameters, the lower the CO, CGA, and F1 scores, and the more severe the relative positional perception bias problem. Finally, we compared the various experimental result metrics of open-source and closed-source models, finding that open-source models are more affected by this phenomenon.
[0028] S4 aims to propose a semantic pairing instruction fine-tuning method, verify the effectiveness of the method through two types of test sets, and improve the accuracy and generalization ability of the model on Chinese sequence relative position question answering tasks.
[0029] S41: Design a semantic pairing strategy based on the SeqKQA training set. First, we analyze the semantic and sentence structure features of all question-answering samples in the SeqKQA training set and group the samples according to the core principle of semantic similarity but different word order. Second, within each group, we label the non-aligned sentences that the model is prone to cognitive bias in, as well as the aligned sentences that the model can stably answer, forming one-to-one semantic pairing sample pairs. Finally, we perform format regularization and validity verification on all paired sample pairs to construct a semantic pairing training dataset suitable for instruction fine-tuning.
[0030] S42: Designing CoT Instructions to Facilitate Semantic Matching Tasks. First, focusing on the core objective of semantic matching, we determined the design logic of CoT instructions to guide the model to first identify the semantic equivalence between unaligned and aligned sentences, and then deduce the word order transformation logic between them. Second, we designed standardized CoT instruction templates, clarifying the guidance steps and expression paradigms of the instructions, including three core links: sentence semantic analysis, word order transformation reasoning, and answer generation and verification, ensuring that the model can complete semantic matching by following fixed logic. Finally, we fused CoT instructions with semantic matching sample pairs to form an integrated fine-tuning input format of "instruction + sample".
[0031] S43: Testing the effect of open-source large model instruction fine-tuning. First, we selected a mainstream open-source LLM as the base model for fine-tuning, and carried out standardized instruction fine-tuning based on semantic pairing training data combined with the "instruction + sample" input format. Second, we used a multi-dimensional evaluation index system to comprehensively evaluate the fine-tuned model on both in-distribution and out-of-distribution test sets. Finally, we compared the differences in evaluation indexes of the model before and after fine-tuning to verify the effectiveness of the semantic pairing instruction fine-tuning method in solving the problem of misunderstanding of relative positions in Chinese sequences.
[0032] As can be seen from the above technical solutions, this invention first identifies cognitive biases in a large model by conducting sequence relative positioning question-answering tests and summarizes iterative sequence knowledge, thus preparing a knowledge foundation for dataset construction. Secondly, based on sequence knowledge, this invention designs question generation rules and obtains a SeqKQA dataset adapted to word order features through human-computer collaborative annotation and partitioning. Then, based on conventional evaluation metrics, this invention adds NA and CGA to address the characteristics of the sequence relative positioning question-answering task, and constructs a multi-dimensional evaluation metric system combined with harmonic mean F1 score to quantitatively represent the model's bias in understanding the relative position of sequences. Finally, based on the SeqKQA training set, this invention designs a semantic pairing strategy to construct training data, combines it with customized CoT instructions to form an integrated fine-tuning input format, performs semantic pairing instruction fine-tuning on the open-source LLM, and verifies the effect through in-distribution and out-of-distribution dual test sets, thereby improving the model's question-answering accuracy on this task. This method has the following significant advantages: (1) Sequence knowledge mining and construction: By identifying cognitive biases in understanding the relative position of sequences through model testing, the sequence knowledge set is specifically abstracted and iteratively expanded to provide knowledge support for the construction of the dataset; (2) Sequence knowledge question answering dataset construction: Based on sequence knowledge, combined with positive and negative question directions and six sentence patterns, diverse questions are generated. Through human-computer collaborative annotation and multiple rounds of verification, a high-quality SeqKQA dataset is obtained, providing high-quality data support for subsequent research; (3) Multidimensional sequence knowledge question answering evaluation system: Breaking through the limitations of conventional evaluation indicators, NA and CGA indicators are added and harmonic average F1 values are integrated to quantify the biases in understanding the relative position of sequences in the model, providing a scientific basis for verifying the model optimization effect; (4) Semantic pairing instruction fine-tuning: Based on the sequence knowledge features, a semantic pairing strategy and customized CoT instructions are designed to make the fine-tuning logic deeply compatible with Chinese semantics, effectively improving the accuracy of the model in the sequence relative position question answering task, and solving the problem of the lack of directional fine-tuning strategies for existing large models. Attached Figure Description
[0033] Figure 1 This is a schematic diagram of the overall process of a large model fine-tuning method for fusion sequence knowledge enhancement proposed in this invention;
[0034] Figure 2 These are schematic diagrams of the six sentence structures proposed in this invention; Detailed Implementation
[0035] This invention proposes a large-scale model fine-tuning method that integrates sequence knowledge enhancement, which mainly includes: expanding the sequence knowledge set by combining large-scale model testing iterations; constructing a sequence knowledge question answering SeqKQA dataset; constructing a multi-dimensional evaluation index system to quantitatively represent the problem of understanding bias in the relative position of sequences; and proposing a semantic pairing instruction fine-tuning method to alleviate the model's understanding bias in the relative position of sequences.
[0036] Step S1: Iterate and expand the sequence knowledge set by combining large model testing. Specifically, this includes:
[0037] S11: Identifying Model Inadequacies in Initial Sequence Knowledge Construction. First, we tested the LLM by constructing a basic question-and-answer task focusing on relative order. We found that the model exhibited comprehension biases in these types of questions (e.g., for the questions "Which season precedes autumn?" and "Which season precedes autumn?", the correct answers are "winter" and "summer," but the model consistently output "summer"). Second, we abstracted this type of content with a clear sequential structure into sequence knowledge (e.g., spring, summer, autumn, winter), and collected and summarized related error types, forming an initial sequence knowledge set encompassing five dimensions: numerical, numerical, temporal, hierarchical, and semantic logic. This type of knowledge can be formally represented as:
[0038]
[0039] in and The first and second of the sequence are respectively The and the first One element, Let be the total length of the sequence, if ,but exist Previously, if ,but exist after.
[0040] S12: Expanding and validating sequence knowledge to construct a set of effective sequence knowledge. First, we rewrite and expand the initial sequence knowledge using GPT-4o to generate a candidate sequence knowledge set. Second, we design prompt words to filter candidate sequence knowledge to eliminate content not mastered by the model. Filtering experiments are conducted on both closed-source and open-source models, ultimately yielding 100 effective sequence knowledge entries. Specific prompt word examples are as follows: {
[0041] "role_definition":"You are an expert in the field of Chinese history."
[0042] "instruction": "Your task is to answer the question that is entered, and the answer must be selected from the options given in the question."
[0043] "input":{"question": "Which dynasty came earlier, the Qing Dynasty or the Ming Dynasty?"}
[0044] }
[0045] Step S2: Construct the Sequence Knowledge Question Answering (SeqKQA) dataset. Specifically, this includes:
[0046] S21: Define subsequence knowledge and sentence structure to establish data construction specifications. First, we define subsequence knowledge as a new sequence formed by selecting some elements from a given sequence of knowledge while retaining their original order. Its formal definition is:
[0047]
[0048] in ,and for subsequences (e.g., given a sequence) Its partial subsequences are (spring, summer, autumn) and (summer, autumn, winter); while (autumn, spring, summer) does not belong to the subsequence knowledge because it violates the original order. Secondly, we define the question sentence as consisting of interrogative words. Relationship words ,subject An ordered sequence (e.g., a sentence structure (which season, before, autumn)) is formally represented as:
[0049]
[0050] This definition, by constraining the order of the core components, characterizes the structural form of the question, supporting the subsequent construction of the dataset.
[0051] S22: Design the question generation rules. First, we categorize questions into two types based on their relational terms: forward and reverse. Forward questions use the relational terms "before" or "before," meaning they are retrieved from the sequence forward. Reverse questions use the relational terms "after" or "after," meaning they are retrieved from the sequence backward. Second, we will... , , The three main components are organized in different orders to form six sentence patterns. Taking the positive example, these include (autumn, which season, before), (which season, before, autumn), (autumn, before, which season), (which season, autumn, before), (before, which season, autumn), and (before, autumn, which season).
[0052] S23: Human-computer collaborative annotation and segmentation of high-quality question-answer samples. First, we sampled each filtered sequence of knowledge using triples to generate a sub-sequence knowledge set. Second, we defined interrogative words and used the middle element of the sub-sequence triples as the question subject. Based on six sentence structures, we generated question templates and determined the answer annotation rules according to the relational words and their relative positions to the subject: when the relational word is "before" or "before" and is immediately adjacent to the subject, the reference answer is the first element of the sub-sequence triple; otherwise, it is the last element. When the relational word is "after" or "after" and is immediately adjacent to the subject, the reference answer is the last element; otherwise, it is the first element (e.g., the triple is (summer, autumn, winter), the sentence structure is (which season, autumn, before), and the question is "Which season precedes autumn?"). If the question template is "?", the answer is summer. Then, we use GPT-4o to generate and optimize the expression of the question template using natural language, forming semantically complete and naturally expressed questions, thus constructing a complete dataset (for example, if the sentence is (autumn, before, which season), the optimized question is "What season is before autumn?"). Finally, we invited two college students to complete the data quality verification to ensure the accuracy of the dataset, and constructed a SeqKQA dataset containing 16067 high-quality samples. The training set and test set were divided in a 3:1 ratio, and the test set was divided into 3923 in-distribution test sets and 648 out-of-distribution test sets according to whether the sequence knowledge types involved in the test samples appeared in the training data.
[0053] Step S3: Construct a multi-dimensional evaluation index system to quantitatively characterize the problem of bias in understanding the relative positions of sequences. Specifically, this includes:
[0054] S31: Constructing Evaluation Indicators to Quantify Experimental Results. First, common indicators are used in the evaluation system, including CO (Coefficient of Performance) to measure the proportion of predicted answers that are completely consistent with the reference answer, and IN (Inconsistent Meaning) to reflect semantic contradictions between the predicted results and the reference answer (e.g., the answer to the question "Which season comes before autumn?" is "Spring, Summer, Autumn, Winter," which includes the reference answer but is still considered incorrect). Second, NA (Not Attempted) and CGA are designed specifically for the characteristics of sequence location question answering tasks (e.g., if the question is "Which season comes before autumn?" and the answer is "There are four seasons in a year," and the generated response is completely unrelated to the reference answer, then the question is considered "unattended") to distinguish between situations where the model does not respond effectively to the semantics of the question and the accuracy of the answers in samples where effective attempts have been made. Finally, the F1 score is calculated by harmonic averaging CO and CGA, forming an evaluation system containing five indicators to quantify the experimental results.
[0055] S32: Using metrics to assess the severity of the relative positional perception bias problem. First, we found that the F1 scores of all model experimental results were far below 100%, proving that positional perception blind spots are prevalent in both closed-source and open-source LLMs of different scales. Second, we categorized the experimental results of open-source models according to their number of parameters, finding that the smaller the number of model parameters, the lower the CO, CGA, and F1 scores, and the more severe the relative positional perception bias problem. Finally, we compared the various experimental result metrics of open-source and closed-source models, finding that open-source models are more affected by this phenomenon.
[0056] Step S4: Propose a semantic pairing instruction fine-tuning method to mitigate the bias in understanding the relative localization of model sequences. Specifically, this includes:
[0057] S41: Design a semantic pairing strategy based on the SeqKQA training set. First, we analyze the semantic and sentence structure features of all question-answering samples in the SeqKQA training set and group the samples according to the core principle of semantic similarity but different word order. Second, within each group, we label the non-aligned sentences that the model is prone to cognitive bias in, as well as the aligned sentences that the model can answer stably, forming one-to-one semantic pairing sample pairs (for example, pairing the questions "What season comes before autumn?" with "What season comes after autumn?"). Finally, we perform format regularization and validity verification on all paired sample pairs to construct a semantic pairing training dataset suitable for fine-tuning instructions.
[0058] S42: Designing CoT Instructions to Facilitate Semantic Matching Tasks. First, focusing on the core objective of semantic matching, we determined the design logic of CoT instructions to guide the model to first identify the semantic equivalence between unaligned and aligned sentence structures, and then deduce their word order transformation logic. Second, we designed standardized CoT instruction templates, clarifying the guidance steps and expression paradigms of the instructions, including three core stages: sentence semantic analysis, word order transformation reasoning, and answer generation and verification. This ensures that the model can complete semantic matching by following fixed logic. The specific instruction template is as follows: {
[0059] "Querry":"Task: Answer questions based on sequence knowledge. You need to transform the sentence structure before answering."
[0060] Given sequence knowledge: The sequence of numbers is 1→2→3→4 (that is, 1 comes before 2, 2 comes before 3, and 3 comes before 4).
[0061] Question (QP1): Before which number does 2 come?
[0062] Require:
[0063] 1. First, convert QP1 into the target sentence QP2 with the same semantics (format reference: "Which number comes after X?").
[0064] 2. Based on the transformed QP2, provide the accurate answer.
[0065] Output format:
[0066] Transformed QP2: [Sentence Transformation Result]
[0067] Final answer: [Answer]
[0068] Finally, the CoT instructions are fused with semantic pairing sample pairs to form an integrated fine-tuning input format of "instruction + sample".
[0069] S43: Testing the effect of open-source large model instruction fine-tuning. First, we selected mainstream open-source LLMs (such as Qwen2.5-7B and Internlm2.5-7B) as base models, and carried out standardized instruction fine-tuning based on semantic pairing training data combined with the "instruction + sample" input format. Second, we used a multi-dimensional evaluation index system to comprehensively evaluate the fine-tuned model on both in-distribution and out-of-distribution test sets. Finally, we compared the differences in evaluation indexes of the model before and after fine-tuning to verify the effectiveness of the semantic pairing instruction fine-tuning method in solving the problem of misunderstanding of relative position in Chinese sequences (experimental results show that on the Tongyi Qianwen 2.5-7B and Internlm2.5-7B models, this method improves the F1 score on the test set by 12.01% and 11.77%, respectively, and on the out-of-distribution test set by 8.58% and 10.84%, respectively).
Claims
1. A method for fine-tuning a large model that incorporates sequence knowledge enhancement, characterized in that... Includes the following steps: S1. Iteratively expanding the sequence knowledge set by combining large model testing: First, through large language model (LM) testing and error type analysis, we identify cognitive biases in the model's understanding of relative positions involving sequence. We abstract this type of knowledge with a clear sequential structure into sequence knowledge and summarize it to form an initial sequence knowledge set. Second, based on this initial set, we use GPT-4o to expand and generalize knowledge generation, and design exclusive filtering prompts to remove knowledge that the model has not mastered, iteratively forming an expanded sequence knowledge set. S2. Constructing the Sequential Knowledge Question Answering (SeqKQA) dataset: First, we define subsequence knowledge as a sequence of partial elements that maintains its original order, and define the ordered structure composed of interrogative words, relational words, and subjects as sentence patterns to support the construction of the dataset. Second, we design questions from both positive and negative question directions, combining different sentence patterns, and use GPT-4o to complete the questions. Finally, we adopt a human-computer combined annotation method, annotating the data in three stages to obtain the SeqKQA dataset and divide it, providing dedicated data support for question analysis and model optimization. S3. Constructing a multi-dimensional evaluation index system to quantitatively represent the problem of bias in understanding the relative position of sequences: First, based on conventional evaluation indexes, we added the Not Attempted (NA) rate and Correct Given Attempted (CGA) rate to suit the task characteristics of sequence knowledge location question answering. The former measures the proportion of model-generated content that is irrelevant to the reference answer, while the latter counts the proportion of correct answers among samples that have made effective attempts to answer the question, so as to more accurately reflect the model's true understanding of the question's semantics. Second, we conducted experiments on open-source and closed-source models and used evaluation indexes to quantitatively analyze the results, revealing the model's bias in understanding the relative position. S4. A semantic pairing instruction fine-tuning method is proposed to alleviate the model's understanding bias in relative positional questions in Chinese sequences: First, we design a semantic pairing strategy based on the SeqKQA training set to obtain training data; second, we design a Chain of Thought (CoT) instruction to assist in the semantic pairing task; finally, we use the training data to fine-tune the instructions of the open-source LLM, and evaluate the model's learning effect under known knowledge patterns and its adaptability under unseen sequence patterns through two test sets, in-distribution and out-of-distribution, respectively, thereby verifying the effectiveness of the proposed method and improving the accuracy of the model in Chinese sequence relative positional question answering tasks.
2. The multimodal reasoning algorithm based on bidirectional chained thinking enhancement according to claim 1, characterized in that, Step S1 specifically includes: S11. Recognizing Model Inadequacy in Constructing Initial Sequence Knowledge: First, we tested the LLM by constructing a basic question-and-answer task focusing on relative order, revealing a comprehension bias in this type of question. Second, we abstracted this type of content with a clear sequential structure into sequence knowledge, and collected and summarized related error types, forming an initial sequence knowledge set around five dimensions: numerical, numerical, chronological, hierarchical, and semantic logic. This type of knowledge can be formally represented as: in and The first and second of the sequence are respectively The and the first One element, Let be the total length of the sequence, if ,but exist Previously, if ,but exist after. S12. Expanding and verifying sequence knowledge to construct a set of effective sequence knowledge: First, we use GPT-4o to rewrite and expand the initial sequence knowledge in S11 to generate a set of candidate sequence knowledge; second, we design prompt words to filter candidate sequence knowledge to eliminate content that the model does not know, and conduct filtering experiments on closed-source and open-source models to finally obtain 100 effective sequence knowledge.
3. The multimodal reasoning algorithm based on bidirectional chained thinking enhancement according to claim 1, characterized in that, Step S2 specifically includes: S21. Defining Subsequence Knowledge and Sentence Structure to Establish Data Construction Specifications: First, we define subsequence knowledge as a new sequence formed by selecting some elements from a given sequence of knowledge while retaining their original order. Its formal definition is: in ,and for The subsequence; secondly, we define the question sentence as consisting of interrogative words. Relationship words ,subject The constructed ordered sequence is formally represented as: This definition, by constraining the order of the core components, characterizes the structural form of the question, supporting the subsequent construction of the dataset. S22. Design Question Generation Rules: First, we categorize questions into two types based on their relational terms: forward and reverse. Forward questions use the relational term "before...", meaning they search forward within the sequence; reverse questions use the relational term "after...", meaning they search backward within the sequence. Second, we will... , , The three main components are organized in different orders to produce six sentence patterns. S23. Human-computer collaborative annotation and segmentation of high-quality question-answer samples: First, we sample each filtered sequence knowledge in S12 into triples according to the definition in S21 to generate a subsequence knowledge set; second, we define question words and use the middle element of the subsequence triples as the question subject, generating question templates based on the six sentence structures in S22. Simultaneously, we determine the answer annotation rules based on the relation words and their relative positions to the subject: when the relation word is "before" or "before" and is immediately adjacent to the subject, the reference answer is the first element of the subsequence triple; otherwise, it is the last element; when the relation word is "after" or "after" and is immediately adjacent to the subject... When defining the subject, the reference answer is the last element; otherwise, it is the first element. Then, we use GPT-4o to generate and optimize the expression of the question template using natural language, forming semantically complete and naturally expressed questions, thus constructing a complete dataset. Finally, we invited two college students to complete the data quality verification to ensure the accuracy of the dataset, and constructed a SeqKQA dataset containing 16,067 high-quality samples. The training set and test set were divided in a 3:1 ratio, and the test set was further divided into an in-distribution test set and an out-of-distribution test set based on whether the sequence knowledge types involved in the test samples appeared in the training data.
4. The multimodal reasoning algorithm based on bidirectional chained thinking enhancement according to claim 1, characterized in that, Step S3 specifically includes: S31. Constructing Evaluation Indicators to Quantify Experimental Results: First, common indicators are used in the evaluation system, including the accuracy rate (CO), which measures the proportion of predicted answers that are completely consistent with the reference answer, and the error rate (IN), which reflects the semantic contradiction between the predicted results and the reference answer. Second, NA and CGA are designed to distinguish between cases where the model does not respond effectively to the semantics of the question and cases where the model answers accurately in samples that have made effective attempts. Finally, the F1 score is obtained by calculating the harmonic average of CO and CGA, forming an evaluation system containing five indicators to quantify the experimental results. S32. Using metrics to assess the severity of the relative positional perception bias problem: First, according to the F1 calculation method defined in S31, the F1 values corresponding to the experimental results of all models are significantly lower than 100%, proving that positional perception blind spots are prevalent in both closed-source and open-source LLMs of different scales; Second, we categorized the experimental results of open-source models according to their parameter count, and found that the smaller the model parameter count, the lower the CO, CGA, and F1 values, and the more severe the relative positional perception bias problem; Finally, we compared the various experimental result metrics of open-source and closed-source models respectively, and found that open-source models are more affected by this phenomenon.
5. The multimodal reasoning algorithm based on bidirectional chained thinking enhancement according to claim 1, characterized in that, Step S4 specifically includes: S41. Design a semantic pairing strategy based on the SeqKQA training set: First, we analyze the semantic and sentence structure features of all question-answering samples in the SeqKQA training set and group the samples according to the core principle of semantic similarity but different word order. Second, within each group, we label the non-aligned sentences that the model is prone to cognitive biases, as well as the aligned sentences that the model can stably answer, forming one-to-one semantic pairing sample pairs. Finally, we perform format regularization and validity verification on all paired sample pairs to construct a semantic pairing training dataset suitable for instruction fine-tuning. S42. Designing CoT Instructions to Assist Semantic Matching Tasks: First, focusing on the core objective of semantic matching, we determined the design logic of CoT instructions to guide the model to first identify the semantic equivalence between unaligned and aligned sentences, and then deduce the word order transformation logic between the two. Second, we designed standardized CoT instruction templates, clarifying the guidance steps and expression paradigms of the instructions, including three core links: sentence semantic analysis, word order transformation reasoning, and answer generation and verification, to ensure that the model can complete semantic matching by following fixed logic. Finally, we integrated CoT instructions with semantic matching sample pairs to form an integrated fine-tuning input format of "instruction + sample". S43. Testing the effect of open-source large model instruction fine-tuning: First, we selected mainstream open-source LLM as the base model for fine-tuning. Based on the semantic pairing training data in S41 and the "instruction + sample" input format, we carried out the standardized instruction fine-tuning designed in S42. Second, we used the multi-dimensional evaluation index system in S31 to comprehensively evaluate the fine-tuned model on both in-distribution and out-of-distribution test sets. Finally, we compared the differences in evaluation indexes of the model before and after fine-tuning to verify the effectiveness of the semantic pairing instruction fine-tuning method in solving the problem of misunderstanding of relative position in Chinese sequences.