Misunderstanding and reordering method for multiple choice questions based on causal big language model
By fine-tuning and quantizing the causal large language model, the problems of candidate misunderstanding confusion and insufficient logical judgment in multiple-choice question misunderstanding identification are solved, achieving efficient and accurate misunderstanding reordering, which is suitable for intelligent education and tutoring in low-resource environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2025-12-24
- Publication Date
- 2026-04-14
AI Technical Summary
Existing technologies for identifying misunderstandings in multiple-choice questions suffer from problems such as confusion of candidate misunderstandings and insufficient logical discrimination ability during the ranking stage, resulting in insufficient accuracy and robustness in misunderstanding identification.
A fine-tuned causal large language model is adopted, and the relevance scores between candidate misunderstandings and query context are calculated through generative or classification head scoring mechanisms and then re-ranked. Combined with low-rank adaptation technology and 4-bit quantization deployment, the model's running efficiency in low-resource environments is optimized.
It improves the accuracy and robustness of misunderstanding reordering, enhances the model's operating efficiency in low-resource environments, and supports precise tutoring in personalized education scenarios.
Smart Images
Figure CN121858600A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a method for reordering misunderstandings in multiple-choice questions based on a causal large language model. Background Technology
[0002] In intelligent tutoring scenarios for multiple-choice math questions, accurately identifying the core cognitive misunderstandings behind students' incorrect options is crucial for providing targeted instructional feedback and helping students correct cognitive biases. Incorrect options in multiple-choice questions are often directly related to specific misunderstandings, and multiple misunderstandings may share similarities in their textual expression. Precisely pinpointing the root cause of the misunderstanding can effectively prevent students from repeatedly making the same mistakes, improving learning efficiency and the targeted nature of tutoring. Therefore, optimizing multiple-choice question misunderstanding identification technology has significant practical value.
[0003] Currently, the process for identifying misunderstandings in multiple-choice questions generally consists of two stages: recall and ranking. The recall stage typically uses a dual-tower vector model to quickly filter out the Top-K candidate misunderstandings semantically similar to the student's incorrect option from a misunderstanding database. The ranking stage often employs models with encoder architectures such as BERT, trained either individually or in pairs, to score the semantic relevance of the "question-incorrect option-candidate misunderstanding" combination, thereby selecting the optimal misunderstanding result. These methods primarily rely on surface-level text feature matching and do not delve into the logical connection between errors and misunderstandings.
[0004] However, existing technologies have significant shortcomings in the ranking stage: Candidate misunderstandings retrieved by vector recall are easily mixed with distractors that are textually similar but logically different (such as "confusing similar items" and "omitting similar items"). Traditional ranking models lack deep logical reasoning capabilities, only able to judge literal relevance, and cannot accurately determine whether a candidate misunderstanding is the root cause of the error. Furthermore, single-point or pairwise training methods ignore the overall distribution characteristics of the candidate list, resulting in insufficient robustness of the model in ranking highly similar distractors, ultimately affecting the accuracy of misunderstanding identification. Therefore, a technical solution with strong logical discrimination capabilities and the ability to optimize the ranking accuracy of candidate misunderstandings is urgently needed. Summary of the Invention
[0005] To address the problems existing in the background art, one aspect of the present invention provides a method for reordering misunderstandings in multiple-choice questions based on a causal large language model, comprising:
[0006] S1: Obtain a set of candidate misunderstandings consisting of the Top-K candidate misunderstandings retrieved from the retrieval database based on the query context, wherein the query context includes: multiple-choice questions and the incorrect options selected by the student;
[0007] S2: Concatenate the query context and each candidate misunderstanding separately to obtain multiple sets of sequences to be judged;
[0008] S3: Input multiple sets of sequences to be discriminated into the fine-tuned causal big language model, and use generative scoring or classification head scoring mechanism to calculate the relevance score between each candidate misconception and the query context;
[0009] S4: Sort the Top-K candidate misconceptions in descending order based on their relevance scores, and output the misconception with the highest score as the final identification result.
[0010] Another aspect of the present invention provides a multiple-choice question misunderstanding reordering system based on a causal large language model. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned multiple-choice question misunderstanding reordering method based on a causal large language model.
[0011] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for reordering misunderstandings in multiple-choice questions based on a causal large language model.
[0012] The present invention has at least the following beneficial effects
[0013] This invention obtains a query context containing multiple-choice questions and students' incorrect options, along with the Top-K candidate misunderstandings retrieved. The query context and each candidate misunderstanding are concatenated into a sequence to be judged and input into a fine-tuned causal big language model. Generative scoring or classification head scoring mechanisms are used to calculate relevance scores and perform re-ranking. This approach leverages the powerful logical reasoning capabilities of the causal big language model to accurately distinguish mathematically difficult negative samples with similar textual expressions but vastly different logical meanings, effectively solving the problem of traditional ranking methods' difficulty in accurately identifying fundamental misunderstandings. This improves the accuracy and robustness of misunderstanding re-ranking. Furthermore, through model fine-tuning and optimization, along with subsequent quantization deployment support, it balances recognition accuracy with model operating efficiency. This provides an efficient and feasible solution for diagnosing mathematical misunderstandings in low-resource edge computing environments, facilitating the precise implementation of personalized tutoring in intelligent education scenarios. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the method flow of the present invention;
[0015] Figure 2 This is a schematic diagram of the model framework process of the present invention. Detailed Implementation
[0016] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0017] Please see Figure 1 One aspect of the present invention provides a method for reordering misunderstandings in multiple-choice questions based on a causal large language model, comprising:
[0018] S1: Obtain a set of candidate misunderstandings consisting of the Top-K candidate misunderstandings retrieved from the retrieval database based on the query context, wherein the query context includes: multiple-choice questions and the incorrect options selected by the student;
[0019] In this embodiment, the Top-K candidate misunderstandings are specifically obtained from the answer record database of the intelligent education platform. After OCR recognition and data cleaning to remove invalid records, multiple-choice question information containing complete question stems, all options, and incorrect options selected by students are extracted to form a query context. At the same time, from the pre-built structured mathematical multiple-choice question misunderstanding database (the misunderstanding entries in the database contain structured information such as core descriptions and related knowledge points and have been preprocessed with semantic vectors), the similarity between the query context and the misunderstanding entries in the database is calculated through a semantic vector retrieval model. The top K candidate misunderstandings are recalled to form the Top-K candidate misunderstanding set, providing basic data pairs for subsequent re-ranking.
[0020] S2: Concatenate the query context and each candidate misunderstanding separately to obtain multiple sets of sequences to be judged;
[0021] Preferably, in step S2, if a generative scoring mechanism is used, the sequence to be judged includes: {multiple-choice questions, the incorrect options selected by the student, candidate misunderstandings, is the misunderstanding the cause of the error? Answer Yes or No}; if a classification head scoring mechanism is used, the sequence to be judged includes: {multiple-choice questions, the incorrect options selected by the student, candidate misunderstandings}.
[0022] In this embodiment, based on the scoring mechanism adopted later, the obtained query context and each candidate misunderstanding in the candidate misunderstanding set are standardized and concatenated. If a generative scoring mechanism is adopted, the sequence to be judged is formed by concatenating a multiple-choice question + the student's incorrect option + the candidate misunderstanding + whether the misunderstanding is the cause of the error (Yes or No). If a classification head scoring mechanism is adopted, the sequence to be judged is formed by concatenating a multiple-choice question + the student's incorrect option + the candidate misunderstanding template. During the concatenation process, the text format is kept consistent and the information is free of redundancy or missing information, so that each sequence can fully present the logical relationship between question, error, and potential misunderstanding, providing standardized input for the subsequent model to accurately calculate the relevance score.
[0023] S3: Input multiple sets of sequences to be discriminated into the fine-tuned causal big language model, and use generative scoring or classification head scoring mechanism to calculate the relevance score between each candidate misconception and the query context;
[0024] Preferably, the generative scoring method for calculating the relevance score between each candidate misunderstanding and the query context includes: inputting the sequence to be judged into a causal large language model, and extracting the EOS corresponding to the vocabulary projection layer. vector Locate the dimension value corresponding to YesToken And calculate the relevance score of each candidate misunderstanding to the query context:
[0025]
[0026] in, A vocabulary set representing a causal large language model; Indicates the EOS corresponding vocabulary projection layer Vector in the th The dimension value corresponding to each word; Represents an exponential function; This represents the relevance score between the candidate misunderstanding and the query context.
[0027] In this embodiment, the sequence to be discriminated, assembled according to the corresponding template, is input into a causal large language model that has been fine-tuned by LoRA to adapt to the mathematical misunderstanding discrimination scenario. The model will perform generative inference token by token until the end symbol (EOS) is output. At this time, the logits vector z output by the vocabulary projection layer corresponding to the EOS is extracted (the dimension of this vector is consistent with the size of the model vocabulary set V, and each dimension value corresponds to the prediction confidence value of a token in the vocabulary). The dimension corresponding to the "YES" token is accurately located in this vector, and the specific value of this dimension is obtained. Then, the calculation is performed using the Softmax function. The ratio of the index value to the sum of the index values of all tokens in the vocabulary set V is used to obtain the relevance score between the candidate misinterpretation and the query context. The score ranges from 0 to 1. A higher score indicates a higher confidence level in the model's determination that the candidate misunderstanding is the root cause of the student's wrong answer, and can accurately quantify the degree of causal relationship between misunderstanding and error in generative scenarios.
[0028] Preferably, the step of calculating the relevance score between each candidate misunderstanding and the query context using the classification head scoring mechanism includes: removing the vocabulary projection layer corresponding to the causal large language model EOS; and adjusting the hidden state vector corresponding to EOS. By mapping the linear layer to a one-dimensional scalar score, the relevance score between each candidate misconception and the query context is obtained:
[0029]
[0030] in, This represents the relevance score between the candidate misinterpretation and the query context; Indicates the bias parameter; Indicates the weighting parameter; This represents the Sigmoid activation function.
[0031] In this embodiment, for the sequence to be discriminated based on the corresponding template, the vocabulary projection layer (LM Head) corresponding to the EOS (end-of-words symbol) in the causal large language model is first removed. Since there is no need to generate subsequent tokens, the hidden state vector corresponding to the EOS is extracted instead. (This vector integrates the semantic information and logical correlation features of the entire sequence to be judged, and is the model's deep semantic representation of the "question-incorrect option-candidate misconception" combination.) Subsequently, this hidden state vector is input with learnable weight parameters. and bias parameters Dimension mapping is performed on the linear layer (where, and (These are the optimal parameters learned by the LoRA model after fine-tuning and adapting to the mathematical misunderstanding discrimination scenario). Finally, the mapping result is compressed to the value range of 0~1 using the Sigmoid activation function σ to obtain the relevance score between the candidate misunderstanding and the query context. The closer the score is to 1, the higher the confidence level of the model in determining that the candidate misunderstanding is the root cause of the student's wrong answer, and the more accurately it can quantify the strength of the causal relationship between the two.
[0032] Preferably, the fine-tuning of the causal large language model employs a low-rank adaptation technique, including: performing low-rank adaptation on the attention weights W of the causal large language model.
[0033]
[0034] in, The attention weights after low-rank adaptation; Attention weights before low-rank adaptation; and This represents the low-rank matrix being inserted; when fine-tuning the causal large language model, the parameters of the causal large language model are frozen, and only the low-rank matrix is fine-tuned. and The parameters.
[0035] In this embodiment, for the attention weight matrix W in the causal large language model, which is responsible for capturing the logical association and semantic features of the text, an adaptation structure is constructed by inserting two low-rank matrices A (dimension d×r, where r is the low-rank dimension) and B (dimension r×d) with dimensions much lower than W. The original attention weights are adjusted using the formula W'=W+A, where W' is the attention weight adapted to the mathematical misunderstanding discrimination scenario. During the fine-tuning process, in order to avoid the loss of the original pre-training knowledge of the large model and reduce the consumption of computing resources, all the original parameters of the causal large language model are frozen, and only the parameters of the low-rank matrices A and B are updated and optimized by gradient. With a small number of parameter adjustments, the model can be accurately adapted to the causal association discrimination task of "question-incorrect option-misunderstanding" in the mathematical field. While ensuring the basic reasoning ability of the model, the fine-tuning efficiency is greatly improved and the memory usage is reduced.
[0036] Preferably, the loss function used in the fine-tuning causal large language model includes:
[0037]
[0038] in, Represents the loss function; ; Represents an exponential function; Indicates query context The corresponding correct misunderstanding The relevance score; Indicates the number of candidate misunderstandings; Indicates query context With respect to correctness and misunderstanding The rest of the outside One candidate misunderstanding The relevance score.
[0039] In this embodiment, the list-level cross-entropy loss function used in fine-tuning the causal large language model is specifically applied to the model fine-tuning process processed by the low-rank adaptation technique. Its core objective is to optimize the model's ranking robustness by mining the overall distribution information of the candidate misunderstanding list. The calculation logic is as follows: for each query context in the training batch... According to its corresponding true misunderstanding As a positive example, consider the misunderstandings of other samples within the same batch. A candidate set of negative examples is constructed, and a scoring mechanism is used to calculate the relevance scores between the query context and the positive examples, as well as each negative example. and Then, the cross-entropy loss is calculated using a formula. This loss function guides the model to focus on the global differences between positive and negative examples during training, avoiding the shortcomings of traditional single-point or pairwise training that ignores the overall information of the list. This enhances the model's ability to distinguish between "difficult negative samples" that are semantically similar but logically different in the mathematical field, and further improves the accuracy of misinterpretation reordering.
[0040] Preferably, the reordering method further includes:
[0041] For the fine-tuned causal big language model, extract all the weight matrices of the causal big language model to construct a weight matrix set;
[0042] The features of each input weight matrix are used as the activation features of that weight matrix, and the scaling factor of the weight matrix is calculated based on the activation features of that weight matrix.
[0043] The weight matrix is compressed into a 4-bit form by combining the scaling factor with a 4-bit quantization function to obtain the compressed weight matrix. An INT4 inference kernel is then constructed based on the compressed weight matrix and the scaling factor and deployed to the edge device.
[0044] In this embodiment, for a causal large language model fine-tuned by LoRA and adapted to mathematical misunderstanding discrimination scenarios, the weight matrices responsible for semantic representation and logical reasoning in all layers are first comprehensively extracted and integrated to form a complete set of weight matrices. Then, the actual data features input to each weight matrix during model inference are used as the activation features of that weight matrix. By analyzing the distribution characteristics of the activation features, a specific scaling factor adapted to the weight matrix is calculated to balance the preservation of accuracy and compression efficiency during the quantization process. Subsequently, based on the scaling factor, each weight matrix is compressed matrix by matrix using a 4-bit quantization function. While converting the weight data from high-precision format to 4-bit integer format, the loss of semantic information and reasoning ability is minimized, resulting in a compressed lightweight weight matrix. Finally, all compressed weight matrices are integrated with the corresponding scaling factors to construct an INT4 inference kernel optimized for low-resource environments. This kernel can be directly deployed on edge devices such as learning machines, mobile apps, and educational smart hardware to achieve efficient inference operation of the model in low-memory and low-computing-power scenarios.
[0045] Preferably, the scaling factor for calculating the weight matrix includes:
[0046]
[0047] in, This represents the optimal scaling factor for the weight matrix; This represents the scaling factor of the weight matrix; Represents the weight matrix; Represents the activation characteristics of the weight matrix; This represents a 4-bit quantization function; This represents the inverse of the scaling factor; This represents the squared L2 norm.
[0048] In this embodiment, based on the extracted weight matrix W and its corresponding activation features X, the core objective is to find the optimal scaling factor s*, so that the product of the weights and activation features after processing by the 4-bit quantization function Q is as close as possible to the original calculation result before quantization, thereby minimizing the model performance loss caused by quantization. This is achieved by solving an optimization problem. This calculation process is performed separately for the activation feature distribution of each weight matrix, ensuring that the scaling factor is adapted to the semantic importance of the corresponding weights. It amplifies and protects the significant weight regions that carry key logical reasoning information, and reasonably compresses non-critical weight regions, providing accurate and adapted parameter support for subsequent 4-bit weight quantization, and ensuring that the model can still maintain high-precision mathematical misunderstanding detection and reordering capabilities after quantization.
[0049] S4: Sort the Top-K candidate misconceptions in descending order based on their relevance scores, and output the misconception with the highest score as the final identification result.
[0050] Another aspect of the present invention provides a multiple-choice question misunderstanding reordering system based on a causal large language model. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned multiple-choice question misunderstanding reordering method based on a causal large language model.
[0051] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for reordering misunderstandings in multiple-choice questions based on a causal large language model.
[0052] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0053] In summary, this invention obtains the query context containing multiple-choice questions and students' incorrect options, along with the Top-K candidate misunderstandings retrieved. The query context and each candidate misunderstanding are concatenated into a sequence to be judged and input into a fine-tuned causal big language model. Generative scoring or classification head scoring mechanisms are used to calculate relevance scores and complete the reordering. This approach leverages the powerful logical reasoning capabilities of the causal big language model to accurately distinguish mathematically difficult negative samples with similar textual expressions but vastly different logical meanings, effectively solving the problem of traditional ranking methods' difficulty in accurately identifying fundamental misunderstandings. This improves the accuracy and robustness of misunderstanding reordering. Furthermore, through model fine-tuning and optimization, along with subsequent quantization deployment support, it balances recognition accuracy with model operating efficiency. This provides an efficient and feasible solution for mathematical misunderstanding diagnosis in low-resource edge computing environments, facilitating the precise implementation of personalized tutoring in intelligent education scenarios.
[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for reordering misunderstandings in multiple-choice questions based on a causal large language model, characterized in that, include: S1: Obtain a set of candidate misunderstandings consisting of the Top-K candidate misunderstandings retrieved from the retrieval database based on the query context, wherein the query context includes: multiple-choice questions and the incorrect options selected by the student; S2: Concatenate the query context and each candidate misunderstanding separately to obtain multiple sets of sequences to be judged; S3: Input multiple sets of sequences to be discriminated into the fine-tuned causal big language model, and use generative scoring or classification head scoring mechanism to calculate the relevance score between each candidate misconception and the query context; S4: Sort the Top-K candidate misconceptions in descending order based on their relevance scores, and output the misconception with the highest score as the final identification result.
2. The method for reordering misunderstandings in multiple-choice questions based on a causal large language model according to claim 1, characterized in that, In step S2, if a generative scoring mechanism is used, the sequence to be judged includes: {multiple-choice questions, the incorrect options selected by the student, candidate misunderstandings, is the misunderstanding the cause of the error? Answer Yes or No}; if a classification head scoring mechanism is used, the sequence to be judged includes: {multiple-choice questions, the incorrect options selected by the student, candidate misunderstandings}.
3. The method for reordering misunderstandings in multiple-choice questions based on a causal large language model according to claim 2, characterized in that, Generative scoring is used to calculate the relevance score between each candidate misunderstanding and the query context, which includes: inputting the sequence to be discriminated into a causal large language model, extracting the EOS corresponding to the vocabulary projection layer. vector Locate the dimension value corresponding to the Yes Token. And calculate the relevance score of each candidate misunderstanding to the query context: in, A vocabulary set representing a causal large language model; Indicates the EOS corresponding vocabulary projection layer Vector in the th The dimension value corresponding to each word; Represents an exponential function; This represents the relevance score between the candidate misunderstanding and the query context.
4. The method for reordering misunderstandings in multiple-choice questions based on a causal large language model according to claim 2, characterized in that, The method of calculating the relevance score between each candidate misunderstanding and the query context using the classification head scoring mechanism includes: removing the vocabulary projection layer corresponding to the causal large language model EOS; and adjusting the hidden state vector corresponding to EOS. By mapping the linear layer to a one-dimensional scalar score, the relevance score between each candidate misconception and the query context is obtained: in, This represents the relevance score between the candidate misinterpretation and the query context; Indicates the bias parameter; Indicates the weighting parameter; This represents the Sigmoid activation function.
5. The method for reordering misunderstandings in multiple-choice questions based on a causal large language model according to claim 1, characterized in that, Fine-tuning of the causal large language model employs low-rank adaptation techniques, including: performing low-rank adaptation on the attention weights W of the causal large language model. in, The attention weights after low-rank adaptation; Attention weights before low-rank adaptation; and This represents the low-rank matrix being inserted; when fine-tuning the causal large language model, the parameters of the causal large language model are frozen, and only the low-rank matrix is fine-tuned. and The parameters.
6. A method for reordering misunderstandings in multiple-choice questions based on a causal large language model, as described in claim 1 or 5, characterized in that... The loss function used in the fine-tuned causal large language model includes: in, Represents the loss function; ; Represents an exponential function; Indicates query context The corresponding correct misunderstanding The relevance score; Indicates the number of candidate misunderstandings; Indicates query context With respect to correctness and misunderstanding The rest of the outside One candidate misunderstanding The relevance score.
7. The method for reordering misunderstandings in multiple-choice questions based on a causal large language model according to claim 1, characterized in that, The reordering method further includes: For the fine-tuned causal big language model, extract all the weight matrices of the causal big language model to construct a weight matrix set; The features of each input weight matrix are used as the activation features of that weight matrix, and the scaling factor of the weight matrix is calculated based on the activation features of that weight matrix. The weight matrix is compressed into a 4-bit form by combining the scaling factor with a 4-bit quantization function to obtain the compressed weight matrix. An INT4 inference kernel is then constructed based on the compressed weight matrix and the scaling factor and deployed to the edge device.
8. A method for reordering misunderstandings in multiple-choice questions based on a causal large language model, as described in claim 7, is characterized in that... The scaling factors for calculating the weight matrix include: in, This represents the optimal scaling factor for the weight matrix; This represents the scaling factor of the weight matrix; Represents the weight matrix; Represents the activation characteristics of the weight matrix; This represents a 4-bit quantization function; This represents the inverse of the scaling factor; This represents the squared L2 norm.
9. A multiple-choice question misunderstanding reordering system based on a causal large language model, characterized in that, The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute a multiple-choice question misunderstanding reordering method based on a causal large language model as described in any one of claims 1 to 8.
10. A computer storage medium, characterized in that, The computer storage medium stores a remote monitoring program, which, when executed by the processor, implements a multiple-choice question misunderstanding reordering method based on a causal large language model as described in any one of claims 1 to 8.