Multi-choice question error analysis retrieval method based on large language model and knowledge distillation

Abstract

The invention belongs to the technical field of machine learning, relates to a multi-choice question error analysis and retrieval method based on a large language model and knowledge distillation, and aims to solve the problems of high labeling cost, lack of hidden logical reasoning and weak fine misunderstanding and distinguishing ability of the existing method. The method comprises the following steps: firstly, acquiring a data set containing a topic text, error interference options and standard misunderstanding description, and generating a synthetic data supplementary sample through clustering and few-sample learning; correct options are extracted from the question text, the questions and error interference options are combined to be input into a second large language model, and error logic reasons are generated and spliced into semantic enhancement query description; constructing a positive and negative sample pair based on the query description and the corresponding standard misunderstanding description, and finely adjusting a second pre-training language model by adopting a low-rank adaptation technology; and finally, constructing a retrieval database, and recalling Top-K standard misunderstanding descriptions through vector similarity. According to the method, the data scarcity is effectively made up, the causal relationship behind the error is accurately captured, the training efficiency and the generalization ability are both considered, the retrieval precision ratio is remarkably improved, and reliable support is provided for personalized teaching feedback and learning condition control.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, and in particular relates to a method for error parsing and retrieval in multiple-choice questions based on large language models and knowledge distillation. Background Technology

[0002] Multiple-choice questions, as an efficient and convenient tool for assessing knowledge in educational evaluation, are widely used in teaching feedback and student learning diagnosis in subjects such as mathematics. Their carefully designed error-prone options often correspond to common student misunderstandings, calculation oversights, or logical deviations. Accurately identifying the underlying cognitive problems behind these incorrect options is key to achieving personalized teaching guidance, optimizing tutoring resource recommendations, and assisting teachers in accurately monitoring student learning progress. This has significant practical implications for improving the relevance and effectiveness of education and teaching.

[0003] Currently, retrieval methods for analyzing errors in multiple-choice questions mainly fall into two categories: one is the traditional manual annotation method, which relies on educational experts or teachers to manually analyze the correspondence between distractors and cognitive misunderstandings, and form a matching library of misunderstandings and incorrect options through experience; the other is the text matching method based on deep learning, which uses text embedding models such as BERT and Word2Vec to semantically encode the questions, incorrect options and misunderstanding descriptions, and realize misunderstanding retrieval through vector similarity calculation.

[0004] However, existing technologies have significant limitations: manual annotation is time-consuming and labor-intensive, difficult to adapt to massive question bank updates, and the annotation results are easily influenced by subjective judgment, resulting in insufficient consistency; while deep learning methods improve efficiency, they face problems in the field of mathematics such as the lack of implicit logical reasoning, scarcity of high-quality annotated data, and insufficient understanding of the mixed context of mathematical symbols and text, making it difficult to accurately distinguish subtle misunderstandings that are semantically similar but essentially different, resulting in the need to improve the precision and generalization ability of retrieval. Therefore, there is an urgent need for an intelligent retrieval method that can deeply understand mathematical logic, adapt to low-resource scenarios, and accurately capture the cognitive misunderstandings behind errors. Summary of the Invention

[0005] To address the problems existing in the background art, one aspect of the present invention provides a method for retrieving and parsing errors in multiple-choice questions based on a large language model and knowledge distillation, comprising:

[0006] S1: Obtain the multiple-choice question misunderstanding dataset, which includes: the question text of the multiple-choice question, multiple incorrect distractor options of the multiple-choice question, and a standard misunderstanding description of each incorrect distractor option;

[0007] S2: Encode the standard misunderstanding description using the first pre-trained language model to obtain the misunderstanding semantic vector; use the K-Means algorithm to cluster the misunderstanding semantic vector, calculate the average semantic distance within each cluster, and select clusters with a distance less than a set threshold as the seed set.

[0008] S3: Sample standard misconception descriptions from the seed set to construct a Few-Shot Prompt template, and use the first language model to generate new multiple-choice question misconception data based on the Few-Shot Prompt template, adding it to the multiple-choice question misconception dataset;

[0009] S4: For the multiple-choice question misleading dataset, extract the correct option from the question text of each multiple-choice question, and input the question text, the correct option, and the incorrect distractor option into the second language model to generate the logical reasons that led to the selection of the incorrect distractor option;

[0010] S5: Combine the question text of a multiple-choice question, the incorrect distractor options of the multiple-choice question, and the logical reasons for choosing the incorrect distractor options to form a semantically enhanced query description;

[0011] S6: Generate positive and negative sample pairs based on the semantically enhanced query description and its corresponding correct standard misinterpretation description, and fine-tune the second pre-trained language model using low-rank adaptation technique;

[0012] S7: Encode the standard misunderstanding descriptions in the multiple-choice misunderstanding dataset into retrieval vectors using a fine-tuned second pre-trained language model to construct a retrieval database;

[0013] S8: Generate semantically enhanced query descriptions based on the incorrect distractor options selected by students in multiple-choice questions, generated through steps S4 and S5. Input the query descriptions into the fine-tuned second pre-trained language model and encode them into query vectors. Calculate the similarity between the query vectors and the retrieval vectors in the retrieval database. Select the Top-K standard misinterpretations with the highest similarity to obtain the retrieval results.

[0014] Another aspect of the present invention provides a multiple-choice question error parsing and retrieval system based on a large language model and knowledge distillation. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned multiple-choice question error parsing and retrieval method based on a large language model and knowledge distillation.

[0015] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for retrieving and parsing errors in multiple-choice questions based on a large language model and knowledge distillation.

[0016] The present invention has at least the following beneficial effects

[0017] This invention addresses the scarcity of high-quality labeled data in existing technologies by acquiring a dataset containing question text, incorrect distracting options, and corresponding standard misunderstanding descriptions. It combines clustering, few-shot learning, and thought chain techniques to generate synthetic data, effectively reducing the cost of manual annotation. Through a knowledge distillation mechanism, a second language model generates logical reasons for errors and concatenates them into semantically enhanced query descriptions, compensating for the lack of implicit logical reasoning in traditional methods and helping to capture the deep causal relationships behind errors. Low-rank adaptation technology is used to fine-tune the second pre-trained language model, achieving efficient adaptation while balancing model generalization ability and training efficiency. Finally, by constructing a retrieval database and recalling the Top-K standard misunderstanding descriptions based on vector similarity, this invention overcomes the limitations of existing deep learning methods in distinguishing subtle misunderstandings in mathematical contexts, significantly improving the precision and efficiency of multiple-choice question error analysis retrieval. This provides reliable technical support for personalized teaching feedback, precise tutoring resource recommendations, and student learning monitoring. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0019] Figure 2 This is a schematic diagram of the model structure of the present invention. Detailed Implementation

[0020] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0021] Please see Figure 1 and Figure 2 One aspect of the present invention provides a method for retrieving errors in multiple-choice questions based on a large language model and knowledge distillation, comprising:

[0022] S1: Obtain the multiple-choice question misunderstanding dataset, which includes: the question text of the multiple-choice question, multiple incorrect distractor options of the multiple-choice question, and a standard misunderstanding description of each incorrect distractor option;

[0023] In this embodiment, the multiple-choice question misunderstanding dataset integrates publicly available educational datasets, historical answer logs from online education platforms, and specialized question banks annotated by subject matter experts. This ensures that the data covers multiple-choice questions across different educational levels and knowledge points. Each data entry must include the complete question text (including the stem and all options, clearly distinguishing between correct and incorrect options), multiple incorrect options verified through teaching practice (corresponding to common student cognitive misconceptions, not randomly generated), and a standard misunderstanding description annotated by subject matter experts based on teaching experience and cognitive patterns. This description must accurately point to the underlying cognitive problems behind the incorrect options (such as conceptual confusion, logical fallacies, calculation oversights, etc.), ensuring standardized and unambiguous expression. Furthermore, the structured organization of the dataset ensures unified and clearly correlated fields, providing high-quality basic data support for subsequent cluster analysis, synthetic data generation, and model training.

[0024] S2: Encode the standard misunderstanding description using the first pre-trained language model to obtain the misunderstanding semantic vector; use the K-Means algorithm to cluster the misunderstanding semantic vector, calculate the average semantic distance within each cluster, and select clusters with a distance less than a set threshold as the seed set.

[0025] Preferably, calculating the average semantic distance within each cluster includes:

[0026]

[0027] in, Indicates the first The average semantic distance within each cluster; Indicates the first The number of misinterpreted semantic vectors in each cluster; Indicates the first The first cluster of the cluster A misunderstanding semantic vector; Indicates the first The first cluster of the cluster A misunderstanding semantic vector; This represents the L2 norm.

[0028] In this embodiment, a first pre-trained language model adapted to deep text semantic encoding (such as BGE-Large, OpenAI text-embedding-3, etc.) is selected to transform each standard misunderstanding description in the multiple-choice question misunderstanding dataset into a high-dimensional misunderstanding semantic vector, ensuring that the vector can accurately capture the core cognitive features and semantic associations of the misunderstanding. Then, the K-Means algorithm is used to perform unsupervised clustering on all misunderstanding semantic vectors. The number of clusters is reasonably set according to the distribution characteristics of misunderstanding types in the dataset. Then, the average semantic distance within each cluster is obtained by calculating the average L2 norm (Euclidean distance) of all semantic vector pairs within each cluster. This distance directly reflects the semantic similarity of misunderstandings within the cluster. Finally, combined with the confusion frequency and data distribution characteristics of misunderstandings in actual teaching scenarios, a reasonable distance threshold is set, and clusters with an average semantic distance within the cluster less than the threshold are selected as seed sets. The misunderstandings within these clusters are easily confused by students due to their high semantic similarity, providing a precise target for the subsequent generation of highly recognizable synthetic data.

[0029] S3: Sample standard misconception descriptions from the seed set to construct a Few-Shot Prompt template, and use the first language model to generate new multiple-choice question misconception data based on the Few-Shot Prompt template, adding it to the multiple-choice question misconception dataset;

[0030] In this embodiment, 2-3 standard misunderstanding descriptions are randomly sampled from each selected seed set (a set of semantically highly similar and easily confused misunderstandings within a cluster). Combined with their corresponding original multiple-choice question structure, a Few-Shot Prompt template is constructed, including example demonstrations, generation logic, and output requirements. The template must explicitly guide the first major language model (GPT-3.5-turbo, GPT-4, etc.) to first reason about the question trap design ideas through a thought chain (how to make different incorrect options correspond to different misunderstandings in the seed set), and then output structured new data. For example, for the easily confused misunderstanding seed set of adding numerators and denominators directly versus adding denominators without finding a common denominator in fraction addition, the Prompt template can be designed as follows: The following are two sets of easily confused mathematical misunderstandings and their corresponding questions: Example 1—Misunderstanding A: In fraction addition, directly add the numerators and denominators; Question: Calculate 1 / 2 +1 / 3, Incorrect option A: 2 / 5 (corresponding to misunderstanding A), Correct option B: 5 / 6; Example 2—Misunderstanding B: Forgetting to find a common denominator when adding fractions, only adding the numerators and keeping the denominator unchanged, Question: Calculate 2 / 3 + 1 / 4, Incorrect option C: 3 / 3 (corresponding to misunderstanding B), Correct option D: 11 / 12. Please refer to the above examples to generate a new multiple-choice question on fraction addition, including the question stem, all options (at least 2 incorrect distractors, corresponding to misunderstanding A and misunderstanding B respectively), the correct option, and the standard misunderstanding description corresponding to each incorrect distractor, ensuring that the question trap can accurately distinguish between the two easily confused misunderstandings, and finally output it in the format of Question: XXX; Incorrect option and corresponding misunderstanding: XXX; Correct option: XXX. Then input this template into the first large language model to generate new multiple-choice question misunderstanding data that meets the requirements, and supplement it to the original dataset to enrich the distribution of hard samples.

[0031] S4: For the multiple-choice question misleading dataset, extract the correct option from the question text of each multiple-choice question, and input the question text, the correct option, and the incorrect distractor option into the second language model to generate the logical reasons that led to the selection of the incorrect distractor option;

[0032] Preferably, the logical reasons generated by the second large language model that lead to the selection of incorrect interference options include:

[0033]

[0034] in, Express the optimal logical reason; This represents all the potential logical reasons that the second largest language model may generate; The text representing the multiple-choice question; Indicates the correct options in a multiple-choice question; Indicates incorrect or distracting options in multiple-choice questions; This represents the model parameters of the second largest language model.

[0035] In this embodiment, the logical reasoning generated by the second large language model that leads to the selection of incorrect distracting options is similar to the generation of new multiple-choice question misinterpretation data in step S3. Both involve constructing a Few-Shot Prompt template to guide the output of the large language model. Specifically, text parsing technology is first used to accurately extract the unique correct option from the multiple-choice question text, ensuring that the correct option is consistent with the question and the incorrect distracting options. Then, a second large language model with strong logical reasoning and causal analysis capabilities (such as GPT-4, Qwen-72B, etc.) is selected to generate the complete question text Q + the correct option. +Target error interference option As input, a Few-Shot Prompt template containing example demonstrations and generation rules is constructed to guide the model to focus on the essence of the error and generate precise logical reasons that include conceptual confusion, calculation omissions, or logical breaks. For example, for problems involving solving linear equations in one variable, the Prompt template can be designed as follows: Please analyze the core logical reasons why students choose the wrong option, requiring the reasons to accurately point to cognitive misconceptions rather than vaguely describing the error: Example 1 - Question Q: Solve the equation 3x-5=7, option A: 4 (correct, Option B: 3 (Incorrect) Logical reasoning R*: The student failed to change -5 to +5 when rearranging terms, incorrectly calculating 3x = 7 - 5 = 2, thus solving for x = 3; Example 2 – Question Q: Solve the equation 2(x + 4) = 14, option A: 3 (correct, Option B: 5 (Incorrect) The logical reasoning is as follows: The student failed to apply the distributive property of multiplication when removing parentheses, incorrectly obtaining 2x + 4 = 14, which leads to x = 5. This overlooks the multiplication of 2 and 4. Please refer to the example above and address the specific problem. Solve the equation 5x + 8 = 23. Option A: 3 (Correct) Option B: 6.2 (Error) This generates the optimal logical reasoning for a student's incorrect choice, option B, requiring it to be concise, clear, and directly address cognitive misconceptions. This template drives the second language model to output logical reasons strongly tied to the incorrect option, providing core support for subsequent semantic enhancement.

[0036] S5: Combine the question text of a multiple-choice question, the incorrect distractor options of the multiple-choice question, and the logical reasons for choosing the incorrect distractor options to form a semantically enhanced query description;

[0037] In this embodiment, according to the title text +Error interference items +Optimal logical reasoning The query description is structured and assembled in a fixed order to form a semantically enhanced query description; first, the complete context of the question and the core information of the student's incorrect options are preserved, and then the optimal logical reasoning is added. The included causal logic of errors ensures that query descriptions not only cover surface textual information but also contain the deeper cognitive logic behind the errors, effectively overcoming the limitations of traditional queries that rely solely on literal matching. At the same time, the standardized format avoids redundant information interference, ensuring that each query description accurately links the complete chain of question scenario, incorrect selection, and cognitive misconception. This provides a logically coherent and information-rich input foundation for subsequent generation of positive and negative sample pairs, model fine-tuning, and semantic vector encoding, helping the model learn the strong correlation features between errors and misunderstandings.

[0038] S6: Generate positive and negative sample pairs based on the semantically enhanced query description and its corresponding correct standard misinterpretation description, and fine-tune the second pre-trained language model using low-rank adaptation technique;

[0039] Preferably, generating positive and negative sample pairs includes: adding semantically enhanced query descriptions. And its corresponding correct standard misunderstanding description As a positive cost pair, it is described according to its corresponding correct standard misunderstanding. The misunderstanding semantic vector is retrieved and compared with the misunderstanding semantic vector. Construct a set of hard-to-bear samples using the top-K most similar standard misinterpretations. ; enhance the semantic description of queries With difficult samples As negative sample pairs, positive and negative sample pairs are generated.

[0040] Preferably, fine-tuning the second pre-trained language model using low-rank adaptation technology includes: performing low-rank adaptation on the attention weights W of the second pre-trained language model.

[0041]

[0042] in, The attention weights after low-rank adaptation; Attention weights before low-rank adaptation; and This represents the low-rank matrix to be inserted; when fine-tuning the second pre-trained language model, the parameters of the second pre-trained language model are frozen, and only the low-rank matrix is ​​fine-tuned. and The parameters.

[0043] Preferably, the loss function used for fine-tuning the second pre-trained language model includes:

[0044]

[0045] in, Represents the InfoNCE loss function; Indicates the number of samples; and Indicates the first The and the first A semantically enhanced query description; and This indicates that the second pre-trained language model is... and Encoded query vector; and express and The corresponding correct standard is a misinterpretation; and This indicates that the second pre-trained language model is... and Encoded retrieval vector; Represents the cosine similarity function; Represents an exponential function; Indicates the temperature coefficient; express The corresponding set of hard-to-bear samples; This indicates that the second pre-trained language model handles difficult samples. Encoded retrieval vector; This represents the weighting coefficient.

[0046] In this embodiment, a semantically enhanced query description is first used. The corresponding correct standard misconception description Construct positive sample pairs, and then based on The misunderstanding semantic vectors are used to select the Top-K standard misunderstanding descriptions with the most similar semantics through similarity retrieval, forming a hard-burden sample set. and with With difficult samples Negative sample pairs are constructed to form positive and negative sample pairs that balance effective signals and discriminative power. Subsequently, for the second pre-trained language model, low-rank adaptation technology is used to insert low-rank matrices A and B into its attention weight layer. During fine-tuning, the original parameters of the model are frozen and only the parameters of matrices A and B are updated. At the same time, the InfoNCE contrastive loss function with hard negative sample mining is introduced. The temperature coefficient τ is used to adjust the smoothness of the similarity distribution and the weight coefficient α is used to balance the contribution of hard negative samples. This forces the model to learn the strong correlation features between query description and correct misunderstanding and the semantic differences between query description and hard negative samples, thereby optimizing the vector space distribution and improving the model's ability to distinguish subtle misunderstandings.

[0047] Preferably, for the second pre-trained language model, a last-term pooling strategy is used to obtain the sentence representation vector. While keeping the causal masking mechanism of the large language model unchanged, the hidden state vector corresponding to the last valid word in the input sequence is extracted from the output of the last hidden layer of the second pre-trained language model using the unidirectional attention characteristic of the decoder architecture. The vector is then normalized to serve as the vector encoded by the input sentence of the second pre-trained language model.

[0048] For sentence representation vector extraction of the second pre-trained language model, a differentiated strategy adapted to the model architecture is adopted to ensure the accuracy of semantic capture: If the model is a Transformer-based encoder architecture (such as BERT, BGE-Large, etc.), the CLS tokens designed specifically for sentence-level semantic aggregation are used to extract the hidden state vectors corresponding to the CLS tokens in the last encoding layer. After normalization, these vectors are used as sentence representation vectors to fully aggregate the overall semantic information of the input text. If the model is a decoder-only architecture (such as LLaMA2, Qwen-72B, etc.), the last-term pooling strategy is continued to maintain the inherent causal masking mechanism of the model. Relying on its unidirectional attention characteristic that can only focus on the preceding information, the last valid word in the input sequence (excluding filler words) is accurately located. The hidden state vector corresponding to this word in the last hidden layer is extracted. After normalization to eliminate the difference in dimensions, it is used as the final sentence representation vector. This ensures that the semantic encoding of different architecture models can adapt to the subsequent vector similarity calculation and misunderstanding retrieval requirements, guaranteeing the consistency and effectiveness of representation.

[0049] S7: Encode the standard misunderstanding descriptions in the multiple-choice misunderstanding dataset into retrieval vectors using a fine-tuned second pre-trained language model to construct a retrieval database;

[0050] In this embodiment, all standard misunderstanding descriptions annotated by subject matter experts are first extracted from the multiple-choice question misunderstanding dataset (including raw and synthetic data). These descriptions are then input one by one into a second pre-trained language model fine-tuned using low-rank adaptation technology. Following a sentence representation strategy adapted to the model architecture, retrieval vectors with uniform dimensions and precise semantics are extracted and normalized to ensure that the vectors can fully capture the core cognitive features and logical connotations of each standard misunderstanding. Subsequently, all standard misunderstanding descriptions are structurally associated with their corresponding retrieval vectors. A retrieval database is constructed using efficient vector storage and indexing tools (such as Faiss). The database must support fast approximate nearest neighbor search while ensuring data integrity and consistency, providing an efficient and comprehensive vector retrieval foundation for real-time misunderstanding recall after receiving student answer records.

[0051] S8: Generate semantically enhanced query descriptions based on the incorrect distractor options selected by students in multiple-choice questions, generated through steps S4 and S5. Input the query descriptions into the fine-tuned second pre-trained language model and encode them into query vectors. Calculate the similarity between the query vectors and the retrieval vectors in the retrieval database. Select the Top-K standard misinterpretations with the highest similarity to obtain the retrieval results.

[0052] In this embodiment, when the student's multiple-choice question answer record (including the complete question text) is obtained... and the incorrect interference items selected by the students After that, follow the process in step S4 from... Extract the correct option ,Will , and The second largest language model is input to generate the corresponding error logic reasons. Then, following the fixed format of step S5, a semantically enhanced query description is formed. Subsequently, this query description is input into the fine-tuned second pre-trained language model. Based on the model architecture, an adapted sentence representation strategy is used to encode the query vector. The approximate nearest neighbor search algorithm is called through the vector indexing tool to calculate the cosine similarity between the query vector and all standard misunderstanding retrieval vectors in the retrieval database. After sorting the similarity from high to low, the Top-K standard misunderstanding descriptions (K value can be set to 1-5 according to the needs of the teaching scenario) are selected as the final retrieval results. This achieves accurate positioning of the deep cognitive misunderstandings behind students' errors, providing a direct basis for subsequent personalized teaching feedback and tutoring guidance.

[0053] Another aspect of the present invention provides a multiple-choice question error parsing and retrieval system based on a large language model and knowledge distillation. The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the aforementioned multiple-choice question error parsing and retrieval method based on a large language model and knowledge distillation.

[0054] Another aspect of the present invention provides a computer storage medium storing a remote monitoring program, which, when executed by a processor, implements the aforementioned method for retrieving and parsing errors in multiple-choice questions based on a large language model and knowledge distillation.

[0055] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0056] In summary, this invention effectively addresses the scarcity of high-quality labeled data in existing technologies by acquiring a dataset containing question text, incorrect distracting options, and corresponding standard misunderstanding descriptions. It combines clustering, few-shot learning, and thought chain techniques to generate synthetic data, reducing the cost of manual annotation. Through a knowledge distillation mechanism, a second language model generates logical reasons for errors and concatenates them into semantically enhanced query descriptions, compensating for the lack of implicit logical reasoning in traditional methods and helping to capture the deep causal relationships behind errors. Low-rank adaptation technology is used to fine-tune the second pre-trained language model, achieving efficient adaptation while balancing model generalization ability and training efficiency. Finally, by constructing a retrieval database and recalling the Top-K standard misunderstanding descriptions based on vector similarity, this invention overcomes the limitation of existing deep learning methods in distinguishing subtle misunderstandings in mathematical contexts, significantly improving the precision and efficiency of multiple-choice question error analysis retrieval. This provides reliable technical support for personalized teaching feedback, precise tutoring resource recommendations, and student learning monitoring.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for error parsing and retrieval in multiple-choice questions based on a large language model and knowledge distillation, characterized in that, include: S1: Obtain the multiple-choice question misunderstanding dataset, which includes: the question text of the multiple-choice question, multiple incorrect distractor options of the multiple-choice question, and a standard misunderstanding description of each incorrect distractor option; S2: Encode the standard misunderstanding description using the first pre-trained language model to obtain the misunderstanding semantic vector; use the K-Means algorithm to cluster the misunderstanding semantic vector, calculate the average semantic distance within each cluster, and select clusters with a distance less than a set threshold as the seed set. S3: Sample standard misunderstanding descriptions from the seed set to construct a Few-Shot Prompt template, and use the first language model to generate new multiple-choice misunderstanding data based on the Few-Shot Prompt template, adding it to the multiple-choice misunderstanding dataset; S4: For the multiple-choice question misleading dataset, extract the correct option from the question text of each multiple-choice question, and input the question text, the correct option, and the incorrect distractor option into the second language model to generate the logical reasons that led to the selection of the incorrect distractor option; S5: Combine the question text of a multiple-choice question, the incorrect distractor options of the multiple-choice question, and the logical reasons for choosing the incorrect distractor options to form a semantically enhanced query description; S6: Generate positive and negative sample pairs based on the semantically enhanced query description and its corresponding correct standard misinterpretation description, and fine-tune the second pre-trained language model using low-rank adaptation technique; S7: Encode the standard misunderstanding descriptions in the multiple-choice misunderstanding dataset into retrieval vectors using a fine-tuned second pre-trained language model to construct a retrieval database; S8: Generate semantically enhanced query descriptions based on the incorrect distractor options selected by students in multiple-choice questions, generated through steps S4 and S5. Input the query descriptions into the fine-tuned second pre-trained language model and encode them into query vectors. Calculate the similarity between the query vectors and the retrieval vectors in the retrieval database. Select the Top-K standard misinterpretations with the highest similarity to obtain the retrieval results.

2. The method for retrieving errors in multiple-choice questions based on a large language model and knowledge distillation as described in claim 1, characterized in that, The calculation of the intra-cluster average semantic distance for each cluster includes: in, Indicates the first The average semantic distance within each cluster; Indicates the first The number of misinterpreted semantic vectors in each cluster; Indicates the first The first cluster of the cluster A misunderstanding semantic vector; Indicates the first The first cluster of the cluster A misunderstanding semantic vector; This represents the L2 norm.

3. The method for error analysis and retrieval in multiple-choice questions based on a large language model and knowledge distillation as described in claim 1, characterized in that, The logical reasons why the second largest language model generates incorrect interference options include: in, Express the optimal logical reason; This represents all the potential logical reasons that the second largest language model may generate; The text representing the multiple-choice question; Indicates the correct options in a multiple-choice question; Indicates incorrect or distracting options in multiple-choice questions; This represents the model parameters of the second largest language model.

4. The method for error analysis and retrieval in multiple-choice questions based on a large language model and knowledge distillation as described in claim 1, characterized in that, The generation of positive and negative sample pairs includes: semantically enhanced query descriptions. And its corresponding correct standard misunderstanding description As a positive cost pair, it is described according to its corresponding correct standard misunderstanding. The misunderstanding semantic vector is retrieved and compared with the misunderstanding semantic vector. Construct a set of hard-to-bear samples using the top-K most similar standard misinterpretations. ; enhance the semantic description of queries With difficult samples As negative sample pairs, generate positive and negative sample pairs.

5. The method for retrieving errors in multiple-choice questions based on a large language model and knowledge distillation as described in claim 1, characterized in that, Fine-tuning the second pre-trained language model using low-rank adaptation involves performing low-rank adaptation on the attention weights W of the second pre-trained language model. in, The attention weights after low-rank adaptation; Attention weights before low-rank adaptation; and This represents the low-rank matrix to be inserted; when fine-tuning the second pre-trained language model, the parameters of the second pre-trained language model are frozen, and only the low-rank matrix is ​​fine-tuned. and The parameters.

6. The method for retrieving errors in multiple-choice questions based on a large language model and knowledge distillation as described in claim 4, characterized in that, The loss function used for fine-tuning the second pre-trained language model includes: in, Represents the InfoNCE loss function; Indicates the number of samples; and Indicates the first The and the first A semantically enhanced query description; and This indicates that the second pre-trained language model is... and Encoded query vector; and express and The corresponding correct standard is a misinterpretation; and This indicates that the second pre-trained language model is... and Encoded retrieval vector; Represents the cosine similarity function; Represents an exponential function; Indicates the temperature coefficient; express The corresponding set of hard-to-bear samples; This indicates that the second pre-trained language model handles difficult samples. Encoded retrieval vector; This represents the weighting coefficient.

7. The method for retrieving errors in multiple-choice questions based on a large language model and knowledge distillation as described in claim 6, characterized in that, For the second pre-trained language model, a last-term pooling strategy is adopted to obtain the sentence representation vector. While keeping the causal masking mechanism of the large language model unchanged, the hidden state vector corresponding to the last valid word in the input sequence is extracted from the output of the last hidden layer of the second pre-trained language model using the unidirectional attention characteristic of the decoder architecture. The vector is then normalized to serve as the vector encoded by the input sentence of the second pre-trained language model.

8. A multiple-choice question error analysis and retrieval system based on large language models and knowledge distillation, characterized in that, The system includes a memory and a processor; the memory is used to store an application program; the processor is used to run the application program and execute the multiple-choice question error parsing retrieval method based on a large language model and knowledge distillation as described in any one of claims 1 to 7.

9. A computer storage medium, characterized in that, The computer storage medium stores a remote monitoring program, which, when executed by the processor, implements a multiple-choice question error parsing and retrieval method based on a large language model and knowledge distillation as described in any one of claims 1 to 7.

Images