A large model-based test question answer analysis system

Abstract

The application relates to the technical field of large models, in particular to a test question answer analysis system based on a large model, which comprises a test question collecting module used for receiving a target test question input by a user; a matching module used for identifying a subject type to which the target test question belongs through a routing model and assigning a corresponding retrieval model; a knowledge base module used for storing test question data of different subject types; an auxiliary retrieval module used for obtaining embedded expression of the target test question through the retrieval model, extracting auxiliary data from the knowledge base according to the embedded expression through a recall method; a test question analysis module used for inputting the target test question and the auxiliary data into a test question analysis model to obtain a target test question analysis result; and a display module used for displaying the target test question analysis result; the application can provide high-quality learning support for the user.

Description

Technical Field

[0001] This invention relates to the field of large-scale model technology, and specifically to a test question answering and analysis system based on large-scale models. Background Technology

[0002] Test answer analysis systems are a hot research topic in online education. These systems can output the correct answers and reasoning processes based on user-input test questions, providing users with detailed analysis to aid their learning. Traditional test answer analysis systems primarily rely on manually compiled question banks or simple keyword matching techniques. While these methods meet basic needs to some extent, they have significant limitations. First, manually compiled question banks are slow to update and struggle to cover all subjects and question types. Second, traditional matching techniques lack semantic understanding, easily leading to mismatches or missed matches, thus failing to provide high-quality answers. Furthermore, traditional methods typically cannot provide customized recommendations based on users' individual needs, resulting in a poor user experience.

[0003] Large Language Models (LLMs), also known as large-scale language models or big models, are artificial intelligence models designed to understand and generate human language. They are trained on large amounts of text data, possess powerful semantic understanding and reasoning capabilities, and can perform a wide range of natural language tasks, including text summarization, translation, and sentiment analysis. While existing large language model technologies have made significant progress, in the field of test question answering, test question answering systems still face considerable challenges in terms of accuracy and answer quality. Summary of the Invention

[0004] To address the above problems, this invention provides a test question answering and analysis system based on a large model, comprising:

[0005] The question acquisition module is used to receive target questions input by the user;

[0006] The matching module includes a routing model, which is used to identify the subject type of the target test question and assign the corresponding retrieval model.

[0007] The knowledge base module includes knowledge bases corresponding to different subject types, used to store test question data for different subject types; the test question data includes test questions, answers to test questions, and problem-solving approaches for test questions; it also stores the embedded expression of each test question;

[0008] The auxiliary retrieval module includes retrieval models for different subjects, which are used to obtain the embedded expression of the target test question through the retrieval model, and to extract auxiliary data from the knowledge base based on the embedded expression using a recall method;

[0009] The test question analysis module includes a test question analysis model, which is used to obtain the target test question analysis result by inputting the target test question and auxiliary data into the test question analysis model. The target test question analysis result includes the answer and problem-solving approach of the target test question. The target test question, the target test question analysis result, and the embedded expression of the target test question are put into the corresponding knowledge base. The test question analysis model is obtained through the large-scale model knowledge distillation method.

[0010] The display module is used to display the analysis results of the target test questions.

[0011] The beneficial effects of this invention are:

[0012] This invention uses data augmentation and model distillation techniques to build dynamically updated knowledge bases for different subject types. This not only covers a wide range of subjects and question types, but also generates new explanations and answers in real time according to user needs, significantly improving the richness and accuracy of the question base, and significantly improving the reasoning ability and deployment efficiency of the model.

[0013] This invention optimizes the routing and retrieval models using Dynamic Balance Loss (DBL) and Subject Comparison Loss (DCL), respectively, achieving more accurate subject classification and question retrieval. This personalized, customized approach based on a large model can provide targeted answers and recommendations according to the user's specific needs, greatly enhancing the user experience. For example, after a user inputs a question, the system can not only quickly classify it into the correct subject but also retrieve the most similar questions and answers. Combining this with the reasoning capabilities of the large model, it generates the final answer, improving service quality and optimizing the user experience.

[0014] This invention uses two-dimensional fusion distance for clustering and recall, ensuring efficient retrieval of the knowledge base and reliable reasoning results. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the system module connections of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This invention provides a test question answering and analysis system based on a large model, such as... Figure 1 As shown, it includes:

[0018] The question acquisition module is used to receive target questions input by the user;

[0019] The matching module includes a routing model, which is used to identify the subject type of the target test question and assign the corresponding retrieval model.

[0020] The knowledge base module includes knowledge bases corresponding to different subject types, used to store test question data for different subject types; the test question data includes test questions, answers to test questions, and problem-solving approaches for test questions; it also stores the embedded expression of each test question;

[0021] The auxiliary retrieval module includes retrieval models for different subjects, which are used to obtain the embedded expression of the target test question through the retrieval model, and to extract auxiliary data from the knowledge base based on the embedded expression using a recall method;

[0022] The test question analysis module includes a test question analysis model, which is used to obtain the target test question analysis result by inputting the target test question and auxiliary data into the test question analysis model. The target test question analysis result includes the answer and problem-solving approach of the target test question. The target test question, the target test question analysis result, and the embedded expression of the target test question are put into the corresponding knowledge base. The test question analysis model is obtained through the large-scale model knowledge distillation method.

[0023] The display module is used to display the analysis results of the target test questions.

[0024] Specifically, the subject types involved in the embodiments of the present invention include seven types: advanced mathematics, linear algebra, probability theory, computer networks, data structures, operating systems, and computer organization principles.

[0025] Specifically, the knowledge base initialization for any subject type includes...

[0026] Collect a set of test questions belonging to the current subject type, wherein the test question set includes multiple test question data;

[0027] For each test question, input the test question from the test question data into the 110B large model (i.e., a large model with 110 billion parameters) to obtain a new answer and a new problem-solving approach. If the new answer is the same as the answer in the test question data, and the new problem-solving approach is different from the problem-solving approach in the test question data, then combine the test question, the answer, and the new problem-solving approach into a new test question data.

[0028] Add all test data to the knowledge base for the current subject type.

[0029] Specifically, the process of obtaining the test question analysis model through the large-scale model knowledge distillation method includes:

[0030] S11. Fine-tune the 32B large model (i.e., a large model with 32 billion parameters) based on the test question data in the knowledge base module, so that the 32B large model can take test questions as input and the answers and solutions of the test questions as output;

[0031] S12. Input the question into the 7B large model (i.e., the large model with 7 billion parameters) to get the first answer and the first solution approach. Input the same question into the finely tuned 32B large model to get the second answer and the second solution approach.

[0032] S13. Calculate the answer similarity loss (ASL) based on the first and second answers, denoted as...

[0033]

[0034] Where N represents the length of the answer token. If the lengths of the two answers are different, N is the larger length, and the shorter answer will be padded to N through special token padding. Let i represent the vector representation of the i-th token in the first answer. Let p represent the vector representation of the i-th token in the second answer. small，a p represents the probability distribution generated by the 7B large model. large，a This represents the probability distribution generated by the 32B large model, and KL(·) represents the KL divergence operation;

[0035] S14. Calculate the inference path loss RPL based on the first and second problem-solving approaches, expressed as follows:

[0036]

[0037] Where M represents the maximum length of the solution token. Let represent the vector representation of the i-th token in the first solution approach. Let p represent the vector representation of the i-th token in the second solution approach. small，t p represents the probability distribution generated by the 7B large model. large，t Π(p) represents the probability distribution generated by the 32B large model, and Wasserstein(·) represents the Wasserstein distance; small，t p large，t Let p represent the set of all possible joint distributions, with marginal distributions p. small，t and p large，t γ(i,j) is a joint distribution, representing the distribution from p small，t The i-th point is transferred to p. large，t The probability of the j-th point, d(i,j) represents p small，t The transition of the i-th point to p large，t The Euclidean distance between the j-th points; since the text length of the solution approach is longer than the text length of the answer, using Wasserstein can provide better robustness.

[0038] S15. Perform knowledge distillation on the 7B large model based on answer similarity loss (ASL) and reasoning path loss (RPL). Repeat steps S12-S15 until the parameters of the 7B large model converge to obtain the test item parsing model.

[0039] Specifically, the routing model includes a backbone module, intermediate dimension fully connected layers, and classification fully connected layers. The training process of the routing model includes:

[0040] S21. Obtain the test question label dataset, where each test question label dataset includes a test question and its corresponding subject type label;

[0041] S22. Input the test questions into the backbone module to obtain the embedding vector, where the backbone module uses a pre-trained BERT model;

[0042] S23. Input the embedding vector into the intermediate dimension fully connected layer. The intermediate dimension fully connected layer maps the embedding vector to a lower dimension to obtain the intermediate dimension vector; the dimension z of the intermediate dimension vector is calculated as follows:

[0043]

[0044] Where, d bert L represents the number of dimensions of the embedding vector. bert H represents the number of layers in the BERT model. bert This indicates the number of multi-head attention heads in the BERT model;

[0045] S24. Input the intermediate dimension vector into the classification fully connected layer to map the output onto 7 probabilities to obtain the predicted label;

[0046] S25. Calculate the DBL loss and train the model parameters based on the DBL loss until the model parameters converge.

[0047] Specifically, the DBL loss calculation formula is as follows:

[0048] L DBL =L CE *exp(L triplet )+L triplet *exp(L CE )

[0049]

[0050] L triplet =max(0, ||f(x)-f(x)) + )|| 2 -||f(x)-f(x - )|| 2 )

[0051] Among them, L DBL L represents the DBL loss. CE Indicates the first loss, L triplet Indicates the second loss; y i p represents the true label of the sample. i Let f(x) represent the probability distribution of the sample obtained through the routing model, and let f(x) represent the embedding vector of the sample. + f(x) represents the embedding vector of another sample belonging to the same subject type as the sample. - ) represents the embedding vector of another sample belonging to a different subject type than the sample. This loss function enhances intra-class compactness and inter-class separation, improving the model's discriminative ability.

[0052] Specifically, the process of obtaining the retrieval model corresponding to any subject type includes:

[0053] S31. Obtain the training dataset, which includes multiple sample pairs, each sample pair consisting of one question and one difficult question; the process of obtaining one sample pair is as follows:

[0054] Obtain the knowledge points tested in the test questions, construct an n-gram similar to the test questions but with different knowledge points tested as a hard sample, and combine the test questions and the hard sample into a sample pair;

[0055] S32. The BGE model is trained using the training dataset, and the loss is calculated using the DCL loss function to finally obtain the retrieval model; the formula for the DCL loss function is:

[0056]

[0057] Where, N all f(x) represents the number of sample pairs. i f'(x) represents the embedded representation of the item in the i-th sample pair obtained by the bge model. i ) represents f(x) i The result after dropout, f(x) i ) - Let L represent the embedded representation of the hard sample in the i-th sample pair obtained by the BGE model. DCL The loss maximizes the similarity between the test item and the positive sample, while maximizing the dissimilarity between the test item and the hard sample (negative sample), thereby bringing the positive sample closer and pushing the hard sample further away in the embedding space.

[0058] Specifically, the recall method used to extract auxiliary data from the knowledge base based on embedded representations includes:

[0059] S41. Based on embedded representation, cluster all test question data in the knowledge base corresponding to the subject type of the target test question by using two-dimensional fusion distance to obtain multiple cluster sets;

[0060] S42. Calculate the two-dimensional fusion distance between the embedded expression of the target test item and the embedded expression of the centroid of each cluster set, and take the cluster set with the smallest two-dimensional fusion distance as the target set;

[0061] S43. Calculate the two-dimensional fusion distance between the embedding expression of the target test item and the embedding expression of each test item data in the target set. Sort all the two-dimensional fusion distances in ascending order of size and select the test item data corresponding to the first two two-dimensional fusion distances as auxiliary data.

[0062] Specifically, the formula for calculating the two-dimensional fusion distance is:

[0063]

[0064] Where D represents the two-dimensional fusion distance, A and B represent the embedding expressions of two different questions, n represents the number of dimensions of the embedding expression, and A i Let B represent the vector of the i-th dimension in the embedding expression A. i This represents the vector of the i-th dimension in the embedding expression B.

[0065] Specifically, the display module also displays auxiliary data obtained by the auxiliary retrieval module. In particular, the auxiliary data consists of two test questions that are most similar to the target test question. The two test questions in the auxiliary data can be displayed to recommend similar test questions to the user, realize personalized recommendations, and guide the user to further learn and consolidate the same knowledge points.

[0066] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "setting," "connection," "fixing," "rotation," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal connection of two components or the interaction between two components. Unless otherwise explicitly limited, those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A test question answering and analysis system based on a large model, characterized in that, include: The question acquisition module is used to receive target questions input by the user; The matching module includes a routing model, which is used to identify the subject type of the target test question and assign the corresponding retrieval model. The knowledge base module includes knowledge bases corresponding to different subject types, used to store test question data for different subject types; the test question data includes test questions, answers to test questions, and problem-solving approaches for test questions; it also stores the embedded expression of each test question; The auxiliary retrieval module includes retrieval models for different subjects, which are used to obtain the embedded expression of the target test question through the retrieval model, and to extract auxiliary data from the knowledge base based on the embedded expression using a recall method; The test question analysis module includes a test question analysis model, which is used to obtain the target test question analysis result by inputting the target test question and auxiliary data into the test question analysis model; the target test question analysis result includes the answer and problem-solving ideas of the target test question; and the target test question, the target test question analysis result, and the embedded expression of the target test question are put into the corresponding knowledge base; The test question analysis model was obtained through the large-scale model knowledge distillation method. The process of obtaining a test question analysis model through the large-scale model knowledge distillation method includes: S11. Fine-tune and train the 32B large model based on the test question data in the knowledge base module, so that the 32B large model can take test questions as input and the answers and solutions of the test questions as outputs. S12. Input the question into the 7B large model to get the first answer and the first solution approach; input the same question into the finely tuned 32B large model to get the second answer and the second solution approach. S13. Calculate the answer similarity loss (ASL) based on the first and second answers, denoted as... Where N represents the length of the answer token. Let i represent the vector representation of the i-th token in the first answer. Let p represent the vector representation of the i-th token in the second answer. small,a p represents the probability distribution generated by the 7B large model. large,a This represents the probability distribution generated by the 32B large model, and KL(·) represents the KL divergence operation; S14. Calculate the inference path loss RPL based on the first and second problem-solving approaches, expressed as follows: Where M represents the maximum length of the solution token. Let represent the vector representation of the i-th token in the first solution approach. Let p represent the vector representation of the i-th token in the second solution approach. small,t p represents the probability distribution generated by the 7B large model. large,t Let Π(p) represent the probability distribution generated by the 32B large model, and Wasserstein(·) represent the Wasserstein distance. small,t ,p large,t Let ) denote the joint distribution set, and γ(i,j) denote the distribution from p. small,t The i-th point is transferred to p. large,t The probability of the j-th point, where d(i,j) represents the Euclidean distance; S15. Perform knowledge distillation on the 7B large model based on the answer similarity loss (ASL) and reasoning path loss (RPL). Repeat steps S12-S15 until the parameters of the 7B large model converge to obtain the test question parsing model. The display module is used to display the analysis results of the target test questions.

2. The test question answering and analysis system based on a large model according to claim 1, characterized in that, Knowledge base initialization for any subject type includes Collect a set of test questions belonging to the current subject type, wherein the test question set includes multiple test question data; For each test question, the test question in the test question data is input into the 110B large model to obtain a new answer and a new problem-solving approach. If the new answer is the same as the answer in the test question data, and the new problem-solving approach is different from the problem-solving approach in the test question data, then the test question, answer and new problem-solving approach are combined into a new test question data. Add all test data to the knowledge base for the current subject type.

3. The test question answering and analysis system based on a large model according to claim 1, characterized in that, The routing model consists of a backbone module, intermediate fully connected layers, and a classification fully connected layer. The training process of the routing model includes: S21. Obtain the test question label dataset, where each test question label dataset includes a test question and its corresponding subject type label; S22. Input the test questions into the backbone module to obtain the embedding vector, where the backbone module uses a pre-trained BERT model; S23. Input the embedding vector into the intermediate dimension fully connected layer. The intermediate dimension fully connected layer maps the embedding vector to a lower dimension to obtain the intermediate dimension vector; the dimension z of the intermediate dimension vector is calculated as follows: Where, d bert L represents the number of dimensions of the embedding vector. bert H represents the number of layers in the BERT model. bert This indicates the number of multi-head attention heads in the BERT model; S24. Input the intermediate dimension vector into the classification fully connected layer to obtain the predicted label; S25. Calculate the DBL loss and train the model parameters based on the DBL loss until the model parameters converge.

4. The test question answering and analysis system based on a large model according to claim 3, characterized in that, The formula for calculating DBL loss is: L DBL L CE *exp(L triplet )+L triplet *exp(L CE ) L triplet =max(0,||f(x)-f(x+)|| 2 -||f(x)-f(x - )|| 2 ) Among them, L DBL L represents the DBL loss. CE Indicates the first loss, L triplet Indicates the second loss; y i p represents the true label of the sample. i Let f(x) represent the probability distribution of the sample obtained through the routing model, and let f(x) represent the embedding vector of the sample. + f(x) represents the embedding vector of another sample belonging to the same subject type as the sample. - ) represents the embedding vector of another sample that belongs to a different subject type than the sample.

5. The test question answering and analysis system based on a large model according to claim 1, characterized in that, The process of obtaining the retrieval model corresponding to any subject type includes: S31. Obtain the training dataset, which includes multiple sample pairs, each sample pair consisting of one question and one difficult question; the process of obtaining one sample pair is as follows: Obtain the knowledge points tested in the test questions, construct an n-gram similar to the test questions but with different knowledge points tested as a hard sample, and combine the test questions and the hard sample into a sample pair; S32. The BGE model is trained using the training dataset, and the loss is calculated using the DCL loss function to finally obtain the retrieval model; the formula for the DCL loss function is: Where, N all f(x) represents the number of sample pairs. i f'(x) represents the embedded representation of the item in the i-th sample pair obtained by the bge model. i ) represents f(x) i The result after dropout, f(x) i ) - This represents the embedded representation of the hard sample in the i-th sample pair obtained by the bge model.

6. The test question answering and analysis system based on a large model according to claim 1, characterized in that, Based on embedded representations, a recall method is used to extract auxiliary data from the knowledge base, including: S41. Based on embedded representation, cluster all test question data in the knowledge base corresponding to the subject type of the target test question by using two-dimensional fusion distance to obtain multiple cluster sets; S42. Calculate the two-dimensional fusion distance between the embedded expression of the target test item and the embedded expression of the centroid of each cluster set, and take the cluster set with the smallest two-dimensional fusion distance as the target set; S43. Calculate the two-dimensional fusion distance between the embedding expression of the target test item and the embedding expression of each test item data in the target set. Sort all the two-dimensional fusion distances in ascending order of size and select the test item data corresponding to the first two two-dimensional fusion distances as auxiliary data.

7. The test question answering and analysis system based on a large model according to claim 6, characterized in that, The formula for calculating the two-dimensional fusion distance is: Where D represents the two-dimensional fusion distance, A and B represent the embedding expressions of two different questions, n represents the number of dimensions of the embedding expression, and A i Let B represent the vector of the i-th dimension in the embedding expression A. i This represents the vector of the i-th dimension in the embedding expression B.

Images