Test question answering analysis system based on large model

Abstract

The invention relates to the technical field of large models, in particular to a test question answering analysis system based on a large model, comprising a test question collection module used for receiving a target test question input by a user; the matching module is used for identifying the subject type to which the target test question belongs through a routing model and distributing a corresponding retrieval model; the knowledge base module is used for storing test question data of different subject types; the auxiliary retrieval module is used for acquiring embedded expressions of the target test questions through a retrieval model and extracting auxiliary data from the knowledge base by adopting a recall method according to the embedded expressions; the test question analysis module is used for inputting the target test question and the auxiliary data into a test question analysis model to obtain a target test question analysis result; the display module is used for displaying the target test question analysis result; according to the invention, high-quality learning support can be provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of large models, and in particular to a test question answering and analyzing system based on large models. Background Art

[0002] The test question answer analysis system is a research hotspot in the field of online education. The test question answer analysis system can output the corresponding correct answers and the answer reasoning process based on the test questions input by the user, so that users can obtain more detailed test question analysis content and assist users in learning. Traditional test question answer analysis systems mainly rely on manually compiled question banks or simple keyword matching technology. Although these methods meet basic needs to a certain extent, they have obvious limitations. First, the manually compiled question bank has a slow update speed and it is difficult to cover all subjects and question types; second, traditional matching technology lacks semantic understanding capabilities, which can easily lead to mismatches or missed matches and cannot provide high-quality answers. In addition, traditional methods are usually unable to make customized recommendations based on the personalized needs of users, resulting in a poor user experience.

[0003] Large Language Model (LLM), also known as large language model or large model, is an artificial intelligence model designed to understand and generate human language. They are trained on a large amount of text data, have strong semantic understanding and reasoning capabilities, and can perform a wide range of natural language tasks, including text summarization, translation, sentiment analysis, etc. Although the existing large language model technology has made significant progress, in the field of test answering, the test answering system still faces great challenges in terms of accuracy and answer quality. Summary of the invention

[0004] To solve the above problems, the present invention provides a test question answering analysis system based on a large model, comprising:

[0005] A test question collection module is used to receive target test questions input by users;

[0006] A matching module, which includes a routing model, for identifying the subject type to which the target test question belongs through the routing model and assigning a corresponding retrieval model;

[0007] A knowledge base module includes knowledge bases corresponding to different subject types, and is used to store test data of different subject types; the test data includes test questions, answers to test questions, and solutions to test questions; and also stores embedded expressions of each test question;

[0008] An auxiliary retrieval module, which includes retrieval models corresponding to different subjects, is used to obtain embedded expressions of target test questions through the retrieval models, and extract auxiliary data from the knowledge base using a recall method based on the embedded expressions;

[0009] The test question analysis module includes a test question analysis model, which is used to obtain a target test question analysis result by inputting a target test question and auxiliary data into the test question analysis model; the target test question analysis result includes an answer to the target test question and a solution idea; the target test question, the target test question analysis result, and the embedded expression of the target test question are put into a corresponding knowledge base; the test question analysis model is obtained by a large model knowledge distillation method;

[0010] The display module is used to display the analysis results of the target test questions.

[0011] Beneficial effects of the present invention:

[0012] The present invention builds a dynamically updated knowledge base for different subject types through data enhancement and model distillation technology, which not only covers a wide range of subjects and question types, but also can generate new analysis and answers in real time according to user needs, significantly improving the richness and accuracy of the question bank, and significantly improving the reasoning ability and deployment efficiency of the model.

[0013] The present invention optimizes the routing model and the retrieval model respectively through the dynamic balance loss (DBL) and the subject contrast loss (DCL), thereby achieving more accurate subject classification and question recall. This personalized and customized method based on a large model can provide targeted answers and recommendations according to the specific needs of users, greatly improving the user experience. For example, after the user enters the test question, the system can not only quickly classify it into the correct subject, but also recall the most similar questions and answers, and generate the final answer in combination with the reasoning ability of the large model, thereby improving the service quality and optimizing the user experience.

[0014] The present invention performs clustering and recall by using a two-dimensional fusion distance, thereby ensuring efficient retrieval of a knowledge base and reliability of reasoning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the connection of the system modules of the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The present invention provides a test question answering analysis system based on a large model, such as Figure 1 As shown, including:

[0018] A test question collection module is used to receive target test questions input by users;

[0019] A matching module, which includes a routing model, for identifying the subject type to which the target test question belongs through the routing model and assigning a corresponding retrieval model;

[0020] A knowledge base module includes knowledge bases corresponding to different subject types, and is used to store test data of different subject types; the test data includes test questions, answers to test questions, and solutions to test questions; and also stores embedded expressions of each test question;

[0021] An auxiliary retrieval module, which includes retrieval models corresponding to different subjects, is used to obtain embedded expressions of target test questions through the retrieval models, and extract auxiliary data from the knowledge base using a recall method based on the embedded expressions;

[0022] The test question analysis module includes a test question analysis model, which is used to obtain a target test question analysis result by inputting a target test question and auxiliary data into the test question analysis model; the target test question analysis result includes an answer to the target test question and a solution idea; the target test question, the target test question analysis result, and the embedded expression of the target test question are put into a corresponding knowledge base; the test question analysis model is obtained by a large model knowledge distillation method;

[0023] The display module is used to display the analysis results of the target test questions.

[0024] Specifically, the subject types involved in the embodiments of the present invention include seven types: advanced mathematics, linear algebra, probability theory, computer networks, data structures, operating systems, and computer organization principles.

[0025] Specifically, the knowledge base initialization of any subject type includes

[0026] Collecting a test question set belonging to the current subject type, wherein the test question set includes a plurality of test question data;

[0027] For each piece of test data, the test questions in the test data are input into the 110B large model (i.e., the large model with 110 billion parameters) to obtain new answers and new solutions. If the new answer is the same as the answer in the test data, and the new solution is different from the solution in the test data, then the test question, the answer and the new solution are combined into a new piece of test data;

[0028] Add all test question data to the knowledge base of the current subject type.

[0029] Specifically, the process of obtaining the test question parsing model through the large model knowledge distillation method includes:

[0030] S11. Fine-tune the 32B large model (i.e., the large model with 32 billion parameters) according to the test data in the knowledge base module, so that the 32B large model can take the test questions as input and the answers and solution ideas of the test questions as output;

[0031] S12. Input the test question into the 7B large model (i.e., the large model with 7 billion parameters) to obtain the first answer and the first solution, and input the same test question into the fine-tuned 32B large model to obtain the second answer and the second solution;

[0032] S13. Calculate the answer similarity loss ASL based on the first answer and the second answer, expressed as

[0033]

[0034] Where N represents the answer token length. If the two answers are of different lengths, N is the larger length and the shorter answer will be padded to N through special token padding. represents the vector representation of the i-th token in the first answer, represents the vector representation of the i-th token in the second answer, p small，a represents the probability distribution generated by the 7B large model, p large，a represents the probability distribution generated by the 32B large model, and KL(·) represents the KL divergence operation;

[0035] S14. Calculate the reasoning path loss RPL according to the first solution idea and the second solution idea, expressed as

[0036]

[0037] Among them, M represents the maximum token length of the solution idea. Represents the vector representation of the i-th token in the first solution. represents the vector representation of the i-th token in the second solution, p small，t represents the probability distribution generated by the 7B large model, p large，t represents the probability distribution generated by the 32B large model, Wasserstein(·) represents the Wasserstein distance; Π(p small，t , p large，t ) represents the set of all possible joint distributions, whose marginal distribution is p small，t and p large，t ;γ(i,j) is a joint distribution, indicating that small，t The i-th point is transferred to p large，t The probability of the jth point, d(i, j) represents p small，t The i-th point transfer and p large，t The Euclidean distance between the j-th points in ; since the text length of the solution is longer than the text length of the answer, using wasserstein can have better robustness.

[0038] S15. Perform knowledge distillation on the 7B large model according to the answer similarity loss ASL and the reasoning path loss RPL, and repeat steps S12-S15 until the parameters of the 7B large model converge to obtain the test question parsing model.

[0039] Specifically, the routing model includes a backbone module, an intermediate dimension fully connected layer, and a classification fully connected layer. The training process of the routing model includes:

[0040] S21. Obtain a test question label data set, wherein each test question label data set includes a test question and its corresponding subject type label;

[0041] S22. Input the test questions into the backbone module to obtain the embedding vector, where the backbone module uses the pre-trained BERT model;

[0042] S23. Input the embedded vector into the intermediate dimension fully connected layer, and map the embedded vector to the low dimension through the intermediate dimension fully connected layer to obtain the intermediate dimension vector; the dimension z of the intermediate dimension vector is calculated as follows:

[0043]

[0044] Among them, d bert represents the dimension of the embedding vector, L bert Indicates the number of layers of the BERT model, H bert Indicates the number of multi-head attention heads of the BERT model;

[0045] S24. Input the intermediate dimension vector into the classification fully connected layer to map the output to 7 probabilities to obtain the predicted label;

[0046] S25. Calculate the DBL loss, and train the model parameters according to the DBL loss until the model parameters converge.

[0047] Specifically, the DBL loss calculation formula is:

[0048] L DBL =L CE *exp(L triplet )+L triplet *exp(L CE )

[0049]

[0050] L triplet =max(0,||f(x)-f(x + )|| 2 -||f(x)-f(x - )|| 2 )

[0051] Among them, L DBL represents DBL loss, L CE represents the first loss, L triplet represents the second loss; y i represents the true label of the sample, p i represents the probability distribution of the sample obtained through the routing model, f(x) represents the embedding vector of the sample, and f(x + ) represents the embedding vector of the remaining samples belonging to the same subject type as the sample, f(x - ) represents the embedding vector of the remaining samples that belong to different subject types. This loss function can enhance the intra-class compactness and inter-class separation, and improve the discriminative ability of the model.

[0052] Specifically, the process of obtaining the retrieval model corresponding to any subject type includes:

[0053] S31. Obtain a training data set, which includes multiple sample pairs, each sample pair includes a test question and a difficult sample; the process of obtaining a sample pair is as follows:

[0054] Obtain the knowledge points tested in the test questions, construct a question that is similar to the n-gram of the test questions but has different knowledge points tested as a difficult sample, and form a sample pair with the test questions and the difficult sample;

[0055] S32. Use the training data set to train the bge model, use the DCL loss function to calculate the loss, and finally obtain the retrieval model; the calculation formula of the DCL loss function is

[0056]

[0057] Among them, N all represents the number of sample pairs, f(x i ) represents the embedded expression of the test question in the i-th sample pair obtained by the bge model, f′(x i ) means f(x i )The result after dropout, f(x i ) - represents the embedded expression of the difficult sample in the i-th sample pair obtained by the bge model. By minimizing L DCL The loss maximizes the similarity between the test questions and the positive samples, and at the same time maximizes the dissimilarity between the test questions and the difficult samples (negative samples), thereby bringing the positive samples closer and pushing the difficult samples further away in the embedding space.

[0058] Specifically, extracting auxiliary data from the knowledge base using a recall method based on embedded expressions includes:

[0059] S41. Based on the embedded expression, all the test question data in the knowledge base corresponding to the subject type of the target test question are clustered through the two-dimensional fusion distance to obtain multiple cluster sets;

[0060] S42. Calculate the two-dimensional fusion distance between the embedded expression of the target test question and the embedded expression of the centroid of each cluster set, and take the cluster set with the smallest two-dimensional fusion distance as the target set;

[0061] S43. Calculate the two-dimensional fusion distance between the embedding expression of the target test question and the embedding expression of each test question data in the target set, arrange all the two-dimensional fusion distances in ascending order, and select the test question data corresponding to the first two two-dimensional fusion distances as auxiliary data.

[0062] Specifically, the calculation formula of the two-dimensional fusion distance is:

[0063]

[0064] Among them, D represents the two-dimensional fusion distance, A and B represent the embedding expressions of two different test questions, n represents the number of embedding expression dimensions, A i represents the i-th dimension vector in the embedded expression A, B i Represents the i-th dimension vector in the embedding expression B.

[0065] Specifically, the display module also displays the auxiliary data obtained by the auxiliary retrieval module. Specifically, the auxiliary data consists of two test questions that are most similar to the target test questions. The two test questions in the auxiliary data can be displayed to recommend similar test questions to users, realize personalized recommendations, and guide users to further study and consolidate the same knowledge points.

[0066] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.

[0067] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A test question answering and analysis system based on a large model, characterized in that: include: A test question collection module is used to receive target test questions input by users; A matching module, which includes a routing model, for identifying the subject type to which the target test question belongs through the routing model and assigning a corresponding retrieval model; A knowledge base module includes knowledge bases corresponding to different subject types, and is used to store test data of different subject types; the test data includes test questions, answers to test questions, and solutions to test questions; and also stores embedded expressions of each test question; An auxiliary retrieval module, which includes retrieval models corresponding to different subjects, is used to obtain embedded expressions of target test questions through the retrieval models, and extract auxiliary data from the knowledge base using a recall method based on the embedded expressions; The test question analysis module includes a test question analysis model, which is used to obtain a target test question analysis result by inputting a target test question and auxiliary data into the test question analysis model; the target test question analysis result includes an answer to the target test question and a solution idea; the target test question, the target test question analysis result, and the embedded expression of the target test question are placed in a corresponding knowledge base; The test question parsing model is obtained through a large model knowledge distillation method; The display module is used to display the analysis results of the target test questions.

2. A test question answering and analyzing system based on a large model according to claim 1, characterized in that: Initialization of the knowledge base for any subject type includes Collecting a test question set belonging to the current subject type, wherein the test question set includes a plurality of test question data; For each piece of test data, the test questions in the test data are input into the 110B large model to obtain new answers and new solution ideas. If the new answer is the same as the answer in the test data, and the new solution idea is different from the solution idea in the test data, then the test question, the answer and the new solution idea are combined into a new piece of test data; Add all test question data to the knowledge base of the current subject type.

3. A test question answering and analyzing system based on a large model according to claim 1, characterized in that: The process of obtaining the test question parsing model through the large model knowledge distillation method includes: S11. Fine-tune the 32B large model according to the test data in the knowledge base module, so that the 32B large model can take the test questions as input and the answers and solution ideas of the test questions as output; S12. Input the test question into the 7B large model to obtain the first answer and the first solution, and input the same test question into the fine-tuned 32B large model to obtain the second answer and the second solution; S13. Calculate the answer similarity loss ASL based on the first answer and the second answer, expressed as Where N represents the length of the answer token. represents the vector representation of the i-th token in the first answer, represents the vector representation of the i-th token in the second answer, p small,a represents the probability distribution generated by the 7B large model, p large,a represents the probability distribution generated by the 32B large model, and KL(·) represents the KL divergence operation; S14. Calculate the reasoning path loss RPL according to the first solution idea and the second solution idea, expressed as Among them, M represents the maximum token length of the solution idea. Represents the vector representation of the i-th token in the first solution. represents the vector representation of the i-th token in the second solution, p small,t represents the probability distribution generated by the 7B large model, p large,t represents the probability distribution generated by the 32B large model, Wasserstein(·) represents the Wasserstein distance, Π(p small,t ,p large,t ) represents the joint distribution set, γ(i,j) represents the distribution from p small,t The i-th point is transferred to p large,t The probability of the j-th point, d(i,j) represents the Euclidean distance; S15. Perform knowledge distillation on the 7B large model according to the answer similarity loss ASL and the reasoning path loss RPL, and repeat steps S12-S15 until the parameters of the 7B large model converge to obtain the test question parsing model.

4. A test question answering and analyzing system based on a large model according to claim 1, characterized in that: The routing model includes a backbone module, an intermediate dimension fully connected layer, and a classification fully connected layer. The training process of the routing model includes: S21. Obtain a test question label data set, wherein each test question label data set includes a test question and its corresponding subject type label; S22. Input the test questions into the backbone module to obtain the embedding vector, where the backbone module uses the pre-trained BERT model; S23. Input the embedded vector into the intermediate dimension fully connected layer, and map the embedded vector to the low dimension through the intermediate dimension fully connected layer to obtain the intermediate dimension vector; the dimension z of the intermediate dimension vector is calculated as follows: Among them, d bert represents the dimension of the embedding vector, L bert Indicates the number of layers of the BERT model, H bert Indicates the number of multi-head attention heads of the BERT model; S24. Input the intermediate dimension vector into the classification fully connected layer to obtain the predicted label; S25. Calculate the DBL loss, and train the model parameters according to the DBL loss until the model parameters converge.

5. A test question answering and analyzing system based on a large model according to claim 4, characterized in that: The DBL loss calculation formula is L DBL =L CE *exp(L triplet )+L triplet *exp(L CE ) L triplet =max(0,||f(x)-f(x + )|| 2 -||f(x)-f(x - )|| 2 ) Among them, L DBL represents DBL loss, L CE represents the first loss, L triplet represents the second loss; y i represents the true label of the sample, p i represents the probability distribution of the sample obtained through the routing model, f(x) represents the embedding vector of the sample, and f(x + ) represents the embedding vector of the remaining samples belonging to the same subject type as the sample, f(x - ) represents the embedding vector of the remaining samples that belong to different subject types than the sample.

6. A test question answer analysis system based on a large model according to claim 1, characterized in that: The process of obtaining the retrieval model corresponding to any subject type includes: S31. Obtain a training data set, which includes multiple sample pairs, each sample pair includes a test question and a difficult sample; the process of obtaining a sample pair is as follows: Obtain the knowledge points tested in the test questions, construct a question that is similar to the n-gram of the test questions but has different knowledge points tested as a difficult sample, and form a sample pair with the test questions and the difficult sample; S32. Use the training data set to train the bge model, use the DCL loss function to calculate the loss, and finally obtain the retrieval model; the calculation formula of the DCL loss function is Among them, N all represents the number of sample pairs, f(x i ) represents the embedded expression of the test question in the i-th sample pair obtained by the bge model, f′(x i ) means f(x i )The result after dropout, f(x i ) - Represents the embedded expression of the difficult sample in the i-th sample pair obtained by the bge model.

7. A test question answer analysis system based on a large model according to claim 1, characterized in that: Extracting auxiliary data from the knowledge base using the recall method based on the embedded expression includes: S41. Based on the embedded expression, all the test question data in the knowledge base corresponding to the subject type of the target test question are clustered through the two-dimensional fusion distance to obtain multiple cluster sets; S42. Calculate the two-dimensional fusion distance between the embedded expression of the target test question and the embedded expression of the centroid of each cluster set, and take the cluster set with the smallest two-dimensional fusion distance as the target set; S43. Calculate the two-dimensional fusion distance between the embedding expression of the target test question and the embedding expression of each test question data in the target set, arrange all the two-dimensional fusion distances in ascending order, and select the test question data corresponding to the first two two-dimensional fusion distances as auxiliary data.

8. A test question answering and analyzing system based on a large model according to claim 7, characterized in that: The calculation formula of the two-dimensional fusion distance is: Among them, D represents the two-dimensional fusion distance, A and B represent the embedding expressions of two different test questions, n represents the number of embedding expression dimensions, A i represents the i-th dimension vector in the embedded expression A, B i Represents the i-th dimension vector in the embedding expression B.

Images