Test question text scoring method and device, electronic equipment and storage medium
By using deep learning to identify key steps and theorem features in mathematical proof problems, the problem of poor reliability caused by the coarse granularity of scoring models in existing technologies is solved, and more accurate automatic scoring is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2022-12-28
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, the scoring models for mathematical proof problems have a coarse modeling granularity, and the scoring principles rely on similarity, resulting in poor reliability of the scoring results.
By using deep learning methods to identify key steps and theorem features in test question texts, and combining these with reasoning features, the system simulates the teacher's grading process to achieve automatic scoring.
This improves the reliability and rationality of scoring mathematical proof problems, avoiding the problem of overly high scores due to coarse modeling granularity.
Smart Images
Figure CN115827828B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, electronic device, and storage medium for scoring test texts. Background Technology
[0002] In daily teaching activities, the scoring and grading of test questions in mathematics are particularly important, as these types of questions can assess students' comprehensive abilities. Among mathematical problem-solving questions, there is a special category: mathematical proof questions. This type of question focuses on assessing students' logical reasoning abilities, with strict logical derivation between each step, possibly based on a theorem or corollary. How to utilize artificial intelligence algorithms to achieve intelligent scoring and grading of mathematical test questions is an important research topic in the fields of education and artificial intelligence.
[0003] Existing scoring schemes for math problem-solving typically involve training a regression model, inputting the question stem, standard answer, and student responses, and ultimately obtaining a score prediction model. The core idea of this scheme is to train a vector that can represent the student's response text. Usually, a language model is trained using a large amount of response data, and then a score prediction model is trained based on the model representation, using the scoring results as a supervision signal.
[0004] These score prediction models are generally used to score regular problem-solving questions. For proof questions, which involve explicit reasoning, the current modeling granularity is relatively coarse, and the scoring principle relies primarily on similarity. Furthermore, students can usually write out some derivation for proof questions, and their answers often contain many invalid or illogical but well-structured arguments. Such arguments can significantly interfere with the scoring, easily leading to overestimation of scores and resulting in unreliable scoring results. Summary of the Invention
[0005] This invention provides a method, apparatus, electronic device, and storage medium for scoring test texts, in order to solve the defects of existing technologies where the modeling granularity is too coarse and the scoring principle basically relies on similarity, resulting in poor reliability of the scoring results.
[0006] This invention provides a method for scoring test questions, comprising:
[0007] Obtain the target test question text;
[0008] Based on the target test question text, obtain the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text;
[0009] Based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach, the scoring result corresponding to the answer text is obtained;
[0010] The answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps; the answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps.
[0011] According to a test question text scoring method provided by the present invention, the step of obtaining the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text, includes:
[0012] Based on the target test question text, determine the input information;
[0013] The input information is fed into the shared feature layer of the text feature prediction model to obtain the shared features of the target test text;
[0014] The shared features are input into the theorem prediction branch model of the text feature prediction model to obtain the key answering steps of the answer text, the theorem features corresponding to the key answering steps, and the key answering steps of each problem-solving approach, as well as the theorem features corresponding to the key answering steps.
[0015] The shared features, the key answering steps, and the key answer steps are input into the inference prediction branch model of the text feature prediction model to obtain the inference features corresponding to the key answering steps of the answer text and the inference features corresponding to the key answer steps of each problem-solving approach.
[0016] According to the test text scoring method provided by the present invention, the text feature prediction model is obtained by joint training based on the loss function of the initial theorem prediction branch model and the loss function of the initial inference prediction branch model in the initial feature prediction model.
[0017] According to a test question text scoring method provided by the present invention, the target test question text includes a question stem text, the answer text, and the response text;
[0018] The step of determining the input information based on the target test question text includes:
[0019] The input information is determined based on the question text, the answer text, and the response text.
[0020] According to a test question text scoring method provided by the present invention, the step of determining the input information based on the question stem text, the answer text, and the response text includes:
[0021] The positions of each step in the problem-solving approach in the answer text and each step in the response text are encoded.
[0022] The input information is obtained by concatenating the question text, the encoded answer text, and the encoded response text.
[0023] According to a test text scoring method provided by the present invention, the step of obtaining the scoring result corresponding to the answer text based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach includes:
[0024] The answer features and the answer features corresponding to each problem-solving approach are input into the feature interaction layer of the scoring prediction model to obtain the interaction features;
[0025] The interactive features are input into the problem-solving approach prediction branch model of the scoring prediction model, and the problem-solving approach that matches the answer text is determined from all the problem-solving approaches in the answer text.
[0026] The problem-solving approach matching the answer text and the interaction features are input into the score prediction branch model of the scoring prediction model to obtain the score corresponding to the answer text.
[0027] The solution approach that matches the answer text and the score are used as the scoring result.
[0028] According to the test text scoring method provided by the present invention, the scoring prediction model is obtained by joint training based on the loss function of the initial score prediction branch model and the loss function of the initial problem-solving strategy prediction branch model in the initial scoring prediction model.
[0029] The present invention also provides a test question text scoring device, comprising:
[0030] The acquisition unit is used to acquire the target test question text;
[0031] The feature extraction unit is used to obtain, based on the target test question text, the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text in the target test question text;
[0032] The scoring unit is used to obtain the scoring result corresponding to the answer text based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach;
[0033] The answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps; the answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps.
[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the test text scoring method as described above.
[0035] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the test text scoring method as described above.
[0036] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the test text scoring method as described above.
[0037] The test text scoring method, apparatus, electronic device, and storage medium provided by this invention identify the target test text to obtain the key answering steps, the theorem features, and reasoning features corresponding to the answer text, as well as the key answering steps, theorem features, and reasoning features corresponding to each problem-solving approach in the answer text. By comprehensively considering the multi-dimensional features such as the derivation steps and derivation logic in the answer text and the answer text, the test text is scored accordingly. This avoids the problem of poor reliability of scoring results due to coarse modeling granularity and the scoring principle relying mainly on similarity, thereby improving the reliability and rationality of test text scoring. Attached Figure Description
[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0039] Figure 1 This is one of the flowcharts illustrating the test text scoring method provided by the present invention;
[0040] Figure 2 This is the second flowchart illustrating the test text scoring method provided by the present invention;
[0041] Figure 3 This is the third flowchart illustrating the test text scoring method provided by the present invention;
[0042] Figure 4 This is a schematic diagram of the scoring prediction model in the test text scoring method provided by the present invention;
[0043] Figure 5 This is a schematic diagram of the test text scoring device provided by the present invention;
[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0046] Generally, although proof questions belong to the category of problem-solving questions, there are many differences between them and simple problem-solving questions. The key difference lies in the fact that proof questions involve more deductive logic and theorems, and have very strict contextual logical relationships, which is the biggest challenge in the refined scoring of proof questions.
[0047] Existing scoring prediction models have a coarse-grained modeling level, and their scoring principles largely rely on similarity. However, for proof questions, which involve explicit reasoning logic, student responses often contain many invalid or illogical but well-structured reasoning steps, significantly interfering with scoring and potentially leading to overestimation of scores. Therefore, it is necessary to further refine the representation of each student's step and the logical relationships between them.
[0048] Typically, grading student work requires teachers to rely on their teaching experience. For mathematical proofs, the grading criteria need to clearly identify the theorems or principles used in each step of the student's answer, and assess the logical consistency of derivations between steps. Based on the student's reasoning, the grading process should be clarified and compared with various approaches to the standard answer. The final score is determined by the percentage of correct and incorrect steps, and whether key steps are addressed. Therefore, accurately predicting the theorems or test points for each key step, as well as the derivation relationships between steps, is a crucial issue that the target industry urgently needs to address.
[0049] Therefore, this embodiment addresses the problem that the existing scoring prediction models have coarse modeling granularity and rely primarily on similarity in their scoring principles, leading to poor reliability of the scoring results. It proposes a test text scoring method that automatically identifies key steps, the theorem features used in these key steps, and the reasoning features between steps using deep learning. Furthermore, it judges the student's answering approach, comprehensively learning the multidimensional important features of the test text and simulating the standards used by teachers in grading test texts. This achieves automatic scoring of test texts, improving the reliability and rationality of test text scoring.
[0050] like Figure 1 The diagram shown is one of the flowcharts illustrating the test text scoring method provided in this embodiment. The method includes the following steps:
[0051] Step 101: Obtain the target test question text;
[0052] The target test question text includes, but is not limited to, the answer text and the written responses to the target test question. This embodiment does not specifically limit these; it may also include the question stem text. The answer text includes the standard answer corresponding to the target test question, and the written responses contain the answers provided by one or more students to the target test question.
[0053] The test question corresponding to the target test question text can be a mathematical proof question or a physical proof question, etc. This embodiment does not make specific limitations on this.
[0054] Optionally, the answer text, the response text, and the question stem of the target test question are obtained to determine the target test question text based on the answer text and the response text, or to determine the target test question text based on the answer text, the response text, and the question stem.
[0055] Step 102: Based on the target test question text, obtain the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text; wherein, the answer features include key answer steps, the theorem features corresponding to the key answer steps, and the reasoning features corresponding to the key answer steps; the answer features include key answer steps, the theorem features corresponding to the key answer steps, and the reasoning features corresponding to the key answer steps;
[0056] The answer text for each target question includes one or more problem-solving approaches (hereinafter also referred to as proof approaches), and this embodiment does not specifically limit the number of approaches.
[0057] The answer characteristics include, but are not limited to, key answer steps, the theorem characteristics corresponding to the key answer steps, and the reasoning characteristics corresponding to the key answer steps; the key answer steps are the necessary steps to answer the target question; the theorem characteristics are the proof theorems used in the key answer steps;
[0058] For example, the logic of answering proof questions is to use the existing conditions in the question stem or the conclusion obtained from the previous proof as known conditions, and then obtain the next conclusion based on a certain theorem or corollary (hereinafter referred to as the theorem). For example, if the answer text contains the content "Because AB / / CD, CD / / EF, therefore AB / / EF", the answer text uses the theorem "Transitivity of parallel lines: If two lines are both parallel to a third line, then the two lines are also parallel to each other." Therefore, "AB / / EF" is determined to be one of the key answering steps in the answer text, and the theorem characteristic corresponding to this key answering step is "transitivity of parallel lines".
[0059] The reasoning characteristics are the reasoning relationships between key answering steps and other steps, including but not limited to deductive relationships or parallel relationships.
[0060] For example, if the answer text contains the answer content "Because AB / / CD, CD / / EF, therefore AB / / EF"; where the reasoning relationship between the two steps "AB / / CD" and "CD / / EF" is a parallel relationship, and the reasoning relationship between "AB / / CD", "D / / EF" and "AB / / EF" is a deductive relationship.
[0061] Optionally, after obtaining the target test question text, all key answering steps, the theorem features corresponding to each key answering step, and the reasoning features corresponding to each key answering step can be identified from the target test question text; and all key answering steps, the theorem features corresponding to each key answering step, and the reasoning features corresponding to each key answering step can be identified from each problem-solving approach in the answer text, so as to determine the student's answering logic based on the answering features corresponding to the answer text and the answering features corresponding to each problem-solving approach in the answer text, thereby achieving accurate scoring of the answer text.
[0062] It should be noted that obtaining the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text in the target test question text, can be achieved through a prediction model constructed using one or more neural networks. For example, based on one prediction model, key step identification, theorem feature prediction, and reasoning feature prediction are performed holistically to obtain the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text in the target test question text; or, based on one of the two prediction models, key step identification and theorem feature prediction are performed locally, and based on the other of the two prediction models, reasoning feature prediction is performed locally. By summarizing the prediction results of this prediction model, the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text in the target test question text, are obtained. This embodiment does not specifically limit this approach.
[0063] Step 103: Based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach, obtain the scoring result corresponding to the answer text;
[0064] The scoring results may include the score corresponding to the answer text, or the problem-solving approach corresponding to the answer text. That is, the prediction model used to achieve the scoring prediction can be a single-task model or a multi-task model, which needs to be determined according to the scoring requirements.
[0065] For example, when the scoring result includes the score and the problem-solving approach, the prediction model used to achieve the score prediction can be a multi-task model; when the scoring result includes the score, the prediction model used to achieve the score prediction can be a single-task model or a multi-task model.
[0066] Optionally, after obtaining the answer features and the answer features corresponding to each problem-solving approach, feature interaction can be performed on the answer features and the answer features corresponding to each problem-solving approach to determine the problem-solving approach to which the answer text belongs;
[0067] The system calculates the score corresponding to the answer text based on the problem-solving approach and answer characteristics; and obtains the scoring result corresponding to the answer text based on the score, or the score and problem-solving approach.
[0068] In addition, the scoring results corresponding to the answer texts can be used to realize functions such as automated test paper grading and error analysis, so as to help teachers improve grading efficiency, reduce the burden of homework grading, and improve the efficiency of student error analysis by automatically scoring the answer texts.
[0069] For example, in the scenario of automated test grading, the score of each key step in the answer text can be graded based on the score of each student's answer text, the standard answer corresponding to the problem-solving approach of the answer text, and the answer characteristics of the answer text. As another example, in the scenario of error analysis, the error causes can be analyzed based on the score of each key step in the answer text and the answer characteristics in the answer text.
[0070] The test text scoring method provided in this embodiment identifies the target test text and obtains the key answering steps, the theorem features, and reasoning features corresponding to the answer text, as well as the key answering steps, theorem features, and reasoning features corresponding to each problem-solving approach in the answer text. By comprehensively considering the derivation steps and derivation logic in both the answer text and the answer text, the method scores the answer text. This avoids the problem of poor reliability of scoring results due to coarse modeling granularity and a scoring principle that relies mainly on similarity, thereby improving the reliability and rationality of test text scoring.
[0071] In some embodiments, such as Figure 2 As shown, step 102 further includes:
[0072] Step 1021: Determine the input information based on the target test question text;
[0073] Step 1022: Input the input information into the shared feature layer of the text feature prediction model to obtain the shared features of the target test text;
[0074] Step 1023: Input the shared features into the theorem prediction branch model of the text feature prediction model to obtain the key answering steps of the answer text, the theorem features corresponding to the key answering steps, and the key answering steps of each problem-solving approach, and the theorem features corresponding to the key answering steps.
[0075] Step 1024: Input the shared features, the key answering steps, and the key answer steps into the inference prediction branch model of the text feature prediction model to obtain the inference features corresponding to the key answering steps of the answer text and the inference features corresponding to the key answer steps of each problem-solving approach.
[0076] Among them, the text feature prediction model is used to extract features from the target test text in order to obtain the answer features corresponding to the answer text, as well as the answer features corresponding to each problem-solving approach in the answer text.
[0077] Text feature prediction models include shared feature layers, theorem prediction branch models, and inference prediction branch models.
[0078] The shared feature layer is a feature extraction layer shared by the theorem prediction branch model and the inference prediction branch model. That is, the theorem prediction branch model and the inference prediction branch model share the same set of underlying vector representations for parameter sharing. The shared feature layer can be built based on BERT (Bidirectional Encoder Representations from Transformer, a pre-trained language representation model) or LSTM (Long Short-Term Memory) models and is used for text feature extraction.
[0079] The theorem prediction branch model performs step-level theorem predictions for key steps in the target test text, and can be generated based on a softmax classification layer.
[0080] The inference prediction branch model is used to predict the inference relationships between each key step and other steps in the answer text and response text, obtaining the inference features corresponding to each key step. The input of the inference prediction branch model is the same as that of the theorem prediction branch model. Each step concatenates the previous step and the next step, and finally concatenates them into a vector for classification to predict the inference features corresponding to each key step. It can also be generated based on a softmax classification layer, but it is independent of the classification layer of the theorem prediction branch model mentioned above.
[0081] Optionally, before performing step 1021, a text feature prediction model can be pre-trained, specifically including the following steps:
[0082] First, an initial feature prediction model is created. This initial feature prediction model is a multi-task model used to implement the theorem prediction task and the inference prediction task, including an initial shared feature layer, an initial theorem prediction branch model, and an initial inference prediction branch model.
[0083] The initial shared feature layer can be a module prepared for feature extraction after parameter initialization, or it can be a pre-trained module with feature extraction function. Similarly, each branch model in the prediction branch model can be a branch model prepared for prediction after parameter initialization, or it can be a pre-trained branch model with prediction function. This embodiment does not make specific limitations on this.
[0084] In addition, sample test question texts, sample answer texts, and answer feature labels corresponding to each problem-solving approach in the sample answer texts can be collected.
[0085] Subsequently, the sample test question text can be used as input to the initial feature prediction model. The initial shared feature layer in the initial feature prediction model extracts shared features from the sample test question text to obtain sample shared features. Then, the initial theorem prediction branch model and the initial inference prediction branch model predict features to output the sample answer features corresponding to the sample answer text, as well as the sample answer features corresponding to each solution approach in the sample answer text. After obtaining the sample answer features and sample answer features, the parameters of the initial feature prediction model can be iteratively trained based on the sample answer features, the sample answer features, the answer feature labels of the sample answer text, and the answer feature labels of the sample answer text, thereby obtaining the text feature prediction model.
[0086] After obtaining the text feature prediction model, the target test question text can be directly used as the input information of the text feature prediction model, or the target test question text can be preprocessed such as encoding and / or splicing before being used as the input information of the text feature prediction model. This embodiment does not make specific limitations on this.
[0087] Then, based on the shared feature layer of the text feature prediction model, shared features are extracted from the input information to obtain the shared features of the target test text; based on the theorem prediction branch model of the text feature prediction model, the key answering steps in the answer text of the target test text are extracted and the theorem features corresponding to the key answering steps are identified, as well as the key answering steps in the answer text of the target test text are extracted and the theorem features corresponding to the key answering steps are identified.
[0088] Next, the inference prediction branch model based on the text feature prediction model obtains the inference relationship between each key answer step and other answer steps (hereinafter also referred to as inference features) and the inference relationship between each key answer step and other answer steps, based on the shared features and the key answer steps and key answer steps obtained in step 1023.
[0089] This embodiment uses a multi-task text feature prediction model to accurately extract the theorems and axioms used in the process of answering questions, as well as the reasoning relationships corresponding to each key step, from the answer text and the response text. Based on this, the answer text is scored. Compared to common techniques that calculate scores based on similarity, the question text scoring method provided in this embodiment performs more refined modeling based on the derivation logic, theorems, and logical relationships in the target question text. This effectively solves the problem of overscoring, making the question text scoring more refined and accurate, thereby improving the reliability and rationality of the question text scoring.
[0090] In some embodiments, the text feature prediction model is obtained by jointly training the loss function of the initial theorem prediction branch model and the loss function of the initial inference prediction branch model in the initial feature prediction model.
[0091] Optionally, the training steps for the text feature prediction model specifically include:
[0092] First, the sample test question text is used as the input of the initial feature prediction model. The initial shared feature layer in the initial feature prediction model extracts the shared features of the sample test question text to obtain the sample shared features. Then, the initial theorem prediction branch model and the initial inference prediction branch model predict and output the sample answer features corresponding to the sample answer text, as well as the sample answer features corresponding to each problem-solving approach in the sample answer text.
[0093] After obtaining the sample response features and sample answer features, the loss function of the initial theorem prediction branch model can be calculated based on the sample key answer steps, the theorem features corresponding to the sample key answer steps in the sample answer features, the sample key response steps, the theorem features corresponding to the sample key response steps in the sample response features, the key answer step labels, the theorem labels corresponding to the key answer step labels in the answer feature labels of the sample answer text, and the key response step labels, the theorem labels corresponding to the key response step labels in the response feature labels of the sample answer text.
[0094] Based on the inference features corresponding to the key answer steps in the sample answer features, the inference features corresponding to the key answer steps in the sample answer features, the inference labels corresponding to the key answer step labels in the answer feature labels of the sample answer text, and the inference labels corresponding to the key answer step labels in the answer feature labels of the sample answer text, the loss function of the initial theorem prediction branch model is calculated.
[0095] The loss functions of the initial theorem prediction branch model and the initial theorem prediction branch model are fused to obtain the total loss function of the initial feature prediction model. Based on the total loss function, backpropagation training is performed to synchronously update the parameters of the multi-task branch model in the initial feature prediction model until the initial feature prediction model meets the training termination condition, thus obtaining the text feature prediction model. The training termination condition includes reaching the maximum number of training iterations or the model performance meeting the optimal performance requirements. This embodiment does not specifically limit these conditions.
[0096] The total loss function can be obtained by weighted summing of the loss functions of the initial theorem prediction branch model and the initial theorem prediction branch model. The weights of the loss function can be adaptively adjusted based on the training performance obtained during the training of the initial feature prediction model using the training dataset.
[0097] In this embodiment, during the text feature prediction modeling process, the loss functions of the initial theorem prediction branch model and the initial inference prediction branch model in the feature prediction model are jointly initialized. Through mutual learning, mutual promotion, mutual supplementation, and mutual sharing among multiple branch models, collaborative training is carried out, which makes the trained text feature prediction model perform better, has stronger semantic rationality, and further improves the accuracy of test text scoring.
[0098] In some embodiments, the target test question text includes the question stem text, the answer text, and the response text;
[0099] The step of determining the input information based on the target test question text includes:
[0100] The input information is determined based on the question text, the answer text, and the response text.
[0101] Among them, the question stem text is the question stem of the target test question, which is used to help improve the efficiency and accuracy of feature extraction from the answer text and the response text.
[0102] Optionally, when extracting features from the answer text and response text in the target test question text, the question stem text, answer text, and response text can be directly combined as input information for the text feature prediction model. Alternatively, one or more combinations of the question stem text, answer text, and response text can be encoded and / or concatenated before being used as input information for the text feature prediction model. This can achieve more accurate and efficient extraction of key steps, theorem features, and reasoning features from the answer text and response text, thereby improving the scoring accuracy of the test question text.
[0103] In some embodiments, determining the input information based on the question stem text, the answer text, and the response text includes:
[0104] The positions of each step in the problem-solving approach in the answer text and each step in the response text are encoded.
[0105] The input information is obtained by concatenating the question text, the encoded answer text, and the encoded response text.
[0106] Optionally, when determining the input information of the text feature prediction model, the positional encoding of each step of the problem-solving approach in the answer text and each step in the response text can be performed in advance, so that the text feature prediction model can more accurately obtain the positional characteristics of each step, and further extract the key steps, theorem features and reasoning features in the answer text and response text more accurately and efficiently, thereby improving the scoring accuracy of the test text.
[0107] For example, a special character can be added before the first character of each step of the problem-solving approach in the answer text and the first character of each step in the answer text to uniquely mark the position information of each step. The question stem text, the encoded answer text, and the encoded answer text can also be concatenated to form a long character text to obtain the input information. When the text feature prediction model outputs based on the input information, it can use the special character to perform vector representation to accurately output the answer features corresponding to the answer text in the target question text, as well as the answer features corresponding to each problem-solving approach in the answer text.
[0108] In some embodiments, such as Figure 3 As shown, step 103, which involves obtaining the scoring result corresponding to the answer text, further includes:
[0109] Step 1031: Input the answer features and the answer features corresponding to each problem-solving approach into the feature interaction layer of the scoring prediction model to obtain the interaction features;
[0110] Step 1032: Input the interaction features into the problem-solving strategy prediction branch model of the scoring prediction model, and determine the problem-solving strategy that matches the answer text among all the problem-solving strategies of the answer text;
[0111] Step 1033: Input the problem-solving approach matching the answer text and the interaction features into the score prediction branch model of the scoring prediction model to obtain the score corresponding to the answer text;
[0112] Step 1034: The problem-solving approach that matches the answer text and the score are used as the scoring result.
[0113] Among them, the scoring prediction model is used to predict the score of the target test question text and identify the proof logic. For example... Figure 4 As shown, the scoring prediction model includes a feature interaction layer, a problem-solving approach prediction branch model, and a score prediction branch model;
[0114] Among them, the feature interaction layer is a shared feature interaction layer between the problem-solving idea prediction branch model and the score prediction branch model. That is, the problem-solving idea prediction branch model and the score prediction branch model share the same set of underlying vector representations for parameter sharing. The feature interaction layer can be generated based on multiple layers of Transformers (deep learning models using self-attention mechanisms) for interactive feature extraction, such as being formed by 12 Transformers layers.
[0115] The problem-solving approach prediction branch model can be built based on a classifier model and used to identify the problem-solving approach to which the answer text belongs.
[0116] The score prediction branch model can be built based on the regressor model and is used to predict the score corresponding to the answer text.
[0117] Optionally, before performing step 1031, the rating prediction model can be pre-trained, specifically including the following steps:
[0118] First, an initial score prediction model is created. This initial score prediction model is a multi-task model used to implement the problem-solving approach prediction task and the score prediction task, including an initial encoding layer, an initial problem-solving approach prediction branch model, and an initial score prediction branch model.
[0119] In addition, sample test question texts, as well as solution approach tags and score tags corresponding to sample answer texts within the sample test question texts, can be collected. Subsequently, the answer features and response features of the sample test question texts can be used as input to the initial scoring prediction model. The initial scoring prediction model, based on the initial encoding layer, the initial solution approach prediction branch model, and the initial score prediction branch model, infers and predicts the solution approach prediction results and score prediction results of the sample answer texts. Based on the solution approach prediction results and score prediction results of the sample answer texts, as well as the solution approach tags and score tags of the sample answer texts, the parameters of the initial scoring prediction model are iteratively trained to obtain the scoring prediction model.
[0120] After obtaining the scoring prediction model, the vector representation of the answer text and the vector representation of the answer text can be concatenated as input information.
[0121] The vector representation of the answer text includes the vector representation of the answer features obtained in step 102, and the vector representation of the answer text includes the vector representation of the answer features corresponding to all problem-solving approaches in the answer text obtained in step 102.
[0122] For example, the vector representation of the answer text is {the vector representation of solution approach 1, the vector representation of solution approach 2, ..., the vector representation of solution approach i, ..., the vector representation of solution approach n}, where n is the number of solution approaches contained in the answer text.
[0123] like Figure 4 As shown, the input information determined by the vector representation of the answer text is input to the feature interaction layer of the scoring prediction model. The feature interaction layer performs feature interaction based on the answer features corresponding to the answer text and the answer features corresponding to each problem-solving approach in the answer text to obtain interactive features.
[0124] The interaction features are input into the problem-solving approach prediction branch model, which then uses a classifier to identify the interaction features and identify the problem-solving approach that matches the answer text among all the problem-solving approaches in the answer text.
[0125] Next, the score prediction branch model, based on the regressor, predicts the score of the answer text according to the interaction features and the problem-solving ideas that match the answer text. The score rate of the answer text is input, and then the score corresponding to the answer text is obtained based on the score rate.
[0126] After obtaining the score, the problem-solving approach and score that match the answer text can be used together as the scoring result of the answer text. This allows for accurate score prediction and precise acquisition of the student's specific problem-solving approach. If the answer text does not reach the full score, the student can conduct a specific error analysis based on the predicted proof approach.
[0127] In this embodiment, by combining key answer steps, theorem features, and reasoning features in the answer text with key answer steps, theorem features, and reasoning features in the answer text, and simultaneously integrating multiple problem-solving approaches in the answer text, the system adaptively evaluates the problem-solving approach to which the answer text belongs, thereby providing a further scoring result. This scheme differs from the scoring methods for general problem-solving questions; it not only focuses on the final proof conclusion but also on the logical derivation between the student's steps and the existence of key steps, thus providing a more reasonable and accurate scoring result.
[0128] In some embodiments, the scoring prediction model is obtained by jointly training the loss function of the initial score prediction branch model and the loss function of the initial problem-solving strategy prediction branch model in the initial scoring prediction model.
[0129] Optionally, the training steps for the rating prediction model specifically include:
[0130] The answer features and response features of the sample test text are used as input to the initial scoring prediction model. The initial scoring prediction model, based on the initial encoding layer, the initial problem-solving approach prediction branch model, and the initial score prediction branch model, infers and predicts the problem-solving approach prediction results and score prediction results of the sample answer text.
[0131] The loss function of the initial problem-solving approach prediction branch model can be obtained based on the deviation between the problem-solving approach prediction result of the sample answer text and the problem-solving approach label of the sample answer text; the loss function of the initial score prediction branch model can be obtained based on the deviation between the score prediction result of the sample answer text and the score label of the sample answer text.
[0132] The loss functions of the initial problem-solving approach prediction branch model and the initial score prediction branch model are fused to obtain the total loss function of the initial score prediction model. Based on the total loss function, backpropagation training is performed to synchronously update the parameters of the multi-task branch model in the initial score prediction model until the initial score prediction model meets the training termination condition, thus obtaining the score prediction model. The training termination condition includes reaching the maximum number of training iterations or the model performance meeting the optimal performance requirements. This embodiment does not specify these conditions.
[0133] The total loss function of the initial score prediction model can be obtained by weighted summing of the loss function of the initial problem-solving approach prediction branch model and the loss function of the initial score prediction branch model.
[0134] In this embodiment, during the training of the scoring prediction model, the loss function of the initial scoring prediction model and the loss function of the initial score prediction branch model in the initial feature prediction model are jointly initialized. Through mutual learning, mutual promotion, mutual supplementation and mutual sharing among multiple branch models, collaborative training is carried out, so that the trained scoring prediction model has better model performance, stronger scoring rationality, and further improves the accuracy of test text scoring.
[0135] The test text scoring device provided by the present invention is described below. The test text scoring device described below can be referred to in correspondence with the test text scoring method described above.
[0136] like Figure 5 The diagram shown is a structural schematic of the test text scoring device provided in this embodiment. The device includes:
[0137] Unit 501 is used to retrieve the target test question text;
[0138] The feature extraction unit 502 is used to obtain, based on the target test question text, the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text in the target test question text;
[0139] The scoring unit 503 is used to obtain the scoring result corresponding to the answer text based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach;
[0140] The answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps; the answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps.
[0141] The test text scoring device provided in this embodiment identifies the target test text and obtains the key answering steps, the theorem features, and reasoning features corresponding to the answer text in the target test text, as well as the key answering steps, theorem features, and reasoning features corresponding to each problem-solving approach in the answer text of the target test text. By comprehensively considering the multi-dimensional features such as the derivation steps and derivation logic in the answer text and the answer text, the device scores the answer text accordingly. This avoids the problem of poor reliability of the scoring results due to coarse modeling granularity and the scoring principle relying mainly on similarity, thereby improving the reliability and rationality of test text scoring.
[0142] In some embodiments, the feature extraction unit 502 is specifically used for:
[0143] Based on the target test question text, determine the input information;
[0144] The input information is fed into the shared feature layer of the text feature prediction model to obtain the shared features of the target test text;
[0145] The shared features are input into the theorem prediction branch model of the text feature prediction model to obtain the key answering steps of the answer text, the theorem features corresponding to the key answering steps, and the key answering steps of each problem-solving approach, as well as the theorem features corresponding to the key answering steps.
[0146] The shared features, the key answering steps, and the key answer steps are input into the inference prediction branch model of the text feature prediction model to obtain the inference features corresponding to the key answering steps of the answer text and the inference features corresponding to the key answer steps of each problem-solving approach.
[0147] In some embodiments, the text feature prediction model is obtained by jointly training the loss function of the initial theorem prediction branch model and the loss function of the initial inference prediction branch model in the initial feature prediction model.
[0148] In some embodiments, the target test question text includes the question stem text, the answer text, and the response text;
[0149] The device also includes a determining unit, specifically used for:
[0150] The input information is determined based on the question text, the answer text, and the response text.
[0151] In some embodiments, the determining unit is further configured to:
[0152] The positions of each step in the problem-solving approach in the answer text and each step in the response text are encoded.
[0153] The input information is obtained by concatenating the question text, the encoded answer text, and the encoded response text.
[0154] In some embodiments, the scoring unit 503 is specifically used for:
[0155] The answer features and the answer features corresponding to each problem-solving approach are input into the feature interaction layer of the scoring prediction model to obtain the interaction features;
[0156] The interactive features are input into the problem-solving approach prediction branch model of the scoring prediction model, and the problem-solving approach that matches the answer text is determined from all the problem-solving approaches in the answer text.
[0157] The problem-solving approach matching the answer text and the interaction features are input into the score prediction branch model of the scoring prediction model to obtain the score corresponding to the answer text.
[0158] The solution approach that matches the answer text and the score are used as the scoring result.
[0159] In some embodiments, the scoring prediction model is obtained by jointly training the loss function of the initial score prediction branch model and the loss function of the initial problem-solving strategy prediction branch model in the initial scoring prediction model.
[0160] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604. The processor 601 can call logical instructions in the memory 603 to execute a test question text scoring method. This method includes: acquiring a target test question text; acquiring, based on the target test question text, answer features corresponding to the answer text in the target test question text, and answer features corresponding to each problem-solving approach in the answer text in the target test question text; acquiring the scoring result corresponding to the answer text based on the answer features and the answer features corresponding to each problem-solving approach; wherein the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps; the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps.
[0161] Furthermore, the logical instructions in the aforementioned memory 603 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0162] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the test text scoring method provided by the above methods. The method includes: obtaining a target test text; obtaining, based on the target test text, answer features corresponding to the answer text in the target test text, and answer features corresponding to each problem-solving approach in the answer text of the target test text; obtaining a scoring result corresponding to the answer text based on the answer features and the answer features corresponding to each problem-solving approach; wherein, the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps; the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps.
[0163] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the test text scoring method provided by the above methods. The method includes: acquiring a target test text; acquiring, based on the target test text, answer features corresponding to answer texts in the target test text, and answer features corresponding to each problem-solving approach in the answer texts of the target test text; acquiring a scoring result corresponding to the answer text based on the answer features and the answer features corresponding to each problem-solving approach; wherein the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps; the answer features include key answer steps, theorem features corresponding to the key answer steps, and reasoning features corresponding to the key answer steps.
[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for scoring test questions, characterized in that, include: Obtain the target test question text; Based on the target test question text, obtain the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text; Based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach, the scoring result corresponding to the answer text is obtained; The answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps; the answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps. The answer features corresponding to the answer text and the answer features corresponding to each of the problem-solving approaches are obtained by identifying key steps, predicting theorem features, and predicting inference features based on a single prediction model as a whole; or, by identifying key steps and predicting theorem features locally based on one of the two prediction models, and predicting inference features locally based on the other of the two prediction models, and then summing the prediction results of the two prediction models. The step of obtaining the scoring result corresponding to the answer text based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach includes: The answer features and the answer features corresponding to each problem-solving approach are input into the feature interaction layer of the scoring prediction model to obtain the interaction features; The interactive features are input into the problem-solving approach prediction branch model of the scoring prediction model, and the problem-solving approach that matches the answer text is determined from all the problem-solving approaches in the answer text. The problem-solving approach matching the answer text and the interaction features are input into the score prediction branch model of the scoring prediction model to obtain the score corresponding to the answer text. The solution approach that matches the answer text and the score are used as the scoring result.
2. The test text scoring method according to claim 1, characterized in that, The step of obtaining the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text, based on the target test question text, includes: Based on the target test question text, determine the input information; The input information is fed into the shared feature layer of the text feature prediction model to obtain the shared features of the target test text; The shared features are input into the theorem prediction branch model of the text feature prediction model to obtain the key answering steps of the answer text, the theorem features corresponding to the key answering steps, and the key answering steps of each problem-solving approach, as well as the theorem features corresponding to the key answering steps. The shared features, the key answering steps, and the key answer steps are input into the inference prediction branch model of the text feature prediction model to obtain the inference features corresponding to the key answering steps of the answer text and the inference features corresponding to the key answer steps of each problem-solving approach.
3. The test text scoring method according to claim 2, characterized in that, The text feature prediction model is obtained by jointly training the loss function of the initial theorem prediction branch model and the initial inference prediction branch model in the initial feature prediction model.
4. The test text scoring method according to claim 2, characterized in that, The target test question text includes the question stem text, the answer text, and the response text; The step of determining the input information based on the target test question text includes: The input information is determined based on the question text, the answer text, and the response text.
5. The test text scoring method according to claim 4, characterized in that, The step of determining the input information based on the question stem text, the answer text, and the response text includes: The positions of each step in the problem-solving approach in the answer text and each step in the response text are encoded. The input information is obtained by concatenating the question text, the encoded answer text, and the encoded response text.
6. The test text scoring method according to claim 1, characterized in that, The scoring prediction model is obtained by jointly training the loss function of the initial score prediction branch model and the loss function of the initial problem-solving strategy prediction branch model in the initial scoring prediction model.
7. A test question text scoring device, characterized in that, include: The acquisition unit is used to acquire the target test question text; The feature extraction unit is used to obtain, based on the target test question text, the answer features corresponding to the answer text in the target test question text, and the answer features corresponding to each problem-solving approach in the answer text of the target test question text; The scoring unit is used to obtain the scoring result corresponding to the answer text based on the answer characteristics and the answer characteristics corresponding to each problem-solving approach; The answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps; the answer features include key answering steps, the theorem features corresponding to the key answering steps, and the reasoning features corresponding to the key answering steps. The answer features corresponding to the answer text and the answer features corresponding to each of the problem-solving approaches are obtained by identifying key steps, predicting theorem features, and predicting inference features based on a single prediction model as a whole; or, by identifying key steps and predicting theorem features locally based on one of the two prediction models, and predicting inference features locally based on the other of the two prediction models, and then summing the prediction results of the two prediction models. The scoring unit is specifically used for: The answer features and the answer features corresponding to each problem-solving approach are input into the feature interaction layer of the scoring prediction model to obtain the interaction features; The interactive features are input into the problem-solving approach prediction branch model of the scoring prediction model, and the problem-solving approach that matches the answer text is determined from all the problem-solving approaches in the answer text. The problem-solving approach matching the answer text and the interaction features are input into the score prediction branch model of the scoring prediction model to obtain the score corresponding to the answer text. The solution approach that matches the answer text and the score are used as the scoring result.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the test text scoring method as described in any one of claims 1 to 6.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the test text scoring method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
Topic analysis method and device, electronic equipment and storage medium
CN111126610A
Text processing method and device, computer equipment and storage medium
CN112329435A