A subject correction method, device, equipment and storage medium

Abstract

The application provides a question correction method, device and equipment and a storage medium, wherein the method comprises the following steps: obtaining a subject title, a subject stem, question data, and standard answers; performing sub-question segmentation on the subject question data and the standard answers to obtain a plurality of sub-questions corresponding to a plurality of triplets, wherein each triplet comprises a question, question data, and a standard answer corresponding to a sub-question; obtaining a plurality of sub-question feature vectors and a whole question feature vector of the subject title according to the subject stem and the plurality of triplets corresponding to the plurality of sub-questions; and predicting the sub-question answering condition of the subject title according to the plurality of sub-question feature vectors and predicting the whole question answering condition of the subject title according to the whole question feature vector. The question correction method provided by the application can not only obtain a whole question correction result, but also obtain a sub-question correction result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a question grading method and device, equipment and a storage medium. BACKGROUND

[0002] In some scenarios (such as teaching evaluation scenarios), it is necessary to grade the answer data of a respondent to a question. The current question grading scheme is mostly a manual grading scheme, that is, a grader reviews and grades the answer data of a respondent to a question by referring to a standard answer. It can be understood that the manual grading scheme consumes a large amount of manpower, and as the grading time increases, the efficiency and accuracy of the grader in grading the question will decrease. SUMMARY

[0003] Therefore, the present application provides a question grading method, device, equipment and storage medium to solve the problem that the current manual grading scheme consumes a large amount of manpower, and as the grading time increases, the efficiency and accuracy of the grader in grading the question will decrease. The technical scheme is as follows:

[0004] A question grading method comprises the following steps:

[0005] Obtaining a question stem, a question, answer data and a standard answer of a subject question;

[0006] Dividing the question, the answer data and the standard answer of the subject question into a plurality of sub-questions to obtain a plurality of triplets corresponding to the plurality of sub-questions, wherein each triplet comprises a question, answer data and a standard answer of a corresponding sub-question;

[0007] Obtaining a plurality of sub-question feature vectors and a whole question feature vector of the subject question according to the question stem and the plurality of triplets corresponding to the plurality of sub-questions;

[0008] Predicting a sub-question answer situation of the subject question according to the plurality of sub-question feature vectors, and predicting a whole question answer situation of the subject question according to the whole question feature vector.

[0009] Optionally, the step of obtaining the plurality of sub-question feature vectors of the subject question according to the question stem and the plurality of triplets corresponding to the plurality of sub-questions comprises:

[0010] For each sub-question in the plurality of sub-questions:

[0011] Splicing the question, the answer data and the standard answer contained in the corresponding triplet to obtain a sub-question text of the sub-question;

[0012] Obtaining a feature vector capable of representing information of the sub-question itself according to the sub-question text of the sub-question.

[0013] fuse the feature vector capable of representing the information of the sub-question itself with the stem feature vector, and take the fused vector as a sub-question feature vector of the sub-question.

[0014] Optionally, the predicting the sub-question answering of the target question based on the sub-question feature vectors comprises:

[0015] for each of the sub-questions:

[0016] predicting the right or wrong of the sub-question based on the sub-question feature vector of the sub-question;

[0017] and / or extracting a key answer from the answering data and / or the standard answer of the sub-question based on the sub-question feature vector of the sub-question.

[0018] Optionally, the extracting a key answer from the answering data and / or the standard answer of the sub-question based on the sub-question feature vector of the sub-question comprises:

[0019] for each text unit in the answering data and / or the standard answer of the sub-question, determining a category label of the text unit from the set category labels based on a feature vector of the text unit in the sub-question feature vector of the sub-question, the category label of the text unit being capable of indicating whether the text unit is part of a key answer segment;

[0020] determining the key answer in the answering data and / or the standard answer of the sub-question based on the category label of each text unit in the answering data and / or the standard answer of the sub-question.

[0021] Optionally, the obtaining the sub-question feature vectors and the whole question feature vector of the target question based on the stem of the target question and the triplets corresponding to the sub-questions, and predicting the sub-question answering of the target question based on the sub-question feature vectors and predicting the whole question answering of the target question based on the whole question feature vector comprises:

[0022] processing the stem of the target question and the triplets corresponding to the sub-questions based on a pre-trained question grading model to obtain the sub-question answering of the target question and the whole question answering of the target question;

[0023] wherein the question grading model is trained based on the stem, the question, the answering data and the standard answer of a training question, and real sub-question answering data and real whole question answering data of the training question.

[0024] Optionally, the pre-trained question correction model is used to process the triple corresponding to the question stem and each of the sub-questions to obtain the sub-question answer of the question and the whole question answer of the question, including:

[0025] The question correction model is used to process the question stem of the question to obtain a question stem feature vector of the question.

[0026] The question correction model is used to process the triple corresponding to the question stem feature vector and each of the sub-questions to obtain a sub-question feature vector of the question.

[0027] The question correction model is used to process the sub-question feature vector of the question to obtain a whole question feature vector of the question.

[0028] The sub-question feature vector of the question is used as a basis to predict the sub-question answer of the question based on the question correction model.

[0029] The whole question feature vector of the question is used as a basis to predict the whole question answer of the question based on the question correction model.

[0030] Optionally, the prediction of the sub-question answer of the question based on the sub-question feature vector of the question and the question correction model includes:

[0031] For each sub-question:

[0032] The feature vector of the sub-question is used as a basis to predict the probability of the sub-question being answered correctly based on the question correction model, and the correctness of the sub-question is determined according to the probability of the sub-question being answered correctly.

[0033] The feature vector of the sub-question is used as a basis to predict the probability of each text unit in the answer data and / or the standard answer of the sub-question being each set class label based on the question correction model, the class label of each text unit is determined according to the probability of each text unit being each set class label, and the key answer in the answer data and / or the standard answer of the sub-question is determined according to the determined class label, wherein the class label of a text unit can indicate whether the text unit is part of a key answer fragment.

[0034] Optionally, the training process of the question correction model includes:

[0035] The question, answer data and standard answer of the training question are divided into sub-questions to obtain a triple corresponding to each of the sub-questions of the training question.

[0036] process the triple corresponding to the stem of the training question and the several sub-questions of the training question based on the question grading model, to obtain the sub-question feature vector and the whole question feature vector of the training question;

[0037] predict, based on the question grading model, the probability of each sub-question of the training question being answered correctly, the standard answer of each sub-question, and / or the probability of each text unit in the answer data being labeled as a set of labels, and the whole question score of the training question, according to the sub-question feature vector and the whole question feature vector of the training question; wherein the label of a text unit can indicate whether the text unit is part of a key answer fragment;

[0038] determine a prediction loss according to the prediction result of the question grading model on the training question, and the true or false label of each sub-question of the training question, the true label of each text unit in the answer text and / or the answer text of each sub-question, and the true whole question score of the training question;

[0039] update the parameters of the question grading model according to the determined prediction loss.

[0040] Optionally, the determination of the prediction loss according to the prediction result of the question grading model on the training question, and the true or false label of each sub-question of the training question, the true label of each text unit in the answer text and / or the answer text of each sub-question, and the true whole question score of the training question, comprises:

[0041] determine a first prediction loss according to the probability of each sub-question of the training question being answered correctly and the true or false label of each sub-question of the training question;

[0042] determine a second prediction loss according to the probability of each text unit in the answer data being labeled as a set of labels and the true label of each text unit in the answer text and / or the answer text of each sub-question of the training question;

[0043] determine a third prediction loss according to the whole question score predicted for the training question and the true whole question score of the training question.

[0044] A question grading device, comprising: a question data acquisition module, a sub-question segmentation module, and a question grading module;

[0045] The question data acquisition module is configured to acquire the stem, the question, the answer data, and the standard answer of a target question;

[0046] The small question segmentation module is configured to segment the questions, the answer data and the standard answers of the subject title into a plurality of small questions respectively corresponding to a plurality of triplets, wherein the triplets comprise the questions, the answer data and the standard answers of the corresponding small questions.

[0047] The question correction module is configured to acquire a plurality of small question feature vectors and a whole question feature vector of the subject title according to the stem of the subject title and the triplets corresponding to the plurality of small questions, predict the small question answer condition of the subject title according to the plurality of small question feature vectors, and predict the whole question answer condition of the subject title according to the whole question feature vector.

[0048] A question correction method and device, comprising a memory and a processor.

[0049] The memory is configured to store a program.

[0050] The processor is configured to execute the program to implement each step of the question correction method of any one of the above.

[0051] A computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement each step of the question correction method of any one of the above.

[0052] The question correction method, device, equipment and storage medium provided by the application, after obtaining the stem, questions, answer data and standard answers of the subject title, firstly segment the questions, answer data and standard answers of the subject title into a plurality of small questions respectively corresponding to a plurality of triplets, then acquire a plurality of small question feature vectors and a whole question feature vector of the subject title according to the stem of the subject title and the triplets corresponding to the plurality of small questions, and finally predict the small question answer condition of the subject title according to the plurality of small question feature vectors of the subject title, and predict the whole question answer condition of the subject title according to the whole question feature vector of the subject title. The question correction method provided by the application does not need human participation, greatly saves manpower, improves the correction efficiency, avoids the problems caused by human subjective factors, and the question correction method provided by the application can not only correct the whole question, but also correct the small question, that is, the final whole question correction result and the small question correction result can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0054] Figure 1 A schematic diagram of a hardware architecture involved in the present application;

[0055] Figure 2 A schematic diagram of a flow of a subject grading method provided by an embodiment of the present application;

[0056] Figure 3 A schematic diagram of a flow of a training process of a subject grading model provided by an embodiment of the present application;

[0057] Figure 4 A schematic diagram of a structure of a subject grading model provided by an embodiment of the present application;

[0058] Figure 5 A schematic diagram of a flow of processing, by a subject grading model provided by an embodiment of the present application and pre-trained, a subject title, a subject stem and a plurality of triplets corresponding to a plurality of sub-questions of the subject title, to obtain a sub-question answering result of the subject title and a whole question answering result of the subject title;

[0059] Figure 6 A schematic diagram of a structure of a subject grading device provided by an embodiment of the present application;

[0060] Figure 7 A schematic diagram of a structure of a subject grading apparatus provided by an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0062] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0063] In view of the fact that the artificial-based correction scheme needs to consume a large amount of manpower, and with the increase of the correction time, the efficiency and accuracy of the correction personnel in correcting the questions will decrease, the applicant tries to propose an automatic correction scheme and researches for this purpose, initially thinks of a correction scheme based on text classification, that is, judging the matching degree of the answer data of the question and the standard answer, so as to obtain a probability value of 0 to 1, and then determining whether the answer is correct or incorrect according to the obtained probability and the set probability threshold, or directly taking the obtained probability value as the normalized correction score.

[0064] The applicant further researches the above-mentioned correction scheme based on text classification and finds that although this scheme saves manpower and improves correction efficiency, it has many defects, two main defects are listed as follows: first, it can only give the score or right or wrong of the whole question, for the question with multiple sub-questions, it cannot locate the sub-questions, and cannot distinguish the answer data of the answerers who answer the same sub-questions incorrectly, for example, for a question with three sub-questions, the whole question score of the answerer who only answers the first sub-question incorrectly and the answerer who only answers the second sub-question incorrectly may be the same, in this case, the two kinds of answer data cannot be distinguished; second, the semantic information amount of the answer data of the question and the standard answer is not equal, the reasons for this problem mainly include two aspects, the first reason is that the answerer himself has divergence, some answerers may write very detailed steps, processes and explanations when answering, while some answerers write more concisely, some answerers list a lot of theorems, formulas and the like when answering, while some answerers write very little or nothing when encountering questions they do not know, the second reason is the difference between the standard answer and the answer data, the standard answer is often written in detail and mixed with some explanatory language, some information only plays an auxiliary explanation role and is not the main point of the question correction, the correction scheme based on text classification does not pay attention to the key answer information that needs to be aligned and compared.

[0065] In view of the fact that the above-mentioned correction scheme based on text classification has many defects, the applicant researches and finally proposes a better question correction method, which perfectly overcomes the defects of the above-mentioned correction scheme based on text classification.

[0066] Before introducing the question correction method provided by the present application, the hardware architecture involved in the present application is described.

[0067] In a possible implementation manner, as shown in Figure 1 The hardware architecture involved in the present application can include an electronic device 101 and a server 102.

[0068] Exemplarily, the electronic device 101 can be any electronic product capable of human-computer interaction, such as a PC, a notebook computer, a tablet computer, a palm computer, a PAD, etc.

[0069] It should be noted that, Figure 1 For example only, the type of the electronic device can be various, and is not limited to Figure 1 a notebook computer in the prior art.

[0070] Exemplarily, the server 102 can be a server, a server cluster composed of multiple servers, or a cloud computing server center. The server 102 can include a processor, a memory, a network interface, etc.

[0071] Exemplarily, the electronic device 101 can establish a connection with the server 102 and communicate with the server 102 through a wireless communication network; and exemplarily, the electronic device 101 can establish a connection with the server 102 and communicate with the server 102 through a wired communication network.

[0072] The electronic device 101 can acquire the related data of the to-be-corrected question, send the related data of the to-be-corrected question to the server 102, and correct the to-be-corrected question according to the question correction method provided by the present application, and send the correction result to the electronic device 101.

[0073] In another possible implementation, the hardware architecture involved in the present application can include an electronic device. The electronic device is an electronic product with strong data processing capability, such as a PC, a mobile phone, etc. The electronic device can acquire the related data of the to-be-corrected question, and correct the to-be-corrected question according to the question correction method provided by the present application.

[0074] Those skilled in the art should understand that the above-mentioned electronic device and server are only examples, and other existing or future electronic devices or servers, such as those applicable to the present application, should also be included in the protection scope of the present application, and are hereby included by reference.

[0075] Next, the question correction method provided by the present application will be introduced through the following embodiments.

[0076] First Embodiment

[0077] Please refer to Figure 2 , which shows a flowchart of the question correction method provided by the embodiment of the present application, which can include:

[0078] Step S201: acquiring the title, the stem, the question, the answer data and the standard answer of the question.

[0079] The subject title is a subject to be corrected. The subject title can be a physics calculation subject, for example. It should be noted that the physics calculation subject is only an example, and the subject title can also be other types of subjects.

[0080] The subject title question stem contains known conditions needed to understand the question, and the subject title question contains a question to be answered. It should be noted that the subject title question stem and the question are usually together. Since the two are used separately later, it is necessary to separate them. The subject title answer data is data for answering the subject title question by the answerer, and the subject title standard answer is the standard answer to the subject title question.

[0081] Step S202: The subject title question, answer data, and standard answer are divided into small questions to obtain a plurality of three-tuples corresponding to the small questions.

[0082] The three-tuple includes the question, answer data, and standard answer of the corresponding small question.

[0083] The subject title in this embodiment can be a subject including a plurality of small questions, or a subject including one small question.

[0084] If the number of small questions contained in the subject title is represented as k, after the subject title question, answer data, and standard answer are divided into small questions, k three-tuples [(Q1, S1, U1),..., (Qk, Sk, Uk)] can be obtained, where Qk represents the question of the kth small question of the subject title, Sk represents the standard answer of the kth small question, and Uk represents the answer data of the kth small question. k k k k k k

[0085] Optionally, the division can be based on a preset division rule. Optionally, the division rule can be based on the serial number information (such as (1), (2), etc.) in the beginning of the text line.

[0086] It should be noted that the subject title question, answer data, and standard answer are divided into small questions, so that the answer data of the subject title can be corrected by small questions later, to solve the problem that the same score cannot be distinguished in the whole subject correction.

[0087] Step S203: According to the subject title question stem and the plurality of three-tuples corresponding to the small questions, a plurality of small question feature vectors and a whole question feature vector of the subject title are obtained.

[0088] ​​​​​​Optionally, the process of obtaining the question stem feature vector and the question feature vector according to the question stem of the subject question and the triple corresponding to each sub-question of the subject question can include:

[0089] Step S2031, obtaining the question stem feature vector of the subject question according to the question stem of the subject question.

[0090] The question stem feature vector of the subject question is a feature vector capable of representing the context information of the question stem of the subject question.

[0091] Specifically, the process of obtaining the question stem feature vector of the subject question according to the question stem of the subject question can include: first, performing word segmentation processing on the question stem of the subject question based on a word segmentation model to obtain a text unit sequence composed of a plurality of text units (the text unit can be a word or a part of a word, i.e., a sub-word) obtained by word segmentation processing, i.e., a text unit sequence corresponding to the question stem of the subject question; then, for each text unit in the text unit sequence corresponding to the question stem of the subject question, obtaining a feature vector fused with the information of the text unit itself, the position information of the text unit, and the text segment information where the text unit is located; finally, calculating the feature vectors corresponding to each text unit in the text unit sequence corresponding to the question stem of the subject question based on a self-attention mechanism to obtain a feature vector capable of representing the context information of the question stem of the subject question, i.e., the question stem feature vector of the subject question.

[0092] In one possible implementation, the word segmentation model described above can use an existing word segmentation model. Considering that the data of some correction tasks has its particularity, for example, there are a large number of formulas, symbols, and numbers in the data of a physics calculation question, in order to obtain better word segmentation results, in another possible implementation, a word segmentation model suitable for the current correction task can be trained based on the BPE algorithm using the training data of the current correction task. For example, if the current correction task is a physics calculation question correction task, a word segmentation model suitable for the physics calculation question correction task can be trained based on the BPE algorithm using a large amount (e.g., ten million) of training text data of physics calculation questions.

[0093] Step S2032, obtaining the question stem feature vector of the subject question and the question feature vector of each sub-question of the subject question according to the question stem feature vector of the subject question and the triple corresponding to each sub-question of the subject question.

[0094] If the number of sub-questions contained in the subject question is denoted as k, then the question stem feature vector of the subject question and the triple (Q1, S1, U1) corresponding to the first sub-question can be used to obtain the question feature vector of the first sub-question, …, the question stem feature vector of the subject question and the triple (Qk, Sk, Uk) corresponding to the kth sub-question can be used to obtain the question feature vector of the kth sub-question. k ,S k ,U k) obtain a question characteristic vector of the kth sub-question.

[0095] Specifically, the process of obtaining the question characteristic vector of the kth sub-question can include:

[0096] For each sub-question of the question, the following steps are performed:

[0097] Step a1, splice the question, answer data and standard answer corresponding to the sub-question to obtain a sub-question text of the sub-question.

[0098] Specifically, a splice symbol <sep>Splice the corresponding triplets of the question, the answer data and the standard answer.

[0099] Step a2, according to the question text of the question, obtain the feature vector capable of representing the information of the question itself.

[0100] Specifically, according to the question text of the question, the process of obtaining the feature vector capable of representing the information of the question itself can include: first, based on the word segmentation model, the question text of the question is segmented to obtain the text unit sequence corresponding to the question; Then, for each text unit in the text unit sequence corresponding to the question, obtain the feature vector corresponding to the text unit, which is fused with the information of the text unit itself, the position information of the text unit, and the text segment information where the text unit is located; Finally, based on the self-attention mechanism, the feature vectors corresponding to each text unit contained in the text unit sequence corresponding to the question are calculated to obtain the feature vector capable of representing the information of the question itself.

[0101] Step a3, the feature vector capable of representing the information of the question itself is informationally fused with the title question stem feature vector, and the fused vector is used as the question feature vector of the question.

[0102] Specifically, based on the self-attention mechanism, the feature vector capable of representing the information of the question itself and the stem feature vector capable of representing the stem context information can be informationally fused to obtain a feature vector fused with the stem context information and the information of the question itself, that is, the question feature vector of the question.

[0103] Step S2033, according to the question feature vectors of the title questions, obtain the whole question feature vector of the title questions.

[0104] Specifically, the self-attention mechanism can be used to calculate the question feature vectors of the title questions to obtain the whole question feature vector of the title questions.

[0105] Step S204: According to the question feature vectors of the title questions, predict the question answering situation of the title questions.

[0106] Optionally, according to the question feature vectors of the title questions, the question answering situation of the title questions can include: for each question, according to the question feature vector of the question, predicting the question is correct or not, and / or according to the question feature vector of the question, extracting the key answer from the answer data and / or the standard answer of the question according to the question feature vector of the question.

[0107] It should be noted that when predicting the small question answering situation, whether to predict the correctness of the small question, or to extract the key answer, or to perform both, can be set according to the specific application situation, and similarly, when extracting the key answer, whether to extract the answer from the answer data or to extract the key answer from the standard answer, or to extract the key answer from both, can also be set according to the specific application situation. Exemplarily, the subject is a physical calculation question, the correctness of the small question can be predicted, and the key answer can be extracted from the answer data and the standard answer of the small question.

[0108] Step S205: According to the whole question feature vector of the subject, the whole question answering situation of the subject is predicted.

[0109] Optionally, according to the whole question feature vector of the subject, the whole question answering situation of the subject is predicted, including: according to the whole question feature vector of the subject, the whole question score of the subject is predicted.

[0110] It should be noted that the execution order of step S204 and step S205 is not limited in this embodiment, step S204 can be executed first, and then step S205 can be executed, or step S205 can be executed first, and then step S204 can be executed, or step S204 and step S205 can be executed in parallel.

[0111] The subject correction method provided by the embodiment of the application first divides the questions, the answer data and the standard answer of the subject into small questions to obtain a plurality of small question feature vectors and a whole question feature vector of the subject according to the stem of the subject and the plurality of small question feature vectors and the whole question feature vector of the subject, and finally, the small question answering situation of the subject is predicted according to the plurality of small question feature vectors of the subject, and the whole question answering situation of the subject is predicted according to the whole question feature vector of the subject. The subject correction method provided by the embodiment of the application does not need human participation, greatly saves manpower, improves correction efficiency, avoids problems caused by human subjective factors, and the subject correction method provided by the embodiment of the application can not only correct the whole question, but also correct the small question, that is, the final whole question correction result and the small question correction result can be obtained, and in addition, when correcting the small question, not only the correctness of the small question can be corrected, but also the key answer can be extracted from the answer data and the standard answer of the small question.

[0112] Second embodiment

[0113] In a possible implementation, steps S201 to S205 in the above embodiments can be implemented based on a pre-trained question correction model, that is, the question title, the question stem of the question title, and the triples corresponding to the several sub-questions of the question title are processed based on the pre-trained question correction model to obtain the sub-question answering conditions and the whole question answering conditions of the question title. This embodiment mainly introduces the training process of the question correction model.

[0114] The question correction model is trained by using training data in a training data set. The training data set includes a plurality of training data. Each training data includes a training sample and a sample label corresponding to the training sample. The training sample includes the stem, the question, the answer data, and the standard answer of a training question. The training sample label includes the real sub-question answering condition data and the real whole question answering condition data of the training question. For example, a training sample includes the stem, the question, the answer data, and the standard answer of a training question x. The sample label corresponding to the training sample includes the real right or wrong label of each sub-question of the training question x, the answer data and the standard answer of each text unit of each sub-question of the training question x, and the real whole question score of the training question x.

[0115] Next, the training process of the question correction model is introduced.

[0116] Please refer to Figure 3 , which shows a flowchart of the training process of the question correction model, which can include:

[0117] Step S301: obtaining training data from a training data set.

[0118] The training data includes the stem, the question, the answer data, and the standard answer of a training question, and the real sub-question answering condition data and the real whole question answering condition data of the training question.

[0119] Step S302: sub-question segmentation is performed on the question, the answer data, and the standard answer of the training question to obtain triples corresponding to the several sub-questions of the training question.

[0120] The triples include the question, the answer data, and the standard answer of the corresponding sub-question.

[0121] Step S303: processing the stem of the training question and the triples corresponding to the several sub-questions of the training question based on the question correction model to obtain the several sub-question feature vectors and the whole question feature vector of the training question.

[0122] Specifically, the process of processing the stem of the training question and the triples corresponding to the several sub-questions of the training question based on the question correction model to obtain the several sub-question feature vectors and the whole question feature vector of the training question can include:

[0123] Step b1: Process the question stems of the training questions based on the question grading model to obtain the question stem feature vectors of the training questions.

[0124] Among them, the feature vector of the training question stem is a feature vector that can represent the contextual information of the training question stem.

[0125] In one possible implementation, such as Figure 4 As shown, the question grading model may include a question stem encoder 401. The process of processing the question stems of training questions based on the question grading model may include: inputting the question stems of training questions into the question stem encoder 401 for encoding, and obtaining the question stem feature vectors of training questions output by the question stem encoder 401.

[0126] Specifically, when inputting the stem of the training question into the stem encoder 401 for encoding, the stem of the training question can first be segmented based on the word segmentation model to obtain the text unit sequence corresponding to the stem of the training question. Then, the text unit sequence corresponding to the stem of the training question is input into the stem encoder 401. For each text unit in the text unit sequence corresponding to the stem of the training question, the stem encoder 401 first obtains the word vector, the position vector, and the representation vector of the text segment in which the text unit is located. The word vector, the position vector, and the representation vector of the text segment in which the text unit is located are fused to obtain the feature vector corresponding to the text unit, which is a fusion of the information of the text unit itself, the position information, and the information of the text segment in which the text unit is located. After obtaining the feature vectors corresponding to each text unit in the text unit sequence corresponding to the stem of the training question, the stem encoder 401 calculates the feature vectors corresponding to each text unit based on the self-attention mechanism to obtain the feature vector that can represent the context information of the stem of the training question, i.e., the stem feature vector of the training question.

[0127] Among them, the stem encoder 401 can calculate the stem feature vector T of the training questions based on the following formula. C :

[0128]

[0129] R in the above formula C =[R c1 R c2 , ...,R cn R is a sequence of feature vectors consisting of the feature vectors corresponding to the n text units in the text unit sequence corresponding to the question stem of the training question. c1 R represents the feature vector corresponding to the first text unit in the sequence of text units corresponding to the stem of the training question, ..., R cn denotes the feature vector corresponding to the nth text unit, W Q , W K , W V is a model parameter, which is a trainable parameter, and d is a fixed parameter set.

[0130] Step b2, based on the question grading model, processing the stem feature vector of the training question and the triple corresponding to each small question of the training question to obtain the small question feature vector of the training question.

[0131] As shown in Figure 4 , the question grading model can include a small question encoder 402, and the process of processing the stem feature vector of the training question and the triple corresponding to each small question of the training question based on the question grading model can include: inputting the triple corresponding to each small question of the training question and the stem feature vector output by the stem encoder 401 into the small question encoder 402 for encoding to obtain the small question feature vector of the training question output by the small question encoder 402.

[0132] Specifically, when inputting the triple corresponding to each small question of the training question into the small question encoder 302, the question, the answer data and the standard answer in the triple corresponding to each small question can be spliced first (a splicing symbol such as <sep>(The text of the question, the standard answer, and the student's answer are concatenated) to obtain the sub-text of each sub-question of the training questions. For example, if there are two sub-questions, the question, the answer data, and the standard answer of the first sub-question are concatenated, and the resulting text is used as the sub-text of the first sub-question. The question, the answer data, and the standard answer of the second sub-question are concatenated, and the resulting text is used as the sub-text of the second sub-question. After obtaining the sub-text of each sub-question of the training questions, the sub-text of each sub-question of the training questions can be segmented based on the word segmentation model to obtain the text unit sequence corresponding to each sub-question of the training questions. After obtaining the word segmentation sequences corresponding to several sub-questions of the training questions, they are input into the sub-question encoder 402.

[0133] For each sub-question in the training questions, the sub-question encoder 402 first obtains the feature vectors corresponding to each text unit in the text unit sequence corresponding to the sub-question. Then, based on the self-attention mechanism, it calculates the feature vectors corresponding to each text unit in the text unit sequence corresponding to the sub-question to obtain the feature vector that can represent the information of the sub-question itself. Finally, based on the self-attention mechanism, it fuses the feature vector that can represent the information of the sub-question itself with the feature vector that can represent the context information of the question stem to obtain the feature vector that integrates the context information of the question stem and the information of the sub-question itself, which is used as the sub-question feature vector.

[0134] It should be noted that the feature vectors corresponding to each text unit in the text unit sequence corresponding to the sub-question are feature vectors that integrate information about the corresponding text unit itself, the position information of the corresponding text unit, and the information of the text segment in which the corresponding text unit is located. The method by which the sub-question encoder 402 obtains the feature vectors corresponding to each text unit in the text unit sequence corresponding to the sub-question is similar to the method by which the question stem encoder 401 obtains the feature vectors corresponding to each text unit in the text unit sequence corresponding to the question stem, and will not be described in detail here.

[0135] Specifically, for the i-th (i∈[1,k]) sub-question of the training questions, the sub-question encoder 402 can calculate the feature vector corresponding to each text unit in the text unit sequence corresponding to the sub-question based on the following formula, so as to obtain the feature vector that can represent the information of the i-th sub-question itself.

[0136]

[0137] R in the above formula iT =[R it1 R it2 , ...,R itm Let R be the feature vector sequence consisting of the feature vectors corresponding to the m text units in the i-th sub-question of the training questions. it1 Let R represent the feature vector corresponding to the first text unit in the sequence of text units corresponding to the i-th sub-question of the training questions, ..., R. itm W represents the feature vector corresponding to the m-th text unit in the above formula. Q W K W V Here are the model parameters, which are trainable parameters, and d is a fixed parameter.

[0138] Specifically, for the i-th (i∈[1,k]) sub-question of the training questions, the sub-question encoder 302 can use the following formula to generate a feature vector that can represent the information of the sub-question itself. The feature vector T of the question stem that can represent the contextual information of the question stem C Information fusion is performed to obtain the feature vector of the i-th sub-question of the training questions.

[0139]

[0140] W in the above formula Q W K W V Here are the model parameters, which are trainable parameters, and d is a fixed parameter.

[0141] Step b3: Based on the question grading model, process the feature vectors of several sub-questions of the training questions to obtain the feature vector of the whole question of the training questions.

[0142] like Figure 4 As shown, the question grading model may include a whole question encoder 403. The process of processing the feature vectors of several sub-questions of the training questions based on the question grading model to obtain the whole question feature vector of the training questions may include: inputting the feature vectors of several sub-questions of the training questions into the whole question encoder 403 for encoding, and obtaining the whole question feature vector of the training questions output by the whole question encoder 403.

[0143] Specifically, the whole-question encoder 403 can calculate the feature vectors of each sub-question based on a self-attention mechanism to obtain the whole-question feature vector of the target question. More specifically, the whole-question encoder 403 can calculate the feature vectors of each sub-question based on the following formula to obtain the whole-question feature vector T of the target question. whole :

[0144]

[0145] In the above formula Among them, T cls1 The feature vector of the first sub-question in the training questions. The feature vector at position [CLS], ..., T clsk The feature vector of the k-th sub-question the feature vector at the [CLS] position in the k small question feature vectors, W Q , W K , W V is a model parameter, which is a trainable parameter, and d is a fixed parameter set.

[0146] It should be noted that when encoding the small question text, a [CLS] mark is generally set at the beginning of the small question text, and the feature vector corresponding to the [CLS] is also determined when determining the feature vector corresponding to each text unit. When calculating the feature vector corresponding to each text unit in the text unit sequence corresponding to the small question based on the self-attention mechanism, the feature vector corresponding to each text unit is calculated with the feature vectors corresponding to other text units, and the feature vector corresponding to the [CLS] is also calculated with the feature vectors corresponding to other text units. In this way, the feature vector at the [CLS] position in the final small question feature vector contains the information of the entire small question. It should be noted that since the feature vector corresponding to each text unit in the text unit sequence corresponding to the small question is calculated with the feature vectors corresponding to other text units, i.e., the feature vector of each text unit in the final small question feature vector contains the information of the entire small question, but considering that [CLS] itself has no semantic interference, the application preferably adopts the feature vector sequence composed of the feature vectors corresponding to the [CLS] in the k small question feature vectors to determine the whole question feature vector T whole . Of course, the feature vector sequence composed of the feature vectors at non-[CLS] positions in the k small question feature vectors can also be used to determine the whole question feature vector T whole , for example, the feature vector sequence composed of the feature vector corresponding to the first text unit in the first small question feature vector,..., the feature vector corresponding to the first text unit in the kth small question feature vector is used to determine the whole question feature vector T whole .

[0147] Step S304: Based on the small question feature vectors of the training question, the probability of each small question being answered correctly is predicted based on the question grading model, and the probability of each text unit in the standard answer and / or answer data being a set class label, and based on the whole question feature vector of the training question, the whole question score of the training question is predicted based on the question grading model.

[0148] The class label of a text unit can indicate whether the text unit is part of a key answer fragment.

[0149] Optionally, each category label can include B (begin), I (inside), and O (outside). If the category label of a text unit is B (begin), it means that the text unit is located at the beginning of a key answer segment. If the category label of a text unit is I (inside), it means that the text unit is contained in a key answer segment. If the category label of a text unit is O (outside), it means that the text unit is not part of a key answer segment.

[0150] Optional, such as Figure 4 As shown, the question grading model may also include: a module 404 for predicting the answer to sub-questions and a module 405 for predicting the answer to the whole question.

[0151] After obtaining the feature vectors of several sub-questions of the training questions, the feature vectors of several sub-questions of the training questions can be input into the sub-question answer prediction module 404. Optionally, the sub-question answer prediction module may include a sub-question correctness classification module 4041 and a key answer extraction module 4042. The feature vectors of several sub-questions of the training questions are respectively input into the sub-question correctness classification module 4041 and the key answer extraction module 4042. For each sub-question, the sub-question correctness classification module 4041 predicts the probability of answering the sub-question correctly based on the sub-question feature vector of the sub-question. The key answer extraction module 4042 predicts the standard answer of the sub-question and / or the probability that the category label of each text unit in the answer data is each set category label (such as B, I, O) based on the sub-question feature vector of the sub-question.

[0152] Optionally, the question-based right / wrong classification module 4041 may include a fully connected layer. For the i-th (i∈[1,k]) question in the training questions, the feature vector of that question can be... The probability of answering the i-th sub-question correctly is obtained by passing the fully connected layer and then the sigmoid function.

[0153]

[0154] in, This indicates that the feature vector of the i-th sub-question is passed through a fully connected layer.

[0155] Optionally, it can be The feature vector T at the [CLS] position clsi The probability of answering the i-th sub-question correctly is obtained by passing the fully connected layer and then the sigmoid function.

[0156] Optionally, the key answer extraction module 3042 can include a fully connected layer, and for the standard answer and / or the jth text unit in the answer data of the ith sub-question of the training question, the corresponding feature vector After the fully connected layer and the softmax function, the class label of the jth text unit in the standard answer and / or the answer data of the ith sub-question of the training question can be obtained, and the class label is the probability of each set class label (such as B, I, O) Probability which can be expressed as:

[0157]

[0158] After obtaining the whole question feature vector T whole of the training question, the whole question feature vector T whole of the training question can be input into the whole question answer condition prediction module 405, and the whole question answer condition prediction module 405 predicts the whole question score of the training question according to the whole question feature vector of the training question.

[0159] Optionally, the whole question answer condition prediction module 405 can include a fully connected layer, and the whole question feature vector T whole of the training question can be input into the fully connected layer, and the whole question score score of the training question can be obtained, and score can be expressed as:

[0160] score = FNN (T whole ) (7)

[0161] Step S305: According to the probability predicted on the sub-question and the whole question score predicted on the whole question of the training question, and the true right or wrong label of each sub-question of the training question, the true class label of each text unit in the answer text and / or the answer text of each sub-question, and the true whole question score of the training question, the prediction loss is determined.

[0162] Specifically, the first prediction loss can be determined according to the probability of each sub-question of the training question being answered correctly and the true right or wrong label of each sub-question of the training question; the second prediction loss can be determined according to the probability of each text unit in the standard answer and / or the answer data of each sub-question of the training question being each set class label and the true class label of each text unit in the answer text and / or the answer text of each sub-question of the training question; and the third prediction loss can be determined according to the whole question score predicted for the training question and the true whole question score of the training question.

[0163] The first prediction loss can be calculated based on the following formula:

[0164]

[0165] wherein y i The true or false label of the i-th sub-question of the training question, y i is 0 if the i-th sub-question is answered incorrectly, k is the number of sub-questions of the training question, the first prediction loss L i is the average of the binary cross-entropy loss of each sub-question. cla

[0166] The second prediction loss can be calculated based on the following formula:

[0167]

[0168] wherein, represents the cross-entropy loss on the j-th text unit in the standard answer and / or answer data of the i-th sub-question of the training question, represents the label vector of the j-th text unit in the standard answer and / or answer data of the i-th sub-question of the training question, which is encoded in one-hot manner, indicating the class label of the text unit, the cross-entropy loss on each text unit in the standard answer and / or answer data of the i-th sub-question is calculated, summed up to obtain the key answer extraction loss of the i-th sub-question, and finally the key answer extraction losses of the k sub-questions are averaged to obtain the final key answer extraction loss L ext of the whole question.

[0169] The third prediction loss can be calculated based on the following formula:

[0170] L reg =(target-score) 2 (10)

[0171] wherein, target represents the true whole question score of the training question, and score is the predicted whole question score for the training question.

[0172] Step S306: According to the determined prediction loss, the question grading model is updated in parameters.

[0173] After obtaining the first prediction loss L cla , the second prediction loss L ext and the third prediction loss L reg , the first prediction loss L cla , the second prediction loss L ext and the third prediction loss L reg can be fused, and the question grading model is updated in parameters according to the fused loss.

[0174] wherein, the first prediction loss L cla , the second prediction loss L ext and the third prediction loss L​ reg There are various ways to perform fusion. In one possible implementation, the first prediction loss L cla , the second prediction loss L ext and the third prediction loss L reg are directly summed, that is:

[0175] L = L cla + L ext + L reg (11)

[0176] In another possible implementation, the first prediction loss L cla , the second prediction loss L ext and the third prediction loss L reg are weighted and summed, that is:

[0177] L = aL cls + bL ext + gL reg (12)

[0178] where a represents the weight of the first prediction loss L cla , b represents the weight of the second prediction loss L ext , and g represents the weight of the third prediction loss L reg . The specific values of a, b and g can be set according to specific scenarios.

[0179] The above steps S301 to S306 are performed multiple times for iterative optimization training until a preset number of iterations is reached or the model converges.

[0180] Third Embodiment

[0181] Based on the second embodiment described above, this embodiment introduces the process of processing the triplets corresponding to the subject title, the subject stem and a number of sub-questions based on the pre-trained subject correction model, obtaining the subject title, the subject stem and a number of sub-questions, and obtaining the subject title, the subject stem and a number of sub-questions.

[0182] Please refer to Figure 5 , which shows the process of processing the triplets corresponding to the subject title, the subject stem and a number of sub-questions based on the pre-trained subject correction model, obtaining the subject title, the subject stem and a number of sub-questions, and obtaining the subject title, the subject stem and a number of sub-questions, including:

[0183] Step S501: input the subject stem of the subject title into the stem encoder of the subject correction model to obtain the subject stem feature vector output by the stem encoder.

[0184] The process of encoding the stems of the test questions by the stem encoder is similar to the process of encoding the stems of the training questions by the stem encoder in the second embodiment, and details can be referred to the relevant part in the second embodiment, which will not be repeated herein.

[0185] Step S502: input the triplets and the stem feature vectors corresponding to the several sub-questions of the test question into the sub-question encoder of the test question grading model, to obtain the sub-question feature vectors of the test question output by the sub-question encoder.

[0186] The process of encoding the triplets and the stem feature vectors corresponding to the several sub-questions of the test question by the sub-question encoder is similar to the process of encoding the triplets and the stem feature vectors corresponding to the several sub-questions of the training questions by the sub-question encoder in the second embodiment, and details can be referred to the relevant part in the second embodiment, which will not be repeated herein.

[0187] Step S503: input the sub-question feature vectors of the test question into the whole-question encoder of the test question grading model, to obtain the whole-question feature vector of the test question output by the whole-question encoder.

[0188] The process of encoding the sub-question feature vectors of the test question by the whole-question encoder is similar to the process of encoding the sub-question feature vectors of the training questions by the whole-question encoder in the second embodiment, and details can be referred to the relevant part in the second embodiment, which will not be repeated herein.

[0189] Step S504: based on the sub-question feature vectors of the test question, predict the sub-question answering situation of the test question by the sub-question answering situation prediction module of the test question grading model.

[0190] As shown in FIG. 8, the sub-question answering situation prediction module includes a sub-question right-wrong classification module and a key answer extraction module. Figure 4 input the sub-question feature vectors of the test question into the sub-question right-wrong classification module, to obtain the probability of each sub-question being answered correctly output by the sub-question right-wrong classification module, and determine the right or wrong of each sub-question according to the probability of each sub-question being answered correctly, and it needs to be noted that the probability of each sub-question being answered correctly can also be directly used as the normalized grading score of the sub-question. input the sub-question feature vectors of the test question into the key answer extraction module, to obtain the probability of each text unit contained in the answering data and / or the standard answer of each sub-question of the test question being a set category label output by the key answer extraction module, determine the category label of each text unit contained in the answering data and / or the standard answer of each sub-question according to the probability output by the key answer extraction module, and determine the key answer in the answering data and / or the standard answer of each sub-question according to the category label of each text unit contained in the answering data and / or the standard answer of each sub-question.

[0191] Optionally, the process of determining the correctness of the sub-question according to the probability of answering the sub-question correctly can comprise: determining whether the probability of answering the sub-question correctly is greater than a preset probability threshold, and if yes, determining that the sub-question is answered correctly, and if not, determining that the sub-question is answered incorrectly.

[0192] Optionally, the process of determining the category label of the text unit according to the probability of each category label set for the text unit can comprise: determining the maximum probability from the probability of each category label set for the text unit, and determining the category label corresponding to the maximum probability as the category label of the text unit.

[0193] Step S505: Taking the whole question feature vector of the question as the prediction basis, the whole question answering condition prediction module of the question grading model predicts the whole question answering condition of the question.

[0194] Specifically, the whole question feature vector of the question is input into the whole question answering condition prediction module of the question grading model, and the whole question score output by the whole question answering condition prediction module is obtained.

[0195] Through the above process, not only the whole question answering condition (such as the whole question answering score) of the question can be obtained, but also the sub-question answering condition (such as the correctness of the sub-question, the sub-question answering data and the key answer in the standard answer) of the question can be obtained.

[0196] Fourth embodiment

[0197] The embodiment of the application further provides a question grading device, and the question grading device provided by the embodiment of the application is described below. The question grading device described below can be correspondingly referred to the question grading method described above.

[0198] Please refer to Figure 6 , which shows the structure diagram of the question grading device provided by the embodiment of the application, which can comprise: a question data acquisition module 601, a sub-question segmentation module 602 and a question grading module 603.

[0199] The question data acquisition module 601 is configured to acquire the question stem, the question, the answering data and the standard answer of the question.

[0200] The sub-question segmentation module 602 is configured to segment the question, the answering data and the standard answer of the question into sub-questions to obtain a plurality of triplets corresponding to the sub-questions respectively.

[0201] The triplet comprises the question, the answering data and the standard answer of the corresponding sub-question.

[0202] The question correction module 603 is configured to: acquire a plurality of sub-question feature vectors and a whole question feature vector of the subject question according to the subject question stem and the triples corresponding to the plurality of sub-questions, respectively; and predict a sub-question answer condition of the subject question according to the plurality of sub-question feature vectors, and predict a whole question answer condition of the subject question according to the whole question feature vector.

[0203] Optionally, when acquiring the plurality of sub-question feature vectors of the subject question according to the question stem feature vector and the triples corresponding to the plurality of sub-questions, the question correction module 603 is specifically configured to:

[0204] For each sub-question in the plurality of sub-questions:

[0205] splice the corresponding triple, the answer data, and the standard answer, and obtain a sub-question text of the sub-question after splicing;

[0206] acquire a feature vector capable of representing information of the sub-question itself according to the sub-question text of the sub-question;

[0207] fuse the feature vector capable of representing information of the sub-question itself and the question stem feature vector, and obtain a vector after fusion as the sub-question feature vector of the sub-question.

[0208] Optionally, when predicting the sub-question answer condition of the subject question according to the plurality of sub-question feature vectors, the question correction module 603 is specifically configured to:

[0209] For each sub-question in the plurality of sub-questions:

[0210] predict the sub-question according to the sub-question feature vector of the sub-question;

[0211] and / or extract a key answer from the answer data and / or the standard answer of the sub-question according to the sub-question feature vector of the sub-question;

[0212] The prediction of the whole question answer condition of the subject question according to the whole question feature vector includes:

[0213] predict a whole question score of the subject question according to the whole question feature vector.

[0214] Optionally, when extracting the key answer from the answer data and / or the standard answer of the sub-question according to the sub-question feature vector of the sub-question, the question correction module 603 is specifically configured to:

[0215] For each text unit in the answer data and / or the standard answer of the question, a category label of the text unit is determined from the set category labels according to the feature vector of the text unit in the question feature vector of the question, the category label of the text unit being capable of indicating whether the text unit is part of a key answer fragment;

[0216] According to the category label of each text unit in the answer data and / or the standard answer of the question, a key answer in the answer data and / or the standard answer of the question is determined.

[0217] Optionally, the question correction module 603 is specifically configured to:

[0218] based on the pre-trained question correction model, process the subject and the triples corresponding to the plurality of sub-questions of the question to obtain the sub-question answer of the question and the whole question answer of the question;

[0219] wherein the question correction model is trained by using the subject, the question, the answer data and the standard answer of the training question, and the real sub-question answer data and the real whole question answer data of the training question.

[0220] Optionally, the question correction module 603 is specifically configured to:

[0221] based on the question correction model, process the subject of the question to obtain the subject feature vector of the question;

[0222] based on the question correction model, process the subject feature vector and the triples corresponding to the plurality of sub-questions to obtain the plurality of sub-question feature vectors of the question;

[0223] based on the question correction model, process the plurality of sub-question feature vectors to obtain the whole question feature vector of the question;

[0224] based on the plurality of sub-question feature vectors of the question, predict the sub-question answer of the question based on the question correction model;

[0225] The whole question feature vector of the question is taken as a basis, and the whole question answering condition of the question is predicted based on the question grading model.

[0226] Optionally, when the question grading module 603 predicts the small question answering condition of the question based on the small question feature vectors of the question, the question grading module 603 is specifically configured to:

[0227] For each small question:

[0228] The probability of answering the small question correctly is predicted based on the feature vector of the small question, and the correctness of the small question is determined according to the probability of answering the small question correctly;

[0229] The probability of each text unit in the answering data and / or the standard answer of the small question being each set category label is predicted based on the feature vector of the small question, the category label of each text unit is determined according to the probability of each text unit being each set category label, and the key answer in the answering data and / or the standard answer of the small question is determined according to the determined category label, wherein the category label of a text unit can indicate whether the text unit is part of a key answer fragment.

[0230] Optionally, the question grading apparatus provided in the embodiment can further include a model training module. The model training module is configured to train the question grading model.

[0231] When the model training module trains the question grading model, the model training module is specifically configured to:

[0232] The questions, the answering data, and the standard answer of the training question are divided into small questions to obtain the triplets corresponding to the small questions of the training question, respectively;

[0233] The stem of the training question and the triplets corresponding to the small questions of the training question are processed based on the question grading model to obtain the small question feature vectors and the whole question feature vector of the training question;

[0234] The probability of each small question of the training question being answered correctly and the probability of each text unit in the standard answer and / or the answering data of each small question being each set category label are predicted based on the small question feature vectors of the training question and the question grading model, and the whole question score of the training question is predicted based on the whole question feature vector of the training question and the question grading model, wherein the category label of a text unit can indicate whether the text unit is part of a key answer fragment;

[0235] determine a prediction loss according to the probability predicted for each sub-question of the training question and the full question score predicted for the training question, and the true right or wrong label of each sub-question of the training question, the true class label of each text unit in the answer text and / or the answer text of each sub-question, and the true full question score of the training question;

[0236] update parameters of the question grading model according to the determined prediction loss.

[0237] Optionally, when determining the prediction loss according to the prediction result of the question grading model on the training question, and the true right or wrong label of each sub-question of the training question, the true class label of each text unit in the answer text and / or the answer text of each sub-question, and the true full question score of the training question, the model training module is specifically configured to:

[0238] determine a first prediction loss according to the probability of each sub-question of the training question being answered correctly and the true right or wrong label of each sub-question of the training question;

[0239] determine a second prediction loss according to the probability of each text unit in the standard answer and / or the answer data of each sub-question being set as a class label and the true class label of each text unit in the answer text and / or the answer text of each sub-question of the training question;

[0240] determine a third prediction loss according to the full question score predicted for the training question and the true full question score of the training question.

[0241] The question grading device provided in the embodiments of the present application, after obtaining the title of the question, the stem, the answer data and the standard answer of the question, first performs sub-question segmentation on the question, the answer data and the standard answer of the question to obtain a plurality of sub-questions respectively corresponding to a triple, then acquires a plurality of sub-question feature vectors and a full question feature vector of the question according to the stem of the question and the plurality of sub-questions respectively corresponding to the triple, and finally predicts the answer to the sub-questions of the question according to the plurality of sub-question feature vectors of the question and predicts the answer to the full question of the question according to the full question feature vector of the question. The question grading device provided in the embodiments of the present application can not only grade the full question, but also grade the sub-questions, that is, not only the full question grading result can be obtained, but also the sub-question grading result can be obtained, and in addition, when grading the sub-questions, not only the right or wrong of the sub-questions can be graded, but also the key answer can be extracted from the answer data and the standard answer of the sub-questions.

[0242] Fifth embodiment

[0243] The embodiments of the present application also provide a question grading device, please refer to Figure 7 , a structural schematic diagram of the question correcting device is shown, the question correcting device can include: at least one processor 701, at least one communication interface 702, at least one memory 703 and at least one communication bus 704;

[0244] In the embodiment of the application, the number of the processor 701, the communication interface 702, the memory 703 and the communication bus 704 is at least one, and the processor 701, the communication interface 702 and the memory 703 complete the communication with each other through the communication bus 704;

[0245] The processor 701 can be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the application, etc.

[0246] The memory 703 can include a high-speed RAM memory, and can also include a non-volatile memory, etc., such as at least one disk memory.

[0247] The memory stores a program, and the processor can call the program stored in the memory, and the program is used for:

[0248] Obtaining the question stem, the question, the answer data and the standard answer of the subject;

[0249] Dividing the question, the answer data and the standard answer of the subject to obtain a plurality of small questions respectively corresponding to a plurality of triplets, wherein the triplet includes the question, the answer data and the standard answer of the corresponding small question;

[0250] According to the question stem of the subject and the plurality of triplets respectively corresponding to the plurality of small questions, obtaining a plurality of small question feature vectors and a whole question feature vector of the subject;

[0251] According to the plurality of small question feature vectors, the small question answer situation of the subject is predicted, and according to the whole question feature vector, the whole question answer situation of the subject is predicted.

[0252] Optionally, the detailed functions and the extended functions of the program can refer to the description above.

[0253] Sixth embodiment

[0254] The embodiment of the application further provides a readable storage medium, the readable storage medium can store a program suitable for processor execution, and the program is used for:

[0255] Obtaining the question stem, the question, the answer data and the standard answer of the subject;

[0256] The questions, answer data and standard answers of the title purpose are divided into a plurality of triads, wherein the triad includes the question, answer data and standard answer of the corresponding small question;

[0257] According to the title purpose question stem and the triad corresponding to the plurality of small questions, a plurality of small question feature vectors and a whole question feature vector of the title purpose are obtained;

[0258] The small question answering situation of the title purpose is predicted according to the plurality of small question feature vectors, and the whole question answering situation of the title purpose is predicted according to the whole question feature vector.

[0259] Optionally, the refinement function and the expansion function of the program can refer to the description above.

[0260] Finally, it should be noted that in this document, the terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a…" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0261] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0262] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.< / sep> < / sep>

Claims

1. A method for grading exam questions, characterized in that, include: Obtain the question stem, question, answer data, and standard answer of the target question, wherein the question stem contains the known conditions needed to solve the problem, and the question contains the question that needs to be answered; The target question, answer data, and standard answer are divided into sub-questions to obtain several triples corresponding to each sub-question. Each triple includes the question, answer data, and standard answer of the corresponding sub-question. Based on the stem of the target question and the triples corresponding to the sub-questions, obtain the feature vectors of the sub-questions and the feature vector of the whole question. The system predicts the answer to each sub-question of the target question based on the feature vectors of the sub-questions, and predicts the answer to the entire target question based on the feature vector of the entire question. The step of obtaining the feature vectors of several sub-questions and the feature vector of the whole question based on the stem of the target question and the triples corresponding to the several sub-questions includes: For each of the aforementioned sub-questions: the question, answer data, and standard answer contained in the corresponding triple are concatenated to obtain the sub-question text; based on the sub-question text, a feature vector that can represent the information of the sub-question itself is obtained; the feature vector that can represent the information of the sub-question itself is fused with the question stem feature vector of the target question, and the resulting vector is used as the sub-question feature vector; Based on the question grading model, the feature vectors of several sub-questions are processed to obtain the whole question feature vector of the target question.

2. The question grading method according to claim 1, characterized in that, The step of predicting the answer status of the target question based on the feature vectors of the several sub-questions includes: For each of the several sub-questions: Based on the feature vector of the question, predict whether the question is correct or incorrect; And / or, based on the feature vector of the question, extract the key answer from the answer data and / or the standard answer of the question.

3. The question grading method according to claim 2, characterized in that, The step of extracting key answers from the answer data and / or standard answers based on the feature vector of the question includes: For each text unit in the answer data and / or standard answer for this question, the category label of the text unit is determined from each set category label based on the feature vector of the text unit in the feature vector of the question. The category label of the text unit can indicate whether the text unit is part of a key answer segment. Based on the category labels of each text unit in the response data and / or standard answer for this question, identify the key answer in the response data and / or standard answer for this question.

4. The question grading method according to claim 1, characterized in that, The step of obtaining feature vectors for several sub-questions and a feature vector for the entire target question based on the stem of the target question and the triples corresponding to the several sub-questions; predicting the answer to the sub-questions of the target question based on the feature vectors for the several sub-questions, and predicting the answer to the entire target question based on the feature vector for the entire question, includes: Based on the question grading model, the stem of the target question and the triples corresponding to the sub-questions are processed to obtain the answer status of the sub-questions and the answer status of the entire target question. The question grading model is trained using the question stems, questions, answer data, and standard answers of the training questions, as well as the actual answer data of the sub-questions and the actual answer data of the whole question.

5. The question grading method according to claim 4, characterized in that, Based on a pre-trained question grading model, the stem of the target question and the triples corresponding to the sub-questions are processed to obtain the sub-question answers and the overall answer to the target question, including: The question stem of the target question is processed based on the question grading model to obtain the question stem feature vector of the target question; Based on the question grading model, the feature vector of the question stem and the triples corresponding to the sub-questions are processed to obtain the feature vectors of the sub-questions of the target question. Based on the question grading model, the feature vectors of the several sub-questions are processed to obtain the whole question feature vector of the target question; Based on the feature vectors of several sub-questions of the target question, the answering situation of the sub-questions of the target question is predicted based on the question grading model; Based on the feature vector of the target question, the overall answer to the target question is predicted using the question grading model.

6. The question grading method according to claim 5, characterized in that, The step of predicting the answers to the sub-questions of the target question based on the feature vectors of several sub-questions of the target question and the question grading model includes: For each question: Based on the feature vector of the question, the probability of answering the question correctly is predicted according to the question grading model, and the correctness of the question is determined according to the probability of answering the question correctly. Based on the feature vector of the question, the probability that the category label of each text unit in the answer data and / or standard answer of the question is each set category label is predicted based on the question grading model. The category label of each text unit is determined according to the probability that the category label of each text unit is each set category label. The key answer in the answer data and / or standard answer of the question is determined according to the determined category label. The category label of a text unit can indicate whether the text unit is part of a key answer segment.

7. The question grading method according to claim 4, characterized in that, The training process of the question grading model includes: The training questions, answer data, and standard answers are divided into sub-questions to obtain triples corresponding to several sub-questions of the training questions; Based on the question grading model, the triplet corresponding to the question stem and several sub-questions of the training question are processed to obtain the feature vectors of several sub-questions and the feature vector of the whole question. Based on the feature vectors of several sub-questions and the feature vector of the whole question in the training question, the probability of answering each sub-question correctly, the probability that the standard answer and / or the category label of each text unit in the answer data of each sub-question are each set category label, and the whole question score of the training question are predicted based on the question grading model; wherein, the category label of a text unit can indicate whether the text unit is part of a key answer segment; Based on the prediction results of the question grading model on the training questions, as well as the true correct / incorrect labels of each sub-question of the training questions, the true category labels of each text unit in the answer text and / or answer text of each sub-question, and the true whole question score of the training questions, the prediction loss is determined. Based on the determined predicted loss, the parameters of the question grading model are updated.

8. The question grading method according to claim 7, characterized in that, The step of determining the prediction loss based on the prediction results of the question grading model on the training questions, and the true correct / incorrect labels for each sub-question of the training questions, the true category labels for each text unit in the answer text and / or answer text of each sub-question, and the true overall score of the training questions, includes: The first prediction loss is determined based on the probability of answering each sub-question of the training questions correctly and the true right or wrong label of each sub-question of the training questions; The second prediction loss is determined based on the probability of each set category label being the category label of each text unit in the standard answer and / or answer data of each sub-question of the training questions, and the true category label of each text unit in the answer text and / or answer text of each sub-question of the training questions. The third prediction loss is determined based on the predicted whole-question score for the training question and the actual whole-question score for the training question.

9. A problem-grading device, characterized in that, include: The module includes a question data acquisition module, a question segmentation module, and a question grading module. The question data acquisition module is used to acquire the question stem, question, answer data and standard answer of the target question, wherein the question stem contains the known conditions needed to solve the question and the question contains the question that needs to be answered. The sub-question segmentation module is used to segment the question, answer data, and standard answer of the target question into sub-questions to obtain a number of triples corresponding to each sub-question. The triples include the question, answer data, and standard answer of the corresponding sub-question. The question grading module is used to obtain the feature vectors of several sub-questions and the feature vector of the whole question based on the question stem of the target question and the triples corresponding to the several sub-questions, predict the answer to the sub-questions of the target question based on the feature vectors of the several sub-questions, and predict the answer to the whole question of the target question based on the feature vector of the whole question. When the question grading module obtains the feature vectors of the sub-questions and the feature vector of the whole question based on the question stem and the triples corresponding to the sub-questions of the target question, it is specifically used for: For each of the aforementioned sub-questions: the question, answer data, and standard answer contained in the corresponding triple are concatenated to obtain the sub-question text; based on the sub-question text, a feature vector that can represent the information of the sub-question itself is obtained; the feature vector that can represent the information of the sub-question itself is fused with the question stem feature vector of the target question, and the resulting vector is used as the sub-question feature vector; Based on the question grading model, the feature vectors of several sub-questions are processed to obtain the whole question feature vector of the target question.

10. A method and device for grading exam questions, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the question grading method as described in any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the question grading method as described in any one of claims 1 to 8.

Images