A subject correction method, device, equipment and storage medium

Abstract

The application provides a question correction method, device and equipment and a storage medium. The question correction method comprises the following steps: obtaining an answer text and a standard answer of a to-be-corrected question; obtaining text sequence features corresponding to the answer text and the standard answer respectively; constructing reasoning graphs corresponding to the answer text and the standard answer respectively, wherein the reasoning graphs can represent reasoning processes of the corresponding texts; obtaining structure features of the reasoning graphs corresponding to the answer text and the structure features of the reasoning graphs corresponding to the standard answer, to obtain reasoning graph structure features corresponding to the answer text and the standard answer respectively; and predicting a score of the answer text according to the text sequence features and the reasoning graph structure features of the answer text, and the text sequence features and the reasoning graph structure features of the standard answer. The question correction method provided by the application does not require human participation, and has high correction efficiency and correction accuracy.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, apparatus, device, and storage medium for grading questions. Background Technology

[0002] In certain scenarios (such as teaching assessments), it is necessary to grade the answers given by respondents to certain types of questions (such as mathematical proofs). Current grading methods are mostly based on manual grading, where graders review and grade the answers by referring to standard answers. Understandably, manual grading requires a significant amount of manpower, and the efficiency and accuracy of graders decrease as grading time increases. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, device, and storage medium for grading questions, to solve the problem that existing manual grading schemes require a large amount of manpower, and that the efficiency and accuracy of grading personnel decrease as grading time increases. The technical solution is as follows:

[0004] A method for grading exam questions includes:

[0005] Obtain the written answers and standard answers for the questions to be graded;

[0006] Obtain the text sequence features corresponding to the answer text and the standard answer, respectively;

[0007] Construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graphs can represent the reasoning process of the corresponding text;

[0008] Obtain the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer respectively;

[0009] Based on the text sequence features and reasoning graph structure features corresponding to the answer text, and the text sequence features and reasoning graph structure features corresponding to the standard answer, the score of the answer text is predicted.

[0010] Optionally, constructing the reasoning graphs corresponding to the answer text and the standard answer respectively includes:

[0011] For each text in the stated response text and the stated standard answer:

[0012] Based on the text sequence features corresponding to the text, multi-type step units are extracted from the text to obtain the step unit sequence corresponding to the text.

[0013] Predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text, so as to obtain the reasoning relationship information of the sequence of step units corresponding to the text;

[0014] Based on the reasoning relationship information of the step unit sequence corresponding to the text, construct the reasoning graph corresponding to the text.

[0015] Optionally, the step of predicting whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text includes:

[0016] The step unit sequence corresponding to the text is taken as the text sequence, and the text sequence feature corresponding to the step unit sequence is obtained as the step unit sequence feature corresponding to the text.

[0017] Based on the sequence features of the step units corresponding to the text, predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text.

[0018] Optionally, predicting whether there is a reasoning relationship between the step units in the step unit sequence corresponding to the text based on the step unit sequence features includes:

[0019] For each step unit in the sequence of step units corresponding to this text:

[0020] Based on the sequence features of the step units corresponding to the text, determine the relevance weight between the step unit and each other step unit;

[0021] Based on the relevance weights of this step unit and each of the other step units, determine whether there is a reasoning relationship between this step unit and each of the other step units.

[0022] Optionally, the reasoning graph includes several nodes and directed connecting lines between the nodes. Each node represents a step unit, and the two step units represented by two nodes connected by the directed connecting lines have a reasoning relationship.

[0023] The process of obtaining the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer includes:

[0024] For each text in the stated response text and the stated standard answer:

[0025] Traverse the nodes in the reasoning graph corresponding to the text: Based on the first feature of the currently traversed node and the first or second feature of the adjacent nodes of the currently traversed node, determine the second feature of the currently traversed node. The first feature of a node is obtained by fusing the text feature of the step unit represented by the node with the type feature of the step unit represented by the node. The text feature of a step unit is obtained based on the text sequence feature corresponding to the text to which the step unit belongs.

[0026] The structural features of the inference graph corresponding to the text are composed of the second features of each node in the inference graph.

[0027] Optionally, predicting the score of the answer text based on the text sequence features and inference graph structure features corresponding to the answer text, and the text sequence features and inference graph structure features corresponding to the standard answer, includes:

[0028] The reasoning graphs corresponding to the answer text and the standard answer are respectively divided into subgraphs from bottom to top to obtain the subgraph sequences corresponding to the answer text and the standard answer respectively;

[0029] The subgraph sequence features corresponding to the answer text are determined based on the subgraph sequence and reasoning graph structure features corresponding to the answer text, and the subgraph sequence features corresponding to the standard answer are determined based on the subgraph sequence and reasoning graph structure features corresponding to the standard answer.

[0030] Based on the features of the subgraph sequence corresponding to the answer text, the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text is retrieved from the pre-built reasoning subgraph library to obtain the similar subgraph sequence of the subgraph sequence corresponding to the answer text;

[0031] Based on the subgraph sequence features of the similar subgraph sequence, the text sequence features corresponding to the answer text and the standard answer, and the subgraph sequence features corresponding to the answer text and the standard answer, the score of the answer text is predicted.

[0032] Optionally, the inference graph includes several nodes and directed connections between the nodes, and the structural features of the inference graph include the features of each node in the corresponding inference graph;

[0033] Based on the subgraph sequence and reasoning graph structure features corresponding to a text in the answer text and the standard answer, the subgraph sequence features corresponding to that text are determined, including:

[0034] For each subgraph contained in the subgraph sequence corresponding to the text, the features of each node contained in the subgraph are obtained from the inference graph structure features corresponding to the text, and the features of the subgraph are determined based on the features of each node contained in the subgraph.

[0035] The features of the subgraph sequence corresponding to the text are composed of the features of each subgraph contained in the subgraph sequence.

[0036] Optionally, predicting the score of the answer text based on the subgraph sequence features of the similar subgraph sequence, the text sequence features corresponding to the answer text and the standard answer, and the subgraph sequence features corresponding to the answer text and the standard answer, includes:

[0037] The text sequence features corresponding to the answer text are fused with the text sequence features corresponding to the standard answer to obtain the first fused feature;

[0038] The subgraph sequence features corresponding to the answer text are fused with the subgraph sequence features corresponding to the standard answer to obtain a second fused feature;

[0039] The subgraph sequence features corresponding to the answer text are fused with the subgraph sequence features of the similar subgraph sequences to obtain the third fused feature;

[0040] The score of the response text is predicted based on the first fusion feature, the second fusion feature, and the third fusion feature.

[0041] Optionally, predicting the score of the response text based on the first fusion feature, the second fusion feature, and the third fusion feature includes:

[0042] The average of the first fused feature and the second fused feature is used as the first target feature.

[0043] The average value of the first fusion feature and the third fusion feature is used as the second target feature.

[0044] The score of the response text is predicted based on the first fusion feature, the first target feature, and the second target feature.

[0045] A question grading device includes: a question information acquisition module, a text sequence feature acquisition module, a reasoning graph acquisition module, a reasoning graph structure feature acquisition module, and a question grading result determination module;

[0046] The question information acquisition module is used to acquire the answer text and standard answer of the question to be graded;

[0047] The text sequence feature acquisition module is used to acquire the text sequence features corresponding to the answer text and the standard answer, respectively;

[0048] The reasoning graph acquisition module is used to construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graph can represent the reasoning process of the corresponding text;

[0049] The reasoning graph structure feature acquisition module is used to acquire the structure features of the reasoning graph corresponding to the answer text and the structure features of the reasoning graph corresponding to the standard answer, so as to obtain the reasoning graph structure features corresponding to the answer text and the standard answer respectively;

[0050] The question grading result determination module is used to predict the score of the answer text based on the text sequence features and reasoning graph structure features corresponding to the answer text, as well as the text sequence features and reasoning graph structure features corresponding to the standard answer.

[0051] A test paper grading device includes: a memory and a processor;

[0052] The memory is used to store programs;

[0053] The processor is used to execute the program to implement each step of the question grading method described above.

[0054] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the question grading method described in any of the preceding claims.

[0055] The question grading method, apparatus, device, and storage medium provided by this invention, after obtaining the answer text and standard answer of the question to be graded, firstly acquires the text sequence features corresponding to the answer text and standard answer respectively, then constructs a reasoning graph that can represent the reasoning process of the answer text and a reasoning graph that can represent the reasoning process of the standard answer, next acquires the structural features of the constructed reasoning graphs to obtain the structural features of the reasoning graphs corresponding to the answer text and standard answer respectively, and finally predicts the score of the answer text based on the text sequence features and reasoning graph structural features corresponding to the answer text and the standard answer. Compared with manual question grading schemes, since the question grading method provided by this invention does not require human intervention, it greatly saves manpower, improves grading efficiency, and avoids problems such as decreased grading accuracy caused by subjective human factors. Furthermore, considering that the answering process for certain types of questions is divergent, leading to significant differences between the answer text and the standard answer at the textual level, predicting the score of the answer text solely based on textual features would result in inaccurate predictions. Therefore, the question grading method provided by this invention, in addition to acquiring textual features, also structures the reasoning process of the answer text and the reasoning process of the standard answer into reasoning graphs, and acquires the structural features of these reasoning graphs. Then, based on the text sequence features corresponding to the answer text and the standard answer, combined with the structural features of the reasoning graphs corresponding to the answer text and the standard answer, the score of the answer text is predicted. For questions with divergent answering processes, combining the structural features of the reasoning graphs allows for the understanding of the reasoning logic of the answer text and the standard answer, thereby enabling a more accurate score prediction for the answer text. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0057] Figure 1 Examples of the written answers and standard answers to questions to be graded provided in embodiments of the present invention;

[0058] Figure 2 This is a schematic diagram of the hardware architecture involved in the present invention;

[0059] Figure 3 A flowchart illustrating the question grading method provided in an embodiment of the present invention;

[0060] Figure 4This is a flowchart illustrating the construction of reasoning graphs corresponding to the answer text and the standard answer, as provided in an embodiment of the present invention.

[0061] Figure 5 Examples of reasoning diagrams corresponding to the answer text and the standard answer provided in this embodiment of the invention;

[0062] Figure 6 This is a schematic diagram of the process for predicting the score of an answer text based on the text sequence features and reasoning graph structure features corresponding to the answer text, as well as the text sequence features and reasoning graph structure features corresponding to the standard answer, provided in an embodiment of the present invention.

[0063] Figure 7 This is a schematic diagram of the process for predicting the score of an answer text based on the text sequence features corresponding to the answer text and the standard answer, the subgraph sequence features corresponding to the answer text and the standard answer, and the subgraph sequence features of the similar subgraph sequences of the subgraph sequence corresponding to the answer text, as provided in this embodiment of the invention.

[0064] Figure 8 This is an example of a question grading model provided in an embodiment of the present invention;

[0065] Figure 9 This is a schematic diagram of the structure of the question correction device provided in an embodiment of the present invention;

[0066] Figure 10 This is a schematic diagram of the structure of the title correction device provided in an embodiment of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Given that manual grading requires a large amount of manpower, and that the efficiency and accuracy of grading decrease as grading time increases, the inventors of this case attempted to propose an automatic grading scheme and conducted research on it. Initially, they considered a text classification-based grading scheme, which involves using a model to determine the degree of matching between the answer text and the standard answer, thereby obtaining a probability value from 0 to 1. Then, based on the obtained probability and a set probability threshold, they determine whether the answer text is correct or incorrect, or directly use the obtained probability value as the normalized score.

[0070] The inventors in this case, through research on the aforementioned approach, discovered that the score for the answer text is determined by judging the degree of matching between the answer text and the standard answer. However, this approach is not effective for grading questions with divergent answer processes (such as mathematical proofs). The main reason is that the divergent nature of the answer process (e.g., the divergent proof process in mathematical proofs) leads to significant differences between the answer text and the standard answer at the textual level. Since the score is determined by judging the degree of matching between the answer text and the standard answer, the above approach will give higher scores to answer texts that are similar to the standard answer, but lower scores to answer texts that differ significantly from the standard answer. In fact, answer texts that differ significantly from the standard answer may also be the correct answer.

[0071] like Figure 1 As shown, the problem asks to prove that ΔAPE∽ΔFPA. A crucial step in proving ΔAPE∽ΔFPA is to utilize the properties of rhombus ABCE, first proving that ∠DAP=∠DCP, as follows. Figure 1 As shown, when proving ∠DAP=∠DCP, the proof approach in the answer text differs from that in the standard answer. The core difference lies in the different triangle congruences being proven in the intermediate steps. The standard answer proves ΔAPD≌ΔCPD (step 5), while the answer text proves ΔABP≌ΔCBP. Both can prove ∠DAP=∠DCP (step 6 in the standard answer, step 9 in the answer text). Therefore, although the answer text and the standard answer differ in their proof approaches, the answer text is still the correct proof.

[0072] In addition, for certain types of questions (such as mathematical proofs), there are a large number of mathematical formulas. When grading, it is necessary to understand the semantics of the formulas. The above approach treats all formulas as "mathematical formulas" in the same way when understanding the semantics of the formulas. This method cannot understand fine-grained semantic information, thus affecting the grading effect.

[0073] Given the numerous shortcomings of the above approach, the inventors of this case conducted further research and, through continuous research, finally proposed a more effective question grading method. This method can automatically grade the answer text and overcome the shortcomings of the above approach, ultimately providing a more accurate score.

[0074] Before introducing the question grading method provided by this invention, the hardware architecture involved in this invention will be described first.

[0075] In one possible implementation, such as Figure 2 As shown, the hardware architecture involved in this invention may include: electronic device 201 and server 202.

[0076] For example, electronic device 201 can be any electronic product that can interact with a user, such as PC, laptop, tablet, PDA, mobile phone, learning machine, smart TV, etc.

[0077] It should be noted that, Figure 2 This is just one example; there can be many types of electronic devices, not limited to... Figure 2 The laptop in the middle.

[0078] For example, server 202 can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. Server 202 may include processors, memory, and network interfaces, etc.

[0079] For example, electronic device 201 can establish a connection and communicate with server 202 through a wireless communication network; for example, electronic device 201 can establish a connection and communicate with server 202 through a wired communication network.

[0080] Electronic device 201 can obtain the answer text and standard answer of the question to be graded, and send the answer text and standard answer of the question to be graded to server 202. Server 202 grades the answer text according to the question grading method provided by the present invention, and sends the grading result of the answer text to electronic device 201.

[0081] In another possible implementation, the hardware architecture involved in this invention may include: an electronic device.

[0082] Electronic devices are electronic products with strong data processing capabilities, such as PCs, laptops, mobile phones, and learning machines. These electronic devices can acquire the answer text and standard answers of the questions to be graded, and grade the answer text according to the question grading method provided in this invention to obtain the graded answer text.

[0083] Those skilled in the art should understand that the above-described electronic devices and servers are merely examples, and other existing or future electronic devices or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0084] The following examples will illustrate the problem-solving method provided by this invention.

[0085] Please see Figure 3 The diagram illustrates a flowchart of a question grading method provided in an embodiment of the present invention, which may include:

[0086] Step S301: Obtain the answer text and standard answer of the question to be graded.

[0087] For example, the question to be graded is a mathematical proof question, the answer text is the answer text of the respondent to the mathematical proof question, and the standard answer is the standard answer to the mathematical proof question. It should be noted that this embodiment does not limit the question to be graded to a mathematical proof question. The question to be graded can also be other questions that have the same or similar characteristics as mathematical proof questions (for example, there is a reasoning process, the reasoning process is divergent, etc.).

[0088] In one possible scenario, the answer text can be obtained by performing OCR recognition on the image of the answer content. For example, in a teaching scenario, students' test papers or homework can be scanned or photographed, and then the image obtained by scanning or photographing can be subjected to OCR recognition to obtain the answer text of the questions to be graded.

[0089] Step S302: Obtain the text sequence features corresponding to the answer text and the standard answer respectively.

[0090] Text sequence encoding can be performed on the answer text and the standard answer separately to obtain the text sequence features corresponding to the answer text and the standard answer respectively. That is, the text sequence features corresponding to the answer text are the features obtained by text sequence encoding the answer text, and similarly, the text sequence features corresponding to the standard answer are the features obtained by text sequence encoding the standard answer.

[0091] Step S303: Construct reasoning diagrams corresponding to the answer text and the standard answer respectively.

[0092] Among them, the reasoning graph can represent the reasoning process of the corresponding text.

[0093] The process of constructing the reasoning graph corresponding to the answer text is the process of structuring the reasoning process of the answer text, that is, structuring the reasoning process of the answer text into a reasoning graph. Similarly, the process of constructing the reasoning graph corresponding to the standard answer is the process of structuring the reasoning process of the standard answer, that is, structuring the reasoning process of the standard answer into a reasoning graph.

[0094] Step S304: Obtain the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer respectively.

[0095] The reasoning graph corresponding to the answer text is structurally encoded to obtain the structural features of the reasoning graph corresponding to the answer text, i.e., the structural features of the reasoning graph corresponding to the answer text. Similarly, the reasoning graph corresponding to the standard answer is structurally encoded to obtain the structural features of the reasoning graph corresponding to the standard answer, i.e., the structural features of the reasoning graph corresponding to the standard answer.

[0096] Step S305: Based on the text sequence features and reasoning graph structure features corresponding to the answer text, as well as the text sequence features and reasoning graph structure features corresponding to the standard answer, predict the score of the answer text.

[0097] For questions with divergent answering processes, such as mathematical proofs, considering the significant differences between the answer text and the standard answer at the textual level, predicting the score of the answer text solely based on textual features will result in inaccurate predictions. To achieve accurate score predictions, this invention proposes, in addition to acquiring textual features (i.e., the text sequence features corresponding to the answer text), constructing reasoning graphs that represent the reasoning process of the answer text and reasoning graphs that represent the reasoning process of the standard answer. Based on this, the structural features of the constructed reasoning graphs are acquired to obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer, respectively. Then, based on the text sequence features and reasoning graph structural features corresponding to the answer text and the standard answer, the score of the answer text is predicted.

[0098] The question grading method provided in this invention, after obtaining the answer text and standard answer of the question to be graded, first acquires the text sequence features corresponding to the answer text and standard answer, respectively. Then, it constructs a reasoning graph that can represent the reasoning process of the answer text and a reasoning graph that can represent the reasoning process of the standard answer. Next, it acquires the structural features of the constructed reasoning graphs to obtain the structural features of the reasoning graphs corresponding to the answer text and standard answer, respectively. Finally, based on the text sequence features and reasoning graph structural features corresponding to the answer text and the standard answer, it predicts the score of the answer text. Compared with manual question grading schemes, since the question grading method provided in this invention does not require human intervention, it greatly saves manpower, improves grading efficiency, and avoids problems such as decreased grading accuracy caused by subjective human factors. Furthermore, considering that the answering process of certain types of questions (such as mathematical proof questions) is divergent, resulting in significant differences between the answer text and the standard answer at the text level, predicting the score of the answer text based solely on text-level features would lead to inaccurate predictions. Therefore, the question grading method provided in this embodiment of the invention, in addition to acquiring text-level features, also structures the reasoning process of the answer text and the reasoning process of the standard answer into reasoning graphs, and acquires the structural features of these reasoning graphs. Then, based on the text sequence features corresponding to the answer text and the standard answer, and combined with the structural features of the reasoning graphs corresponding to the answer text and the standard answer, the score of the answer text is predicted. For questions with divergent answering processes, combining the structural features of the reasoning graphs allows for the understanding of the reasoning logic between the answer text and the standard answer, thereby enabling a more accurate score prediction for the answer text.

[0099] In another embodiment of the present invention, the specific implementation process of "step S303: constructing reasoning graphs corresponding to the answer text and the standard answer respectively" in the above embodiment is described.

[0100] Please see Figure 4 The diagram illustrates the process of constructing reasoning graphs corresponding to the response text and the standard answer, which may include:

[0101] Step S401: For each text in the answer text and the standard answer, extract multi-type step units from the text according to the text sequence features corresponding to the text to obtain the step unit sequence corresponding to the text.

[0102] Specifically, based on the text sequence features corresponding to the text, the process of extracting multiple types of step units from the text may include: extracting step units from the text based on the text sequence features corresponding to the text, and determining the type of each extracted step unit from multiple set types (such as text type and multiple formula types), thus obtaining multiple types of step units, thereby obtaining the step unit sequence corresponding to the text, and the type of each step unit contained in the step unit sequence corresponding to the text.

[0103] In order to construct a reasoning graph that better represents the reasoning process, this invention extracts fine-grained step units and categories from the answer text and standard answer. These mainly involve different types of mathematical formulas and Chinese descriptions. That is, this invention distinguishes mathematical formulas in a fine-grained way. For example, “AD=CD” is of the type “line segment equation”, “∠ADP=∠CDP” is of the type “angle equation”, and “ΔAPD≌ΔCPD” is of the type “triangle congruence”.

[0104] Table 1 below shows examples of step units and step unit types obtained by extracting multiple types of step units from the answer text. Table 2 below shows examples of step units and step unit types obtained by extracting multiple types of step units from the standard answer:

[0105] Table 1 shows examples of step units and step unit types extracted from the response text.

[0106] 1 BD is the diagonal of rhombus ABCD Chinese description 2 \angle CBD=\angle ABP Angle equations 3 Quadrilateral ABCD is a rhombus. Chinese description 4 AB = CB Line segment equation 5 BP = BP Line segment equation 6 \triangle ABP\cong\triangle CBP Triangle congruence 7 \angle PAB = \angle PCB Angle equations 8 DCB = DAB Angle equations 9 DAP = DCP Angle equations

[0107] Table 2 shows examples of step units and step unit types extracted from the standard answers.

[0108]

[0109]

[0110] Step S402: Predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text, so as to obtain the reasoning relationship information of the sequence of step units corresponding to the text.

[0111] Specifically, the process of predicting whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text can include:

[0112] Step S4021: Take the step unit sequence corresponding to the text as a text sequence, and obtain the text sequence feature corresponding to the step unit sequence corresponding to the text as the step unit sequence feature corresponding to the text.

[0113] By taking the sequence of step units corresponding to the text as a text sequence and encoding the sequence of step units corresponding to the text, the text sequence feature corresponding to the sequence of step units corresponding to the text can be obtained. This text sequence feature contains the features of each step unit contained in the sequence of step units corresponding to the text.

[0114] Step S4022: Based on the step unit sequence features corresponding to the text, predict whether there is a reasoning relationship between the step units in the step unit sequence corresponding to the text, so as to obtain the reasoning relationship information of the step unit sequence corresponding to the text.

[0115] Specifically, for each step unit in the step unit sequence corresponding to the text: based on the features of the step unit sequence corresponding to the text, determine the relevance weight between the step unit and each other step unit; based on the relevance weight between the step unit and each other step unit, determine whether there is a reasoning relationship between the step unit and each other step unit.

[0116] For example, the step unit sequence corresponding to the text includes 5 step units, namely step 1, step 2, step 3, step 4, and step 5: For step 1, based on the feature f1 of step 1 in the step unit sequence features corresponding to the text and the feature f2 of step 2 in the step unit sequence features corresponding to the text, the relevance weight w between step 1 and step 2 is determined. 12 Based on feature f1 and feature f3 of step 3 in the sequence of step units corresponding to the text, the relevance weight w between step 1 and step 3 is determined. 13 Based on feature f1 and feature f4 of step 4 in the sequence of step units corresponding to the text, the relevance weight w between step 1 and step 4 is determined. 14 Based on feature f1 and feature f5 of step 5 in the sequence of step units corresponding to the text, the relevance weight w between step 1 and step 5 is determined. 15 In obtaining w 12 ~w 15 After that, it can be based on w 12 ~w 15 and the preset weight threshold w th Determine whether there is a reasoning relationship between step 1 and step 2, ..., whether there is a reasoning relationship between step 1 and step 5, for example, w 12 Greater than w th Then it can be determined that there is a reasoning relationship between step 1 and step 2, w 13 w 14 w 15 All less than w thIf we can determine that there is no reasoning relationship between step 1 and step 3, step 1 and step 4, and step 1 and step 5, then we can determine whether there is a reasoning relationship between step 2 and steps 1, 3, 4, and 5 in the same way, and so on. Similarly, we can determine whether there is a reasoning relationship between step 5 and steps 1, 2, 3, and 4 in the same way.

[0117] Optionally, the reasoning relationship information of the step unit sequence corresponding to the text can be represented by a reasoning relationship matrix. Assuming that the step unit sequence corresponding to the text includes M step units, the reasoning relationship matrix can be an M*M matrix. The elements in the first row of the matrix represent whether there is a reasoning relationship between the first step unit and each step unit, the elements in the second row of the matrix represent whether there is a reasoning relationship between the second step unit and each step unit, and so on. The elements in the Mth row of the matrix represent whether there is a reasoning relationship between the Mth step and each step unit. It should be noted that if there is a reasoning relationship between two step units, the corresponding element in the matrix is ​​represented by a first identifier, such as "1". If there is no reasoning relationship between two step units, the corresponding element in the matrix is ​​represented by a second identifier, such as "0". The first element in the first row, the second element in the second row, and the Mth element in the Mth row of the matrix are all the first identifiers.

[0118] Step S403: Construct a reasoning graph corresponding to the text based on the reasoning relationship information of the step unit sequence corresponding to the text.

[0119] Specifically, first, nodes representing step units are constructed, and then directed connections between nodes are constructed based on reasoning relationship information to obtain a reasoning graph. That is, the reasoning graph includes several nodes and directed connections between nodes. Each node represents a step unit, and two step units represented by two nodes connected by directed connections have a reasoning relationship.

[0120] Please see Figure 5 , Figure 5 Figure (a) in the table is a reasoning graph constructed based on the reasoning relationship information of the step unit sequence in Table 1 (i.e., the reasoning graph corresponding to the answer text). Figure 5 Figure (b) in the table is a reasoning graph constructed based on the reasoning relationship information of the step unit sequence in Table 2 (i.e., the reasoning graph corresponding to the standard answer). It should be noted that... Figure 5The dashed circles in the diagram represent virtual root nodes, which represent the problem. Nodes 1, 3, and 5 in Figure (a) represent the known conditions in the problem. Nodes 2, 4, 6, 7, and 8 in Figure (a) represent intermediate conclusions. Node 9 in Figure (a) represents the final conclusion. Nodes 1 and 4 in Figure (b) represent the known conditions in the problem. Nodes 2, 3, and 5 in Figure (b) represent intermediate conclusions. Node 6 represents the final conclusion.

[0121] Through the above process, reasoning graphs corresponding to the answer text and the standard answer can be obtained respectively. Furthermore, since the reasoning graph is constructed based on fine-grained step units, it is a fine-grained reasoning graph, which can better represent the reasoning process of the corresponding text.

[0122] In another embodiment of the present invention, the specific implementation process of "step S304: obtaining the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtaining the structural features of the reasoning graph corresponding to the answer text and the standard answer respectively" in the above embodiment will be described.

[0123] The process of obtaining the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer may include performing the following steps on each text in the answer text and the standard answer:

[0124] Step a1: Traverse the nodes in the reasoning graph corresponding to the text: Determine the second feature of the currently traversed node based on the first feature of the currently traversed node and the first or second feature of the adjacent nodes of the currently traversed node.

[0125] It should be noted that the second feature of a node is the feature of that node in the reasoning graph.

[0126] Optionally, when traversing the nodes in the inference graph corresponding to the text, the nodes in the inference graph corresponding to the text can be traversed from bottom to top.

[0127] The first feature of a node is obtained by fusing the text feature of the step unit represented by the node with the type feature of the step unit represented by the node. The text feature of a step unit is obtained based on the text sequence feature corresponding to the text to which the step unit belongs.

[0128] Specifically, the process of obtaining the first feature of a node in the inference graph corresponding to the text includes: obtaining the word vectors of each word contained in the step unit represented by the node from the text sequence features corresponding to the text; calculating the average of the word vectors of each word contained in the step unit represented by the node; using the average as the text feature of the step unit represented by the node; obtaining the type feature of the step unit represented by the node according to the type of the step unit represented by the node; and fusing the text feature of the step unit represented by the node with the type feature of the step unit represented by the node, using the fused feature as the first feature of the node.

[0129] As mentioned above, when determining the second feature of the currently traversed node, it is necessary to use either the first or second feature of the neighboring nodes of the currently traversed node. Whether to use the first or second feature for the neighboring nodes of the currently traversed node depends on whether the neighboring node has been traversed (or, in other words, whether the second feature of the neighboring node has been obtained). If the neighboring node has not been traversed (i.e., the second feature of the neighboring node has not been obtained), then the first feature of the neighboring node is used. If the neighboring node has been traversed (i.e., the second feature of the neighboring node has been obtained), then the second feature of the neighboring node is used.

[0130] For example, for Figure 5 The reasoning graph shown in Figure (a) assumes that the first node to be traversed is node 1, and the adjacent nodes of node 1 are node 2, node 4, and node 5. Since node 2, node 4, and node 5 have not yet been traversed, the second feature of node 1 is determined by using the first feature of node 1, the first feature of node 2, the first feature of node 4, and the first feature of node 5. Assuming that the second node to be traversed is node 2, the adjacent nodes of node 2 are node 1 and node 6. Since node 1 has been traversed, it has the second feature, while node 6 has not yet been traversed and does not have the second feature. Therefore, the second feature of node 2 can be determined by using the first feature of node 2, the second feature of node 1, and the first feature of node 6. The second features of other nodes are obtained in the same way.

[0131] Optionally, the process of determining the second feature of the currently traversed node based on the first feature of the currently traversed node and the first or second features of the neighboring nodes of the currently traversed node may include: determining the attention weight between the currently traversed node and its neighboring nodes based on the first feature of the currently traversed node and the first or second features of its neighboring nodes; and determining the second feature of the currently traversed node based on the first or second features of its neighboring nodes and the attention weight between the currently traversed node and its neighboring nodes.

[0132] Step a2: The second features of each node in the inference graph corresponding to the text constitute the structural features of the inference graph corresponding to the text, that is, the structural features of the inference graph corresponding to the text.

[0133] Through steps a1 and a2 above, the reasoning graph structure features corresponding to the answer text and the reasoning graph structure features corresponding to the standard answer can be obtained.

[0134] In another embodiment of the present invention, the specific implementation process of "step S505: predicting the score of the answer text based on the text sequence features and reasoning graph structure features corresponding to the answer text, and the text sequence features and reasoning graph structure features corresponding to the standard answer" in the above embodiment will be described.

[0135] Please see Figure 6 This diagram illustrates a process for predicting the score of a response text based on the text sequence features and inference graph structure features corresponding to the response text, as well as the text sequence features and inference graph structure features corresponding to the standard answer. The process may include:

[0136] Step S601: Decompose the reasoning graphs corresponding to the answer text and the standard answer into subgraphs from bottom to top to obtain the subgraph sequences corresponding to the answer text and the standard answer respectively.

[0137] In order to make full use of the explicitly known reasoning substructures in the reasoning graph without disrupting the reasoning order inherent in the reasoning graph, this embodiment uses a bottom-up approach to split the reasoning graphs corresponding to the answer text and the standard answer.

[0138] for Figure 5 The reasoning graph shown in Figure (a) can be subgraphed from bottom to top to obtain the subgraphs in Table 3 below:

[0139] Table 3 shows the subgraphs obtained by breaking down the reasoning graph corresponding to the answer text.

[0140] 1 1->2，3->4，3->8，6->7 Single condition 2 (2,4,5)->6，(7,8)->9 Multiple conditions

[0141] It should be noted that a single-condition subgraph refers to a subgraph from which a conclusion is drawn based on one condition (a subgraph where one node points to another node), while a multi-condition subgraph refers to a subgraph from which a conclusion is drawn based on multiple conditions (a subgraph from which multiple nodes point to another node).

[0142] After obtaining the subgraphs in Table 3, the subgraph sequence corresponding to the answer text is formed by the subgraphs in Table 3: {1->2, 3->4, (2,4,5)->6, 6->7, 3->8, (7,8)->9}. It should be noted that the order of the subgraphs is determined according to the reasoning process.

[0143] Step S602: Based on the subgraph sequence and reasoning graph structure features corresponding to the answer text, determine the subgraph sequence features corresponding to the answer text, and based on the subgraph sequence and reasoning graph structure features corresponding to the standard answer, determine the subgraph sequence features corresponding to the standard answer.

[0144] Specifically, based on the subgraph sequence and reasoning graph structure features corresponding to a text in the answer text and the standard answer, the subgraph sequence features corresponding to the text are determined, including: for each subgraph contained in the subgraph sequence corresponding to the text, the features of each node contained in the subgraph are obtained from the reasoning graph structure features corresponding to the text, and the features of the subgraph are determined based on the features of each node contained in the subgraph; the subgraph sequence features corresponding to the text are composed of the features of each subgraph contained in the subgraph sequence corresponding to the text.

[0145] The process of determining the features of a subgraph based on the features of each node it contains can include: averaging the features of each node in the subgraph, and using the average as the feature of the subgraph. It should be noted that node features are usually multidimensional features; when averaging the features of each node in the subgraph, the average of the same-dimensional features of each node can be used. It should also be noted that besides averaging the features of each node, the features of the subgraph can be obtained in other ways, such as taking the maximum value of the same-dimensional features of each node and using that value as the feature of the subgraph.

[0146] Step S603: Based on the subgraph sequence features corresponding to the answer text, retrieve the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text from the pre-built reasoning subgraph library to obtain the similar subgraph sequence to the subgraph sequence corresponding to the answer text.

[0147] The reasoning subgraph library may include several subgraphs and features of each subgraph. The construction process of the reasoning subgraph library may include: obtaining standard answers to several questions and / or full-mark answer texts for several questions; constructing a reasoning graph corresponding to each text (the reasoning graph construction method is the same as the reasoning graph construction method provided in the above embodiment); splitting the reasoning graph corresponding to each text from bottom to top into subgraphs to obtain several subgraphs; determining the features of each subgraph (the method for determining the features of each subgraph is the same as the method for determining the features of a subgraph provided above), and the reasoning subgraph library is composed of several subgraphs and the features of each subgraph. It should be noted that the reasoning subgraph library may also include only several subgraphs. When it is necessary to retrieve the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text to be graded from the reasoning subgraph library, the features of each subgraph in the reasoning subgraph library are then determined.

[0148] The process of retrieving the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text from a pre-constructed inference subgraph library, based on the subgraph sequence features corresponding to the answer text, to obtain a similar subgraph sequence of the subgraph sequence corresponding to the answer text, may include: for each subgraph in the subgraph sequence corresponding to the answer text, calculating the similarity between the features of the subgraph and the features of each subgraph in the inference subgraph library, obtaining the feature similarity between the subgraph and each subgraph in the inference subgraph library, and determining the subgraph with the highest feature similarity to the subgraph in the inference graph as the similar subgraph of the subgraph; and forming a similar subgraph sequence by arranging the similar subgraphs of each subgraph in the subgraph sequence corresponding to the answer text in the order of the subgraphs contained in the subgraph sequence corresponding to the answer text, to obtain a similar subgraph sequence of the subgraph sequence corresponding to the answer text. For example, if the subgraph sequence corresponding to the answer text is {g1, g2, g3, g4}, the similar subgraph of subgraph g1 is g1′, the similar subgraph of subgraph g2 is g2′, the similar subgraph of subgraph g3 is g3′, and the similar subgraph of subgraph g4 is g4′, then the similar subgraph sequence of the subgraph sequence corresponding to the answer text is {g1′, g2′, g3′, g4′}.

[0149] Step S604: Based on the text sequence features corresponding to the answer text and the standard answer, the subgraph sequence features corresponding to the answer text and the standard answer, and the subgraph sequence features of the similar subgraph sequences of the subgraph sequence corresponding to the answer text, predict the score of the answer text.

[0150] Please see Figure 7 This diagram illustrates a flowchart for predicting the score of a response text based on the text sequence features corresponding to the response text and the standard answer, the subgraph sequence features corresponding to the response text and the standard answer, and the subgraph sequence features of similar subgraph sequences to the subgraph sequence corresponding to the response text. The flowchart may include:

[0151] Step S701a: Fuse the text sequence features corresponding to the answer text with the text sequence features corresponding to the standard answer to obtain the first fused feature.

[0152] This step involves feature fusion between the response text and the standard answer at the text sequence level.

[0153] Step S701b: Fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features corresponding to the standard answer to obtain the second fused feature.

[0154] This step involves feature fusion at the subgraph sequence level between the subgraph sequence corresponding to the answer text and the subgraph sequence corresponding to the standard answer.

[0155] Step S701c: Fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features of similar subgraph sequences corresponding to the answer text to obtain the third fused feature.

[0156] This step involves feature fusion at the subgraph sequence level between the subgraph sequence corresponding to the answer text and similar subgraph sequences corresponding to the answer text.

[0157] Step S702: Based on the first fusion feature, the second fusion feature, and the third fusion feature, predict the score of the response text.

[0158] Specifically, the average of the first fusion feature and the second fusion feature is used as the first target feature, and the average of the first fusion feature and the third fusion feature is used as the second target feature. The score of the answer text is predicted based on the first fusion feature, the first target feature and the second target feature.

[0159] The score for the answer text can be obtained through the above process.

[0160] Optionally, after obtaining the score of the answer text, the correctness of the answer text can be determined by combining it with the set score threshold. Specifically, it is determined whether the score of the answer text is greater than the set score threshold. If the score of the answer text is greater than the set score threshold, the answer text is determined to be correct. If the score of the answer text is less than or equal to the set score threshold, the answer text is determined to be incorrect.

[0161] In one possible implementation, steps S302 to S305 can be implemented based on a pre-trained question grading model. That is, after obtaining the answer text and standard answer of the question to be graded, the answer text and standard answer are input into the pre-trained question grading model. The question grading model obtains the text sequence features corresponding to the answer text and standard answer, respectively, and constructs inference graphs corresponding to the answer text and standard answer, respectively. Based on the structural features of the inference graphs corresponding to the answer text and the standard answer, the structural features of the inference graphs corresponding to the answer text and the standard answer are obtained. The score of the answer text is predicted and output based on the text sequence features and inference graph structural features corresponding to the answer text and the standard answer. It should be noted that this invention does not limit steps S302 to S305 to be implemented based on a model; that is, this invention does not limit the specific implementation form of steps S302 to S305.

[0162] Please see Figure 8 This demonstrates an example of a question grading model that enables question grading. The following section will discuss models based on... Figure 8 The process of grading questions using the illustrated question grading model will be introduced.

[0163] like Figure 8As shown, the question grading model may include: a first text sequence encoding module 801, a step unit extraction and classification module 802, a second text sequence encoding module 803, a reasoning graph construction and encoding module 804, a reasoning graph splitting and subgraph feature determination module 805, a reasoning subgraph retrieval module 806, a text sequence feature fusion module 807, a subgraph sequence feature fusion module 808, and a score prediction module 809.

[0164] based on Figure 8 The question grading model shown can include the following processes for grading questions:

[0165] Step b1: Obtain the answer text and standard answer of the questions to be graded.

[0166] Step b2: Input the answer text and the standard answer into the first text sequence encoding module 801 of the question grading model for text sequence encoding to obtain the text sequence features corresponding to the answer text and the standard answer respectively.

[0167] Optionally, the first text sequence encoding module 801 can be a Transformer-based encoder. Of course, this embodiment is not limited to this, and the first text sequence encoding module 801 can also be other modules capable of text sequence encoding.

[0168] Step b3: For each text in the answer text and the standard answer, input the text sequence features corresponding to the text into the step unit extraction and classification module 802 to extract and classify the step units, so as to obtain the step unit sequence corresponding to the text and the type of each step unit contained in the step unit sequence corresponding to the text.

[0169] For each text in the response text and the standard answer, the step unit extraction and classification module 802 predicts the start and end positions of different types of step units in the text to obtain different types of step units. Optionally, the step unit extraction and classification module 802 may, but is not limited to, using the GlobalPointer method to predict the start and end positions of different types of step units. It should be noted that GlobalPointer is a decoding method for multi-class entity extraction.

[0170] Step b4: For each text in the answer text and the standard answer, input the step unit sequence corresponding to the text into the second text sequence encoding module 803 for text sequence encoding to obtain the step unit sequence feature corresponding to the text.

[0171] The second text sequence encoding module 803 takes the input step unit sequence as a text sequence and performs text sequence encoding to obtain the step unit sequence features.

[0172] Step b5: For each text in the answer text and the standard answer, input the step unit sequence features corresponding to the text into the reasoning graph construction and encoding module 804 to obtain the reasoning graph structure features corresponding to the text.

[0173] For each text in the answer text and the standard answer, the reasoning graph construction and encoding module 804 first predicts whether there is a reasoning relationship between the step units in the step unit sequence corresponding to the text based on the step unit sequence features corresponding to the text, so as to obtain the reasoning relationship information of the step unit sequence corresponding to the text. Then, based on the reasoning relationship information of the step unit sequence corresponding to the text, the reasoning graph corresponding to the text is constructed. Finally, the reasoning graph corresponding to the text is encoded and the structural features of the reasoning graph corresponding to the text are output.

[0174] When encoding the inference graph corresponding to a text, the inference graph construction and encoding module 804 traverses the nodes in the inference graph corresponding to the text. Based on the first feature of the currently traversed node and the first or second feature of the adjacent nodes of the currently traversed node, the second feature of the currently traversed node is determined. After obtaining the second features of each node in the inference graph corresponding to the text, the second features of each node in the inference graph corresponding to the text are used to form the structural features of the inference graph corresponding to the text.

[0175] Optionally, the reasoning graph construction and encoding module 804 can determine the second feature of the currently traversed node based on a multi-head attention mechanism. Assuming the currently traversed node is node i, the second feature of node i can be determined according to the following formula:

[0176]

[0177] Where, n i N represents the feature of node i in the inference graph corresponding to the text, i.e., the second feature. i Let K represent the set of neighboring nodes of node i, containing all neighboring nodes of node i. Let K represent the number of attention heads, and W represent the number of attention heads. k This represents the weight matrix of the k-th attention head. This represents the attention weight between node i and its neighbor j, determined based on the k-th attention head. This indicates that the feature vectors obtained after K attention processing are concatenated and integrated.

[0178] In the above formula, α ij It can be calculated using the following formula:

[0179]

[0180]

[0181] It should be noted that Equation (3) represents the concatenation of the first feature of node i with the first or second feature of node j (if node j has been traversed, then the second feature of node j is used; otherwise, the first feature of node j is used) and then passed through a fully connected FNN layer to obtain the attention score e of node i and node j. ij Equation (2) represents the expression for e ij Normalization is performed to obtain the attention weights α between node i and node j. ij It should also be noted that the method for obtaining the first feature of a node can be found in the relevant parts of the above embodiments, and will not be repeated here.

[0182] For a more detailed implementation process and related explanation of the inference graph construction and encoding module 804 for constructing the inference graph and obtaining the structural features of the inference graph, please refer to the relevant parts of the above embodiments. This embodiment will not repeat the details here.

[0183] Step b6: For each text in the answer text and the standard answer, input the inference graph and inference graph structure features corresponding to the text into the inference graph splitting and subgraph feature determination module 805 to obtain the subgraph sequence and subgraph sequence features corresponding to the text.

[0184] For each text in the answer text and the standard answer, the reasoning graph splitting and subgraph feature determination module 805 first splits the reasoning graph of the text from bottom to top to obtain the subgraph sequence corresponding to the text. Then, based on the subgraph sequence corresponding to the text and the structural features of the reasoning graph, it determines the subgraph sequence features corresponding to the text. The specific implementation process of the reasoning graph splitting and subgraph feature determination module 805 in splitting the reasoning graph into subgraphs and determining the subgraph sequence features can be found in the relevant parts of the above embodiments, and will not be repeated here.

[0185] Step b7: Input the subgraph sequence features corresponding to the answer text into the reasoning subgraph retrieval module 806 to obtain the similar subgraph sequence to the subgraph sequence corresponding to the answer text.

[0186] The reasoning subgraph retrieval module 806 retrieves the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text from the pre-built reasoning subgraph library based on the subgraph sequence features corresponding to the answer text, so as to obtain the similar subgraph sequence of the subgraph sequence corresponding to the answer text. For a more specific implementation process, please refer to the relevant parts of the above embodiments, which will not be repeated here.

[0187] Step b8-a: Input the text sequence features corresponding to the answer text and the standard answer into the text sequence feature fusion module 807 for feature fusion to obtain the first fused feature.

[0188] Optionally, the text sequence feature fusion module 807 can fuse the text sequence features corresponding to the answer text with the text sequence features corresponding to the standard answer in the following manner to obtain the first fused feature S:

[0189]

[0190] Among them, S S S represents the text sequence features corresponding to the standard answer. U W represents the text sequence features corresponding to the answer text. Q W K d and d are both model parameters.

[0191] Step b8-b: Input the subgraph sequence features corresponding to the answer text and the subgraph sequence features corresponding to the standard answer into the subgraph sequence feature module 808 for feature fusion to obtain the second fused feature. Then, input the subgraph sequence features corresponding to the answer text and the subgraph sequence features of similar subgraph sequences of the subgraph sequence corresponding to the answer text into the subgraph sequence feature module 808 for feature fusion to obtain the third fused feature.

[0192] Optionally, the subgraph sequence feature module 808 can fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features corresponding to the standard answer using the following formula to obtain a second fused feature.

[0193]

[0194] in, This represents the subgraph sequence features corresponding to the standard answer. This represents the subgraph sequence features corresponding to the answer text.

[0195] Optionally, the subgraph sequence feature module 808 can fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features of similar subgraph sequences of the subgraph sequence corresponding to the answer text using the following formula to obtain a third fused feature.

[0196]

[0197] in, Subgraph sequence features representing similar subgraph sequences to the subgraph sequence corresponding to the answer text.

[0198] Step b9: Input the first fusion feature, the second fusion feature and the third fusion feature into the score prediction module 809 to predict the score and obtain the score of the answer text.

[0199] Optionally, the score prediction module 809 may include a pooling layer and a fully connected layer, and the first fused feature S and the second fused feature S... The input pooling layer performs average pooling processing, and the resulting feature is used as the first target feature T1. The first fused feature S and the third fused feature are then combined. The input pooling layer performs average pooling to obtain the feature T2 as the second target feature. The first fusion feature S, the first target feature T1, and the second target feature T2 are concatenated and input into the fully connected layer. The output of the fully connected layer is passed through the Sigmoid function to obtain the score of the answer text.

[0200] It should be noted that for each text in the answer text and the standard answer, when performing text sequence encoding, a [CLS] marker is generally set at the beginning of the text. [CLS] also participates in the encoding process and interacts with each text unit within the text. Therefore, the feature at the [CLS] position ultimately contains information about the entire text. Since the text sequence features corresponding to the answer text and the standard answer each contain the feature at the [CLS] position, the first fused feature S obtained by fusing the text sequence features corresponding to the answer text and the text sequence features corresponding to the standard answer will also contain the fused feature at the [CLS] position. Thus, the fused feature S at the [CLS] position in the first fused feature S can be... CLS The features of the first and second targets are concatenated and then input into the fully connected layer.

[0201] In this embodiment, the question grading model uses the answer texts and standard answers of the training questions as training samples, and the actual scores of the answer texts in the training samples as sample labels. When training the question grading model, the answer texts and standard answers of the training questions are first input into the model to obtain the scores of the answer texts of the training questions output by the model. Then, the prediction loss of the question grading model is determined based on the scores output by the model and the actual scores of the answer texts of the training questions. Finally, the parameters of the question grading model are updated based on the prediction loss. The question grading model is trained multiple times using different training data in the above manner until the training termination condition is met (e.g., model convergence, or reaching a preset number of training iterations).

[0202] In one possible implementation, the step unit extraction and classification module in the initial question grading model can use a pre-trained step unit extraction and classification model. During the training of the question grading model, the parameters of the step unit extraction and classification module are fixed. In another possible implementation, the step unit extraction and classification module uses an untrained step unit extraction and classification model. During the training of the question grading model, the parameters of the step unit extraction and classification module are updated. For the second implementation, in addition to labeling the true scores of the answers to the training questions, it is also necessary to label the position and type of each step unit in the answer text of the training questions, as well as the position and type of each step unit in the standard answer of the training questions. Furthermore, when determining the prediction loss, in addition to determining the prediction loss of the question grading model on the score prediction task, the prediction loss of the question grading model on the step unit extraction and classification task can also be determined. Finally, the parameters of the question grading model are updated based on these two losses (the two losses are fused, and the parameters of the question grading model are updated based on the fused loss).

[0203] Compared to manual question grading methods, the question grading method based on a question grading model provided in this invention requires no human intervention, thus significantly saving manpower, improving grading efficiency, and avoiding problems such as decreased accuracy due to subjective human factors. Furthermore, in the question grading method provided in this invention, the question grading model, in addition to acquiring text-level features, also structures the reasoning process of the answer text and the reasoning process of the standard answer into reasoning graphs. During structuring, the question grading model extracts fine-grained step units and categories from the answer text and the standard answer. The resulting reasoning graphs can better represent the reasoning process. After obtaining the reasoning graphs, further features of the reasoning graphs are acquired. Then, based on the text sequence features corresponding to the answer text and the standard answer, combined with the structural features of the reasoning graphs corresponding to the answer text and the standard answer, the score of the answer text is predicted. For questions with divergent answering processes, the question grading model, combined with the structural features of the reasoning graphs, can understand the reasoning logic between the answer text and the standard answer, thereby predicting a more accurate score for the answer text.

[0204] This invention also provides a question grading device. The question grading device provided in this invention will be described below. The question grading device described below can be referred to in correspondence with the question grading method described above.

[0205] Please see Figure 9The diagram shows a schematic of the structure of the question grading device provided in an embodiment of the present invention, which may include: a question information acquisition module 901, a text sequence feature acquisition module 902, a reasoning graph acquisition module 903, a reasoning graph structure feature acquisition module 904, and a question grading result determination module 905.

[0206] The question information acquisition module 901 is used to acquire the answer text and standard answer of the questions to be graded.

[0207] The text sequence feature acquisition module 902 is used to acquire the text sequence features corresponding to the answer text and the standard answer, respectively.

[0208] The reasoning graph acquisition module 903 is used to construct reasoning graphs corresponding to the answer text and the standard answer, respectively. The reasoning graphs can represent the reasoning process of the corresponding text.

[0209] The reasoning graph structure feature acquisition module 904 is used to acquire the structure features of the reasoning graph corresponding to the answer text and the structure features of the reasoning graph corresponding to the standard answer, so as to obtain the reasoning graph structure features corresponding to the answer text and the standard answer respectively.

[0210] The question grading result determination module 905 is used to predict the score of the answer text based on the text sequence features and reasoning graph structure features corresponding to the answer text, as well as the text sequence features and reasoning graph structure features corresponding to the standard answer.

[0211] Optionally, the reasoning graph acquisition module 903 includes: a step unit extraction and classification module and a reasoning graph construction module.

[0212] The step unit extraction and classification module is used to extract multiple types of step units from each text in the answer text and the standard answer, based on the text sequence features corresponding to that text, so as to obtain the step unit sequence corresponding to that text.

[0213] The reasoning graph construction module is used to predict whether there is a reasoning relationship between the step units in the step unit sequence corresponding to the text, so as to obtain the reasoning relationship information of the step unit sequence corresponding to the text, and construct the reasoning graph corresponding to the text based on the reasoning relationship information of the step unit sequence corresponding to the text.

[0214] Optionally, when predicting whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text, the reasoning graph construction module is specifically used for:

[0215] The step unit sequence corresponding to the text is taken as the text sequence, and the text sequence feature corresponding to the step unit sequence is obtained as the step unit sequence feature corresponding to the text.

[0216] Based on the sequence features of the step units corresponding to the text, predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text.

[0217] Optionally, when the inference graph construction module predicts whether there is an inference relationship between the step units in the sequence of step units corresponding to the text based on the features of the step unit sequence corresponding to the text, it is specifically used for:

[0218] For each step unit in the sequence of step units corresponding to this text:

[0219] Based on the sequence features of the step units corresponding to the text, determine the relevance weight between the step unit and each other step unit;

[0220] Based on the relevance weights of this step unit and each of the other step units, determine whether there is a reasoning relationship between this step unit and each of the other step units.

[0221] Optionally, the reasoning graph includes several nodes and directed connecting lines between the nodes. Each node represents a step unit, and two step units represented by two nodes connected by a directed connecting line have a reasoning relationship.

[0222] When acquiring the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, the reasoning graph structure feature acquisition module 904 is specifically used for:

[0223] For each text in the stated response text and the stated standard answer:

[0224] Traverse the nodes in the reasoning graph corresponding to the text: Based on the first feature of the currently traversed node and the first or second feature of the adjacent nodes of the currently traversed node, determine the second feature of the currently traversed node. The first feature of a node is obtained by fusing the text feature of the step unit represented by the node with the type feature of the step unit represented by the node. The text feature of a step unit is obtained based on the text sequence feature corresponding to the text to which the step unit belongs.

[0225] The structural features of the inference graph corresponding to the text are composed of the second features of each node in the inference graph.

[0226] Optionally, the question grading result determination module 905 includes: a reasoning graph splitting module, a subgraph sequence feature determination module, a reasoning subgraph retrieval module, and a grading result prediction module.

[0227] The reasoning graph splitting module is used to split the reasoning graphs corresponding to the answer text and the standard answer into subgraphs from bottom to top, so as to obtain the subgraph sequences corresponding to the answer text and the standard answer respectively.

[0228] The subgraph sequence feature determination module is used to determine the subgraph sequence features corresponding to the answer text based on the subgraph sequence and reasoning graph structure features corresponding to the answer text, and to determine the subgraph sequence features corresponding to the standard answer based on the subgraph sequence and reasoning graph structure features corresponding to the standard answer.

[0229] The reasoning subgraph retrieval module is used to retrieve the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text from a pre-built reasoning subgraph library based on the subgraph sequence features corresponding to the answer text, so as to obtain the similar subgraph sequence of the subgraph sequence corresponding to the answer text.

[0230] The grading result prediction module is used to predict the score of the answer text based on the subgraph sequence features of the similar subgraph sequence, the text sequence features corresponding to the answer text and the standard answer, and the subgraph sequence features corresponding to the answer text and the standard answer, respectively.

[0231] Optionally, the inference graph includes several nodes and directed connections between the nodes, and the structural features of the inference graph include the features of each node in the corresponding inference graph.

[0232] When determining the subgraph sequence feature of a text based on the subgraph sequence and reasoning graph structure features corresponding to that text in the answer text and the standard answer, the subgraph sequence feature determination module is specifically used for:

[0233] For each subgraph contained in the subgraph sequence corresponding to the text, the features of each node contained in the subgraph are obtained from the inference graph structure features corresponding to the text, and the features of the subgraph are determined based on the features of each node contained in the subgraph.

[0234] The features of the subgraph sequence corresponding to the text are composed of the features of each subgraph contained in the subgraph sequence.

[0235] Optionally, the grading result prediction module includes: a text sequence feature fusion module, a subgraph sequence feature fusion module, and a score prediction module.

[0236] The text sequence feature fusion module is used to fuse the text sequence features corresponding to the answer text with the text sequence features corresponding to the standard answer to obtain the first fused feature.

[0237] The subgraph sequence feature fusion module is used to fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features corresponding to the standard answer to obtain a second fused feature, and to fuse the subgraph sequence features corresponding to the answer text with the subgraph sequence features of the similar subgraph sequences to obtain a third fused feature;

[0238] The score prediction module is used to predict the score of the answer text based on the first fusion feature, the second fusion feature and the third fusion feature.

[0239] Optionally, when predicting the score of the response text based on the first fusion feature, the second fusion feature, and the third fusion feature, the score prediction module is specifically used for:

[0240] The average of the first fused feature and the second fused feature is used as the first target feature.

[0241] The average value of the first fusion feature and the third fusion feature is used as the second target feature.

[0242] The score of the response text is predicted based on the first fusion feature, the first target feature, and the second target feature.

[0243] The question grading device provided in this invention eliminates the need for manual intervention during question grading, significantly saving manpower, improving grading efficiency, and avoiding problems such as decreased accuracy due to subjective human factors. Furthermore, considering the divergent nature of the answering process for certain types of questions (such as mathematical proofs), which can lead to significant differences between the answer text and the standard answer at the textual level, predicting scores based solely on textual features would result in inaccurate predictions. Therefore, in addition to acquiring textual features, the question grading device in this invention also structures the reasoning processes of the answer text and the standard answer into reasoning graphs, respectively, and acquires the features of these reasoning graphs. Based on the text sequence features corresponding to the answer text and the standard answer, and combined with the structural features of the reasoning graphs corresponding to the answer text and the standard answer, the device predicts the score of the answer text. For questions with divergent answering processes, combining the structural features of the reasoning graphs allows for the understanding of the reasoning logic between the answer text and the standard answer, thereby enabling a more accurate score prediction.

[0244] This invention also provides a question grading device; please refer to [link / reference]. Figure 10 The diagram shows the structure of the problem correction device, which may include: at least one processor 1001, at least one communication interface 1002, at least one memory 1003 and at least one communication bus 1004.

[0245] In this embodiment of the invention, the number of processor 1001, communication interface 1002, memory 1003 and communication bus 1004 is at least one, and processor 1001, communication interface 1002 and memory 1003 communicate with each other through communication bus 1004.

[0246] The processor 1001 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0247] The memory 1003 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0248] The memory stores a program, which the processor can call. The program is used for:

[0249] Obtain the written answers and standard answers for the questions to be graded;

[0250] Obtain the text sequence features corresponding to the answer text and the standard answer, respectively;

[0251] Construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graphs can represent the reasoning process of the corresponding text;

[0252] Obtain the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer respectively;

[0253] Based on the text sequence features and reasoning graph structure features corresponding to the answer text, and the text sequence features and reasoning graph structure features corresponding to the standard answer, the score of the answer text is predicted.

[0254] Optionally, the refined and extended functions of the program can be found in the description above.

[0255] This invention also provides a readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0256] Obtain the written answers and standard answers for the questions to be graded;

[0257] Obtain the text sequence features corresponding to the answer text and the standard answer, respectively;

[0258] Construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graphs can represent the reasoning process of the corresponding text;

[0259] Obtain the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer respectively;

[0260] Based on the text sequence features and reasoning graph structure features corresponding to the answer text, and the text sequence features and reasoning graph structure features corresponding to the standard answer, the score of the answer text is predicted.

[0261] Optionally, the refined and extended functions of the program can be found in the description above.

[0262] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0263] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0264] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for grading exam questions, characterized in that, include: Obtain the written answers and standard answers for the questions to be graded; Obtain the text sequence features corresponding to the answer text and the standard answer, respectively; Construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graphs can represent the reasoning process of the corresponding text; Obtain the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer, and obtain the structural features of the reasoning graphs corresponding to the answer text and the standard answer respectively; Based on the text sequence features and reasoning graph structure features corresponding to the answer text, and the text sequence features and reasoning graph structure features corresponding to the standard answer, the score of the answer text is predicted; The step of predicting the score of the answer text based on the text sequence features and inference graph structure features corresponding to the answer text, and the text sequence features and inference graph structure features corresponding to the standard answer, includes: The reasoning graphs corresponding to the answer text and the standard answer are respectively divided into subgraphs from bottom to top to obtain the subgraph sequences corresponding to the answer text and the standard answer respectively; For each text in the answer text and the standard answer, the subgraph sequence features corresponding to the text are determined based on the subgraph sequence and reasoning graph structure features corresponding to the text. Based on the features of the subgraph sequence corresponding to the answer text, the subgraph with the highest feature similarity to each subgraph in the subgraph sequence corresponding to the answer text is retrieved from the pre-built reasoning subgraph library to obtain the similar subgraph sequence of the subgraph sequence corresponding to the answer text; Based on the subgraph sequence features of the similar subgraph sequence, and the text sequence features and subgraph sequence features corresponding to the answer text and the standard answer, respectively, the score of the answer text is predicted.

2. The question grading method according to claim 1, characterized in that, The construction of reasoning graphs corresponding to the answer text and the standard answer includes: For each text in the stated response text and the stated standard answer: Based on the text sequence features corresponding to the text, multi-type step units are extracted from the text to obtain the step unit sequence corresponding to the text. Predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text, so as to obtain the reasoning relationship information of the sequence of step units corresponding to the text; Based on the reasoning relationship information of the step unit sequence corresponding to the text, construct the reasoning graph corresponding to the text.

3. The question grading method according to claim 2, characterized in that, The step of predicting whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text includes: The step unit sequence corresponding to the text is taken as the text sequence, and the text sequence feature corresponding to the step unit sequence is obtained as the step unit sequence feature corresponding to the text. Based on the sequence features of the step units corresponding to the text, predict whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text.

4. The question grading method according to claim 3, characterized in that, The step of predicting whether there is a reasoning relationship between the step units in the sequence of step units corresponding to the text, based on the sequence features of the step units corresponding to the text, includes: For each step unit in the sequence of step units corresponding to this text: Based on the sequence features of the step units corresponding to the text, determine the relevance weight between the step unit and each other step unit; Based on the relevance weights of this step unit and each of the other step units, determine whether there is a reasoning relationship between this step unit and each of the other step units.

5. The method for grading questions according to claim 1, characterized in that, The reasoning graph includes several nodes and directed connecting lines between the nodes. Each node represents a step unit, and the two step units represented by two nodes connected by a directed connecting line have a reasoning relationship. The process of obtaining the structural features of the reasoning graph corresponding to the answer text and the structural features of the reasoning graph corresponding to the standard answer includes: For each text in the stated response text and the stated standard answer: Traverse the nodes in the reasoning graph corresponding to the text: Based on the first feature of the currently traversed node and the first or second feature of the adjacent nodes of the currently traversed node, determine the second feature of the currently traversed node. The first feature of a node is obtained by fusing the text feature of the step unit represented by the node with the type feature of the step unit represented by the node. The text feature of a step unit is obtained based on the text sequence feature corresponding to the text to which the step unit belongs. The structural features of the inference graph corresponding to the text are composed of the second features of each node in the inference graph.

6. The method for grading questions according to claim 1, characterized in that, The inference graph includes several nodes and directed connections between nodes, and the structural features of the inference graph include the features of each node in the corresponding inference graph. The step of determining the subgraph sequence features corresponding to the text based on the subgraph sequence and inference graph structure features includes: For each subgraph contained in the subgraph sequence corresponding to the text, the features of each node contained in the subgraph are obtained from the inference graph structure features corresponding to the text, and the features of the subgraph are determined based on the features of each node contained in the subgraph. The features of the subgraph sequence corresponding to the text are composed of the features of each subgraph contained in the subgraph sequence.

7. The method for grading questions according to claim 1, characterized in that, The step of predicting the score of the answer text based on the subgraph sequence features of the similar subgraph sequence, and the text sequence features and subgraph sequence features corresponding to the answer text and the standard answer, respectively, includes: The text sequence features corresponding to the answer text are fused with the text sequence features corresponding to the standard answer to obtain the first fused feature; The subgraph sequence features corresponding to the answer text are fused with the subgraph sequence features corresponding to the standard answer to obtain a second fused feature; The subgraph sequence features corresponding to the answer text are fused with the subgraph sequence features of the similar subgraph sequences to obtain the third fused feature; The score of the response text is predicted based on the first fusion feature, the second fusion feature, and the third fusion feature.

8. A problem-grading device, characterized in that, include: The module includes: question information acquisition module, text sequence feature acquisition module, reasoning graph acquisition module, reasoning graph structure feature acquisition module, and question grading result determination module. The question information acquisition module is used to acquire the answer text and standard answer of the question to be graded; The text sequence feature acquisition module is used to acquire the text sequence features corresponding to the answer text and the standard answer, respectively; The reasoning graph acquisition module is used to construct reasoning graphs corresponding to the answer text and the standard answer, respectively, wherein the reasoning graph can represent the reasoning process of the corresponding text; The reasoning graph structure feature acquisition module is used to acquire the structure features of the reasoning graph corresponding to the answer text and the structure features of the reasoning graph corresponding to the standard answer, so as to obtain the reasoning graph structure features corresponding to the answer text and the standard answer respectively; The question grading result determination module is used to predict the score of the answer text based on the text sequence features and reasoning graph structure features corresponding to the answer text, as well as the text sequence features and reasoning graph structure features corresponding to the standard answer. The question grading result determination module is specifically used to perform bottom-up sub-graph splitting on the reasoning graphs corresponding to the answer text and the standard answer, respectively, to obtain the sub-graph sequences corresponding to the answer text and the standard answer. For each text in the answer text and the standard answer, the sub-graph sequence features corresponding to the text are determined based on the sub-graph sequence and reasoning graph structural features. Based on the sub-graph sequence features corresponding to the answer text, the sub-graph with the highest feature similarity to each sub-graph in the sub-graph sequence corresponding to the answer text is retrieved from the pre-constructed reasoning sub-graph library to obtain the similar sub-graph sequence of the sub-graph sequence corresponding to the answer text. Based on the sub-graph sequence features of the similar sub-graph sequence, as well as the text sequence features and sub-graph sequence features corresponding to the answer text and the standard answer, the score of the answer text is predicted.

9. A test paper grading device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the question grading method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the question grading method as described in any one of claims 1 to 7.

Images