Automatic Scoring Method, Device, Electronic Device and Storage Medium for Subjective Question Answers

Abstract

The present invention provides a method, device, electronic device and storage medium for automatically grading subjective question answers, belonging to the technical field of natural language processing. The method includes: obtaining the student answers, standard answers, questions and scoring point information of subjective questions; inputting the student answers, standard answers, questions and scoring point information of subjective questions into the automatic subjective question answer grading model to output the grading result of the student answers; the automatic subjective question answer grading model is constructed based on a first language processing model, a second language processing model and a third language processing model; the first language processing model is used to determine the initial grading of the student answers; the second language processing model is used to extract the first keywords of the student answers and the second keywords of the standard answers, and the third language processing model is used to determine the grading result of the student answers. The present invention can improve the grading efficiency of subjective question answers while also improving the analysis ability to comprehensively and accurately evaluate students' subjective question answers.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, electronic device and storage medium for automatically grading answers to subjective questions. Background Art

[0002] In recent years, with the rapid rise of online education, the way of examination has gradually changed from offline to online. In this development trend, online assessment has become the core link of online education, and automatic evaluation technology has attracted much attention. Among them, due to its unique openness, the automatic scoring of subjective questions is still highly challenging.

[0003] At present, the scoring of subjective questions is still mainly based on the traditional manual marking method. However, the traditional manual method has the disadvantages of heavy workload and low efficiency, which is especially evident in large-scale online education.

[0004] Therefore, how to better achieve automatic scoring of answers to subjective questions has become a technical problem that needs to be urgently solved in the industry. Summary of the invention

[0005] The present invention provides a method, device, electronic device and storage medium for automatically scoring answers to subjective questions, so as to better realize automatic scoring of answers to subjective questions.

[0006] The present invention provides a method for automatically grading answers to subjective questions, comprising:

[0007] Obtain student answers, standard answers, questions and scoring information for subjective questions;

[0008] Inputting the student answers, standard answers, questions and scoring point information of the subjective questions into the automatic scoring model for subjective questions, and outputting the scoring results of the student answers;

[0009] The automatic scoring model for answers to subjective questions is constructed based on a first language processing model, a second language processing model and a third language processing model; the first language processing model is used to determine an initial score for the student answer based on the student answer and the standard answer; the second language processing model is used to extract a first keyword for the student answer and a second keyword for the standard answer; and the third language processing model is used to determine a scoring result for the student answer based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question and the scoring point information.

[0010] According to a method for automatically scoring answers to subjective questions provided by the present invention, the first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer and a similarity matching layer; the step of obtaining the initial score of the student's answer includes:

[0011] Input the student answer and the standard answer into the sentence vector representation processing layer respectively, to obtain a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer;

[0012] Input the first sentence vector representation and the second sentence vector representation into the first similarity calculation layer, to obtain the similarity between paragraphs in the student answer and paragraphs in the standard answer output by the first similarity calculation layer;

[0013] Input the similarity between paragraphs in the student answer and paragraphs in the standard answer into the similarity matching layer, to obtain an initial score of the student answer output by the similarity matching layer.

[0014] According to a method for automatically grading subjective question answers provided by the present invention, the sentence vector representation processing layer includes a first BERT network layer and a first mean pooling layer connected in sequence, and a second BERT network layer and a second mean pooling layer connected in sequence; the first BERT network layer and the second BERT network layer share each other's network parameters; the step of inputting the student answer and the standard answer into the sentence vector representation processing layer respectively, to obtain a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer, includes:

[0015] Input the student answer into the first BERT network layer, to obtain a word vector representation of the student answer output by the first BERT network layer, and input the standard answer into the second BERT network layer, to obtain a word vector representation of the standard answer output by the second BERT network layer;

[0016] Input the word vector representation of the student answer into the first mean pooling layer, to obtain a first sentence vector representation of the student answer output by the first mean pooling layer, and input the word vector representation of the standard answer into the second mean pooling layer, to obtain a second sentence vector representation of the standard answer output by the second mean pooling layer.

[0017] According to a method for automatically grading subjective question answers provided by the present invention, the second language processing model includes a word embedding model, a third BERT network layer, a second similarity calculation layer, and a keyword extraction layer; the steps of obtaining a first keyword of the student answer and a second keyword of the standard answer include:

[0018] Input the student answer and the standard answer into the word embedding model respectively to obtain the first candidate word of the student answer and the second candidate word of the standard answer output by the word embedding model;

[0019] Input the student answer and the first candidate word into the third BERT network layer respectively to obtain the word vector representations of the student answer and the first candidate word output by the third BERT network layer, and input the standard answer and the second candidate word into the third BERT network layer respectively to obtain the word vector representations of the standard answer and the second candidate word output by the third BERT network layer;

[0020] Input the word vector representations of the student answer and the first candidate word into the second similarity calculation layer to obtain the similarity between the first candidate word and the student answer output by the second similarity calculation layer, and input the word vector representations of the standard answer and the second candidate word into the second similarity calculation layer to obtain the similarity between the second candidate word and the standard answer output by the second similarity calculation layer;

[0021] Input the similarity between the first candidate word and the student answer into the keyword extraction layer to obtain the first keyword of the student answer output by the keyword extraction layer, and input the similarity between the second candidate word and the standard answer into the keyword extraction layer to obtain the second keyword of the standard answer output by the keyword extraction layer.

[0022] According to a method for automatically grading subjective question answers provided by the present invention, the third language processing model is constructed based on a large language model; the steps for obtaining the grading result of the student answer include:

[0023] Generate the target prompt information of the large language model based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question and the scoring point information;

[0024] Input the initial score and the target prompt information into the large language model to obtain the grading result of the student answer output by the large language model; the grading result includes the target score, the explanatory information and reasoning information of the target score, and the comment information of the student answer.

[0025] According to a method for automatically grading subjective question answers provided by the present invention, before inputting the student answer, the standard answer, the question and the scoring point information of the subjective question into the subjective question answer automatic grading model to output the grading result of the student answer, the method further includes:

[0026] Obtain the student answers, standard answers, questions, answer scores, scoring points information, and score explanation information of the subjective question samples;

[0027] Input the student answers and standard answers of the subjective question samples into the first language processing model in the initial automatic scoring model for subjective question answers, obtain the similarity between the third sentence vector representation of the student answers of the subjective question samples and the fourth sentence vector representation of the standard answers of the subjective question samples, and determine the sentence similarity loss based on the similarity between the third sentence vector representation and the fourth sentence vector representation and the answer score;

[0028] Input the student answers and standard answers of the subjective question samples into the second language processing model in the initial automatic scoring model for subjective question answers, and obtain the keywords of the student answers of the subjective question samples and the keywords of the standard answers of the subjective question samples;

[0029] Train the third language processing model in the initial automatic scoring model for subjective question answers using an autoregressive method, and determine the autoregressive loss based on the student answers, standard answers, questions, answer scores, scoring points information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples;

[0030] Based on the sentence similarity loss and the autoregressive loss, perform parameter iteration on the initial automatic scoring model for subjective question answers to obtain a trained automatic scoring model for subjective question answers.

[0031] The present invention also provides an automatic scoring device for subjective question answers, including:

[0032] An acquisition module, configured to acquire the student answers, standard answers, questions, and scoring points information of subjective questions;

[0033] A scoring module, configured to input the student answers, standard answers, questions, and scoring points information of the subjective questions into the automatic scoring model for subjective question answers and output the scoring result of the student answers;

[0034] The subjective question answer automatic grading model is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine an initial grade of the student answer based on the student answer and the standard answer; the second language processing model is used to determine a first keyword of the student answer and a second keyword of the standard answer, and the third language processing model is used to determine a grading result of the student answer based on the student answer, the standard answer, the first keyword, the second keyword, the initial grade, the question, and the scoring point information.

[0035] According to a subjective question answer automatic grading device provided by the present invention, the first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer, and a similarity matching layer; the device is specifically configured to:

[0036] Input the student answer and the standard answer into the sentence vector representation processing layer respectively, and obtain a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer;

[0037] Input the first sentence vector representation and the second sentence vector representation into the first similarity calculation layer, and obtain the similarity between paragraphs in the student answer and paragraphs in the standard answer output by the first similarity calculation layer;

[0038] Input the similarity between paragraphs in the student answer and paragraphs in the standard answer into the similarity matching layer, and obtain an initial grade of the student answer output by the similarity matching layer.

[0039] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, it implements the subjective question answer automatic grading method as described in any one of the above.

[0040] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the subjective question answer automatic grading method as described in any one of the above.

[0041] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the subjective question answer automatic grading method as described in any one of the above.

[0042] The automatic scoring method, device, electronic device and storage medium for subjective question answers provided by the present invention construct an automatic scoring model for subjective question answers by using multiple types of language processing models in combination, and adopt a multi-level scoring mechanism. After obtaining the student answers, standard answers, questions and scoring points information of subjective questions, a type of language processing model is used to perform primary scoring on the student answers and standard answers of subjective questions to obtain the initial score of the student answers. Then, another type of language processing model is used to extract the keywords of the student answers and standard answers of subjective questions, and based on the student answers and their keywords, standard answers and their keywords, initial scores, subjective questions and preset scoring points information, another type of language processing model is used to verify the initial scores, and finally the scoring results of the student answers are output. It can improve the efficiency of scoring subjective question answers while also enhancing the analytical ability to comprehensively and accurately evaluate students' subjective question answers, and improve the rationality of automatic scoring of subjective question answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0044] Figure 1 is one of the flow diagrams of the automatic scoring method for subjective question answers provided by the present invention;

[0045] Figure 2 is the flow diagram of primary scoring in the automatic scoring method for subjective question answers provided by the present invention;

[0046] Figure 3 is the second flow diagram of the automatic scoring method for subjective question answers provided by the present invention;

[0047] Figure 4 is the structural diagram of the automatic scoring device for subjective question answers provided by the present invention;

[0048] Figure 5 is the physical structural diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] To make the objectives, technical solutions and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention fall within the protection scope of the present invention.

[0050] In the description of the invention, it should be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", and "linked" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] The following combines Figures 1 - 5 to describe the subjective question answer automatic scoring method, device, electronic device, and storage medium of the present invention.

[0052] Figure 1 is one of the flow diagrams of the subjective question answer automatic scoring method provided by the present invention. The execution subject of each step in this method can be the device for constructing the subjective question answer automatic scoring model. This device can be implemented by software and / or hardware and can be integrated in an electronic device. The electronic device can be a terminal device (such as a smart phone, a personal computer, etc.), a server (such as a local server or a cloud server, or a server cluster, etc.), a processor, or a chip, etc. As Figure 1 shown, this method includes:

[0053] Step 110, obtaining the student answer, standard answer, question, and scoring point information of the subjective question;

[0054] Step 120, inputting the student answer, standard answer, question, and scoring point information of the subjective question into the subjective question answer automatic scoring model, and outputting the scoring result of the student answer;

[0055] The subjective question answer automatic scoring model is constructed based on the first language processing model, the second language processing model, and the third language processing model; the first language processing model is used to determine the initial score of the student answer based on the student answer and the standard answer; the second language processing model is used to extract the first keywords of the student answer and the second keywords of the standard answer; the third language processing model is used to determine the scoring result of the student answer based on the student answer, the standard answer, the first keywords, the second keywords, the initial score, the question, and the scoring point information.

[0056] Specifically, the first language processing model described in the embodiments of the present invention is used to analyze the student answers and standard answers of subjective questions and give the initial scores of the student answers. It can be specifically constructed based on a single language representation model (Bidirectional Encoder Representations from Transformers, BERT) model, or based on two BERT models of a siamese network.

[0057] The first keyword described in the embodiments of the present invention refers to the keyword obtained by extracting the text information in the student answers of subjective questions.

[0058] The second keyword described in the embodiments of the present invention refers to the keyword obtained by extracting the text information in the standard answers of subjective questions.

[0059] The second language processing model described in the embodiments of the present invention is used to extract the first keyword of the student answers of subjective questions and the second keyword of the standard answers. It can be specifically constructed based on the BERT model and the similarity measurement algorithm. Among them, the similarity measurement algorithm adopted can include the Max Sum Distance (MSD) method, or the Maximal Marginal Relevance (MMR) method.

[0060] The third language processing model described in the embodiments of the present invention is used to perform natural language processing and analysis on the student answers, standard answers, questions and scoring point information of subjective questions, as well as the first keyword, second keyword and initial score, and output the scoring result of the student answers. It can be specifically constructed using a large language model.

[0061] The scoring result described in the embodiments of the present invention includes the target score, the explanatory information and reasoning information of the target score, and the comment information of the student answers. Among them, the target score represents the final score of the evaluation of the student answers.

[0062] In the embodiments of the present invention, in step 110, the student answers, standard answers, questions and scoring point information of subjective questions are obtained. Among them, the scoring point information can be pre-written at the front end by means of manual review according to the standard answers of subjective questions.

[0063] In an embodiment of the present invention, in step 120, the student answer, standard answer, question, and scoring point information of the subjective question obtained in step 110 are respectively input into the automatic subjective question answer scoring model. The first language processing model scores the input student answer and standard answer to determine the initial score of the student answer; the second language processing model extracts keywords from the input student answer and standard answer to obtain the keywords of the student answer of the subjective question and the keywords of the standard answer of the subjective question.

[0064] Further, in step 120, through the third language processing model, natural language processing is performed on the input student answer, standard answer, first keyword, second keyword, initial score, question, and scoring point information to deeply understand the meaning of various types of texts, and the initial score of the student answer is inspected and inferred. Finally, the scoring result of the student answer is output, including the target score of the student answer, the explanatory information and inference information of the target score, and the comment information of the student answer.

[0065] The automatic subjective question answer scoring method of the embodiment of the present invention jointly constructs an automatic subjective question answer scoring model by using multiple types of language processing models, and adopts a multi-level scoring mechanism. After obtaining the student answer, standard answer, question, and scoring point information of the subjective question, a type of language processing model is used to perform a primary score on the student answer and standard answer of the subjective question to obtain the initial score of the student answer. Furthermore, another type of language processing model is used to extract the keywords of the student answer of the subjective question and the keywords of the standard answer, and based on the student answer and its keywords, the standard answer and its keywords, the initial score, the subjective question, and the preset scoring point information, another type of language processing model is used to inspect the initial score. Finally, the scoring result of the student answer is output, which can improve the scoring efficiency of the subjective question answer while also improving the analysis ability to comprehensively and accurately evaluate the student's subjective question answer, and improving the rationality of the automatic subjective question answer scoring.

[0066] Based on the content of the above embodiment, as an alternative embodiment, the first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer, and a similarity matching layer; the steps for obtaining the initial score of the student answer include:

[0067] The student answer and the standard answer are respectively input into the sentence vector representation processing layer to obtain the first sentence vector representation of the student answer and the second sentence vector representation of the standard answer output by the sentence vector representation processing layer;

[0068] The first sentence vector representation and the second sentence vector representation are input into the first similarity calculation layer to obtain the similarity between paragraphs in the student answer and paragraphs in the standard answer output by the first similarity calculation layer;

[0069] The similarity between paragraphs in the student's answer and paragraphs in the standard answer is input into the similarity matching layer to obtain the initial score of the student's answer output by the similarity matching layer.

[0070] Specifically, the first sentence vector representation described in the embodiments of the present invention refers to the vector representation obtained by extracting paragraph-level features from the subjective question student's answer.

[0071] The second sentence vector representation described in the embodiments of the present invention refers to the vector representation obtained by extracting paragraph-level features from the subjective question standard answer.

[0072] The sentence vector representation processing layer described in the embodiments of the present invention is used to extract paragraph-level features from the student's answer and the standard answer of subjective questions. It can be specifically constructed based on a single BERT model or two BERT models of a siamese network.

[0073] The first similarity calculation layer described in the embodiments of the present invention is used to calculate the similarity between the first sentence vector representation and the second sentence vector representation to characterize the similarity between paragraphs in the student's answer and paragraphs in the standard answer. It can be specifically constructed by adopting the cosine similarity calculation method.

[0074] The similarity matching layer described in the embodiments of the present invention is used to match the paragraph positions of the student's answer and the standard answer according to the similarity between paragraphs in the student's answer and paragraphs in the standard answer, and determine the initial score of the student's answer according to the matching result. It can be specifically constructed by adopting the optimal matching algorithm, such as the Hungarian algorithm (Kuhn-Munkres algorithm).

[0075] In the embodiments of the present invention, the first language processing model can be constructed based on the sentence vector representation processing layer, the first similarity calculation layer, and the similarity matching layer.

[0076] In the embodiments of the present invention, the initial score of the subjective question student's answer can be obtained through the following specific implementation methods.

[0077] Specifically, in the embodiments of the present invention, the student's answer and the standard answer are respectively input into the sentence vector representation processing layer, and through the text feature processing of the student's answer and the standard answer by the sentence vector representation processing layer, the first sentence vector representation of the student's answer and the second sentence vector representation of the standard answer can be obtained.

[0078] Figure 2 It is a schematic flowchart of the primary scoring in the automatic scoring method for subjective question answers provided by the present invention, such as Figure 2As shown in the figure, the sentence vector representation processing layer 200 includes a first BERT network layer 210 and a first mean pooling layer 220 connected in sequence, and a second BERT network layer 230 and a second mean pooling layer 240 connected in sequence; the network parameters between the first BERT network layer 210 and the second BERT network layer 230 are shared; inputting the student answer and the standard answer into the sentence vector representation processing layer respectively to obtain the first sentence vector representation of the student answer and the second sentence vector representation of the standard answer output by the sentence vector representation processing layer 200, including:

[0079] Input the student answer into the first BERT network layer 210 to obtain the word vector representation of the student answer output by the first BERT network layer 210, and input the standard answer into the second BERT network layer 230 to obtain the word vector representation of the standard answer output by the second BERT network layer 230;

[0080] Input the word vector representation of the student answer into the first mean pooling layer 220 to obtain the first sentence vector representation of the student answer output by the first mean pooling layer 220, and input the word vector representation of the standard answer into the second mean pooling layer 240 to obtain the second sentence vector representation of the standard answer output by the second mean pooling layer 240.

[0081] Specifically, in the embodiment of the present invention, the sentence vector representation processing layer can be constructed based on two Sentence-BERT models with the same structure of a siamese network, and the network parameters are shared between the two Sentence-BERT models. Both of the two Sentence-BERT models can be trained by a pre-trained BERT model. It can be understood that the first BERT network layer is one of the two Sentence-BERT models, and the second BERT network layer is the other Sentence-BERT model of the two Sentence-BERT models.

[0082] In the embodiment of the present invention, when inputting the text of the student answer and the standard answer of a subjective question, two special identifiers [CLS] and [SEP] can be added to the head and tail respectively. Among them, the [CLS] identifier is used to identify the beginning of the sentence, and the [SEP] identifier is used to identify the end of the sentence. Then, use a tokenizer to split the input text into several independent characters and input them into the corresponding BERT network layer.

[0083] Further, for the input student answers and standard answers, the corresponding Sentence-BERT model performs text encoding respectively to obtain the word vector representations of the student answers and the standard answers. Subsequently, through the corresponding mean pooling layer, pooling operations are performed on the obtained word vector representations of the student answers and the standard answers, and the entire semantic vector representations of the text sentences of the student answers and the standard answers can be obtained, that is, the first sentence vector representation of the student answers and the second sentence vector representation of the standard answers are obtained.

[0084] The method of the embodiment of the present invention can better capture the semantic information and structural information of sentences in the student answers and standard answers of subjective questions by adopting a Sentence-BERT model based on a siamese network to construct a sentence vector representation processing layer, improving the accuracy and robustness of subjective question answer evaluation.

[0085] Further, in the embodiment of the present invention, the first sentence vector representation and the second sentence vector representation are input into the first similarity calculation layer, and the similarity between each pair of paragraphs in the student answers and the paragraphs in the standard answers is obtained through the similarity calculation algorithm in the first similarity calculation layer.

[0086] The above similarity calculation algorithm uses cosine similarity, and the calculation method is as follows:

[0087]

[0088] In the formula, CosineSimilarity(A,B) represents the similarity between each pair of paragraphs in the student answers and the paragraphs in the standard answers; A i and B i respectively represent the vector representations of the i-th group of corresponding words in the paragraphs of the student answers and the paragraphs of the standard answers.

[0089] However, in actual student answers, there are often situations where the paragraph order is not completely consistent with the paragraph order in the standard answers due to carelessness or other reasons. Therefore, it is also necessary to match the paragraphs of the standard answers and the student answers.

[0090] Figure 3 is the second flowchart of the automatic subjective question answer scoring method provided by the present invention, as Figure 3As shown, in the embodiments of the present invention, through the processing of the BERT model based on the Siamese network and the mean pooling layer, the first sentence vector representation of the student answer and the second sentence vector representation of the standard answer are obtained. After calculating the cosine similarity between each pair of paragraphs in the student answer and the standard answer through the first similarity calculation layer, the respective similarities can be input into the similarity matching layer. The similarity matching layer matches the paragraphs of the standard answer and the student answer to obtain the best matching score, that is, the initial score of the student answer is obtained.

[0091] Among them, in the embodiments of the present invention, the combination of the BERT model based on the Siamese network, the mean pooling layer, and the first similarity calculation layer can be described as a paragraph-level similarity calculation module, and the similarity matching layer can be described as a paragraph-level similarity matching module.

[0092] More specifically, in the embodiments of the present invention, the similarity matching layer can be constructed by the Kuhn-Munkres algorithm. Among them, the Kuhn-Munkres algorithm is an algorithm for solving the optimal matching problem of a weighted bipartite graph, and it can be used for the matching between multiple paragraph similarities. After obtaining the similarity between each pair of paragraphs in the student answer and the standard answer, the Kuhn-Munkres algorithm is used to match the paragraphs. Full marks are given to the student answers with the same paragraph order, and a certain degree of penalty points are given to the student answers with disordered order.

[0093] Among them, the specific steps of the above Kuhn-Munkres algorithm are as follows:

[0094] Step S0: Create an m×n matrix, where each element represents the similarity between each pair of the m paragraphs of the standard answer and the n paragraphs of the student answer. Subsequently, extract the element with the largest similarity in the matrix, and generate a cost matrix by subtracting each element.

[0095] Step S1: For each row of the cost matrix, find the element with the smallest similarity in it, and subtract it from each element in that row, and then execute Step S2.

[0096] Step S2: Find a "0" in the matrix obtained in Step S1. If there is no starred "0" in that row or column, then mark this "0" as "0*". Repeat the above operation for each element in the matrix, and then execute Step S3.

[0097] Step S3: Cover each column containing "0*", and "0*" describes an optimal matching of a group of paragraphs. If all columns in the matrix are covered, go to DONE at this time, otherwise execute Step S4.

[0098] Step S4: Find an uncovered "0" in the matrix and mark it as "0'" (a "0" with a prime). If there is no "0*" in the row containing this "0'", go to Step S5. Otherwise, cover this row and uncover the column containing the "0*". Continue in this way until there are no uncovered "0"s. Save the smallest uncovered value and then perform Step S6.

[0099] Step S5: Construct a series of alternating "0'" and "0*". Let Z 0 represent the uncovered "0'" found in Step S4. If there is a "0*" in the column where Z 0 is located, let it be Z 1 . Let Z 2 represent the "0'" in the row where Z 1 is located. Continue doing this until no "0*" can be found in this sequence, ending with a "0'" (where there is no "0*" in its column). Remove the asterisk from each "0*" in the sequence, add an asterisk to each "0'" in the sequence and erase all primes, uncover each row in the covered matrix, and return to Step S3.

[0100] Step S6: Add the value saved in Step S4 to each element in each covered row and subtract it from each uncovered element. Do not change any asterisks, primes, or covering lines, and return to Step S4.

[0101] In this way, through the above steps, each column of the cost matrix contains a "0*". The corresponding position is the best match for the scores of the standard answer paragraph and the student answer paragraph. Furthermore, based on the matching results of the above standard answer paragraphs and the student answer paragraphs, the initial score of the student answer can be calculated.

[0102] The method of the embodiment of the present invention constructs the first language processing model by adopting a sentence vector representation processing layer, a first similarity calculation layer, and a similarity matching layer, enabling the model to pay more attention to the text features of each paragraph of the subjective question answer, and can effectively solve the analysis and evaluation of the paragraph similarity between the student answer and the standard answer, which helps to improve the accuracy and efficiency of the evaluation.

[0103] Continuing to refer to Figure 3 , in some embodiments, the second language processing model includes a word embedding model, a third BERT network layer, a second similarity calculation layer, and a keyword extraction layer; the steps for obtaining the first keywords of the student answer and the second keywords of the standard answer include:

[0104] Input the student answer and the standard answer into the word embedding model respectively to obtain the first candidate words of the student answer and the second candidate words of the standard answer output by the word embedding model;

[0105] The student answer and the first candidate word are respectively input into the third BERT network layer to obtain the word vector representations of the student answer and the first candidate word output by the third BERT network layer, and the standard answer and the second candidate word are respectively input into the third BERT network layer to obtain the word vector representations of the standard answer and the second candidate word output by the third BERT network layer;

[0106] The word vector representations of the student answer and the first candidate word are input into the second similarity calculation layer to obtain the similarity between the first candidate word and the student answer output by the second similarity calculation layer, and the word vector representations of the standard answer and the second candidate word are input into the second similarity calculation layer to obtain the similarity between the second candidate word and the standard answer output by the second similarity calculation layer;

[0107] The similarity between the first candidate word and the student answer is input into the keyword extraction layer to obtain the first keyword of the student answer output by the keyword extraction layer, and the similarity between the second candidate word and the standard answer is input into the keyword extraction layer to obtain the second keyword of the standard answer output by the keyword extraction layer.

[0108] Specifically, the first candidate word described in the embodiments of the present invention refers to a word obtained by randomly extracting phrase groups from the student answer through a word embedding model.

[0109] The second candidate word described in the embodiments of the present invention refers to a word obtained by randomly extracting phrase groups from the standard answer through a word embedding model.

[0110] In the embodiments of the present invention, the third BERT network layer is constructed based on a single BERT model (i.e., BERT Encoder). The second similarity calculation layer is used to calculate the cosine similarity between the student answer and its candidate word, and calculate the cosine similarity between the standard answer and its candidate word. The keyword extraction layer can be constructed by the MSD algorithm or the MMR algorithm.

[0111] In the embodiments of the present invention, in order to identify the key information in the student answer and the standard answer document, the Key-BERT model is used to find the sub-phrase in the document that is most similar to the document itself. First, a word embedding model is used to extract N-gram phrases to obtain the first candidate word of the student answer and the second candidate word of the standard answer. Then, the BERT model is used for word embedding to obtain the document-level word vector representation, from which the word vector representations of the student answer and the first candidate word, or the word vector representations of the standard answer and the second candidate word can be obtained.

[0112] Secondly, in this embodiment, the second similarity calculation layer uses cosine similarity to calculate the similarity between the first candidate word and the student's answer, or the similarity between the second candidate word and the standard answer. Through similarity calculation, the word most similar to the document can be found from the candidate words. Finally, the MSD algorithm or the MMR algorithm is used to increase the diversity of keywords or key phrases, so that the words that can best describe the entire document can be obtained, that is, the first keyword of the student's answer or the second keyword of the standard answer can be obtained.

[0113] Among them, in the MSD algorithm, the purpose is to extract some keywords from some candidate words so that their similarity to the document is the largest, while keeping the similarity between them the smallest to increase diversity. Specifically, assuming that Top_n keywords need to be selected, 2×Top_n candidate words most similar to the document are selected by using cosine similarity. Subsequently, all Top_n combinations are extracted from the 2×Top_n candidate words, and finally, the combination with the smallest similarity between candidate words is selected through cosine similarity to ensure the diversity of keywords.

[0114] In the MMR algorithm, the similarity between the key phrase and the document, as well as the similarity between the selected keywords and key phrases, also need to be considered. The purpose is to reduce the redundancy of the sorting results while ensuring the relevance of the results. The formula of the MMR method is as follows:

[0115]

[0116] Among them, R\S represents the set of unselected keywords, R represents the set of selected keywords, Q represents the document of the student's answer or the standard answer, K i ,K j respectively represent a certain candidate word in the document of the student's answer or the standard answer, and a certain selected keyword in the document of the student's answer or the standard answer, and λ represents the importance trade-off coefficient.

[0117] The method of the embodiment of the present invention constructs a second language processing model by using a word embedding model, a third BERT network layer, a second similarity calculation layer, and a keyword extraction layer, making the model pay more attention to the similarity between the words in the student's answer and the standard answer and the document itself, identifying the key information in the student's answer and the standard answer document, and can effectively extract the keywords in the student's answer and the standard answer, which is beneficial to improving the accuracy and effectiveness of the evaluation of the student's subjective question answers.

[0118] Continue to refer to Figure 3 , based on the content of the above embodiment, as an optional embodiment, the third language processing model is constructed based on a large language model; the steps for obtaining the scoring result of the student's answer include:

[0119] Generate the target prompt information for the large language model based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question, and the scoring point information;

[0120] Input the initial score and the target prompt information into the large language model to obtain the scoring result of the student answer output by the large language model.

[0121] Specifically, in the embodiment of the present invention, the third language processing model is constructed based on the large language model and can be described as an evaluation result analysis module based on the large language model. Among them, the large language model in the embodiment of the present invention can be trained through a pre-trained large language model. It specifically uses an autoregressive mechanism to encode and decode text. Its model structure is similar to the Transformer-Decoder model structure, and its attention mechanism enables the vector representation after the sentence to see the vector representation in front.

[0122] It should be noted that by pre-training a large amount of corpus data, the large language model can learn the rules and patterns in natural language and has achieved excellent performance in generative tasks, being able to better provide explanations and analysis processes for subjective question evaluation. The advantage of the large language model is that it can utilize the knowledge obtained during pre-training to achieve better performance on the target task without a large amount of labeled data.

[0123] More specifically, in the embodiment of the present invention, by making the large language model act as a teacher, the initial score of the student answer obtained above and the keywords of the student answer and the standard answer extracted are used for result verification, and the analysis process, explanation, and comments of the score are given. First, the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question, and the scoring point information are fused into the prompt of the large language model, that is, the target prompt information is generated.

[0124] Furthermore, in the embodiment of the present invention, the initial score and the obtained target prompt information are input into the large language model. According to the target prompt information, the large language model can verify the initial score of the student answer and finally output the scoring result of the student answer. Among them, if the judgment result is reasonable, the large language model is made to give the analysis process of the score, including the reasoning process of the score and the comments on the student answer; if the judgment result is unreasonable, the large language model will be made to give a new score and give the explanation or reason for the final result, and finally give the reasoning process of the score and the comment information on the student answer.

[0125] The method according to the embodiments of the present invention combines a large language model with a text paragraph semantic matching technology. By leveraging the powerful capabilities of the large language model and deeply understanding and simulating the structure and context of natural language, it brings more comprehensive and accurate analysis capabilities to the evaluation of subjective questions, providing high-availability guarantees for the evaluation of subjective questions. At the same time, the model output results provide more interpretability for graders, enabling them to more deeply understand students' answers, which helps to improve the rationality of evaluation. It also provides comments for students, promoting students' in-depth thinking about their own answers, and thus contributing to stimulating students' learning interest and motivation for progress.

[0126] Based on the content of the above embodiments, as an optional embodiment, before inputting the student answers, standard answers, questions, and scoring point information of subjective questions into the automatic subjective question answer scoring model and outputting the scoring results of the student answers, the method further includes:

[0127] Obtain the student answers, standard answers, questions, answer scores, scoring point information, and score explanation information of subjective question samples;

[0128] Input the student answers and standard answers of the subjective question samples into the first language processing model in the initial automatic subjective question answer scoring model to obtain the similarity between the third sentence vector representation of the student answers of the subjective question samples and the fourth sentence vector representation of the standard answers of the subjective question samples, and determine the sentence similarity loss based on the similarity between the third sentence vector representation and the fourth sentence vector representation and the answer score;

[0129] Input the student answers and standard answers of the subjective question samples into the second language processing model in the initial automatic subjective question answer scoring model to obtain the keywords of the student answers of the subjective question samples and the keywords of the standard answers of the subjective question samples;

[0130] Train the third language processing model in the initial automatic subjective question answer scoring model using an autoregressive method, and determine the autoregressive loss based on the student answers, standard answers, questions, answer scores, scoring point information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples;

[0131] Based on the sentence similarity loss and the autoregressive loss, perform parameter iteration on the initial automatic subjective question answer scoring model to obtain a trained automatic subjective question answer scoring model.

[0132] Specifically, in the embodiments of the present invention, before inputting the student answers, standard answers, questions, and scoring point information of subjective questions into the automatic subjective question answer scoring model and outputting the scoring results of the student answers, it is also necessary to train the automatic subjective question answer scoring model.

[0133] Specifically, in the embodiments of the present invention, first, subjective question samples are obtained, along with their corresponding student answers, standard answers, questions, answer scores, as well as manually written scoring points information and score explanation information to form a training data set. The training data set can be divided into a training set and a test set. The training set is used for model training, and the test set is used for model testing.

[0134] In the embodiments of the present invention, first, the student answers, questions, standard answers, answer scores, question scoring points, and score explanations corresponding to the subjective question samples are obtained to form the original training data. Then, the above original training data is processed manually to obtain the sample student answers and their corresponding subjective question titles, standard answers, answer scores, question scoring points, and score explanations. Among them, the manual processing steps include at least one of constructing a prompt template and constructing a question-and-answer pair. Specifically, it can include integrating the key information in the original student answers and their corresponding subjective question titles, standard answers, answer scores, question scoring points, and score explanations into the prompt template, or designing them in the form of a question-and-answer pair.

[0135] Further, in the embodiments of the present invention, the student answers and standard answers of the subjective question samples are input into the first language processing model in the initial subjective question answer automatic scoring model for initial scoring processing to obtain the similarity between the third sentence vector representation of the student answers of the subjective question samples and the fourth sentence vector representation of the standard answers of the subjective question samples. And using a preset loss function, based on the similarity between the third sentence vector representation and the fourth sentence vector representation and the answer score, the sentence similarity loss is calculated.

[0136] Among them, the sentence similarity loss can specifically adopt the mean squared error loss L MSE . In some embodiments, the calculation formula of the sentence similarity loss can be expressed as follows:

[0137]

[0138] In the formula, n is the number of samples, y i represents the answer score corresponding to the i-th sample, represents the predicted score given by the model for the i-th sample.

[0139] Further, in the embodiments of the present invention, the student answers and standard answers of the subjective question samples are input into the second language processing model in the initial subjective question answer automatic scoring model again for keyword extraction, and the keywords of the student answers of the subjective question samples and the keywords of the standard answers of the subjective question samples can be obtained.

[0140] In an embodiment of the present invention, an autoregressive method is used to train the third language processing model in the initial subjective question answer automatic scoring model, and a preset loss function is used to calculate the autoregressive loss based on the student answers, standard answers, questions, answer scores, scoring points information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples of the subjective question samples.

[0141] Among them, for each word, the autoregressive loss can specifically adopt the cross-entropy loss L reg , in some embodiments, the autoregressive loss calculation formula can be expressed as follows:

[0142]

[0143] In the formula, y j represents the probability of the j-th word in the sentences of the student answers, standard answers, questions, answer scores, scoring points information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples of the subjective question samples.

[0144] On this basis, based on the sentence similarity loss and the autoregressive loss, parameter iteration can be performed on the initial subjective question answer automatic scoring model to obtain a trained subjective question answer automatic scoring model.

[0145] In some embodiments, the total loss of the model can be expressed as:

[0146] L all =αL reg +βL MSE ;

[0147] Among them, α and β represent adjustable hyperparameters, L MSE represents the sentence similarity loss, and L reg represents the autoregressive loss. By minimizing the total loss function, parameter iteration is performed on the initial subjective question answer automatic scoring model. The subjective question answer automatic scoring model obtained thereby can be used to implement the automatic evaluation and analysis of multi-paragraph articles.

[0148] The method of the embodiment of the present invention trains the subjective question answer automatic scoring model by combining the sentence similarity loss and the autoregressive loss, and controls the model loss value within a preset range, thereby facilitating the improvement of the accuracy of the subjective question answer automatic scoring model for automatic evaluation of subjective questions.

[0149] Next, a subjective question answer automatic scoring device provided by the present invention will be described. The subjective question answer automatic scoring device described below can be correspondingly referred to the subjective question answer automatic scoring method described above.

[0150] Figure 4 It is a schematic structural diagram of the subjective question answer automatic grading device provided by the present invention. As Figure 4 shown, it includes an acquisition module 410 and a grading module 420 connected in sequence.

[0151] Among them, the acquisition module 410 is used to acquire the student answers, standard answers, questions, and scoring point information of subjective questions;

[0152] The grading module 420 is used to input the student answers, standard answers, questions, and scoring point information of subjective questions into the subjective question answer automatic grading model, and output the grading result of the student answers;

[0153] The subjective question answer automatic grading model is constructed based on the first language processing model, the second language processing model, and the third language processing model; the first language processing model is used to determine the initial grading of the student answers based on the student answers and the standard answers; the second language processing model is used to determine the first keywords of the student answers and the second keywords of the standard answers, and the third language processing model is used to determine the grading result of the student answers based on the student answers, the standard answers, the first keywords, the second keywords, the initial grading, the questions, and the scoring point information.

[0154] The subjective question answer automatic grading device described in this embodiment can be used to execute the above-mentioned subjective question answer automatic grading method embodiment, and its principle and technical effect are similar, so it will not be elaborated here.

[0155] The subjective question answer automatic grading device of the embodiment of the present invention constructs a subjective question answer automatic grading model by jointly using multiple types of language processing models, and adopts a multi-level grading mechanism. After acquiring the student answers, standard answers, questions, and scoring point information of subjective questions, it uses one type of language processing model to perform primary grading on the student answers and standard answers of subjective questions to obtain the initial grading of the student answers. Furthermore, it uses another type of language processing model to extract the keywords of the subjective question student answers and the keywords of the standard answers, and based on the student answers and their keywords, the standard answers and their keywords, the initial grading, the subjective question, and the preset scoring point information, it uses another type of language processing model to verify the initial grading, and finally outputs the grading result of the student answers, which can improve the grading efficiency of subjective question answers while also improving the analysis ability to comprehensively and accurately evaluate the student's subjective question answers, and improve the rationality of the subjective question answer automatic grading.

[0156] Optionally, the first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer, and a similarity matching layer; specifically, the device is used for:

[0157] The student answer and the standard answer are respectively input into the sentence vector representation processing layer, and a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer are obtained;

[0158] The first sentence vector representation and the second sentence vector representation are input into the first similarity calculation layer, and the similarity between each pair of paragraphs in the student answer and the paragraphs in the standard answer output by the first similarity calculation layer is obtained;

[0159] The similarity between each pair of paragraphs in the student answer and the paragraphs in the standard answer is input into the similarity matching layer, and an initial score of the student answer output by the similarity matching layer is obtained.

[0160] Figure 5 It is a schematic physical structure diagram of the electronic device provided by the present invention. As Figure 5 shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 complete mutual communication through the communication bus 540. The processor 510 can call the logical instructions in the memory 530 to execute the subjective question answer automatic scoring method provided by each of the above methods. The method includes: obtaining the student answer, the standard answer, the question, and the scoring point information of the subjective question; inputting the student answer, the standard answer, the question, and the scoring point information of the subjective question into the subjective question answer automatic scoring model, and outputting the scoring result of the student answer; the subjective question answer automatic scoring model is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine the initial score of the student answer based on the student answer and the standard answer; the second language processing model is used to extract a first keyword of the student answer and a second keyword of the standard answer, and the third language processing model is used to determine the scoring result of the student answer based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question, and the scoring point information.

[0161] In addition, when the logical instructions in the above-mentioned memory 530 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0162] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the subjective question answer automatic scoring method provided by the above-mentioned various methods. The method includes: obtaining the student answer, standard answer, question, and scoring point information of the subjective question; inputting the student answer, standard answer, question, and scoring point information of the subjective question into the subjective question answer automatic scoring model, and outputting the scoring result of the student answer; the subjective question answer automatic scoring model is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine the initial score of the student answer based on the student answer and the standard answer; the second language processing model is used to extract the first keywords of the student answer and the second keywords of the standard answer, and the third language processing model is used to determine the scoring result of the student answer based on the student answer, the standard answer, the first keywords, the second keywords, the initial score, the question, and the scoring point information.

[0163] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the subjective question answer automatic scoring method provided by the above-mentioned various methods. The method includes: obtaining the student answer, standard answer, question, and scoring point information of the subjective question; inputting the student answer, standard answer, question, and scoring point information of the subjective question into the subjective question answer automatic scoring model, and outputting the scoring result of the student answer; the subjective question answer automatic scoring model is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine the initial score of the student answer based on the student answer and the standard answer; the second language processing model is used to extract the first keywords of the student answer and the second keywords of the standard answer, and the third language processing model is used to determine the scoring result of the student answer based on the student answer, the standard answer, the first keywords, the second keywords, the initial score, the question, and the scoring point information.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative labor.

[0165] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically scoring answers to subjective questions, characterized in that: include: Obtain student answers, standard answers, questions and scoring information for subjective questions; Inputting the student answers, standard answers, questions and scoring point information of the subjective questions into the automatic scoring model for subjective questions, and outputting the scoring results of the student answers; The automatic scoring model for answers to subjective questions is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine an initial score for the student answer based on the student answer and the standard answer; the second language processing model is used to extract a first keyword for the student answer and a second keyword for the standard answer; the third language processing model is used to determine a scoring result for the student answer based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question, and the scoring point information; Before inputting the student answer, standard answer, question and scoring point information of the subjective question into the automatic scoring model for subjective question answers and outputting the scoring result of the student answer, the method further includes: Obtain student answers, standard answers, questions, answer scores, scoring point information and score explanation information of subjective question samples; Inputting the student answer and the standard answer of the subjective question sample into the first language processing model in the initial subjective question answer automatic scoring model, obtaining the similarity between the third sentence vector representation of the student answer of the subjective question sample and the fourth sentence vector representation of the standard answer of the subjective question sample, and determining the sentence similarity loss based on the similarity between the third sentence vector representation and the fourth sentence vector representation and the answer score; Inputting the student answers and standard answers of the subjective question samples into the second language processing model in the initial subjective question answer automatic scoring model to obtain keywords of the student answers of the subjective question samples and keywords of the standard answers of the subjective question samples; The third language processing model in the automatic scoring model for the initial subjective question answers is trained using an autoregressive method, and the autoregressive loss is determined based on the student answers, standard answers, questions, answer scores, scoring point information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples; Based on the sentence similarity loss and the autoregressive loss, the parameters of the initial automatic scoring model for subjective questions are iterated to obtain a trained automatic scoring model for subjective questions.

2. The method for automatically scoring answers to subjective questions according to claim 1, characterized in that: The first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer and a similarity matching layer; the step of obtaining the initial score of the student's answer includes: Inputting the student answer and the standard answer into the sentence vector representation processing layer respectively, obtaining a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer; Inputting the first sentence vector representation and the second sentence vector representation into the first similarity calculation layer, and obtaining the similarity between the paragraphs in the student answer and the paragraphs in the standard answer output by the first similarity calculation layer; The similarities between the paragraphs in the student answer and the paragraphs in the standard answer are input into the similarity matching layer to obtain the initial score of the student answer output by the similarity matching layer.

3. The method for automatically grading answers to subjective questions according to claim 2, characterized in that: The sentence vector representation processing layer includes a first BERT network layer and a first mean pooling layer connected in sequence, and a second BERT network layer and a second mean pooling layer connected in sequence; the first BERT network layer and the second BERT network layer share each other's network parameters; the student answer and the standard answer are respectively input into the sentence vector representation processing layer to obtain a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer, including: Input the student answer to the first BERT network layer to obtain the word vector representation of the student answer output by the first BERT network layer, and input the standard answer to the second BERT network layer to obtain the word vector representation of the standard answer output by the second BERT network layer; The word vector representation of the student's answer is input into the first mean pooling layer to obtain a first sentence vector representation of the student's answer output by the first mean pooling layer, and the word vector representation of the standard answer is input into the second mean pooling layer to obtain a second sentence vector representation of the standard answer output by the second mean pooling layer.

4. The method for automatically grading answers to subjective questions according to claim 1, characterized in that: The second language processing model includes a word embedding model, a third BERT network layer, a second similarity calculation layer and a keyword extraction layer; the steps of obtaining the first keyword of the student answer and the second keyword of the standard answer include: Inputting the student answer and the standard answer into the word embedding model respectively, and obtaining a first candidate word for the student answer and a second candidate word for the standard answer output by the word embedding model; Input the student answer and the first candidate word into the third BERT network layer respectively, and obtain the word vector representation of the student answer and the word vector representation of the first candidate word output by the third BERT network layer, and input the standard answer and the second candidate word into the third BERT network layer respectively, and obtain the word vector representation of the standard answer and the word vector representation of the second candidate word output by the third BERT network layer; Inputting the word vector representation of the student's answer and the word vector representation of the first candidate word into the second similarity calculation layer to obtain the similarity between the first candidate word and the student's answer output by the second similarity calculation layer, and inputting the word vector representation of the standard answer and the word vector representation of the second candidate word into the second similarity calculation layer to obtain the similarity between the second candidate word and the standard answer output by the second similarity calculation layer; The similarity between the first candidate word and the student's answer is input into the keyword extraction layer to obtain the first keyword of the student's answer output by the keyword extraction layer, and the similarity between the second candidate word and the standard answer is input into the keyword extraction layer to obtain the second keyword of the standard answer output by the keyword extraction layer.

5. The method for automatically grading answers to subjective questions according to claim 1, characterized in that: The third language processing model is constructed based on the large language model; the steps of obtaining the scoring result of the student's answer include: Generate target prompt information of the large language model based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question and the scoring point information; The initial score and the target prompt information are input into the large language model to obtain a score result of the student answer output by the large language model; the score result includes a target score, explanation information and reasoning information of the target score, and comment information of the student answer.

6. An automatic scoring device for subjective question answers, characterized in that: include: The acquisition module is used to obtain student answers, standard answers, questions and scoring point information for subjective questions; A scoring module, which is used to input the student answers, standard answers, questions and scoring point information of the subjective questions into the automatic scoring model of the subjective questions, and output the scoring results of the student answers; The automatic scoring model for answers to subjective questions is constructed based on a first language processing model, a second language processing model, and a third language processing model; the first language processing model is used to determine an initial score for the student answer based on the student answer and the standard answer; the second language processing model is used to determine a first keyword for the student answer and a second keyword for the standard answer; the third language processing model is used to determine a scoring result for the student answer based on the student answer, the standard answer, the first keyword, the second keyword, the initial score, the question, and the scoring point information; The automatic scoring device for subjective question answers is also used to obtain student answers, standard answers, questions, answer scores, scoring point information and score explanation information of subjective question samples; Inputting the student answer and the standard answer of the subjective question sample into the first language processing model in the initial subjective question answer automatic scoring model, obtaining the similarity between the third sentence vector representation of the student answer of the subjective question sample and the fourth sentence vector representation of the standard answer of the subjective question sample, and determining the sentence similarity loss based on the similarity between the third sentence vector representation and the fourth sentence vector representation and the answer score; Inputting the student answers and standard answers of the subjective question samples into the second language processing model in the initial subjective question answer automatic scoring model to obtain keywords of the student answers of the subjective question samples and keywords of the standard answers of the subjective question samples; The third language processing model in the automatic scoring model for the initial subjective question answers is trained using an autoregressive method, and the autoregressive loss is determined based on the student answers, standard answers, questions, answer scores, scoring point information, score explanation information, keywords of the student answers of the subjective question samples, and keywords of the standard answers of the subjective question samples; Based on the sentence similarity loss and the autoregressive loss, the parameters of the initial automatic scoring model for subjective questions are iterated to obtain a trained automatic scoring model for subjective questions.

7. The automatic scoring device for subjective questions according to claim 6, characterized in that: The first language processing model includes a sentence vector representation processing layer, a first similarity calculation layer and a similarity matching layer; the device is specifically used for: Inputting the student answer and the standard answer into the sentence vector representation processing layer respectively, obtaining a first sentence vector representation of the student answer and a second sentence vector representation of the standard answer output by the sentence vector representation processing layer; Inputting the first sentence vector representation and the second sentence vector representation into the first similarity calculation layer, and obtaining the similarity between the paragraphs in the student answer and the paragraphs in the standard answer output by the first similarity calculation layer; The similarities between the paragraphs in the student answer and the paragraphs in the standard answer are input into the similarity matching layer to obtain the initial score of the student answer output by the similarity matching layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for automatically grading answers to subjective questions as described in any one of claims 1 to 5 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for automatically scoring answers to subjective questions as described in any one of claims 1 to 5 is implemented.

Images