A method and system for automatic scoring of subjective questions based on knowledge graph

Abstract

The present invention relates to a method and system for automatic scoring of subjective questions based on knowledge graphs. Firstly, a triple extraction component is used to extract knowledge triples from student answers and standard answers to subjective questions respectively, and then the knowledge triples matched in the student answers are input into a trained knowledge graph embedding model, and the similarity score of each successfully matched triple is output. Finally, the score of each scoring point in the triple score sequence is determined according to the scoring rule to obtain the total score of the student answers to the subjective questions. The trained knowledge graph embedding model is trained according to the knowledge graph library of the student's professional field, rather than using a general public knowledge graph, so that the system is more accurate in professional detection and scoring, thereby realizing accurate automatic scoring of subjective questions.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method and system for automatically grading subjective questions based on a knowledge graph. Background Art

[0002] As labor costs continue to rise, traditional manual marking methods can no longer meet the needs of modern society, and society is increasingly demanding automatic marking. At present, automatic marking technology for objective questions is very mature and widely used. However, due to the complexity of Chinese and the difficulty of implementing natural language processing technology, there are still many shortcomings in marking subjective questions, such as not considering the impact of semantics, word order and other issues on scoring accuracy.

[0003] At present, the technical methods for automatic marking systems mainly include: rule-based and template-based methods and traditional machine learning-based methods. Rule-based and template-based methods rely on manually designed features and templates, and the trained models are not generalizable. In traditional machine learning-based methods, it is generally necessary to manually define features and use regression, classification, or a combination of the two to obtain a score. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for automatic scoring of subjective questions based on knowledge graphs, so as to achieve accurate automatic scoring of subjective questions.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for automatic scoring of subjective questions based on knowledge graph, the method comprising:

[0007] The knowledge triples are extracted from the student answers and the standard answers of the subjective questions respectively by using the triple extraction component, so as to obtain multiple triples of the student answers and multiple triples of the standard answers;

[0008] Matching multiple triples of student answers with multiple triples of standard answers to obtain successfully matched triples in student answers;

[0009] Input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library in the student's professional field;

[0010] The score of each scoring point in the triple scoring sequence is determined according to the scoring rules to obtain the total score of the student's answer to the subjective question.

[0011] Optionally, the method of extracting knowledge triples from the student answers and the standard answers of the subjective questions respectively by using the triple extraction component to obtain multiple triples of the student answers and multiple triples of the standard answers may also include:

[0012] Use open domain knowledge extraction tools to extract knowledge triple samples from historical data in the student's professional field to form a knowledge graph library; the historical data includes the examination tutorials, relevant knowledge points of the examination questions, and previous final examination papers of the student's professional courses;

[0013] The TransE model is trained using the knowledge graph library to obtain a trained knowledge graph embedding model.

[0014] Optionally, the use of an open domain knowledge extraction tool to extract knowledge triple samples from historical data in the student's professional field to form a knowledge graph library specifically includes:

[0015] Use open domain knowledge extraction tools to extract knowledge triple samples from historical data in the students' professional fields to obtain multiple triple samples;

[0016] Add multiple triple samples to the CSV file to form a knowledge graph library.

[0017] Optionally, the method further comprises: extracting knowledge triples from the student answers and the standard answers of the subjective questions respectively by using the triple extraction component to obtain multiple triples of the student answers and multiple triples of the standard answers, and then:

[0018] Add multiple triples of standard answers to the knowledge graph library.

[0019] Optionally, determining the score of each scoring point in the triple scoring sequence according to the scoring rule to obtain the total score of the student's answer to the subjective question specifically includes:

[0020] Match each score point with the triple of each element in the triple score sequence to determine multiple matching elements for each score point;

[0021] According to the score of each scoring point and the similarity scores of multiple matching elements of each scoring point, the formula is used Determine the score of the student's answer at each scoring point; where f i is the score of the student's answer at the i-th scoring point, S j is the similarity score of the jth matching element of the i-th scoring point, F i is the score of the i-th scoring point, and J is the total number of matching elements of the i-th scoring point;

[0022] Sum up the scores of students' answers at each scoring point to obtain the total score of students' answers to subjective questions.

[0023] A system for automatically grading subjective questions based on a knowledge graph, the system comprising:

[0024] A triple extraction module is used to extract knowledge triples from the student answers and standard answers of the subjective questions using the triple extraction component, so as to obtain multiple triples of the student answers and multiple triples of the standard answers;

[0025] A triple matching module, used for matching multiple triples of student answers with multiple triples of standard answers, and obtaining successfully matched triples in student answers;

[0026] A triple score sequence formation module is used to input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library in the student's professional field;

[0027] The total score acquisition module is used to determine the score of each scoring point in the triple scoring sequence according to the scoring rules, and obtain the total score of the student's answer to the subjective question.

[0028] Optionally, the system further includes:

[0029] The knowledge graph library construction module is used to extract knowledge triple samples from the historical data of the student's professional field using an open domain knowledge extraction tool to construct a knowledge graph library; the historical data includes the examination tutorials, relevant knowledge points of the examination questions, and previous final examination papers of the student's professional courses;

[0030] The training module is used to train the TransE model using the knowledge graph library to obtain a trained knowledge graph embedding model.

[0031] Optionally, the knowledge graph library constitutes a module, specifically including:

[0032] The triple sample acquisition submodule is used to extract knowledge triple samples from historical data in the professional field of the students using an open domain knowledge extraction tool to obtain multiple triple samples;

[0033] The knowledge graph library constitutes a submodule, which is used to add multiple triple samples to the CSV file to constitute the knowledge graph library.

[0034] Optionally, the system further includes:

[0035] The updating module is used to add multiple triples of standard answers to the knowledge graph library.

[0036] Optionally, the total score obtaining module specifically includes:

[0037] A matching element determination submodule, used for matching each score point with the triple of each element in the triple score sequence, and determining multiple matching elements for each score point;

[0038] The score determination submodule is used to determine the score of each score point and the similarity scores of multiple matching elements of each score point using the formula Determine the score of the student's answer at each scoring point; where f i is the score of the student's answer at the i-th scoring point, S j is the similarity score of the jth matching element of the i-th scoring point, F i is the score of the i-th scoring point, and J is the total number of matching elements of the i-th scoring point;

[0039] The total score acquisition submodule is used to sum the scores of the students' answers at each scoring point to obtain the total score of the students' answers to the subjective questions.

[0040] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0041] The present invention discloses a method and system for automatic scoring of subjective questions based on knowledge graph. Firstly, a triple extraction component is used to extract knowledge triples from student answers and standard answers to subjective questions respectively, and then the knowledge triples are matched. Then, the successfully matched triples in the student answers are input into a trained knowledge graph embedding model, and the similarity score of each successfully matched triple is output. Finally, the score of each scoring point in the triple score sequence is determined according to the scoring rule to obtain the total score of the student answers to the subjective questions. The trained knowledge graph embedding model is trained according to the knowledge graph library of the student's professional field, rather than using a general public knowledge graph, so that the system is more accurate in professional detection and scoring, thereby realizing accurate automatic scoring of subjective questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 A flowchart of the method for automatic scoring of subjective questions based on knowledge graph provided by the present invention;

[0044] Figure 2 A schematic diagram of the automatic scoring method for subjective questions based on knowledge graph provided by the present invention;

[0045] Figure 3 Schematic diagram of the knowledge graph embedding model training process provided by the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The purpose of the present invention is to provide a method and system for automatic scoring of subjective questions based on knowledge graphs, so as to achieve accurate automatic scoring of subjective questions.

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] The present invention provides a method for automatic scoring of subjective questions based on knowledge graph, such as Figure 1-2 As shown, the method includes:

[0050] Step 101, using a triple extraction component to extract knowledge triples from student answers and standard answers to subjective questions respectively, to obtain multiple triples of student answers and multiple triples of standard answers.

[0051] For a student answer and a standard answer that need to be graded, the triple extraction component is used to perform relationship extraction to obtain two sets of triplets containing the head entity, the relationship, and the tail entity.

[0052] Step 102, matching multiple triples of student answers with multiple triples of standard answers, and obtaining successfully matched triples in student answers.

[0053] Compare the triplets of the true answer and the student's answer, and filter out the triplets in the student's answer that are not sub-elements of the true answer, that is, match each triple in the student's answer with all the triplets in the standard answer. If there are successfully matched entity pairs and relations, they are retained, and the remaining triplets that fail to match are filtered.

[0054] Step 103, input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library of the student's professional field.

[0055] The filtered triplets are taken as input and the trained TransE model is used to score the triplets, generating a score for each triplet. This score and the corresponding triplet form a triplet score sequence.

[0056] Step 104, determine the score of each score point in the triple score sequence according to the scoring rule, and obtain the total score of the student's answer to the subjective question.

[0057] Specifically include:

[0058] Match each score point with the triple of each element in the triple score sequence to determine multiple matching elements for each score point;

[0059] According to the score of each scoring point and the similarity scores of multiple matching elements of each scoring point, the formula is used Determine the score of the student's answer at each scoring point; where f i is the score of the student's answer at the i-th scoring point, S j is the similarity score of the jth matching element of the i-th scoring point, F i is the score of the i-th scoring point, and J is the total number of matching elements of the i-th scoring point;

[0060] Sum up the scores of students' answers at each scoring point to obtain the total score of students' answers to subjective questions.

[0061] Before step 101, you need to obtain the trained knowledge graph embedding model, refer to Figure 3 , the acquisition process is:

[0062] Step 1: Use open domain knowledge extraction tools to extract knowledge triple samples from the historical data of the student’s professional field to form a knowledge graph library; historical data includes the examination tutorials of the student’s professional courses, knowledge points related to the examination questions, and previous final examination papers.

[0063] For different majors (taking logistics as an example here), exam tutorials, exam-related knowledge points, and historical data of previous final exams, we use the OpenIE (open domain knowledge extraction tool) component to extract knowledge triples. We take the text information of the historical data as input and add the extracted triples to a CSV file. Each line contains the format of head, tail, and relation, and the CSV file is named triples.csv, thus forming our own knowledge graph library.

[0064] Step 2: Use the knowledge graph library to train the TransE model and obtain the trained knowledge graph embedding model.

[0065] The TransE model (knowledge graph embedding model) is trained using the knowledge graph formed in step 1. The model generation component is based on the triples.csv file in step 1. The component then generates four files: entity2id.txt, relation2id.txt, train2id.txt, and type_constrain.txt.

[0066] entity2id.txt: This file contains all entities in the triples.csv file and the IDs corresponding to the entities;

[0067] relation2id.txt: This file contains all the relations in the triples.csv file, as well as the IDs corresponding to the relations;

[0068] train2id.txt: This file contains a list of all triples in triples.csv. Each triple in train2id.txt is represented by a set of head entity ID, relation ID, and tail entity ID, which match the entity ID in entity2id.txt and the relation ID in relation2id.txt;

[0069] type_constrain.txt: This file contains the type constraints for each relation. This means that for each entity, all head entities and all tail entities are specified.

[0070] The model generation component uses OpenKE, which takes the above three files as input and generates a TransE-based embedding model.

[0071] The present invention adopts the method of automatically constructing a knowledge graph based on logistics profession rather than adopting a general public knowledge graph, which makes the system more accurate in professional detection and scoring.

[0072] During the scoring process of subjective questions, multiple triplets of standard answers are added to the knowledge graph library, so that the constructed knowledge graph can continue to grow and improve itself.

[0073] Knowledge extraction systems can be divided into two categories: closed information extraction and open information extraction. The first method requires predefined templates for extracting information, usually for specific fields and situations. On the other hand, open information extraction is not used to find relations in a specific field. The present invention adopts OpenIE as an open information extraction component, which enables the system to extract unseen relational facts without a pre-specified vocabulary.

[0074] The advantage of TransE as a knowledge graph embedding model is that it has lower computational cost and good performance than other more complex embedding models.

[0075] The present invention constructs a continuously growing knowledge graph based on test points and historical data in different fields (such as logistics, economic management, etc.), converts its training into a knowledge embedding model, and uses the model to automatically score subjective questions in different professional fields.

[0076] The present invention also provides a system for automatically grading subjective questions based on a knowledge graph, the system comprising:

[0077] A triple extraction module is used to extract knowledge triples from the student answers and standard answers of the subjective questions using the triple extraction component, so as to obtain multiple triples of the student answers and multiple triples of the standard answers;

[0078] A triple matching module, used for matching multiple triples of student answers with multiple triples of standard answers, and obtaining successfully matched triples in student answers;

[0079] The triple score sequence formation module is used to input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library in the student's professional field;

[0080] The total score acquisition module is used to determine the score of each scoring point in the triple scoring sequence according to the scoring rules, and obtain the total score of the student's answer to the subjective question.

[0081] The system also includes:

[0082] The knowledge graph library construction module is used to extract knowledge triple samples from the historical data of the student's professional field using the open domain knowledge extraction tool to form a knowledge graph library; the historical data includes the examination tutorials of the student's professional courses, knowledge points related to the examination questions, and the final examination papers of previous years;

[0083] The training module is used to train the TransE model using the knowledge graph library to obtain a trained knowledge graph embedding model.

[0084] The knowledge graph library consists of the following modules:

[0085] The triple sample acquisition submodule is used to extract knowledge triple samples from historical data in the professional field of the students using an open domain knowledge extraction tool to obtain multiple triple samples;

[0086] The knowledge graph library constitutes a submodule, which is used to add multiple triple samples to the CSV file to constitute the knowledge graph library.

[0087] The system also includes:

[0088] The updating module is used to add multiple triples of standard answers to the knowledge graph library.

[0089] The total score acquisition module includes:

[0090] A matching element determination submodule, used for matching each score point with the triple of each element in the triple score sequence, and determining multiple matching elements for each score point;

[0091] The score determination submodule is used to determine the score of each score point and the similarity scores of multiple matching elements of each score point using the formula Determine the score of the student's answer at each scoring point; where f i is the score of the student's answer at the i-th scoring point, S j is the similarity score of the jth matching element of the i-th scoring point, F i is the score of the i-th scoring point, and J is the total number of matching elements of the i-th scoring point;

[0092] The total score acquisition submodule is used to sum the scores of the students' answers at each scoring point to obtain the total score of the students' answers to the subjective questions.

[0093] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0094] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for automatic scoring of subjective questions based on knowledge graph, characterized in that: The method comprises: The knowledge triples are extracted from the student answers and the standard answers of the subjective questions respectively by using the triple extraction component, so as to obtain multiple triples of the student answers and multiple triples of the standard answers; Matching multiple triples of student answers with multiple triples of standard answers to obtain successfully matched triples in student answers; Input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library in the student's professional field; Determine the score of each scoring point in the triple scoring sequence according to the scoring rules, and obtain the total score of the student's answer to the subjective question; The method of using the triple extraction component to extract knowledge triples from the student answers and the standard answers of the subjective questions respectively, and obtaining multiple triples of the student answers and multiple triples of the standard answers, also includes: Use open domain knowledge extraction tools to extract knowledge triple samples from historical data in the student's professional field to form a knowledge graph library; the historical data includes the examination tutorials, relevant knowledge points of the examination questions, and previous final examination papers of the student's professional courses; Using the knowledge graph library to train the TransE model to obtain a trained knowledge graph embedding model; Determining the score of each scoring point in the triple scoring sequence according to the scoring rule to obtain the total score of the student's answer to the subjective question specifically includes: Match each score point with the triple of each element in the triple score sequence to determine multiple matching elements for each score point; According to the score of each scoring point and the similarity scores of multiple matching elements of each scoring point, the formula is used , determine the score of the student's answer at each scoring point; where, For students to answer i The score of points, For the i Points j The similarity scores of the matching elements, For the i The score value of each scoring point is J For the i The total number of matching elements with score points; Sum up the scores of students' answers at each scoring point to obtain the total score of students' answers to subjective questions.

2. The method for automatic scoring of subjective questions based on knowledge graph according to claim 1 is characterized in that: The open domain knowledge extraction tool is used to extract knowledge triple samples from the historical data of the student's professional field to form a knowledge graph library, which specifically includes: Use open domain knowledge extraction tools to extract knowledge triple samples from historical data in the students' professional fields to obtain multiple triple samples; Add multiple triple samples to the CSV file to form a knowledge graph library.

3. The method for automatic scoring of subjective questions based on knowledge graph according to claim 1 is characterized in that: The method further comprises: extracting knowledge triples from the student answers and the standard answers of the subjective questions respectively by using the triple extraction component to obtain multiple triples of the student answers and multiple triples of the standard answers, and then: Add multiple triples of standard answers to the knowledge graph library.

4. A subjective question automatic scoring system based on knowledge graph, characterized in that: The system comprises: A triple extraction module is used to extract knowledge triples from the student answers and standard answers of the subjective questions using the triple extraction component, so as to obtain multiple triples of the student answers and multiple triples of the standard answers; A triple matching module, used for matching multiple triples of student answers with multiple triples of standard answers, and obtaining successfully matched triples in student answers; A triple score sequence formation module is used to input the successfully matched triples in the student's answer into the trained knowledge graph embedding model, output the similarity score of each successfully matched triple, and form a triple score sequence with each successfully matched triple and the corresponding similarity score; the trained knowledge graph embedding model is trained according to the knowledge graph library in the student's professional field; A total score acquisition module is used to determine the score of each score point in the triple score sequence according to the scoring rules, and obtain the total score of the student's answer to the subjective question; The system further comprises: The knowledge graph library construction module is used to extract knowledge triple samples from the historical data of the student's professional field using an open domain knowledge extraction tool to construct a knowledge graph library; the historical data includes the examination tutorials, relevant knowledge points of the examination questions, and previous final examination papers of the student's professional courses; A training module, used to train the TransE model using the knowledge graph library to obtain a trained knowledge graph embedding model; The total score obtaining module specifically includes: A matching element determination submodule, used for matching each score point with the triple of each element in the triple score sequence, and determining multiple matching elements for each score point; The score determination submodule is used to determine the score of each score point and the similarity scores of multiple matching elements of each score point using the formula , determine the score of the student's answer at each scoring point; where, For students to answer i The score of points, For the i Points j The similarity scores of the matching elements, For the i The score value of each scoring point is J For the i The total number of matching elements with score points; The total score acquisition submodule is used to sum the scores of the students' answers at each scoring point to obtain the total score of the students' answers to the subjective questions.

5. The automatic scoring system for subjective questions based on knowledge graph according to claim 4 is characterized in that: The knowledge graph library constitutes a module, specifically including: The triple sample acquisition submodule is used to extract knowledge triple samples from historical data in the professional field of the students using an open domain knowledge extraction tool to obtain multiple triple samples; The knowledge graph library constitutes a submodule, which is used to add multiple triple samples to the CSV file to constitute the knowledge graph library.

6. The automatic scoring system for subjective questions based on knowledge graph according to claim 4 is characterized in that: The system further comprises: The updating module is used to add multiple triples of standard answers to the knowledge graph library.

Images