Multi-question-type interaction self-editing system and method

Abstract

The invention provides a self-editing system and method for multi-question-type interaction, and belongs to the field of online question banks.The method comprises the steps that S1, subjects in a question bank are subjected to subject classification, preprocessed through a natural language processing model under subjects and then sent to a machine learning model, and new converted questions are obtained; s2, evaluating the validity of the new question, including sending the new question to an automatic evaluation model and then sending the new question to manual auditing; s3, evaluating the quality and difficulty of the new questions, and performing difficulty grading marking on the effective new questions; s4, counting the quality of the new questions and the difficulty after marking, and optimizing a machine learning model. According to the method, a clear and definite mapping rule between the input questions and the target question types is constructed, and more accurate new questions and richer new question types are obtained.

Description

Technical Field

[0001] This specification relates to the field of computer question banks, and particularly to a self-editing system and method for multi-type question interaction. Background Art

[0002] Due to the limited number of question banks, it is necessary to regularly replace the original questions to update the exam content. The number of courses to be updated is large, and there are also many questions. If the questions are replaced manually, there are problems such as high cost and poor efficiency. Automatically converting questions through existing pre-trained models or large language models will also have problems such as questions violating objective facts, misinterpreting the meaning of questions, and the answers not matching the options.

[0003] Chinese Patent CN112216168A, an intelligent question type conversion system and method based on a multiple-choice question editor, discloses an intelligent question type conversion system and method based on a multiple-choice question editor. The system includes: an input module for obtaining the required questions in an automated manner; a textification module for textifying the questions; an analysis module for analyzing the textified questions to obtain the question stem and the correct answer; a wrong answer recommendation module for generating multiple wrong answers for the user to choose according to the text characteristics of the question stem and the correct answer; an integrated editing module for transmitting the analyzed content and the corresponding generated wrong answers to the multiple-choice question editor; and a multiple-choice question editor for automatically generating multiple-choice questions according to the input content and allowing the user to further edit.

[0004] The above technology aims to convert the original multiple-choice questions through system modular work to obtain updated multiple-choice questions. Although the efficiency is high, the question types after conversion are too single.

[0005] Therefore, there is a need to provide a self-editing system and method for multi-type question interaction to achieve automatic editing and conversion of multiple question types. Summary of the Invention

[0006] One embodiment of this specification provides a self-editing system and method for multi-type question interaction, which can train the original question bank through a large model to expand and obtain more types of new questions. While the question types and content are updated, a question feasibility verification unit and a difficulty rating unit are set up, which can be adaptively adjusted according to the needs of the test takers.

[0007] In some embodiments, the self-editing method for multi-type question interaction S1, classify the questions in the question bank by subject, and after preprocessing through a natural language processing model under the subject, send them to a machine learning model to obtain the converted new questions; S2, evaluate the validity of the new questions, including sending them to an automated evaluation model first and then to manual review; S3. Evaluate the quality and difficulty of the new questions, and mark the difficulty levels of the valid new questions. S4. Statistically analyze the quality of the new questions and the marked difficulty, and optimize the machine learning model.

[0008] Furthermore, the working principle of the natural language processing model includes successively performing sentence structure conversion, synonym replacement, and semantic analysis on the original questions. Sentence structure conversion includes generating a new problem description through text generation-related algorithms, and changing the question structure through dependency syntax analysis and constituent syntax analysis. Synonym replacement generates new questions through a synonym replacement engine. Semantic analysis includes identifying subject-specific nouns and analyzing the predicate-argument structure in the sentence through semantic role annotation, extracting the core components of the question, and reorganizing them.

[0009] Furthermore, the machine learning model includes a knowledge graph model and a transformation rule model. The knowledge graph model is used to associate professional nouns with question type templates. The knowledge graph model hierarchically associates the professional nouns of each subject, and the associated professional nouns can be replaced in the question type templates. The transformation rule model includes multiple question type templates. By defining the transformation rules of the question templates, variables are replaced to generate new question types.

[0010] Furthermore, the question type templates include sequence generation templates, direct replacement templates, formula recombination templates, and logical equivalence transformation templates. The sequence generation templates include generating multiple-choice questions and true / false questions from open text through the GPT system and the Transformer system. The direct replacement template is used to swap the answers and variables of the original question. The formula recombination template transforms the form of math questions through a symbolic calculation library and LaTeX parsing.

[0011] Furthermore, the validity evaluation includes an automated evaluation model. By sending the new questions to this model, if a predicate-argument structure exists in the sentence determined through semantic analysis, it is defined as a valid new question. Manual review optimizes the semantic analysis algorithm by randomly checking valid new questions.

[0012] A self-editing system with multi-question type interaction includes sending the questions in the question bank to the natural language processing unit for question type conversion, performing question type optimization through the machine learning unit, and then processing through the difficulty grading and marking unit, and optimizing the rules in the machine learning unit according to the difficulty grading results; The natural language processing unit includes text repetition rules, syntactic structure conversion rules, entity replacement and masking rules; The machine learning unit includes generative rules, knowledge graph rules, and logical reasoning rules; The evaluation and verification unit includes verifying the uniqueness of the question answers through the BLEU evaluation system and the Z3 solver; The difficulty grading and marking unit includes item response theory rules.

[0013] Furthermore, the generative rule generates new questions by replacing variables based on a predefined question type template, and converts the form of math questions based on a symbolic calculation library and LaTeX parsing.

[0014] Furthermore, the new questions are sent to different types of students for practice according to the difficulty level. When it is necessary to lower or raise the difficulty level of the newly generated questions, the question parameters are dynamically adjusted through the item response theory, as well as the number of questions converted by each rule in the machine learning unit. After sampling the questions under the new quantity, only the new question conversion under a single rule is performed for the same knowledge point.

[0015] The beneficial effects of the present invention are: 1. Using the large model to perform question type conversion according to the trained algorithm rules can effectively expand the quantity of the question bank by 10 - 20 times; 2. Adopting an automated script or tool, combined with the method of manual review, to ensure that the selected questions are representative and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] This specification will be further described in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where: Figure 1 is a schematic diagram of the working principle shown in some embodiments of this specification; Figure 2 is a schematic diagram of a virtual classroom shown in some embodiments of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To more clearly illustrate the technical solutions of the embodiments of this specification, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the drawings represent the same structure or operation.

[0018] It should be understood that the "system", "device", "unit" and / or "module" used herein is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0019] As shown in this specification and the claims, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the previous or subsequent operations do not necessarily need to be executed precisely in sequence. On the contrary, the steps can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0021] Embodiment 1: Please refer to Figure 1 , including, S1. Classify the questions in the question bank by subject, and after preprocessing through a natural language processing model under the subject, send them to a machine learning model to obtain new converted questions; S2. Evaluate the validity of the new questions, including first sending them to an automated evaluation model and then sending them to manual review; S3. Evaluate the quality and difficulty of the new questions, and mark the difficulty levels of the valid new questions; S4. Statistically analyze the quality and marked difficulty of the new questions, and optimize the machine learning model.

[0022] The working principle of the natural language processing model includes sequentially performing sentence structure conversion, synonym replacement, and semantic analysis on the original questions. Sentence structure conversion includes generating a new question description through text generation-related algorithms, changing the question structure through dependency syntax analysis and constituent syntax analysis. Synonym replacement generates new questions through a synonym replacement engine. Semantic analysis includes identifying subject-specific nouns and through semantic role annotation, analyzing the predicate-argument structure in the sentence, extracting the core components of the question, and reorganizing them.

[0023] The machine learning model includes a knowledge graph model and a conversion rule model. The knowledge graph model is used to associate professional nouns with question type templates. Among them, the knowledge graph model performs multi-level association division on the professional nouns of each subject. The associated professional nouns can be replaced in the question type templates. The conversion rule model includes multiple question type templates. By defining the conversion rules of the question templates, variables are replaced to generate new question types.

[0024] The question type templates include sequence generation templates, direct replacement templates, formula recombination templates, and logical equivalence conversion templates. The sequence generation templates include generating multiple-choice questions and true / false questions from open text through GPT systems and Transformer systems. The direct replacement templates are used to swap the answers and variables of the original questions. The formula recombination templates convert the forms of math problems through symbolic calculation libraries and LaTeX parsing. The effectiveness evaluation includes an automated evaluation model. By sending new questions to this model, if a predicate-argument structure exists in the sentence determined through semantic analysis, it is defined as a valid new question. Manual review optimizes the semantic analysis algorithm by randomly checking valid new questions.

[0025] Find the most suitable large model and classify it according to the attributes of the courses, that is, the hierarchical structure of first-level discipline - second-level discipline - specialty. Use each mainstream large model to answer the questions in the existing question bank. The number of questions for each subject specialty should be no less than 2000. It is mainly evaluated from five aspects: concept, logic, reasoning, application, and practice to determine the most suitable large model under different subject specialty courses. For example, for the advanced mathematics specialty course under the mathematics discipline, select no less than 2000 questions, answer them using multiple large models respectively, and then evaluate and compare from the above five dimensions.

[0026] The detailed rules are as follows: Convert multiple-choice questions to true / false questions (taking the case where the answer to the question is four): If the answer to the multiple-choice question is two: When converting to a true / false question, according to the question stem, fill in the two correct answers or two wrong answers to form a true / false question.

[0027] If the answer to the multiple-choice question is three: When converting to a true / false question, according to the question stem, fill in the correct answers to form a true / false question.

[0028] If the answer to the multiple-choice question is four: When converting to a true / false question, according to the question stem, fill in the correct answers to form a true / false question.

[0029] Convert multiple-choice questions to single-choice questions: If the answer to the multiple-choice question is two: When converting to a single-choice question, remove one wrong answer and reverse the question stem to obtain the only correct single-choice answer.

[0030] 2. If the answer to the multiple-choice question is three: When converting to a single-choice question, reverse the question stem to obtain the only correct single-choice answer.

[0031] 3. If the answer to the multiple-choice question is four: When converting to a single-choice question, add the option "All other options are correct". For converting single-choice questions to true / false questions, randomly select a wrong answer or a correct answer as the question stem for judgment. For converting true / false questions to true / false questions, just reverse the question stem.

[0032] Multiple-choice questions are converted into multiple-choice questions: If the answer to a multiple-choice question is two: When converting to a multiple-choice question, reverse the stem of the question, and the original two incorrect answers become the correct answers.

[0033] True or false questions are converted into true or false questions: Reverse the stem of the question, and also reverse the answer to the true or false question.

[0034] Single-choice questions are converted into multiple-choice questions: Reverse the stem of the single-choice question, and the original incorrect answer becomes the correct answer.

[0035] Single-choice questions are converted into true or false questions: Fill in the correct answer at the option part in the stem of the question, then the answer to this true or false question is correct; Fill in the incorrect answer at the option part in the stem of the question, then the answer to this true or false question is incorrect.

[0036] The natural language processing technology mentioned in the present application above is specifically the NLP technology, including text paraphrasing - based on T5, BART models, Seq2Seq models, rewriting the description of the question into a synonymous sentence pattern, being able to convert the stems of a large number of multiple-choice questions into fill-in-the-blank questions, and also including a syntactic analysis system, which identifies the types of each word in the sentence, changes the active voice to the passive voice, the question type remains unchanged, and the missing nouns are changed, etc.

[0037] The present application also includes the ability to achieve cross - language question conversion. For example, after translating from Chinese to English, multiple-choice questions are generated. In summary, through each model, the present application sequentially converts and expands the questions in the original question bank into new questions. The conversion and expansion algorithm is more accurate and effective. It sequentially goes through text recognition, noun extraction to noun expansion, and question type conversion. There are intelligent algorithms for calibration for the conversion process and the validity of the final questions. And through the difficulty analysis of the final questions, by controlling the difficulty parameters, the efficiency, quantity, and depth of model conversion are corrected. The question type conversion algorithm can effectively support the diverse needs in the fields of educational technology, online evaluation, etc.

[0038] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are suggested in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0039] Meanwhile, this specification uses specific terms to describe the embodiments of this specification. For example, "an embodiment", "one embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that the "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0040] In addition, unless clearly stated in the claims, the order of the processing elements and sequences, the use of numerical letters, or the use of other names described in this specification are not used to limit the order of the processes and methods in this specification. Although some currently considered useful embodiments of the invention are discussed through various examples in the above disclosure, it should be understood that such details only serve the purpose of illustration. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0041] Similarly, it should be noted that, in order to simplify the expression of the disclosure in this specification and thus help the understanding of one or more embodiments of the invention, in the previous description of the embodiments of this specification, sometimes multiple features are merged into one embodiment, drawing, or description thereof. However, this disclosure method does not mean that the features required by the subject of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are less than all the features of the individual embodiments disclosed above.

[0042] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other deformations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification can be considered to be in accordance with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments clearly introduced and described in this specification.

Claims

1. A self-editing method for multi-question interaction, characterized in that: include, S1, classify the questions in the question bank by subject, pre-process them by the natural language processing model under the subject, and then send them to the machine learning model to obtain the converted new questions; S2, evaluates the effectiveness of new questions, including sending them to the automated evaluation model before sending them to manual review; S3, evaluate the quality and difficulty of new questions, and mark the difficulty level of effective new questions; S4, count the quality of new questions and the difficulty after marking, and optimize the machine learning model.

2. The multi-question interactive self-editing method according to claim 1, characterized in that: The working principle of the natural language processing model includes performing sentence structure conversion, synonym replacement and semantic analysis on the original question in sequence. Sentence structure conversion includes generating new question descriptions through text generation-related algorithms, changing the question structure through dependency syntax analysis and component syntax analysis, and synonym replacement generates new questions through a synonym replacement engine. Semantic analysis includes identifying subject professional nouns and analyzing the predicate-argument structure in sentences through semantic role labeling, extracting the core components of the question and reorganizing them.

3. The multi-question interactive self-editing method according to claim 2, characterized in that: The machine learning model includes a knowledge graph model and a conversion rule model. The knowledge graph model is used to associate professional terms with question type templates. The knowledge graph model divides the professional terms of various disciplines into multi-level associations. The associated professional terms can be replaced in the question type templates. The conversion rule model includes a variety of question type templates. By defining the conversion rules of the question template, the variables are replaced to generate a new question type.

4. The multi-question interactive self-editing method as claimed in claim 3, characterized in that: Question type templates include sequence generation templates, direct replacement templates, formula reorganization templates, and logical equivalent conversion templates. The sequence generation template uses the GPT system and the Transformer system to generate multiple-choice questions and true-or-false questions from open text. The direct replacement template is used to swap the answers and variables of the original questions. The formula reorganization template converts the questions into mathematical question forms through the symbolic calculation library and LaTeX parsing.

5. The multi-question interactive self-editing method according to claim 4, characterized in that: The effectiveness evaluation includes an automated evaluation model, which sends new questions to the model. If a predicate-argument structure is determined in the sentence through semantic analysis, it is defined as a valid new question. Manual review optimizes the semantic analysis algorithm by spot-checking valid new questions.

6. A multi-question interactive self-editing system, applied to the multi-question interactive self-editing method according to claim 5, characterized in that: Including sending the questions in the question bank to the natural language processing unit for question type conversion, optimizing the question type through the machine learning unit and then processing it by the difficulty grading marking unit, and optimizing the rules in the machine learning unit according to the difficulty grading results; Natural language processing unit, including text retelling rules, syntactic structure transformation rules, entity replacement and masking rules; Machine learning units, including generative rules, knowledge graph rules, and logical reasoning rules; Evaluation and verification unit, including verification of the uniqueness of the answer to the question through the BLEU evaluation system and the Z3 solver; Difficulty level marking units, including item response theory rules.

7. The multi-question interactive self-editing system according to claim 6, characterized in that: Generative rules generate new questions by replacing variables based on predefined question templates, and convert the form of mathematical questions based on symbolic calculation libraries and LaTeX parsing.

8. The multi-question interactive self-editing system as claimed in claim 7, characterized in that: New questions are sent to different types of students for practice based on difficulty levels. When the difficulty level of the newly generated questions needs to be lowered or increased, the question parameters and the number of questions converted by each rule in the machine learning unit are dynamically adjusted through the item response theory. After random sampling of the questions under the new number, new questions are only converted under a single rule for the same knowledge point.

Images