Multi-target test question generation method and system based on double-model engine

Abstract

The invention discloses a multi-target test question generation method and system based on a double-model engine, and the method comprises the steps: firstly determining a design target of a test question, including a target knowledge point and a target cognition level, constructing a cue word through a multi-round multi-target comprehensive prompt method, and inputting the cue word into a question generation model, thereby obtaining a candidate question set; constructing cue words through a multi-sample cue method, inputting the candidate questions into a question evaluation model, and judging whether the questions conform to a target cognitive hierarchy or not; and if the questions accord with and are not recorded, adding the questions into a result set, otherwise, generating and evaluating again. The system is composed of the question generation model and the question evaluation model, through a double-model cooperative working mode, questions meeting multi-dimensional targets at the same time can be generated, the diversity and accuracy of the questions can be improved, meanwhile, the storage and calculation expenses of the system are greatly reduced, the question generation efficiency and quality are improved, and the system is suitable for popularization and application. And a universally applicable solution is provided for an intelligent test generation system.

Claims

1. A method for generating multi-objective test questions based on a dual model engine, characterized in that: The following steps are involved: S1: Determine the target knowledge points and target cognitive levels of the test items, including Bloom's cognitive hierarchy; S2: The target knowledge points and target cognitive levels determined in S1 are constructed into multi-target comprehensive question generation prompt words through a multi-round multi-target comprehensive prompt method, and the question generation prompt words are input into the question generation engine. The question generation model in the question evaluation engine generates a series of candidate questions to obtain a candidate question set; S3: Use a multi-sample prompt method to construct question evaluation prompt words, input the question evaluation prompt words into the question evaluation engine, and use the question evaluation model in the question evaluation engine to determine whether the questions in the candidate question set meet the target cognitive level; S4: Use the question evaluation engine to determine whether the question that meets the requirements in S2 is already in the result set. If not, add the question to the result set. If so, return to S2 and regenerate the question. S5: Output the result set.

2. The method for generating multi-objective test questions based on a dual model engine according to claim 1, characterized in that: The problem generation model, the construction method thereof comprises: A question generation fine-tuning dataset is collected, and the benchmark model ChatGLM2-6B is fine-tuned using the parameter efficient fine-tuning method P-Tuningv2 to obtain the question generation model.

3. The method for generating multi-objective test questions based on a dual model engine according to claim 1, characterized in that: The problem assessment model, its construction method includes: A fine-tuning dataset is generated for the problem, a problem evaluation fine-tuning dataset is constructed using inverse formation, and the benchmark model ChatGLM2-6B is fine-tuned using the parameter efficient fine-tuning method P-Tuning v2 to obtain the problem evaluation model.

4. The method for generating multi-objective test questions based on a dual model engine according to claim 2, characterized in that: The acquisition problem generates a fine-tuning dataset, which specifically includes: 1) Obtain a public dataset of general domains containing correspondences of Bloom’s cognitive hierarchy, covering the six cognitive levels of memory, understanding, application, analysis, evaluation, and creation; 2) Extract the questions in the data set described in step 1) and reconstruct and splice them according to the following prompt words: "'Prompt': Please generate a question belonging to 'Memory / Understanding / Application / Analysis / Evaluation / Creation' in Bloom's cognitive hierarchy; 'Answer': '[Question]'"; 3) Merge the data obtained in step 2) to form a fine-tuning dataset for problem generation.

5. The method for generating multi-objective test questions based on a dual model engine according to claim 3, characterized in that: The use of reverse formation to construct a problem evaluation fine-tuning dataset specifically includes: 1) Obtain a public dataset of general domains containing correspondences of Bloom’s cognitive hierarchy, covering the six cognitive levels of memory, understanding, application, analysis, evaluation, and creation; 2) Extract the questions in the data set described in step 1) and reconstruct and splice them according to the following prompt words: "Hint": Please determine what Bloom's cognitive level the question [question] belongs to? "Answer": "Memory / understanding / application / analysis / evaluation / creation"; 3) Merge the data obtained in step 2) to form a problem evaluation fine-tuning dataset.

6. The method for generating multi-objective test questions based on a dual model engine according to claim 1, characterized in that: The step S2 of constructing a multi-objective comprehensive topic and generating prompt words through a multi-round multi-objective comprehensive prompt method specifically includes: 1) Determine the target knowledge points and target Bloom’s cognitive level; 2) Construct the first prompt word based on the target Bloom’s cognitive level, input it into the question generation model, and generate the corresponding question of the target Bloom’s cognitive level; 3) Construct a second prompt word based on the target knowledge point, input it into the question generation model, and generate the corresponding question of the target knowledge point; 4) The cognitive level questions and knowledge point questions generated in step 2) and step 3) are combined with the preset prompt template and integrated using the text splicing method to form prompt words for generating multi-objective comprehensive questions.

7. The method for generating multi-objective test questions based on a dual model engine according to claim 1, characterized in that: The step S3 uses a multi-sample prompt method to construct a question evaluation prompt word, specifically including: According to Bloom's cognitive hierarchy theory, a case set containing all cognitive levels is constructed. Each cognitive level contains a sample question and its corresponding classification. The Bloom's cognitive levels are memory, understanding, application, analysis, evaluation and creation. The multi-sample prompt method constructs problem evaluation prompt words together with the specific problems to be classified and the case set.

8. A multi-objective test question generation system based on a dual model engine, characterized in that: include: Question generation engine: including question generation model and multi-round dialogue prompt module. The question generation model is obtained by fine-tuning the baseline language model through prompts of the question generation dataset. The multi-round dialogue prompt module inputs the question generation model through the question generation prompt method to generate a series of candidate questions and obtain a preliminary candidate question set; Question evaluation engine: It includes question evaluation model, cognitive level judgment module and question repetition recognition module. The question evaluation model is obtained by fine-tuning the baseline language model through prompts from the question evaluation fine-tuning dataset. The cognitive level judgment module uses the stratified sample prompt method to prompt the question evaluation model to evaluate whether the questions in the candidate question set meet the target Bloom's cognitive level; The question repetition identification module is used to identify whether a question is generated repeatedly.

Images