Education-oriented big language model questioning ability evaluation device

Abstract

The invention provides an education-oriented big language model question-asking ability evaluation device, which is characterized by comprising a data storage module used for storing a plurality of common domain contexts, professional domain contexts and interdisciplinary domain contexts; the common domain problem generation module is used for generating six levels of common domain problems; the professional subject domain problem generation module is used for generating six-level specialized domain problems; the interdisciplinary field question generation module is used for generating interdisciplinary questions; the common domain evaluation module is used for obtaining a common domain questioning ability result according to all the six-level common domain questions; the specialized subject field evaluation module is used for obtaining a specialized field questioning ability result according to all six-level specialized field questions; and the interdisciplinary field evaluation module is used for obtaining an interdisciplinary field questioning ability result according to all interdisciplinary questions. In a word, the device can quantitatively evaluate the teaching questioning ability of the large language model from three fields of common, professional and interdisciplinary.

Description

Technical Field

[0001] The present invention relates to the field of questioning ability assessment, and in particular to a large language model questioning ability assessment device for education. Background Art

[0002] Teachers play a vital role in imparting knowledge and answering questions. Considering large language models as teachers is significant in multiple ways. First, it can be applied in educational settings, such as in automated teaching or assisted learning applications. Second, exploring how they transmit knowledge can provide insights into the development of efficient educational tools to meet the needs of learners. In addition, this can advance the understanding of how humans acquire information and new skills from machines, driving innovation in artificial intelligence in education. Finally, it further promotes how to design algorithms with effective teaching capabilities, laying the foundation for future educational technology.

[0003] Previous studies have mainly viewed models from the perspective of students, focusing on reading comprehension and test assessment. However, these tasks focus on adopting context to passively answer questions or make inferences, while these tests treat large language models as students and assess their ability by how they answer questions. Therefore, the questioning ability of large language models as teachers has not been deeply studied.

[0004] Current education-related research is far from sufficient to determine the questioning ability of large language models as teachers, and there is no benchmark for comprehensive teaching ability research using large language models as teachers. Although some role-playing tasks simulate professional conversations, they do not truly evaluate the teaching ability of large language models.

[0005] In summary, the existing techniques fail to comprehensively evaluate the question-asking ability of large language models as teachers and give quantitative results. Summary of the invention

[0006] The present invention is made to solve the above-mentioned problems, and its purpose is to provide a large language model questioning ability evaluation device for education.

[0007] The present invention provides a large language model questioning ability evaluation device for education, which is used to obtain the questioning ability evaluation result of the large language model to be evaluated, and has the following characteristics: a data storage module, which is used to store multiple general domain contexts, professional subject domain contexts and professional subject domain contexts; a general domain question generation module, which is used to enable the large language model to be evaluated to generate corresponding six-level general domain questions according to each general domain context; a professional subject domain question generation module, which is used to enable the large language model to be evaluated to generate corresponding six-level professional domain questions according to each professional subject domain context; an interdisciplinary domain question generation module, which is used to enable the large language model to be evaluated to generate corresponding six-level professional domain questions according to each professional subject domain context The subject domain context generates corresponding interdisciplinary questions; the general domain assessment module is used to evaluate all six-level general domain questions according to the corresponding general domain context, and obtain the general domain questioning ability results; the professional subject domain assessment module is used to evaluate all six-level specialized domain questions according to the corresponding professional subject domain context, and obtain the specialized domain questioning ability results; the interdisciplinary domain assessment module is used to evaluate all interdisciplinary questions according to the corresponding professional subject domain context, and obtain the interdisciplinary domain questioning ability results, among which the questioning ability evaluation results include general domain questioning ability results, specialized domain questioning ability results and interdisciplinary domain questioning ability results.

[0008] The large language model questioning ability assessment device for education provided by the present invention may also have the following features: the six levels of general domain problems include general domain problems at the memory level, general domain problems at the understanding level, general domain problems at the application level, general domain problems at the analysis level, general domain problems at the evaluation level, and general domain problems at the creativity level, which respectively correspond to the six cognitive levels of Anderson and Kraswohl's educational classification method.

[0009] In the large language model questioning ability assessment device for education provided by the present invention, it can also have the following characteristics: wherein, the general domain assessment module includes: a general scoring unit, including a trained large language model, which is used to obtain the consistency score, relevance score and representativeness score of each question in each six-level general domain question through the general domain context corresponding to the six-level general domain question, as well as the coverage score corresponding to each general domain context; a general ability calculation unit, which is used to calculate the consistency average score, relevance average score and representativeness average score of each cognitive level, as well as the general domain coverage average score as the general domain questioning ability result based on all consistency scores, relevance scores, coverage scores and representativeness scores.

[0010] In the large language model questioning ability assessment device for education provided by the present invention, it can also have the following characteristics: among them, the six levels of specialized field problems include memory level specialized field problems, understanding level specialized field problems, application level specialized field problems, analysis level specialized field problems, evaluation level specialized field problems and creativity level specialized field problems, which respectively correspond to the six cognitive levels of Anderson and Kraswohl's educational classification method.

[0011] In the large language model questioning ability assessment device for education provided by the present invention, it can also have the following characteristics: wherein, the professional subject field assessment module includes: a professional scoring unit, including a trained large language model, which is used to obtain the consistency score, relevance score and representativeness score of each question in each six-level professional subject field problem through the professional subject field context corresponding to the six-level professional subject field problem, as well as the coverage score corresponding to each professional subject field context; a professional ability calculation unit, which is used to calculate the consistency average score, relevance average score and representativeness average score of each cognitive level, as well as the coverage average score of the professional field as the professional field questioning ability result based on all consistency scores, relevance scores, coverage scores and representativeness scores.

[0012] In the large language model questioning ability assessment device for education provided by the present invention, it can also have the following characteristics: wherein, the values ​​of the consistency score, relevance score, coverage score and representative score are all 1 or 0, and the corresponding judgment content is judged by the large language model to obtain the corresponding values ​​respectively; the large language model is used to judge whether the question is consistent with the corresponding cognitive level, if so, the consistency score is 1, if not, the consistency score is 0; the large language model is used to judge whether the question is related to the content or theme of the corresponding context, if so, the relevance score is 1, if not, the relevance score is 0; the large language model is used to judge whether the percentage of all questions corresponding to each context in the total content of the context is greater than a preset threshold, if so, the coverage score is 1, if not, the coverage score is 0; the large language model is used to judge whether the question captures the main content or core idea of ​​the context, if so, the representative score is 1, if not, the representative score is 0.

[0013] The large language model questioning ability assessment device for education provided by the present invention may also have the following feature: the large language model makes three judgments on each judgment content, and selects the values ​​corresponding to the two identical judgment results as the values.

[0014] The large language model questioning ability assessment device for education provided by the present invention may also have the following feature: wherein the preset threshold is 50%.

[0015] In the large language model questioning ability assessment device for education provided by the present invention, it can also have the following characteristics: wherein, interdisciplinary problems are problems involving multiple subject areas and can reflect the characteristics of each subject area, and the interdisciplinary field evaluation module includes: an interdisciplinary scoring unit, including a trained large language model, which is used to obtain corresponding relevance scores and representative scores for each interdisciplinary problem through the corresponding professional subject field context; an interdisciplinary ability calculation unit, which is used to calculate the average relevance score and the average representative score as the interdisciplinary field questioning ability result based on all the relevance scores and representative scores.

[0016] Functions and Effects of the Invention

[0017] According to the large language model questioning ability evaluation device for education involved in the present invention, because, through the general field question generation module, the professional subject field question generation module and the interdisciplinary field question generation module, the large language model to be evaluated generates different teaching questions in the general field, the specialized field and the interdisciplinary field in turn; through the general field evaluation module, the professional subject field evaluation module and the interdisciplinary field evaluation module, combined with the scoring guide words, the teaching questions in different fields are scored from different scoring angles, thereby obtaining a quantitative result that can reflect the teaching questioning ability of the large language model to be evaluated in different fields. Therefore, the large language model questioning ability evaluation device for education of the present invention can quantitatively evaluate the teaching questioning ability of the large language model from the three fields of general, professional and interdisciplinary. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of an educational questioning ability assessment in an embodiment of the present invention;

[0019] Figure 2 is a block diagram of a large model educational questioning and evaluation device in an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of scoring performed by a common scoring unit in an embodiment of the present invention;

[0021] Figure 4 It is a flow chart of the large language model teaching questioning ability assessment in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate the large language model questioning ability assessment device for education of the present invention.

[0023] In this embodiment, a large language model questioning ability evaluation device for education is provided, hereinafter referred to as the large model education questioning evaluation device, which is used to obtain the questioning ability evaluation result of the large language model to be evaluated.

[0024] Among them, the questioning ability evaluation results include the questioning ability results in the general field, the questioning ability results in the specialized field, and the questioning ability results in the interdisciplinary field. In this embodiment, the questioning ability results in the general field correspond to the general field, i.e., general knowledge tasks; the questioning ability results in the specialized field correspond to the specialized field, i.e., single-discipline tasks; and the questioning ability results in the interdisciplinary field correspond to the interdisciplinary field, i.e., cross-disciplinary tasks.

[0025] Figure 1 Schematic diagram of educational questioning ability assessment in an embodiment of the present invention.

[0026] like Figure 1 As shown, in the general knowledge task, the large model educational question evaluation device evaluates the questions asked by the large language model to be evaluated at the memory level, understanding level, application level, analysis level, evaluation level and creation level respectively.

[0027] In the single-subject task, the large model educational question assessment device assesses the questions of the large language model to be evaluated at the memory level, understanding level, application level, analysis level, evaluation level and creation level. For example, in the single-subject field of chemistry, the question at the memory level is "ask for the chemical formula of ammonium sulfide.", the question at the understanding level is "describe the chemical reaction equation between sodium bicarbonate and hydrochloric acid.", the question at the application level is "select a substance suitable for reacting with hydrochloric acid to produce carbon dioxide.", the question at the analysis level is "compare the reactions of sodium bicarbonate and ammonium sulfide with strong acids and strong bases.", the question at the evaluation level is "discuss the advantages and disadvantages of using sodium bicarbonate and hydrochloric acid to prepare carbon dioxide.", and the question at the creation level is "design an experiment to test the rate of CO2 production when different concentrations of hydrochloric acid react with NaHCO3.".

[0028] In the interdisciplinary task, the large model education question evaluation device evaluates the interdisciplinary questions of the large language model to be evaluated. For example, in the interdisciplinary field of literature and geography, the corresponding question is "In Li Bai's "The Rise of the Sun", "Who whips the four movements (spring, summer, autumn, and winter)? All things rise and fall naturally." What geographical phenomenon does this sentence describe?", that is, the question uses "The Rise of the Sun" in the field of literature as the background material to raise questions related to the field of geography. In the interdisciplinary field of art and geography, the corresponding question is "Shaanxi local folk song "Sheep belly, hand towel, three lines of blue, it's easy for us to meet each other, but it's difficult to talk, one is on that mountain, and the other is in that ditch. . . "This folk song has a high-spirited singing voice, and what are the natural geographical features?", that is, the question uses Shaanxi local folk songs in the field of art as the background material to raise questions related to the field of geography.

[0029] Figure 2 It is a block diagram of a large-scale model educational questioning and evaluation device in an embodiment of the present invention.

[0030] like Figure 2 As shown, the large-scale model educational questioning and evaluation device 100 includes a data storage module 10, a general field question generation module 20, a professional subject field question generation module 30, an interdisciplinary field question generation module 40, a general field evaluation module 50, a professional subject field evaluation module 60, an interdisciplinary field evaluation module 70 and a control module 80 for controlling the above modules.

[0031] The data storage module 10 is used to store a plurality of general domain contexts and professional subject domain contexts.

[0032] In this embodiment, the general domain context is obtained from the existing SQuAD dataset, which is a reading comprehension dataset extracted from Wikipedia articles. The professional subject domain context is generated by using GPT-4 based on each multiple-choice question in the existing MMLU dataset. The professional subject domain context includes basic information related to the question and all corresponding candidate items.

[0033] The general domain question generation module 20 is used to enable the large language model to be evaluated to generate corresponding six-level general domain questions according to various general domain contexts.

[0034] Among them, the six levels of general domain problems include general domain problems at the memory level, general domain problems at the understanding level, general domain problems at the application level, general domain problems at the analysis level, general domain problems at the evaluation level and general domain problems at the creativity level, which correspond to the six cognitive levels of Anderson and Kraswohl's educational classification, namely the memory level, the understanding level, the application level, the analysis level, the evaluation level and the creativity level.

[0035] The professional subject field question generation module 30 is used to enable the large language model to be evaluated to generate corresponding six-level professional subject field questions according to the context of each professional subject field.

[0036] Among them, the six levels of specialized field problems include specialized field problems at the memory level, specialized field problems at the understanding level, specialized field problems at the application level, specialized field problems at the analysis level, specialized field problems at the evaluation level and specialized field problems at the creativity level, which also correspond to the six cognitive levels of Anderson and Kraswohl's educational classification method.

[0037] The interdisciplinary field question generation module 40 is used to enable the large language model to be evaluated to generate corresponding interdisciplinary questions according to the context of each professional discipline field.

[0038] Among them, interdisciplinary problems are those that involve multiple disciplines and can reflect the characteristics of each discipline.

[0039] The general domain evaluation module 50 is used to evaluate all level 6 general domain questions according to the corresponding general domain contexts to obtain general domain questioning ability results. The general domain evaluation module 50 includes a general scoring unit 501 and a general ability calculation unit 502.

[0040] The general scoring unit 501 includes a trained large language model, which is used to obtain the consistency score, relevance score and representative score of each question in each of the six-level general domain questions through the general domain context corresponding to the six-level general domain questions, as well as the coverage score corresponding to each general domain context.

[0041] The general ability calculation unit 502 is used to calculate the consistency average score, relevance average score and representativeness average score of each cognitive level, and the general domain coverage average score as the general domain questioning ability result according to all consistency scores, relevance scores, coverage scores and representativeness scores.

[0042] The professional subject field evaluation module 60 is used to evaluate all six-level professional subject field questions according to the corresponding professional subject field context to obtain professional subject field questioning ability results, including a professional scoring unit 601 and a professional ability calculation unit 602.

[0043] The specialized scoring unit 601 includes a trained large language model, which is used to obtain the consistency score, relevance score and representative score of each question in each of the six-level specialized field questions through the professional subject field context corresponding to the six-level specialized field questions, as well as the coverage score corresponding to each professional subject field context.

[0044] The specialty ability calculation unit 602 is used to calculate the consistency average score, relevance average score and representativeness average score of each cognitive level, as well as the coverage average score of the specialty field as the specialty field questioning ability result according to all consistency scores, relevance scores, coverage scores and representativeness scores.

[0045] The interdisciplinary field evaluation module 70 is used to evaluate all interdisciplinary questions according to the corresponding professional subject field context to obtain interdisciplinary field questioning ability results, including an interdisciplinary scoring unit 701 and an interdisciplinary ability calculation unit 702.

[0046] The interdisciplinary scoring unit 701 includes a trained large language model, which is used to obtain corresponding relevance scores and representativeness scores for each interdisciplinary problem through the corresponding professional subject field context.

[0047] The interdisciplinary ability calculation unit 702 is used to calculate the average relevance score and the average representative score as the interdisciplinary questioning ability result according to all the relevance scores and representative scores.

[0048] In this embodiment, the consistency score, relevance score, coverage score and representative score of the general domain evaluation module 50 and the professional subject domain evaluation module 60, as well as the relevance score and representative score of the interdisciplinary domain evaluation module 70 are all 1 or 0, and the values ​​are obtained by judging the corresponding judgment content through the large language model in the corresponding module, that is, the corresponding judgment content is judged by the large language model to obtain the corresponding value. In this embodiment, the trained large language models in the general scoring unit 501, the specialized scoring unit 601 and the interdisciplinary scoring unit 701 are all GPT-4.

[0049] The large language model is used to determine whether the question is consistent with the corresponding cognitive level. If so, the consistency score is 1, otherwise, the consistency score is 0.

[0050] The large language model is used to determine whether the question is related to the content or topic of the corresponding context. If so, the relevance score is 1, otherwise, the relevance score is 0.

[0051] The large language model is used to determine whether the percentage of all questions corresponding to each context in the total content of the context is greater than a preset threshold. If so, the coverage score is 1, and if not, the coverage score is 0. The preset threshold is 50%.

[0052] The large language model is used to determine whether the question captures the main content or core idea of ​​the context. If so, the representative score is 1, otherwise, the representative score is 0.

[0053] In this embodiment, the large language model makes three judgments on each judgment content, and selects the values ​​corresponding to the two identical judgment results as the values. For example, the general scoring unit 501 uses the large language model it contains to score a certain question at the memory level three times in a row in the order of consistency score, relevance score, representative score and coverage score, and obtains the corresponding results of [1,1,1,1], [1,0,1,1] and [1,0,0,1]. In the end, the consistency score of the question is 1, the relevance score is 0, the representative score is 1, and the coverage score is 1. In this embodiment, all the scores of a single indicator of the model in the field are processed to obtain the corresponding total consistency score, total relevance score, total coverage score and total representative score.

[0054] Figure 3 It is a schematic diagram of scoring by a common scoring unit in an embodiment of the present invention.

[0055] like Figure 3 As shown, the general scoring unit 501 takes the comprehensive prompt containing the question of the memory level, the corresponding general domain context, and the scoring guide words corresponding to the consistency score, relevance score, representative score and coverage score as the input of the large language model, so that the large language model obtains the scoring result of the question, and then refines it into a concise representation for the consistency score, relevance score, coverage score and representative score.

[0056] The control module 80 stores a control program for controlling the operation of each module.

[0057] The following describes the process of evaluating the ability of large language model teaching questioning using the large model educational questioning evaluation device 100 in conjunction with the accompanying drawings.

[0058] Figure 4 It is a flow chart of the large language model teaching questioning ability assessment in an embodiment of the present invention.

[0059] like Figure 4 As shown in the figure, the evaluation of the question-asking ability of the large language model teaching includes the following steps:

[0060] Step S1, using the general domain question generation module 20 to enable the large language model to be evaluated to generate corresponding six-level general domain questions according to various general domain contexts.

[0061] Step S2, using the professional subject field question generation module 30 to enable the large language model to be evaluated to generate corresponding six-level professional subject field questions according to the context of each professional subject field.

[0062] Step S3, using the interdisciplinary field question generation module 40 to enable the large language model to be evaluated to generate corresponding interdisciplinary questions according to the context of each professional discipline field.

[0063] Step S4, using the general domain assessment module 50 to assess all six-level general domain questions according to the corresponding general domain contexts, and obtaining general domain questioning ability results.

[0064] Step S5, using the professional subject field assessment module 60 to assess all six-level professional subject field questions according to the corresponding professional subject field contexts, and obtaining professional subject field questioning ability results.

[0065] Step S6, using the interdisciplinary field assessment module 70 to assess all interdisciplinary questions according to the corresponding professional subject field context, and obtaining interdisciplinary field questioning ability results.

[0066] In this embodiment, the existing BLOOM-7B model, BLOOM-176B model and GPT4 model are selected as test models, and the large model educational questioning evaluation device 100 is used to evaluate the teaching questioning ability. In addition, the tester uses each model multiple times and performs subjective scoring, and the educational questioning ability ranking of each model is: GPT4 model>BLOOM-176B model>BLOOM-7B model.

[0067] The evaluation results of each model in the general field are shown in the following table:

[0068] Con Rel Cov Rep BLOOM-7B 45.5 56.4 26.6 32.1 BLOOM-176B 97.7 100.0 53.7 78.2 GPT4 98.5 100.0 54.5 80.1

[0069] The first column in the table above is each model, and the second to fifth columns are the consistency scores of each model in the general field, namely Con, the relevance score, namely Rel, the coverage score, namely Cov, and the representative score, namely Rep. For example, the cell in the second row and the second column indicates that the consistency score of the BLOOM-7B model in the general field is 45.5.

[0070] The evaluation results of each model in the specialty field are shown in the following table:

[0071] Con Rel Cov Rep BLOOM-7B 28.7 61.1 47.8 55.9 BLOOM-176B 58.3 88.1 62.6 71.1 GPT4 77.2 93.4 81.9 81.6

[0072] The first column in the table above is for each model, and the second to fifth columns are the consistency scores of each model in the specialty field, namely Con, the relevance score, namely Rel, the coverage score, namely Cov, and the representative score, namely Rep. For example, the cell in the second row and second column indicates that the consistency score of the BLOOM-7B model in the specialty field is 28.7.

[0073] The evaluation results of each model in the interdisciplinary field are shown in the following table:

[0074]

[0075]

[0076] The first column in the table above is the various models, and the second to third columns are the total relevance scores of each model in the interdisciplinary field, namely Rel and the total representativeness scores, namely Rep. For example, the cell in the second row and second column indicates that the total relevance score of the BLOOM-7B model in the interdisciplinary field is 29.9.

[0077] It can be seen that the evaluation results obtained by using the large-model educational questioning and evaluation device 100 are the same as the ranking results of the subjective evaluation, and the performance differences of each model are further reflected through the quantification of each indicator.

[0078] Functions and Effects of the Embodiments

[0079] According to the large language model questioning ability evaluation device for education involved in this embodiment, through the general field question generation module, the professional subject field question generation module and the interdisciplinary field question generation module, the large language model to be evaluated generates different teaching questions in the general field, the specialized field and the interdisciplinary field in turn; through the general field evaluation module, the professional subject field evaluation module and the interdisciplinary field evaluation module, combined with the scoring guide words, the teaching questions in different fields are scored from different scoring angles, thereby obtaining quantitative results that can reflect the teaching questioning ability of the large language model to be evaluated in different fields. In short, this device can quantitatively evaluate the teaching questioning ability of the large language model in the three fields of general, professional and interdisciplinary.

[0080] Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A large language model questioning ability evaluation device for education, used to obtain the questioning ability evaluation result of the large language model to be evaluated, characterized in that: include: A data storage module, used for storing a plurality of general domain contexts and professional subject domain contexts; A general domain question generation module, used to enable the large language model to be evaluated to generate corresponding six-level general domain questions according to each of the general domain contexts; A professional subject field question generation module is used to enable the large language model to be evaluated to generate corresponding six-level professional subject field questions according to each professional subject field context; An interdisciplinary field question generation module is used to enable the large language model to be evaluated to generate corresponding interdisciplinary questions according to the context of each professional subject field; A general domain evaluation module, used to evaluate all the six-level general domain questions according to the corresponding general domain contexts to obtain general domain questioning ability results; A professional subject field evaluation module is used to evaluate all the six-level professional subject field questions according to the corresponding professional subject field contexts to obtain professional subject field questioning ability results; The interdisciplinary field assessment module is used to assess all the interdisciplinary questions according to the corresponding professional subject field contexts to obtain interdisciplinary field questioning ability results. Among them, the questioning ability evaluation results include questioning ability results in general fields, questioning ability results in specialized fields, and questioning ability results in interdisciplinary fields.

2. The large language model questioning ability evaluation device for education according to claim 1 is characterized by: in, The six levels of general domain problems include general domain problems at the memory level, general domain problems at the understanding level, general domain problems at the application level, general domain problems at the analysis level, general domain problems at the evaluation level, and general domain problems at the creativity level, which correspond to the six cognitive levels of Anderson and Kraswohl's educational classification method respectively.

3. The large language model questioning ability evaluation device for education according to claim 2, Features: Wherein, the general domain assessment module includes: A general scoring unit, including a trained large language model, is used to obtain, for each of the six levels of general domain questions, a consistency score, a relevance score and a representative score of each of the six levels of general domain questions through the general domain contexts corresponding to the six levels of general domain questions, as well as a coverage score corresponding to each of the general domain contexts; The general ability calculation unit is used to calculate the consistency average score, relevance average score and representative average score of each cognitive level and the general domain coverage average score as the general domain questioning ability result according to all the consistency scores, the relevance scores, the coverage scores and the representative scores.

4. The large language model questioning ability evaluation device for education according to claim 1, characterized in that: in, The six levels of specialized field problems include specialized field problems at the memory level, specialized field problems at the understanding level, specialized field problems at the application level, specialized field problems at the analysis level, specialized field problems at the evaluation level, and specialized field problems at the creation level, which correspond to the six cognitive levels of Anderson and Kraswohl's educational classification method respectively.

5. The large language model questioning ability evaluation device for education according to claim 4, Features: The professional subject area assessment modules include: A specialized scoring unit, including a trained large language model, is used to obtain a consistency score, a relevance score and a representative score for each of the six-level specialized field questions, and a coverage score corresponding to each of the professional subject field contexts, for each of the six-level specialized field questions; The specialized ability calculation unit is used to calculate the consistency average score, relevance average score and representativeness average score of each cognitive level and the coverage average score of the specialized field as the result of the specialized field questioning ability based on all the consistency scores, the relevance scores, the coverage scores and the representativeness scores.

6. The large language model questioning ability evaluation device for education according to claim 3 or 5, characterized in that: in, The values ​​of the consistency score, the relevance score, the coverage score and the representative score are all 1 or 0, and the corresponding values ​​are obtained by judging the corresponding judgment contents through the large language model respectively. The large language model is used to determine whether the question is consistent with the corresponding cognitive level. If so, the consistency score is 1; if not, the consistency score is 0. The large language model is used to determine whether the question is related to the content or topic of the corresponding context. If so, the relevance score is 1; if not, the relevance score is 0. The large language model is used to determine whether the percentage of all the questions corresponding to each context in the total content of the context is greater than a preset threshold. If so, the coverage score is 1; if not, the coverage score is 0. The large language model is used to determine whether the question captures the main content or core idea of ​​the context. If so, the representative score is 1; otherwise, the representative score is 0.

7. The large language model questioning ability evaluation device for education according to claim 6, characterized in that: in, The large language model performs three judgments on each of the judgment contents, and selects values ​​corresponding to two identical judgment results as values.

8. The large language model questioning ability assessment device for education according to claim 6, characterized in that: in, The preset threshold is 50%.

9. The large language model questioning ability evaluation device for education according to claim 1, Features: The interdisciplinary problems are problems involving multiple disciplines and can reflect the characteristics of each discipline. The cross-disciplinary field assessment modules include: An interdisciplinary scoring unit, including a trained large language model, is used to obtain a corresponding relevance score and a representative score for each of the interdisciplinary questions through the corresponding professional subject field context; The interdisciplinary ability calculation unit is used to calculate the average relevance score and the average representativeness score as the interdisciplinary field questioning ability result according to all the relevance scores and the representativeness scores.

Images