Teacher artificial intelligence literacy dynamic evaluation method and system based on generative technology

Abstract

The invention relates to a teacher artificial intelligence literacy dynamic evaluation method based on a generative technology. The method comprises the steps of obtaining teacher background information; a multi-dimensional evaluation task is dynamically generated according to the teacher background information by utilizing a generative technology, and the multi-dimensional evaluation task comprises case evaluation and question evaluation; obtaining performance data of completing the multi-dimensional evaluation task by the teacher, automatically processing the performance data by using a natural language processing technology, and automatically generating a multi-dimensional evaluation result based on a multi-dimensional evaluation index; and generating a feedback report according to the multi-dimensional evaluation result, and storing the multi-dimensional evaluation task, the performance data, the multi-dimensional evaluation result and the feedback report into an evaluation database. Compared with the prior art, the three-layer structure of knowledge, skills and attitudes is innovatively combined, complex attitudes definition is converted into operable evaluation tasks, and scientificity and comprehensiveness of evaluation are ensured.

Description

Technical Field

[0001] The present invention relates to the field of educational intelligence, and in particular to a dynamic evaluation method and system for teachers' artificial intelligence literacy based on generative technology. Background Art

[0002] With the rapid development of artificial intelligence technology, teachers' artificial intelligence literacy has become an important concern in the field of modern education. Teachers' artificial intelligence literacy not only includes the mastery of basic artificial intelligence knowledge, but also includes the ability to apply artificial intelligence technology to teaching practice, the awareness of artificial intelligence ethics issues, and the comprehensive ability to effectively integrate artificial intelligence tools in teaching design. However, current evaluation methods for teachers' artificial intelligence literacy mainly rely on traditional means, lacking dynamism, personalization, and multi-dimensional coverage. For teachers with different quality levels, the evaluation effect is not ideal and the pertinence is poor.

[0003] Traditional evaluation methods mainly include questionnaires, interviews and observations, and online evaluation systems.

[0004] Questionnaires: The most commonly used method for evaluating teachers' literacy at present, aiming to evaluate teachers' artificial intelligence knowledge and attitudes through a series of fixed questions. This method is easy to operate, but lacks dynamic evaluation of teachers' actual abilities. The evaluation results mainly reflect teachers' subjective cognitive levels and cannot deeply measure their application abilities and teaching integration abilities.

[0005] Interviews and observations: Some scholars obtain the comprehensive ability performance of teachers by organizing experts to conduct face-to-face interviews or classroom observations on teachers. However, such methods are time-consuming and laborious, and are subject to the subjectivity of expert evaluation, making it difficult to be promoted to large-scale applications.

[0006] Online evaluation systems: Some intelligent evaluation systems attempt to improve the evaluation of teachers' literacy using online tools. For example, tools such as Quizlet and Kahoot provide adaptive tests of basic knowledge through question banks, while learning platforms such as Knewton use artificial intelligence algorithms to recommend personalized learning paths. Although these systems have improved the evaluation efficiency to a certain extent, they are mainly limited to the evaluation at the knowledge level and are difficult to generate diverse evaluation tasks adapted to different teacher backgrounds.

[0007] In some academic research, attempts have been made to reflect teachers' technology application capabilities through evaluation methods based on case evaluation analysis, virtual scenarios, or teacher-student mutual evaluation. Chinese Patent CN113487213A discloses a vocational education teaching evaluation method based on big data. This method constructs an intelligent decision-making evaluation index system, establishes a vocational education teaching evaluation model, and visually displays teaching quality evaluation data. Patent No. CN113989081A discloses a smart evaluation system for college students' projects with hybrid enhanced intelligence, which introduces student mutual evaluation and teacher mutual evaluation into artificial intelligence evaluation to form a hybrid enhanced intelligence evaluation system with humans in the loop. However, the evaluations of both patents lack dynamism and personalization, are less targeted at the individual situations of different teachers, and are inefficient at the same time, making it difficult to meet the needs of large-scale applications.

[0008] Teachers in different school stages, subjects, and teaching experiences have different usage requirements and experiences with artificial intelligence. For example, the application scenarios of artificial intelligence for mathematics teachers will be data analysis or chart generation, while those for science teachers will focus on experimental design or simulated chemistry teaching. If only general artificial intelligence knowledge or universal scenario simulations are evaluated, it will not be able to provide targeted feedback and improvement references for teachers, which will restrict the continuous development of teachers in the era of artificial intelligence. Summary of the Invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a dynamic evaluation method and system for teachers' artificial intelligence literacy based on generative technology.

[0010] The purpose of the present invention can be achieved through the following technical solutions:

[0011] A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology, the method comprising:

[0012] Obtain teachers' background information;

[0013] Using generative technology, dynamically generate multi-dimensional evaluation tasks according to the teachers' background information, the multi-dimensional evaluation tasks including case evaluation and question evaluation;

[0014] Obtain the performance data of the teachers completing the multi-dimensional evaluation tasks, automatically process the performance data using natural language processing technology, and automatically generate multi-dimensional evaluation results based on multi-dimensional evaluation indicators;

[0015] Generate a feedback report according to the multi-dimensional evaluation results, and store the teachers' background information, multi-dimensional evaluation tasks, performance data, multi-dimensional evaluation results, and feedback report in an evaluation database.

[0016] Further, the teacher background information includes the school stage, subject background, teaching experience, and artificial intelligence knowledge level.

[0017] Furthermore, the multi-dimensional assessment tasks include knowledge assessment, skill assessment, and attitude assessment. The dynamic generation process of the multi-dimensional assessment tasks includes:

[0018] Classify the teachers based on their subject backgrounds according to the preset artificial intelligence requirements for subject teaching;

[0019] Determine the initial weights of the knowledge assessment, skill assessment, and attitude assessment according to the teacher's school stage and teaching experience, and adjust the initial weights in combination with the artificial intelligence knowledge level to obtain the final weights, which are used to adjust the proportion of the generated assessment case numbers;

[0020] Use generative technology to generate knowledge assessments, skill assessments, and attitude assessments in corresponding classification fields and quantities according to the classification of the artificial intelligence requirements for subject teaching of the teachers and the final weights, where

[0021] The knowledge assessment includes multiple-choice questions and essay questions, the skill assessment includes scenario tasks and design tasks, and the attitude assessment includes case analysis and reflective tasks.

[0022] Further, the multi-dimensional assessment tasks are generated based on the five dimensions and three-layer assessment structure of the UNESCO framework. The five dimensions include people-centered thinking, artificial intelligence ethics, artificial intelligence foundation and application, artificial intelligence pedagogy, and artificial intelligence career development. The three-layer assessment structure includes knowledge, skills, and attitudes.

[0023] Further, the performance data includes knowledge assessment data, skill assessment data, and attitude assessment data. The process of automatically processing the performance data includes:

[0024] Use natural language processing technology to perform data cleaning, missing value processing, outlier detection, data normalization, word segmentation, part-of-speech tagging, named entity recognition, syntactic analysis, and sentiment analysis on the performance data, and extract the keywords and key phrases of the knowledge assessment data, the logical structure and key elements of the skill assessment data, and the logical structure, key elements, and sentiment tendency of the attitude assessment data in the performance data.

[0025] Furthermore, the multi-dimensional assessment indicators include knowledge assessment indicators, skill assessment indicators, and attitude assessment indicators. The process of automatically generating the multi-dimensional assessment results includes:

[0026] Use natural language processing technology to semantically match the extracted keywords and key phrases with the knowledge graph, calculate the semantic similarity, and output the corresponding knowledge assessment indicator scores according to the semantic similarity;

[0027] Using natural language processing technology, based on the logical structure and key elements of the extracted skill assessment data, analyze the integrity, rationality, and innovation of the skill assessment data, and output the corresponding skill assessment index scores according to the analysis results;

[0028] Using natural language processing technology, according to the logical structure, key elements, and sentiment tendency of the extracted attitude assessment data, judge the teacher's attitude towards artificial intelligence ethics issues, and output the corresponding attitude assessment index scores according to the judgment results;

[0029] According to the preset weights, comprehensively calculate the multi-dimensional assessment total score based on the knowledge assessment index scores, skill assessment index scores, and attitude assessment index scores, and output it as the multi-dimensional assessment result.

[0030] Furthermore, the knowledge graph is formed by performing information extraction, knowledge fusion, graph construction, and graph quality assessment and optimization on the data in the assessment database.

[0031] Furthermore, the process of generating the feedback report includes:

[0032] Obtain the knowledge assessment index scores, skill assessment index scores, attitude assessment index scores, and multi-dimensional assessment total score, determine the teacher's level in each dimension according to the preset scoring criteria, and convert it into a visual chart;

[0033] Generate improvement suggestions using generative technology based on the teacher's level in each dimension;

[0034] Integrate the visual chart and the improvement suggestions and output them as a feedback report.

[0035] Furthermore, the performance data, assessment results, and feedback reports are classified and stored in the assessment database, and duplicate removal processing is performed on the performance data, assessment results, and feedback reports before storage.

[0036] A dynamic assessment system for teachers' artificial intelligence literacy based on generative technology, the system includes:

[0037] An input module that obtains the teacher's background information;

[0038] A task generation module that uses generative technology to dynamically generate multi-dimensional assessment tasks according to the teacher's background information, and the multi-dimensional assessment tasks include case evaluation and question evaluation;

[0039] A performance evaluation module that obtains the performance data of the teacher completing the multi-dimensional assessment tasks, automatically analyzes the performance data based on natural language processing technology and multi-dimensional assessment indicators, and automatically generates multi-dimensional assessment results;

[0040] A feedback improvement module that generates a feedback report based on the multi-dimensional evaluation results;

[0041] A data storage and management module that stores, queries, and manages the teacher background information, multi-dimensional evaluation tasks, performance data, multi-dimensional evaluation results, and feedback reports.

[0042] Compared with the prior art, the beneficial effects of the present invention include:

[0043] 1. The present invention dynamically generates tasks according to the teacher background, covering case evaluation and question evaluation, with various forms, breaking through the problem of fixed traditional evaluation tasks; when dynamically generating multi-dimensional evaluation tasks, first obtain the artificial intelligence requirements for subject teaching based on the teacher's subject background, then obtain the initial weights according to the school stage and teaching experience, adjust the initial weights in combination with the artificial intelligence knowledge level to obtain the final weights, and finally generate knowledge evaluation, skill evaluation, and attitude evaluation in the corresponding classification fields and quantities according to the subject teaching artificial intelligence requirement classification and final weights of the teacher. This generation method of dynamically adjusting the proportion and difficulty of task types according to the teacher's personal situation is more targeted and fairer than the fixed method of calling questions from the database for question setting, and the scoring of teachers will be more objective, making the final evaluation results more in line with the actual capabilities of teachers, more accurate, improving the applicability and personalization of the evaluation;

[0044] 2. Existing methods often focus on the knowledge level and ignore other important dimensions. The present invention combines the evaluation contents of five dimensions with the knowledge, skill, and attitude hierarchical structures, making the task design more refined. It innovatively combines the three-layer structures of knowledge, skill, and attitude, transforms the complex literacy definition into an operable evaluation task, and ensures the scientificity and comprehensiveness of the evaluation;

[0045] 3. From task generation, data analysis to feedback generation, the present invention realizes automation through the system, significantly improves the evaluation efficiency, supports the application of a large-scale teacher group, and reduces the evaluation cost;

[0046] 4. The present invention establishes a data storage and management module to realize the effective storage, query, and management of evaluation data, provides data support for subsequent result analysis and research, and can also perform statistical analysis on a large amount of teacher evaluation data in the future, which helps to promote the improvement of teacher literacy and the optimization of artificial intelligence education policies. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of the method of the present invention;

[0048] Figure 2 It is a system structure diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0049] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0050] Embodiment 1

[0051] A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology, the method includes:

[0052] S1. Obtain teachers' background information;

[0053] S2. Use generative technology to dynamically generate multi-dimensional evaluation tasks according to teachers' background information. The multi-dimensional evaluation tasks include case evaluation and question evaluation;

[0054] S3. Obtain the performance data of teachers' completion of multi-dimensional evaluation tasks, automatically process the performance data using natural language processing technology, and automatically generate multi-dimensional evaluation results based on multi-dimensional evaluation indicators;

[0055] S4. Generate a feedback report according to the multi-dimensional evaluation results, and store the teachers' background information, multi-dimensional evaluation tasks, performance data, multi-dimensional evaluation results and feedback reports in the evaluation database.

[0056] In step S1, the teachers' background information includes school stage, subject background, teaching experience and artificial intelligence knowledge level. The school stage includes primary school, junior high school or senior high school, etc., reflecting the specific teaching stage of teachers.

[0057] This information will be used to determine the relevance and depth of the generated tasks. The teachers' artificial intelligence cognitive level is obtained through questionnaires or short self-assessment surveys. The information collection questions include whether the teacher is familiar with the basic concepts of artificial intelligence, whether they understand the application of artificial intelligence in education, etc.

[0058] In step S2, the multi-dimensional evaluation tasks include knowledge evaluation, skill evaluation and attitude evaluation.

[0059] Knowledge evaluation: multiple-choice questions, short-answer questions, etc., to evaluate teachers' understanding of the basic concepts, application scenarios and ethical issues of artificial intelligence.

[0060] Skill evaluation: scenario tasks, design tasks, etc., to evaluate whether teachers can apply artificial intelligence to teaching design, how to use AI tools for classroom management or personalized learning.

[0061] Attitude assessment: Through case analysis, reflective tasks, etc., evaluate teachers' awareness and sense of responsibility regarding artificial intelligence ethical issues, as well as their acceptance and thinking about the application of AI technology in education.

[0062] The dynamic generation process of multi-dimensional assessment tasks is as follows:

[0063] Classify teachers based on their subject backgrounds according to the preset artificial intelligence requirements for subject teaching.

[0064] Determine the initial weights of knowledge assessment, skill assessment, and attitude assessment based on the teachers' school levels and teaching experience, and adjust the initial weights in combination with their artificial intelligence knowledge levels to obtain the final weights, which are used to adjust the proportion of the generated assessment case quantities.

[0065] Using generative technology, generate knowledge assessments, skill assessments, and attitude assessments in the corresponding classification fields and quantities according to the classification of artificial intelligence requirements for subject teaching of the teachers and the final weights. Among them,

[0066] Knowledge assessments include multiple-choice questions and essay questions, skill assessments include scenario tasks and design tasks, and attitude assessments include case analysis and reflective tasks.

[0067] In this embodiment, assume a novice teacher in the primary school stage. The initial weight of the knowledge assessment is set to 0.4, the initial weight of the skill assessment is 0.3, and the initial weight of the attitude assessment is 0.3. As the teachers' school levels increase and their teaching experience grows, the weights will be adjusted accordingly. For example, for a teacher in the high school stage with rich teaching experience, the initial weight of the knowledge assessment may be adjusted to 0.3, the initial weight of the skill assessment is increased to 0.4, and the weight of the attitude assessment remains at 0.3.

[0068] Subsequently, optimize the initial weights in combination with the teachers' artificial intelligence knowledge levels. If a teacher has participated in artificial intelligence-related training or is proficient in using artificial intelligence tools in actual teaching, the weight of the artificial intelligence knowledge part in the knowledge assessment can be appropriately increased, such as from the initial 0.3 to 0.4. Correspondingly, the weights of other assessment dimensions will be reduced proportionally to ensure that the total weight is 1. After this series of adjustments, the final weights are obtained, which will be used to accurately adjust the proportion of the generated assessment case quantities.

[0069] For knowledge assessment, when it is determined that the weight of a certain type of teacher's knowledge assessment is relatively high, a larger number of knowledge assessment questions will be generated. Among them, multiple-choice questions can cover options such as basic concepts of artificial intelligence and application scenarios of artificial intelligence related to the subject. For example, "In mathematics teaching, which operation using artificial intelligence is most helpful for improving students' understanding of function graphs? A. Intelligent homework grading B. Constructing function models C. Automatically generating teaching plans"; essay questions require teachers to elaborate on the specific applications and advantages of artificial intelligence in subject teaching. For example, "Please explain in detail how artificial intelligence can assist in Chinese reading teaching and improve students' reading comprehension ability."

[0070] For skills assessment, if the weight of skills assessment is relatively high, more scenario tasks and design tasks will be generated. The scenario task may be set as "Suppose you are teaching a junior high school physics experiment. There is an artificial intelligence-assisted experimental device available. Describe how you will use it to optimize the experimental course and improve students' participation"; the design task may require teachers to design a plan for using artificial intelligence tools to review high school chemistry courses.

[0071] In terms of attitude assessment, when the weight of attitude assessment is relatively high, more case analyses and reflective tasks will be generated. The case analysis can provide an actual case of the failure of applying artificial intelligence in subject teaching, allowing teachers to analyze the reasons and propose improvement measures; the reflective task may let teachers reflect on their attitudes towards the use of artificial intelligence tools in teaching in the past semester and how to improve to better serve teaching. In this way, according to different classifications and weights, multi-dimensional assessment tasks corresponding to the classification fields and quantities are generated.

[0072] The multi-dimensional assessment tasks are generated based on the five dimensions and three-layer assessment structure of the UNESCO framework. The five dimensions include people-centered thinking, artificial intelligence ethics, artificial intelligence foundation and application, artificial intelligence pedagogy, and artificial intelligence career development. The three-layer assessment structure includes knowledge, skills, and attitude.

[0073] 1. Assess the people-centered concept of artificial intelligence

[0074] Background information: Teachers input their school stage (such as primary school) and subject background (such as mathematics).

[0075] Task type: Case analysis (attitude assessment)

[0076] Task example:

[0077] Question: Suppose your school has adopted an AI tool to automatically assess students' math homework. Although this AI tool is efficient, some students have reported that their personalized learning needs have not been fully considered. In this case, how do you think we can balance the application of technology and the personalized development of students?

[0078] Evaluation Dimensions:

[0079] Knowledge: Whether one understands the people-centered thinking principle and can identify possible defects in AI applications.

[0080] Skills: Whether one can propose solutions on how to adjust the application of AI tools to pay more attention to individual student differences.

[0081] Attitude: Whether one shows a high degree of attention to students' personalized development and needs, and whether one believes that AI should serve students' all-round growth.

[0082] 2. Evaluate AI Ethics

[0083] Background Information: Teachers input their school stage (such as high school) and subject background (such as sociology).

[0084] Task Type: Reflective Questions (Knowledge Assessment)

[0085] Task Example:

[0086] Question: In an AI-based learning platform, students' personal learning data is used to predict grades and generate personalized recommendations. If the platform shares the data with third parties without the consent of students and parents, what ethical issues do you think this behavior has?

[0087] Evaluation Dimensions:

[0088] Knowledge: Whether one can identify data privacy and ethical issues and understand the importance of data sharing and the right to privacy.

[0089] Skills: Whether one can analyze potential ethical issues in the application of AI technology and propose appropriate improvement suggestions.

[0090] Attitude: Whether one shows a high sense of responsibility for students' data protection and ethical issues and can propose improvement measures.

[0091] 3. Evaluate the Basics and Applications of AI

[0092] Background Information: Teachers input their school stage (such as junior high school) and subject background (such as physics).

[0093] Task Type: Scenario Task (Skills Assessment)

[0094] Task Example:

[0095] Question: You are designing an AI-based physics experiment to simulate the relationship between current and voltage using AI. Please describe how you would use AI tools to design this experiment and apply it to classroom teaching.

[0096] Evaluation Dimensions:

[0097] Knowledge: Whether one understands the basic applications of AI in physical experiments and knows how to apply AI to simulate the experimental process.

[0098] Skills: Whether one is able to design an AI-driven experimental plan that can effectively help students understand the relationship between current and voltage.

[0099] Attitude: Whether one can fully demonstrate the application potential of AI technology in teaching and is able to integrate AI technology into teaching design with innovative thinking.

[0100] 4. Evaluate the AI teaching method

[0101] Background information: Teachers input their school stage (such as primary school) and subject background (such as English).

[0102] Task type: Design task (skill assessment)

[0103] Task example:

[0104] Question: Design an AI-based English reading class where AI will provide personalized learning resources and tasks according to students' reading progress and comprehension ability. Please describe how you use AI technology to design the course content to ensure that each student can obtain a suitable learning experience.

[0105] Evaluation dimensions:

[0106] Knowledge: Whether one understands the basic application methods of AI in teaching and knows how to use AI to achieve personalized learning.

[0107] Skills: Whether one is able to design a personalized AI teaching plan that can adjust content according to students' needs.

[0108] Attitude: Whether one can consider the application of AI technology from the perspective of educational equity to ensure that the learning needs of each student are met.

[0109] 5. Evaluate the career development of AI

[0110] Background information: Teachers input their school stage (such as junior high school) and subject background (such as Chinese).

[0111] Task type: Reflective question (attitude assessment)

[0112] Task example:

[0113] Question: In the next 5 years, how do you think artificial intelligence will further develop in teaching? How will you use these technologies to improve your teaching effectiveness?

[0114] Evaluation dimensions:

[0115] Knowledge: Can you describe the potential and trends of artificial intelligence in the field of education?

[0116] Skills: Can you propose specific solutions for applying AI to improve teaching effectiveness?

[0117] Attitude: Are you actively receptive to and implementing the application of AI technology in future teaching?

[0118] In step S3, the performance data of teachers completing the multi-dimensional evaluation tasks includes knowledge evaluation data, skills evaluation data, and attitude evaluation data.

[0119] The process of automatically processing the performance data includes:

[0120] Using natural language processing technology to perform data cleaning, missing value handling, outlier detection, data normalization, word segmentation, part-of-speech tagging, named entity recognition, syntactic analysis, and sentiment analysis on the performance data, and extracting keywords and key phrases of the knowledge evaluation data, logical structures and key elements of the skills evaluation data, as well as logical structures, key elements, and sentiment tendencies of the attitude evaluation data in the performance data.

[0121] Data cleaning: Clean the collected performance data to remove duplicate, incorrect, or incomplete data. For example, remove garbled characters and data records with incorrect formats caused by system failures, etc., to ensure the accuracy and integrity of the data and provide a reliable data source for subsequent analysis.

[0122] Missing value handling: For missing values in the knowledge evaluation data, skills evaluation data, and attitude evaluation data, they can be filled using mean filling, mode filling, or model-based prediction methods. For example, for missing values of certain skill indicators in the skills evaluation data, they can be reasonably estimated and filled according to the skill levels of similar samples.

[0123] Outlier detection: Use statistical methods such as box plot analysis and density-based spatial clustering algorithms to identify outliers in the data. Taking the knowledge evaluation data as an example, if there are extreme values with scores far higher or lower than the normal range, it is necessary to further verify its authenticity and determine whether it is caused by data entry errors or other special circumstances.

[0124] Data normalization: Scale the data uniformly to a specific interval to make different types of performance data comparable.

[0125] Word segmentation: Use word segmentation tools such as jieba to segment continuous text into individual words for subsequent analysis.

[0126] Part-of-speech tagging: Identify the part of speech of each word, such as nouns, verbs, adjectives, etc. This helps to understand the grammatical structure and semantic information of the text, and is of great significance for the extraction of keywords in knowledge assessment data and the sentiment analysis of attitude assessment data.

[0127] Named entity recognition: Identify specific entities such as personal names, place names, and organization names in the text. In skills assessment data, it may involve specific technical names, project names, etc. Accurately identifying these entities is crucial for sorting out the logical structure and key elements of skills assessment data.

[0128] Syntactic analysis: Used to analyze the grammatical structure of sentences, determine the subject, predicate, object, attributive, adverbial, complement, etc. of the sentence. Through syntactic analysis, it is possible to better understand the meaning of complex sentences in knowledge assessment data, extract key information, and also help to analyze the grammatical structure of emotional expressions in attitude assessment data.

[0129] Sentiment analysis: For attitude assessment data, use natural language processing technology to judge the sentiment tendency expressed by the text, whether it is positive, negative or neutral. Here, natural language processing technology includes using sentiment dictionary methods or machine learning algorithms for processing. Sentiment dictionary methods such as the sentiment dictionary based on HowNet judge the sentiment of the text according to the sentiment polarity (positive, negative, neutral) of the words in the dictionary. In terms of machine learning algorithms, commonly used classification algorithms such as Naive Bayes and Support Vector Machine are used to train on a large amount of text data with sentiment annotations to build a sentiment classification model, so as to judge the sentiment tendency of attitude assessment data.

[0130] The multi-dimensional evaluation indicators include knowledge evaluation indicators, skills evaluation indicators and attitude evaluation indicators.

[0131] In step S3, the process of automatically generating multi-dimensional evaluation results includes:

[0132] Using natural language processing technology, semantically match the extracted keywords and key phrases with the knowledge graph, calculate the semantic similarity, and output the corresponding knowledge evaluation index score according to the semantic similarity;

[0133] Using natural language processing technology, based on the logical structure and key elements of the extracted skills assessment data, analyze the integrity, rationality and innovation of the skills assessment data, and output the corresponding skills evaluation index score according to the analysis results;

[0134] Using natural language processing technology, according to the logical structure, key elements and sentiment tendency of the extracted attitude assessment data, judge the teacher's attitude towards artificial intelligence ethics issues, and output the corresponding attitude evaluation index score according to the judgment results;

[0135] According to the preset weights, comprehensively combine the scores of the knowledge evaluation index, the skill evaluation index, and the attitude evaluation index, calculate the total multi-dimensional evaluation score, and output it as the multi-dimensional evaluation result.

[0136] In this embodiment, for knowledge evaluation tasks such as multiple-choice questions and essay questions, natural language processing technology uses semantic understanding algorithms to judge the accuracy of teachers' answers. Semantically match the teachers' answers with the standard answers or correct concepts in the knowledge graph. When answering the question "What are the main application fields of artificial intelligence?", the system will extract key information in the teachers' answers, such as domain words like "medical", "education", "transportation", etc., and compare them with the preset correct application fields. Using the cosine similarity algorithm or a semantic matching model based on deep learning, through in-depth analysis of the semantic information represented by keywords and key phrases and the nodes and edges in the knowledge graph, calculate the semantic similarity. If the similarity reaches the preset threshold, it indicates that the teacher has a correct understanding of this knowledge point and corresponding scores can be given.

[0137] The knowledge graph is formed by performing information extraction, knowledge fusion, graph construction, and graph quality evaluation and optimization on the data in the evaluation database.

[0138] The generation process of the knowledge graph includes: data collection, data screening and sorting, information extraction, knowledge fusion, knowledge graph construction, and knowledge graph quality evaluation and optimization.

[0139] Knowledge evaluation index:

[0140] Excellent (90 - 100 points): The teacher has a profound understanding of relevant knowledge, can accurately explain the basic concepts, technical principles of AI and its applications in education. Can clearly elaborate on the ethical issues, educational theories and technical applications of AI.

[0141] Good (70 - 89 points): The teacher has a relatively solid grasp of basic knowledge, can understand and apply the basic concepts and technologies of AI relatively accurately. Although there are a few inaccuracies, the overall understanding is good.

[0142] Qualified (50 - 69 points): The teacher has some understanding of the basic knowledge of AI, but there are certain understanding loopholes, and there may be deviations in the application of some core concepts and technologies.

[0143] Unqualified (0 - 49 points): The teacher's understanding of AI knowledge is relatively shallow, lacking a grasp of basic concepts or having serious errors in understanding.

[0144] In scenario tasks and design tasks of skills assessment tasks, teachers usually describe teaching plans in text. Natural language processing technology can parse this text and extract key elements of the teaching plan, such as teaching objectives, teaching methods, AI tools used, etc. The teaching design ability of teachers is evaluated by analyzing the integrity, rationality, and innovation of these elements. When analyzing the integrity of skills assessment data, check whether all key aspects that should be included in the skill description are covered, such as whether the objectives, operation steps, scope of application, etc. of the skill are clearly defined. By comparing with a pre-set standard template for skill integrity, judge the missing parts of the data and make corresponding scoring adjustments. For rationality analysis, based on industry common sense, practical experience, and relevant skill specifications, evaluate whether the design and implementation methods of the skill are reasonable, such as whether the difficulty of the skill matches the set level, and whether the operation process of the skill is efficient. Innovation analysis focuses on whether unique ideas, novel methods, or innovative improvements to traditional skills are demonstrated in the skills assessment data. By comprehensively considering the analysis results of these three aspects of integrity, rationality, and innovation, output the corresponding skill assessment index scores. If the data performs well in terms of integrity, rationality, and innovation, the skill assessment index scores will be higher; otherwise, the scores will be lower.

[0145] Skill assessment indicators:

[0146] Excellent (90 - 100 points): Teachers can proficiently use AI tools to design teaching plans, can innovatively integrate AI technology into teaching, and effectively solve practical problems in teaching. They have relatively high technical operation capabilities.

[0147] Good (70 - 89 points): Teachers can use AI tools to design basic teaching plans and effectively apply AI technology to the classroom. Although the plans may sometimes be relatively conventional, their overall operation capabilities are relatively strong.

[0148] Qualified (50 - 69 points): Teachers can basically use AI tools for teaching design, but lack the ability of innovation or flexibility, and the ways to solve practical problems are relatively simplified.

[0149] Unqualified (0 - 49 points): Teachers cannot effectively use AI technology to design teaching or solve problems, or there are major technical operation mistakes during the application process.

[0150] Regarding the attitude evaluation data, in texts such as teachers' discussion records and feedback questionnaires on topics related to AI ethics, through syntactic analysis and semantic understanding techniques, clarify the logical structure of the text, and identify the levels and contexts of teachers' expressed viewpoints. Extract key elements, such as specific aspects of AI ethics issues mentioned by teachers, the cases cited, and the elaboration of different viewpoints. At the same time, use sentiment analysis techniques, with the sentiment dictionary method and machine learning-based sentiment classification models, to judge whether the sentiment tendency expressed in the text is positive, negative, or neutral. Integrate this information to accurately judge the teachers' attitudes. If a teacher actively mentions the importance of AI ethics issues in a logically clear elaboration, proposes reasonable countermeasures, and has a positive sentiment tendency, then the score of their attitude evaluation index will be relatively high; conversely, if a teacher has a vague understanding of AI ethics issues, is logically confused, and has a negative sentiment tendency, then the score of the attitude evaluation index will be relatively low. Output the corresponding attitude evaluation index score according to the judgment result.

[0151] Attitude evaluation index:

[0152] Excellent (90 - 100 points): Teachers show a high degree of recognition and enthusiasm for AI, actively explore the ethical and educational applications of AI technology, have a self-driven learning attitude, and are able to reflect on the challenges and growth in their own use of AI.

[0153] Good (70 - 89 points): Teachers have a positive attitude towards AI, are willing to apply AI tools in teaching and continue to learn. Although they may be a bit hesitant or uncertain, overall they are willing to accept and develop relevant capabilities.

[0154] Qualified (50 - 69 points): Teachers' attitudes towards AI are relatively neutral. Although they are willing to try, they have certain doubts or resistance to the application of AI technology and lack sufficient learning motivation.

[0155] Unqualified (0 - 49 points): Teachers have a negative attitude towards AI, reject or highly distrust AI technology, and lack the motivation for reflection and learning.

[0156] Comprehensive scoring method, example scoring framework:

[0157] Knowledge level: 30% weight

[0158] Skill level: 40% weight

[0159] Attitude level: 30% weight

[0160] For example:

[0161] Score for knowledge level: 85 points (weight 30%) -> 85 * 0.3 = 25.5

[0162] Skill level score: 75 points (weight 40%) -> 75 * 0.4 = 30

[0163] Attitude level score: 90 points (weight 30%) -> 90 * 0.3 = 27

[0164] Total score = 25.5 + 30 + 27 = 82.5 points

[0165] In step S4, the process of generating the feedback report includes:

[0166] Obtain the scores of knowledge evaluation indicators, skill evaluation indicators, attitude evaluation indicators, and the total multi-dimensional evaluation score, determine the teacher's level in each dimension according to the preset scoring criteria, and convert it into a visual chart;

[0167] According to the teacher's level in each dimension, use generative technology to generate improvement suggestions;

[0168] Integrate the visual chart and the improvement suggestions as the feedback report for output.

[0169] The performance data, evaluation results, and feedback report need to be classified when stored in the evaluation database, and duplicate removal processing is performed on the performance data, evaluation results, and feedback report before storage.

[0170] Embodiment 2

[0171] Based on the above Embodiment 1, this embodiment discloses a dynamic evaluation system for teachers' artificial intelligence literacy based on generative technology. The system includes:

[0172] Input module 1, which obtains the teacher's background information;

[0173] Task generation module 2, which uses generative technology to dynamically generate multi-dimensional evaluation tasks according to the teacher's background information. The multi-dimensional evaluation tasks include case evaluation and question evaluation;

[0174] Performance evaluation module 3, which obtains the performance data of the teacher completing the multi-dimensional evaluation tasks, automatically analyzes the performance data based on natural language processing technology and multi-dimensional evaluation indicators, and automatically generates multi-dimensional evaluation results;

[0175] Feedback improvement module 4, which generates a feedback report according to the multi-dimensional evaluation results;

[0176] Data storage and management module 5, which stores, queries, and manages the teacher's background information, multi-dimensional evaluation tasks, performance data, multi-dimensional evaluation results, and feedback reports.

[0177] The specific details of the above modules can be understood by referring to the relevant descriptions and effects in Embodiment 1.

[0178] Embodiment 3

[0179] Based on Embodiment 1, this embodiment provides an electronic device, including: one or more processors and a memory. The memory stores one or more programs, and the one or more programs include instructions for executing the foregoing dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology.

[0180] At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, there may also be other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the foregoing dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology. Of course, in addition to the software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or a logic device.

[0181] The memory may include non-permanent memory in computer-readable media, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0182] Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0183] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology, characterized in that, The method includes: Obtaining teacher background information; Using generative technology to dynamically generate multi-dimensional assessment tasks according to the teacher background information, where the multi-dimensional assessment tasks include case assessment and question assessment; Obtaining the performance data of the teacher completing the multi-dimensional assessment tasks, automatically processing the performance data using natural language processing technology, and automatically generating multi-dimensional assessment results based on multi-dimensional evaluation indicators; Generating a feedback report according to the multi-dimensional assessment results, and storing the teacher background information, multi-dimensional assessment tasks, performance data, multi-dimensional assessment results and feedback report in the evaluation database.

2. The dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 1, wherein, The teacher background information includes school stage, subject background, teaching experience and artificial intelligence knowledge level.

3. The dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 2, characterized in that, The multi-dimensional assessment tasks include knowledge assessment, skill assessment and attitude assessment. The dynamic generation process of the multi-dimensional assessment tasks includes: Classifying teachers based on the subject background of the teachers according to the preset artificial intelligence requirements for subject teaching; Determining the initial weights of the knowledge assessment, skill assessment and attitude assessment according to the teacher's school stage and teaching experience, and adjusting the initial weights in combination with the artificial intelligence knowledge level to obtain the final weights, which are used to adjust the proportion of the generated assessment case quantity; Using generative technology to generate knowledge assessment, skill assessment and attitude assessment in the corresponding classification fields and quantities according to the classification of the artificial intelligence requirements for subject teaching of the teachers and the final weights, where The knowledge assessment includes multiple-choice questions and essay questions, the skill assessment includes scenario tasks and design tasks, and the attitude assessment includes case analysis and reflective tasks.

4. A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 1, characterized in that, The multi-dimensional assessment tasks are generated based on the five dimensions and three-layer assessment structure of the UNESCO framework. The five dimensions include people-centered thinking, artificial intelligence ethics, artificial intelligence foundation and application, artificial intelligence pedagogy, and artificial intelligence career development. The three-layer assessment structure includes knowledge, skills and attitudes.

5. The dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 1, characterized in that, The performance data includes knowledge assessment data, skill assessment data and attitude assessment data. The process of automatically processing the performance data includes: Using natural language processing technology to perform data cleaning, missing value processing, outlier detection, data normalization, word segmentation, part-of-speech tagging, named entity recognition, syntactic analysis and sentiment analysis on the performance data, and extracting the keywords and key phrases of the knowledge assessment data, the logical structure and key elements of the skill assessment data, and the logical structure, key elements and sentiment tendency of the attitude assessment data in the performance data.

6. The dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 5, characterized in that, The multi-dimensional evaluation indicators include knowledge evaluation indicators, skill evaluation indicators and attitude evaluation indicators. The process of automatically generating the multi-dimensional assessment results includes: Using natural language processing technology to perform semantic matching between the extracted keywords and key phrases and the knowledge graph, calculating the semantic similarity, and outputting the corresponding knowledge evaluation indicator scores according to the semantic similarity; Using natural language processing technology to analyze the integrity, reasonableness and innovation of the skill assessment data based on the logical structure and key elements of the extracted skill assessment data, and outputting the corresponding skill evaluation indicator scores according to the analysis results; Using natural language processing technology, based on the logical structure, key elements and sentiment tendency of the extracted attitude evaluation data, judge the teacher's attitude towards artificial intelligence ethics issues, and output the corresponding attitude evaluation index scores according to the judgment results; According to the preset weights, comprehensively calculate the multi-dimensional evaluation total score by combining the knowledge evaluation index score, skill evaluation index score and attitude evaluation index score, and output it as the multi-dimensional evaluation result.

7. A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 6, characterized in that, The knowledge graph is formed by performing information extraction, knowledge fusion, graph construction, and graph quality evaluation and optimization on the data in the evaluation database.

8. A dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 6, characterized in that The process of generating the feedback report includes: Obtain the knowledge evaluation index score, skill evaluation index score, attitude evaluation index score and multi-dimensional evaluation total score, determine the teacher's level in each dimension according to the preset scoring criteria, and convert it into a visual chart; Generate improvement suggestions using generative technology according to the teacher's level in each dimension; Integrate the visual chart and the improvement suggestions and output them as a feedback report.

9. The dynamic evaluation method for teachers' artificial intelligence literacy based on generative technology according to claim 1, characterized in that The performance data, evaluation results and feedback reports are classified and stored in the evaluation database, and the performance data, evaluation results and feedback reports are deduplicated before storage.

10. A dynamic evaluation system for teachers' artificial intelligence literacy based on generative technology, characterized in that, The system includes: An input module to obtain the teacher's background information; A task generation module that uses generative technology to dynamically generate multi-dimensional evaluation tasks according to the teacher's background information. The multi-dimensional evaluation tasks include case evaluation and question evaluation; A performance evaluation module that obtains the performance data of the teacher completing the multi-dimensional evaluation tasks, automatically analyzes the performance data based on natural language processing technology and multi-dimensional evaluation indicators, and automatically generates multi-dimensional evaluation results; A feedback improvement module that generates a feedback report according to the multi-dimensional evaluation results; A data storage and management module that stores, queries and manages the teacher's background information, multi-dimensional evaluation tasks, performance data, multi-dimensional evaluation results and feedback reports.

Images