Teaching quality evaluation method for AI-assisted college teacher teaching

Abstract

This invention relates to the field of AI-assisted teaching technology, specifically disclosing a teaching quality evaluation method for AI-assisted teaching by university teachers. The method includes the following steps: obtaining the knowledge point coverage rate of courseware, and obtaining prompt words and corresponding response content for each input; segmenting the prompt words to obtain sub-prompt words, and obtaining target content based on the sub-prompt words; determining candidate groups based on the similarity between the response content and the target content; obtaining reduced words based on the number of sub-prompt words in the candidate groups, and calculating the redundancy ratio; calculating the sum of the redundancy ratios, and weighted summing the sum with the knowledge point coverage rate to obtain the evaluation score. This invention can objectively evaluate the AI-assisted teaching ability of university teachers.

Description

Technical Field

[0001] This invention relates to the field of AI-assisted teaching technology, specifically to a teaching quality evaluation method for AI-assisted teaching by university teachers. Background Technology

[0002] In higher education, AI technology is becoming a powerful tool for teachers, providing multi-dimensional support for instruction. For example, with the help of AI-powered courseware generation tools, teachers can quickly integrate diverse materials such as text, images, audio, and video, automatically formatting them into clearly structured multimedia courseware.

[0003] As the core carrier of knowledge transmission in higher education, the quality of courseware largely determines whether students can accurately understand the course content and directly impacts the overall teaching effectiveness. The application of AI-powered courseware generation tools has demonstrated significant advantages in improving courseware production efficiency and multimedia integration capabilities; however, its effectiveness still depends on the teacher's skill level and professional judgment.

[0004] When different teachers use the same AI tools, varying levels of proficiency often lead to different results: some teachers can quickly and accurately create ideal courseware, while others may spend more time but produce less satisfactory results. This disparity reveals a prominent problem in current teaching practice: the lack of a scientific and objective evaluation system for teachers' AI-assisted teaching abilities. The absence of this system not only makes it difficult for teachers to obtain effective improvement guidance but also, to some extent, limits the in-depth application and optimization of AI tools in higher education. Summary of the Invention

[0005] The purpose of this invention is to provide a teaching quality evaluation method for AI-assisted teaching by university teachers, and to solve the following technical problems:

[0006] There is a lack of a scientific and objective evaluation system for teachers' AI-assisted teaching abilities in the current technology.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] A teaching quality evaluation method for AI-assisted teaching by university teachers includes the following steps:

[0009] The system retrieves courseware generated by university teachers using AI tools, assesses the knowledge point coverage of the courseware, and retrieves the prompts and corresponding responses that university teachers input each time they generate courseware using AI tools.

[0010] The input prompt word Ci is segmented to obtain several sub-prompt words. The sub-prompt words are grouped, and different groups contain at least one different sub-prompt word. The sub-prompt words in the same group are combined into a target word. The target word is input into the AI ​​tool to obtain the AI ​​tool's response content, which is denoted as the target content. i is a positive integer and i∈[1,n], where n represents the total number of times the university teacher inputs prompt words when generating courseware through the AI ​​tool.

[0011] Get the response content Ci' corresponding to the prompt word Ci, get the similarity between the response content Ci' and the target content, and select the groups with the similarity greater than a preset threshold as candidate groups;

[0012] Get the number of sub-prompt words in the candidate group, and record it as the selection number. Take the target word of the candidate group with the smallest selection number as the reduced word. Calculate the redundancy ratio D=D1 / D2, where D1 and D2 represent the number of sub-reduced words and sub-prompt words, respectively.

[0013] The redundancy ratios of the prompts from the first to the nth input are summed to obtain the total Dtot. The total redundancy ratio Dtot is then weighted and summed with the knowledge point coverage rate to obtain the evaluation score of the college teacher's AI-assisted teaching ability.

[0014] As a further aspect of the present invention: obtaining the knowledge point coverage of the courseware includes:

[0015] Establish a knowledge point standard library to store the knowledge points of the course, and obtain the total number of knowledge points S1 in the knowledge point standard library;

[0016] Text is extracted from courseware using OCR, keywords are extracted from the text, and the number of keywords that are knowledge points is obtained as S2.

[0017] The knowledge point coverage rate is S2 divided by S1.

[0018] As a further aspect of the present invention, the process of obtaining the prompt words also includes:

[0019] Obtain the semantic vectors of the prompt words, sort the semantic vectors according to the timeline, and obtain the first sort;

[0020] Obtain the difference position k in the first sort, and use the semantic vector from the first position to the difference position k in the first sort as the merge vector;

[0021] The prompt words corresponding to the merged vectors are merged into a new prompt word A, and the response content corresponding to the semantic vector at the difference position k is used as the response content of prompt word A. Prompt word A is used as the prompt word for the kth input.

[0022] Remove the merged vector from the first sort to obtain a new first sort. Repeat the above steps to obtain new difference positions, and obtain new prompt words and corresponding reply content based on the new difference positions, until there are no difference positions in a new first sort.

[0023] As a further aspect of the present invention: the difference position k satisfies the following constraints:

[0024] P 1，j ≥Pys and P 1，k+1 <Pys, j∈[2,k], P 1，j P represents the cosine of the angle between the semantic vector of the first position in the first sort and the semantic vector of the j-th position in the first sort. 1，k+1 Pys represents the cosine of the angle between the semantic vector at the first position in the first sort and the semantic vector at the (k+1)th position in the first sort. Pys represents the preset threshold value for the cosine of the angle.

[0025] As a further aspect of the present invention: if P 1，2 If <Pys, then the semantic vector of the first position in the first sort will be used as the merge vector.

[0026] As a further aspect of the present invention: after obtaining the evaluation score of university teachers' AI-assisted teaching ability, the method further includes:

[0027] If the evaluation score is lower than the preset evaluation score threshold, the prompt words and corresponding reduced words will be displayed to the user visually.

[0028] As a further aspect of the present invention: during the process of grouping sub-prompt words, groups with fewer than a preset number of sub-prompt words are removed.

[0029] The beneficial effects of this invention compared to the prior art are as follows:

[0030] 1. This invention can simultaneously consider the knowledge point coverage rate and the redundancy in the input of prompt words in the teaching quality evaluation, thereby achieving a comprehensive evaluation of teachers' AI-assisted teaching ability. By weighting and combining the results of knowledge point coverage rate and redundancy ratio, it can effectively reflect the breadth of teachers' knowledge points and the effectiveness of their input prompt words in the courseware generation process, avoiding the distortion of evaluation results caused by a single indicator. This reflects the rationality of teachers' input and the richness of the generated results in the process of teachers using AI to generate courseware, making the final evaluation results more objective and comprehensive, and providing a reliable basis for improving teaching quality.

[0031] 2. In the processing of prompt words, this invention merges consecutive prompt word inputs within the same generation step (e.g., when the user wants the AI ​​tool to provide a table) into a new prompt word based on semantic vector sorting and difference position determination, and uses the response corresponding to the difference position as the response content of the prompt word; when entering the next generation step, merging stops due to differences in semantic vectors; it is understood that the user may only make minor adjustments to the previous version of the prompt word in the same step, and the evaluation can be based on the last response of the step, thereby avoiding the inclusion of multiple minor adjustments in the evaluation of the same step separately, and ensuring that the evaluation result can reflect the final situation of the step in a concentrated manner. Attached Figure Description

[0032] The invention will now be further described with reference to the accompanying drawings.

[0033] Figure 1 This is a flowchart illustrating the teaching quality evaluation method for AI-assisted teaching by university teachers, as described in this invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 As shown, this invention is a teaching quality evaluation method for AI-assisted teaching by university teachers, comprising the following steps:

[0036] The system retrieves courseware generated by university teachers using AI tools, assesses the knowledge point coverage of the courseware, and retrieves the prompts and corresponding responses that university teachers input each time they generate courseware using AI tools.

[0037] In a specific embodiment of the present invention, firstly, with the authorization of the university teacher, the content of the courseware generated by the university teacher using artificial intelligence tools is obtained; the courseware is an important carrier for teachers to present knowledge in the process of lesson preparation and teaching, and usually contains text, charts, pictures and various teaching materials. In order to ensure the objectivity and completeness of the evaluation, it is necessary to extract information from the courseware finally generated by the teacher, identify the scope and coverage of knowledge involved, and determine the knowledge points contained in the courseware accordingly.

[0038] At the same time, it is also necessary to obtain all the prompt words input by teachers during their interaction with the AI ​​tool; prompt words are the core instructions for teachers to communicate with AI. Teachers express the content and form to be generated by inputting prompt words, and the AI ​​tool will generate corresponding responses or results based on these prompt words; record the content of each prompt word input by the teacher, and simultaneously save the response results generated by the AI ​​tool for that prompt word;

[0039] This process encompasses the teacher's initial input, as well as the teacher's revised input when dissatisfied with the generated results, and the continuous responses generated by the AI ​​tool based on the modification prompts. By fully acquiring the courseware content, knowledge point coverage, and interaction information between the prompts and responses, it can provide a comprehensive data foundation for subsequent analysis and evaluation.

[0040] In a preferred embodiment of the present invention, obtaining the knowledge point coverage of courseware includes:

[0041] First, a knowledge point standard library is established to store all the knowledge points involved in the course. The so-called knowledge point standard library is to organize and summarize the important knowledge content specified in the course syllabus, textbook content and teaching objectives. For example, the knowledge points of a database course may include relational model, normalization, SQL statement, transaction management and other content. These knowledge points are stored in the standard library in a structured way as a benchmark for comparison.

[0042] The courseware generated by teachers using artificial intelligence tools is processed, and optical character recognition technology is used to extract the text information in the courseware. The principle of optical character recognition is to analyze the image features in the courseware page, identify the text area and convert it into processable text. In this way, whether the courseware exists in the form of scanned images or in the form of slides containing text, the text content can be extracted.

[0043] After obtaining the text, it is necessary to further identify which words are associated with the course knowledge points. This can usually be achieved through keyword extraction. For example, in a courseware about databases, if words such as "transaction," "index," and "view" appear in the text, it can be determined that these words match the corresponding entries in the knowledge point standard library, thus confirming them as knowledge point keywords, and thereby obtaining the actual number of knowledge points covered in the courseware.

[0044] By comparing the number of knowledge points actually appearing in the courseware with the total number of knowledge points in the standard library, we can obtain the knowledge point coverage rate of the courseware.

[0045] In another preferred embodiment of the present invention, the process of obtaining the prompt words further includes:

[0046] In the process of obtaining prompt words, it is also necessary to further process the semantic vectors of the prompt words. The semantic vector is to transform the textual information of the prompt words into a form that can express their semantic features. This allows for comparison of the closeness between different prompt words in mathematical space. For example, if a teacher inputs the prompt word "generate a table" in one step, and then changes it to "generate a three-column table" in the next step, the distance between the semantic vectors of these two prompt words will be relatively close, because they belong to the same type of instruction semantically, with only minor adjustments in details.

[0047] After obtaining these semantic vectors, they are sorted according to the timeline to obtain the first sort. This preserves the order in which the prompt words appear and ensures that subsequent processing follows the logical flow of the teacher's input.

[0048] Obtain the difference position k in the first sort. The difference position refers to finding a continuous portion in the semantic vectors arranged chronologically that maintains a high degree of similarity to the initial semantic vector, and identifying the first position that no longer meets the similarity requirement. It's important to note that only when a difference occurs does it indicate that the teacher has entered a new task or made significant modifications to the original task. For example, from "generating a three-column table and adding titles" to "generating a flowchart," there will be a significant difference between the semantic vectors; this position can be used as the difference position k.

[0049] After identifying the discrepancy position, the semantic vector from the first position in the first sort to the discrepancy position is treated as a whole, called the merge vector. The prompt words corresponding to the merge vector are merged into a new prompt word A, and the response content corresponding to the discrepancy position k is used as the response content of prompt word A. For example, in the process of creating a table, if the teacher first enters "generate table" and then enters "generate three-column table", these two inputs will be merged to obtain a new prompt word A. The merging method is to concatenate them according to the timeline order, that is, in the example above, prompt word A is "generate table, generate three-column table". The response generated by the artificial intelligence at the discrepancy position is used as the corresponding content of A. At the same time, prompt word A is used as the prompt word for the kth input, and the new prompt word obtained later is used as the prompt word for the Kth input. K represents the sorting position of the new discrepancy position in the original first sort.

[0050] Remove the merged vectors from the first sort to obtain a new first sort. Then repeat the above steps to find new differences and merge them again to generate new prompts and corresponding replies. Repeat this process until there are no more differences in the new first sort, that is, the cosine of the angle between the semantic vectors at subsequent sort positions and the semantic vector at the first position in a new first sort is greater than or equal to Pys.

[0051] In this process, each merge can group multiple similar prompts into one category, and use the response corresponding to the last difference as a representative, thus recording the teacher's continuous fine-tuning under the same task as a correspondence between prompt input and response. When the teacher starts a new task, a new correspondence between prompts and responses is naturally formed.

[0052] In a preferred embodiment, the difference position k satisfies the following constraint:

[0053] P 1，j ≥Pys and P 1，k+1 <Pys, j∈[2,k], P 1，j P represents the cosine of the angle between the semantic vector of the first position in the first sort and the semantic vector of the j-th position in the first sort. 1，k+1 Pys represents the cosine of the angle between the semantic vector at the first position in the first sort and the semantic vector at the (k+1)th position in the first sort.

[0054] It should be noted that if P 1，2 If <Pys, then the semantic vector of the first position in the first sort will be used as the merge vector;

[0055] The input prompt word Ci is segmented to obtain several sub-prompt words. The sub-prompt words are grouped, and different groups contain at least one different sub-prompt word. The sub-prompt words in the same group are combined into a target word. The target word is input into the AI ​​tool to obtain the AI ​​tool's response content, which is denoted as the target content. i is a positive integer and i∈[1,n], where n represents the total number of times the university teacher inputs prompt words when generating courseware through the AI ​​tool.

[0056] It should be noted that during the process of grouping sub-suggestion words, groups with fewer than the preset number of sub-suggestion words are removed;

[0057] It is important to note that after segmenting and grouping the i-th prompt word to obtain the target word, the previous dialogue content should remain unchanged when inputting the target word into the AI ​​tool. This is because the AI ​​tool usually combines contextual reasoning when generating the response content. If the previous dialogue changes, the generated content may deviate from the teacher's original intention.

[0058] Get the response content Ci' corresponding to the prompt word Ci, get the similarity between the response content Ci' and the target content, and select the groups with the similarity greater than a preset threshold as candidate groups;

[0059] Get the number of sub-prompt words in the candidate group, and record it as the selection number. Take the target word of the candidate group with the smallest selection number as the reduced word. Calculate the redundancy ratio D=D1 / D2, where D1 and D2 represent the number of sub-reduced words and sub-prompt words, respectively.

[0060] The redundancy ratios of the prompts from the first to the nth input are summed to obtain the total Dtot. The total redundancy ratios Dtot are then weighted and summed with the knowledge point coverage rate to obtain the evaluation score of the AI-assisted teaching ability of college teachers.

[0061] After obtaining the evaluation scores for university teachers' AI-assisted teaching abilities, the following are also included:

[0062] If the evaluation score is lower than the preset evaluation score threshold, the prompt words and corresponding reduced words will be displayed to the user visually.

[0063] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the present invention should still fall within the scope of the present invention.

Claims

1. A method for evaluating teaching quality for AI-assisted high school teachers, characterized in that, Includes the following steps: The system retrieves courseware generated by university teachers using AI tools, assesses the knowledge point coverage of the courseware, and retrieves the prompts and corresponding responses that university teachers input each time they generate courseware using AI tools. The input prompt word Ci is segmented to obtain several sub-prompt words. The sub-prompt words are grouped, and different groups contain at least one different sub-prompt word. The sub-prompt words in the same group are combined into a target word. The target word is input into the AI ​​tool to obtain the AI ​​tool's response content, which is denoted as the target content. i is a positive integer and i∈[1,n], where n represents the total number of times the university teacher inputs prompt words when generating courseware through the AI ​​tool. Get the response content Ci' corresponding to the prompt word Ci, get the similarity between the response content Ci' and the target content, and select the groups with the similarity greater than a preset threshold as candidate groups; Get the number of sub-prompt words in the candidate group, and record it as the selection number. Take the target word of the candidate group with the smallest selection number as the reduced word. Calculate the redundancy ratio D=D1 / D2, where D1 and D2 represent the number of reduced words and sub-prompt words, respectively. The redundancy ratios of the prompts from the first to the nth input are summed to obtain the total Dtot. The total Dtot is then weighted and summed with the knowledge point coverage rate to obtain the evaluation score of the college teacher's AI-assisted teaching ability. 2.The teaching quality evaluation method for AI-assisted high school teacher teaching of claim 1, wherein, The knowledge point coverage of the courseware includes: Establish a knowledge point standard library to store the knowledge points of the course, and obtain the total number of knowledge points S1 in the knowledge point standard library; Text is extracted from courseware using OCR, keywords are extracted from the text, and the number of keywords that are knowledge points is obtained as S2. The knowledge point coverage rate is S2 divided by S1. 3.The teaching quality evaluation method for AI-assisted high school teacher teaching of claim 1, wherein, The process of obtaining prompt words also includes: Obtain the semantic vectors of the prompt words, sort the semantic vectors according to the timeline, and obtain the first sort; Obtain the difference position k in the first sort, and use the semantic vector from the first position to the difference position k in the first sort as the merge vector; The prompt words corresponding to the merged vectors are merged into a new prompt word A, and the response content corresponding to the semantic vector at the difference position k is used as the response content of prompt word A. Prompt word A is used as the prompt word for the Yth input, where Y∈[1,k]; Remove the merged vector from the first sort to obtain a new first sort. Repeat the above steps to obtain new difference positions, and obtain new prompt words and corresponding reply content based on the new difference positions, until there are no difference positions in a new first sort. The difference position k satisfies the following constraints: P 1，j ≥Pys and P 1，k+1 <Pys, j∈[2,k], P 1，j P represents the cosine of the angle between the semantic vector of the first position in the first sort and the semantic vector of the j-th position in the first sort. 1，k+1 Pys represents the cosine of the angle between the semantic vector at the first position in the first sort and the semantic vector at the (k+1)th position in the first sort. Pys represents the preset threshold value for the cosine of the angle.

4. The teaching quality evaluation method for AI-assisted teaching by university teachers according to claim 3, characterized in that, If P 1，2 If Pys, the first semantic vector in the first ranking is taken as the merged vector. 5.The teaching quality evaluation method for AI-assisted high school teacher teaching of claim 1, wherein, After obtaining the evaluation scores for university teachers' AI-assisted teaching abilities, the following are also included: If the evaluation score is lower than the preset evaluation score threshold, the prompt words and corresponding reduced words will be displayed to the user visually. 6.The teaching quality evaluation method for AI-assisted high school teacher teaching of claim 1, wherein, During the process of grouping sub-prompt words, groups with fewer than the preset number of sub-prompt words are removed.

Images