A learning effect evaluation system and method for an artificial intelligence image generation training course

Abstract

The application discloses a learning effect evaluation system and method for an artificial intelligence image generation training course, and relates to the technical field of computers. In order to solve the technical problem that a learning management system cannot timely feedback learning conditions of students in the prior art, the technical scheme provided by the application is as follows: a plurality of modules for student learning process data acquisition and analysis are included, and the system comprises: a student single course homework quality evaluation module, which is used for performing quality evaluation on single homework of a student in an artificial intelligence image generation course, and obtaining a quality score; an overall learning ability evaluation module, which performs comprehensive learning state analysis on quality scores of multiple homework of the student and homework process logs, and obtains an overall learning ability score of the student; and a visual feedback module, which is used for generating a student learning diary according to changes in homework quality and learning ability of the student. The application is suitable for being applied to learning effect evaluation work for the artificial intelligence image generation course.

Description

Technical Field

[0001] This relates to the field of computer technology, specifically to a learning effectiveness evaluation system and method for artificial intelligence image generation training courses. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, especially breakthroughs in the field of image generation, the education industry has also ushered in a new transformation. Traditional teaching assessment methods are gradually being replaced by more intelligent and dynamic learning outcome evaluation systems, especially in the teaching of AI image generation courses, where this shift is particularly evident.

[0003] Existing learning management systems (LMS) are already able to record various event logs of students during the learning process, including learning time, task completion status, and interaction frequency. This data provides a foundation for evaluating students' learning outcomes.

[0004] However, existing learning outcome evaluation systems still have some shortcomings.

[0005] Current learning management systems (LMS) can record event logs of students' learning processes, including learning time, task completion status, and interaction frequency. While this data provides a foundation for evaluating the effectiveness of learning records, existing evaluation systems have shortcomings in the following aspects: First, they lack precise assessment of individual student differences, making it difficult to achieve personalized learning outcome evaluation; second, existing systems are mainly static evaluation methods, lacking real-time capabilities, resulting in the inability to provide timely feedback on student learning progress; and third, for AI image generation courses, existing evaluation methods struggle to incorporate students' creativity and technical mastery during the creative process. Although some research has attempted to combine formative assessment with comprehensive ability assessment to improve the accuracy and timeliness of learning evaluation, its application in AI image generation courses remains scarce. Therefore, developing a learning outcome evaluation system that enables personalized student learning assessment and possesses dynamic and personalized functions is both urgent and pressing. Summary of the Invention

[0006] To address the technical problems of existing learning management systems lacking accurate assessment of individual student differences, making it difficult to achieve personalized learning outcome evaluation; and existing systems primarily using static evaluation methods, lacking real-time capabilities and resulting in the inability to provide timely feedback on student learning progress, the technical solution provided by this invention is as follows:

[0007] A learning effectiveness evaluation system for artificial intelligence image generation training courses includes multiple modules for data collection and analysis of student learning processes.

[0008] The system includes:

[0009] The Student Single Course Assignment Quality Evaluation Module is used to evaluate the quality of students' single assignments in the Artificial Intelligence Image Generation course and obtain a quality score.

[0010] The overall learning ability assessment module conducts a comprehensive analysis of students' learning status based on the quality scores of multiple assignments and assignment process logs, and obtains the students' overall learning ability score.

[0011] The visual feedback module is used to generate student learning diaries based on changes in the quality of students' assignments and their learning abilities.

[0012] Furthermore, a preferred embodiment is provided: a student single course assignment quality evaluation module, based on a feature extraction method of a pre-trained deep model, calculates the similarity by using the feature allocation distance between the target image and the single assignment generated image, and obtains a quality score.

[0013] Furthermore, a preferred embodiment is provided in which the feature assignment distance is achieved through the Frechette revelation distance.

[0014] Furthermore, a preferred implementation method is provided: an overall learning ability evaluation module that uses a decision tree model constructed based on the gradient boosting tree algorithm to evaluate the student's learning status at each stage.

[0015] Furthermore, a preferred implementation is provided in which the overall learning ability evaluation module accumulates the scores of students' learning status at different learning stages through an exponential weighted accumulation method, thereby obtaining the students' overall learning ability score.

[0016] Furthermore, a preferred implementation is provided: a visual feedback module that provides personalized visual feedback on students' learning diaries and offers it to students and teachers through a front-end display page of the system.

[0017] Based on the same inventive concept, this invention also provides a method for evaluating the learning effectiveness of training courses for artificial intelligence image generation.

[0018] The method is implemented based on the system described above, and includes:

[0019] The steps for evaluating the quality of a student's single course assignment are as follows: to evaluate the quality of a student's single assignment in the Artificial Intelligence Image Generation course and obtain a quality score;

[0020] The overall learning ability assessment module conducts a comprehensive analysis of students' learning status based on the quality scores of multiple assignments and assignment process logs, and obtains the students' overall learning ability score.

[0021] The visual feedback module generates student learning diaries based on changes in the quality of students' assignments and their learning abilities.

[0022] Based on the same inventive concept, the present invention also provides a computer storage medium for storing a computing program, characterized in that when the computer reads the computer program, the computer executes the method described thereon.

[0023] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium, characterized in that when the processor reads a computer program stored in the storage medium, the computer executes the method described thereon.

[0024] Based on the same inventive concept, the present invention also provides a computer program product, which, as a computer program, is characterized in that, when the computer program is executed, it implements the method described.

[0025] Compared with the prior art, the advantages of the technical solution provided by the present invention are as follows:

[0026] By using image set similarity estimation to automatically evaluate students' single image generation assignments, the problems of inconsistency and lack of emphasis in manual grading are solved, grading efficiency is improved, teachers' workload is reduced, and the fairness and consistency of the evaluation are ensured. This automated evaluation capability enables students to receive scientific and fair quality evaluation at every step of the learning process, thus replacing existing manual grading methods and having higher reliability and traceability.

[0027] By constructing a decision tree model using the gradient boosting tree algorithm, a comprehensive analysis of students' learning status can be achieved based on their assignment quality scores and assignment process logs, thus providing a holistic evaluation of their learning progress. Compared to existing single assignment quality scoring systems, this model offers a comprehensive assessment of students' overall learning abilities, helping teachers and students identify weaknesses in the learning process and adjust teaching strategies accordingly.

[0028] An exponential weighting method is used to accumulate learning status according to different learning stages, which pays more attention to the changes in students' learning status in the later stages of the course and evaluates the overall learning ability.

[0029] Compared with existing static evaluation systems, this progressive learning ability evaluation method can more accurately reflect students' learning progress and problem-solving abilities, and helps to better understand students' ability improvement throughout the learning process, especially their performance in the later stages of learning.

[0030] The system is designed with a personalized, visual feedback mechanism, providing customized feedback based on students' learning progress. Compared to traditional "one-size-fits-all" evaluation methods, it pays more attention to students' unique needs. Through this personalized display function, students can receive initial suggestions for improving their learning initiative, enhancing their engagement and motivation, and improving learning outcomes. Furthermore, teachers can gain a deeper understanding of students' learning status through visual feedback, thereby enabling them to make appropriate adjustments to their teaching.

[0031] It is suitable for use in evaluating the learning outcomes of AI image generation courses. Attached Figure Description

[0032] Figure 1 This is a system composition framework diagram of the present invention.

[0033] Figure 2 This is a flowchart of the single-course assignment quality evaluation module of the present invention.

[0034] Figure 3 This is a flowchart of the student overall learning ability evaluation module of the present invention.

[0035] Figure 4 This is a flowchart of the FID distance extreme value estimation process for the single course assignment quality evaluation module of this invention.

[0036] Figure 5 Example of generating an image. Detailed Implementation

[0037] To make the advantages and benefits of the technical solution provided by the present invention clearer, the technical solution provided by the present invention will now be described in further detail with reference to the accompanying drawings, specifically:

[0038] Currently, although some platforms and companies have begun to explore combining process-based assessment, comprehensive ability assessment, and dynamic assessment to improve the accuracy and timeliness of teaching feedback, specific application cases for AI image generation courses are still limited. This presents teachers with certain challenges in evaluating student learning outcomes, especially in how to quantitatively assess students' creative abilities, technical mastery, and understanding of course content.

[0039] Therefore, developing a learning effectiveness evaluation system and method for AI-based image generation courses is particularly necessary. This system should integrate multiple data sources and utilize advanced machine learning and data mining techniques to monitor and analyze students' learning processes in real time, enabling a comprehensive assessment of students' abilities. Furthermore, the system should have the function of personalized learning path recommendations, dynamically adjusting teaching strategies and content based on students' learning progress to improve educational outcomes.

[0040] In conclusion, given the rapid development of the field of artificial intelligence image generation, establishing an efficient and intelligent learning outcome evaluation system will provide strong support for teachers, helping them to better understand and guide students' learning process, and ultimately achieve further optimization and improvement of teaching.

[0041] Implementation Method 1: The system provided in this implementation method, in evaluating the learning outcomes of artificial intelligence image generation courses, includes the following specific steps:

[0042] Step 1: Initialize the student single course assignment quality evaluation module

[0043] Feature extraction is performed on the image generation assignments submitted by students using a pre-trained deep model, and the similarity between image sets is calculated.

[0044] Detailed Description: First, the system receives AI-generated images submitted by students and extracts the generated image set and corresponding target image set for each assignment. Then, based on a pre-trained deep model (e.g., the InceptionV3 model), the system performs high-dimensional visual feature extraction on these images to generate a feature set. The system then calculates the feature distribution distance between the target image set and the generated image set using the Frechette cue distance (FID), and standardizes the distance to obtain a similarity score (between 0 and 1). The similarity score is the quality score for a single assignment.

[0045] Step 2: Construction of the Overall Learning Ability Assessment Module

[0046] Based on student assignment scores and assignment logs, a decision tree model is constructed using the gradient boosting tree algorithm to progressively assess students' learning status.

[0047] Detailed Description: The system inputs the quality score of each assignment and relevant log data (such as the number of times the assignment was completed, the assignment time, and the task completion time) into the decision tree model. Based on this data, a learning status evaluation decision tree model is constructed using the gradient boosting tree algorithm to quantify the student's learning status. The learning status is derived through a comprehensive analysis of assignment completion, efficiency, and quality scores. The model uses an exponential weighted summation method to calculate the student's overall learning ability score at different learning stages, in order to better reflect the student's progress and ability improvement during the learning process.

[0048] Step 3: The personalized visual feedback module generates feedback information.

[0049] Generate personalized feedback based on changes in students' learning quality and abilities, and display it to students and teachers.

[0050] Detailed Description: Based on students' academic performance, the system extracts the quality score and learning ability score for each assignment, generating corresponding visual data (such as line charts). This visual feedback primarily includes trends in student assignment quality scores and dynamic changes in learning status scores, helping students and teachers clearly understand students' learning progress. The system presents these visualizations to students and teachers upfront, enabling dynamic tracking of the learning process and personalized teaching adjustment suggestions.

[0051] Step 4: Personalized Learning Path Recommendation

[0052] Teaching strategies and learning content are dynamically adjusted based on students' learning performance, and personalized learning path suggestions are provided.

[0053] Detailed Description: Based on students' overall learning ability scores and assignment quality scores, the system can personalize teaching strategies, such as recommending suitable learning materials, adjusting learning difficulty, or providing additional exercises. The recommendation process uses an adaptive recommendation algorithm, combining students' historical learning data and current learning performance to generate personalized learning paths that better meet students' learning needs.

[0054] Implementation Method Two: Combination Figure 1-4 This embodiment describes the technical solution provided above in further detail, specifically:

[0055] A learning effectiveness evaluation system and method for AI image generation courses includes: a module for evaluating the quality of individual student assignments, a module for evaluating overall learning ability, and a visualization feedback module. The method automatically evaluates the quality of individual student assignments based on their answering behavior in target image generation assignments for AI image generation courses, using image set similarity estimation. Based on the quality scores of multiple student assignments and assignment process logs, a learning state evaluation decision tree model is constructed using a gradient boosting tree algorithm. This model performs an overall learning ability evaluation based on the progressive accumulation of learning states at each stage, providing feedback to students on the overall learning effectiveness of the course and enabling personalized presentation. This implementation can automatically provide a quantitative and fair evaluation of students' problem-solving ability and overall learning ability based on their answering behavior. It systematically provides a solution for the necessary assignment evaluation and teaching effectiveness evaluation process in AI image generation courses, enabling faster and more accurate feedback on student learning outcomes, improving teaching quality, and facilitating closed-loop teaching lifecycle management.

[0056] like Figure 1 This implementation includes a module for evaluating the quality of individual student assignments, a module for evaluating overall learning ability, and a module for visual feedback.

[0057] Module 1: Student Single Course Assignment Quality Evaluation Module

[0058] Extract AI-generated images submitted by students for a single assignment, use image set similarity estimation method, and calculate the similarity between the submitted generated image set and the real image set of the assignment target based on the Frechette revelation distance. Evaluate the quality of the AIGC image generation assignment completed by students based on the similarity between the submitted assignment and the real images.

[0059] like Figure 2 The student single course assignment quality evaluation module includes the following steps:

[0060] (1-1) For single-course assignments, the teacher or teaching platform system will assign them based on teaching experience and needs. These assignments include assignment number i and one or more target images, forming a set of real target images. Where n i Let be the number of target images for the i-th assignment. The assignment requires students to use AIGC technology to generate a set of images matching the target images. Images that are as similar as possible. After a student submits their assignment, a set of AIGC-generated images from their i-th assignment submission is extracted to form the generated image set. Where m i This represents the number of images submitted by students for the i-th assignment.

[0061] (1-2) Use a feature extraction method based on a pre-trained deep model to process the target real image set. Image sets generated with assignments From all images, obtain the feature set of the target real image. Image feature set generated by the assignment

[0062] (1-3) Calculate the feature distribution distance FID(r,g) between the target real image set and the task-generated image set.

[0063] (1-4) The distribution distance is standardized based on the maximum and minimum values ​​to obtain a similarity score in the range of 0-1, which is used as the evaluation of the job quality.

[0064]

[0065] FID o The feature distribution distance calculated in (1-3), FID max FID min The maximum and minimum values ​​of the estimated real and generated image FID distances are given.

[0066] (1-5) Record the highest quality score, number of submissions, time from assignment publication to first successful submission, and submission time for each student's assignment, and store these records in the system's assignment process log database. Specifically, an assignment is considered successful if its quality score is greater than 0.7.

[0067] The first step (1-2) of the module is based on the feature extraction method of the pre-trained deep model. The deep model is pre-trained on a massive dataset, the image is input into the pre-trained model, and average pooling is performed on the output of the deepest layer to extract the high-dimensional visual features of the image.

[0068] The feature distribution distance calculation step (1-3) of module one includes using the mean and covariance matrices of the real and generated image sets to approximate the Fréchet Inception Distance (FID) between the two distributions, that is:

[0069]

[0070] Where μ r μ g Let μ be the mean of the feature distributions of the real image and the generated image. r μ g Let be the covariance of the two characteristic distributions.

[0071] The standardization steps for the maximum and minimum values ​​of the feature distribution distance in module one (1-4) are characterized in that the maximum and minimum values ​​of the FID distance are obtained through statistical estimation by conducting multi-model generated image sampling experiments. The experiment design is based on different positive and negative prompt words and different AIGC generation models to generate image groups with different similarities. The similarity of the samples within and between groups is calculated, and the extreme values ​​are statistically analyzed. Specifically, multiple images that are similar within groups and very dissimilar between groups are generated using different AIGC generation models as samples. First, the FID extreme values ​​are estimated by randomly grouping within each sample group. Then, the multi-model sample groups are mixed into a whole and randomly grouped again to estimate the FID extreme values. The maximum and minimum values ​​of all FID extreme values ​​are taken to normalize the FID distance between the target assignment image and the student-generated image distribution to obtain the assignment score.

[0072] Module Two: Overall Learning Ability Assessment Module

[0073] The system extracts the quality scores of students' completed assignments generated in Module 1, and obtains statistical data on the assignment process, such as the number of assignment submissions, assignment time, and assignment scores, from the answer behavior log. It then uses the gradient boosting tree algorithm to construct a decision tree model for evaluating the overall learning situation, builds simulation data to train the model, and applies the decision results of the trained model to generate an evaluation of the students' overall learning ability based on the learning status at each stage.

[0074] like Figure 3 The overall learning evaluation module includes the following steps:

[0075] (2-1) Construction of a quantitative framework for student learning ability, establishment of learning ability evaluation indicators, including three indicators: homework completion rate, homework completion efficiency, and homework completion quality, and determination of relevant homework process log data elements for each evaluation indicator;

[0076] (2-2) Establish a student learning status detection model, obtain training samples of the log data of the currently assigned homework, establish a learning status detection model, and determine the detection model through sample training.

[0077] (2-3) Evaluation of students’ overall learning ability: Apply the student learning status detection model, analyze the log data of the homework process to be tested, obtain the detection results of relevant data elements of the evaluation indicators, establish a learning ability evaluation algorithm based on the progressive weighted accumulation of learning status at each stage, and obtain the evaluation of students’ overall learning ability.

[0078] (2-4) Store the results of students’ learning status tests at each stage and the overall learning ability score into the system’s learning ability log database.

[0079] The second step (2-1) of the module, which constructs a quantitative framework for students' overall learning ability, is characterized by establishing learning ability evaluation indicators, including three indicators: homework completion rate, homework completion efficiency, and homework completion quality.

[0080] Assignment completion rate: Used to evaluate students' pass rate for all currently assigned assignments;

[0081] Homework completion efficiency: Used to evaluate the time efficiency of students in completing all currently assigned homework;

[0082] Assignment completion quality: Used to evaluate the quality of students' completion of all currently assigned assignments;

[0083] The second step (2-1) of module 2 determines the relevant work process log data elements for each evaluation indicator, characterized in that:

[0084] The data elements in the job completion log include: the total number of jobs and the number of jobs that have been completed, which can be represented as:

[0085]

[0086] For a single assignment, a submission is considered passed if at least one of the submitted answers has a quality score greater than 0.7.

[0087] The data elements in the job completion efficiency log include: total number of job submissions, number of jobs that have passed, time taken for the job to pass the platform for the first time, and time taken for a single submission when the job first passed. These can be represented as: ),

[0088] The data elements in the job completion quality-related job process log include: the quality score for a single job calculated by Module 1, expressed as:

[0089]

[0090] The second step (2-2) of the module, the establishment of the student learning status detection model, is characterized by the following specific process:

[0091] (2-2-1) Obtain training samples of student homework process logs, and randomly generate simulation data elements of the homework process logs described in step (2) of module two as training samples for learning ability evaluation indicators. Sample labeling: According to the simulation data elements, the data samples are manually labeled to obtain student learning status score labels.

[0092] (2-2-2) Construct an XGBoost decision tree model and define the model features as data elements of three indicators from step (1) of module two: homework completion rate, homework completion efficiency, and homework completion quality. The label is the student's learning status score label.

[0093] (2-2-3) Decision tree model training and determination: Set model training parameters, use simulation data of the work process log for model training, and evaluate the model performance using precision P, recall R, and F1 score. The specific calculation formulas are as follows:

[0094]

[0095] In this context, Precision represents accuracy, Recall represents recall, TP refers to the number of times the true value was correctly predicted, FP refers to the number of times the false value was correctly predicted, and FN refers to the number of times the true value was incorrectly predicted.

[0096] The second step (2-3) of the module, which evaluates the overall learning ability of students, is characterized by the following learning ability evaluation algorithm process based on the accumulation of learning states:

[0097] (2-3-1) Maintain a list of student learning states, state. When the i-th assignment is completed or due, calculate the student's learning state score for the first t assignments completed. This is added to a list, which calculates and saves the learning status scores of students who have completed the first assignment, the first two assignments, ..., the first t assignments, denoted as...

[0098] (2-3-2) Set an index weight based on time stage, and sum the learning states of each stage in the learning state list in (3-1) to obtain the overall learning ability score of the students who have completed the first t assignments:

[0099]

[0100] Among them, w i The weights increase exponentially with time. Therefore, this method of establishing a student's overall learning ability score can, to some extent, reflect the improvement in the student's learning status over time, and better reflect the student's overall learning ability through self-improvement and problem-solving through multiple assignments. The overall learning ability score obtained after the final assignment is the student's learning ability score for this AI-generated image course.

[0101] Module 3: Visual Feedback Module

[0102] Based on the student assignment quality evaluation module and the overall learning ability evaluation module, the changes in students' problem-solving and learning abilities during the answering process are calculated, and the learning trajectory of each student is displayed in a personalized manner. This mainly includes:

[0103] (3-1) Extract the highest quality score of each student's assignment calculated in Module 1 from the system's student assignment process log database, and draw a line graph to show the changes in problem-solving ability;

[0104] (3-2) Extract the student's learning status score and overall learning ability score calculated by Module 2 from the system's student learning ability log database, and use a line graph to show the changes in student learning status and learning ability;

[0105] (3-3) Feedback the visualization results to teachers or students.

[0106] Compared with the prior art, this implementation method has the following advantages:

[0107] (1) Automated Evaluation Capability: Compared with existing technologies, the learning effect evaluation system of this embodiment possesses the capability to automatically evaluate the quality of AI-generated image assignments. Through image set similarity estimation, the system can automatically evaluate the quality of a student's single image generation assignment, eliminating the subjectivity and inconsistency of manual scoring. This automated evaluation not only improves scoring efficiency and reduces the workload of teachers, but also ensures the fairness and consistency of the evaluation.

[0108] (2) Comprehensive Learning Status Evaluation: Existing evaluation systems are often limited to the quality score of a single assignment, lacking a comprehensive assessment of students' overall learning ability. In contrast, the decision tree model constructed by the gradient boosting tree algorithm in this implementation can comprehensively analyze students' learning status based on multiple assignment quality scores and assignment process logs.

[0109] (3) Progressive learning ability assessment: Based on different learning stages, the learning status is weighted exponentially, which pays more attention to the changes in the student's learning status in the later stages of the course and evaluates the overall learning ability. This comprehensive assessment can more accurately reflect the student's learning progress and problem-solving ability, and helps to identify the student's weaknesses in the learning process in a timely manner.

[0110] (4) Personalized and Visualized Feedback Mechanism: The system in this implementation is designed with a personalized display function, providing customized feedback based on students' learning progress. Compared with the traditional "one-size-fits-all" evaluation method, this pays more attention to the unique needs of each student and can provide targeted learning suggestions, thereby improving learning effectiveness and motivation. The personalized feedback mechanism allows teachers to better understand each student's learning status and adjust their teaching strategies accordingly.

[0111] (5) Closed-Loop Management of the Teaching Process: This implementation system provides a systematic solution that supports closed-loop management of the teaching process through comprehensive analysis of homework evaluation and learning outcomes. Compared with existing technologies, this system can provide faster and more accurate feedback on student learning outcomes, thereby enabling dynamic adjustments to teaching content and methods and improving overall teaching quality. This closed-loop management approach ensures continuous improvement of the teaching process and helps achieve higher-level educational goals.

[0112] Detailed Implementation Method Three: This implementation method further details the technical solution provided in Implementation Method Two above, specifically:

[0113] Module 1: Student Single Course Assignment Quality Evaluation Module

[0114] For each assignment in the AI ​​image generation course, a set of target real images is provided from a given assignment image library. Students are required to use AIGC technology to generate images that are as similar as possible to the target set of real images and upload them to the system. The generated image sets uploaded by students... The quality score is determined by its comparison with the target real image set. The similarity score is calculated based on the Frechette heuristic distance between the feature distributions of the target image and the generated image. The extreme values ​​of the distance are estimated through multi-model image generation sampling experiments, and the distance between the target image and the feature distribution of the student assignment image set is standardized to obtain a similarity score within the range of 0-1. Figure 2 As shown, the quality evaluation of a single course assignment is formed by the following steps:

[0115] (1-1) Obtain the target real image set for the i-th task. Image sets generated from student-uploaded assignments Where n i m i These represent the number of target images for the assignment and the number of images uploaded by the student, respectively. Specifically, when... When only one target image is provided, different random Gaussian noises are applied to the target image to generate multiple target images, which are then added to the target real image set to obtain the updated image. Right now:

[0116]

[0117] Wherein, GN(·) represents the operation of applying random Gaussian noise to the image;

[0118] (1-2) A deep neural network G, pre-trained on a large-scale dataset, is preloaded as a deep feature extractor for processing all images. For images I∈R H*W After processing by a deep neural network G, high-dimensional features are obtained from the output of the deepest layer. These features are then subjected to global average pooling to obtain the final features.

[0119] f = AvgPool(G(I)); f ∈ R 1*C ,

[0120] Where C is the dimension of the deepest channel in the deep convolutional neural network G. After deep feature extraction of all images from the target real image set and the student assignment image set, the target real image feature set is obtained. Image feature set generated by the assignment

[0121] There are multiple methods for implementing image feature extraction based on pre-trained deep neural networks in step (1-2) of module one. This specification provides a specific embodiment for reference:

[0122] Specifically, the pre-trained InceptionV3 model built into the PyTorch-Fid library is preloaded, and the deepest layer dimension parameter is specified as 2048, serving as the deep feature extractor for all images. Table 1 shows the internal structure of this model. For the image to be processed, I∈R... H*W The image features are input into the pre-trained InceptionV3 model, which outputs the image features of layer 11 in Table 1. Global average pooling is then performed to obtain the deep image features f∈R. 1*2048 .

[0123] Table 1 Internal Structure of the InceptionV3 Model

[0124]

[0125] (1-3) Calculate the mean and covariance of the feature distributions of the target real image and the generated image, respectively:

[0126]

[0127] (1-4) Calculation of the characteristic distribution distance: The mean and covariance obtained in (1-3) are used to approximate the Fréchet Inception Distance (FID) between the two distributions, i.e.:

[0128]

[0129] (1-5) Standardize the distance obtained in (1-4) to obtain the similarity estimate in the 0-1 interval, which is used as the quality score for a student's single assignment:

[0130]

[0131] FID max FID min The FID extreme values ​​estimated based on the generated image sampling experiments are stored as hyperparameters after the initial estimation. score(r,g) i The calculations are then saved in the student's personal learning process log data.

[0132] The extreme value FID of the first step (1-5) of the module max FID min The estimation was accomplished through separate multi-model generated image sampling experiments, such as... Figure 4 As shown, the estimation experiment procedure is as follows:

[0133] (1-5-1) Select a Stable Diffusion model, give the prompt and other generation parameters, and generate several images (referred to as the generation results).

[0134] (1-5-2) Group the generated results into groups of 8 images each, discarding samples that are not large enough for a group. Randomly select one image from each group as the real image, and the remaining images as generated images. Calculate and record the FID value within each group. Specifically, image grouping includes internal grouping of images generated by a single generative model, as well as hybrid grouping of images generated by multiple generative models.

[0135] (1-5-3) After calculating the FID values ​​for all groups, return the maximum, minimum, and average FID values.

[0136] For the extreme value FID of step (1-5) of module one max FID min There are various ways to implement the generation model and prompt selection in the multi-model image sampling experiment during estimation. This manual presents one method: extreme value FID. max FID min Specific implementation examples for estimation are provided for reference:

[0137] For example, two different Stable Diffusion generation models are used to generate 200 images each based on different prompts, with each batch consisting of 8 images and different random seeds used between batches.

[0138] Specifically, for the first model (hereinafter referred to as Model A), a retrained model specifically designed for generating anime character images, downloaded from the website LiblibAI, was selected. This model was trained based on the algorithm sd-v1-5.ckpt. In practice, when using this model with numerous and detailed prompts, the generated images exhibited relatively similar structures. The prompts shown in Table 2 were selected for image generation. Figure 5 (a) provides some examples of generated images.

[0139] Table 2 Generated Model Parameter Attributes

[0140]

[0141] For the second model (hereinafter referred to as Model B), the Stable Diffusion official sd-v1-5.ckpt training model was selected. This model is more difficult to use than Model A, and the composition and content of the generated images under the same prompt can differ significantly when there are fewer prompt words. The prompt words shown in Table 2 were selected for image generation. Figure 5 (b) provides some examples of generated images.

[0142] Based on the characteristics of FID, the smaller the FID, the closer the feature vectors of the two sets of images are, and the higher the similarity, indicating good image quality. FID is 0 when the two sets of images are completely identical. Table 3 shows the extreme value calculation results of FID under the internal grouping of models A and B, as well as the mixed grouping method. The FID value range of images generated by model A (with relatively stable generation results) in the same prompt is approximately between 100 and 210, while the FID value range of images generated by model B (with unstable generation results) in the same prompt is approximately between 200 and 500. After shuffling and randomly grouping the images generated by models A and B, it can be considered that the generated images and real images are almost unrelated, and the calculated FID value range is approximately between 260 and 500. Therefore, the maximum FID value represents complete dissimilarity at around 500, and the minimum FID value represents visually identifiable similarity at around 100. Thus, the maximum and minimum FID values ​​are 500 and 100, respectively. This extreme value is suitable for similarity calculation when using the above models A, B, and their similar models to generate images.

[0143] Table 3. Calculation results of FID extreme values ​​under different grouping methods

[0144]

[0145] (6) Record the highest score for each student's submitted assignment for each assignment. i Number of assignment submissions (sub) i Pass by the number of assignment submissions. i The time from when the assignment is published to the first submission via the platform. The time it takes for a single submission to pass on the first submission The results are stored in the system's job process log database. Specifically, a job is considered passed if the job quality score is greater than 0.7. Module Two, the overall learning ability evaluation module, such as... Figure 3 It is formed by the following steps:

[0146] (2-1) Construction of a quantitative framework for student learning ability, establishment of learning ability evaluation indicators, including three indicators: homework completion rate, homework completion efficiency, and homework completion quality, and determination of relevant homework process log data elements for each evaluation indicator;

[0147] Specifically, the three evaluation indicators are as follows:

[0148] Assignment completion rate: This is used to evaluate students' completion status of all currently assigned assignments. Relevant assignment process log data elements include: number of assignment questions, number of questions passed, etc.

[0149] Homework completion efficiency: This is used to evaluate the time efficiency of students in completing all currently assigned homework. Relevant homework process log data elements include: total number of homework submissions, number of homework that has been passed, time taken for the first time a homework is passed on the platform, and time taken for a single submission when the homework is first passed.

[0150] Homework completion quality: Used to evaluate the quality of students’ completion of all currently assigned homework. Relevant homework process log data elements include: the quality score of a single homework assignment calculated by Module 1, etc.

[0151] There are various implementation methods for defining and calculating evaluation indicators based on various data elements. This implementation method provides an example of calculating various evaluation indicators:

[0152] Specifically, at time T, the job completion rate can be represented by the ratio of the total number of completed jobs to the total number of published jobs:

[0153]

[0154] in, N represents the evaluation index of student's homework completion at time T. T The score represents the total number of jobs published at time T. i Let num(·) be the highest individual score of a student completing the i-th assignment task, extracted from the system assignment process log database.

[0155] The evaluation index for task completion efficiency consists of two parts: the average submission pass rate for each task and the average first-time pass time efficiency for each task.

[0156]

[0157] in, The sub represents the evaluation index of the efficiency of task completion at time T. i pass i , These are, respectively, the number of times a student submits an assignment when completing the i-th assignment, the number of times an assignment is passed, the time from assignment publication to the first submission passing the platform, and the single submission time for the first submission passing the platform, all extracted from the system assignment process log database.

[0158] The quality evaluation index for completed assignments is represented by the average score of each assignment item:

[0159]

[0160] (2-2) Establishment of a student learning status detection model: Obtain training samples from the log data of currently assigned homework to establish a learning status detection model, and determine the detection model through sample training; specifically, the model establishment process is as follows:

[0161] (2-2-1) Obtain training samples of student homework process logs, and randomly generate simulation data elements of the homework process logs described in step (2) of module two as training samples for learning ability evaluation indicators. Sample labeling: According to the simulation data elements, the data samples are manually labeled to obtain student learning status score labels.

[0162] (2-2-2) Construct an XGBoost decision tree model and define the model features as data elements of three indicators from step (1) of module two: homework completion rate, homework completion efficiency, and homework completion quality. The label is the student's learning status score label.

[0163] (2-2-3) Decision tree model training and determination: Set model training parameters, use simulation data of the work process log for model training, and evaluate the model performance using precision P, recall R, and F1 score. The specific calculation formulas are as follows:

[0164]

[0165] In this context, Precision represents accuracy, Recall represents recall, TP refers to the number of times the true value was correctly predicted, FP refers to the number of times the false value was correctly predicted, and FN refers to the number of times the true value was incorrectly predicted.

[0166] There are multiple implementation methods for establishing a learning state detection model. This specification provides an example of simulation data generation, XGBoost decision tree model construction, and training, specifically:

[0167] Randomly generated task log data were used to construct simulation training samples. Labels were generated for the randomly generated log data according to the rules shown in Table 4: label 0 represents a student's poor learning status, and label 1 represents a student's good learning status. When the total number of tasks, the number of completed tasks, the number of submitted tasks, the average score for each task, the average submission time for the first completion, and the average platform usage time for the first completion all meet the data ranges given in the table, the corresponding label was assigned to the sample. The simulation samples were divided into training and testing sets, with the training set accounting for 80% and the testing set accounting for 20%.

[0168] Table 4. Examples of Simulation Operation Process Log Data Labeling Rules

[0169]

[0170] Construct and train the XGBoost decision tree model according to the parameter settings shown in Table 5. Organize and calculate all sample data according to the three evaluation indicators in step (1), and concatenate all evaluation indicator features as the input of the XGBoost model for a single training sample. The objective function for model training is the mean squared error function, and L1 and L2 regularization terms are added to control the model complexity;

[0171] Table 5 XGBoost Decision tree model parameter settings

[0172]

[0173] Table 6 shows the evaluation performance of the models trained on different training sets on the test set. It can be seen that the model performs better on the simulation dataset.

[0174] Table 6. Model performance under different training data sizes

[0175]

[0176] (2-3) Evaluation of students' overall learning ability, the specific implementation process is as follows:

[0177] (2-3-1) In the system's student learning ability log database, maintain student learning status data and update it at the end of each task. Specifically, at the end of the t-th assignment, calculate the student's learning status score for completing the previous t assignments. The results are then saved to a log database, which stores a list of student learning statuses, which can be represented as follows:

[0178] (2-3-2) Set exponential weights based on time stages, and sum the weighted learning states of each stage in the learning state list (3-1) to obtain the student's overall learning ability score at time T:

[0179]

[0180] Where, N T w represents the total number of jobs published at time T. i The weights increase exponentially with time. Therefore, this method of establishing a student's overall learning ability score can, to some extent, reflect the improvement in the student's learning status over time, and better reflect the student's overall learning ability through self-improvement and problem-solving through multiple assignments. The overall learning ability score obtained after the final assignment is the student's learning ability score for this AI-generated image course.

[0181] (2-4) Record the overall learning ability score calculated at the end of each task in the system’s student learning ability log database. That is, the student learning ability log database stores the student’s learning status score and overall learning ability score at the end of each assignment task.

[0182] Module 3: Visual Feedback Module

[0183] Based on the student assignment quality evaluation module and the overall learning ability evaluation module, changes in students' problem-solving and learning abilities during the answering process are calculated, and students' learning trajectories are displayed in a personalized manner for different students. Specific implementation steps include:

[0184] (3-1) Extract the highest quality score of each student's assignment calculated in Module 1 from the system's student assignment process log database, and draw a line graph to show the changes in problem-solving ability;

[0185] (3-2) Extract the student's learning status score and overall learning ability score calculated by Module 2 from the system's student learning ability log database, and use a line graph to show the changes in student learning status and learning ability;

[0186] (3-3) Use graphing tools to present the visualization results to teachers or students through the system's front-end page.

[0187] The above description of several specific embodiments further details the technical solution provided by the present invention in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, combinations of embodiments, and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A learning effectiveness evaluation system for artificial intelligence image generation training courses, comprising multiple modules for data collection and analysis of student learning processes, characterized in that: The system includes: The Student Single Course Assignment Quality Evaluation Module is used to evaluate the quality of students' single assignments in the Artificial Intelligence Image Generation course and obtain a quality score. The overall learning ability assessment module conducts a comprehensive analysis of students' learning status based on the quality scores of multiple assignments and assignment process logs, and obtains the students' overall learning ability score. The visual feedback module is used to generate student learning diaries based on changes in the quality of students' assignments and their learning abilities. The student single course assignment quality evaluation module uses a feature extraction method based on a pre-trained deep model to calculate the similarity between the target image and the image generated from the single assignment by allocating the feature distance, and obtains the quality score. The feature assignment distance is achieved through the Frechette revelation distance; The overall learning ability evaluation module uses a decision tree model built based on the gradient boosting tree algorithm to evaluate students' learning status at different stages. The overall learning ability evaluation module accumulates the scores of students' learning status at different learning stages using an exponential weighting method to obtain the students' overall learning ability score.

2. The learning effectiveness evaluation system for artificial intelligence image generation training courses according to claim 1, characterized in that: The visual feedback module provides personalized visual feedback on students' learning diaries and offers it to students and teachers through a front-end display page in the system.

3. A method for evaluating the learning effectiveness of training courses for artificial intelligence image generation, characterized by: The method is implemented based on the system described in claim 1, and the method includes: The steps for evaluating the quality of a student's single course assignment are as follows: to evaluate the quality of a student's single assignment in the Artificial Intelligence Image Generation course and obtain a quality score; The overall learning ability assessment module conducts a comprehensive analysis of students' learning status based on the quality scores of multiple assignments and assignment process logs, and obtains the students' overall learning ability score. The visual feedback module generates student learning diaries based on changes in the quality of students' assignments and their learning abilities.

4. A computer storage medium for storing computing programs, characterized in that, When the computer reads the computer program, the computer executes the method of claim 3.

5. A computer, including a processor and a storage medium, characterized in that, When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 3.

6. A computer program product, as a computer program, is characterized by: When the computer program is executed, it implements the method of claim 3.

Images