Learning effect evaluation system and method for artificial intelligence image generation training course

Abstract

The invention discloses a learning effect evaluation system and method for artificial intelligence image generation training courses, and relates to the technical field of computers. In order to solve the technical problem that in the prior art, a learning management system cannot feed back learning conditions of students in time, the technical scheme provided by the invention is that the system comprises a plurality of modules used for collecting and analyzing data in the learning process of the students, and comprises a student single course homework quality evaluation module, the quality evaluation module is used for performing quality evaluation on a single homework of a student in an artificial intelligence image generation course to obtain a quality score; the overall learning ability evaluation module is used for carrying out comprehensive learning state analysis on the quality scores and the homework process logs of the multiple homeworks of the students to obtain overall learning ability scores of the students; and the visual feedback module is used for generating learning diaries of the students according to the change of the homework quality and the learning ability of the students. The method is suitable for being applied to learning effect evaluation work of artificial intelligence image generation courses.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a learning effect evaluation system and method for an artificial intelligence image generation training course. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, especially the breakthrough in the field of image generation, the education industry has also ushered in new changes. Traditional teaching evaluation methods are gradually replaced by more intelligent and dynamic learning effect evaluation systems, especially in the teaching of artificial intelligence image generation courses. This change is particularly obvious.

[0003] Existing learning management systems (LMS) are already able to record various event logs of students during the learning process, including learning time, task completion, interaction frequency, etc. These data provide a basis for evaluating students' learning outcomes.

[0004] However, the existing learning effect evaluation system still has some shortcomings.

[0005] The current learning management system (LMS) can record the event logs of students' learning process, including learning time, task completion, interaction frequency, etc. Although these data provide a certain basis for the evaluation of learning record effects, the existing evaluation system still has defects in the following aspects: First, there is a lack of accurate evaluation of individual differences among students, making it difficult to achieve personalized learning effect evaluation; second, the existing system is mainly a static evaluation method, which lacks real-time performance and cannot provide timely feedback on students' learning situation; third, for the AI ​​image generation course, the evaluation method of the existing system is difficult to combine the students' innovative ability and technical mastery in the creative process. Although some studies have tried to combine process evaluation with comprehensive ability evaluation to improve the accuracy and timeliness of learning evaluation, its application in AI image generation courses is still very scarce. Therefore, it is very urgent to develop a learning effect evaluation system that can realize personalized learning evaluation for students and has dynamic and personalized functions. Summary of the invention

[0006] In order to solve the technical problems that the learning management system in the prior art lacks accurate assessment of individual differences of students and is difficult to achieve personalized learning effect evaluation; the existing system is mainly a static evaluation method, lacks real-time performance, and cannot provide timely feedback on students' learning status, the technical solution provided by the present invention is:

[0007] A learning effect evaluation system for artificial intelligence image generation training courses, including multiple modules for student learning process data collection and analysis.

[0008] The system comprises:

[0009] The student single course assignment quality evaluation module is used to evaluate the quality of students' single assignments in the AI ​​image generation course and obtain quality scores;

[0010] The overall learning ability evaluation module conducts a comprehensive learning status analysis on the quality scores of multiple assignments and the assignment process logs of students to obtain the overall learning ability scores of students;

[0011] The visual feedback module is used to generate student learning diaries based on changes in student homework quality and learning ability.

[0012] Furthermore, a preferred implementation is provided, in which a student single course assignment quality evaluation module calculates the similarity based on a feature extraction method of a pre-trained deep model using the feature allocation distance between a target image and an image generated from the single assignment to obtain a quality score.

[0013] Furthermore, a preferred embodiment is provided, wherein the feature allocation distance is implemented by Frechette's revelation distance.

[0014] Furthermore, a preferred implementation scheme is provided, namely, an overall learning ability evaluation module, which uses a decision tree model constructed based on a gradient boosting tree algorithm to evaluate the learning status of students at each stage.

[0015] Furthermore, a preferred implementation is provided, wherein the overall learning ability evaluation module accumulates the learning status scores of students at different learning stages by exponential weight accumulation, thereby obtaining the overall learning ability scores of students.

[0016] Furthermore, a preferred implementation is provided, a visual feedback module, which provides personalized visual feedback on students' learning diaries and provides it to students and teachers through the system front display page.

[0017] Based on the same inventive concept, the present invention also provides a learning effect evaluation method for an artificial intelligence image generation training course.

[0018] The method is implemented based on the system, and the method includes:

[0019] The quality evaluation step of a student's single course assignment is to evaluate the quality of a student's single assignment in the AI ​​image generation course and obtain a quality score;

[0020] The overall learning ability evaluation module conducts a comprehensive learning status analysis on the quality scores of multiple assignments and the assignment process logs of students to obtain the overall learning ability scores of students;

[0021] The visual feedback module generates student learning diaries based on changes in student homework quality and learning ability.

[0022] Based on the same inventive concept, the present invention also provides a computer storage medium for storing a computer program, characterized in that when the computer reads the computer program, the computer executes the described method.

[0023] Based on the same inventive concept, the present invention also provides a computer, including a processor and a storage medium, characterized in that when the processor reads the computer program stored in the storage medium, the computer executes the described method.

[0024] Based on the same inventive concept, the present invention also provides a computer program product, as a computer program, characterized in that when the computer program is executed, the described method is implemented.

[0025] Compared with the prior art, the technical solution provided by the present invention is beneficial in that:

[0026] The image set similarity estimation method is used to automatically evaluate students' single image generation assignments, which solves the problems of emphasis and inconsistency in manual scoring, improves scoring efficiency, reduces the workload of teachers, and ensures the fairness and consistency of evaluation. This automated evaluation capability enables students to receive scientific and fair quality evaluation at every step in the learning process, thereby replacing the existing manual scoring method with higher reliability and traceability.

[0027] By building a decision tree model through the gradient boosting tree algorithm, we can comprehensively analyze the students' learning status based on their homework quality scores and homework process logs, thereby providing a comprehensive evaluation of their learning status. Compared with the existing single homework quality scoring system, this model can provide a comprehensive evaluation of students' overall learning ability, help teachers and students identify weak links in the learning process, and adjust teaching strategies in a timely manner.

[0028] The exponential weight accumulation method is used to accumulate the learning status exponentially according to different learning stages, focusing more on the changes in students' learning status in the later stage of course learning and evaluating their overall learning ability.

[0029] Compared with the existing static evaluation system, this progressive learning ability evaluation method can more accurately reflect students' learning progress and problem-solving ability, and help to better understand students' ability improvement throughout the learning process, especially their performance in later learning.

[0030] The system is designed with a personalized and visual feedback mechanism to provide customized feedback based on students' learning situation. Compared with the traditional "one-size-fits-all" evaluation method, it pays more attention to students' unique needs. Through this personalized display function, students can get preliminary suggestions on learning initiative, which enhances their learning enthusiasm and initiative and improves learning results. In addition, teachers can have a deeper understanding of students' learning status through visual feedback, so as to achieve the expected teaching adjustments.

[0031] It is suitable for use in the learning effect evaluation of artificial intelligence image generation courses. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a framework diagram of the system composition of the present invention.

[0033] Figure 2 This is a flow chart of the single course homework quality evaluation module of the present invention.

[0034] Figure 3 This is a flow chart of the student overall learning ability evaluation module of the present invention.

[0035] Figure 4 This is a flow chart of FID distance extreme value estimation in the single course homework quality evaluation module of the present invention.

[0036] Figure 5 To generate an image example. DETAILED DESCRIPTION

[0037] In order to make the advantages and benefits of the technical solution provided by the present invention more clearly reflected, the technical solution provided by the present invention is now further described in detail with reference to the accompanying drawings, specifically:

[0038] At present, although some platforms and companies have begun to explore ways to combine process evaluation, comprehensive ability evaluation and dynamic evaluation to improve the accuracy and timeliness of teaching feedback, there are still few specific application cases for AI image generation courses. This makes teachers face certain challenges when evaluating students' learning effects, especially in how to quantitatively evaluate students' creative ability, technical mastery and understanding of course content.

[0039] Therefore, it is particularly necessary to develop a learning effect evaluation system and method for AI image generation courses. The system can integrate multiple data sources and use advanced machine learning and data mining techniques to monitor and analyze students' learning process in real time to achieve a comprehensive evaluation of students' abilities. In addition, the system should have the function of personalized recommendation of learning paths and be able to dynamically adjust teaching strategies and content according to students' learning situation, thereby improving educational effects.

[0040] In short, in the face of the rapid development in the field of artificial intelligence image generation, establishing an efficient and intelligent learning effect evaluation system will provide strong support for teachers, help them better understand and guide students' learning process, and ultimately achieve further optimization and improvement of teaching.

[0041] Implementation method 1: The system provided in this implementation method has the following specific steps in the learning effect evaluation work for the artificial intelligence image generation course:

[0042] Step 1: Initialize the student single course assignment quality evaluation module

[0043] The pre-trained deep model is used to extract features from the image generation assignments submitted by students and calculate the similarity between image sets.

[0044] Detailed description: First, the system receives the AI-generated images submitted by students, extracts the generated image set and the corresponding target image set submitted for each assignment. Then, based on the pre-trained deep model (such as the InceptionV3 model), the system extracts high-dimensional visual features from these images and generates a feature set. The system then calculates the feature distribution distance between the target image set and the generated image set using the Frechette hint distance (FID), and normalizes the distance to obtain a similarity score (between 0 and 1). The similarity score is the quality score of a single assignment.

[0045] Step 2: Construction of overall learning ability evaluation module

[0046] Based on the scores and homework logs of students' homework, a decision tree model is constructed using the gradient boosting tree algorithm to gradually evaluate the students' learning status.

[0047] Detailed description: The system inputs the quality score of each assignment and the relevant log data of the assignment process (such as the number of assignments completed, assignment time, task completion time, etc.) into the decision tree model. Based on these data, the gradient boosting tree algorithm is used to build a learning status evaluation decision tree model to quantify the student's learning status. The learning status is obtained through a comprehensive analysis of the assignment completion, efficiency and quality scores. The model uses an exponential weight accumulation method to calculate the overall learning ability score of students at different learning stages, so as to better reflect the progress and ability improvement of students in the learning process.

[0048] Step 3: Personalized visual feedback module generates feedback information

[0049] Generate personalized feedback based on student learning quality and learning ability changes and display it to students and teachers.

[0050] Detailed description: The system extracts the quality score of each homework and the learning ability score based on the student's academic performance, and generates corresponding visual data (such as a line chart). These visual feedbacks mainly include the changing trend of the student's homework quality score and the dynamic changes of the learning status score, so as to help students and teachers clearly understand the student's learning progress. The system will display these visual results to students and teachers in advance, so as to achieve dynamic tracking of the learning process and personalized teaching adjustment suggestions.

[0051] Step 4: Personalized learning path recommendation

[0052] Dynamically adjust teaching strategies and learning content based on students' learning performance, and provide personalized learning path recommendations.

[0053] Detailed description: Based on the student's overall learning ability score and homework quality score, the system can make personalized adjustments to the teaching strategy, such as recommending suitable learning materials, adjusting the learning difficulty, or providing additional exercises. The recommendation process uses an adaptive recommendation algorithm, which combines the student's historical learning data and current learning performance to generate a personalized learning path to better meet the student's learning needs.

[0054] Implementation Method 2: Combination Figure 1-4 This embodiment further describes the technical solution provided above in detail, specifically:

[0055] A learning effect evaluation system and method for an artificial intelligence image generation course, including: a student single course homework quality evaluation module, an overall learning ability evaluation module, and a visual feedback module. The method is based on the student's answering behavior for the target image generation homework of the artificial intelligence image generation course, and uses the image set similarity estimation method to automatically evaluate the quality of the student's single image generation homework; based on the quality scores of students' multiple homeworks and the homework process log, a learning state evaluation decision tree model is constructed using the gradient boosting tree algorithm, and an overall learning ability evaluation based on the progressive accumulation of learning states at each stage is performed, and the overall learning effect of the course is fed back to the students to achieve personalized display. This implementation method can automatically give a quantitative and fair evaluation of the student's problem-solving ability and overall learning ability based on the student's answering behavior, and systematically provides a solution process for homework evaluation and teaching effect evaluation necessary for the teaching process of the artificial intelligence image generation course, which can feedback students' learning effects faster and more accurately, improve teaching quality, and help closed-loop teaching full life cycle management.

[0056] like Figure 1 This implementation method includes a student single course assignment quality evaluation module, an overall learning ability evaluation module, and a visual feedback module:

[0057] Module 1: Student single course assignment quality evaluation module

[0058] Extract the AI-generated images submitted by students in a single assignment, use the image set similarity estimation method, calculate the similarity between the submitted generated image set and the target real image set of the assignment based on the Frechette revelation distance, and make a quality evaluation of the AIGC image generation assignment completed by the students based on the similarity between the submitted assignment and the real image;

[0059] like Figure 2 The student single course assignment quality evaluation module includes the following steps:

[0060] (1-1) For a single course assignment, the teacher or the teaching platform system assigns it according to teaching experience and needs, including the assignment number i and one or more assignment target pictures, which constitute the target real picture set where n i is the number of target images for the i-th assignment. The assignment requires students to use AIGC technology to generate images that match the target image set. After the student submits the assignment, a set of AIGC generated images submitted for the i-th assignment is extracted to form the generated image set Where m i The number of images submitted by students for the i-th assignment.

[0061] (1-2) Use the feature extraction method based on the pre-trained deep model to process the target real image set Generate image collection with assignments All images in the target image are obtained to obtain the target real image feature set Generate image feature sets with jobs

[0062] (1-3) Calculate the feature distribution distance FID(r,g) between the target real image set and the job generated image set.

[0063] (1-4) The distribution distance is normalized based on the maximum and minimum values ​​to obtain a similarity score in the range of 0-1, which is used as the job quality evaluation, namely:

[0064]

[0065] Where FID o is the feature distribution distance calculated in (1-3), FID max 、、FID min It is the maximum and minimum value of the estimated true and generated image FID distance.

[0066] (1-5) Record the highest score of the homework quality submitted by students for each homework task, the number of homework submissions, the time from the homework release to the first submission passing the platform, the single submission time of the first submission passing, etc., and store them in the system homework process log database. In particular, when the homework quality score is greater than 0.7, the homework is considered to have passed.

[0067] The module 1 step (1-2) is based on the feature extraction method of the pre-trained deep model, loads the deep model pre-trained on a massive data set, inputs the image into the pre-trained model, and performs average pooling on the deepest layer output to extract the high-dimensional visual features of the image.

[0068] The step (1-3) of the module 1 is a feature distribution distance calculation step, which includes using the mean and covariance matrix of the real and generated image sets to approximately calculate the Fréchet Inception Distance (FID) between the two distributions, namely:

[0069]

[0070] where μ r , μ g is the mean of the feature distribution of the real image and the generated image, μ r , μ g is the covariance of the two feature distributions.

[0071] The step (1-4) of the module 1 is a step of normalizing the maximum and minimum values ​​of the feature distribution distance, characterized in that the maximum and minimum values ​​of the FID distance are obtained by statistically estimating the multi-model generated image sampling experiment, and the experimental design is based on different positive and negative prompt words and different AIGC generation models to generate image groups with different similarities, and the similarity of the samples within and between groups is calculated to calculate the extreme values. Specifically, different AIGC generation models are used to generate multiple pictures that are similar within the group and very dissimilar between groups as samples, and random grouping is first performed in each sample group to estimate the FID extreme value, and then the multi-model sample groups are mixed into a population and randomly grouped to estimate the FID extreme value, and the maximum and minimum values ​​of all FID extreme values ​​are taken to normalize the FID distance between the target homework image and the student-generated image distribution to obtain the homework score.

[0072] Module 2: Overall learning ability evaluation module

[0073] The quality scores of students' completed assignments generated by module 1 are extracted. At the same time, statistical data of the assignment process, such as the number of assignment submissions, assignment time, and assignment scores, are obtained from the answering behavior log. A decision tree model for overall learning evaluation is constructed using the gradient boosting tree algorithm. Simulation data is constructed to train the model. The decision results of the model obtained through training are applied to generate an evaluation of students' overall learning ability based on the learning status of each stage.

[0074] like Figure 3 , the overall learning evaluation module includes the following steps:

[0075] (2-1) Construct a quantitative framework for students’ learning ability and establish evaluation indicators for learning ability, including three indicators: homework completion, homework completion efficiency, and homework completion quality, and determine the data elements of the homework process log related to each evaluation indicator;

[0076] (2-2) Establishing a student learning status detection model, obtaining training samples of the current assigned homework process log data, establishing a learning status detection model, and determining the detection model through sample training;

[0077] (2-3) Evaluation of students’ overall learning ability: Apply the student learning status detection model, analyze the log data of the homework process to be detected, obtain the detection results of the data elements related to the evaluation indicators, establish a learning ability evaluation algorithm based on the progressive weighted accumulation of the learning status of each stage, and obtain the overall learning ability evaluation of students.

[0078] (2-4) The learning status test results of each stage of the students and the overall learning ability score are stored in the system learning ability log database.

[0079] The second module, step (2-1), constructs a quantitative framework for students' overall learning ability, which is characterized by establishing learning ability evaluation indicators, including three indicators: homework completion, homework completion efficiency, and homework completion quality, among which:

[0080] Homework completion: used to evaluate students' completion of all currently assigned homework;

[0081] Homework completion efficiency: used to evaluate the time efficiency of students in completing all currently assigned homework;

[0082] Homework completion quality: used to evaluate the quality of students' completion of all currently assigned homework;

[0083] The second step (2-1) of the module determines the log data elements of the operation process related to each evaluation indicator, which is characterized by:

[0084] The operation process log data elements related to the completion of the operation include: the total number of operation tasks and the number of completed operation tasks, which can be expressed as:

[0085]

[0086] For a single homework task, as long as there is one submission with a quality score greater than 0.7 among all the submitted answers, the homework is passed.

[0087] The operation process log data elements related to the operation completion efficiency include: the total number of operation submissions, the number of completed operations, the time it takes for the operation to pass the platform for the first time, and the single submission time when the operation passes for the first time, which can be expressed as: ),

[0088] The data elements of the operation process log related to the completion quality of the operation include: the single operation quality score calculated by module 1, expressed as:

[0089]

[0090] The second step (2-2) of the module is to establish a student learning status detection model, which is characterized in that the specific process of model establishment is as follows:

[0091] (2-2-1) Obtain the student's homework process log training sample, randomly generate the homework process log simulation data elements described in step (2) of module 2, as the learning ability evaluation index training sample. Sample labeling, according to the simulation data elements, manually label the data sample to obtain the student's learning status score label.

[0092] (2-2-2) Construct an XGBoost decision tree model and define the model features including the data elements of the three indicators of homework completion, homework completion efficiency, and homework completion quality in step (1) of module 2. The label is the student learning status score label.

[0093] (2-2-3) Decision tree model training and determination: Set model training parameters, use the operation process log simulation data training samples for model training, and use precision P, recall rate R, and F1 score to evaluate the performance of the model. The specific calculation formula is as follows:

[0094]

[0095] Among them, Precision represents the accuracy, Recall represents the recall, TP refers to the number of correct predictions of the true value, FP refers to the number of correct predictions of the wrong value, and FN refers to the number of wrong predictions of the true value.

[0096] The module 2 step (2-3) of the overall learning ability evaluation of students is characterized in that the learning ability evaluation algorithm process of the learning state accumulation is:

[0097] (2-3-1) Maintain a student learning status list state, and when the i-th homework task is completed or expired, calculate the learning status score of the student who completed the previous t homeworks And add it to the list, that is, the list calculates and saves the learning status scores of students who have completed the first homework, completed the first two homeworks, ..., and completed the first t homeworks, expressed as

[0098] (2-3-2) Set the exponential weight based on the time stage, and perform the weighted summation of the learning status of each stage in the learning status list state in (3-1) to obtain the overall learning ability score of the students who have completed the first t homework:

[0099]

[0100] Among them, w i The weight increases exponentially as time goes by. Therefore, the overall learning ability score of students established in this way can pay attention to the improvement of students' learning status over time to a certain extent, and can better reflect the overall learning ability of students to improve themselves and solve problems through multiple assignments. The overall learning ability score obtained after the last assignment is the learning ability score of the student in the AI-generated image course.

[0101] Module 3: Visual Feedback Module

[0102] Based on the student homework quality obtained from the single course homework quality evaluation module and the student overall learning ability from the overall learning ability evaluation module, the problem-solving ability and learning ability changes of students in the process of answering questions are calculated respectively, and the learning trajectory of students is displayed in a personalized way for different students. It mainly includes:

[0103] (3-1) Extract the highest quality score of each student's homework calculated in module 1 from the system student homework process log database, and draw a line graph to show the changes in problem-solving ability;

[0104] (3-2) Extract the student's learning status score and overall learning ability score calculated by module 2 from the system's student learning ability log database, and use a line chart to display the student's learning status and learning ability changes;

[0105] (3-3) Provide visualization results to teachers or students.

[0106] Compared with the prior art, this embodiment has the following advantages:

[0107] (1) Automated evaluation capability: Compared with the prior art, the learning effect evaluation system of this embodiment has the ability to automatically evaluate the quality of AI image generation assignments. Through the image set similarity estimation method, the system can automatically evaluate the quality of a student's single image generation assignment, eliminating the subjectivity and inconsistency of manual scoring. This automated evaluation not only improves the efficiency of scoring and reduces the workload of teachers, but also ensures the fairness and consistency of the evaluation.

[0108] (2) Comprehensive learning status evaluation: Existing evaluation systems are often limited to the quality rating of a single assignment and lack a comprehensive assessment of the student's overall learning ability. In contrast, the decision tree model constructed by the gradient boosting tree algorithm in this embodiment can comprehensively analyze the student's learning status based on the student's multiple assignment quality ratings and assignment process logs.

[0109] (3) Progressive learning ability evaluation: The learning status is exponentially weighted according to different learning stages, with more emphasis on the changes in the learning status of students in the later stages of course learning, and the overall learning ability is evaluated. This comprehensive evaluation can more accurately reflect students’ learning progress and problem-solving ability, and help to promptly identify students’ weak links in the learning process.

[0110] (4) Personalized and visualized feedback mechanism: The system of this implementation is designed with a personalized display function to provide customized feedback based on the student's learning situation. Compared with the traditional "one-size-fits-all" evaluation method, this pays more attention to the unique needs of each student and can provide students with targeted learning suggestions, thereby improving learning effects and enthusiasm. The personalized feedback mechanism enables teachers to better understand the learning status of each student and adjust teaching strategies accordingly.

[0111] (5) Closed-loop management of the teaching process: This implementation system provides a systematic solution that supports closed-loop management of the teaching process through comprehensive analysis of homework evaluation and learning outcomes. Compared with existing technologies, this system can provide faster and more accurate feedback on student learning outcomes, thereby achieving dynamic adjustments to teaching content and methods and improving overall teaching quality. This closed-loop management approach ensures continuous improvement of the teaching process and helps achieve higher-level educational goals.

[0112] Specific implementation method 3: This implementation method further refines the technical solution provided in the above-mentioned implementation method 2, specifically:

[0113] Module 1: Student single course assignment quality evaluation module

[0114] For each assignment in the AI ​​image generation course, a target real image set is given from the given assignment image library. Students are required to use AIGC technology to generate images that are as similar as possible to the target real image set and upload them to the system. The quality score is calculated by comparing it with the target real image set The similarity is calculated based on the Frechette revelation distance between the target image and the generated image. The distance extremes are estimated through multi-model image generation sampling experiments, and the distance between the target image and the student homework image set is standardized to obtain a similarity score in the range of 0-1. Figure 2 As shown, the single coursework quality evaluation is formed by the following steps:

[0115] (1-1) Obtain the target real image set for the i-th job Generate image sets with student uploaded assignments where n i 、m i are the number of homework target images and the number of images uploaded by students. In particular, when That is, when only one target image is provided, different random Gaussian noises are applied to the target image to generate multiple target images, which are added to the target real image set to obtain the updated Right now:

[0116]

[0117] Where GN(·) represents the operation of applying random Gaussian noise to the image;

[0118] (1-2) Preload the deep neural network G pre-trained on a large-scale dataset as a deep feature extractor for processing all images. H*W , processed by the deep neural network G, after the deepest layer outputs the high-dimensional features, the features are obtained after global average pooling, namely:

[0119] f = AvgPool(G(I)); f∈R 1*C ,

[0120] Where C is the deepest channel dimension of the deep convolutional neural network G. After deep feature extraction of all images in the target real image set and the student homework generated image set, the target real image feature set is obtained. Generate image feature sets with jobs

[0121] There are many methods for implementing the image feature extraction based on the pre-trained deep neural network in step (1-2) of the module 1. This specification provides a specific embodiment as a reference:

[0122] Specifically, the pre-trained InceptionV3 model built into the Pytorch-Fid library is pre-loaded, and the deepest dimension parameter of the model is specified as 2048, as the deep feature extractor for extracting all images. Table 1 shows the internal structure of the model. H*W , input it into the pre-trained InceptionV3 model, output the image features of the 11th layer in Table 1, and perform global average pooling to obtain the deep image features f∈R 1*2048 .

[0123] Table 1 InceptionV3 model internal structure

[0124]

[0125] (1-3) Calculate the mean and covariance of the target real image feature distribution and the job generated image feature distribution respectively:

[0126]

[0127] (1-4) Feature distribution distance calculation: Use the mean and covariance calculated in (1-3) to approximately calculate the Fréchet Inception Distance (FID) between the two distributions, that is:

[0128]

[0129] (1-5) Normalize the distance in (1-4) to obtain a similarity estimate in the range of 0-1, which is used as the quality score of the student's single assignment:

[0130]

[0131] Where FID max 、FID min The FID extreme value estimated from the generated image sampling experiment is stored as a hyperparameter after the first estimation. score(r,g) i After the calculation is completed, it is saved in the student's personal learning process log data.

[0132] The module 1 step (1-5) extreme value FID max 、FID min The estimation is done by a separate multi-model generated image sampling experiment, such as Figure 4 As shown, the estimation experiment process is as follows:

[0133] (1-5-1) Select a Stable Diffusion model, give prompt and other generation parameters, and generate several pictures (referred to as generation results).

[0134] (1-5-2) Group the generated results into groups of 8, and discard the samples that do not fit into the group. Randomly select one image in the group as the real image, and the remaining images as the generated images. Calculate the FID value in the group and record it. In particular, the image grouping includes the internal grouping method for images generated by a single generation model, and the mixed grouping method for images generated by mixed multiple generation models.

[0135] (1-5-3) After calculating the FID values ​​of all groups, return the maximum, minimum and average FID values.

[0136] For the module one step (1-5) extreme FID max 、FID min There are many ways to generate models and prompt selection during the multi-model image sampling experiment. This manual gives an extreme FID max 、FID min The specific examples of estimation are provided as reference:

[0137] For example, two different Stable Diffusion generation models are used to generate 200 images each according to different prompts, and each model generates 8 images as a batch, with different random seeds used between batches.

[0138] Specifically, for the first model (hereinafter referred to as Model A), a model that has been retrained and specially used to generate anime characters was selected from the website LiblibAI. The model was trained based on the sd-v1-5.ckpt algorithm. In use, when the prompts are large and detailed, the structure and content of the pictures generated by this model are similar. The prompt words shown in Table 2 are selected for image generation. Figure 5 (a) shows some examples of generated images.

[0139] Table 2 Generated model parameter attributes

[0140]

[0141] For the second model (hereinafter referred to as model B), the official sd-v1-5.ckpt training model of Stable Diffusion is selected. This model is more difficult to use than model A. When there are fewer prompt words, the composition and content of the generated pictures under the same prompt will be quite different. The prompt words shown in Table 2 are selected for image generation. Figure 5 (b) shows some examples of generated images.

[0142] According to the characteristics of FID, the smaller the FID, the closer the feature vectors of the two groups of pictures are, the higher the similarity is, indicating that the quality of the generated pictures is good. When the two groups of pictures are completely consistent, the FID is 0. Table 3 shows the extreme value calculation results of FID under the internal grouping and mixed grouping of models A and B. The FID value range of the pictures generated by the same prompt of model A with relatively stable generation results is roughly between 100 and 210, and the FID value range of the pictures generated by the same prompt of model B with unstable generation results is roughly between 200 and 500. After the pictures generated by models A and B are shuffled and randomly grouped, it can be considered that the generated pictures and the real pictures are almost irrelevant, and the calculated FID value range is roughly between 260 and 500. It can be concluded that the complete dissimilarity represented by the maximum FID value is about 500 in value, and the similarity visible to the naked eye represented by the minimum FID value is about 100 in value. Therefore, the maximum and minimum FID values ​​are 500 and 100 respectively. This extreme value is suitable for similarity calculation when using the above-mentioned models A, B and similar models to generate pictures.

[0143] Table 3 FID extreme value calculation results under different grouping methods

[0144]

[0145] (6) Record the highest score of the homework quality submitted by students for each homework task i , Number of job submissions sub i , pass the number of job submissions pass i , Time from the publication of the assignment to the first submission through the platform Single submission time for first submission pass etc., and stored in the system operation process log database. In particular, when the operation quality score is greater than 0.7, the operation is considered to have passed. Module 2: Overall learning ability evaluation module, such as Figure 3 , formed by the following steps:

[0146] (2-1) Construct a quantitative framework for students’ learning ability and establish evaluation indicators for learning ability, including three indicators: homework completion, homework completion efficiency, and homework completion quality, and determine the data elements of the homework process log related to each evaluation indicator;

[0147] Specifically, the three evaluation indicators are:

[0148] Homework completion: used to evaluate students' completion of all currently assigned homework. The relevant homework process log data elements include: the number of homework questions, the number of questions passed, etc.

[0149] Homework completion efficiency: used to evaluate the time efficiency of students in completing all currently assigned homework. The relevant homework process log data elements include: the total number of homework submissions, the number of homework passes, the time it takes for the homework to pass the platform for the first time, and the time it takes for a single submission when the homework passes for the first time, etc.

[0150] Homework completion quality: used to evaluate the quality of students' completion of all currently assigned homework. The relevant homework process log data elements include: the single homework quality score calculated in Module 1, etc.

[0151] There are many implementation methods for defining and calculating evaluation indicators according to various data elements. This implementation method provides an example of calculating various evaluation indicators:

[0152] Specifically, when the current time is T, the job completion degree can be expressed by the ratio of the total number of completed job tasks to the total number of published job tasks:

[0153]

[0154] in, represents the evaluation index of the student's homework completion at time T, N T is the total number of published jobs at time T, score i is the highest individual score of the student completing the i-th homework task extracted from the system homework process log database, and num(·) is a counting function.

[0155] The evaluation index of job completion efficiency is composed of the average of the submission pass rate of each job and the average of the first pass time efficiency of each job:

[0156]

[0157] in, represents the evaluation index of the job completion efficiency at time T, sub i 、pass i , They are respectively the number of homework submissions when students complete the i-th homework task, the number of successful homework submissions, the time from homework release to the first successful submission on the platform, and the single submission time for the first successful submission, which are extracted from the system homework process log database.

[0158] The evaluation index of the quality of homework completion is represented by the average score of each homework:

[0159]

[0160] (2-2) Establishing a student learning status detection model, obtaining the current assigned homework process log data training sample, establishing a learning status detection model, and determining the detection model through sample training; specifically, the specific process of model establishment is as follows:

[0161] (2-2-1) Obtain the student's homework process log training sample, randomly generate the homework process log simulation data elements described in step (2) of module 2, as the learning ability evaluation index training sample. Sample labeling, according to the simulation data elements, manually label the data sample to obtain the student's learning status score label.

[0162] (2-2-2) Construct an XGBoost decision tree model and define the model features including the data elements of the three indicators of homework completion, homework completion efficiency, and homework completion quality in step (1) of module 2. The label is the student learning status score label.

[0163] (2-2-3) Decision tree model training and determination: Set model training parameters, use the operation process log simulation data training samples for model training, and use precision P, recall rate R, and F1 score to evaluate the performance of the model. The specific calculation formula is as follows:

[0164]

[0165] Among them, Precision represents the accuracy, Recall represents the recall, TP refers to the number of correct predictions of the true value, FP refers to the number of correct predictions of the wrong value, and FN refers to the number of wrong predictions of the true value.

[0166] There are many implementation methods for establishing a learning state detection model. This specification provides an example of generating simulation data, building an XGBoost decision tree model, and training. Specifically:

[0167] Randomly generate job process log data and construct simulation training samples. Generate labels for the randomly generated log data according to the rules shown in Table 4. Label 0 represents that the student has a poor learning state, and label 1 represents a good learning state. When the total number of homework tasks, the number of passed homeworks, the number of submitted homeworks, the average score of each task, the average submission time for the first pass, and the average platform usage time for the first pass represented by the sample all meet the data range given in the table, mark the sample with the corresponding label. Divide the simulation samples into training sets and test sets, with the training set accounting for 80% and the test set accounting for 20%;

[0168] Table 4. Examples of log data annotation rules for simulation operation process

[0169]

[0170] The XGBoost decision tree model was constructed and trained according to the parameter settings shown in Table 5. All sample data were organized and calculated according to the three evaluation indicators in step (1), and all evaluation indicator features were concatenated as the XGBoost model input for a single training sample. The objective function of the model training was the mean square error function, and L1 and L2 regularization terms were added to control the model complexity;

[0171] Table 5 XGBoost Decision tree model parameter settings

[0172]

[0173] Table 6 shows the evaluation results of the models trained with different scale training sets on the test set. It can be seen that the model performance is better on the simulation data set.

[0174] Table 6 Model performance under different scale training data

[0175]

[0176] (2-3) Evaluation of students’ overall learning ability. The specific implementation process is as follows:

[0177] (2-3-1) In the system student learning ability log database, the student learning status data is maintained and updated at the end of each task. Specifically, when the tth homework task is due, the student learning status score of the student who has completed the previous t homework tasks is calculated. The results are saved in the log database. The student learning status list stored in the database can be expressed as

[0178] (2-3-2) Set the exponential weight based on the time stage, and perform the weighted summation of the learning status of each stage in the learning status list state in (3-1) to obtain the overall learning ability score of the student at time T:

[0179]

[0180] Among them, N T is the total number of published jobs at time T, w i The weight increases exponentially as time goes by. Therefore, the overall learning ability score of students established in this way can pay attention to the improvement of students' learning status over time to a certain extent, and can better reflect the overall learning ability of students to improve themselves and solve problems through multiple homework assignments. The overall learning ability score obtained after the last homework assignment is the student's learning ability score for the AI-generated image course.

[0181] (2-4) The overall learning ability score calculated at the end of each task is recorded in the system's student learning ability log database, that is, the student learning ability log database stores the student's learning status score and overall learning ability score at the end of each homework task.

[0182] Module 3: Visual Feedback Module

[0183] Based on the student homework quality obtained from the single course homework quality evaluation module and the student overall learning ability from the overall learning ability evaluation module, we calculate the changes in students' problem-solving ability and learning ability during the answering process, and display the learning trajectory of students in a personalized way for different students. The specific implementation steps are:

[0184] (3-1) Extract the highest quality score of each student's homework calculated in module 1 from the system student homework process log database, and draw a line graph to show the changes in problem-solving ability;

[0185] (3-2) Extract the student's learning status score and overall learning ability score calculated by module 2 from the system's student learning ability log database, and use a line chart to display the student's learning status and learning ability changes;

[0186] (3-3) Use drawing tools to provide visualization results to teachers or students through the system front-end page.

[0187] The technical solution provided by the present invention is further described in detail above through several specific implementation modes in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the several specific implementation modes described above are not intended to be used as limitations on the present invention. Any reasonable modification and improvement of the present invention, combination of implementation modes and equivalent substitution within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A learning effect evaluation system for an artificial intelligence image generation training course, including multiple modules for collecting and analyzing student learning process data, characterized by: The system comprises: The student single course assignment quality evaluation module is used to evaluate the quality of students' single assignments in the AI ​​image generation course and obtain quality scores; The overall learning ability evaluation module conducts a comprehensive learning status analysis on the quality scores of multiple assignments and the assignment process logs of students to obtain the overall learning ability scores of students; The visual feedback module is used to generate student learning diaries based on changes in student homework quality and learning ability.

2. The learning effect evaluation system for artificial intelligence image generation training courses according to claim 1 is characterized by: The student single course homework quality evaluation module is based on the feature extraction method of the pre-trained deep model, and uses the feature allocation distance between the target image and the image generated by the single homework to calculate the similarity and obtain the quality score.

3. The learning effect evaluation system for artificial intelligence image generation training courses according to claim 2 is characterized by: The feature assignment distance is implemented by Frechette revelation distance.

4. The learning effect evaluation system for artificial intelligence image generation training courses according to claim 1 is characterized by: The overall learning ability evaluation module uses a decision tree model built based on the gradient boosting tree algorithm to evaluate the learning status of students at different stages.

5. The learning effect evaluation system for artificial intelligence image generation training courses according to claim 4 is characterized by: The overall learning ability evaluation module accumulates the scores of students' learning status in different learning stages by exponential weight accumulation, so as to obtain the students' overall learning ability scores.

6. The learning effect evaluation system for artificial intelligence image generation training courses according to claim 1 is characterized by: The visual feedback module provides personalized visual feedback on students' learning diaries and provides it to students and teachers through the system's front display page.

7. A learning effect evaluation method for an artificial intelligence image generation training course, characterized by: The method is implemented based on the system according to claim 1, and the method comprises: The quality evaluation step of a student's single course assignment is to evaluate the quality of a student's single assignment in the AI ​​image generation course and obtain a quality score; The overall learning ability evaluation module conducts a comprehensive learning status analysis on the quality scores of multiple assignments and the assignment process logs of students to obtain the overall learning ability scores of students; The visual feedback module generates student learning diaries based on changes in student homework quality and learning ability.

8. A computer storage medium for storing a computing program, characterized in that: When the computer reads the computer program, the computer executes the method of claim 7.

9. A computer, comprising a processor and a storage medium, characterized in that: When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 7 .

10. A computer program product, being a computer program, characterized in that When the computer program is executed, the method of claim 7 is implemented.

Images