An online intelligent teaching system based on multimodal shared representation learning

Abstract

The present invention discloses an online intelligent teaching system based on multimodal shared representation learning, which specifically relates to the field of online intelligent teaching technology, including a user learning preference module, a user learning ability module, a course test and evaluation data module, and a course recommendation model; the user learning preference module is used to collect the user's learning habit preference data in terms of learning type, learning time period, course presentation form, etc.; the user learning ability module is used to build a user memory model, including a concentration test and a memory rule test, and collect the user's personal learning ability data; the course test and evaluation data module is used to obtain the user's post-evaluation score and points, and determine whether the user enters the next chapter of learning. The present invention realizes the evaluation of the subjective and objective learning ability of online users, intelligently generates courses that meet the user's learning ability, and promotes users to learn better and faster.

Description

Technical Field

[0001] The present invention relates to the technical field of online intelligent teaching, and in particular to an online intelligent teaching system based on multimodal shared representation learning. Background Art

[0002] With the continuous development of digitalization and information technology, online smart teaching has gradually attracted widespread attention as a modern and flexible teaching method. With the help of the Internet and digital tools, online smart teaching provides students with a more personalized and diversified learning experience. Through online platforms, virtual classrooms and interactive content, students can choose their own learning time and place, and adjust their learning progress and methods according to their own needs. This teaching method breaks the time and space limitations of traditional learning and promotes knowledge sharing and cooperation on a global scale. Online smart teaching has obvious advantages in providing personalized learning, rich resources and flexibility, but it also faces challenges such as reduced face-to-face communication, high technology dependence and self-discipline requirements. Summary of the invention

[0003] In order to solve the problems faced by existing online intelligent teaching, such as reduced face-to-face communication, high technology dependence and self-discipline requirements, the present invention provides an online intelligent teaching system based on multimodal shared representation learning to solve the problems raised in the background technology.

[0004] In order to achieve the above-mentioned object, the present invention provides the following technical solutions: an online intelligent teaching system based on multimodal shared representation learning, the teaching system at least comprising a user learning preference module, a user learning ability module, a course test and evaluation data module and a course recommendation model;

[0005] User learning preference module, used to collect user learning habit preference data in terms of learning type, learning time period, course presentation format, etc.;

[0006] User learning ability module, used to build a user memory model, including concentration test and memory pattern test, and collect user's personal learning ability data;

[0007] The course test and evaluation data module is used to obtain the user's test scores and points, and decide whether the user should proceed to the next chapter;

[0008] The user learning preference module, user learning ability module, course test and evaluation data module are used as modal input data, and courses that meet the user's learning ability are generated through the course recommendation model; users who complete courses that meet the user's learning ability will receive user level upgrades, course tests and evaluations, and the course test and evaluation data are used as input data for the course test and evaluation data module;

[0009] Among them, the course recommendation model is designed as follows:

[0010] Input data set D, represented as D = [(x r ,y r ),(x a ,y a ),(x s ,y s )], where (x r ,y r ) represents learning habit preference data, (x a ,y a ) represents learning ability data, (x s ,y s ) represents the course test and evaluation data. Each mode is associated with a prediction function, which is expressed as f m =g·h m , where m is {r, a, s}, r represents learning preference data, a represents learning ability data, and s represents course test and evaluation data, representing three modes; where function h m As a specific encoder, the function g represents the shared head among all modalities;

[0011] Given T total training steps, the model receives data from only one modality per iteration;

[0012] In each training step t∈T, the corresponding unimodal data is minimized The predicted risk L of the training set in t To iteratively optimize the course recommendation model:

[0013]

[0014] Among them, (x, y) in the above formula refers to the input data and labels. and are the encoders at time point t respectively. and the learnable parameters of the shared head g, l represents the loss function; represents the prediction function at time point t; m t represents the mode under t step size;

[0015] After learning the data of the three modalities, modal fusion is performed by learning cross-modal information: the course recommendation model uses a shared head g in all given modalities, so that cross-modal interaction information can be captured throughout the process;

[0016] After the course recommendation model is optimized, its prediction process is as follows:

[0017] For a given test example (x, y), the prediction is calculated as follows:

[0018]

[0019] Among them, the values ​​of m are 1, 2, and 3, 1 represents learning preference data r, 2 represents learning ability data a, and 3 represents course test and evaluation data s; represents the prediction function under three modes, indicating the importance of mode m in predicting labels, λ m The calculation is as follows:

[0020]

[0021] in, e v With e m They all represent different modes, e m represents x using the entropy of each individual modal output, e m -e v To calculate the difference in entropy between the three modes, the Softmax function converts the output logarithm into probability Represents the prediction function for the total set with a variable number of modalities.

[0022] Preferably, a multidimensional learning habit assessment scale is designed to collect user learning preference data;

[0023] The multidimensional learning habit assessment scale is designed as follows:

[0024] Learning type preference: text, voice, picture, text and voice combination, text and picture combination, voice and picture combination, text, voice and picture combination;

[0025] Study time preference: 8:00-10:00, 10:00-12:00, 14:00-16:00, 16:00-18:00, 19:00-20:00;

[0026] Preference for course presentation format: self-study, seminar, practical training, question-and-answer;

[0027] By having users complete a multi-dimensional learning habit assessment scale, we collect data on users’ learning preferences in terms of learning type, learning time period, and course presentation format.

[0028] Preferably, a learning ability assessment method is designed to collect the user's personal learning ability data;

[0029] The learning ability assessment method is designed as follows:

[0030] Build a user memory model, including concentration test and memory regularity test;

[0031] Concentration test: calculate the accuracy of the user clicking the target within a given time t;

[0032] Memory pattern test: observe the position of a group of objects or symbols, then restore or confirm the position on the screen and calculate the accuracy.

[0033] Preferably, the user level upgrade is designed as follows:

[0034] Completing the current teaching task will earn you a medal and upgrade your grade as part of your regular performance evaluation. The medals include:

[0035] Primary Medal: Completed the course, but the time spent on learning was unreasonable, the completion degree was poor, and it could not be upgraded;

[0036] Intermediate Badge: Complete the course with a reasonable time and a reasonable degree of completion, and be promoted to the next level;

[0037] Advanced Medal: Complete the course with a reasonable time and high completion rate, and upgrade two levels;

[0038] Among them, learning time: define the maximum completion time of each chapter Q max and the minimum completion time Q min , if learning time Q min or Q>Q max , then the evaluation is unreasonable, otherwise it is reasonable;

[0039] Completion degree: Each course chapter includes several pages of courseware, including basic knowledge, difficult knowledge, and extended knowledge. If only basic knowledge is completed or the time spent is not within a reasonable range, it will be evaluated as unreasonable completion degree; if basic knowledge and difficult knowledge are completed and the time spent is within a reasonable range, it will be evaluated as reasonable completion degree; if basic knowledge, difficult knowledge, and extended knowledge are completed and the time spent is within a reasonable range, it will be evaluated as a high degree of completion;

[0040] Level: Failure to upgrade indicates lower-level scores, and the online learning scores are S1 (S1>0); upgrading one level indicates middle-level scores, and the online learning scores are S2 (S2>S1); upgrading two levels indicates upper-level scores, and the online learning scores are S3 (S3>S2). The total score of online learning is NP score Add up and average the scores for all chapters, as shown below:

[0041]

[0042] In the formula, NP score is the total score of online learning, N i (i=1,2,...,n) is the chapter score of chapter i.

[0043] ​Preferably, the course test and evaluation design is as follows:

[0044] After completing the chapter study, questions are randomly selected from the question bank for assessment, and students are required to score the chapter study type, study time period, and course performance. Points are distributed according to the assessment results and the completion of the scoring. Users with an assessment score of S2 or more and a completed course evaluation can obtain points. After obtaining points, they can proceed to the next chapter study. Otherwise, students are required to retake the question assessment. If they fail to obtain points for three consecutive times, they will be reminded to restudy the chapter.

[0045] The present invention has the following advantages:

[0046] (1) Through the evaluation of user learning habits and user concentration and memory patterns, the subjective and objective learning ability of online users can be evaluated, and courses that meet the user's learning ability can be intelligently generated to promote better and faster learning of users;

[0047] (2) The designed course recommendation model based on multimodal shared representation learning can capture rich representations of all available modalities, the number of which can vary (not limited to the above-mentioned modalities), while avoiding the multimodal model only learning the dominant modality information (i.e., modality laziness). At the same time, the model does not require paired multimodal data during the training phase, which makes it naturally suitable for scenarios with extreme modality laziness, such as learning with missing modalities.

[0048] (3) By upgrading medal levels and unlocking courses with points, students are encouraged to learn scientifically and proactively, and their independent learning ability is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the overall framework diagram of the present invention;

[0050] Figure 2 It is a schematic diagram of the model optimization process of the present invention;

[0051] Figure 3 A schematic diagram of the online course intelligent generation process of the present invention;

[0052] Figure 4 A schematic diagram of the medal level upgrade process of the present invention;

[0053] Figure 5 This is a schematic diagram of the process of unlocking courses with points according to the present invention. DETAILED DESCRIPTION

[0054] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Example 1: Figure 1 As shown, this embodiment provides an online intelligent teaching system based on multimodal shared representation learning, and the teaching system at least includes a user learning preference module, a user learning ability module, a course test and evaluation data module, and a course recommendation model;

[0056] User learning preference module, used to collect user learning habit preference data in terms of learning type, learning time period, course presentation format, etc.;

[0057] Specifically, Figure 3 As shown, a multidimensional learning habit assessment scale is designed to collect user learning preference data;

[0058] The multidimensional learning habit assessment scale is designed as follows:

[0059] Learning type preference: text, voice, picture, text and voice combination, text and picture combination, voice and picture combination, text, voice and picture combination;

[0060] Study time preference: 8:00-10:00, 10:00-12:00, 14:00-16:00, 16:00-18:00, 19:00-20:00;

[0061] Preference for course presentation format: self-study, seminar, practical training, question-and-answer;

[0062] By having users complete a multi-dimensional learning habit assessment scale, we collect data on users’ learning preferences in terms of learning type, learning time period, and course presentation format.

[0063] User learning ability module, used to build user memory model, including concentration test and memory rule test, and collect user's personal learning ability data;

[0064] Specifically, a learning ability assessment method is designed to collect the user's personal learning ability data;

[0065] The learning ability assessment method is designed as follows:

[0066] Build a user memory model, including concentration test and memory regularity test;

[0067] Concentration test: calculate the accuracy of the user clicking the target within a given time t;

[0068] Memory pattern test: observe the position of a group of objects or symbols, then restore or confirm the position on the screen and calculate the accuracy.

[0069] The course test and evaluation data module is used to obtain the user's test scores and points, and decide whether the user should proceed to the next chapter;

[0070] The user learning preference module, user learning ability module, course test and evaluation data module are used as modal input data, and courses that meet the user's learning ability are generated through the course recommendation model; users who complete courses that meet the user's learning ability will obtain user level upgrades, course tests and evaluations, and the course test and evaluation data are used as input data for the course test and evaluation data module.

[0071] Specifically, Figure 4 As shown, the user level upgrade design is as follows:

[0072] Completing the current teaching task will earn you a medal and upgrade your grade as part of your regular performance evaluation. The medals include:

[0073] Primary Medal: Completed the course, but the time spent on learning was unreasonable, the completion degree was poor, and it could not be upgraded;

[0074] Intermediate Badge: Complete the course with a reasonable time and a reasonable degree of completion, and be promoted to the next level;

[0075] Advanced Medal: Complete the course with a reasonable time and high completion rate, and upgrade two levels;

[0076] Among them, learning time: define the maximum completion time of each chapter Q max and the minimum completion time Q min , if learning time Q min or Q>Q max , then the evaluation is unreasonable, otherwise it is reasonable;

[0077] Completion degree: Each course chapter includes several pages of courseware, including basic knowledge, difficult knowledge, and extended knowledge. If only basic knowledge is completed or the time spent is not within a reasonable range, it will be evaluated as unreasonable completion degree; if basic knowledge and difficult knowledge are completed and the time spent is within a reasonable range, it will be evaluated as reasonable completion degree; if basic knowledge, difficult knowledge, and extended knowledge are completed and the time spent is within a reasonable range, it will be evaluated as a high degree of completion;

[0078] ​Level: Failure to upgrade means lower-level scores, with online learning scores of 60 points; upgrading one level means middle-level scores, with online learning scores of 80 points; upgrading two levels means upper-level scores, with online learning scores of 100 points. score Add up and average the scores for all chapters, as shown below:

[0079]

[0080] In the formula, NP score is the total score of online learning, N i (i=1,2,...,n) is the chapter score of chapter i.

[0081] Specifically, Figure 5 As shown, the course test and evaluation design is as follows:

[0082] After completing the chapter study, questions are randomly selected from the question bank for assessment, and students are required to score the chapter study type, study time period, and course performance. Points are awarded based on the assessment results and the completion of the scoring. Users with an assessment score of 80 points or more and a completed course evaluation can obtain points. After obtaining points, they can proceed to the next chapter study. Otherwise, students are required to retake the question assessment. If they fail to obtain points for three consecutive times, they will be reminded to restudy the chapter.

[0083] like Figure 2 As shown, the course recommendation model is designed as follows:

[0084] Input data set D, represented as D = [(x r ,y r ),(x a ,y a ),(x s ,y s )], where (x r ,y r ) represents learning habit preference data, (x a ,y a ) represents learning ability data, (x s ,y s ) represents the course test and evaluation data. Each mode is associated with a prediction function, which is expressed as f m =g·h m , where m is {r, a, s}, r represents learning preference data, a represents learning ability data, and s represents course test and evaluation data, representing three modes; where function h m As a specific encoder, the function g represents the shared head among all modalities;

[0085] Given T total training steps, the model receives data from only one modality per iteration;

[0086] In each training step t, the corresponding unimodal data is minimized The predicted risk L of the training set in t To iteratively optimize the course recommendation model:

[0087]

[0088] Among them, (x, y) in the above formula refers to the input data and labels. and are the encoders at time point t respectively. and the learnable parameters of the shared head g, l represents the loss function; represents the prediction function at time point t; m t represents the mode under t step size;

[0089] After learning the data of the three modalities, modal fusion is performed by learning cross-modal information: the course recommendation model uses a shared head g in all given modalities, so that cross-modal interaction information can be captured throughout the process;

[0090] After the course recommendation model is optimized, its prediction process is as follows:

[0091] For a given test example (x, y), the prediction is calculated as follows:

[0092]

[0093] Among them, the values ​​of m are 1, 2, and 3, 1 represents learning preference data r, 2 represents learning ability data a, and 3 represents course test and evaluation data s; represents the prediction function under three modes, λ m represents the importance of modality m in predicting labels, λ m The calculation is as follows:

[0094]

[0095] in, e v With e m They all represent different modes, e m represents x using the entropy of each individual modal output, e m -e v To calculate the difference in entropy between the three modes, the Softmax function converts the output logarithm into probability Represents the prediction function for the total set with a variable number of modalities.

[0096] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An online intelligent teaching system based on multimodal shared representation learning, characterized by: The teaching system at least includes a user learning preference module, a user learning ability module, a course test and evaluation data module, and a course recommendation model; User learning preference module, used to collect user learning habit preference data in terms of learning type, learning time period, course presentation format, etc.; User learning ability module, used to build a user memory model, including concentration test and memory pattern test, and collect user's personal learning ability data; The course test and evaluation data module is used to obtain the user's test scores and points, and decide whether the user should proceed to the next chapter; The user learning preference module, user learning ability module, course test and evaluation data module are used as modal input data, and courses that meet the user's learning ability are generated through the course recommendation model; users who complete courses that meet the user's learning ability will receive user level upgrades, course tests and evaluations, and the course test and evaluation data are used as input data for the course test and evaluation data module; Among them, the course recommendation model is designed as follows: Input data set D, represented as D = [(x r ,y r ),(x a ,y a ),(x s ,y s )], where (x r ,y r ) represents learning habit preference data, (x a ,y a ) represents learning ability data, (x s ,y s ) represents the course test and evaluation data. Each mode is associated with a prediction function, which is expressed as f m =g·h m , where m is {r, a, s}, r represents learning preference data, a represents learning ability data, and s represents course test and evaluation data, representing three modes; where function h m As a specific encoder, the function g represents the shared head among all modalities; Given T total training steps, the model receives data from only one modality per iteration; In each training step t∈T, the corresponding unimodal data is minimized The predicted risk L of the training set in t To iteratively optimize the course recommendation model: Among them, (x, y) in the above formula refers to the input data and labels. and are the encoders at time point t respectively. and the learnable parameters of the shared head g, l represents the loss function; represents the prediction function at time point t; m t represents the mode under t step size; After learning the data of the three modalities, modal fusion is performed by learning cross-modal information: the course recommendation model uses a shared head g in all given modalities, so that cross-modal interaction information can be captured throughout the process; After the course recommendation model is optimized, its prediction process is as follows: For a given test example (x, y), the prediction is calculated as follows: Among them, the values ​​of m are 1, 2, and 3, 1 represents learning preference data r, 2 represents learning ability data a, and 3 represents course test and evaluation data s; represents the prediction function under three modes, λ m represents the importance of modality m in predicting labels, λ m The calculation is as follows: in, e v With e m They all represent different modes, e m represents x using the entropy of each individual modal output, e m -e v To calculate the difference in entropy between the three modes, the Softmax function converts the output logarithm into probability p m , Represents the prediction function for the total set with a variable number of modalities.

2. The online intelligent teaching system based on multimodal shared representation learning according to claim 1, characterized in that: Design a multi-dimensional learning habit assessment scale to collect user learning preference data; The multidimensional learning habit assessment scale is designed to include learning type preference, learning time period preference and course presentation form preference. By having users complete the multidimensional learning habit assessment scale, the user's learning preference data in terms of learning type, learning time period and course presentation form is collected.

3. The online intelligent teaching system based on multimodal shared representation learning according to claim 2 is characterized by: Design a learning ability assessment method to collect personal learning ability data of users; The learning ability assessment method is designed as follows: Build a user memory model, including concentration test and memory pattern test.

4. The online intelligent teaching system based on multimodal shared representation learning according to claim 3 is characterized by: The user level upgrade design is as follows: Completing the current teaching task will earn you a medal and upgrade your grade as part of your regular performance evaluation. The medals include: Primary Medal: Completed the course, but the time spent on learning was unreasonable, the completion degree was poor, and it could not be upgraded; Intermediate Badge: Complete the course with a reasonable time and a reasonable degree of completion, and be promoted to the next level; Advanced Medal: Complete the course with a reasonable time and high completion rate, and upgrade two levels; Among them, learning time: define the maximum completion time of each chapter Q max and the minimum completion time Q min , if learning time Q min or Q>Q max , then the evaluation is unreasonable, otherwise it is reasonable;​ Completion degree: Each course chapter includes several pages of courseware, including basic knowledge, difficult knowledge, and extended knowledge. If only basic knowledge is completed or the time spent is not within a reasonable range, it will be evaluated as unreasonable completion degree; if basic knowledge and difficult knowledge are completed and the time spent is within a reasonable range, it will be evaluated as reasonable completion degree; if basic knowledge, difficult knowledge, and extended knowledge are completed and the time spent is within a reasonable range, it will be evaluated as a high degree of completion; Level: Failed to upgrade means lower-level scores, and the online learning scores are S1 (S1>0); Upgrading one level means medium-level scores, and the online learning scores are S2 (S2>S1); Upgrading two levels means upper-level scores, and the online learning scores are S3 (S3>S2); The total online learning scores are NP score Add up and average the scores for all chapters, as shown below: In the formula, NP score is the total score of online learning, N i (i=1,2,...,n) is the chapter score of chapter i.

5. The online intelligent teaching system based on multimodal shared representation learning according to claim 4 is characterized by: The course test and evaluation design is as follows: After completing the chapter study, questions are randomly selected from the question bank for assessment, and students are required to score the learning type, learning time period and course performance of the chapter. Points are awarded based on the assessment results and scoring completion status; users whose assessment scores are greater than or equal to S2 points and who have completed the course evaluation can obtain points, and after obtaining points, they can proceed to the next chapter study. Otherwise, students are required to re-take the question assessment. If points cannot be obtained for three consecutive times, students will be reminded to re-study the chapter.

Images