An online course recommendation method and device based on class concentration detection

Abstract

The application discloses an online course recommendation method and device based on class concentration detection, which first collects basic information data of students, vectorizes the basic data of the students, and generates an initial recommendation list by using a user-based collaborative filtering method; then, concentration detection is performed on video records of the students during class, the proportion of the concentration state is calculated, and the proportion is taken as a score of the students on the course; finally, the existing score is converted into a matrix form, a recommendation result is obtained by using a matrix decomposition-based collaborative filtering method, and the recommendation result is fused with the initial recommendation result to obtain a final course recommendation result. The application combines the concentration of the students during class, can better feedback the interest degree of the students on the course, and obtains a more accurate student-course score; and the application can improve the sparsity of the student-course score matrix, and has a better effect when the matrix decomposition-based collaborative filtering method is applied.

Description

Technical Field

[0001] This invention relates to the field of recommendation algorithm technology, specifically to the recommendation field in the context of course recommendation, and particularly to an online course recommendation method and apparatus based on classroom attention detection. Background Technology

[0002] Currently, online education in my country is developing rapidly, with a series of online education platforms growing significantly. Meanwhile, during periods when going out is inconvenient, universities across the country have shifted most courses from in-person to online, making online learning a popular learning method. With the development of online education, the number of courses and students is increasing, and how to accurately match students' interests and recommend courses has become a research topic for experts.

[0003] Unlike video and product recommendation scenarios where users have explicit ratings for items, in course recommendation scenarios, students' learning records vary widely after course selection. Many students have no learning records at all, and student selection alone cannot reflect their course preferences. Furthermore, statistics show that less than 10% of students rate courses after completion, leading to a lack of student-course ratings and data sparsity. This makes traditional collaborative filtering methods perform poorly in this environment and also suffers from cold start problems. Currently, most scholars analyze student course preferences through learning behaviors such as video viewing logs and problem-solving records, and propose course recommendation methods based on this. However, these methods only consider features such as the length of videos watched and rewatch rate, without considering the student's state of mind during viewing. Summary of the Invention

[0004] The purpose of this invention is to address the problems of lack of student-course ratings and data sparsity in the existing technology by proposing an online course recommendation method and device based on classroom attention detection.

[0005] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides an online course recommendation method based on classroom attention detection, comprising the following steps:

[0006] (1) Collect basic information data of students, including their gender, age, major, hobbies and courses they are currently studying;

[0007] (2) Use a user-based collaborative filtering method to obtain the student's recommendation list, and use the recommendation list as the initial result of the student's course recommendation;

[0008] (3) Collect students' course selection records and their corresponding video recordings of class sessions;

[0009] (4) For each course, the ResNet(18)+TCN model is used to detect the students’ focus on the course based on the collected video recording information, so as to obtain the score of each student for each course, and obtain the student-course score matrix based on the score of each student for each course.

[0010] (5) The student’s rating for each course is calculated using the collaborative filtering method of matrix factorization based on the student-course rating matrix. The rating is then merged with the student’s initial course recommendation results and sorted in descending order to obtain the student’s course recommendation results.

[0011] Furthermore, step (2) includes the following sub-steps:

[0012] (2.1) Use one-hot encoding to vectorize the basic information data of each student to obtain the student's feature vector;

[0013] (2.2) The cosine similarity between students is calculated based on their feature vectors. The expression is as follows:

[0014]

[0015] Where sim(i,j) represents the similarity between student i and student j, v i Let v be the feature vector of student i. j Let j be the feature vector of student j;

[0016] (2.3) Based on the similarity between each pair of students, the following formula is used to calculate the student's rating for each course, in order to obtain the student's rating for all courses:

[0017]

[0018] Among them, R ic Let U be the student i's rating of course c, and let U be the set of students who selected course c. j This represents the j-th student in set U;

[0019] (2.4) For each student, sort the student’s ratings of all courses in descending order, and use the list of course ratings in descending order as the initial result of the student’s course recommendations.

[0020] Furthermore, step (4) includes the following sub-steps:

[0021] (4.1) For each course, the collected video recording information is input into the ResNet(18)+TCN model to obtain the focus sequence;

[0022] (4.2) Merge the student’s attention sequence for all videos of each course, calculate the proportion of the number of times “focus” appears in the merged attention sequence, and use it as the student’s score for the course to obtain the score of each student for each course.

[0023] (4.3) Obtain the student-course rating matrix based on each student's rating of each course.

[0024] Furthermore, the ResNet(18)+TCN model includes 17 convolutional layers, 1 fully connected layer, and an LSTM layer. The input of the ResNet(18)+TCN model is the image in the video recording information of the student during class, and the output of the ResNet(18)+TCN model is the result of being focused or not focused.

[0025] Furthermore, step (5) includes the following sub-steps:

[0026] (5.1) The student-course rating matrix is ​​transformed into a student-rating matrix using a collaborative filtering method based on matrix factorization. The transformation process is as follows: Assuming the number of students is M and the number of courses is N, the rating of the i-th student for the j-th course is the element in the i-th row and j-th column of the student-rating matrix; when the m-th student has no rating information for the n-th course, the element in the m-th row and n-th column of the student-rating matrix is ​​0.

[0027] (5.2) The least squares method is used to decompose the student-rating matrix into two smaller matrices X and Y with size M×N, respectively. The mean square error is used as the loss function, and the loss function is minimized to obtain the updated matrices X and Y.

[0028] (5.3) Multiply the updated matrices X and Y to obtain the updated student-course rating matrix, and merge it with the initial results of the students' course recommendations to obtain the students' final ratings for each course;

[0029] (5.4) For each student, sort the student’s final ratings for each course in descending order to obtain the N courses with the highest ratings, and use them as the student’s course recommendation results.

[0030] Furthermore, the expression for the loss function is:

[0031]

[0032] Where Loss(X,Y) represents the loss function, r ui X represents the element in the u-th row and i-th column of the original student-rating matrix R. u Let Y represent the vector in the u-th row of matrix X.i Represents the i-th column vector in matrix Y;

[0033] Minimizing the loss function to obtain the updated matrices X and Y specifically includes the following sub-steps:

[0034] (5.2.1) Initialize matrices X and Y;

[0035] (5.2.2) Solve for u = 1 up to u = n. Get x u x1~x n The new matrix X is obtained by splicing the two matrices together.

[0036] (5.2.3) Solve for i = 1 up to i = m. Get y i y1~y n The new matrix Y is obtained by splicing the two matrices together.

[0037] (5.2.4) Calculate the new loss based on the obtained new matrices X and Y;

[0038] (5.2.5) Repeat steps (5.2.2)-(5.2.4) until the loss converges.

[0039] A second aspect of the present invention provides an online course recommendation device based on classroom attention detection, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described online course recommendation method based on classroom attention detection.

[0040] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-described online course recommendation method based on classroom attention detection.

[0041] The beneficial effects of this invention are that it combines students' attention during class to better reflect their interest in the course, resulting in more accurate student-course ratings; and the online course recommendation method of this invention improves the sparsity problem of the student-course rating matrix, making it more effective when applying matrix factorization-based collaborative filtering methods. Attached Figure Description

[0042] Figure 1 This is a flowchart of the online course recommendation method based on classroom attention detection according to the present invention;

[0043] Figure 2 This is a flowchart of the user-based collaborative filtering method of the present invention;

[0044] Figure 3 This is a flowchart of the least squares matrix decomposition method of the present invention;

[0045] Figure 4 This is a schematic diagram of an online course recommendation device based on classroom attention detection according to the present invention. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not intended to limit this application.

[0047] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0048] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to determination," or "includes." Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process or method. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0049] See Figure 1 The online course recommendation method based on classroom attention detection of the present invention specifically includes the following steps:

[0050] (1) Collect basic information data of students. The basic information data includes, but is not limited to: the student's gender, age, major, hobbies and courses being studied, etc.

[0051] (2) Use a user-based collaborative filtering method to obtain the student's recommendation list and use the recommendation list as the initial result of the student's course recommendation.

[0052] Specifically, a user-based collaborative filtering method is used to obtain a student recommendation list, that is, to recommend courses that students with similar majors, interests, etc., like these. Figure 2 As shown, the specific steps include:

[0053] (2.1) Use the one-hot encoding method to vectorize the basic information data of each student to obtain the student's feature vector.

[0054] Specifically, assuming there are N pieces of basic information data about students, such as gender, major, and hobbies, then each student's basic information data can be represented by an N-dimensional vector u, where the i-th element of the vector ui is... i Let i represent whether student u has the i-th piece of information; assuming the i-th piece of information represents the student's major as software engineering, then u i =1 indicates that this student has this information, u i =0 indicates that the student does not exist.

[0055] (2.2) The cosine similarity between students is calculated based on their feature vectors. The expression is as follows:

[0056]

[0057] Where sim(i,j) represents the similarity between student i and student j, v i Let v be the feature vector of student i. j Let be the feature vector of student j.

[0058] (2.3) Based on the similarity between each pair of students, the following formula is used to calculate the student's rating for each course, in order to obtain the student's rating for all courses:

[0059]

[0060] Among them, R ic Let U be the student i's rating of course c, and let U be the set of students who selected course c. j Let j represent the j-th student in set U.

[0061] (2.4) For each student, sort the student’s ratings of all courses in descending order, and use the list of course ratings in descending order as the initial result of the student’s course recommendations.

[0062] (3) Collect students’ course selection records and their corresponding video recordings of class sessions.

[0063] (4) For each course, the ResNet(18)+TCN model is used to detect the students’ focus on the course based on the collected video recording information, so as to obtain the score of each student for each course, and obtain the student-course score matrix based on the score of each student for each course.

[0064] It should be understood that each student's rating for each course refers to the student's level of focus on that course.

[0065] (4.1) For each course, the collected video recording information is input into the ResNet(18)+TCN model to obtain the attention sequence.

[0066] It should be understood that the ResNet(18)+TCN model is a model for participation detection in videos. Its input is a sequence of L original video frames, and its output is the order category detected based on the student's participation in the video.

[0067] In this embodiment, the ResNet(18)+TCN model contains 17 convolutional layers, 1 fully connected layer, and an LSTM layer. The input to the ResNet(18)+TCN model is an image from a specific moment in a video recording of a student attending class, i.e., a sequence of L original video frames. The output is a focus / non-focus result. Similarly, the ResNet(18)+TCN model is used to detect the student's state. For every 10 frames of the video, a focus or non-focus result is obtained. For the video recording information of each course, a focus sequence can be obtained, which includes the focus or non-focus result. Testing showed that the model achieved a recognition accuracy of 70% on the public dataset DaiSEE.

[0068] (4.2) Merge the student’s attention sequence for all videos of each course, calculate the proportion of the number of times “focus” appears in the merged attention sequence, and use it as the student’s score for the course to obtain the score of each student for each course.

[0069] (4.3) Obtain the student-course rating matrix based on each student's rating of each course.

[0070] It should be understood that once the student's ratings for all courses are obtained, the matrix consisting of a student's ratings for all courses is the student-course rating matrix for that student.

[0071] (5) The student’s rating for each course is calculated using the collaborative filtering method of matrix factorization based on the student-course rating matrix. The rating is then merged with the student’s initial course recommendation results and sorted in descending order to obtain the student’s course recommendation results.

[0072] It should be understood that the recommended list is obtained after sorting in descending order, which is the course recommendation result for the student. Based on the course recommendation result, courses that the student is interested in are recommended.

[0073] (5.1) The student-course rating matrix is ​​transformed into a student-rating matrix using a collaborative filtering method based on matrix factorization. The transformation process is as follows: Assuming the number of students is M and the number of courses is N, the rating of the i-th student for the j-th course is the element in the i-th row and j-th column of the student-rating matrix; when the m-th student has no rating information for the n-th course, the element in the m-th row and n-th column of the student-rating matrix is ​​0.

[0074] (5.2) The student-rating matrix is ​​decomposed using the least squares method to decompose the M×N student-rating matrix into two smaller matrices X and Y with sizes of M×k and k×N respectively. The mean squared error (MSE) is used as the loss function, and the loss function is minimized to obtain the updated matrices X and Y.

[0075] In this embodiment, the product of the two smaller matrices is approximately equal to the original student-rating matrix, that is:

[0076] R M×N ≈X M×k ·Y k×N

[0077] Among them, R M×N For the original student-rating matrix, X M×k and Y k×N These are the two smaller matrices obtained after decomposition.

[0078] In this embodiment, the expression for the loss function is:

[0079]

[0080] Where Loss(X,Y) represents the loss function, r ui X represents the element in the u-th row and i-th column of the original student-rating matrix R. u Let Y represent the vector in the u-th row of matrix X. i Let represent the i-th column vector in matrix Y.

[0081] In this embodiment, the mean squared error (MSE) is used as the loss function. The loss function is minimized to obtain the updated matrices X and Y, as follows: Figure 3 As shown, the specific steps include the following:

[0082] (5.2.1) Initialize matrices X and Y.

[0083] (5.2.2) Solve for u = 1 up to u = n. Get x u x1~x n The new matrix X is obtained by splicing the matrices together.

[0084] (5.2.3) Solve for i = 1 up to i = m. Get y i y1~y n The new matrix Y is obtained by splicing the matrices together.

[0085] (5.2.4) Calculate the new loss based on the new matrices X and Y.

[0086] (5.2.5) Repeat steps (5.2.2)-(5.2.4) until the loss converges.

[0087] (5.3) Multiply the updated matrices X and Y to obtain the updated student-course rating matrix, and merge it with the initial results of the students' course recommendations to obtain the students' final ratings for each course.

[0088] Specifically, the updated student-course rating matrix is ​​obtained by multiplying the updated matrices X and Y. The initial course recommendation result of the student obtained in step (2) is then fused with the updated student-course rating matrix. That is, the student's rating for the course in step (2) is added to the student's rating for the course in the current student-course rating matrix to obtain the student's final rating for the course.

[0089] (5.4) For each student, sort the student’s final ratings for each course in descending order to obtain the N courses with the highest ratings, and use them as the student’s course recommendation results.

[0090] Corresponding to the aforementioned embodiments of the online course recommendation method based on classroom attention detection, the present invention also provides embodiments of an online course recommendation device based on classroom attention detection.

[0091] See Figure 4The present invention provides an online course recommendation device based on classroom attention detection, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the online course recommendation method based on classroom attention detection in the above embodiment.

[0092] The embodiments of the online course recommendation device based on classroom attention detection of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4 The diagram shown is a hardware structure diagram of any data processing-capable device that houses the online course recommendation device based on classroom attention detection according to the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0093] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0094] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0095] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the online course recommendation method based on class attention detection described in the above embodiments.

[0096] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0097] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An online course recommendation method based on classroom attention detection, characterized in that, Includes the following steps: (1) Collect basic information data of students, including their gender, age, major, hobbies and courses they are currently studying; (2) Use a user-based collaborative filtering method to obtain the student's recommendation list, and use the recommendation list as the initial result of the student's course recommendation; (3) Collect students' course selection records and corresponding video recordings of their classes; (4) For each course, the ResNet(18)+TCN model is used to detect the students' focus on the course based on the collected video recording information, so as to obtain the score of each student for each course, and to obtain the student-course score matrix based on the score of each student for each course; step (4) includes the following sub-steps: (4.1) For each course, the collected video recording information is input into the ResNet(18)+TCN model to obtain the focus sequence; the ResNet(18)+TCN model includes 17 convolutional layers, 1 fully connected layer and LSTM layer. The input of the ResNet(18)+TCN model is the image in the video recording information of the student during class. The output of the ResNet(18)+TCN model is the result of focus or non-focus. (4.2) Merge the attention sequences of all videos for each course and calculate the proportion of the number of times "focus" appears in the merged attention sequence. Use this as the student's score for the course to obtain the score of each student for each course. (4.3) Obtain the student-course rating matrix based on each student's rating of each course; (5) The student’s rating for each course is calculated using the collaborative filtering method of matrix factorization based on the student-course rating matrix, and then merged with the student’s initial course recommendation results and sorted in descending order to obtain the student’s course recommendation results.

2. The online course recommendation method based on classroom attention detection according to claim 1, characterized in that, Step (2) includes the following sub-steps: (2.1) Use one-hot encoding to vectorize the basic information data of each student to obtain the student's feature vector; (2.2) The cosine similarity between students is calculated based on their feature vectors. The expression is as follows: ； in, This represents the similarity between student i and student j. Let i be the feature vector of student i. Let j be the feature vector of student j; (2.3) Based on the similarity between each pair of students, the following formula is used to calculate the student's rating for each course, in order to obtain the student's rating for all courses: ； in, For student i, rate course c. U This represents the set of students who selected course c. This set U The Middle j 100 students; (2.4) For each student, sort the student’s ratings of all courses in descending order, and use the list of course ratings in descending order as the initial result of the student’s course recommendations.

3. The online course recommendation method based on classroom attention detection according to claim 1, characterized in that, Step (5) includes the following sub-steps: (5.1) The student-course rating matrix is ​​transformed into a student-rating matrix using a matrix factorization collaborative filtering method. The transformation process is as follows: Assuming that the number of students is M and the number of courses is N, the rating of the i-th student for the j-th course is the element in the i-th row and j-th column of the student-rating matrix; when the m-th student has no rating information for the n-th course, the element in the m-th row and n-th column of the student-rating matrix is ​​0. (5.2) Perform matrix decomposition on the student-rating matrix using the least squares method to reduce the matrix size to a minimum. The student-rating matrix is ​​decomposed into a matrix of size . and Given two small matrices X and Y, the mean squared error is used as the loss function. The loss function is minimized to obtain the updated matrices X and Y. (5.3) Multiply the updated matrices X and Y to obtain the updated student-course rating matrix, and merge it with the initial results of student course recommendations to obtain the final rating of each student for each course; (5.4) For each student, sort the student’s final ratings for each course in descending order to obtain the N courses with the highest ratings, and use them as the student’s course recommendation results.

4. The online course recommendation method based on classroom attention detection according to claim 3, characterized in that, The expression for the loss function is: ； in, Represents the loss function. This represents the element in the i-th row and u-th column of the original student-rating matrix R. Let the vector in the i-th row of matrix X be denoted as . Represents the u-th column vector in matrix Y; Minimizing the loss function to obtain the updated matrices X and Y specifically includes the following sub-steps: (5.2.1) Initialize matrices X and Y; (5.2.2) Solve for u=1 up to u=n. get ,Will The new matrix X is obtained by splicing the two matrices together. (5.2.3) Solve for i=1 up to i=m. get ,Will The new matrix Y is obtained by splicing the two matrices together. (5.2.4) Calculate the new loss based on the obtained new matrices X and Y; (5.2.5) Repeat steps (5.2.2)-(5.2.4) until the loss converges.

5. An online course recommendation device based on classroom attention detection, comprising one or more processors and a memory, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the online course recommendation method based on class attention detection as described in any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, is used to implement the online course recommendation method based on class attention detection as described in any one of claims 1-4.

Images