Student course recommendation method, device and equipment based on face recognition
Through methods based on face recognition and micro-expression recognition, combined with students' current learning status and historical data, the problem of low accuracy in course recommendations on online learning platforms is solved, and more accurate and personalized course recommendations are achieved.
Patent Information
- Application Number
- CN202310529492.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2043-05-11
AI Technical Summary
When recommending courses, existing online learning platforms cannot effectively combine students' current mastery of the course and dynamic feedback during the learning process, resulting in low accuracy of recommendation data.
Using a face recognition-based method, by obtaining students' face images and historical learning video data, using pre-trained face recognition models and micro-expression recognition models, combining course learning trajectory data, the candidate recommended course data set is determined, and then the candidate recommended course data is provided. Final recommended course.
It improves the accuracy of course recommendations and conforms to user learning progress, providing more personalized and dynamic learning suggestions.
Smart Images

Figure CN116561421B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of face recognition technology, and in particular to a student course recommendation method, device and equipment based on face recognition. Background Art
[0002] At present, online learning has become one of the learning methods for many users (such as students, etc.) to acquire more knowledge. When learning online, users need to log in to the online learning platform, and then the online learning platform determines the courses that users can choose for this online learning based on the user's purchase course records and historical learning records.
[0003] However, the course data to be recommended each time a student logs into the online learning platform is comprehensively determined by the above-mentioned historical learning records and course purchase records. It is not possible to effectively combine the students' current mastery of the course and various dynamic feedback in the learning process (such as difficulties or incomprehension in the learning process) to recommend learning course data, resulting in low accuracy of the learning course recommendation data of the online learning platform. Summary of the invention
[0004] The embodiments of the present invention provide a student course recommendation method, apparatus, computer equipment and medium based on face recognition, aiming to solve the problem in the prior art that each time a student logs into an online learning platform, the platform determines the courses that the student can choose for this online learning based on the user's course purchase records and historical learning records, but fails to effectively combine the student's current mastery of the course and various dynamic feedbacks during the learning process, resulting in low accuracy of the learning course recommendation data of the online learning platform.
[0005] In a first aspect, an embodiment of the present invention provides a student course recommendation method based on face recognition, which includes:
[0006] In response to a course recommendation instruction, obtaining a student face image and current course information corresponding to the course recommendation instruction;
[0007] Obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result;
[0008] Acquire student history learning video data corresponding to the first recognition result;
[0009] Obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result;
[0010] Determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship;
[0011] The course learning trajectory data corresponding to the student's historical learning video data is obtained, and a final recommended course data set is determined based on the course learning trajectory data and the candidate recommended course data sets.
[0012] In a second aspect, an embodiment of the present invention further provides a student course recommendation device based on face recognition, which includes:
[0013] A facial image information acquisition unit, configured to respond to a course recommendation instruction and acquire a student facial image and current course information corresponding to the course recommendation instruction;
[0014] A face recognition unit, used to obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result;
[0015] A target video data acquisition unit, used to acquire the student's historical learning video data corresponding to the first recognition result;
[0016] A micro-expression recognition unit, used to obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result;
[0017] A candidate course data acquisition unit, configured to determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship;
[0018] The final course data acquisition unit is used to acquire the course learning trajectory data corresponding to the student's historical learning video data, and determine the final recommended course data set based on the course learning trajectory data and the candidate recommended course data set.
[0019] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method described in the first aspect is implemented.
[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method described in the first aspect above can be implemented.
[0021] The embodiment of the present invention provides a method, device and equipment for recommending student courses based on face recognition, the method comprising: in response to a course recommendation instruction, obtaining a student face image and current course information corresponding to the course recommendation instruction; obtaining a pre-trained face recognition model, inputting the student face image into the face recognition model, and obtaining a first recognition result; obtaining student history learning video data corresponding to the first recognition result; obtaining a pre-trained micro-expression recognition model, inputting a number of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtaining a second recognition result; determining a candidate recommended course data set corresponding to the student face image based on the current course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship; obtaining course learning trajectory data corresponding to the student history learning video data, and determining a final recommended course data set based on the course learning trajectory data and the candidate recommended course data set. The embodiment of the present invention can perform expression recognition and micro-expression recognition based on the learning video recorded during the user's learning process, as well as the user's course learning trajectory data to jointly determine a final recommended course data set that is more accurate and in line with the user's learning progress in multiple dimensions. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.
[0023] Figure 1 A schematic diagram of an application scenario of the student course recommendation method based on face recognition provided by an embodiment of the present invention;
[0024] Figure 2 A flowchart of a student course recommendation method based on face recognition provided by an embodiment of the present invention;
[0025] Figure 3 A schematic block diagram of a student course recommendation device based on face recognition provided by an embodiment of the present invention;
[0026] Figure 4 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.
[0029] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0030] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0031] The embodiment of the present invention provides a student course recommendation method based on face recognition, an apparatus, a computer device and a medium. The student course recommendation method based on face recognition in the embodiment of the present invention is applied to a server, and the server is provided with one or more processors, a memory, and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the student course recommendation method based on face recognition.
[0032] like Figure 1 As shown, Figure 1 Schematic diagram of a scenario of a student course recommendation method based on face recognition in an embodiment of the present invention. The student course recommendation scenario based on face recognition in an embodiment of the present invention includes a server 10 and a plurality of user terminals 20, wherein the server 10 is integrated with a student course recommendation device based on face recognition, and runs a storage medium corresponding to the student course recommendation method based on face recognition to execute the steps of the student course recommendation method based on face recognition.
[0033] Understandably, Figure 1The server 10 in the specific application scenario of the student course recommendation method based on face recognition, or the devices included in the server 10, does not constitute a limitation on the embodiments of the present invention, that is, the number of devices and types of devices included in the specific application scenario of the student course recommendation method based on face recognition, or the number of devices and types of devices included in each device do not affect the overall implementation of the technical solution in the embodiments of the present invention, and can all be regarded as equivalent replacements or derivatives of the technical solution claimed to be protected in the embodiments of the present invention.
[0034] In the embodiment of the present invention, the server 10 is mainly used for: responding to a course recommendation instruction, obtaining a student face image and current learning course information corresponding to the course recommendation instruction; obtaining a pre-trained face recognition model, inputting the student face image into the face recognition model, and obtaining a first recognition result; obtaining student history learning video data corresponding to the first recognition result; obtaining a pre-trained micro-expression recognition model, inputting a number of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtaining a second recognition result; determining a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship; obtaining course learning trajectory data corresponding to the student history learning video data, and determining a final recommended course data set based on the course learning trajectory data and the candidate recommended course data set.
[0035] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the solution of the present invention and does not constitute a limitation on the application scenario of the solution of the present invention. Other application environments may also include Figure 1 More or fewer servers 10 or user terminals 20, or network connection relationships of the servers 10, such as Figure 1 Only one server 10 is shown. It can be understood that the specific application scenario of the student course recommendation method based on face recognition may also include one or more other servers 10, which are not specifically limited here; the server 10 may also include a memory.
[0036] Figure 2 FIG. 1 is a flow chart of a method for recommending student courses based on face recognition provided by an embodiment of the present invention. Figure 2 As shown, the method includes the following steps S110-S160.
[0037] S110. In response to a course recommendation instruction, obtain a student face image and current course information corresponding to the course recommendation instruction.
[0038] In this embodiment, the technical solution is described with the server as the execution subject. The server is the background server corresponding to the online learning platform, and a large amount of learning video data is stored in the server. When the student user uses the user terminal to enter the login account and password and successfully logs in to the online learning platform corresponding to the server, the course recommendation instruction can be triggered by the active operation. For example, there is a virtual operation button for course recommendation on the homepage of the online learning platform. After the student user presses the virtual operation button for course recommendation, the course recommendation instruction can be generated. The server can also automatically generate course recommendation instructions based on each successful login of the student user, that is, the student user uses the user terminal to automatically generate course recommendation instructions after entering the login account and password and successfully logging in to the online learning platform corresponding to the server.
[0039] For example, this application takes a student user A logging into an online learning platform as an example. When the server obtains the course recommendation instruction corresponding to the operation of student user A logging into the online learning platform, the server further obtains the student face image and current learning course information corresponding to student user A. That is, when the course recommendation instruction is detected in the server, a user portrait collection instruction will be sent to the user terminal used by student user A. After the user receives the user portrait collection instruction and authorizes the collection, the student face image of the student user is collected by the user terminal and uploaded to the server by the user terminal. Moreover, after student user A enters the online learning platform, the server can also obtain the name of the course that the user currently clicks to open for study in real time as the current learning course information, or the server can also obtain the course name corresponding to the course that student user A has recently completed online learning as the current learning course information. The student face image and the current learning course information are obtained in the server, which can be used as part of the reference data for the subsequent acquisition of the user's student face image and current learning course information.
[0040] S120, obtaining a pre-trained face recognition model, inputting the student face image into the face recognition model, and obtaining a first recognition result.
[0041] In this embodiment, since a pre-trained face recognition model is stored in the server, after obtaining the student's face image, it can be input into the face recognition model for face recognition to obtain a first recognition result. It can be seen that the user's identity can be quickly determined based on the face recognition method.
[0042] For example, still referring to the above example, after student user A logs in to the online learning platform, the server can obtain the current user identity information (such as user name, user unique identification number, etc.) based on the user's account information, and then compare the current user identity information with the first recognition result obtained based on face recognition. If the current user identity information and the first recognition result both correspond to the same user, it means that student user A himself has logged in to the online learning platform for online learning, and no one else (such as student user B, etc.) has logged in to student user A's account to conduct online learning. After confirming that it is student user A himself who has logged in for online learning, further course recommendations can be made for student user A.
[0043] In one embodiment, step S120 includes:
[0044] Inputting the student's face image into the convolutional neural network corresponding to the face recognition model to extract features, and obtaining the student's face features;
[0045] Based on the feature similarity between the student's facial features and each facial feature in a preset facial feature library, determining a target facial feature having the greatest feature similarity with the student's facial features;
[0046] Obtain user identity information corresponding to the target facial feature as the first recognition result.
[0047] In this embodiment, a convolutional neural network is used as a face recognition model in the server. When the student face image is input into the convolutional neural network, convolution processing and pooling processing can be performed in sequence based on the convolutional layer and pooling layer of the convolutional neural network to obtain student face features corresponding to the student face image.
[0048] Since a preset facial feature library is also stored in the server, the feature similarity between the student's facial features and each facial feature in the preset facial feature library can be calculated at this time, such as calculating the cosine similarity between the student's facial features and each facial feature in the preset facial feature library as the feature similarity. After obtaining the feature similarity between the student's facial features and each facial feature in the preset facial feature library, select the target facial feature in the preset facial feature library that has the greatest feature similarity with the student's facial features. Since the user identity information of each facial feature in the preset facial feature library is known, after obtaining the target facial feature, the user identity information corresponding to the target facial feature can be used as the first recognition result. It can be seen that based on the above method, facial recognition can be quickly performed to accurately obtain user identity information.
[0049] S130. Obtain student history learning video data corresponding to the first recognition result.
[0050] In this embodiment, since a large amount of learning video data recorded by users during the learning process is stored in the server, after obtaining the first recognition result, it can be used as a screening condition to obtain the historical learning video data of students stored in the server. In order to reduce the amount of data stored in the server, only the learning video data within the last month can be stored for each learning user.
[0051] In one embodiment, step S130 includes:
[0052] Obtain the current system time, and determine the historical data acquisition time interval according to the preset statistical time length and the current system time; wherein the historical data acquisition time interval is [current system time-preset statistical time length, current system time);
[0053] Acquire historical video data in the local storage area whose video data acquisition time is within the historical data acquisition time interval and whose corresponding acquisition user identity information is the same as the first recognition result to form the student historical learning video data.
[0054] In this embodiment, when the server acquires the student's historical learning video data, it specifically first acquires the current system time and the preset statistical duration at the same time. Then, the historical data acquisition time interval is acquired with the current system time minus the preset statistical duration as the starting time of the historical data acquisition time interval, and the historical data acquisition time interval is acquired with the current system time as the ending time of the historical data acquisition time interval. The historical data acquisition time interval and the first recognition result are used as the screening conditions to screen and acquire the video data acquisition time in the local storage area of the server (such as the hard disk of the server) in the historical data acquisition time interval, and the corresponding historical video data of the collected user's identity information is the same as the first recognition result, so as to form the student's historical learning video data. It can be seen that the student's historical learning video data can be accurately screened based on the above method.
[0055] Among them, the preset statistical duration is generally less than the maximum duration of user video storage in the server (such as 1 month in the above example), that is, the student historical learning video data is also screened based on the video data that is actually effectively stored in the server.
[0056] S140, obtaining a pre-trained micro-expression recognition model, inputting a number of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtaining a second recognition result.
[0057] In this embodiment, still referring to the above example, after the student history learning video data of student user A is screened in the server, the video can be split to obtain multiple student history learning video images. For example, the student history learning video data corresponds to the learning video data of the user in the past 15 days, and 3 student history learning video images are randomly screened for each day's learning video data. Finally, 45 student history learning video images are used as the screening results and input into the micro-expression recognition model to obtain the second recognition result.
[0058] The micro-expression recognition model used is a comprehensive model of ACNN (i.e., convolutional neural network with attention mechanism) and Bi-LSTM (i.e., bidirectional long short-term memory network). The 45 student history learning video images obtained in the above example are used as screening results and input into the ACNN model and the Bi-LSTM model to obtain the micro-expression recognition results corresponding to the 45 student history learning video images. Afterwards, a comprehensive analysis of the micro-expression recognition results corresponding to the 45 student history learning video images is performed to obtain a final second recognition result.
[0059] In one embodiment, step S140 includes:
[0060] Inputting a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model respectively, and obtaining micro-expression recognition results corresponding to the plurality of student history learning video images respectively;
[0061] The micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to a number of student history learning video images is used as the second recognition result.
[0062] In this embodiment, when the second recognition result is finally determined based on the micro-expression recognition results corresponding to several student history learning video images, the micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to several student history learning video images is used as the second recognition result. For example, there are 20 student history learning video images with micro-expression recognition results corresponding to joy, 13 student history learning video images with micro-expression recognition results corresponding to doubt, and 12 student history learning video images with micro-expression recognition results corresponding to normal expressions, among which the micro-expression recognition result with the highest frequency is joy, and joy is used as the second recognition result. In this way, a comprehensive analysis of the micro-expression recognition results corresponding to several student history learning video images is achieved, and an accurate and objective second recognition result is finally obtained.
[0063] S150, determining a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship.
[0064] In this embodiment, referring to the above example, after the first recognition result and the second recognition result of the student user A are determined in the server, the candidate recommended course data set corresponding to the student face image can be determined in combination with the locally pre-stored micro-expression-recommended course mapping relationship.
[0065] For example, the current learning course information of student user A is course B1, the first recognition result corresponds to student user A, and the second recognition result corresponds to pleasure, which means that student user A is in a pleasant mood when learning course B1, which also shows that student user A understands the course and has no difficulty in understanding the knowledge points when learning course B1. In order to further recommend the next course to student user A in combination with learning course B1, the difficulty level of the course to be recommended can be obtained by the mapping relationship between micro-expression and recommended course and the second recognition result, and then the related courses related to course B1 (such as basic learning course B1A, basic learning course B1B and basic learning course B1C of course B1, or advanced learning course B1D and advanced learning course B1E of course B1) are recommended in combination with the difficulty level of the course to be recommended and course B1 to determine the advanced learning course B1D. It can be seen that the candidate recommended course data set obtained based on multi-dimensional parameter analysis can be more suitable for users to further learn.
[0066] In one embodiment, step S150 includes:
[0067] Obtain the current course name corresponding to the current learning course information;
[0068] Determine an initial candidate recommended course data set based on the current course name and the second recognition result;
[0069] The candidate recommended course data set is determined in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship.
[0070] In this embodiment, since the current course name corresponding to the current learning course information has been obtained before, the initial candidate recommended course data set is determined by combining the current course name (such as course B1 in the above example) and the second recognition result (such as pleasure in the above example). For example, the initial candidate recommended course data set is the advanced learning course B1D (with advanced difficulty level one) and advanced learning course B1E (with advanced difficulty level two) and advanced learning course B1F (with advanced difficulty level one) of course B1.
[0071] Afterwards, since the difficulty level of the recommended course can also be determined based on the second recognition result and the micro-expression-recommended course mapping relationship, for example, when the second recognition result is joy, the micro-expression-recommended course mapping relationship is used to query the advanced difficulty level one corresponding to joy, and when the second recognition result is happiness, the micro-expression-recommended course mapping relationship is used to query the advanced difficulty level two corresponding to happiness. When the second recognition result is joy, the advanced difficulty level one is determined based on the micro-expression-recommended course mapping relationship. At this time, the advanced learning courses B1D and B1F with advanced difficulty level one are screened from the initial candidate recommended course data set to form the candidate recommended course data set. It can be seen that based on the above screening method, the micro-expressions generated by student users in understanding knowledge points during the learning process are fully considered to determine the difficulty of the next course learning and the specific recommended courses, and the obtained candidate recommended course data set is more reference and objective.
[0072] In one embodiment, determining an initial candidate recommended course dataset based on the current course title and the second recognition result includes:
[0073] If it is determined that the second recognition result is a positive micro-expression recognition result, then obtaining an advanced course data set corresponding to the current course name to form the initial candidate recommended course data set;
[0074] If it is determined that the second recognition result is a neutral micro-expression recognition result, obtaining a course data set of the same level corresponding to the current course name to form the initial candidate recommended course data set;
[0075] If it is determined that the second recognition result is a negative micro-expression recognition result, obtaining a reduced-level course data set corresponding to the current course name to form the initial candidate recommended course data set;
[0076] The determining the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship includes:
[0077] If there is a target micro-expression identical to the second recognition result in the micro-expression-recommended course mapping relationship, obtaining a target recommended course difficulty level corresponding to the target micro-expression;
[0078] Initial candidate recommended course data having the target recommended course difficulty level is obtained from the initial candidate recommended course data set to form the candidate recommended course data set.
[0079] In this embodiment, when the server determines the initial candidate recommended course data set based on the current course name and the second recognition result, it is necessary to further determine whether the second recognition result is a positive, neutral or negative micro-expression recognition result. For example, happiness and joy are positive micro-expression recognition results, normal is a neutral micro-expression recognition result, and frustration and disgust are negative micro-expression recognition results. After determining the positivity, neutrality or negativity of the second recognition result, it can be determined whether to recommend an advanced course data set, a course data set of the same level or a downgraded course data set to the student user as the candidate recommended course data set.
[0080] Moreover, after determining the initial candidate recommended course data set, it is also possible to determine the target micro-expression that is the same as the second recognition result from the micro-expression-recommended course mapping relationship, and obtain the initial candidate recommended course data with the target recommended course difficulty level in the initial candidate recommended course data set according to the target recommended course difficulty level corresponding to the target micro-expression, so as to form the candidate recommended course data set, such as in the above example, the advanced learning course B1D and advanced learning course B1F with advanced difficulty level 1 are screened in the initial candidate recommended course data set to form the candidate recommended course data set. It can be seen that based on the above screening method, the positive and negative directions of the micro-expressions generated by the student users in understanding the knowledge points during the learning process and the comparison results of the micro-expression mapping relationship are fully considered, and the determined course learning difficulty and specific recommended courses are more accurate and objective.
[0081] S160: Obtain course learning trajectory data corresponding to the student's historical learning video data, and determine a final recommended course data set based on the course learning trajectory data and the candidate recommended course data sets.
[0082] In this embodiment, continuing to refer to the above example, since the server also stores the course learning trajectory data obtained after analyzing the student user A's historical learning of various knowledge points, for example, course C2-course D3-course B1 can be regarded as one course learning trajectory data, after obtaining the course learning trajectory data of student user A and the candidate recommended course data set, the final recommended course data set of student user A can be obtained by comprehensive analysis.
[0083] In one embodiment, step S160 includes:
[0084] Obtaining the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data;
[0085] The candidate recommended course data having the greatest correlation with the course learning trajectory data is obtained to form the final recommended course data set.
[0086] In this embodiment, when determining the final recommended course data set based on the course learning trajectory data and the candidate recommended course data set, specifically, firstly obtain the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data. For example, advanced learning courses B1D and advanced learning courses B1F constitute the candidate recommended course data set, and the correlation between advanced learning course B1D and course C2-course D3-course B1 is 0.9 (for example, when calculating the correlation between advanced learning course B1D and the course learning trajectory data of course C2-course D3-course B1, the cosine similarities of the course name feature vector corresponding to advanced learning course B1D and the above three course name feature vectors are calculated respectively, and then the sum is obtained to obtain the correlation between advanced learning course B1F and the course learning trajectory data of course C2-course D3-course B1), and the correlation between advanced learning course B1D and course C2-course D3-course B1 is 0.8. At this time, the candidate recommended course data set, advanced learning course B1D, which has the greatest correlation with the course learning trajectory data, constitutes the final recommended course data set, and is finally recommended to student user A for the next step of advanced learning.
[0087] It can be seen that the embodiment of the method can perform expression recognition and micro-expression recognition based on the learning videos recorded during the user's learning process, as well as the user's course learning trajectory data to jointly determine a final recommended course data set that is more accurate and in line with the user's learning progress in multiple dimensions.
[0088] Figure 3 FIG. 1 is a schematic block diagram of a student course recommendation device based on face recognition provided by an embodiment of the present invention. Figure 3 As shown, corresponding to the above-mentioned student course recommendation method based on face recognition, the present invention also provides a student course recommendation device 100 based on face recognition. The student course recommendation device 100 based on face recognition includes a unit for executing the above-mentioned student course recommendation method based on face recognition. Figure 3 The student course recommendation device 100 based on face recognition includes: a face image information acquisition unit 110, a face recognition unit 120, a target video data acquisition unit 130, a micro-expression recognition unit 140, a candidate course data acquisition unit 150 and a final course data acquisition unit 160.
[0089] The face image information acquisition unit 110 is used to respond to a course recommendation instruction and acquire a student face image and current course information corresponding to the course recommendation instruction.
[0090] In this embodiment, the technical solution is described with the server as the execution subject. The server is the background server corresponding to the online learning platform, and a large amount of learning video data is stored in the server. When the student user uses the user terminal to enter the login account and password and successfully logs in to the online learning platform corresponding to the server, the course recommendation instruction can be triggered by the active operation. For example, there is a virtual operation button for course recommendation on the homepage of the online learning platform. After the student user presses the virtual operation button for course recommendation, the course recommendation instruction can be generated. The server can also automatically generate course recommendation instructions based on each successful login of the student user, that is, the student user uses the user terminal to automatically generate course recommendation instructions after entering the login account and password and successfully logging in to the online learning platform corresponding to the server.
[0091] For example, this application takes a student user A logging into an online learning platform as an example. When the server obtains the course recommendation instruction corresponding to the operation of student user A logging into the online learning platform, the server further obtains the student face image and current learning course information corresponding to student user A. That is, when the course recommendation instruction is detected in the server, a user portrait collection instruction will be sent to the user terminal used by student user A. After the user receives the user portrait collection instruction and authorizes the collection, the student face image of the student user is collected by the user terminal and uploaded to the server by the user terminal. Moreover, after student user A enters the online learning platform, the server can also obtain the name of the course that the user currently clicks to open for study in real time as the current learning course information, or the server can also obtain the course name corresponding to the course that student user A has recently completed online learning as the current learning course information. The student face image and the current learning course information are obtained in the server, which can be used as part of the reference data for the subsequent acquisition of the user's student face image and current learning course information.
[0092] The face recognition unit 120 is used to obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result.
[0093] In this embodiment, since a pre-trained face recognition model is stored in the server, after obtaining the student's face image, it can be input into the face recognition model for face recognition to obtain a first recognition result. It can be seen that the user's identity can be quickly determined based on the face recognition method.
[0094] For example, still referring to the above example, after student user A logs in to the online learning platform, the server can obtain the current user identity information (such as user name, user unique identification number, etc.) based on the user's account information, and then compare the current user identity information with the first recognition result obtained based on face recognition. If the current user identity information and the first recognition result both correspond to the same user, it means that student user A himself has logged in to the online learning platform for online learning, and no one else (such as student user B, etc.) has logged in to student user A's account to conduct online learning. After confirming that it is student user A himself who has logged in for online learning, further course recommendations can be made for student user A.
[0095] In one embodiment, the face recognition unit 120 is specifically used for:
[0096] Inputting the student's face image into the convolutional neural network corresponding to the face recognition model to extract features, and obtaining the student's face features;
[0097] Based on the feature similarity between the student's facial features and each facial feature in a preset facial feature library, determining a target facial feature having the greatest feature similarity with the student's facial features;
[0098] Obtain user identity information corresponding to the target facial feature as the first recognition result.
[0099] In this embodiment, a convolutional neural network is used as a face recognition model in the server. When the student face image is input into the convolutional neural network, convolution processing and pooling processing can be performed in sequence based on the convolutional layer and pooling layer of the convolutional neural network to obtain student face features corresponding to the student face image.
[0100] Since a preset facial feature library is also stored in the server, the feature similarity between the student's facial features and each facial feature in the preset facial feature library can be calculated at this time, such as calculating the cosine similarity between the student's facial features and each facial feature in the preset facial feature library as the feature similarity. After obtaining the feature similarity between the student's facial features and each facial feature in the preset facial feature library, select the target facial feature in the preset facial feature library that has the greatest feature similarity with the student's facial features. Since the user identity information of each facial feature in the preset facial feature library is known, after obtaining the target facial feature, the user identity information corresponding to the target facial feature can be used as the first recognition result. It can be seen that based on the above method, facial recognition can be quickly performed to accurately obtain user identity information.
[0101] The target video data acquisition unit 130 is used to acquire the student's historical learning video data corresponding to the first recognition result.
[0102] In this embodiment, since a large amount of learning video data recorded by users during the learning process is stored in the server, after obtaining the first recognition result, it can be used as a screening condition to obtain the historical learning video data of students stored in the server. In order to reduce the amount of data stored in the server, only the learning video data within the last month can be stored for each learning user.
[0103] In one embodiment, the target video data acquisition unit 130 is specifically used for:
[0104] Obtain the current system time, and determine the historical data acquisition time interval according to the preset statistical time length and the current system time; wherein the historical data acquisition time interval is [current system time-preset statistical time length, current system time);
[0105] Acquire historical video data in the local storage area whose video data acquisition time is within the historical data acquisition time interval and whose corresponding acquisition user identity information is the same as the first recognition result to form the student historical learning video data.
[0106] In this embodiment, when the server acquires the student's historical learning video data, it specifically first acquires the current system time and the preset statistical duration at the same time. Then, the historical data acquisition time interval is acquired with the current system time minus the preset statistical duration as the starting time of the historical data acquisition time interval, and the historical data acquisition time interval is acquired with the current system time as the ending time of the historical data acquisition time interval. The historical data acquisition time interval and the first recognition result are used as the screening conditions to screen and acquire the video data acquisition time in the local storage area of the server (such as the hard disk of the server) in the historical data acquisition time interval, and the corresponding historical video data of the collected user's identity information is the same as the first recognition result, so as to form the student's historical learning video data. It can be seen that the student's historical learning video data can be accurately screened based on the above method.
[0107] Among them, the preset statistical duration is generally less than the maximum duration of user video storage in the server (such as 1 month in the above example), that is, the student historical learning video data is also screened based on the video data that is actually effectively stored in the server.
[0108] The micro-expression recognition unit 140 is used to obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result.
[0109] In this embodiment, still referring to the above example, after the student history learning video data of student user A is screened in the server, the video can be split to obtain multiple student history learning video images. For example, the student history learning video data corresponds to the learning video data of the user in the past 15 days, and 3 student history learning video images are randomly screened for each day's learning video data. Finally, 45 student history learning video images are used as the screening results and input into the micro-expression recognition model to obtain the second recognition result.
[0110] The micro-expression recognition model used is a comprehensive model of ACNN (i.e., convolutional neural network with attention mechanism) and Bi-LSTM (i.e., bidirectional long short-term memory network). The 45 student history learning video images obtained in the above example are used as screening results and input into the ACNN model and the Bi-LSTM model to obtain the micro-expression recognition results corresponding to the 45 student history learning video images. Afterwards, a comprehensive analysis of the micro-expression recognition results corresponding to the 45 student history learning video images is performed to obtain a final second recognition result.
[0111] In one embodiment, the micro-expression recognition unit 140 is specifically used for:
[0112] Inputting a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model respectively, and obtaining micro-expression recognition results corresponding to the plurality of student history learning video images respectively;
[0113] The micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to a number of student history learning video images is used as the second recognition result.
[0114] In this embodiment, when the second recognition result is finally determined based on the micro-expression recognition results corresponding to several student history learning video images, the micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to several student history learning video images is used as the second recognition result. For example, there are 20 student history learning video images with micro-expression recognition results corresponding to joy, 13 student history learning video images with micro-expression recognition results corresponding to doubt, and 12 student history learning video images with micro-expression recognition results corresponding to normal expressions, among which the micro-expression recognition result with the highest frequency is joy, and joy is used as the second recognition result. In this way, a comprehensive analysis of the micro-expression recognition results corresponding to several student history learning video images is achieved, and an accurate and objective second recognition result is finally obtained.
[0115] The candidate course data acquisition unit 150 is used to determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship.
[0116] In this embodiment, referring to the above example, after the first recognition result and the second recognition result of the student user A are determined in the server, the candidate recommended course data set corresponding to the student face image can be determined in combination with the locally pre-stored micro-expression-recommended course mapping relationship.
[0117] For example, the current learning course information of student user A is course B1, the first recognition result corresponds to student user A, and the second recognition result corresponds to pleasure, which means that student user A is in a pleasant mood when learning course B1, which also shows that student user A understands the course and has no difficulty in understanding the knowledge points when learning course B1. In order to further recommend the next course to student user A in combination with learning course B1, the difficulty level of the course to be recommended can be obtained by the mapping relationship between micro-expression and recommended course and the second recognition result, and then the related courses related to course B1 (such as basic learning course B1A, basic learning course B1B and basic learning course B1C of course B1, or advanced learning course B1D and advanced learning course B1E of course B1) are recommended in combination with the difficulty level of the course to be recommended and course B1 to determine the advanced learning course B1D. It can be seen that the candidate recommended course data set obtained based on multi-dimensional parameter analysis can be more suitable for users to further learn.
[0118] In one embodiment, the candidate course data acquisition unit 150 is specifically used to:
[0119] Obtain the current course name corresponding to the current learning course information;
[0120] Determine an initial candidate recommended course data set based on the current course name and the second recognition result;
[0121] The candidate recommended course data set is determined in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship.
[0122] In this embodiment, since the current course name corresponding to the current learning course information has been obtained before, the initial candidate recommended course data set is determined by combining the current course name (such as course B1 in the above example) and the second recognition result (such as pleasure in the above example). For example, the initial candidate recommended course data set is the advanced learning course B1D (with advanced difficulty level one) and advanced learning course B1E (with advanced difficulty level two) and advanced learning course B1F (with advanced difficulty level one) of course B1.
[0123] Afterwards, since the difficulty level of the recommended course can also be determined based on the second recognition result and the micro-expression-recommended course mapping relationship, for example, when the second recognition result is joy, the micro-expression-recommended course mapping relationship is used to query the advanced difficulty level one corresponding to joy, and when the second recognition result is happiness, the micro-expression-recommended course mapping relationship is used to query the advanced difficulty level two corresponding to happiness. When the second recognition result is joy, the advanced difficulty level one is determined based on the micro-expression-recommended course mapping relationship. At this time, the advanced learning courses B1D and B1F with advanced difficulty level one are screened from the initial candidate recommended course data set to form the candidate recommended course data set. It can be seen that based on the above screening method, the micro-expressions generated by student users in understanding knowledge points during the learning process are fully considered to determine the difficulty of the next course learning and the specific recommended courses, and the obtained candidate recommended course data set is more reference and objective.
[0124] In one embodiment, determining an initial candidate recommended course dataset based on the current course title and the second recognition result includes:
[0125] If it is determined that the second recognition result is a positive micro-expression recognition result, then obtaining an advanced course data set corresponding to the current course name to form the initial candidate recommended course data set;
[0126] If it is determined that the second recognition result is a neutral micro-expression recognition result, obtaining a course data set of the same level corresponding to the current course name to form the initial candidate recommended course data set;
[0127] If it is determined that the second recognition result is a negative micro-expression recognition result, obtaining a reduced-level course data set corresponding to the current course name to form the initial candidate recommended course data set;
[0128] The determining the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship includes:
[0129] If there is a target micro-expression identical to the second recognition result in the micro-expression-recommended course mapping relationship, obtaining a target recommended course difficulty level corresponding to the target micro-expression;
[0130] Initial candidate recommended course data having the target recommended course difficulty level is obtained from the initial candidate recommended course data set to form the candidate recommended course data set.
[0131] In this embodiment, when the server determines the initial candidate recommended course data set based on the current course name and the second recognition result, it is necessary to further determine whether the second recognition result is a positive, neutral or negative micro-expression recognition result. For example, happiness and joy are positive micro-expression recognition results, normal is a neutral micro-expression recognition result, and frustration and disgust are negative micro-expression recognition results. After determining the positivity, neutrality or negativity of the second recognition result, it can be determined whether to recommend an advanced course data set, a course data set of the same level or a downgraded course data set to the student user as the candidate recommended course data set.
[0132] Moreover, after determining the initial candidate recommended course data set, it is also possible to determine the target micro-expression that is the same as the second recognition result from the micro-expression-recommended course mapping relationship, and obtain the initial candidate recommended course data with the target recommended course difficulty level in the initial candidate recommended course data set according to the target recommended course difficulty level corresponding to the target micro-expression, so as to form the candidate recommended course data set, such as in the above example, the advanced learning course B1D and advanced learning course B1F with advanced difficulty level 1 are screened in the initial candidate recommended course data set to form the candidate recommended course data set. It can be seen that based on the above screening method, the positive and negative directions of the micro-expressions generated by the student users in understanding the knowledge points during the learning process and the comparison results of the micro-expression mapping relationship are fully considered, and the determined course learning difficulty and specific recommended courses are more accurate and objective.
[0133] The final course data acquisition unit 160 is used to acquire the course learning trajectory data corresponding to the student's historical learning video data, and determine the final recommended course data set based on the course learning trajectory data and the candidate recommended course data set.
[0134] In this embodiment, continuing to refer to the above example, since the server also stores the course learning trajectory data obtained after analyzing the student user A's historical learning of various knowledge points, for example, course C2-course D3-course B1 can be regarded as one course learning trajectory data, after obtaining the course learning trajectory data of student user A and the candidate recommended course data set, the final recommended course data set of student user A can be obtained by comprehensive analysis.
[0135] In one embodiment, the final course data acquisition unit 160 is specifically used for:
[0136] Obtaining the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data;
[0137] The candidate recommended course data having the greatest correlation with the course learning trajectory data is obtained to form the final recommended course data set.
[0138] In this embodiment, when determining the final recommended course data set based on the course learning trajectory data and the candidate recommended course data set, specifically, firstly obtain the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data. For example, advanced learning courses B1D and advanced learning courses B1F constitute the candidate recommended course data set, and the correlation between advanced learning course B1D and course C2-course D3-course B1 is 0.9 (for example, when calculating the correlation between advanced learning course B1D and the course learning trajectory data of course C2-course D3-course B1, the cosine similarities of the course name feature vector corresponding to advanced learning course B1D and the above three course name feature vectors are calculated respectively, and then the sum is obtained to obtain the correlation between advanced learning course B1F and the course learning trajectory data of course C2-course D3-course B1), and the correlation between advanced learning course B1D and course C2-course D3-course B1 is 0.8. At this time, the candidate recommended course data set, advanced learning course B1D, which has the greatest correlation with the course learning trajectory data, constitutes the final recommended course data set, and is finally recommended to student user A for the next step of advanced learning.
[0139] It can be seen that the implementation embodiment of the device can perform expression recognition and micro-expression recognition based on the learning videos recorded during the user's learning process, as well as the user's course learning trajectory data to jointly determine a final recommended course data set that is more accurate and in line with the user's learning progress in multiple dimensions.
[0140] The above-mentioned student course recommendation device based on face recognition can be implemented in the form of a computer program. The computer program can be used in Figure 4 The shown face recognition based student course is run on the recommended device.
[0141] See also Figure 4 , Figure 4 The present invention is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device integrates any of the student course recommendation devices based on face recognition provided by an embodiment of the present invention.
[0142] See also Figure 4 The computer device includes a processor 402 , a memory and a network interface 405 connected via a system bus 401 , wherein the memory may include a storage medium 403 and an internal memory 404 .
[0143] The storage medium 403 can store an operating system 4031 and a computer program 4032. The computer program 4032 includes program instructions, and when the program instructions are executed, the processor 402 can execute a student course recommendation method based on face recognition.
[0144] The processor 402 is used to provide computing and control capabilities to support the operation of the entire computer device.
[0145] The internal memory 404 provides an environment for the operation of the computer program 4032 in the storage medium 403. When the computer program 4032 is executed by the processor 402, the processor 402 can execute the above-mentioned student course recommendation method based on face recognition.
[0146] The network interface 405 is used to communicate with other devices over the network. Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0147] The processor 402 is used to run the computer program 4032 stored in the memory to implement the following steps:
[0148] In response to a course recommendation instruction, obtaining a student face image and current course information corresponding to the course recommendation instruction;
[0149] Obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result;
[0150] Acquire student history learning video data corresponding to the first recognition result;
[0151] Obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result;
[0152] Determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship;
[0153] The course learning trajectory data corresponding to the student's historical learning video data is obtained, and a final recommended course data set is determined based on the course learning trajectory data and the candidate recommended course data sets.
[0154] It should be understood that in the embodiment of the present invention, the processor 402 may be a central processing unit (CPU), and the processor 402 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0155] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.
[0156] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium, and the computer-readable storage medium may be a non-volatile storage medium or a non-volatile storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps:
[0157] In response to a course recommendation instruction, obtaining a student face image and current course information corresponding to the course recommendation instruction;
[0158] Obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result;
[0159] Acquire student history learning video data corresponding to the first recognition result;
[0160] Obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result;
[0161] Determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship;
[0162] The course learning trajectory data corresponding to the student's historical learning video data is obtained, and a final recommended course data set is determined based on the course learning trajectory data and the candidate recommended course data sets.
[0163] The storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, or any other computer-readable storage medium that can store program codes.
[0164] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0165] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0166] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a terminal, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.
[0168] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A student course recommendation method based on face recognition, It is characterized in that include: In response to a course recommendation instruction, a student face image and current course information corresponding to the course recommendation instruction are obtained; wherein the course recommendation instruction is generated when a user successfully logs into an online learning platform in a server based on a login account and password and clicks a virtual operation button, or is generated when a user successfully logs into an online learning platform in a server based on a login account and password; Obtain a pre-trained face recognition model, input the student's face image into the face recognition model, and obtain a first recognition result; if it is determined that the current user identity information corresponding to the user logging into the server corresponds to the same user as the first recognition result, it is determined that the user himself has logged into the online learning platform in the server for online learning; Acquire the student's historical learning video data corresponding to the first recognition result, wherein the student's historical learning video data is learning video data stored in the server within the past month; Obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result; Determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship; Acquire course learning trajectory data corresponding to the student's historical learning video data, and determine a final recommended course data set based on the course learning trajectory data and the candidate recommended course data sets; The determining of a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship includes: Obtain the current course name corresponding to the current learning course information; Determine an initial candidate recommended course data set based on the current course name and the second recognition result; Determine the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship; The determining of an initial candidate recommended course data set based on the current course name and the second recognition result includes: If it is determined that the second recognition result is a positive micro-expression recognition result, then obtaining an advanced course data set corresponding to the current course name to form the initial candidate recommended course data set; If it is determined that the second recognition result is a neutral micro-expression recognition result, obtaining a course data set of the same level corresponding to the current course name to form the initial candidate recommended course data set; If it is determined that the second recognition result is a negative micro-expression recognition result, obtaining a reduced-level course data set corresponding to the current course name to form the initial candidate recommended course data set; The determining the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship includes: If there is a target micro-expression identical to the second recognition result in the micro-expression-recommended course mapping relationship, obtaining a target recommended course difficulty level corresponding to the target micro-expression; Acquire initial candidate recommended course data having the target recommended course difficulty level from the initial candidate recommended course data set to form the candidate recommended course data set; The acquiring of the course learning trajectory data corresponding to the student's historical learning video data, and determining the final recommended course data set based on the course learning trajectory data and the candidate recommended course data set, includes: Obtaining the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data; Acquire candidate recommended course data having the greatest correlation with the course learning trajectory data to form the final recommended course data set; In the step of obtaining the degree of association between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data, when determining the degree of association between each candidate recommended course data and the course learning trajectory data, the cosine similarities of the course name feature vector of the candidate recommended course data and the course name feature vector in the course learning trajectory data are respectively obtained and summed to determine the degree of association between the candidate recommended course data and the course learning trajectory data; The step of inputting the student face image into the face recognition model to obtain a first recognition result includes: Inputting the student's face image into the convolutional neural network corresponding to the face recognition model to extract features, and obtaining the student's face features; Based on the feature similarity between the student's facial features and each facial feature in a preset facial feature library, determining a target facial feature having the greatest feature similarity with the student's facial features; Obtaining user identity information corresponding to the target facial feature as the first recognition result; The acquiring of the student's historical learning video data corresponding to the first recognition result includes: Obtain the current system time, and determine the historical data acquisition time interval according to the preset statistical time length and the current system time; wherein the historical data acquisition time interval is [current system time-preset statistical time length, current system time); Acquire in the local storage area historical video data whose video data acquisition time is within the historical data acquisition time interval and whose corresponding acquisition user identity information is the same as the first recognition result, to form the student historical learning video data; The step of inputting a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model to obtain a second recognition result includes: Inputting a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model respectively, and obtaining micro-expression recognition results corresponding to the plurality of student history learning video images respectively; The micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to a number of student history learning video images is used as the second recognition result.
2. A student course recommendation device based on face recognition, It is characterized in that include: A facial image information acquisition unit, configured to respond to a course recommendation instruction and acquire a student facial image and current course information corresponding to the course recommendation instruction; wherein the course recommendation instruction is generated when a user successfully logs into an online learning platform in a server based on a login account and password and clicks a virtual operation button, or when a user successfully logs into an online learning platform in a server based on a login account and password; A face recognition unit is used to obtain a pre-trained face recognition model, input the student face image into the face recognition model, and obtain a first recognition result; if it is determined that the current user identity information corresponding to the user logging into the server corresponds to the same user as the first recognition result, it is determined that the user himself has logged into the online learning platform in the server for online learning; A target video data acquisition unit, used to acquire the student's historical learning video data corresponding to the first recognition result, wherein the student's historical learning video data is the learning video data stored in the server within the past month; A micro-expression recognition unit, used to obtain a pre-trained micro-expression recognition model, input a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model, and obtain a second recognition result; A candidate course data acquisition unit, configured to determine a candidate recommended course data set corresponding to the student face image based on the current learning course information, the second recognition result, and a preset micro-expression-recommended course mapping relationship; A final course data acquisition unit, configured to acquire course learning trajectory data corresponding to the student's historical learning video data, and determine a final recommended course data set based on the course learning trajectory data and the candidate recommended course data set; The candidate course data acquisition unit is specifically used for: Obtain the current course name corresponding to the current learning course information; Determine an initial candidate recommended course data set based on the current course name and the second recognition result; Determine the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship; The determining of an initial candidate recommended course data set based on the current course name and the second recognition result includes: If it is determined that the second recognition result is a positive micro-expression recognition result, then obtaining an advanced course data set corresponding to the current course name to form the initial candidate recommended course data set; If it is determined that the second recognition result is a neutral micro-expression recognition result, obtaining a course data set of the same level corresponding to the current course name to form the initial candidate recommended course data set; If it is determined that the second recognition result is a negative micro-expression recognition result, obtaining a reduced-level course data set corresponding to the current course name to form the initial candidate recommended course data set; The determining the candidate recommended course data set in the initial candidate recommended course data set based on the second recognition result and the micro-expression-recommended course mapping relationship includes: If there is a target micro-expression identical to the second recognition result in the micro-expression-recommended course mapping relationship, obtaining a target recommended course difficulty level corresponding to the target micro-expression; Acquire initial candidate recommended course data having the target recommended course difficulty level from the initial candidate recommended course data set to form the candidate recommended course data set; The final course data acquisition unit is specifically used for: Obtaining the correlation between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data; Acquire candidate recommended course data having the greatest correlation with the course learning trajectory data to form the final recommended course data set; In the step of obtaining the degree of association between each candidate recommended course data in the candidate recommended course data set and the course learning trajectory data, when determining the degree of association between each candidate recommended course data and the course learning trajectory data, the cosine similarities of the course name feature vector of the candidate recommended course data and the course name feature vector in the course learning trajectory data are respectively obtained and summed to determine the degree of association between the candidate recommended course data and the course learning trajectory data; The face recognition unit is specifically used for: Inputting the student's face image into the convolutional neural network corresponding to the face recognition model to extract features, and obtaining the student's face features; Based on the feature similarity between the student's facial features and each facial feature in a preset facial feature library, determining a target facial feature having the greatest feature similarity with the student's facial features; Obtaining user identity information corresponding to the target facial feature as the first recognition result; The target video data acquisition unit is specifically used for: Obtain the current system time, and determine the historical data acquisition time interval according to the preset statistical time length and the current system time; wherein the historical data acquisition time interval is [current system time-preset statistical time length, current system time); Acquire in the local storage area historical video data whose video data acquisition time is within the historical data acquisition time interval and whose corresponding acquisition user identity information is the same as the first recognition result, to form the student historical learning video data; The micro-expression recognition unit is specifically used for: Inputting a plurality of student history learning video images selected from the student history learning video data into the micro-expression recognition model respectively, and obtaining micro-expression recognition results corresponding to the plurality of student history learning video images respectively; The micro-expression recognition result with the highest frequency among the micro-expression recognition results corresponding to a number of student history learning video images is used as the second recognition result.
3. A computer device, It is characterized in that The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the student course recommendation method based on face recognition as claimed in claim 1 when executing the computer program.
4. A computer-readable storage medium, It is characterized in that The storage medium stores a computer program, which includes program instructions. When the program instructions are executed by a processor, the student course recommendation method based on face recognition as claimed in claim 1 can be implemented.
Citation Information
Patent Citations
Student history and real-time learning state parameter-based course recommendation realization method and system
CN106528656A
Content pushing method and device based on expressions
CN112507243A
