Learning ability and preference comprehensive evaluation method and device, electronic equipment and storage medium
By evaluating and analyzing students' historical learning data and learning status, personalized learning solutions are provided, which solves the problem that the existing online learning system cannot adapt to the individual differences of learners, and improves learning effectiveness and efficiency.
Patent Information
- Application Number
- CN202411452631.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-03-04
AI Technical Summary
Existing online learning systems are difficult to provide appropriate learning resources and programs based on individual differences among learners, resulting in poor learning results.
By obtaining students' historical learning test data and learning status data, assessing learning ability and analyzing learning preferences, generating comprehensive assessment results and outputting recommended learning plans.
A more comprehensive and accurate understanding of learners' learning status, providing personalized learning suggestions, and improving learning efficiency and effectiveness.
Smart Images
Figure CN119558511A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a learning ability and preference comprehensive evaluation method, device, electronic equipment and storage medium. Background Art
[0002] With the rapid development of educational informatization, people's demand for education is also growing. Online learning models in the field of education are becoming more and more common, providing learners with more and more convenient learning methods.
[0003] Although online learning has the advantages of convenience, low cost, and no restrictions on space and time, it also has problems such as large amount of learning resources and lack of specificity. Especially for content that requires independent learning, the learner's self-discipline and habits have a great impact on their learning outcomes.
[0004] Some current learning systems usually release learning plans based on preset rules. When learners are learning, they may not be suitable for their own situations, making it difficult to complete or ineffective. It can be seen that the current learning system resources and learning plans lack consideration of individual differences in learners' learning situations and learning states, and cannot more appropriately meet learners' learning needs and efficiency needs. Summary of the invention
[0005] The main purpose of the present invention is to provide a method, device, electronic device and storage medium for comprehensive evaluation of learning ability and preference, which can obtain the learner's learning status more comprehensively and accurately, and provide targeted assistance in learning to meet the learner's learning needs and efficiency requirements.
[0006] To achieve the above objectives, the present application provides a comprehensive evaluation method for learning ability and preference, the method comprising:
[0007] Acquire historical learning test data of the student, evaluate the historical learning test data, and determine the learning ability evaluation result of the student;
[0008] Determining a learning preference analysis result of the student according to the student's learning status data and learning habit data;
[0009] Based on the learning ability assessment results and the learning preference analysis results, a comprehensive assessment result of the student is generated, and a recommended learning plan for the student is output.
[0010] Optionally, obtaining the student's historical learning test data includes:
[0011] Obtaining the test results of the student's target subject, the speed of answering questions, the study time and the learning speed of the student for the target subject, wherein the test results include the student's correct rate in the test of the target subject and the individual ranking of the test;
[0012] The step of evaluating the historical learning test data to determine the learning ability evaluation result of the student includes:
[0013] The historical learning test data is input into a preset evaluation model, and the learning ability evaluation result is output, wherein the learning ability evaluation result includes the learning ability grade or score of the student in the target subject.
[0014] Optionally, the learning status data includes the degree of concentration during the learning process of the target subject;
[0015] Before determining the student's learning preference analysis result according to the student's learning status data and learning habit data, the method further includes:
[0016] Collecting video footage of the student during the learning process of the target subject;
[0017] Performing image analysis on the video screen to determine the concentration of the student in the video screen;
[0018] Based on the concentration of the student in the video image, the degree of concentration during the learning of the target subject is determined.
[0019] Optionally, performing image analysis on the video screen to determine the concentration of the student in the video screen includes:
[0020] Identify and extract the head portrait area image in the video screen;
[0021] Inputting the head portrait area image into a learning state analysis model for processing, and determining an attention type corresponding to the head portrait area image, wherein the attention type is focused attention or distracted attention;
[0022] The determining of the degree of concentration of the student in the process of learning the target subject based on the concentration of the student in the video screen includes:
[0023] The degree of concentration during the target subject learning process is determined according to the proportion of the video frames on which the attention is concentrated in all the video frames.
[0024] Optionally, the training method of the learning state analysis model includes:
[0025] Acquire a sample image training set, the sample image training set including sample images of an unobstructed head area and sample images of a head area obstructed by hand movements, the sample images carrying key point information and a label indicating an attention type and a posture label; the key point information includes feature points located for a head area, eyes, eyeballs, and mouth of a human face in the sample images, the feature points forming contour information of the head area, eyes, eyeballs, and mouth of the human face;
[0026] The preset neural network model is trained using the sample image training set until the parameters reach a preset convergence condition or the cycle reaches a preset number of times, thereby obtaining a trained learning state analysis model.
[0027] Optionally, the learning habit data includes browsing speed data and learning time habit data during the learning process of the target subject;
[0028] The method further comprises:
[0029] Acquiring browsing speed data of the student during the learning process of the target subject;
[0030] generating a target browsing speed curve based on the browsing speed data, wherein the target browsing speed curve represents the browsing speed of the student in different time periods during the learning process of the target subject;
[0031] Determining the browsing preference type corresponding to the target browsing speed curve according to a preset correspondence relationship between the browsing speed curve and the browsing preference type;
[0032] Counting the study time during the target subject study process within a preset period and the duration of each study to obtain the study time habit data, wherein the study time includes the date and specific time period of the current study;
[0033] Determining the student's learning preference analysis result according to the student's learning status data and learning habit data includes:
[0034] Use a learning preference analysis model to analyze the student's learning status data and learning habit data, associate the student's study time during the target subject learning process with the corresponding learning status and learning habits, and generate the student's learning preference analysis results, including the learning status and learning habits corresponding to different study times during the target subject learning process.
[0035] Optionally, the recommended learning plan includes a recommended learning mode, a learning progress plan, and recommended test questions;
[0036] The learning progress plan includes a fixed learning time period and corresponding learning content;
[0037] The recommended learning mode and the learning progress plan are determined based on the learning ability assessment result and the learning preference analysis result;
[0038] The recommended test questions are determined based on the learning status and the learning ability evaluation result of the student.
[0039] On the other hand, the present application provides a learning ability and preference comprehensive evaluation device, comprising:
[0040] A learning ability evaluation module is used to obtain the student's historical learning test data, evaluate the historical learning test data, and determine the student's learning ability evaluation result;
[0041] A learning preference analysis module, used to determine the learning preference analysis result of the student according to the learning status data and learning habit data of the student;
[0042] An assessment recommendation module is used to generate a comprehensive assessment result of the student based on the learning ability assessment result and the learning preference analysis result, and output a recommended learning plan for the student.
[0043] On the other hand, the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes each step of the comprehensive evaluation method of learning ability and preference as described in the first aspect.
[0044] On the other hand, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes each step of the comprehensive evaluation method of learning ability and preference as described in the first aspect.
[0045] The present application provides a method, device, electronic device and storage medium for comprehensive evaluation of learning ability and preference. The method obtains historical learning test data of students, evaluates the historical learning test data, and determines the evaluation result of the student's learning ability; determines the analysis result of the student's learning preference based on the student's learning status data and learning habit data; generates a comprehensive evaluation result of the student based on the learning ability evaluation result and the learning preference analysis result, and outputs a recommended learning plan for the student. The method can evaluate the learning ability based on the student's learning test data, and understand the student's learning preference based on the learning status and learning habits, so as to further analyze and obtain a comprehensive evaluation result of the student's learning situation, and can put forward reasonable suggestions for the learning plan in a targeted manner, so as to obtain the learner's learning status more comprehensively and accurately, and meet the learner's learning needs and efficiency needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0047] in:
[0048] Figure 1 A flowchart of a method for comprehensive evaluation of learning ability and preference provided in an embodiment of the present application;
[0049] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;
[0050] Figure 3 A schematic diagram of the structure of a learning ability and preference comprehensive evaluation device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0052] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.
[0053] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0054] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.
[0055] See also Figure 1 , is a flow chart of a method for comprehensive evaluation of learning ability and preference provided in an embodiment of the present application. Figure 1 As shown, the method includes:
[0056] 101. Obtain the student's historical learning test data, evaluate the historical learning test data, and determine the learning ability evaluation result of the student.
[0057] The method in the embodiment of the present application can be executed by a learning ability and preference comprehensive evaluation device, which can specifically be a terminal device.
[0058] The above historical learning test data refers to the learning test data within the historical time period, and the above historical time period can be set as needed, for example (based on the current time node) the past week or 10 days. Learning test data includes data related to answering questions such as tests and exams, and may mainly include test results, answering speed, learning time and learning speed.
[0059] In the embodiment of the present application, the student's historical learning test data can be evaluated, the student's learning ability can be analyzed to a certain extent, and the learning ability evaluation result can be obtained.
[0060] In an optional implementation manner, the above-mentioned obtaining of the student's historical learning test data includes:
[0061] Obtain the test results of the target subject of the above-mentioned student, the speed of answering questions, the study time and learning speed of the above-mentioned student for the above-mentioned target subject, and the above-mentioned test results include the correct rate of the above-mentioned student in this test of the above-mentioned target subject and the individual ranking of this test;
[0062] The above-mentioned historical learning test data is evaluated to determine the learning ability evaluation results of the above-mentioned students, including:
[0063] The above historical learning test data is input into a preset evaluation model to output the above learning ability evaluation result, which includes the learning ability level or score of the above student in the above target subject.
[0064] 102. Determine the learning preference analysis results of the students based on the learning status data and learning habit data of the students.
[0065] In the embodiment of the present application, the learning preference of the student is determined mainly through the state and habits of the student during learning, and the learning preference analysis result is obtained. The learning state can reflect the interest in the learning content on the one hand, and can also reflect the student's attention status, etc.
[0066] On the one hand, the learning state data can be mainly analyzed and processed by a preset learning state analysis model, which can be selected or set as needed. In an optional implementation, the training method of the model can refer to Figure 3 The specific description of the illustrated embodiments will not be repeated here.
[0067] In an optional embodiment, the learning status data includes the degree of concentration during the learning process of the target subject;
[0068] Before the above step 102, the above method may further include:
[0069] 011. Collect video footage of the above-mentioned students in the process of learning the above-mentioned target subjects;
[0070] 012. Perform image analysis on the video to determine the concentration of the student in the video;
[0071] 013. Based on the students’ concentration in the video, determine their concentration level during the learning of the target subject.
[0072] The method in the embodiment of the present application is applicable to online learning, when students turn on the camera while listening to online classes. In this learning scenario, the system can obtain the video or image of the student's learning process in real time through the camera, that is, the above-mentioned video screen can be obtained, and the student's learning status can be analyzed through video monitoring to determine the student's concentration level in the learning process. Specifically, image analysis processing can be performed on each frame of the collected video screen or the extracted multiple frames of the video screen to determine the student's concentration level in each video screen, and then based on this, the concentration level in the target subject learning process can be further determined.
[0073] The specific method for determining the concentration of students in the video will be introduced later. Figure 2 The specific description of the illustrated embodiments will not be repeated here.
[0074] Optionally, for the same learning resource, such as a video courseware, the degree of attention obtained by different students can be analyzed to evaluate the learning resource and determine its attractiveness or evaluate relevant information such as resource quality; the attention concentration obtained by different students at different times in the same video courseware can also be analyzed to analyze the time nodes in the video courseware where the degree of attention is high or low, so as to adjust the courseware in time, or mark the learning points according to the time nodes with high concentration, so as to prompt students to pay attention and optimize learning efficiency.
[0075] On the other hand, the source of learning habit data can mainly be the user operation data during learning recorded in the system, and the learning habit data within a specific period (such as one week or 15 days) can be counted for analysis.
[0076] In an optional implementation, the learning habit data includes browsing speed data and learning time habit data during the learning process of the target subject;
[0077] Before the above step 102, the above method may further include:
[0078] 021. Obtaining browsing speed data of the above-mentioned students during the learning process of the above-mentioned target subjects;
[0079] 022. Generate a target browsing speed curve based on the browsing speed data, wherein the target browsing speed curve represents the browsing speed of the student in different time periods during the learning process of the target subject;
[0080] 023. Determine the browsing preference type corresponding to the target browsing speed curve according to the correspondence between the preset browsing speed curve and the browsing preference type;
[0081] 024. Count the study time and the duration of each study in the process of learning the above-mentioned target subject within the preset period to obtain the above-mentioned study time habit data, wherein the above-mentioned study time includes the date and specific time period of the current study;
[0082] The above step 102 may include:
[0083] Use the learning preference analysis model to analyze the learning status data and learning habit data of the above-mentioned students, associate the learning time of the above-mentioned students in the process of learning the above-mentioned target subjects with the corresponding learning status and learning habits, and generate the learning preference analysis results of the above-mentioned students, including the learning ability level or score in the above-mentioned target subjects, and the learning status and learning habits corresponding to different learning times.
[0084] The learning preference analysis model in the embodiments of the present application can be selected, set or adjusted as needed. In an optional implementation, the model can be trained based on sample data to obtain a trained learning preference analysis model.
[0085] Browsing speed data may mainly include data on browsing operations on video or text learning resources within a specific period, such as actual viewing time, speed-up time, browsing speed, interval time, pause time, etc.
[0086] In an optional implementation, for learning status data, data such as speed duration, browsing speed and interval duration may also be considered; that is, for the learning preference analysis model, learner status characteristics can be obtained from speed duration, browsing speed and interval duration, and learner learning content preference characteristics can be obtained from information such as actual viewing time and number of views.
[0087] In an optional implementation, a target browsing speed curve may be generated based on the browsing speed data. The curve may represent the student's browsing speed in different time periods during the target subject learning process, and may be understood as a curve showing the relationship between time and browsing speed within a specific period.
[0088] In the embodiment of the present application, the correspondence between the browsing speed curve and the browsing preference type can be set as needed, that is, template curves of multiple browsing preference types can be set; the browsing preference type corresponding to the target browsing speed curve is determined by calculating the similarity between the generated target browsing speed curve and the template curve.
[0089] The above browsing preference types can be set as needed, such as a common one: always browsing at 1.5 times, or another common one: always pausing browsing, and so on.
[0090] For example, one browsing mode may be: the browsing speed data shows that the browsing speed in the non-key chapter time period is greater than one speed, and the browsing speed in the key chapter is not greater than one speed, which can be set as the first browsing preference type;
[0091] For another example, a browsing mode may be: the browsing speed in the entire time period is greater than one time, which may be set as the second browsing preference type.
[0092] Optionally, different browsing preference types may be distinguished based on parameters such as an average value of browsing speed in a specific period, and the embodiment of the present application does not impose any limitation on this.
[0093] For study time habit data, the study time during the target subject study process within a preset period and the duration of each study can be counted to obtain the above study time habit data, wherein the study time may include relevant time information such as the date and specific time period of the current study.
[0094] The learning preferences in the embodiments of the present application are mainly based on two aspects of analysis, namely, learning status and learning habits. In one embodiment, since the data for determining the learning status and learning preferences usually occur in chronological order, such as the screen of a monitoring video of a learning process over a period of time, or the speed of browsing videos over a period of time, etc., there may be a corresponding relationship between the learning status data and the learning habit data within the same time period. For example, when a student is watching a video to learn, the current video screen of the student captured by the camera corresponds to the current browsing speed of the student.
[0095] In another embodiment, a specific learning time unit can be set to associate learning status and learning habits. For example, if it is set to one day, the learning status data and learning habit data of that day will be associated and analyzed, and the learning preference analysis results can show the daily learning status and learning habit related preferences.
[0096] The above-mentioned learning preference analysis results can show the learning status and learning habits corresponding to different learning times in the learning process of the target subject. Students or parents can understand the students' learning situation, such as the learning status and learning habits on a certain day, as well as the regularity of the learning status and learning habits over a period of time, so as to adjust the learning methods or make appropriate self-improvements. In addition to understanding the students' learning situation, platforms or teachers can also make suggestions on students' learning methods based on this, and specifically assist different students to better learn according to their own preferences.
[0097] Optionally, the learning preference analysis model can integrate the learning status data and the learning habit data to perform learning preference analysis. Specifically, a trained network model can be used, or weighted calculations can be performed based on a preset algorithm to obtain the final learning preference analysis results.
[0098] 103. Based on the above learning ability assessment results and the above learning preference analysis results, generate the above student's comprehensive assessment results and output the above student's recommended learning plan.
[0099] The comprehensive evaluation results in the embodiments of the present application may take into account the above-mentioned learning ability assessment results and the above-mentioned learning preference analysis results, demonstrate the students' learning ability and learning preferences, and recommend learning plans based on the students' learning ability and learning preferences.
[0100] In an optional implementation, the recommended learning plan includes a recommended learning mode, a learning progress plan, and recommended test questions;
[0101] The above learning progress plan includes fixed learning time periods and corresponding learning contents;
[0102] The above-mentioned recommended learning mode and the above-mentioned learning progress plan are determined based on the above-mentioned learning ability assessment results and the above-mentioned learning preference analysis results;
[0103] The recommended test questions are determined based on the browsing preference type of the student and the learning ability assessment result.
[0104] Specifically, the recommended learning mode and learning progress plan can mainly take into account the results of learning ability assessment and learning preference analysis. For example, the learning mode is divided into difficult, general and simple modes corresponding to the level of learning ability. There are differences in the difficulty of the corresponding learning content, teaching methods, learning time, etc. In addition, the learning status and learning habits corresponding to different learning times in the learning process are considered, and specific learning time periods are arranged for learning or push.
[0105] The recommended test questions can be determined based on the students' learning status and learning ability assessment results. The number, difficulty or type of test questions corresponding to different learning statuses and learning abilities can be pre-set. For example, the higher the learning ability, the higher the difficulty of the test questions. The better the current learning status, the higher the difficulty and the greater the number of test questions. When the learning status is poor, simpler and fewer test questions will be pushed, and so on. This application does not impose any restrictions on this.
[0106] The present application provides a method, device, electronic device and storage medium for comprehensive evaluation of learning ability and preference. The method obtains historical learning test data of students, evaluates the historical learning test data, and determines the evaluation result of the student's learning ability; determines the analysis result of the student's learning preference based on the student's learning status data and learning habit data; generates a comprehensive evaluation result of the student based on the learning ability evaluation result and the learning preference analysis result, and outputs a recommended learning plan for the student. The method can evaluate the learning ability based on the student's learning test data, and understand the student's learning preference based on the learning status and learning habits, so as to further analyze and obtain a comprehensive evaluation result of the student's learning situation, and can put forward reasonable suggestions for the learning plan in a targeted manner, so as to obtain the learner's learning status more comprehensively and accurately, and meet the learner's learning needs and efficiency needs.
[0107] See also Figure 2 , Figure 2 A schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application. Figure 2 As shown in the figure, the method mainly introduces the steps of identifying video images to determine the degree of attention concentration by learning a state analysis model:
[0108] 201. Identify and extract a head portrait region image in a video screen; if the head portrait region image is not identified in the video screen, determine that the student in the video screen has left;
[0109] 202. Input the head portrait region image into a learning state analysis model for processing, and determine the attention type corresponding to the head portrait region image, where the attention type is focused attention or distracted attention;
[0110] 203. Determine the degree of concentration during the learning of the target subject according to the proportion of the video footage with the concentration in all the video footage.
[0111] Specifically, the avatar area image in the video screen can be identified and extracted through the avatar recognition model. The avatar area image is not necessarily a complete face image, and may include a head area that is not facing straight or an obstructed face area, such as a head area with a buried head posture, or a face area avatar with the hand touching the eyes.
[0112] If the head portrait area image is not recognized in the video screen, it can be determined that the student in the video screen has left. At this time, the concentration level corresponding to the video screen is low, for example, it can be directly recorded as 0.
[0113] The head portrait area image is input into the learning state analysis model, and the learning state analysis model can obtain the probability distribution of the corresponding attention type to determine the current attention type. The above attention type can be divided into focused attention (1) or distracted attention (0); optionally, the attention level can also be set, for example, it can be divided into three types: high attention, general attention, and low attention, which is not limited in the embodiment of the present application.
[0114] After determining the degree of concentration corresponding to each video image to be identified, the proportion of video images to be identified with concentrated attention (or high attention) in all video images to be identified can be calculated to determine the degree of concentration during the learning process of the above target subject. The proportion is positively correlated with the degree of concentration. Here, the degree of concentration can be understood as the degree of concentration of the student watching the video during the learning period; and the type of attention obtained by processing a certain video screen is the student's attention at the current moment, and the type of attention at different moments may be different.
[0115] In an optional implementation, the training method of the above learning state analysis model includes:
[0116] a. Obtain a sample image training set, wherein the sample image training set includes sample images of an unobstructed head area and sample images of a head area obstructed by hand movements, wherein the sample images carry key point information and labels indicating attention types and posture labels; the key point information includes feature points located for the head area, eyes, eyeballs and mouth of the human face in the sample images, and the feature points form contour information of the head area, eyes, eyeballs and mouth;
[0117] b. Use the above sample image training set to train the preset neural network model until the parameters reach the preset convergence conditions or the cycle reaches the preset number of times, and obtain a trained learning state analysis model.
[0118] The learning state analysis model in the embodiment of the present application can select a neural network model as needed. In one embodiment, a deep learning framework (such as TensorFlow or PyTorch) can be used to train a neural network model. The model architecture may include convolutional neural networks (CNNs), such as VGG, ResNet, or MobileNet. During the training process, the model will learn how to map the input head image area to a specific category, that is, the above-mentioned attention type.
[0119] The above-mentioned sample images in the embodiments of the present application can refer to the avatar area in the aforementioned video image, mainly including character images in the learning process, usually including the head area, which can include a complete face area, that is, a sample image of an unobstructed avatar area, or an incomplete face, such as a sample image of an avatar area obstructed by hand movements.
[0120] The image preprocessing in the embodiments of the present application may include face detection and positioning, cropping, scale normalization, equalization, de-illumination, fiber compensation, homomorphic filtering, grayscale, Gaussian smoothing, etc. on similar face images.
[0121] Key point information is annotated in the sample images, mainly the feature points located in the head area, eyes, eyeballs and mouth of the face, to form corresponding contour information, because these features affect the type of attention. For example, if the eyes are not looking directly at the screen, it can be considered that the person is not paying attention, or if the person yawns (with eyes squinting) and has an open mouth, it can be considered that the person is not paying attention, etc. At the same time, the sample images are annotated with corresponding attention type labels and posture labels.
[0122] The above-mentioned posture labels mainly indicate the posture information of the current head area, such as turning the head and burying the head, and may also include hand movement information related to the head area, such as rubbing the eyes and supporting the face.
[0123] Specifically, the posture features, eye features and expression features related to attention of the sample image can be extracted, and these features can be trained according to a specific algorithm to obtain a trained learning state analysis model.
[0124] Among them, the method of obtaining posture features may include:
[0125] Determine head posture features and angle information based on feature points in the head area;
[0126] The posture feature is obtained according to the head posture feature and angle information, and the position blocked by the hand action. The posture feature may correspond to the above-mentioned posture label.
[0127] Among them, the method of obtaining eye features may include:
[0128] Determine the position information of the eyeball relative to the eye socket according to the contour information of the eye feature points and the contour information of the eyeball feature points;
[0129] The eye expression feature is obtained according to the head posture feature and angle information, and the position information of the eyeball relative to the eye socket.
[0130] Among them, the method of obtaining expression features may include:
[0131] The eye feature points, eyeball feature points and mouth feature points are analyzed to determine the expression features.
[0132] Based on such sample images, the preset neural network model can be trained until the parameters reach the preset convergence conditions or the cycle reaches a preset number of times to obtain a trained learning state analysis model. The embodiments of the present application do not limit the specific parameters and conditions.
[0133] Based on the above description, for example, the neural network architecture design steps in the embodiment of the present application may include the following steps and considerations:
[0134] Determine the network depth: Deeper networks can learn more abstract features, but they are also more complex and require more training data and computing resources. For attention focus recognition, similar to expression recognition, you can choose a medium-depth network such as ResNet-50 or VGG-16, which performs well on multiple visual tasks.
[0135] Select the type of convolutional layer: Standard 2D convolutional layers are suitable for processing image data. In the embodiment of the present application, a convolutional layer with a certain number of filters can be used to extract facial features.
[0136] Use activation function: ReLU activation function is often used to speed up training and improve the nonlinear expression ability of the model.
[0137] Consider the pooling layer: The pooling layer can reduce the number of parameters and computation while making feature detection more robust. Max Pooling is a commonly used pooling strategy.
[0138] Add normalization layers: Batch Normalization can speed up the training process and improve the generalization ability of the model.
[0139] Use Dropout: prevent overfitting and improve the generalization performance of the model.
[0140] Fully connected layer: A fully connected layer is used at the end of the network to map the learned features to the final output category.
[0141] Consider network width: Increasing the width of the network (i.e. increasing the number of units in the convolutional or fully connected layers) can improve the learning ability of the model, but it also increases the computational cost.
[0142] Use Island Loss: During the training process, Island Loss is combined with softmax loss to reduce intra-class differences and increase inter-class differences, thereby improving the discriminative ability of the model.
[0143] Hyperparameter tuning: Determine the optimal hyperparameters such as learning rate, batch size, weight decay, etc. through experiments.
[0144] In a specific implementation, an IL-CNN network architecture may be used, which includes three convolutional layers, followed by Prelu and Batch Normalization (BN), each BN followed by a MaxPooling, and finally connected to two fully connected layers. Driven by softmax and Island Loss, SGD+momentum is used for optimization, batch_size is set to 300, weight_decay is set to 0.05, and each fully connected layer is followed by a dropout layer.
[0145] The Island Loss mentioned in the embodiment of the present application is a new loss function that can be used to solve the problem of class imbalance in expression recognition. It focuses on samples that are difficult to classify by adjusting the weight of the loss function.
[0146] In the embodiment of the present application, a loss function can be defined, which calculates the softmax loss and the Island Loss and combines them. In one embodiment, the softmax loss can be calculated first, then the Island Loss can be calculated, and finally the two can be combined for back propagation. It should be noted that the features are detached, which means that when calculating the Island Loss, the gradient of the softmax loss can be not calculated, which helps to prevent gradient conflicts. In practical applications, it can be adjusted according to the specific model and data set.
[0147] Island Loss uses the Center Loss part to reduce intra-class variations, that is, to make the feature vectors of samples of the same class closer to the center of the class. Then, the Penalty part is used to expand the inter-class differences, that is, to make the centers of different classes as far apart as possible. The Penalty part calculates the cosine distance between the class centers and ensures that this distance is large enough to maximize the inter-class differences.
[0148] In practical applications, Island Loss can be used in combination with the softmax loss function. Softmax is responsible for basic classification tasks, while Island Loss is responsible for improving the discriminative ability of features.
[0149] The final loss function in the embodiment of the present application is the softmax loss (L softmax ) and Island Loss(L island ) is the weighted sum of:
[0150] L=L softmax +λL island
[0151] Where λ is the weight coefficient used to balance the softmax loss and Island Loss.
[0152] In an optional implementation, the softmax loss in the embodiment of the present application may mainly include the following losses: posture features, eye features, and expression features.
[0153] Further optionally, weights can be assigned to these features in the loss function as needed to adjust their contribution in the loss function, such as setting feature importance weights.
[0154] Specifically, the weights corresponding to the posture feature, the eye expression feature, and the expression feature may be set to decrease in sequence, for example, 0.4, 0.35, and 0.25, respectively, which is not limited in the present embodiment.
[0155] During the training process, the weight coefficients usually need to be adjusted through experiments to find the best performance balance. For example, the weight coefficients in Island Loss can be determined through cross-validation to determine their optimal values. The labels required for training samples are usually category labels, which are labels indicating which expression category each sample belongs to.
[0156] In addition, the implementation steps of Island Loss include feature extraction, definition of loss function, forward propagation, loss calculation, back propagation and parameter update. In practical applications, it may be necessary to iterate these steps multiple times to continuously adjust and optimize the model to achieve the best performance.
[0157] Considering that features between different categories may overlap, the distance between centers of different categories should be maximized.
[0158] Island loss is an improvement on the calculation of inter-class feature distance based on center loss, with the aim of further increasing the inter-class feature distance and improving the accuracy of expression recognition. The specific calculation formula is as follows:
[0159]
[0160] Among them, L C is the center loss;
[0161] β is the balancing factor; N is the set of attention type centers in each mini-batch;
[0162] C j and C k is the category center of attention type labels j and k; δ is the stability coefficient.
[0163] The first term in the above formula represents the penalty for the distance between the training sample and the cluster center; the second term represents the constraint on the distance between centers of different categories. By minimizing the isolation loss, samples of the same category are close in the feature space, while samples of different categories are separated in the feature space.
[0164] The center loss mentioned in the embodiment of the present application means providing a category center for each category and minimizing the distance between each sample in a small batch and the corresponding category center, so as to achieve the purpose of reducing the distance within the category.
[0165] The calculation formula of center loss is as follows:
[0166]
[0167] Among them, x i represents the feature vector of the i-th sample, y i represents the category label of the i-th sample, cyi It is the cluster center of all samples of the same class in each mini-batch, which has the same dimension as the feature. m represents the number of samples in a mini-batch. By minimizing the center loss, samples of the same class will move closer to the cluster center in the feature space, so the overall intra-class difference will be reduced.
[0168] The center loss is introduced into the convolutional neural network to assist the maximum soft margin loss in multi-classification training, so as to reduce the intra-class feature differences.
[0169] In order to further improve the performance, the Island Loss can be improved. In the embodiment of the present application, an exponential loss can be introduced to reduce the intra-class distance, and the cosine distance loss and the Euclidean metric loss can be used to increase the inter-class distance.
[0170] Based on the description of the aforementioned method embodiment, the embodiment of the present application also provides a learning ability and preference comprehensive evaluation device.
[0171] Figure 3 A schematic diagram of a learning ability and preference comprehensive evaluation device provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the device 300 includes:
[0172] The learning ability evaluation module 310 is used to obtain the student's historical learning test data, evaluate the historical learning test data, and determine the learning ability evaluation result of the student;
[0173] A learning preference analysis module 320, used to determine the learning preference analysis result of the student according to the learning status data and learning habit data of the student;
[0174] The evaluation recommendation module 330 is used to generate a comprehensive evaluation result of the student based on the learning ability evaluation result and the learning preference analysis result, and output a recommended learning plan for the student.
[0175] It can be understood that the relevant contents of each module in the above device have been described in detail in the aforementioned method embodiment, and the details can be referred to the contents in the method embodiment; that is, a learning ability and preference comprehensive evaluation device provided in this application can be executed as follows Figure 1 or Figure 2 Any steps in the illustrated embodiment will not be described in detail here.
[0176] In one embodiment of the present application, an electronic device is also provided. The electronic device may include a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the execution Figure 1 or Figure 2Any step in the method embodiment shown. The electronic device may also include an input / output device, etc. In a specific implementation, the electronic device may be a terminal device, etc.
[0177] In one embodiment, a computer-readable storage medium is further provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes any step in the above method embodiment.
[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0180] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A comprehensive evaluation method for learning ability and preference, characterized in that: The method comprises: Acquire historical learning test data of the student, evaluate the historical learning test data, and determine the learning ability evaluation result of the student; Determining a learning preference analysis result of the student according to the student's learning status data and learning habit data; Based on the learning ability assessment results and the learning preference analysis results, a comprehensive assessment result of the student is generated, and a recommended learning plan for the student is output.
2. The comprehensive evaluation method for learning ability and preference according to claim 1 is characterized in that: The obtaining of the student's historical learning test data includes: Obtaining the test results of the student's target subject, the speed of answering questions, the study time and the learning speed of the student for the target subject, wherein the test results include the student's correct rate in the test of the target subject and the individual ranking of the test; The step of evaluating the historical learning test data to determine the learning ability evaluation result of the student includes: The historical learning test data is input into a preset evaluation model, and the learning ability evaluation result is output, wherein the learning ability evaluation result includes the learning ability grade or score of the student in the target subject.
3. The comprehensive evaluation method for learning ability and preference according to claim 2 is characterized in that: The learning status data includes the degree of concentration during the learning of the target subject; Before determining the student's learning preference analysis result according to the student's learning status data and learning habit data, the method further includes: Collecting video footage of the student during the learning process of the target subject; Performing image analysis on the video screen to determine the concentration of the student in the video screen; Based on the concentration of the student in the video image, the degree of concentration during the learning of the target subject is determined.
4. The comprehensive evaluation method for learning ability and preference according to claim 3 is characterized in that: The performing image analysis on the video screen to determine the concentration of the student in the video screen includes: Identify and extract the head portrait area image in the video screen; Inputting the head portrait area image into a learning state analysis model for processing, and determining an attention type corresponding to the head portrait area image, wherein the attention type is focused attention or distracted attention; The determining of the degree of concentration of the student in the process of learning the target subject based on the concentration of the student in the video screen includes: The degree of concentration during the target subject learning process is determined according to the proportion of the video frames on which the attention is concentrated in all the video frames.
5. The method for comprehensive evaluation of learning ability and preference according to claim 4, characterized in that: The training method of the learning state analysis model includes: Acquire a sample image training set, the sample image training set including sample images of an unobstructed head area and sample images of a head area obstructed by hand movements, the sample images carrying key point information, a label indicating an attention type, and a posture label; the key point information includes feature points located for a head area, eyes, eyeballs, and mouth of a human face in the sample images, the feature points forming contour information of the head area, eyes, eyeballs, and mouth of the human face; The preset neural network model is trained using the sample image training set until the parameters reach a preset convergence condition or the cycle reaches a preset number of times, thereby obtaining a trained learning state analysis model.
6. The comprehensive evaluation method for learning ability and preference according to claim 3 is characterized in that: The learning habit data includes browsing speed data and learning time habit data during the learning process of the target subject; The method further comprises: Acquiring browsing speed data of the student during the learning process of the target subject; generating a target browsing speed curve based on the browsing speed data, wherein the target browsing speed curve represents the browsing speed of the student in different time periods during the learning process of the target subject; Determining the browsing preference type corresponding to the target browsing speed curve according to a preset correspondence relationship between the browsing speed curve and the browsing preference type; Counting the study time during the target subject study process within a preset period and the duration of each study to obtain the study time habit data, wherein the study time includes the date and specific time period of the current study; Determining the student's learning preference analysis result according to the student's learning status data and learning habit data includes: Use a learning preference analysis model to analyze the student's learning status data and learning habit data, associate the student's study time during the target subject learning process with the corresponding learning status and learning habits, and generate the student's learning preference analysis results, including the learning status and learning habits corresponding to different study times during the target subject learning process.
7. The method for comprehensive evaluation of learning ability and preference according to claim 6, characterized in that: The recommended learning plan includes a recommended learning mode, a learning progress plan, and recommended test questions; The learning progress plan includes a fixed learning time period and corresponding learning content; The recommended learning mode and the learning progress plan are determined based on the learning ability assessment result and the learning preference analysis result; The recommended test questions are determined based on the learning status and the learning ability evaluation result of the student.
8. A learning ability and preference comprehensive evaluation device, characterized in that: include: A learning ability evaluation module is used to obtain the student's historical learning test data, evaluate the historical learning test data, and determine the student's learning ability evaluation result; A learning preference analysis module, used to determine the learning preference analysis result of the student according to the learning status data and learning habit data of the student; An assessment recommendation module is used to generate a comprehensive assessment result of the student based on the learning ability assessment result and the learning preference analysis result, and output a recommended learning plan for the student.
9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 7.