Concentration degree monitoring method and device based on visual tracking, equipment and storage medium

Abstract

The invention relates to the technical field of concentration degree monitoring, solves the problems that an existing target user concentration degree evaluation method is not comprehensive enough and low in accuracy, and provides a concentration degree monitoring method and device based on visual tracking, equipment and a storage medium. The method comprises the following steps: acquiring video data including a target user; target information is obtained according to the video data and a preset neural network model, and the target information comprises head posture information, pupil position information and reading material position information of the target user; obtaining eye focus identification information of the target user according to the target information; and according to the sight focus identification information and a preset concentration scoring rule, obtaining a concentration score of the target user. According to the invention, the concentration degree of the target user can be monitored in real time, the target user is reminded to concentrate attention through sound or visual prompt, and the concentration degree of the target user and the concentration strategy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of concentration monitoring, and in particular relates to a concentration monitoring method, device, equipment and storage medium based on visual tracking. Background Art

[0002] Education is the cornerstone of a country, and the quality of teaching also represents the quality of the country's new generation. Improving teaching quality is a topic that every educator is exploring, and students' learning concentration is an important evaluation indicator of teaching quality. Traditional technology uses eye tracking technology to record students' gaze patterns when reading and observing teaching content to monitor students' concentration. Based on eye tracking data, educators can provide personalized feedback and guidance to help students optimize their learning methods and concentration strategies.

[0003] Patent No. CN 113283334 B discloses a method, device and storage medium for analyzing classroom concentration. The method obtains video data and performs multimodal concentration detection consisting of head posture detection, facial expression detection, action detection and mobile phone detection on students based on the video data. Finally, the multimodal features are put into a multimodal rule fusion network based on a gated filtering mechanism for fusion to obtain the student's concentration score. The method can quantify and measure the student's listening concentration, thereby helping educators to understand the student's listening situation more simply and effectively. Although the existing evaluation technology has achieved certain success, there are still some defects. The method does not distinguish the concentration evaluation of students' specific learning tasks and set corresponding attention scoring rules. At the same time, it does not divide the learning area of ​​students' reading materials in specific learning scenarios and assign concentration scoring weights. This may have certain limitations in accuracy and cannot comprehensively evaluate students' concentration. In addition, tracking eye movement information cannot cover a variety of situations when students are learning. Eye tracking technology is especially useful in non-standard learning environments. For example, when students are immersed in reading materials, the technology may not be able to capture students' eye tracking information due to the limitations of the acquisition equipment, and therefore cannot accurately capture students' true state of learning concentration.

[0004] Therefore, how to more accurately capture and analyze students' eye movements and head postures, and then evaluate the distribution of students' learning attention, and help students optimize their learning methods and concentration strategies is an urgent problem to be solved. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a concentration monitoring method, device, equipment and storage medium based on visual tracking, so as to solve the problem that the existing student learning concentration evaluation method is not comprehensive enough and has low accuracy.

[0006] In a first aspect, an embodiment of the present invention provides a method for monitoring concentration based on visual tracking, the method comprising:

[0007] Acquire video data including the target user;

[0008] Acquire target information according to the video data and a preset neural network model, wherein the target information includes head posture information, pupil position information, and reading material position information of the target user;

[0009] According to the target information, obtaining the gaze focus identification information of the target user;

[0010] The concentration score of the target user is obtained according to the gaze focus recognition information and preset concentration scoring rules.

[0011] Preferably, the acquiring target information according to the video data and the preset neural network model comprises:

[0012] Acquire an image sequence according to the video data, wherein the image sequence includes a plurality of image frames;

[0013] Inputting the image sequence into a pre-built neural network model to obtain a target detection image, wherein the target detection image includes an eye region image, a head region image, and a reading material region image;

[0014] The target information is obtained according to the target detection image.

[0015] Preferably, obtaining the target information according to the target detection image includes:

[0016] Acquiring the pupil position information according to the eye region image;

[0017] Calculating a head posture rotation matrix of the target user according to the head region image to obtain the head posture information;

[0018] The reading material position information is acquired according to the reading material area image, wherein the pupil position information, the head posture information and the reading material position information all include a direction vector and a center coordinate.

[0019] Preferably, acquiring the pupil position information according to the eye region image comprises:

[0020] According to a preset Gaussian processing rule, Gaussian processing is performed on the eye region image to obtain an optimized image;

[0021] Performing grayscale transformation on the optimized image to obtain a grayscale image;

[0022] Performing binarization processing on the grayscale image to obtain a binarized image;

[0023] Acquiring edge positioning information according to the binary image;

[0024] Acquiring edge contour information according to the edge positioning information;

[0025] According to the edge contour information, obtain the circularity of several contours;

[0026] According to the circularity of each contour, obtaining a first contour with the largest circularity;

[0027] Pupil position information is acquired according to the first contour.

[0028] Preferably, performing Gaussian processing on the eye region image to obtain an optimized image according to a preset Gaussian processing rule comprises:

[0029] Performing Gaussian blur processing on the eye region image according to a preset Gaussian kernel standard deviation to obtain a first image;

[0030] According to preset histogram equalization parameters, performing histogram equalization processing on the first image to obtain a second image;

[0031] Perform Gaussian blur processing on the second image to obtain an optimized image after Gaussian blur processing.

[0032] Preferably, the step of acquiring the gaze focus identification information of the target user according to the target information;

[0033] Acquire the sight direction according to the pupil position information and the head posture information;

[0034] According to the sight line direction, obtaining a sight line direction vector;

[0035] According to the reading material position information, obtaining a reading material direction vector;

[0036] According to the sight line direction vector and the reading material direction vector, the two are calculated according to a preset calculation rule to obtain the gaze focus identification information of the target user, and the gaze focus identification information includes the angle between the sight line direction and the reading material.

[0037] Preferably, obtaining the concentration score of the target user according to the gaze focus identification information and a preset concentration scoring rule comprises:

[0038] In each preset scoring cycle, when the angle between the sight direction and the reading material is less than or equal to a preset angle threshold, the concentration time of the target user is timed to obtain the initial concentration time;

[0039] When the initial concentration time is greater than or equal to a preset time threshold, the initial concentration time is used as the target concentration time;

[0040] Get a basic score based on the target focus time;

[0041] Obtain reward scores based on preset concentration scoring rules;

[0042] The concentration score of the target user is obtained according to the basic score and the reward score, and a reminder is issued if the concentration score is less than or equal to a preset total score fixed value.

[0043] In a second aspect, an embodiment of the present invention provides a device for monitoring student learning concentration based on visual tracking technology, the device comprising:

[0044] A data acquisition module, which acquires video data including the target user;

[0045] A data training module, which acquires target information according to the video data and a preset neural network model, wherein the target information includes head posture information, pupil position information, and reading material position information of the target user;

[0046] A gaze focus recognition module, which obtains gaze focus recognition information of the target user according to the target information;

[0047] The concentration scoring reminder module obtains the concentration score of the target user according to the gaze focus recognition information and preset concentration scoring rules.

[0048] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect in the above-mentioned embodiment.

[0049] In a fourth aspect, an embodiment of the present invention provides a storage medium having computer program instructions stored thereon, which implements the method of the first aspect in the above-mentioned embodiment when the computer program instructions are executed by a processor.

[0050] In summary, the beneficial effects of the present invention are as follows:

[0051] The embodiment of the present invention provides a method, device, equipment and storage medium for monitoring concentration based on visual tracking, which obtains video data including a target user; obtains target information according to the video data and a preset neural network model, and the neural network model has obvious advantages over the traditional manual monitoring method. The neural network model can identify the complex concentration mode and behavior characteristics of the target user through a deep learning algorithm, so that the system can more accurately monitor the state of the target user; obtains the eye focus recognition information of the target user according to the target information, and uses Gaussian blur, Canny edge extraction and binarization image processing algorithms, and through the preset YoloV8s neural network model, accurately captures the visual focus of the target user and identifies the focus point of the target user in the environment; obtains the concentration score of the target user according to the eye focus recognition information and the preset concentration scoring rule, and presets a dynamic scoring rule, which can automatically adjust the scoring standard according to the duration and frequency of the student's behavior. When the system detects that the concentration of the target user drops below a certain threshold, it will remind the target user to concentrate through sound or visual prompts, thereby helping to improve the learning efficiency of the target user. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the drawings required for use in the embodiment of the present invention will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work, and these are all within the protection scope of the present invention.

[0053] Figure 1 It is a flow chart of a method for monitoring concentration based on visual tracking according to an embodiment of the present invention.

[0054] Figure 2 It is a flow chart of a method for monitoring concentration based on visual tracking according to an embodiment of the present invention.

[0055] Figure 3 It is another flow chart of an embodiment of the present invention for a method for monitoring concentration based on visual tracking.

[0056] Figure 4 It is another flow chart of an embodiment of the present invention for a method for monitoring concentration based on visual tracking.

[0057] Figure 5 It is a structural schematic diagram of a concentration monitoring device based on visual tracking according to an embodiment of the present invention.

[0058] Figure 6 It is a schematic diagram of the structure of a device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present invention and are not configured to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0061] It should be noted that all actions of acquiring signals, information or data in the present invention are performed in compliance with the corresponding data protection laws and policies of the location and with the authorization given by the owner of the corresponding device.

[0062] Example 1

[0063] See also Figure 1 , an embodiment of the present invention provides a method for monitoring concentration based on visual tracking, the method comprising:

[0064] S1. Acquire video data including the target user;

[0065] Video data can monitor the concentration of target users in real time, provide timely feedback, and help target users adjust their learning status. Teachers can analyze the learning results of target users based on video data, formulate targeted teaching strategies, improve teaching effectiveness, and analyze changes in the learning status of target users, which can provide reference for long-term learning planning and help discover potential problems.

[0066] Specifically, the video data of the target user is acquired in real time through the camera device.

[0067] S2. Acquire target information according to the video data and a preset neural network model;

[0068] Current target detection models are divided into single-stage and dual-stage models. The dual-stage model requires a two-stage processing flow, the first stage is usually to generate candidate frames, and the second stage is to classify and regress the candidate frames, which causes the model to be slow in recognition speed. The single-stage model directly predicts bounding boxes and categories, generates bounding boxes and corresponding category probabilities at one time, and usually uses dense prediction grids to cover the entire input image. Each grid unit directly outputs multiple bounding boxes and their corresponding category confidences. Since the detection task is completed directly in a single forward propagation, the single-stage model is usually faster and is suitable for the real-time teaching monitoring scenario of the present invention. Common single-stage models include Yolov8s model and SSD model. The Yolo series model is famous for its extremely fast detection speed. Yolov8s is further optimized on the basis of the series, and can perform real-time target detection at high frame rates. By improving the network structure and loss function of the original Yolo series, it is usually possible to provide higher detection accuracy and accuracy while maintaining the speed advantage. The needs of mobile devices and embedded devices are taken into consideration at the same time during design, and the model is lightweight and easy to deploy. In view of the requirements of real-time, accuracy and easy deployment in teaching classroom scenarios, the present invention adopts the Yolov8s neural network model.

[0069] Preferably, see Figure 2 , the acquiring target information according to the video data and the preset neural network model includes:

[0070] S21. Acquire an image sequence according to the video data, where the image sequence includes a plurality of image frames;

[0071] Neural network model training usually uses image input for training, so the target user's video data needs to be decoded and extracted frame by frame to obtain several image frames for training.

[0072] Specifically, the video data is decoded into a sequence of original image frames, and then image frames are extracted from the decoded video data, each frame representing a still image in the video, and finally the extracted image frames are organized into an image sequence in chronological order for subsequent image processing or analysis.

[0073] S22, inputting the image sequence into a pre-built neural network model to obtain a target detection image, wherein the target detection image includes an eye region image, a head region image, and a reading material region image;

[0074] By analyzing the position of the eyes and head, you can determine the target user’s attention area and help them focus better.

[0075] Specifically, the picture sequence is input into the Yolov8s model, each image is processed frame by frame and the target detection task is performed. The model will analyze the target position and category in each frame image and generate a target detection image, which includes an eye area image, a head area image and a reading material area image.

[0076] S23, obtaining the target information according to the target detection image;

[0077] Target detection images can provide multi-dimensional information, such as the location information of eyes, head, and reading material areas. By analyzing the location information, teachers can fully understand the learning status of target users, develop personalized learning plans that meet their needs, and improve learning outcomes.

[0078] S231, acquiring the pupil position information according to the eye region image;

[0079] By accurately locating the pupil, the user's attention level can be monitored in real time, thereby optimizing the learning or working environment, reducing human intervention and errors, and continuously monitoring the user's eye movements to provide dynamic attention analysis.

[0080] Preferably, acquiring the pupil position information according to the eye region image comprises:

[0081] S2311, performing Gaussian processing on the eye region image according to a preset Gaussian processing rule to obtain an optimized image;

[0082] The purpose of Gaussian processing is to reduce image noise and improve contrast to facilitate subsequent analysis and processing. Circularity provides a method to quantify shape features, which is conducive to target recognition and tracking. The circularity values ​​of different shapes can be used to establish feature vectors to improve the accuracy of target recognition.

[0083] Specifically, according to a preset Gaussian processing rule, Gaussian processing is performed on the eye region image, and the Gaussian processing process is as follows:

[0084] I′ eye (t) = G(H(G(I eye (t),σ),α)

[0085] Where G represents Gaussian blur, ο is the standard deviation of the Gaussian kernel, which is used to control the blur degree, H represents the histogram equalization operation, which is the parameter of α histogram equalization, which is used to adjust the contrast of the image, and I eye (t) represents the eye region image. eye(t) Gaussian blur processing is performed using the standard deviation ο of the Gaussian kernel to obtain a first image, histogram equalization processing is performed on the first image using a control parameter α to obtain a second image, and Gaussian blur processing is performed on the second image to obtain an optimized image.

[0086] S2312, performing grayscale transformation on the optimized image to obtain a grayscale image;

[0087] Grayscale images have only one channel and are more efficient in calculation and processing than color images. They can work under a variety of lighting conditions and are easy to extract effective features. When extracting the pupil position, a clear grayscale image can help the algorithm identify and locate the pupil more accurately. Grayscale images are the basis for subsequent processing such as binarization and edge detection, ensuring the effectiveness of the entire process.

[0088] Specifically, the threshold function of the OpenCV image processing library is used to perform grayscale transformation on the optimized image to obtain a grayscale image.

[0089] S2313, performing binarization processing on the grayscale image to obtain a binarized image;

[0090] Binarization converts grayscale information into black and white, reducing the amount of data for easy subsequent processing, emphasizing important features in the image such as edges and contours, facilitating subsequent contour extraction and analysis. Since only two color values ​​need to be processed, the binarized image processing speed is faster. At the same time, threshold processing can also effectively suppress some noise and highlight the main structure of the image.

[0091] Specifically, the OpenCV image processing library is used to perform binarization processing and adaptive threshold segmentation on the grayscale image to obtain a binarized image.

[0092] S2314, acquiring edge positioning information according to the binarized image;

[0093] Edge detection can accurately identify contours and boundaries in images, which is crucial for subsequent shape analysis and feature extraction. Edge images only retain important structural information, reducing the complexity of processing. At the same time, the Canny algorithm has good robustness to noise and can work effectively under different lighting conditions.

[0094] Specifically, the Canny function in OpenCV is used to obtain edge location information of the binary image;

[0095] S2315, acquiring edge contour information according to the edge positioning information;

[0096] Contour extraction can accurately obtain the shape of objects in images, which is helpful for subsequent shape analysis and recognition. The contour represents the shape and boundary of the object, reduces the amount of data to be processed, and improves efficiency. At the same time, multiple geometric features (such as area and perimeter) can be calculated through the contour for further analysis.

[0097] Specifically, the findContours function of OpenCV is used to extract the edge contour information of the binary image from the edge location information.

[0098] S2316, obtaining several contour circularities according to the edge contour information;

[0099] Circularity provides a simple way to quantify the shape of a contour, making it easier to compare and analyze. Contours with high circularity usually represent objects that are close to circular, which helps to identify specific shapes in an image. By calculating the circularity, contours that meet the preset conditions can be screened out more accurately.

[0100] Specifically, according to the edge contour information, the contour circularity is used as an evaluation index to calculate the circularity of each contour, and the perimeter and area of ​​each contour are calculated. The calculation formula is as follows:

[0101]

[0102] S2317, obtaining a first contour with the largest contour circularity according to the contour circularities;

[0103] The maximum circularity indicates the best match of the target and is suitable for application scenarios that require high-precision recognition. After determining the first contour, subsequent operations, such as pupil position extraction, can be performed directly to avoid processing unnecessary contours and improve processing efficiency.

[0104] Specifically, the circularity of all contours is sorted to obtain a circularity sequence, and the contour corresponding to the maximum circularity is the outer contour of the pupil, which is recorded as the first contour.

[0105] S2318. Acquire pupil position information according to the first contour;

[0106] Pupil position information provides a basis for further processing such as user concentration assessment.

[0107] Specifically, the center position of the first contour is acquired to obtain pupil position information P(t).

[0108] S232, calculating the head posture rotation matrix of the target user according to the head region image, and acquiring the head posture information;

[0109] The steps of the traditional multi-person head pose estimation method are to train a dense detection network to directly predict a set of head objects, which contains a set of head bounding boxes and corresponding head poses, and the intrinsic correlation between head detection and pose estimation is combined. Then the head pose is regarded as an additional head attribute and connected with its head position to construct a joint representation of the head object. The two tasks are integrated in a unified framework using joint prediction. A suitable head bounding box has strong local features and weak global features for its head direction estimation, and the DirectMHP (Direct Multi-Person Head Pose) model proposes a novel end-to-end trainable method to achieve joint head detection and pose estimation through shared features. This method integrates head pose as a neighboring attribute into typical target prediction, supports arbitrary pose representation, and enables the detection network to learn their intrinsic relationship. After training, non-maximum suppression is used to filter out false positives and redundant bounding boxes for the predicted object set. By calculating the confidence of each predicted object, there is no need to modify the common maximum suppression step of obtaining the positive head bounding box, which simplifies the detection process of the traditional multi-person head pose estimation model and is more robust and reliable than traditional methods.

[0110] Specifically, the present invention uses an end-to-end DirectMHP model to calculate the head rotation matrix Rhead(t) of multiple heads, obtains the head information image of the target user and calculates the head posture. The various coordinate information of the rotation matrix Rhead(t) of the head posture includes the objectivity probability O of the head target, the candidate bounding box (x, y, w, h), the head target score C, and the candidate Euler angle (pitch angle around the X axis, yaw angle around the Y axis, and roll angle around the Z axis). The total training loss function L of the model is as follows:

[0111]

[0112] Where Nbs is the training batch size, α, β and γ are the loss function L box , L obj and L mse The loss function is used to measure the consistency between the actual annotation result and the model output result, and the head posture information is obtained through the head posture rotation matrix Rhead(t).

[0113] S233, acquiring the reading material position information according to the reading material area image;

[0114] By monitoring the location of reading materials, real-time feedback can be provided to help target users stay focused on the learning content. Teachers can use the reading material location information to guide discussions and interactions and enhance the target users' sense of participation.

[0115] Specifically, the reading material position information is obtained from the bounding box generated by the Yolov8s model, wherein the reading material position information includes the position coordinates of the reading material (the coordinates of the upper left corner and the lower right corner of the rectangular box).

[0116] S3, obtaining the gaze focus identification information of the target user according to the target information;

[0117] The gaze focus data for each target user can help teachers develop personalized learning feedback and suggestions, understand the target user's learning habits and preferences, and provide a basis for subsequent learning plans. By analyzing the correlation between gaze focus and learning materials, the target user's concentration level can be accurately assessed and distractions can be identified in a timely manner.

[0118] Preferably, see Figure 3 , according to the target information, obtaining the gaze focus identification information of the target user includes:

[0119] S31, obtaining a sight line direction according to the pupil position information and the head posture information;

[0120] The head posture is represented by a rotation matrix, and the pupil position directly reflects the gaze direction of the eyes. By subtracting the position of the image center point, the coordinates of the pupil position can be mapped to a position relative to the center of the screen, so that the final line of sight direction is accurate relative to the screen coordinate system.

[0121] Specifically, the line of sight direction V(t) is calculated as follows:

[0122] V(t)=Rhead(t)·(P(t)-P ref )

[0123] Among them, P ref is a reference point. This point is chosen for calibration. It is generally believed that the line of sight will be concentrated in the center of the image under normal circumstances. The difference P(t)-P ref Represents the offset of the pupil position relative to the center of the image, that is, the specific direction the pupil is looking at is offset relative to the center of the screen. By applying the head posture rotation matrix Rhead(t) to the offset (P(t)-P ref ), get the sight direction vector V(t) under the head direction. This vector represents the direction the pupil is actually looking in based on the head orientation. The sight direction is determined by the head orientation and the pupil position.

[0124] S32, obtaining a sight direction vector according to the sight direction;

[0125] Since the gaze direction vector is calculated in real time, the system can quickly evaluate the target user's concentration and promptly remind them when their concentration decreases. This helps the target user to adjust their status in time and improve their learning efficiency.

[0126] Specifically, it is the same as S31 and will not be repeated here.

[0127] S33, acquiring a reading material direction vector according to the reading material position information;

[0128] By calculating the direction vector from the camera reference point to the center of the reading material, the system can quickly and concisely describe the position of the reading material, and because the direction vector has been standardized, subsequent angle calculations will be more efficient. The standardized reading material direction vector can be directly compared with the user's line of sight direction vector to calculate the angle between the user's eye focus and the reading material. This comparison can quickly determine whether the student is focusing on the learning material, improving the real-time and accuracy of concentration monitoring.

[0129] Specifically, according to the central point P of the reading material material and the camera reference point P ref , calculate the direction vector V of the reading material material , the calculation formula is:

[0130] V material =P material -P ref

[0131] This direction vector represents the direction from the camera reference point to the center of the reading material. In order to obtain a unit direction vector, the system will material Perform normalization, that is, convert it into a vector of length 1, so that it can be compared with the sight direction vector later, as follows:

[0132]

[0133] Among them, ||V material || is the magnitude of the vector, i.e. its length.

[0134] S34, according to the sight line direction vector and the reading material direction vector, calculating the two according to a preset calculation rule to obtain the gaze focus identification information of the target user, wherein the gaze focus identification information includes the angle between the sight line direction and the reading material;

[0135] The angle can provide a quantitative indicator to evaluate and quantify the distribution and concentration of the target user's attention, and can effectively evaluate and understand the target user's concentration. The method is highly adaptable to the target user's head posture and the position of the reading material. No matter where the reading material is placed or how the target user moves or turns his head, as long as the system can obtain the line of sight direction and the reading material direction, the angle calculation can be dynamically adjusted.

[0136] Specifically, the gaze focus identification information refers to the relevant angle θ(t) between the gaze direction and the reading material, and the calculation method thereof is as follows:

[0137] Among them, V material is the vector pointing from the center of the camera to the center of the reading material, and is calculated as follows:

[0138] V material =P material -Pref

[0139] Among them, P material Refers to the center coordinates of the detected reading material. The angle θ(t) is calculated using the dot product and modulus of the vector. V(t)·V material is V(t) and V material The dot product of ||V(t)|| and ||V material || are their moduli respectively. The smaller θ(t) is, the closer the eye's sight direction is to the center of the learning material, indicating that the target user's attention is more focused on the reading material.

[0140] S4. Obtaining a concentration score of the target user according to the gaze focus identification information and a preset concentration scoring rule;

[0141] Traditional scoring mechanisms are usually based on fixed measurement standards and cannot reflect the dynamic changes of target users in real time. The dynamic scoring mechanism can monitor the focus and learning progress of target users in real time, and make timely adjustments and feedback based on real-time data. It can track and evaluate the learning behavior and progress of target users more accurately, rather than just quantifying the results. The preset focus scoring rules mainly consider the focus angle and duration to calculate the focus score. This method can help educators understand the learning process of target users more comprehensively, help to more effectively personalize teaching and support the learning needs of target users.

[0142] Preferably, see Figure 4 , according to the gaze focus identification information and the preset concentration scoring rule, obtaining the concentration score of the target user includes:

[0143] S41, in each preset scoring period, when the angle between the sight direction and the reading material is less than or equal to a preset angle threshold, timing the concentration time of the target user to obtain an initial concentration time;

[0144] The preset scoring period T = 30 minutes, and the preset angle threshold θ threshold = 15°, when the angle θ(t) between the sight direction and the reading material is less than or equal to the preset angle threshold θ threshold , the target user is considered to be concentrating on reading materials and the initial concentration time starts to be counted.

[0145] S42: when the initial concentration time is greater than or equal to a preset time threshold, taking the initial concentration time as the target concentration time;

[0146] Preset a time threshold T focused =15 minutes, i.e., the shortest time that the target user continuously focuses on the reading material, which is used to filter short-term concentration. When the initial concentration time is greater than or equal to the preset time threshold, the initial concentration time is used as the target concentration time. When the initial concentration time is less than the preset time threshold, the concentration time does not meet the set minimum duration requirement, and the initial concentration time is not regarded as the target concentration time.

[0147] S43. Obtain a basic score based on the target focus time;

[0148] Specifically, the initial total score S is preset total =0, preset a time threshold T focused = 10 minutes, which is the shortest time that the target user can continuously focus on reading materials, and is used to filter short-term attention. In each learning cycle, if θ(t)≤θ threshold , then the basic score is accumulated, and the basic score accumulation amount S base One point is added for each minute and the total score is obtained as the basic score.

[0149] S44. Obtaining a reward score according to a preset concentration scoring rule;

[0150] Specifically, if the target user's focus time exceeds T focused , then the score in the corresponding time period will receive an additional bonus score S bonus , S bonus For every minute exceeding five minutes, an additional bonus of 0.5 points will be awarded. Any time less than five minutes will be counted as five minutes.

[0151] S45, obtaining the concentration score of the target user according to the basic score and the reward score, and issuing a reminder if the concentration score is less than or equal to a preset total score fixed value;

[0152] By presetting the concentration score threshold and monitoring the target user's score in real time, the dynamic scoring mechanism can promptly issue reminders when the target user is distracted or needs help, thereby prompting the target user to refocus.

[0153] Specifically, a concentration score threshold is preset, and the sum of the basic score and the reward score is the concentration score of the target user. If the concentration score is less than or equal to the preset concentration score threshold, a reminder is issued.

[0154] Here is a specific example:

[0155] θ threshold =15°

[0156] S base =1min / minute

[0157] S bonus = 0.5 points / every time exceeding 5 minutes

[0158] T focused =10 minutes

[0159] In a 30-minute learning cycle, if the target user has 20 minutes of θ(t) ≤ 15°, and there is a continuous 15-minute period of time in which the target user focuses on the reading material, the target user's concentration score can be calculated as follows:

[0160] Basic score: 20×1=20 points

[0161] Sustained focus reward: (15-10) / 5×0.5=0.5 points

[0162] The total score is the sum of the basic score and the sustained concentration bonus score, 20+0.5=20.5 points.

[0163] Example 2

[0164] Different from Example 1, this embodiment makes further improvements in step S2 and step S3. Different types of learning tasks are identified according to the video data and the preset neural network model. Different concentration monitoring models are designed and adopted according to different types of learning tasks instead of adopting a unified standard. The concentration monitoring strategy can be dynamically adjusted according to the characteristics of the task. A concentration monitoring method based on visual tracking is provided, and the method includes:

[0165] S2-1, obtaining a learning task type according to the video data and a preset neural network model;

[0166] The image frames of the video data are detected by the preset neural network model YOLOv8s to perform content recognition, and the YOLOv8s model is trained using a labeled data set (including samples of the learning task type) to iteratively optimize the model parameters to improve classification accuracy. The feature vectors obtained in real time are input into the trained model, and the model classifies according to the learned features and outputs the current learning task type.

[0167] Preferably, obtaining the learning task type according to the video data and the preset neural network model includes:

[0168] S2-11, acquiring a target detection image according to the video data;

[0169] Generating target detection images in real time can provide users or educators with immediate feedback to help adjust learning methods. The generated target detection images provide important data support for subsequent learning task analysis and user behavior research.

[0170] Specifically, the video data is decomposed into continuous image frames to obtain target detection images for subsequent analysis.

[0171] S2-12, obtaining a feature vector according to the target detection image and a preset neural network model;

[0172] The neural network model can extract high-level features from images and provide richer information representation than traditional feature extraction methods. The extracted feature vectors can be used for subsequent learning task classification, behavior recognition, etc., to improve the classification performance of the model, making the application more flexible and adaptable to different learning scenarios.

[0173] Specifically, the target detection image is cropped to the input size of 640x640 required by the neural network model, and the image pixel values ​​are normalized to meet the input requirements of the model. The pixel values ​​are usually scaled to the range of [0,1] or [-1,1]. The preprocessed target detection image is input into the neural network model for forward propagation calculation. The output of the fully connected layer of the neural network is extracted to obtain feature vectors. These feature vectors can effectively represent the main features of the input image.

[0174] S2-13. Obtaining a learning task type according to the feature vector and the preset neural network model;

[0175] The preset neural network model has been trained to provide high-accuracy classification of learning task types and adapt to various learning scenarios. Learning task types include reading, writing, problem solving and watching videos. It uses feature vectors for fast classification and supports real-time feedback. It is suitable for online learning systems. The model can adapt according to different feature vectors and identify different types of learning tasks.

[0176] Specifically, the preprocessed feature vector is input into the neural network model for forward propagation, and the model generates an output vector representing the probability distribution of each learning task type. Usually, the length of the output vector is equal to the number of possible learning task types. The output vector is converted into a probability value using the softmax function, and the category with the highest probability is selected as the final identified learning task type. The learning task type is obtained and the identified learning task type is fed back to the system.

[0177] S2-2, establishing a gaze angle relationship according to the gaze focus identification information and the learning task type of the target user;

[0178] The gaze angle can effectively measure the user's concentration level and help determine whether they are paying attention to the learning materials. The collected angle data can be used for long-term analysis of users' learning habits and behavior patterns, provide decision support for educators, calculate the angle in real time and provide feedback, support dynamic adjustment of learning rhythm and content, and improve learning efficiency.

[0179] Preferably, establishing the gaze angle relationship according to the gaze focus identification information of the target user and the learning task type includes:

[0180] S2-21, obtaining the angle between the sight direction and the reading material according to the gaze focus identification information of the target user;

[0181] Same as S34, no further details are given.

[0182] S2-22, if the learning task is a reading task, establishing a reading task gaze angle relationship according to the angle and a preset first threshold;

[0183] When a learning task is identified as a reading task, the system can more accurately monitor the user's concentration during the reading process. Since reading tasks require the eyes to focus on the material, the system can track the user's concentration in real time according to the set threshold and provide timely feedback. When the system determines that the user is performing a reading task, it will strictly monitor the user according to the concentration requirements of the task, and issue reminders in time when the user is distracted or the concentration decreases, thereby helping students maintain concentration and improve learning efficiency.

[0184] Specifically, if the current learning task is a "reading task", the preset first threshold is 10°, and the target user's gaze angle needs to be continuously smaller than the first threshold within 30 seconds.

[0185] S2-23, if the learning task is a writing task, establishing a writing task gaze angle relationship according to the angle, the preset first threshold and the preset second threshold;

[0186] Writing tasks are usually accompanied by a short thinking process, and the eyes occasionally leave the material. Therefore, by presetting the first threshold and the second threshold, the system can more accurately capture the eye jumping behavior in the writing task and avoid misjudging it as inattention. According to the characteristics of the writing task, since the focus of the eyes changes a lot in the writing task, the use of a single angle threshold may be misjudged as inattention. Therefore, the system allows the eyes to leave the writing material moderately, which avoids misjudgment of the thinking process. Compared with the stricter monitoring standards for reading tasks, the monitoring standards for writing tasks are more flexible, reduce misjudgments, and allow the target user to concentrate on writing without being disturbed too much.

[0187] Specifically, if the current learning task is a "writing task", the second threshold is preset to 15°. Since writing tasks are usually accompanied by thinking and a brief period of time away from the material, a moderate change in the angle is in line with the characteristics of a writing task. The system continuously detects the user's gaze focus within 30 seconds. During this period, the target user's gaze angle must be greater than the first threshold of 10° and less than the second threshold.

[0188] S2-24, if the learning task is a problem-solving task, establishing a gaze angle relationship for the problem-solving task according to the angle, the preset second threshold and the preset third threshold;

[0189] Problem-solving tasks are often accompanied by frequent eye-jumping, such as switching between the problem and tools such as paper, pen, and calculator. By presetting the second angle threshold and the third angle threshold, this behavioral feature of the problem-solving task can be effectively identified to avoid misjudgment as inattention. The system uses a wide range of angle thresholds to reduce invalid reminders to users during the problem-solving process, avoid frequent interruptions when users switch their gaze, and improve the concentration and fluency of problem-solving tasks.

[0190] Specifically, if the current learning task is a "problem-solving task", the third threshold of 20° is preset. The system detects whether the target user's gaze focus continues to remain within a range greater than the second threshold of 15° and less than the third threshold of 20° through continuous monitoring within 30 seconds. When the target user's gaze angle is greater than the second threshold of 15° and less than the third threshold of 20°, it indicates that the target user's gaze frequently switches between learning materials and other tools, and a gaze angle relationship for the problem-solving task is established.

[0191] S2-25, if the learning task is a video watching task, establishing a gaze angle relationship for the video watching task according to the angle and a preset third threshold;

[0192] In the task of watching a video, the user's eyes are usually focused on the center of the screen, and the head and eyes move less. By setting a larger angle threshold, the system can accurately identify the task status of the user watching a video. This method effectively captures this feature, ensures the accuracy of task identification, and helps the system dynamically adjust the monitoring standard to avoid frequent system reminders due to short-term deviation of the user's eyes, improves the user experience, and also reduces misjudgment.

[0193] Specifically, if the current learning task is a "video watching task", the system detects that the target user's gaze angle must be continuously greater than the third threshold of 20° within 30 seconds, indicating that the user's gaze is basically focused on watching the screen content and the gaze does not frequently jump to other areas, thereby establishing a gaze angle relationship for the video watching task.

[0194] S2-3, obtaining a dynamic concentration monitoring strategy according to the learning task type;

[0195] By dynamically adjusting the monitoring strategy according to the task type, the system can more accurately adapt to the concentration characteristics of each task, reduce misjudgments, and improve the accuracy of monitoring. If a unified standard is used to monitor different types of tasks, frequent false reminders may occur, interfering with the user's learning experience. Dynamic adjustment of the monitoring strategy can avoid frequent interruptions to users when the eyes naturally jump or temporarily deviate, thereby reducing unnecessary reminders. The system can provide personalized concentration feedback and learning suggestions based on different task types. Through monitoring strategies tailored for each learning task, users can focus more on the current task. The system can provide timely reminders when concentration decreases, which helps users stay focused, avoid distractions, and improve overall learning efficiency.

[0196] Preferably, according to the learning task type, obtaining a dynamic concentration monitoring strategy includes:

[0197] S2-31, if the learning task is a reading task, obtaining a dynamic reading concentration strategy;

[0198] The system accurately captures the user's concentration state in the reading task based on the user's gaze focus and head posture. Through dynamic adjustment of the angle, the system can effectively determine whether the user is focusing on the reading material to avoid misjudgment due to a brief absence of sight. The system dynamically adjusts the monitoring standard based on the user's gaze focus and duration to reduce unnecessary reminders during the reading process, which can help users not be disturbed while reading and avoid frequent reminders. The dynamic monitoring strategy moderately adjusts the monitoring intensity based on the user's reading behavior. In long-term reading tasks, the system will not frequently issue reminders due to the user's brief absence of sight, which can help users maintain their attention and avoid excessive fatigue.

[0199] Specifically, the system dynamically monitors the user's gaze focus to evaluate their concentration performance. If the user's gaze remains in the center of the material for more than 15 minutes, the system will give positive feedback and award points S. bonus For every five minutes exceeding the reading time, 0.5 points of additional bonus points will be awarded. If the reading time is less than five minutes, it will be counted as five minutes. If the user's eyes are away from the material for more than 2 minutes, the system will issue a visual or sound reminder to prompt the user to return to the reading task.

[0200] S2-32, if the learning task is a writing task, obtain a dynamic writing concentration strategy;

[0201] The system generates personalized focus feedback based on the user's writing task performance, and provides bonus points when the user maintains focus for a long time, encouraging users to develop good writing habits. In long-term writing tasks, the system's dynamic monitoring strategy can adapt to the user's writing rhythm and will not frequently disturb the user due to short-term gaze departure, ensuring the flexibility and adaptability of monitoring. The system automatically adjusts the monitoring parameters dynamically according to the writing task, and the user does not need to manually set or frequently adjust them, which improves the user's experience of focusing on writing tasks and makes the learning process smoother. Through the system's real-time reminders and dynamic adjustments, users can maintain a higher level of focus in writing tasks and reduce the chance of distraction, thereby improving the overall efficiency of learning and work.

[0202] Specifically, the system allows users to think for a short time without focusing on the writing area, but if they do not return to the material for more than 2 minutes, a focus reminder will be issued. During the writing process, the system monitors the user's gaze state and angle changes in real time. If the user's gaze stays on the material for more than 15 minutes, the system will give a bonus score S bonus For every minute exceeding five minutes, 0.5 extra bonus points will be given. If it is less than five minutes, it will be counted as five minutes.

[0203] S2-33, if the learning task is a problem-solving task, obtaining a dynamic problem-solving concentration strategy;

[0204] Through concentration monitoring and feedback, the system can help the target user maintain a high level of concentration in the problem-solving task. When the target user is distracted or the concentration decreases, the system will issue a prompt to prompt the target user to return to the task and improve the efficiency of problem-solving. Problem-solving tasks may involve long-term reasoning and calculations. The system's dynamic monitoring strategy can adapt to the user's changes in vision during different problem-solving steps, and will not interfere with the user's problem-solving process due to short-term thinking or eye jumps, ensuring the flexibility of concentration monitoring.

[0205] Specifically, it is the same as S2-32 and will not be repeated here.

[0206] S2-34, if the learning task is a video watching task, obtaining a dynamic video watching concentration strategy;

[0207] In the video watching task, the system can dynamically adjust the monitoring parameters to avoid frequent reminders due to the target user's brief absence from the screen or slight head movement, reduce unnecessary interference, and optimize the user experience. The system helps the target user maintain consistent attention while watching the video through real-time feedback and reminders. When the target user's eyes are away from the screen for a long time, the system will issue a reminder in time to prompt the user to return to the video content and improve learning efficiency.

[0208] Specifically, it is the same as S2-32 and will not be repeated here.

[0209] S2-4, obtaining a concentration score of the target user according to the dynamic concentration monitoring strategy and the gaze angle relationship;

[0210] Based on the type of learning task, the system sets the scoring criteria according to the characteristics of the current task type. The scoring mechanism includes factors such as gaze angle relationship and gaze stability time.

[0211] Preferably, according to the dynamic concentration monitoring strategy and the gaze angle relationship, obtaining the concentration score of the target user includes:

[0212] S2-41, obtaining a basic concentration score according to the dynamic concentration monitoring strategy and the gaze angle relationship;

[0213] The system evaluates the target user's concentration in each preset scoring cycle. When the target user is in a specific learning task type, the system selects the corresponding dynamic concentration monitoring strategy for the target user, counts the concentration time, and increases the basic score to obtain a basic concentration score. The reason for setting a gradient score is that there are significant differences in the concentration requirements and eye focus behavior patterns of different learning tasks. By setting a concentration score gradient according to different task types, the system can more accurately and reasonably evaluate the user's concentration performance. The following are the specific reasons for setting a gradient score:

[0214] Reading tasks usually require users to maintain a high degree of gaze concentration, with less eye jumps, and to focus on a fixed area of ​​the reading material. Therefore, a lower gaze angle threshold is set to establish a gaze angle relationship and a lower score is set to reflect its stability requirement; problem-solving tasks require frequent jumps between different areas such as materials and tools. Therefore, users who moderately relax the gaze angle to establish a gaze angle relationship and focus on a variety of materials should receive a higher score to reflect the complexity and dynamism of the problem-solving task; in writing tasks, users may think from time to time and their eyes may leave the writing materials. Therefore, a larger gaze angle is allowed to establish a gaze angle relationship, and since writing tasks require higher creativity, a higher score is set to reflect their concentration; in video watching tasks, users' eyes are mainly focused on the video content, and the task itself is relatively simple. Even if the user's gaze angle is larger, they will continue to focus on the screen content. A higher score is set to reflect the relative low difficulty of maintaining concentration in video tasks.

[0215] Specifically, in each preset scoring period T = 30 minutes, if the learning task is a reading task and the gaze angle is less than 10°, the basic concentration score is increased by 0.5 points / minute; if the learning task is a problem-solving task and the gaze angle is greater than 10° and less than 15°, the basic concentration score is increased by 1 point / minute; if the learning task is a writing task and the gaze angle is greater than 15° and less than 20°, the basic concentration score is increased by 1.5 points / minute; if the learning task is a video watching task and the gaze angle is greater than 20°, the basic concentration score is increased by 2 points / minute.

[0216] S2-42. Obtain reward scores according to the preset concentration scoring rules;

[0217] Same as S44, no further details will be given.

[0218] S2-43, obtaining the concentration score of the target user according to the basic concentration score and the reward score, and issuing a reminder if the concentration score is less than or equal to a preset total score fixed value;

[0219] Same as S45, no further details will be given.

[0220] Example 3

[0221] Based on Example 1, this embodiment makes further improvements in step S3. On the basis of gaze focus detection, the entire reading material and the target user's sight scene are intelligently divided into multiple key areas for the reading material, and the target user's gaze dwelling time in different areas is analyzed. A concentration monitoring method based on visual tracking is provided, and the method includes:

[0222] S3-1, acquiring the gaze focus identification information of the target user according to the target information, wherein the target information includes the head posture information, pupil position information and reading material position information of the target user;

[0223] Same as S3, no further details.

[0224] S3-2, obtaining a material classification area according to the reading material location information;

[0225] The regional division of materials can help the system more accurately analyze the user's attention to different content during the learning process. For example, by analyzing the dwell time in the title, text, and exercise areas, it can be understood whether the user is focusing on important content or staying in irrelevant areas. The system can dynamically adjust the learning strategy based on the user's dwell time in different areas. By segmenting the material areas, the system can better guide users to focus on important learning areas. The weight allocation of different areas helps the system to fine-tune the assessment of concentration and help users improve their concentration performance in learning tasks. If the target user spends too much time on certain specific areas of the material, the system can identify potential learning obstacles or difficulties.

[0226] The system divides the reading materials into several key areas intelligently, and divides them into corresponding areas according to the content of the materials. Each area has clear boundaries to facilitate the subsequent accurate tracking of the user's line of sight.

[0227] Preferably, according to the reading material location information, obtaining the material classification area includes:

[0228] S3-21, obtaining the type of reading material according to the reading material location information;

[0229] Material type classification can help the system dynamically adjust monitoring parameters according to the needs of different learning tasks. By identifying the material type, the system can provide users with personalized learning feedback. For example, if the system recognizes that the target user is reading a chart-type material, it recommends that the target user focus on analyzing the chart area and provides corresponding learning support. Material type classification can help users enter the learning state faster. If the target user stays in the title area for a long time or does not enter the text area, the system can remind the target user to start the learning task, thereby improving learning efficiency.

[0230] Specifically, the system uses the YOLOV8s model to extract the content of the reading material page using the reading material location information, conducts a preliminary analysis based on the format and layout characteristics of the material, and identifies the arrangement of the text, pictures, titles and other elements of the material. The system preliminarily classifies the materials according to the structural characteristics of the page, and the types include articles, exercises and other non-learning areas. The following are the three types of classification basis: article-type materials are mainly composed of continuous text paragraphs, usually including titles, text and paragraph marks; exercise-type materials are mainly questions, options and analysis, with multiple segments and numbers, such as questions in exercise books; non-learning area materials include margin notes and illustration non-learning areas.

[0231] S3-22, obtaining a material classification area according to the type of the reading material;

[0232] The system can adaptively adjust the area division rules according to the material type. Through area division and concentration assessment, it can guide users to focus on important learning content. For example, if a user stays in the title area for a long time, the system can remind them to enter the text area, thereby helping users quickly enter the learning state and improve learning efficiency.

[0233] Specifically, the system accurately defines the boundaries of each area according to the different material contents, including the coordinate information of rectangles or polygons to identify different parts of the material. The boundaries of each area will be stored for subsequent eye tracking and analysis. The following are the rules for dividing the three types of areas: Article-type materials mark the boundaries of the title area, text area, and annotation area respectively; Exercise-type materials include three major areas: marked questions, options, and analysis, ensuring that the questions and answers are independent of each other; Non-learning area materials include margin notes and illustration non-learning areas.

[0234] S3-3, obtaining the gaze dwell time of each area according to the material classification area and the gaze focus identification information of the target user;

[0235] By recording how long the user's gaze stays in different areas, the system can accurately evaluate the user's concentration performance. For example, the time a user stays in the main text area of ​​an article-type material can directly reflect his or her concentration when learning the main content, while staying too long in a non-learning area may indicate distraction. Based on the time spent in each area, the system can analyze the user's learning status in real time and adjust strategies. If the user stays too long in certain areas, such as the exercise area, the system can identify that the user may have difficulties in that part, and provide help or suggestions based on this analysis to help the user overcome learning bottlenecks.

[0236] Preferably, according to the material classification area and the target user's gaze focus identification information, obtaining the gaze dwell time of each area includes:

[0237] S3-31, obtaining a gaze focus area according to the material classification area and the gaze focus identification information of the target user;

[0238] By obtaining the specific area where the user's gaze falls, the system can accurately evaluate the user's attention distribution in different material areas. This helps to understand whether the user is focused on important learning areas or frequently distracted by secondary content, and can more accurately analyze the user's concentration behavior.

[0239] Specifically, the system matches the user's gaze focus with the coordinates of the boundary of the material classification area. If the sight direction vector intersects with the boundary of the area coordinates, the gaze falls on this area. By detecting whether the user's gaze focus is within the classification area, the specific area where the user's current gaze focus is located can be determined, and the gaze landing area can be obtained.

[0240] S3-32, according to the gaze falling area, continuously measure the time for obtaining the gaze dwelling time in the area;

[0241] When the user's eyes jump frequently or stay in non-critical areas for a long time, the system can monitor and remind the user in real time. This can help the user reduce distracting behaviors, maintain learning continuity, and improve learning efficiency. Based on the user's stay time in different areas, the system can provide personalized learning feedback.

[0242] Specifically, once the system confirms that the user's gaze falls on a certain area, the timer for that area is immediately started. The system records the start time of the user's stay in that area and continuously monitors the user's stay status in that area. Each area has an independent timer, which automatically records the user's stay time when it is started. If the user's gaze switches from one area to another, the system stops the timer for the current area and records the user's total stay time in that area. At the same time, the system starts a new timer for the newly entered area and continues to record the user's stay time. When the system detects that the user's sight focus switches from one area to another, it stops the timer for the previous area and records the stay time.

[0243] S3-4, obtaining a reading material area concentration score according to each preset classification area weight and the gaze dwelling time of each area;

[0244] By combining area weights and gaze dwell time, the system can more accurately evaluate the user's concentration performance in different areas. In particular, for important content such as the main text area or exercise area, weighted scores can better reflect the user's concentration level, while long stays in secondary areas or non-learning areas will reduce the score. The system can identify the user's learning bottlenecks and provide feedback.

[0245] Preferably, obtaining the concentration score of the reading material area according to the weight of each preset classification area and the gaze dwelling time of each area includes:

[0246] S3-41, obtaining a concentration score for each area according to the weight of each preset classification area and the gaze dwelling time of each area;

[0247] Based on the time and weight of the user's stay in each area, the system can accurately evaluate the user's concentration in different areas, especially for key learning areas (such as the text or exercise area), the system can accurately reflect whether the user is concentrating. Based on the concentration score of each area, the system can provide personalized feedback to users, monitor the user's gaze in non-learning areas in real time, and deduct points through the design of negative weights to help users reduce distraction and remind them to return to the core learning content to improve learning efficiency.

[0248] Specifically, the system divides the materials into multiple key areas according to different types of learning materials, and assigns a preset weight to each area, which is set according to the importance of the area. The weights of each area are set as follows:

[0249] Text area: 0.5

[0250] Title area: 0.1

[0251] Annotation area: 0.2

[0252] Exercise area: 0.3

[0253] Non-learning area: -0.2

[0254] According to different types of learning materials, the system divides the materials into multiple key areas based on the material type, and assigns a preset weight to each area. The weight value is set according to the importance of the area. The main text area is the core part of the article material, usually containing the most important learning content and knowledge points. The user's concentration in the main text area directly determines their understanding and absorption of the content. Therefore, the main text area has the greatest impact on learning outcomes. In order to emphasize the importance of learning in this area, a weight of 0.5 is given; the title area usually provides an introduction to chapters and paragraphs or a summary of the content. Although the title helps users understand the structure of the material, it does not directly affect the user's understanding of the specific learning content. Therefore, the title area is given a weight of 0.1; the importance of the annotation area is lower than the main text, but higher than the title area, because it provides additional reference information, but should not occupy most of the user's attention, so the annotation area is given a weight of 0.2; the exercise area is the part where users practice and review the learning content. The core purpose of the three major exercise areas is to consolidate the learning content, so no further weight division is performed on them, but weights are given to the whole. Although exercises are important, their core value still depends on the main text. Therefore, the weight of the exercise area should be lower than that of the main text area, and a weight of 0.3 is given. The non-learning area includes the margin note area and the illustration area. If the user's eyes stay in these areas, it means that they are distracted. Ensure that the system recognizes and deducts the user's concentration score, prompting the user to spend less time in these areas. Therefore, a strong negative weight of -0.2 is given.

[0255] S3-42, according to the concentration scores of each area, accumulating and obtaining the concentration scores of the reading material area;

[0256] By accumulating the concentration scores of each area, the system can provide users with the concentration performance throughout the reading material, helping users understand their concentration in each learning link. The total score can help users identify in which areas their concentration is insufficient. The cumulative scoring mechanism can flexibly adapt to different learning tasks. The system provides personalized learning feedback and suggestions based on the user's concentration performance in different areas, helping users adjust their learning strategies and improve learning results. In addition, the system detects whether users spend too much time in non-learning areas or secondary areas by accumulating the concentration scores of each area, and then issues reminders to guide users to focus on important content and reduce distracting behaviors.

[0257] Specifically, the system calculates the concentration score for all classification areas separately. Each area corresponds to a separate concentration score, which represents the user's concentration performance in each area. Then the concentration scores of each area are accumulated to obtain the comprehensive concentration score of the entire reading material. The accumulation formula is as follows:

[0258] Reading material area concentration score = ∑ (each area concentration score)

[0259] Here is an example calculation:

[0260] Main text area: 20 minutes x 0.5 = 10 minutes

[0261] Title area: 5 minutes x 0.1 = 0.5 minutes

[0262] Exercise area: 10 minutes x 0.3 = 3 minutes

[0263] Annotation area: 2 minutes x 0.2 = 0.4 minutes

[0264] Non-learning area: 3 minutes x (-0.2) = -0.6 minutes

[0265] The final reading material concentration score is: 10+0.5+3+0.4-0.6=13.3 points.

[0266] Example 4

[0267] See also Figure 5 , an embodiment of the present invention provides a concentration monitoring device based on visual tracking, the device comprising:

[0268] Data acquisition module, acquiring target user video data;

[0269] A data training module, which acquires target information according to the video data and a preset neural network model, wherein the target information includes head posture information, pupil position information, and reading material position information of the target user;

[0270] A gaze focus recognition module, which obtains gaze focus recognition information of the target user according to the target information;

[0271] The concentration scoring reminder module obtains the concentration score of the target user according to the gaze focus recognition information and preset concentration scoring rules.

[0272] It should be noted that the modules and units in the concentration monitoring device based on visual tracking in this embodiment correspond one-to-one to the steps in the concentration monitoring method based on visual tracking in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned concentration monitoring method based on visual tracking, which will not be repeated here.

[0273] Example 5

[0274] In addition, combined Figure 5 The concentration monitoring method based on visual tracking described in the embodiment of the present invention can be implemented by an electronic device, characterized in that it is used to implement the method as described in any one of claims 1-7. Figure 6A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention is shown.

[0275] An electronic device may include a processor and a memory storing computer program instructions.

[0276] Specifically, the processor may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

[0277] The memory may include a large capacity memory for data or instructions. By way of example and not limitation, the memory may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory may include a removable or non-removable (or fixed) medium. Where appropriate, the memory may be inside or outside a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.

[0278] The processor implements any one of the methods for extracting facial features based on thermal infrared images in the above embodiments by reading and executing computer program instructions stored in the memory.

[0279] In one example, the electronic device may further include a communication interface and a bus. Figure 6 As shown, the processor 401, the memory 402, and the communication interface 403 are connected via a bus 410 and communicate with each other.

[0280] The communication interface is mainly used to implement communication between the modules, devices, units and / or equipment in the embodiments of the present invention.

[0281] Bus includes hardware, software or both, and the parts of electronic equipment are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In suitable cases, bus may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0282] Example 6

[0283] In addition, in combination with a method for monitoring concentration based on visual tracking in the above embodiment, an embodiment of the present invention may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the methods for monitoring concentration based on visual tracking in the above embodiment is implemented.

[0284] In summary, the embodiments of the present invention provide a method, device, equipment and storage medium for monitoring concentration based on visual tracking.

[0285] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0286] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0287] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0288] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A concentration monitoring method based on visual tracking, characterized in that: The method comprises: Acquire video data including the target user; Acquire target information according to the video data and a preset neural network model, wherein the target information includes head posture information, pupil position information, and reading material position information of the target user; According to the target information, obtaining the gaze focus identification information of the target user; The concentration score of the target user is obtained according to the gaze focus recognition information and preset concentration scoring rules.

2. The method for monitoring concentration based on visual tracking according to claim 1, characterized in that: The acquiring target information according to the video data and the preset neural network model includes: Acquire an image sequence according to the video data, wherein the image sequence includes a plurality of image frames; Inputting the image sequence into a pre-built neural network model to obtain a target detection image, wherein the target detection image includes an eye region image, a head region image, and a reading material region image; The target information is obtained according to the target detection image.

3. The method for monitoring concentration based on visual tracking according to claim 2, characterized in that: The obtaining the target information according to the target detection image comprises: Acquiring the pupil position information according to the eye region image; Calculating a head posture rotation matrix of the target user according to the head region image to obtain the head posture information; The reading material position information is acquired according to the reading material area image, wherein the pupil position information, the head posture information and the reading material position information all include a direction vector and a center coordinate.

4. The method for monitoring concentration based on visual tracking according to claim 3, characterized in that: The acquiring the pupil position information according to the eye region image comprises: According to a preset Gaussian processing rule, Gaussian processing is performed on the eye region image to obtain an optimized image; Performing grayscale transformation on the optimized image to obtain a grayscale image; Performing binarization processing on the grayscale image to obtain a binarized image; Acquiring edge positioning information according to the binary image; Acquiring edge contour information according to the edge positioning information; According to the edge contour information, obtain the circularity of several contours; According to the circularity of each contour, obtaining a first contour with the largest circularity; Pupil position information is acquired according to the first contour.

5. The method for monitoring concentration based on visual tracking according to claim 4, characterized in that: The performing Gaussian processing on the eye region image to obtain an optimized image according to a preset Gaussian processing rule comprises: Performing Gaussian blur processing on the eye region image according to a preset Gaussian kernel standard deviation to obtain a first image; According to preset histogram equalization parameters, performing histogram equalization processing on the first image to obtain a second image; Perform Gaussian blur processing on the second image to obtain an optimized image after Gaussian blur processing.

6. A method for monitoring concentration based on visual tracking according to any one of claims 1 to 3, characterized in that: acquiring the gaze focus identification information of the target user according to the target information; Acquire the sight direction according to the pupil position information and the head posture information; According to the sight line direction, obtaining a sight line direction vector; According to the reading material position information, obtaining a reading material direction vector; According to the sight line direction vector and the reading material direction vector, the two are calculated according to a preset calculation rule to obtain the gaze focus identification information of the target user, and the gaze focus identification information includes the angle between the sight line direction and the reading material.

7. The method for monitoring concentration based on visual tracking according to claim 6, characterized in that: The obtaining the concentration score of the target user according to the gaze focus identification information and the preset concentration score rule comprises: In each preset scoring cycle, when the angle between the sight direction and the reading material is less than or equal to a preset angle threshold, the concentration time of the target user is timed to obtain an initial concentration time; When the initial concentration time is greater than or equal to a preset time threshold, the initial concentration time is used as the target concentration time; Get a basic score based on the target focus time; Obtain reward scores based on preset concentration scoring rules; The concentration score of the target user is obtained according to the basic score and the reward score, and a reminder is issued if the concentration score is less than or equal to a preset total score fixed value.

8. A concentration monitoring device based on visual tracking, characterized in that: The device comprises: A data acquisition module, which acquires video data including the target user; A data training module, which acquires target information according to the video data and a preset neural network model, wherein the target information includes head posture information, pupil position information, and reading material position information of the target user; A gaze focus recognition module, which obtains gaze focus recognition information of the target user according to the target information; The concentration scoring reminder module obtains the concentration score of the target user according to the gaze focus recognition information and preset concentration scoring rules.

9. An electronic device, characterized in that: include: At least one processor, at least one memory and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, implement the method according to any one of claims 1 to 7.

10. A storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images