Teacher visual data analysis system and method based on eye movement technology

Abstract

The invention discloses a teacher visual data analysis system based on an eye movement technology and a method thereof, and aims to provide a key cognitive mechanism for teacher visual data report interpretation, which can be widely applied to schools, teaching and training industries and the like to train demonstration examples and strategies of common teachers, help the common teachers to master data interpretation ability, and improve the teaching efficiency. And teaching is optimized by better utilizing data. The implementation method comprises the following steps: acquiring information such as a fixation point and a sight track of a teacher in a process of interpreting a visual data report through eye movement tracking equipment so as to obtain a visual attention mode of the teacher on the data; the thinking activity of the teacher in the interpretation process is recorded through audio thinking, so that the thinking strategy and the cognitive process of the teacher are deeply excavated; through conjoint analysis of eye movement data and voiced thinking, key cognitive modes of experts and teachers in data report interpretation are extracted, and theoretical support is provided for designing an intervention strategy of teacher data literacy cultivation; and finally, common teachers are helped to master effective visual report interpretation skills.

Description

Technical Field

[0001] The present invention relates to the field of educational information technology, and in particular to a teacher visual data analysis system and method based on eye movement technology. Background Art

[0002] Eye tracking technology has been widely used in the fields of cognitive and behavioral analysis. Due to the concern about teachers' teaching behaviors, some educational researches have applied eye tracking technology to classroom interaction research to analyze teachers' visual behavior patterns. However, currently these systems mainly focus on visual behaviors such as teachers' gaze following, and the application scenarios are mostly limited to remote teaching or the interaction optimization of general classrooms, lacking support for teachers' cognitive processes in the interpretation of complex data reports.

[0003] Due to the development of educational informatization and intelligence, some schools have gradually introduced data-driven teaching support systems, providing teachers with visual data reports on students' learning situations. Although these reports play an important role in teaching, ordinary teachers often face difficulties in interpreting complex data and are hard to quickly grasp the key information. In contrast, expert teachers have unique cognitive patterns and can quickly identify the core information in the data and formulate teaching plans accordingly. However, this cognitive pattern of expert teachers is difficult for ordinary teachers to directly copy and master. Summary of the Invention

[0004] Based on the technical problems existing in the background art, the present invention proposes a teacher visual data analysis system and method based on eye movement technology.

[0005] On the one hand, a teacher visual data analysis system based on eye movement technology proposed by the present invention includes a data collection and observation experiment module, a design and verification module for demonstration examples and technical specifications, an intelligent recognition module, and an interactive feedback module. Among them,

[0006] The data collection and observation experiment module is used to analyze and refine the data thinking mode of expert teachers through experimental observation, and verify the feasibility of applying eye movement thinking demonstration examples to the cultivation of teachers' report interpretation ability from a theoretical perspective;

[0007] The design and verification module for demonstration examples and technical specifications is used to verify the effectiveness of example learning in the cultivation of teachers' visual data report ability through quasi-experimental research;

[0008] The intelligent recognition module and the interactive feedback module are used to convert the obtained fixation sequences of expert teachers on different types of visual data reports and the verbal thinking contents at each fixation point into question scaffolds in the form of questions, etc., and store them in a relational database according to the report type, and develop human-computer interaction eye movement intelligent recognition to realize eye tracking when ordinary teachers watch screen reports.

[0009] Preferably, the specific process of the experimental observation and analysis is as follows: Select a subject and recruit two groups of teachers, namely expert teachers and ordinary teachers. Observe the specific eye movements and verbal thinking of these two types of teachers during the process of interpreting data reports, that is, fixation points, scanning paths, thinking processes and strategies, to understand the cognitive mechanism of teachers during this process. In a specific experiment, the system provides class reports, individual reports, and reports of different performance levels to achieve the purpose of experimental observation, analyze the problems existing in the interpretation of data reports by ordinary teachers, and explore the data thinking patterns of expert teachers.

[0010] Preferably, the specific process of the verification from the theoretical perspective is as follows: Analyze the cognitive mechanism of expert teachers' interpretation of visual reports, and analyze whether there are differences in the ability of different expert teachers and ordinary teachers to interpret visual reports that can be captured by eye movement and verbal thinking methods, and analyze whether the problems existing in the ability of ordinary teachers to interpret visual reports can be demonstrated by eye movement and verbal thinking.

[0011] Preferably, when the intelligent recognition module and the interactive feedback module are started, the built-in camera of the computer is called to record the line-of-sight position of ordinary teachers. After obtaining the fixation points of ordinary teachers, according to the current report type, select the content in the corresponding expert knowledge base for comparison. If it is within the error range of the expert fixation point, give the thinking scaffold when the expert pays attention to this place, and then give the position of the next fixation point of the expert; if it is different from the expert fixation point position, prompt the position that ordinary teachers should pay attention to at this time.

[0012] On the other hand, the present invention also proposes a method for visual data analysis of teachers based on eye movement technology, including the following steps:

[0013] S1: Perform face key point annotation on the publicly available face dataset;

[0014] S2: Preprocess the dataset. Based on the cleaned and standardized annotation data generated in S1, further perform preprocessing operations such as face alignment, normalization, noise removal, and data augmentation on it to ensure the spatial consistency and diversity of the data;

[0015] S3: Input the annotated dataset into the PFLD neural network model for training;

[0016] S4: Use the existing face detection dataset for the training of the YOLOv8 neural network;

[0017] S5: Output the face detection result;

[0018] S6: Backpropagation is used to optimize the network weights of the YOLOv8 neural network;

[0019] S7: Apply the trained neural network for face detection to the actual video of the teacher facing the computer screen;

[0020] S8: Read and decode the real-time video of the ordinary teacher facing the computer screen to obtain video frames, and use the YOLOv8 neural network to perform face detection on the extracted video frames;

[0021] S9: Crop and obtain the face image according to the face detection box;

[0022] S10: After preprocessing the cropped face image, input it into the trained face key point detection network model PFLD;

[0023] S11: The PFLD neural network model outputs the recognition result to obtain the key point coordinates of the human eyes;

[0024] S12: After obtaining the human eye coordinates, use the detected human eye key points to establish an eye movement model, estimate the focus of the human eye's line of sight, infer the orientation of the eyeball, and finally obtain the focus coordinates of the real-time window;

[0025] S13: Call the "imshow" function in the OpenCV library to generate a real-time window on the computer side. This window is scaled proportionally to the computer screen, and a coordinate mapping relationship is established between the two. After obtaining the coordinate position of the human eye on the real-time window, through the mapping function, the coordinate position of the human eye on the computer screen can be obtained;

[0026] S14: Based on the coordinate position of the human eye on the computer screen obtained in S13, create a list, add the new focus coordinates of the line of sight to the list, and write and save them in the log. Then traverse the coordinate list and use the "cv2.circle" function to draw the eye movement trajectory, which is superimposed on the original face image. As the program runs, this list will be continuously updated to contain more coordinate points. Before drawing each frame, the program will traverse this list and use "cv2.circle" to draw small dots at all these coordinates to form a trajectory. At the same time, if the new focus coordinates of the line of sight are within a certain range of the previous focus coordinates of the line of sight, that is, there is no significant movement, then calculate the difference between the current timestamp and the previous recorded timestamp. If it exceeds 100 ms, it will be judged as a fixation point.

[0027] Preferably, in S1, the face key point annotation is initially automatically annotated through the HOG feature of Dlib and the set regression tree model. The specific steps include:

[0028] S101: Calculate the distribution histogram of the local gradient direction, and generate a stable face feature representation after normalization. The formal definition is as follows:

[0029]

[0030] Among them, Grad i represents the gradient direction distribution of the i-th pixel in the image, and w i is the weight of a specific window, and Z is the normalization factor;

[0031] S102. Based on this feature, the Dlib classifier can quickly locate the rectangular frame area of the face. In the face key point detection stage, combined with the method based on the Ensemble of Regression Trees (ERT), by performing multiple regression optimizations on the initial position of each key point to gradually approach the true position. For a given key point prediction error, the model corrects the key point position ΔP in each iteration:

[0032] ΔP = R(X) + ∈

[0033] where R(X) is the offset predicted by the regression tree model, and ∈ is the residual. The initial 68-point model obtains a more accurate result through multiple rounds of iteration;

[0034] S103. To expand to 98 key points, it is necessary to process and supplement the annotation of the publicly available face dataset. Select publicly available datasets such as 300-W and WFLW. These datasets cover a variety of complex scenarios to ensure the diversity and generalization ability of the annotated data. Based on the initial 68 points generated by Dlib, the initial positions of the newly added 30 points are deduced by the method of geometric interpolation. The calculation formula is:

[0035] P new = αP i + (1 - α)P j , α ∈ [0, 1]

[0036] where P i and P j are the positions of adjacent points respectively, and α is the proportionality factor;

[0037] S104. For the newly generated points after geometric interpolation, use the manual correction tool to adjust their positions one by one to make them more conform to the true distribution of facial features. After manual adjustment, through the method of cross-validation, different annotators review the same batch of data to eliminate potential annotation biases. The annotation consistency is evaluated by calculating the mean square error (MSE):

[0038]

[0039] The finally output annotated data is cleaned and standardized, and the unified format is JSON or CSV.

[0040] Preferably, in the step S2, the specific steps for preprocessing the dataset include:

[0041] S201. Align the human faces in the data to eliminate geometric differences caused by head postures. Through the affine transformation method, calculate the affine transformation matrix M according to the key point positions of the two eyes and the tip of the nose in the labeled data to achieve the standardized alignment of the facial region. The formula for affine transformation is:

[0042] P′ = M·P

[0043] where P′ and P are the coordinates of the aligned and original images respectively, M is the affine transformation matrix, and the transformation matrix M is calculated by the least squares fitting method by setting the corresponding relationship between the reference points and the target points;

[0044] S202. Uniformly adjust the size of the aligned images to 112×112, and normalize the pixel values to reduce the size and brightness distribution differences of the input images, ensuring that the model can process inputs with a unified scale and statistical distribution. Normalize the original image pixel values to a standard normal distribution with a mean of 0 and a variance of 1. The formula is:

[0045]

[0046] I(x,y) is the pixel value of the original image, μ and σ are the mean and standard deviation of the pixel values respectively, W and H are the width and height of the original image, and W′, H′ are the normalized sizes. The normalization process effectively reduces the dynamic range differences of the image data and improves the stability and convergence speed of the training process;

[0047] S203. Use a Gaussian filter to smooth the noise regions to further improve the data quality. The filter kernel function is defined as:

[0048]

[0049] where σ controls the width of the Gaussian distribution and determines the smoothness of the filter. The result of the convolution of the Gaussian filter and the image makes the local pixel values more uniform, thus effectively suppressing the interference of noise;

[0050] S204. In the data augmentation stage, to expand the diversity of samples and improve the generalization ability of the model, randomly apply cropping, rotation, brightness adjustment, and flipping transformations to each image to expand the diversity of training samples and enhance the generalization ability of the model. The brightness adjustment is achieved by applying a linear transformation to the pixel values. The formula is:

[0051] I′(x,t) = α·I(x,y) + β

[0052] where α is the brightness scaling factor and β is the brightness offset value, and training samples with different brightness characteristics are generated by randomly sampling different values.

[0053] Preferably, in the step S3, after the key point annotation and data preprocessing are completed, the generated data is used to train the PFLD neural network model to achieve high-precision positioning of facial key points. The specific steps of the training include:

[0054] S301. Input the preprocessed image into the global feature extraction module of PFLD, and extract multi-scale global features through a lightweight convolutional network. The calculation formula for the convolution operation is:

[0055] F (l+1) =σ(W (l) *F (l) +b (l) )

[0056] where F (l) is the feature map of the l-th layer, W (l) and b (l) are the convolution kernel weights and bias terms respectively, * represents the convolution operation, and σ is the non-linear activation function. The global features extracted through this process contain the overall semantic information of the face and can describe the geometric features of the face;

[0057] S302. The global features are input into the local refinement regression module to gradually optimize the predicted position of the key points. Assuming that the initial predicted key point is P init , the local refinement module will learn the predicted offset ΔP and gradually approximate the position of the true key point through the following formula:

[0058] P final =P init +ΔP

[0059] S303. In order to measure the error between the model prediction value and the true value, PFLD uses the L2 loss function as the optimization objective, which is defined as:

[0060]

[0061] where N is the number of key points, and represent the predicted value and the true value of the i-th key point respectively. Selecting the L2 loss can impose a higher penalty on larger prediction deviations, thereby promoting faster convergence of the model. During the training process, the data set is divided into a training set, a validation set, and a test set, with proportions of 70%, 20%, and 10% respectively;

[0062] S304. The training set is used to optimize the model parameters, the validation set is used to evaluate the performance of the model on unseen data, and the test set is used for the final generalization performance evaluation. The Adam optimizer is used during the optimization process, which combines the advantages of momentum and adaptive learning rate. The parameter update formula is:

[0063]

[0064] Among them, and are the unbiased estimators of the first moment and the second moment of the gradient respectively, η is the learning rate, and ∈ is a stability factor to avoid a zero denominator;

[0065] S305. To ensure that the model can converge faster and not overfit, a cosine annealing strategy is adopted to dynamically adjust the learning rate, which is defined as:

[0066]

[0067] Among them, η t is the learning rate of the t-th iteration, η min and η max are the minimum and maximum values of the learning rate respectively, and T is the total number of training epochs;

[0068] S306. After the training is completed, Normalized Mean Error (NME) is used to evaluate the model performance, which is defined as:

[0069]

[0070] Among them, d is the standard scale for normalization, and the distance between the two eyes is selected. NME can intuitively reflect the localization accuracy of the model on the validation set. During the test process, the prediction results of the model are compared with the ground truth in a visual way, and the adaptability of the model in various scenarios is further manually checked to train a reliable key point prediction model.

[0071] Preferably, in the step S4, the specific steps of training include:

[0072] S401. Before the model training, the preprocessed image data in S2 is further converted into the YOLO format. The annotation information of each image is represented by a five-tuple (c, x, y, w, h), where c is the class label, (x, y) is the center coordinate of the bounding box, and (w, h) are the width and height of the bounding box. All values are normalized to the interval [0, 1]. The normalization formula is:

[0073]

[0074] where W and H are the width and height of the image respectively; after the annotation is completed, the dataset is divided into a training set, a validation set, and a test set for model training, performance verification, and final evaluation;

[0075] S402. YOLOv8 extracts multi-scale features through the backbone network and fuses feature maps of different resolutions through the Feature Pyramid Network (FPN). Finally, the detection head regresses the bounding box coordinates and predicts the confidence. The prediction formula for the bounding box coordinates is:

[0076] t x = σ(conv(F)) + c x ,t y = σ(conv(F)) + c y ,t w = e conv(F) ,t h = e conv(F)

[0077] Among them, (t x ,t y ) is the predicted value of the bounding box center coordinates, which is the offset based on the center of the prior box (c x ,c y ). t w ,t h is the width and height of the bounding box, which are obtained by scaling the ratio of the prior box. The predicted value is determined by the feature output conv(F) of the convolutional layer. σ is the Sigmoid function, which is used to limit the center point prediction within the range of [0,1];

[0078] S403. During the training process, the optimization objective of the model is to minimize the classification error, bounding box regression error, and confidence error. The total loss function is defined as:

[0079] L = λ cls L cls + λ box L box + λ conf L conf

[0080] During training, the bounding box regression loss L box is mainly optimized, and the Generalized Intersection over Union (GIoU) is used as the bounding box optimization metric. The expression of GIoU is:

[0081]

[0082] Among them, IoU is the Intersection over Union of the predicted box and the ground truth box. C is the area of the smallest enclosing region of the predicted box and the ground truth box. A and B are the areas of the predicted box and the ground truth box respectively. Compared with the ordinary IoU, GIoU improves the stability of the bounding box prediction by penalizing the uncovered region.

[0083] S404. The model performance is evaluated by calculating the mean Average Precision (mAP), which is defined as:

[0084]

[0085] Among them, P(r) is the precision at recall rate r, and AP is the average precision of a single class.

[0086] Preferably, the specific steps of the step S12 include:

[0087] S121. In the initial stage, estimate the initial velocity by calculating the displacement of the center of the eyeball, which is obtained by calculating the change in the key point coordinates between multiple frames;

[0088] S122. Calculate the center key point coordinates of the pupils of the left and right eyes respectively through the coordinates of the upper and lower eyelids and the left and right eye corners around the human eye, set them as the initial coordinates, and consider that the position coordinates corresponding to the initial coordinates are the center position coordinates of the line of sight focus in the subsequently created real-time window;

[0089] S123. Calculate the displacement of the center key point coordinates of the ordinary teacher's pupil relative to the initial coordinates, and then map it to the displacement of the line of sight focus relative to the center position of the real-time window;

[0090] S124. Finally, integrate the movements in the horizontal and vertical directions into a comprehensive eye movement model to obtain the real-time coordinates of the human eye line of sight focus, and display them on the computer window in real time;

[0091] S125. In order to make the eye movement model more accurate, calibrate and adjust the eye movement model to ensure the matching degree between the model and the actual eye movement.

[0092] Compared with the existing technology, the beneficial effects of the present invention are:

[0093] 1) It proposes the key cognitive mechanism for expert teachers to interpret data: through eye movement tracking and think-aloud methods, the present invention deeply analyzes the thinking mode of expert teachers when interpreting complex data reports. This provides an important theoretical basis for designing data interpretation training for ordinary teachers, helps to accurately improve their report interpretation ability, and ultimately optimizes teaching quality.

[0094] 2) It realizes the intelligent intervention strategy: by transforming the thinking mode of experts into an intelligent intervention module, the present invention provides the function of real-time feedback on the teacher's line of sight and interpretation path. The system will rely on the online teaching platform to help ordinary teachers effectively improve their data interpretation ability, thereby improving the accuracy of classroom teaching.

[0095] 3) It forms technical specifications and has the value of popularization: through feedback and verification, the present invention constructs a set of technical specifications covering key links such as eye movement data analysis and thinking guidance. This specification can be popularized in different disciplines to help teachers in a wider range improve their data literacy.

[0096] 4) Social benefits: The present invention can help the vast majority of ordinary teachers and teachers in remote areas improve their data analysis and interpretation capabilities, narrow the skill gap among teachers, promote the improvement of education quality, and achieve educational equity.

[0097] 5) Economic benefits: In the actual application stage, the present invention adopts an intelligent eye movement tracking system based on an ordinary camera instead of a professional eye tracker, reducing the equipment cost, having strong popularization potential, and effectively improving the teaching cost-effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 It is a flowchart of the key cognitive mechanism for interpreting the visualized data report of teachers based on eye movement technology provided by the present invention;

[0099] Figure 2 It is a technical solution for intelligent eye movement recognition of human-computer interaction provided by the present invention;

[0100] Figure 3 It is a schematic diagram of key points around the human eye provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0101] The present invention will be further explained below with reference to specific embodiments.

[0102] As Figure 1 shown, this embodiment proposes a teacher visualized data analysis system based on eye movement technology, including a data collection and observation experiment module, a design and verification module for demonstration examples and technical specifications, an intelligent recognition module, and an interactive feedback module. Among them,

[0103] The data collection and observation experiment module is used to analyze and refine the data thinking mode of expert teachers through experimental observation, including the following steps:

[0104] Adopt the experimental observation method, select two groups of expert teachers and ordinary teachers in a subject, observe the specific eye movements and overt thinking of these two types of teachers during the process of data report interpretation, that is, fixation points, scanning paths, thinking processes and strategies, etc., to understand the cognitive mechanism of teachers during this process.

[0105] In a specific experiment, different report types of the subjects provided by the system are covered, such as class reports, individual reports, and reports of different performance levels, etc., to better achieve the purpose of experimental observation. At the same time, the above multi-modal data will be further analyzed to discover the problems existing in the data report interpretation of ordinary teachers and to explore the data thinking mode of expert teachers.

[0106] And verify the feasibility of applying the eye movement thinking demonstration example to the cultivation of teachers' report interpretation ability from a theoretical perspective, including:

[0107] 1) The cognitive mechanism of expert teachers' interpretation of visual reports;

[0108] 2) Whether there are differences in the ability of different teachers (experts and ordinary) to interpret visual reports that can be captured by eye movement and think-aloud methods;

[0109] 3) Whether the problems existing in the ability of ordinary teachers to interpret visual reports can be revealed by eye movement and think-aloud.

[0110] The design and verification module of demonstration examples and technical specifications is used to verify the effectiveness of example learning applied to the cultivation of teachers' visual data report ability through quasi-experimental research. The specific implementation process is as follows:

[0111] It includes the following steps:

[0112] First, use the expert method (such as organizing expert meetings, Delphi method, or brainstorming method, etc.) to analyze the problems existing in the interpretation of visual data reports by ordinary teachers, and combine the optimization of the interpretation process of expert teachers' visual data reports to solve these problems, and initially implement the eye movement and thinking strategy demonstration example intervention strategy in the form of videos. The expert group includes both subject experts and information technology teaching experts, and the number is not less than 7.

[0113] Second, use the quasi-experimental research method, which is mainly divided into the following key steps:

[0114] 1) All teachers will participate in a pre-test questionnaire survey (mainly including personal information, etc.) and necessary training for participating in the experiment.

[0115] 2) The experimental group will watch the eye movement and thinking example videos made by the researcher according to the expert thinking mode. This video will achieve the purpose of guiding teachers to learn the interpretation process of expert teachers' visual data reports through methods such as local sequential highlighting and text prompts; the control group will not receive any intervention.

[0116] 3) All teachers also need to complete a post-test questionnaire and a visual data report interpretation task. During the post-test task, the eye movement and think-aloud data of teachers in the interpretation of typical data reports will be collected simultaneously, and a combination of quantitative and qualitative analysis will be used to compare the performance differences between the two groups of teachers after receiving the visual data report interpretation training based on example learning.

[0117] 4) Selectively interview the teachers in the experimental group to collect user feedback information to further revise and refine the technical specification document.

[0118] Finally, the expert method (adding experts in system design and development in addition to the above-mentioned expert group members) is used again to evaluate the technical specifications of the eye movement and thinking strategy demonstration examples formed. If it passes, it can be used as the scientific basis for the next stage; otherwise, the eye movement and thinking strategy demonstration examples need to be modified, the quasi-experiment research needs to be carried out again, and user feedback needs to be collected for improvement until the expert group passes.

[0119] The intelligent recognition module and the interactive feedback module are used to convert the obtained fixation sequences of expert teachers on different types of visual data reports and the think-aloud content at each fixation point into question scaffolds in the form of questions and store them in the relational database according to the report type. Develop human-computer interaction eye movement intelligent recognition to achieve eye movement tracking when ordinary teachers watch the screen report. In the function modules of the actual developed intelligent intervention system, the example learning expert knowledge base and the human-computer interaction eye movement intelligent recognition will be further combined, which is reflected in:

[0120] 1) When starting the expert system, the system first uses the built-in camera of the computer (or an ordinary USB external camera). At this time, the camera can record the line-of-sight position of ordinary teachers by simulating a professional eye movement tracking device through this system.

[0121] 2) After obtaining the fixation points of ordinary teachers, according to the current report type, select the content in the corresponding expert knowledge base for comparison. If it is within the error range of the expert fixation points, give the thinking scaffold (presented in the form of questions to promote in-depth thinking of ordinary teachers) when the expert pays attention to this place, and then give the position of the expert's next fixation point; if it is different from the expert fixation point position, prompt the position that ordinary teachers should pay attention to at this time.

[0122] As Figure 2 shown, this embodiment also proposes a method for analyzing teachers' visual data based on eye movement technology, including the following steps:

[0123] S1: Use the Dlib library to combine manual operation to annotate 98 face key points on the publicly available face dataset.

[0124] The key point annotation task completes the initial automatic annotation through the HOG features of Dlib and the set regression tree model. First, calculate the distribution histogram of the local gradient direction, and generate a stable face feature representation after normalization. The formal definition is as follows:

[0125]

[0126] Among them, Grad i represents the gradient direction distribution of the i-th pixel in the image, w i is the weight of a specific window, and Z is the normalization factor.

[0127] Based on this feature, Dlib's classifier can quickly locate the rectangular area of the face. In the face landmark detection stage, combined with the method based on Ensemble of Regression Trees (ERT), the true position is gradually approximated by performing multiple regression optimizations on the initial position of each landmark. For a given landmark prediction error, the model corrects the landmark position ΔP in each iteration:

[0128] ΔP = R(X) + ∈

[0129] where R(X) is the offset predicted by the regression tree model, and ∈ is the residual. The initial 68-point model obtains relatively accurate results through multiple rounds of iteration.

[0130] To extend to 98 landmarks, it is necessary to process and supplement the annotation of the publicly available face dataset. Select publicly available datasets such as 300-W and WFLW. These datasets cover a variety of complex scenarios (such as different lighting conditions, occlusion, expression changes, etc.) to ensure the diversity and generalization ability of the annotation data. Based on the initial 68 points generated by Dlib, the initial positions of the newly added 30 points are calculated by geometric interpolation. The calculation formula is:

[0131] P new = αP i + (1 - α)P j , α ∈ [0, 1]

[0132] where P i and P j are the positions of adjacent points respectively, and α is the scaling factor.

[0133] For the newly generated points after geometric interpolation, an artificial correction tool (LabelImg and VGG ImageAnnotator) is used to adjust their positions one by one to make them more conform to the true distribution of facial features. After manual adjustment, through cross-validation, different annotators review the same batch of data to eliminate potential annotation biases. The annotation consistency is evaluated by calculating the mean squared error (MSE):

[0134]

[0135] The finally output annotated data is cleaned and standardized, and the unified format is JSON or CSV.

[0136] S2: Preprocess the dataset.

[0137] Based on the cleaned and standardized annotation data generated in step one, further preprocessing operations such as face alignment, normalization, noise removal, and data augmentation are performed on it to ensure the spatial consistency and diversity of the data.

[0138] First, perform alignment operations on the faces in the data to eliminate geometric differences caused by head poses (such as rotation, translation, etc.). Through the affine transformation method, according to the key point positions of the two eyes and the tip of the nose in the labeled data, calculate the affine transformation matrix M to achieve the standardized alignment of the facial region. The formula for affine transformation is:

[0139] P′ = M · P

[0140] where P′ and P are the coordinates of the aligned and original images respectively, M is the affine transformation matrix, and by setting the correspondence between the reference points (such as the positions of the two eyes and the tip of the nose) and the target points, calculate the transformation matrix M through the method of least squares fitting.

[0141] The size of the aligned images is uniformly adjusted to 112×112, and the pixel values are normalized to reduce the size and brightness distribution differences of the input images, ensuring that the model can process inputs with a unified scale and statistical distribution. Normalize the pixel values of the original image to a standard normal distribution with a mean of 0 and a variance of 1. The formula is:

[0142]

[0143] Here, I(x,y) is the pixel value of the original image, μ and σ are the mean and standard deviation of the pixel values respectively, W, H are the width and height of the original image, and W′, H′ are the normalized sizes. Normalization effectively reduces the dynamic range differences of the image data and improves the stability and convergence speed of the training process.

[0144] Use a Gaussian filter to smooth the noise regions to further improve the data quality. The filter kernel function is defined as:

[0145]

[0146] where σ controls the width of the Gaussian distribution and determines the smoothness of the filter. The result of the convolution of the Gaussian filter and the image makes the local pixel values more uniform, thus effectively suppressing the interference of noise.

[0147] In the data augmentation stage, to expand the diversity of samples and improve the generalization ability of the model, randomly apply transformations such as cropping, rotation, brightness adjustment, and flipping to each image. Expand the diversity of training samples and enhance the generalization ability of the model.

[0148] The brightness adjustment here is achieved by applying a linear transformation to the pixel values. The formula is:

[0149] I′(x,y) = α · I(x,y) + β

[0150] Among them, α is the brightness scaling factor and β is the brightness offset value. Training samples with different brightness characteristics are generated by randomly sampling different values.

[0151] S3: Input the labeled dataset into the PFLD neural network model for training.

[0152] After completing key point annotation and data preprocessing, the generated data is used to train the PFLD (Progressive Face Localization and Detection) neural network model to achieve high-precision localization of face key points. During the model training process, first, the preprocessed image is input into the global feature extraction module of PFLD, and multi-scale global features are extracted through a lightweight convolutional network. The calculation formula for the convolution operation is:

[0153] F (l+1) =σ(W (l) *F (l) +b (l) )

[0154] Among them, F (l) is the feature map of the l-th layer, W (l) and b (l) are the convolution kernel weights and bias terms respectively, * represents the convolution operation, and σ is the non-linear activation function. The global features extracted through this process contain the overall semantic information of the face and can describe the geometric features of the face.

[0155] The global features are input into the local refinement regression module to gradually optimize the predicted position of the key points. Assuming the initial key point prediction is P init , the local refinement module will learn to predict the offset ΔP and gradually approximate the position of the true key point through the following formula:

[0156] P final =P init +ΔP

[0157] To measure the error between the model prediction value and the true value, PFLD uses the L2 loss function as the optimization objective, which is defined as:

[0158]

[0159] Among them, N is the number of key points, and They respectively represent the predicted value and the ground truth of the i-th key point. Selecting the L2 loss can impose a higher penalty on larger prediction deviations, thus promoting faster convergence of the model. During the training process, the dataset is divided into a training set, a validation set, and a test set, with proportions of 70%, 20%, and 10% respectively. The training set is used to optimize the model parameters, the validation set is used to evaluate the performance of the model on unseen data, and the test set is used for the final generalization performance evaluation. The Adam optimizer is used during the optimization process. It combines the advantages of momentum and adaptive learning rate, and the parameter update formula is:

[0160]

[0161] where and are the unbiased estimates of the first moment and the second moment of the gradient respectively, η is the learning rate, and ∈ is a stabilizing factor to avoid division by zero. To ensure that the model can converge faster and not overfit, the CosineAnnealing strategy is adopted to dynamically adjust the learning rate, which is defined as:

[0162]

[0163] where η t is the learning rate at the t-th iteration, η min and η max are the minimum and maximum values of the learning rate respectively, and T is the total number of training epochs. After the training is completed, the Normalized Mean Error (NME) is used to evaluate the model performance, which is defined as:

[0164]

[0165] where d is the standard scale for normalization, and the distance between the two eyes is selected. NME can intuitively reflect the localization accuracy of the model on the validation set. During the testing process, the predicted results of the model are compared with the ground truth in a visual way, and the adaptability of the model in various scenarios is further manually checked to train a reliable key point prediction model.

[0166] S4: Use the existing face detection dataset for the training of the YOLOv8 neural network.

[0167] It should be noted that although YOLOv11 is the latest version of the YOLO object detection algorithm, we choose the mature YOLOv8 model to complete this face detection task considering stability and architecture integrity.

[0168] Before model training, the preprocessed image data in step two is further converted into the YOLO format. The annotation information of each image is represented by a five-tuple (c, x, y, w, h), where c is the class label (fixed as 1 representing a face in this task), (x, y) is the center coordinate of the bounding box, and (w, h) are the width and height of the bounding box. All values are normalized to the range [0, 1]. The normalization formula is:

[0169]

[0170] where W and H are the width and height of the image respectively. After annotation, the dataset is divided into a training set, a validation set, and a test set for model training, performance validation, and final evaluation.

[0171] YOLOv8 extracts multi-scale features through the backbone network and fuses feature maps of different resolutions through the Feature Pyramid Network (FPN). Finally, the detection head regresses the bounding box coordinates and predicts the confidence. The prediction formula for the bounding box coordinates is:

[0172] t x = σ(conv(F)) + c x ,t y = σ(conv(F)) + c y ,t w = e conv(F) ,t h = e conv(F)

[0173] where (t x ,t y ) is the predicted value of the bounding box center coordinate, which is the offset based on the center of the prior box (c x ,c y ). t w ,t h are the width and height of the bounding box, obtained by scaling the ratio of the prior box. The predicted value is determined by the feature output conv(F) of the convolutional layer, and σ is the Sigmoid function used to limit the center point prediction to the range [0, 1].

[0174] During the training process, the optimization objective of the model is to minimize the classification error, the bounding box regression error, and the confidence error. The total loss function is defined as:

[0175] L = λ cls L cls + λ box L box + λ conf L conf

[0176] During training, the bounding box regression loss L box, the Generalized Intersection over Union (GIoU) is adopted as the bounding box optimization metric. The expression of GIoU is:

[0177]

[0178] where IoU is the Intersection over Union of the predicted box and the ground truth box, C is the area of the smallest enclosing region of the predicted box and the ground truth box, and A and B are the areas of the predicted box and the ground truth box respectively. Compared with the ordinary IoU, GIoU improves the stability of bounding box prediction by penalizing the uncovered region.

[0179] The model performance is evaluated by calculating the mean Average Precision (mAP), which is defined as:

[0180]

[0181] where P(r) is the precision at recall rate r, and AP is the average precision of a single class.

[0182] S5: Output the result of face detection.

[0183] S6: Backpropagation, which is used to optimize the network weights of the YOLOv8 neural network.

[0184] S7: Apply the trained face detection neural network to the actual video of the teacher facing the computer screen.

[0185] S8: Read and decode the real-time video of the ordinary teacher facing the computer screen to obtain video frames, and perform face detection on the extracted video frames using the YOLOv8 neural network.

[0186] S9: Crop the face image according to the face detection box.

[0187] S10: After preprocessing the cropped face image, input it into the trained face key point detection network model PFLD.

[0188] S11: The PFLD neural network model outputs the recognition result to obtain the key point coordinates of the human eyes.

[0189] S12: After obtaining the human eye coordinates, use the detected human eye key points to establish an eye movement model (the formula is shown below) to estimate the focus of the human eye's line of sight, infer the orientation of the eyeball, and finally obtain the focus coordinates of the line of sight in the real-time window.

[0190] New horizontal position = Old horizontal position + Horizontal speed * Time interval New vertical position = Old vertical position + Vertical speed * Time interval

[0191] In the initial stage, the initial speed can be estimated by calculating the displacement of the center of the eyeball. It can be calculated from the change of the key point coordinates between multiple frames.

[0192] Calculate the center key-point coordinates of the left and right eye pupils respectively (as shown in the following formula) through the key points around the human eye (the coordinates of the upper and lower eyelids, the coordinates of the left and right eye corners; see Figure 3 ). Set them as the initial coordinates, and consider that the coordinates corresponding to the initial coordinates are the center position coordinates of the line of sight focus in the subsequently created real-time window. Then, calculate the displacement of the center key-point coordinates of the ordinary teacher's pupil relative to the initial coordinates through the above formula, and then map it to the displacement of the line of sight focus relative to the center position of the real-time window. Finally, integrate the movements in the horizontal and vertical directions into a comprehensive eye movement model to obtain the real-time coordinates of the human eye line of sight focus. And display it on the computer window in real time.

[0193]

[0194] Note: The calculation formula for the center position of the right eye pupil is the same by analogy.

[0195] Finally, in order to make the eye movement model more accurate, the eye movement model can also be calibrated and adjusted to ensure the matching degree between the model and the actual eye movement. Some known calibration points can be used for adjustment, and a 3D model of the human head can be introduced, or calibration can be carried out through the user feedback mechanism. At the same time, a Kalman filter is introduced for trajectory prediction to better capture the dynamic characteristics of eye movement.

[0196] S13: Call the "imshow" function in the OpenCV library to generate a real-time window on the computer side. This window is scaled in proportion to the computer screen, and a coordinate mapping relationship is established between the two. Therefore, when we obtain the coordinate position of the human eye on the real-time window, through the mapping function, we can obtain the coordinate position of the human eye on the computer screen. Among them, the product ratio of the resolution between the computer screen and the real-time window is a key parameter Ratio, and the relationship is defined as follows:

[0197]

[0198] This ratio directly affects the mapping accuracy of the human eye's line of sight focus coordinates between the two. When the area ratio is larger, the distortion of mapping the line of sight focus coordinates from the real-time window to the computer screen is less, and the accuracy is higher. When selecting the product ratio of the resolution of the computer screen and the real-time window, a trade-off needs to be made between high precision and program running efficiency. A large ratio can provide more accurate coordinate mapping, but may lead to a decrease in program running efficiency, especially when dealing with a large amount of real-time data. On the contrary, a small ratio can improve the program running efficiency, but at the cost of sacrificing a certain amount of accuracy. In this study, the function of the system is to roughly track the eye movements of ordinary teachers, and a certain error is allowed in the subsequent comparison with the eye movement fixation points of expert teachers (not the exact position). Therefore, the above implementation method is feasible.

[0199] S14: Based on the coordinate position of the human eye on the computer screen obtained in step 13, create a list, add the new line of sight focus coordinates to the list, and write and save them in the log. Then traverse the coordinate list and use the "cv2.circle" function to draw the eye movement trajectory, which is superimposed on the original face image. As the program runs, this list will be continuously updated to contain more coordinate points. Before drawing each frame, the program will traverse this list and use "cv2.circle" to draw small dots at all these coordinates to form a trajectory. At the same time, if the new line of sight focus coordinates and the previous line of sight focus coordinates are within a certain range (i.e., there is no significant movement), then calculate the difference between the current timestamp and the timestamp recorded last time. If it exceeds 100 ms, it is judged as a fixation point.

[0200] In the data collection and observation experiments, demonstration examples, and design and verification stages of technical specifications of the present invention, a cognitive analysis method combining eye movement technology and think-aloud protocol recording is adopted.

[0201] A professional eye tracker is used to analyze the teacher's fixation situation, path, etc. The eye movement device used is the GP3HD eye movement device of Gazepoint Company, and its sampling frequency is 150 Hz. In the experimental part, the eye movement device will be used to collect eye movement data such as the teacher's fixation points, attention duration and frequency, and scanning path, and analyze them separately for the expert and ordinary teacher groups to extract the eye movement behavior characteristics and fixation sequences of experts. In the design and verification part of technical specifications, it is necessary to collect the eye movement data of teachers in the experimental group and the control group during the process of solving problems by themselves (where the experimental group receives sample learning intervention in advance), such as fixation points, attention duration and frequency, and scanning path, and use quantitative and qualitative analysis methods such as t-test, eye movement heat map, and path analysis to compare whether there are significant differences between the two to reveal whether learning based on eye movement and thinking strategy demonstration examples can promote the acquisition of teachers' visualization data report interpretation ability.

[0202] To minimize the interference of the think-aloud process to teachers, all teachers were trained before starting the task to be able to express their thinking processes verbally while carrying out the task; in the formal experiment, the think-aloud of teachers was recorded and analyzed for thinking strategies in combination with eye movement data. The experimental part refined the problem-solving strategies of expert teachers and revealed potential problems of ordinary teachers from the collected think-aloud. The technical specification design verification part collected the think-aloud information of teachers in the experimental group and the control group, and qualitatively coded it according to the analysis framework to reveal the thinking differences between the two groups of teachers with and without the support of learning from eye movement and thinking strategy demonstration examples.

[0203] This analysis method combining eye movement technology and think-aloud makes up for the deficiency of traditional cognitive analysis relying solely on a single technology, and more comprehensively and accurately demonstrates the multi-dimensional cognitive behaviors of teachers in the interpretation process.

[0204] As described above, it is only the preferred specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered within the protection scope of the present invention.

Claims

1. A teacher visualization data analysis system based on eye movement technology, characterized in that: It includes data collection and observation experiment module, demonstration sample and technical specification design and verification module, intelligent recognition module and interactive feedback module. The data collection and observation experiment module is used to refine the data thinking mode of expert teachers through experimental observation and analysis, and to verify the feasibility of applying the eye movement thinking demonstration examples to the cultivation of teachers' report interpretation ability from a theoretical perspective; The design and verification module of the demonstration examples and technical specifications is used to verify the effectiveness of example learning in cultivating teachers' visual data reporting ability through quasi-experimental research; The intelligent recognition module and the interactive feedback module are used to convert the acquired gaze sequences of expert teachers on different types of visual data reports and the vocal thinking content at each gaze point into question scaffolds in the form of questions, etc., and store them in a relational database according to the report type, develop intelligent eye movement recognition for human-computer interaction, and realize eye movement tracking of ordinary teachers when watching screen reports.

2. According to claim 1, a teacher visualization data analysis system based on eye movement technology is characterized in that: The specific process of the experimental observation and analysis is: select a subject and recruit two groups of expert teachers and ordinary teachers, observe the specific eye movements and vocal thinking of these two types of teachers in the process of interpreting data reports, that is, gaze points, scanning paths, thinking processes and strategies, and understand the cognitive mechanisms of teachers in this process. In the specific experiment, the system provides class reports, personal reports, and reports of different performance levels to achieve the purpose of experimental observation, analyze and discover the problems existing in the interpretation of data reports by ordinary teachers and explore the data thinking patterns of expert teachers.

3. According to claim 1, a teacher visualization data analysis system based on eye movement technology is characterized in that: The specific process of verification from the theoretical perspective is: the cognitive mechanism of expert teachers' interpretation of visual reports, analyzing whether there are differences in the visual report interpretation abilities of different expert teachers and ordinary teachers that can be captured by eye movements and think-aloud methods, and analyzing whether the problems of ordinary teachers in their visual report interpretation abilities can be revealed through eye movements and think-aloud methods.

4. The teacher visualization data analysis system based on eye movement technology according to claim 1 is characterized in that: When the intelligent recognition module and the interactive feedback module are started, the computer's built-in camera is called to record the line of sight position of the ordinary teacher. After obtaining the ordinary teacher's gaze point, it is compared with the content in the corresponding expert knowledge base according to the current report type. If the error range is within the expert's gaze point, the thinking scaffolding of the expert's attention is given, and then the expert's next gaze point position is given; if it is different from the expert's gaze point position, the ordinary teacher is prompted with the position that he should pay attention to at this time.

5. A teacher visualization data analysis method based on eye movement technology, characterized in that: The following steps are involved: S1: Annotate the facial key points of the public face dataset; S2: Preprocessing the data set, based on the cleaned and standardized annotated data generated in S1, further performing preprocessing operations such as face alignment, normalization, noise removal and data enhancement to ensure the spatial consistency and diversity of the data; S3: Input the labeled data set into the PFLD neural network model for training; S4: Use the existing face detection dataset for training the YOLOv8 neural network; S5: Output the result of face detection; S6: Back propagation, used to optimize the network weights of the YOLOv8 neural network; S7: Apply the trained face detection neural network to the video of the actual teacher facing the computer screen; S8: Read and decode the real-time video of ordinary teachers facing the computer screen to obtain video frames, and use the YOLOv8 neural network to perform face detection on the extracted video frames; S9: Crop and obtain a face image according to the face detection frame; S10: After preprocessing, the cropped face image is input into the trained face key point detection network model PFLD; S11: The PFLD neural network model outputs the recognition results and obtains the key point coordinates of the human eye; S12: After obtaining the eye coordinates, using the detected eye key points, an eye movement model is established to estimate the eye focus, infer the direction of the eye, and finally obtain the eye focus coordinates of the real-time window; S13: calling the "imshow" function in the OpenCV library to generate a real-time window on the computer side. The window is proportional to the computer screen, and a coordinate mapping relationship is established between the two. After obtaining the coordinate position of the human eye on the real-time window, the coordinate position of the human eye on the computer screen can be obtained through the mapping function; S14: Based on the coordinate position of the human eye on the computer screen obtained in S13, create a list, add the new sight focus coordinates to the list, and write them into the log for storage. Then, traverse the coordinate list and use the "cv2.circle" function to draw the eye movement trajectory, and superimpose it on the original face image. As the program runs, the list will be continuously updated to include more coordinate points. Before drawing each frame, the program will traverse this list and use "cv2.circle" to draw small dots on all these coordinates to form a trajectory. At the same time, if the new sight focus coordinates are within a certain range with the previous sight focus coordinates, that is, no significant movement has occurred, then calculate the difference between the current timestamp and the last recorded timestamp, and if it exceeds 100ms, it will be judged as the gaze point.

6. The teacher visualization data analysis method based on eye movement technology according to claim 5 is characterized in that: In S1, the facial key points are annotated by using the HOG feature and ensemble regression tree model of Dlib to complete the initial automatic annotation. The specific steps include: S101, calculate the distribution histogram of the local gradient direction, and generate a stable facial feature representation after normalization. The formal definition is as follows: Among them, Grad i represents the gradient direction distribution of the i-th pixel in the image, w i is the weight of a specific window, and Z is the normalization factor; S102. Based on this feature, Dlib's classifier can quickly locate the rectangular frame area of ​​the face. In the face key point detection stage, combined with the method based on ensemble regression trees (ERT), the initial position of each key point is gradually approached to the true position by multiple regression optimizations. For a given key point prediction error, the model corrects the key point position AP in each iteration: AP=R(X)+∈ Among them, R(X) is the offset predicted by the regression tree model, and ∈ is the residual. The initial 68-point model obtains more accurate results after multiple rounds of iterations; S103. In order to expand to 98 key points, the annotated public face datasets need to be processed and supplemented with annotations. Public datasets such as 300-W and WFLW are selected. These datasets cover a variety of complex scenes to ensure the diversity and generalization ability of the annotated data. Based on the initial 68 points generated by Dlib, the initial positions of the additional 30 points are calculated by the geometric interpolation method. The calculation formula is: P new =αP i +(1-α)P j ,α∈[0,1] Among them, P i and P j are the positions of adjacent points, and α is the scale factor; S104. For the new points generated after geometric interpolation, the positions are adjusted one by one using manual correction tools to make them more consistent with the actual distribution of facial features. After manual adjustment, different annotators review the same batch of data through cross-checking to eliminate potential annotation bias. The annotation consistency is evaluated by calculating the mean square error (MSE): The final output annotation data is cleaned and standardized, and the unified format is JSON or CSV.

7. The teacher visualization data analysis method based on eye movement technology according to claim 5 is characterized in that: In S2, the specific steps of preprocessing the data set include: S201, align the faces in the data to eliminate the geometric differences caused by the head posture, and calculate the affine transformation matrix M according to the key point positions of the eyes and the nose tip in the annotated data through the affine transformation method to achieve standardized alignment of the facial area. The formula of affine transformation is: P′=MP Among them, P′ and P are the coordinates of the aligned and original images respectively, and M is the affine transformation matrix. By setting the correspondence between the reference point and the target point, the transformation matrix M is calculated by the least squares fitting method; S202, the aligned images are uniformly resized to 112×112, and the pixel values ​​are normalized to reduce the size and brightness distribution differences of the input images, ensuring that the model can handle inputs with uniform scale and statistical distribution. The original image pixel values ​​are normalized to a standard normal distribution with a mean of 0 and a variance of 1, and the formula is: I(x, y) is the pixel value of the original image, μ and σ are the mean and standard deviation of the pixel value, W and H are the width and height of the original image, and W′ and H′ are the normalized sizes. Normalization effectively reduces the dynamic range difference of image data and improves the stability and convergence speed of the training process; S203, using a Gaussian filter to smooth the noise area to further improve the data quality. The filter kernel function is defined as: Among them, σ controls the width of the Gaussian distribution and determines the smoothness of the filter. The result of convolution of the Gaussian filter and the image makes the local pixel values ​​more uniform, thereby effectively suppressing the interference of noise; S204. In the data enhancement stage, in order to expand the diversity of samples and improve the generalization ability of the model, cropping, rotation, brightness adjustment and flipping transformations are randomly applied to each image to expand the diversity of training samples and enhance the generalization ability of the model. Brightness adjustment is achieved by applying a linear transformation to the pixel value. The formula is: I′(x, y)=α·I(x, y)+β Among them, α is the brightness scaling factor, β is the brightness offset value, and training samples with different brightness characteristics are generated by random sampling of different values.

8. The teacher visualization data analysis method based on eye movement technology according to claim 5 is characterized in that: In S3, after completing key point labeling and data preprocessing, the generated data is used to train the PFLD neural network model to achieve high-precision positioning of facial key points. The specific steps of the training include: S301, input the preprocessed image into the global feature extraction module of PFLD, and extract multi-scale global features through a lightweight convolutional network. The calculation formula of the convolution operation is: F (l+1) =σ(W (l) *F (l) +b (l) ) Among them, F (l) is the feature map of the lth layer, W (l) and b (l) are the convolution kernel weight and bias term respectively, * represents the convolution operation, σ is the nonlinear activation function. The global features extracted through this process contain the overall semantic information of the face and can describe the geometric features of the face; S302: The global features are input into the local refinement regression module to gradually optimize the predicted positions of the key points. Assuming that the initial key point prediction is P init , the local refinement module learns to predict the offset AP and gradually approaches the position of the real key point through the following formula: P final =P init +AP S303. In order to measure the error between the model prediction value and the true value, PFLD uses the L2 loss function as the optimization target, which is defined as: Where N is the number of key points, and Represent the predicted value and true value of the i-th key point respectively. Selecting L2 loss can impose higher penalties on larger prediction deviations, thereby promoting faster convergence of the model. During the training process, the dataset is divided into training set, validation set and test set, with proportions of 70%, 20% and 10% respectively; S304, the training set is used to optimize the model parameters, the validation set is used to evaluate the performance of the model on unseen data, and the test set is used for the final generalization performance evaluation. The Adam optimizer is used in the optimization process, which combines the advantages of momentum and adaptive learning rate. The parameter update formula is: in, and are the unbiased estimates of the first and second moments of the gradient, η is the learning rate, and ∈ is a stabilization factor to avoid the denominator being zero; S305. To ensure that the model can converge faster and not overfit, the cosine annealing strategy is used to dynamically adjust the learning rate, which is defined as: Among them, η t is the learning rate at the tth iteration, η min and η max are the minimum and maximum values ​​of the learning rate, respectively, and T is the total number of training rounds; S306. After training is completed, the Normalized Mean Error (NME) is used to evaluate the model performance, which is defined as: Among them, d is the standard scale used for normalization, and the distance between the two eyes is selected. NME can intuitively reflect the positioning accuracy of the model on the validation set. During the test, the prediction results of the model are compared with the true values ​​in a visual way, and the adaptability of the model in various scenarios is further manually checked to train a reliable key point prediction model.

9. The teacher visualization data analysis method based on eye movement technology according to claim 5 is characterized in that: In S4, the specific steps of training include: S401. Before model training, the preprocessed image data in S2 is further converted into the YOLO format. The annotation information of each image is represented by a five-tuple (c, x, y, w, h), where c is the category label, (x, y) is the coordinate of the center of the border, (w, h) is the width and height of the border, and all values ​​are normalized to the interval [0, 1]. The normalization formula is: Where W and H are the width and height of the image respectively. After the annotation is completed, the dataset is divided into training set, validation set and test set for model training, performance verification and final evaluation. S402 and YOLOv8 extract multi-scale features through the backbone network and fuse feature maps of different resolutions through the feature pyramid network (FPN). Finally, the detection head regresses the bounding box coordinates and prediction confidence. The prediction formula of the bounding box coordinates is: t x =σ(conv(F))+c x ,t y =σ(conv(F))+c y ,t w =e conv(F) ,t h =e conv (F) Among them, (t x , t y ) is the predicted value of the center coordinate of the bounding box, based on the prior box center (c x , c y ) offset, t w , t h is the width and height of the border, obtained by scaling the prior box. The predicted value is determined by the feature output conv(F) of the convolutional layer, and σ is the Sigmoid function, which is used to limit the center point prediction to the range of [0, 1]; S403. During the training process, the optimization goal of the model is to minimize the classification error, bounding box regression error and confidence error. The total loss function is defined as: L=λ cls L cls +λ box L box +λ conf L conf During training, we focus on optimizing the bounding box regression loss L box , the generalized intersection-over-union (GIoU) ​​is used as the border optimization indicator. The expression of GIoU is: Among them, IoU is the intersection-over-union ratio of the predicted box and the real box, C is the minimum enclosing area of ​​the predicted box and the real box, A and B are the areas of the predicted box and the real box respectively. Compared with ordinary IoU, GIoU improves the stability of border prediction by penalizing uncovered areas. S404, model performance evaluation is calculated by the mean average precision (mAP), which is defined as: Among them, P(r) is the precision at recall rate r, and AP is the average precision of a single category.

10. The teacher visualization data analysis method based on eye movement technology according to claim 5 is characterized in that: The specific steps of step S12 include: S121, in the initial stage, the initial velocity is estimated by calculating the displacement of the center of the eyeball, which is calculated by the change of the coordinates of the key points between multiple frames; S122, calculating the coordinates of the center key points of the left and right pupils through the upper and lower eyelid coordinates and the left and right eye corners, setting them as initial coordinates, and considering that the initial coordinates correspond to the center position coordinates of the sight focus in the real-time window created subsequently; S123, calculating the displacement of the coordinates of the key point of the pupil center of the ordinary teacher relative to the initial coordinates, and then mapping it to the displacement of the sight focus relative to the center position of the real-time window; S124, finally integrating the movements in the horizontal and vertical directions into a comprehensive eye movement model to obtain real-time human eye sight focus coordinates, and displaying them in real time on a computer window; S125. In order to make the eye movement model more accurate, the eye movement model is calibrated and adjusted to ensure the matching degree between the model and the actual eye movement.

Images