System for evaluating reading comprehension behavior in combination with an eye-tracking system

Abstract

A system for evaluating reading comprehension behavior in combination with an eye-tracking system installed in an electronic device, comprising the following: a camera unit for capturing a user's facial image; and a processing unit connected to the camera unit, comprising the following: an image processing module for analyzing the facial image in order to analyze multiple eye movement data of the user, including a gaze target position, eye movement path and gaze dwell time; an article focus data module that automatically analyzes the semantic structure, text distribution, or paragraph hierarchy of the article content based on its characteristics, in order to create multiple text areas and multiple corresponding weightings of comprehension quality; a reading behavior analysis module that is linked to the image processing module and serves to create several reading behavior indicators based on the user's eye movement data; and A comprehension assessment module, linked to the article focus data module and the reading behavior analysis module, serves to weight the reading behavior indicators taking into account the weightings of comprehension quality in order to assess the user's level of understanding in relation to the article content.

Description

Field of invention

[0001] The present invention relates to an electronic device and in particular a system for evaluating reading comprehension behavior in combination with an eye-tracking system. State of the art

[0002] Existing eye-tracking techniques are primarily used in the control of human-machine interfaces, research on visual attention, emotion recognition, and the analysis of gaze behavior when viewing advertisements. Common methods utilize infrared light sources in combination with high-resolution cameras to capture pupillary and corneal reflex points and calculate the eye fixation positions and eye movement path of a user. Some systems can also analyze visual behaviors such as saccades, fixations, and flashbacks to provide information about a user's fixation characteristics when viewing objects or stimulus materials.However, such systems often rely on special hardware, permanently installed light sources, or complex calibration procedures, which increases costs and the threshold for use, making them less suitable for general educational scenarios or reading training situations with a large number of learners.

[0003] In contrast, existing educational technology products can log reading times, test results, or page navigation behavior, but they largely remain at the level of estimating learning status based on "user response tests" and are unable to directly reflect the visual attention distribution or the learners' interactive comprehension behaviors during reading. Similarly, it is difficult to determine which paragraphs of an article a user is reading, which sections are being skipped, or in which areas a lack of focus occurs, thus preventing a precise assessment of comprehension quality.Furthermore, existing eye-tracking products primarily focus on the "correspondence between gaze and target objects" (that is, "it is only known where the user is looking") as a superficial form of behavioral recognition, without further considering the article content, text structure, paragraph emphases, or levels of comprehension (that is, "how the user reads and what they read"). Therefore, such systems are unsuitable for assessing a reader's reading comprehension performance.

[0004] In view of the above-described shortcomings of the prior art and future requirements, the present invention proposes a system for evaluating reading comprehension behavior in combination with an eye-tracking system, the specific architecture and embodiments of which are described in detail below. Object of the invention

[0005] One object of the present invention is to provide a system for evaluating reading comprehension behavior in combination with an eye-tracking system, wherein the system first automatically analyzes the semantic structure of an article and creates weightings of text areas in order to solve the problem of the prior art in which the text content is not taken into account and only a superficial analysis of the correspondence between gazes and target objects is carried out.

[0006] An object of the present invention is to provide a system for evaluating reading comprehension behavior in combination with an eye-tracking system, based on statistical characteristics, to assess a user's reading comprehension level by using the distribution of eye dwell time, the reading marking operations performed by the user, and the automatic matching analysis results with text focal points to evaluate the reading comprehension level and thus solve the prior art problem whereby the user's comprehension quality with respect to individual article paragraphs is not quantifiable.

[0007] One object of the present invention is to provide a system for evaluating reading comprehension behavior in combination with an eye-tracking system that is capable of recognizing user reading comprehension behavior patterns, including saccades, fixations, skimming, looking back, line tracking and line-space jumps, in order to solve the prior art problem where only fixation points can be logged and no recognition of reading strategies or reading behaviors is possible.

[0008] An object of the present invention is to provide a system for evaluating reading comprehension behavior in combination with an eye-tracking system, which performs non-invasive eye-tracking by means of the front camera or the externally connected camera of an electronic device and combines the gaze target position, the eye-tracking path and the reading behavior-related feedback with a text content, in order to solve the problem of the prior art which has to resort to special, expensive and costly infrared eye trackers, as well as to solve the further problem that the prior art can only detect fixation positions, but is not able to reflect the actual reading behavior of a user and its relationship to the article content.

[0009] To solve the aforementioned problems, the present invention provides a system for evaluating reading comprehension behavior in combination with an eye-tracking system, installed in an electronic device, comprising: a camera unit for capturing a user's facial image; and a processing unit connected to the camera unit, comprising: an image processing module for analyzing the facial image to analyze multiple eye-tracking data points of the user, including a gaze target position, eye movement path, and gaze dwell time; an article focus data module that automatically analyzes the semantic structure, text distribution, or paragraph hierarchy of the article content based on features of the article content in order to generate multiple text areas and multiple corresponding comprehension quality weightings;a reading behavior analysis module, which is linked to the image processing module and serves to create several reading behavior indicators based on the user's eye movement data; and a comprehension assessment module, which is linked to the article focus data module and the reading behavior analysis module and serves to weight the reading behavior indicators, taking into account the weightings of comprehension quality, in order to assess the user's level of understanding with regard to the article content.

[0010] According to an embodiment of the present invention, the reading behavior indicators include an eye movement time, a total saccade path, a direction of movement, a ratio of forward to return movement path, a dwell pattern, a logging of skipping reading, a direction of visual trajectory, a distribution of the article focal points on which the gaze lingers, and dwell duration statistics.

[0011] According to an embodiment of the present invention, the reading behavior indicators further comprise at least one of the following parameters: review rate, skimming rate, line start point deviation, fixation dispersion, forward and backward trajectory distribution, off-screen rate, and defocus rate.

[0012] According to an embodiment of the present invention, the processing unit further comprises a posture correction module that creates a head posture frame and, upon detection of a head deviation of a user that exceeds the area of ​​the head posture frame, adjusts or discards the eye movement data acquired during this time in order to correct recognition errors caused by eye deviations of the user.

[0013] According to an embodiment of the present invention, the image processing module identifies the gaze target position based on at least one of the following features: eye features, pupil position, corneal reflection and facial features.

[0014] According to an embodiment of the present invention, the article focus data module creates text areas and weightings of comprehension quality based on the text features, the paragraph structure or the semantic distribution of an article content.

[0015] According to an embodiment of the present invention, the comprehension assessment module performs an evaluation of the level of understanding based on the correspondence relationship between the gaze target position and the article content, wherein the evaluation includes the quality of understanding and the reading accuracy, wherein the reading accuracy includes the reading precision, the reading stability and the reading efficiency in order to evaluate the reading behavior and the quality of understanding of a user.

[0016] According to an embodiment of the present invention, the processing unit further comprises a reporting module which is connected to the comprehension assessment module and serves to output at least one visualization report on reading behavior, including fixation heatmap, saccade path diagram, reading radar diagram, comprehension curve or other results of reading diagnostics.

[0017] According to an embodiment of the present invention, the reading behavior analysis module or the reporting module further creates a learning curve or a long-term comprehension curve based on several reading data sets.

[0018] According to one embodiment of the present invention, the camera unit is either the front camera or the externally connected camera of an electronic device. Brief description of the drawings Fig. Figure 1 shows a block diagram of the system according to the invention for evaluating reading comprehension behavior in combination with an eye-tracking system; Fig. Figure 2 shows a block diagram of another embodiment of the system according to the invention for evaluating reading comprehension behavior in combination with an eye-tracking system. Detailed description of the exemplary implementations

[0019] The technical solution contained in the exemplary embodiments of the present invention is described in detail and completely below with reference to the drawings of those embodiments. The described exemplary embodiments naturally represent only some of the exemplary embodiments of the present invention and not all of them.

[0020] It will be on Fig. Reference is made to Figure 1, which shows a block diagram of the system according to the invention for evaluating reading comprehension behavior in combination with an eye-tracking system. The system according to the invention for evaluating reading comprehension behavior in combination with an eye-tracking system 10 is installed in an electronic device 11. The electronic device 11 can be any commercially available or portable electronic device with a screen and a front-facing camera or an externally connected camera, including, but not limited to, tablets, smartphones, laptops, all-in-one PCs, desktop computers with an externally connected camera, educational reading devices, or other electronic products with processor and camera functionality.The system for evaluating reading comprehension behavior in combination with an eye-tracking system 10 comprises a camera unit 12 and a processing unit 14, the processing unit 14 being electrically connected to the camera unit 12. In one embodiment, the camera unit 12 can be a front-facing camera, a camera connected externally via USB, a camera built into a laptop, or another image sensor device for capturing visible light. The camera unit 12 can use CMOS or CCD image sensors and have standard functions such as automatic exposure, autofocus, or face detection.In one embodiment, the processing unit 14 can be any electronic processing hardware with computing capacity, including but not limited to CPU, GPU, digital signal processor (DSP), SoC, edge computing chip and other logical computing units capable of executing program instructions, and can be combined with dynamic memory, flash memory or other storage media.

[0021] The camera unit 12 is used to capture a user's facial image, including head position, rotation angle, and eye image. The processing unit 14 is used to process the image data and analyze reading behavior and comprises an image processing module 142, a reading behavior analysis module 144, an article focus data module 146, and a comprehension evaluation module 148. The image processing module 142 is connected to the reading behavior analysis module 144, and the reading behavior analysis module 144 and the article focus data module 146 are connected to the comprehension evaluation module 148. The structure and function of the individual components are explained in more detail below.

[0022] The image processing module 142 is used to process the facial images captured by the camera unit 12 and to analyze multiple eye movement data of a user using an eye-tracking system (not shown), including gaze target position, eye movement path and gaze dwell time, wherein the image processing module 142 first detects the face and eye area, localizes the user's face area and eye position using facial feature detection algorithms, face boundary frame detection models or other image processing techniques, then captures eye position and pupil features in the eye area to determine features such as pupil center, positions of scleral and corneal reflection and upper and lower eyelid margins.The image processing module 142 estimates the relative gaze direction based on the pupil's position relative to orbital or facial feature points to calculate the user's current gaze target position. The image processing module 142 links successive gaze target positions into an eye movement path to record reading behaviors such as saccades, flashbacks, and line skips. Gaze dwell time is determined by measuring the duration the gaze target remains within a specific area to assess fixation behavior, and this dwell time is recorded.

[0023] The reading behavior analysis module 144 serves to generate several reading behavior indicators based on a user's eye-tracking data analyzed by the image processing module 142, in order to reflect the user's behavioral characteristics during reading. These reading behavior indicators include eye-tracking time, total saccade path, direction of movement, ratio of forward to return path, dwell pattern, logging of skipped reading, direction of visual trajectory, distribution of article focal points where the gaze lingers, and dwell duration statistics.In one embodiment, the reading behavior analysis module 144 can perform saccadic behavior, fixation behavior, skimming, flashbacks, line-following ability, dwell patterns, logging of skipping, fixation dispersion, off-screen and defocusing behavior, correspondence of visual trajectories with text content, and the integration of gaze behavior data and pattern recognition. Saccadic behavior analysis can detect rapid eye movements between different positions, determining the number, amplitude, and speed of saccades and the total saccade path to evaluate visual search efficiency during reading. Fixation behavior analysis can determine the duration and frequency of gaze dwells on a text area to reflect the user's attentional state.By analyzing skimming, the proportion of text not captured by the gaze (i.e., skipped over and not read intensively by the user) can be identified to determine the degree of skimming. Through backtracking analysis, the reversal of eye movements back to previous paragraphs or text sections can be detected, calculating the backtracking count, backtracking ratio, or the ratio of forward to backward movement paths to reflect potential comprehension problems during reading. Line-tracking ability analysis can determine whether the user's gaze lands correctly on the new line after a line break, evaluating line-turn accuracy and the user's visual tracking ability.Analyzing dwell patterns, based on the positions of gaze pauses and quickly scanned text areas, reveals whether the user pauses or skips text, reflecting their reading habits, preferred reading speed, and attentional distribution. Analyzing fixation dispersion, based on the distribution of fixation points within the article, determines whether the user's attention remains consistent with the article's content structure, calculating a fixation dispersion indicator. Analyzing off-screen and defocus behavior reveals whether the gaze wanders from the screen or is not focused on specific text areas, indicating that the reading gaze does not follow expected patterns. The off-screen rate or defocus rate serves as an auxiliary indicator of reading stability or attentional state.The analysis of the correspondence between visual trajectories and text content is performed by correlating eye movement paths with key text sections of an article. This allows the system to recognize whether the user is focusing on important paragraphs, logging the time spent looking at these key areas. By integrating eye-tracking data and pattern recognition, the aforementioned behavioral indicators can be aggregated into a set of visual behavioral features, which is then used for subsequent evaluation of comprehension or identification of reading strategies. Through these analyses, the reading behavior analysis module 144 can generate multiple reading behavior indicators.To comprehensively reflect the visual and behavioral characteristics of a user during reading, the reading behavior analysis module includes at least one of the following features: eye movement time, total saccade path, direction of movement, ratio of forward to backward movement path, dwell pattern, logging of skipping reading, direction of visual trajectory, distribution of article focal points where the gaze lingers, dwell time statistics, look-back rate, skimming rate, deviation of line start meeting point, fixation dispersion, distribution of forward and backward trajectories, off-screen rate, and defocus rate; however, the present invention is not subject to any limitations in this respect.

[0024] The following explains the basic formulas for the indicators mentioned above:

[0025] In one embodiment, the reading behavior analysis module 144 performs quantitative calculations of several reading behavior indicators based on the gaze comparison data from the eye movement and time series data acquired by eye tracking (including fixation coordinates, timestamps, and validity markers). Examples include: 1. Saccade: Defined as the eye movement between two successive fixation points where the gaze displacement distance exceeds a predetermined angular threshold (e.g., 1° visual angle) and the duration is below a first time threshold (e.g., 60 ms). The saccade length can be calculated as follows: S=√((x2−x1)2+(y2−y1)2)

[0026] Saccade length and saccade frequency can be determined statistically.

[0027] 2. Fixation: Refers to the gaze remaining on the same block of text, where the displacement is below a distance threshold and the duration is within a second time threshold (e.g., 100 to 500 ms). The fixation duration can be defined as: F=∑Δti where Δt i represents the successive valid sampling times.

[0028] 3. Skimming: This behavior is characterized by low average fixation time per unit of time, long saccade lengths, and insufficient text coverage. The skimming index can be calculated as follows: Sk = (average saccade length ÷ average fixation duration) × weighting of text coverage

[0029] 4. Retrospective: Defined as the movement of the gaze from later text sections to earlier text sections. The relationship can be represented as: R = Number of regressive saccades ÷ Total number of saccades

[0030] 5. Line tracking capability: This is calculated based on the consistency and success rate of the saccade direction from the end of one line to the beginning of the next. If the saccade angle is outside the expected line direction, this is recorded as a line error.

[0031] 6. Fixation dispersion: This can be reflected by the standard deviation of the fixation points or the convex hull surface to determine whether the gaze is focused on semantically relevant areas: D=√(σx2+σy2)

[0032] 7. Using the off-screen time fraction, the number of defocus events, and the correspondence between gaze and semantic text blocks, a multidimensional reading vector is created, which is used for subsequent pattern recognition and ability assessment.

[0033] The article focus data module 146 serves to automatically analyze the semantic structure, text distribution, or paragraph hierarchy of article content based on its characteristics, in order to create several text areas and their associated weightings of comprehension quality. Furthermore, the article focus data module 146 first analyzes the entered article content, segmenting it into paragraphs, sentences, or words to define basic text areas. Subsequently, the article focus data module 146 analyzes semantic features and determines the relative importance of paragraphs, sentences, or words based on semantic relevance, word features, grammatical structure, or semantic distribution of the article content. Depending on the topic, type of knowledge (e.g.,Based on the narrative, explanatory, or argumentative writing style or content structure of an article, the article focus data module 146 automatically generates a comprehension quality weighting for each text area. Important sentences, key paragraphs, or main concepts of the article are marked as key areas and receive a higher comprehension quality weighting.

[0034] The Comprehension Assessment Module 148 serves to weight the reading behavior indicators analyzed by the Reading Behavior Analysis Module 144, taking into account the weightings of comprehension quality assigned by the Article Focus Data Module 146, in order to assess the user's level of understanding with regard to the article content. The level of understanding encompasses comprehension quality and reading accuracy, with reading accuracy including reading correctness, reading stability, and reading efficiency, in order to assess a user's reading behavior and comprehension quality. First, the eye dwell time on individual text areas is correlated with the comprehension quality weightings generated by the Article Focus Data Module to determine whether the gaze was directed at important text areas, recording the dwell time and dwell ratio.Subsequently, the Comprehension Assessment Module 148 statistically evaluates the user's level of comprehension based on dwell time data, eye movement paths, and center of gravity weightings, without requiring a specific mathematical formula. Furthermore, the Comprehension Assessment Module 148 can draw on additional information from the Reading Behavior Analysis Module 144, such as skimming rate, retrospective behavior, fixation dispersion, and line-following ability, which serve as reference indicators for assessing the level of comprehension.

[0035] It will be on Fig.Reference is made to Figure 2, which shows a block diagram of another embodiment of the present invention. To improve the accuracy of eye tracking, the processing unit 14 further comprises a posture correction module 143, which is connected to the image processing module 142 and the reading behavior analysis module 144 and creates a head posture frame and, upon detection of a user head deviation exceeding the area of ​​the head posture frame, adjusts or discards the eye movement data acquired during this period in order to correct recognition errors caused by the user's gaze deviations. In one embodiment, the posture correction module 143 analyzes image data using facial features or facial recognition models to estimate head posture parameters such as head pitch, yaw, and roll.Based on the user's normal sitting or starting posture while reading, a predefined head posture frame is created, which serves as a reference for detecting head deviations. If the user's head angle deviates from the predefined range of the head posture frame, a deviation condition is detected, which can lead to inaccurate calculations of the gaze target position. Therefore, if a deviation condition is detected, the posture correction module 143 can reduce the reliability of the eye-tracking data for that period or exclude the data for that period from interpretation to prevent misinterpretation of eye tracking caused by head movements.

[0036] In summary, the three modules described in the present invention primarily serve the following purposes:

[0037] The posture correction module 143 primarily serves to correct gaze deviations caused by a user's head movements in real time. Based on the relative positions of key facial points (e.g., eyes, nose tip, corners of the mouth), this module calculates head tilt, yaw angle, and roll angle and creates a posture compensation matrix to convert the original eye movement coordinates into standardized gaze coordinates relative to the screen area. This design is suitable for non-restrictive reading situations where the user can move their head naturally.

[0038] The image processing module 142 receives the image data captured by the camera, performs eye area localization, pupil center detection, reflection point detection, and noise filtering, and converts the results into continuous gaze vectors and time-series data. This module can also integrate posture correction results to correct eye movement features in the image in real time. This avoids tracking inaccuracies caused by changing lighting conditions or viewing angles.

[0039] The Reading Behavior Analysis Module 144 uses calibrated eye-tracking data and compares it with user-selected text areas to analyze saccadic, fixation, and review behavior. By integrating reading behavior characteristics and pattern recognition, it generates indicators of reading comprehension. In application scenarios, this module can operate in real time during digital reading exercises or reading assessments, providing results to backend systems or teaching interfaces for learning analysis and skills feedback.

[0040] The processing unit 14 may further include a reporting module 149, which is connected to the comprehension assessment module 148 to output the level of comprehension determined by the comprehension assessment module 148 in at least one visualization report on reading behavior. This visualization report may include a fixation heatmap, saccade path diagram, reading radar diagram, comprehension curve, or other results of reading diagnostics. The reading behavior analysis module or the reporting module also creates a learning history or a long-term comprehension curve based on multiple reading data sets.

[0041] In summary, the system according to the invention for evaluating reading comprehension behavior in combination with an eye-tracking system has the following technical advantages: 1. Non-invasive eye tracking is performed using the front-facing camera or an externally connected camera of an electronic consumer device. This eliminates the need for special infrared eye trackers to capture eye images and eye movement data, significantly reducing the costs and implementation effort of eye tracking systems; 2. The image processing module can accurately estimate projection positions and eye movement paths. The posture correction module can eliminate errors caused by head deviations, thus enabling stable tracking quality in natural body postures and improving the accuracy of eye movement data in natural reading situations. 3. By using reading behavior indicators such as saccades, fixation, skimming, looking back, skipping, line tracking, fixation dispersion, and off-screen rate, the system can create a complete profile of reading behavior, rather than simply showing where the gaze is directed. By comparing eye movement paths with important text areas, the system can estimate, based on gaze dwell time, whether the reader is focusing on relevant paragraphs, thus increasing the accuracy of the reading comprehension assessment. 4. The reports can be divided into versions for students, parents, and teachers, each containing informative visual summaries and suggestions for improvement. Based on various tracking results at different stages, the system can generate learning curves and risk alerts to enable individualized diagnoses and long-term monitoring in education.

[0042] The foregoing description represents only preferred embodiments of the present invention and is not intended to limit the scope of protection of the present invention. All equivalent changes and modifications that correspond to the features described in the claims and the spirit of the present invention are within the scope of protection of the present invention. Reference symbol list 10. System for evaluating reading comprehension behavior in combination with an eye-tracking system 11 electronic device 12 camera units 14 processing units 142 Image processing module 143 Posture Correction Module 144 Reading Behavior Analysis Module 146 Article focus data module 148 Comprehension Assessment Module 149 Reporting module

Images