Intelligent reading control system
By collecting multimodal data and using deep learning models to assess user status, dynamically generating adaptive difficulty questions and personalized recommendations, the problem of mismatch between learning content and needs in existing intelligent reading control systems is solved, thereby improving learning efficiency and focus.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HENAN ZHONGXIANG EDUCATION TECHNOLOGY CO LTD
- Filing Date
- 2025-12-24
- Publication Date
- 2026-04-24
AI Technical Summary
Existing intelligent reading control systems struggle to dynamically adapt to users' real-time knowledge status and cognitive pace, suffer from weak feedback mechanisms, mismatch between learning content and needs, low learning efficiency, and difficulty in maintaining user focus.
By collecting multimodal data, including eye-tracking, voice interaction, body posture, and biometrics, behavioral cognitive feature vectors are generated. These vectors are then combined with deep learning models to assess reading comprehension, focus, and emotional state, dynamically generating adaptive difficulty questions and providing personalized recommendations and real-time intervention.
It enables real-time monitoring and immediate intervention of users' learning status, dynamically matches learning content with needs, improves learning efficiency and focus, and optimizes learning outcomes through multi-channel feedback.
Smart Images
Figure CN121919787A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent reading technology, and in particular to an intelligent reading control system. Background Technology
[0002] Intelligent reading control systems analyze reader behavior and text characteristics to dynamically adjust reading speed, content filtering, and information presentation, representing a revolution in knowledge management in the digital age. They automatically extract key information and generate personalized summaries, helping readers accurately pinpoint valuable content amidst the information deluge and improving cognitive efficiency. Through speech synthesis, difficulty adjustment, and real-time multilingual translation, they remove reading barriers for children, the elderly, visually impaired individuals, and non-native speakers, promoting knowledge accessibility. By analyzing eye-tracking and dwell time, they identify areas of difficulty for readers, dynamically inserting background annotations or visual charts to facilitate knowledge internalization, shifting reading from passive reception to active interaction and providing sustainable momentum for the knowledge evolution of individuals and society.
[0003] Currently, existing intelligent reading control systems typically rely on preset templates and uniform paths, making it difficult to dynamically adapt to students' real-time knowledge status and cognitive pace. This leads to a mismatch between learning content and actual needs. At the same time, the feedback mechanism is weak, often only providing right or wrong judgments without in-depth analysis and targeted guidance, making it difficult to internalize knowledge. In addition, system design often neglects learning motivation and emotional experience, and the monotonous interaction makes it difficult to maintain focus, resulting in low learning efficiency for users.
[0004] Therefore, there is an urgent need for an intelligent reading control system with high user learning efficiency. Summary of the Invention
[0005] This invention provides an intelligent reading control system. The system includes:
[0006] The acquisition module is used to collect multimodal raw data in real time during the user's reading process. The multimodal raw data includes eye movement trajectory data, voice interaction data, body posture data, and biometric data.
[0007] An extraction module, connected to the acquisition module, is used to clean and standardize the multimodal raw data, extract the deep features of the multimodal raw data, integrate heterogeneous features through a multimodal fusion network to generate behavioral cognitive feature vectors, and calculate feature fusion coefficients.
[0008] An evaluation module, connected to the extraction module, is used to input the behavioral cognitive feature vector into a pre-trained deep learning model to evaluate the user's reading comprehension level, reading fluency, vocabulary mastery, concentration, and emotional state, and output the evaluation results and evaluation confidence.
[0009] An adaptation module, connected to the evaluation module, is used to perform deep semantic analysis on the reading material and construct a knowledge graph, and calculate the semantic association strength; based on the evaluation results, evaluation confidence and semantic association strength, it generates multi-type and multi-level adaptive difficulty questions.
[0010] The recommendation module, connected to the evaluation module, is used to construct a user capability profile based on the evaluation results, combine user interests and preferences with knowledge graph coverage, calculate interest matching degree, and recommend personalized books, learning tasks and supplementary materials.
[0011] The intervention module is connected to the evaluation module and the recommendation module respectively. It is used to monitor the key state parameters in the evaluation results in real time, determine whether intervention is needed based on the preset intervention trigger threshold, and if intervention is needed, it will perform real-time intervention through tactile feedback, voice prompts or display parameter adjustments, and record the intervention effect data.
[0012] Further, the extraction of deep features from the multimodal raw data includes:
[0013] Eye movement features are extracted from eye movement trajectory data, including fixation duration, scan rate, and pupil change amplitude.
[0014] Extract speech features from voice interaction data, including speech rate, pause interval, and pronunciation accuracy.
[0015] Posture features are extracted from limb posture data, including sitting stability, page turning frequency, and hand gestures.
[0016] Physiological features are extracted from biometric data, including heart rate variability and skin conductance.
[0017] The multimodal fusion network integrates the deep features of each modality through a weighted summation method. The feature fusion coefficient is calculated based on the signal-to-noise ratio of each modality data. The value range of the feature fusion coefficient is 0-1. The higher the signal-to-noise ratio, the larger the corresponding feature fusion coefficient. The dimension of the behavioral cognition feature vector is consistent with the total dimension of the deep features of each modality.
[0018] Furthermore:
[0019] The pre-trained deep learning model includes a feature mapping layer, a classification prediction layer, and a confidence calculation layer.
[0020] The classification prediction layer outputs preliminary scores for each assessment dimension, which includes reading comprehension level score, reading fluency score, vocabulary mastery score, concentration score, and emotional state score.
[0021] The confidence calculation layer calculates the evaluation confidence based on the prediction error of each evaluation dimension and the completeness of the original multimodal data. The evaluation confidence ranges from 0.5 to 1.0. The smaller the prediction error and the higher the data completeness, the greater the evaluation confidence.
[0022] Furthermore, the process of constructing the knowledge graph by the adaptation module includes:
[0023] The reading material is segmented into words, identified as entities, and its relationships are extracted to determine the core knowledge points and the relationships between them.
[0024] Calculate the semantic association strength between adjacent knowledge points. The semantic association strength is determined based on the co-occurrence frequency of knowledge points, semantic similarity, and logical dependency, and the value range is 0-1.
[0025] A hierarchical knowledge graph is constructed by using core knowledge points and semantic association strength as node and edge weights. The knowledge graph includes a basic knowledge point layer, an applied knowledge point layer, and an extended knowledge point layer.
[0026] Furthermore, the process by which the adaptation module generates adaptive difficulty questions includes:
[0027] If the assessment confidence level is greater than or equal to the first confidence threshold, then based on the reading comprehension level score and semantic association strength, inference, critical and multimodal questions are generated;
[0028] If the assessment confidence level is less than the first confidence threshold but greater than or equal to the second confidence threshold, then factual and general questions are generated based on the vocabulary mastery score and the semantic association of the basic knowledge point layer.
[0029] If the assessment confidence level is less than the second confidence threshold, then basic memory-based questions are generated; the first confidence threshold is greater than the second confidence threshold.
[0030] Furthermore, the recommendation process of the recommendation module includes:
[0031] Based on the scores of each dimension in the evaluation results and the user's historical reading records, the user capability profile is constructed, which includes capability weakness dimension, strength dimension and interest tendency tag;
[0032] Calculate the interest matching degree between recommended books and the user's ability profile. The interest matching degree is determined based on the fit between the book's theme and interest tendency tags, and the complementarity between the book's knowledge points and ability shortcomings.
[0033] Calculate the knowledge graph coverage, which is the proportion of knowledge points covered by the recommended content to knowledge points that the user has not mastered;
[0034] If the interest matching degree is greater than or equal to the matching degree threshold and the knowledge graph coverage is greater than or equal to the coverage threshold, then the book and its accompanying learning tasks will be recommended first, and the frequency of recommendation is positively correlated with the interest matching degree.
[0035] Furthermore, the intervention process of the intervention module includes:
[0036] The focus score and reading fluency score in the evaluation results are monitored in real time, and the focus score is compared with the focus threshold, and the reading fluency score is compared with the fluency threshold.
[0037] If the focus score is less than the focus threshold and the duration is greater than the preset duration, or if the reading fluency score is less than the fluency threshold and the pronunciation error rate is greater than the error rate threshold, then it is determined that real-time intervention is required.
[0038] The intervention trigger thresholds include the focus threshold, fluency threshold, and error rate threshold, and the choice of intervention method is determined based on the emotional state score. If the emotional state score is negative, voice prompts are used as the preferred intervention method.
[0039] Furthermore, the calculation process of the feature fusion coefficients includes:
[0040] Calculate the signal-to-noise ratio (SNR) of the raw data for each mode, where the SNR is the ratio of the effective signal strength to the noise signal strength for each mode.
[0041] The signal-to-noise ratio of each mode is normalized to obtain the normalized signal-to-noise ratio value;
[0042] Based on the normalized signal-to-noise ratio value, the feature fusion coefficient is calculated using a weighted average method. The feature fusion coefficient is used to adjust the weights of each modality's deep features during the integration process, so as to optimize the representation accuracy of the behavioral cognitive feature vector.
[0043] Furthermore, the calculation process for the assessment confidence level includes:
[0044] Calculate the prediction error of the preliminary score for each evaluation dimension, where the prediction error is the average of the absolute differences between the model prediction value and the manually labeled value;
[0045] Calculate the integrity of the multimodal raw data, where integrity is the proportion of valid data to the total collected data.
[0046] The prediction error is reverse normalized to obtain the error correction coefficient, and the evaluation confidence level is the product of the error correction coefficient and the data integrity.
[0047] Furthermore, the dynamic adjustment process of the intervention trigger threshold includes:
[0048] Record the effect feedback data after each intervention, including the improvement in focus, the degree of fluency recovery, and user feedback ratings after the intervention.
[0049] Calculate the comprehensive intervention effect index, which is a weighted sum of the improvement, recovery degree, and user feedback rating;
[0050] If the comprehensive index of the intervention effect is greater than the effect threshold, the corresponding intervention trigger threshold will be lowered by a preset ratio;
[0051] If the comprehensive index of the intervention effect is less than or equal to the effect threshold, the corresponding intervention trigger threshold will be increased by a preset ratio.
[0052] The intervention trigger threshold is adjusted to 50%-150% of the initial threshold to ensure the rationality of the intervention timing and compatibility with the user experience.
[0053] Compared with existing technologies, the beneficial effects of this invention are as follows: By collecting multimodal data in real time, changes in learning status can be detected in a timely manner, providing immediate intervention and avoiding ineffective learning; based on user ability assessment, interest preferences, and knowledge graphs, learning content and questions are dynamically recommended to match individual levels and needs; by comprehensively considering multiple indicators such as reading comprehension, concentration, and emotion, weak links can be accurately identified, and learning strategies can be adjusted in a targeted manner; adaptive questions are dynamically generated based on assessment results to maintain the challenge and achievability of learning tasks and promote effective learning; through real-time intervention via multiple channels such as touch, voice, and display, users can adjust their state in a timely manner, improving learning concentration and efficiency; and the intervention effect and learning data are continuously recorded to iteratively optimize the model and recommendation strategy, thereby improving the user's learning efficiency.
[0054] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description
[0055] Figure 1 A block diagram of an intelligent reading control system according to an embodiment of the present invention is shown. Detailed Implementation
[0056] Figure 1 A block diagram of an intelligent reading control system according to an embodiment of the present invention is shown. The system includes:
[0057] The acquisition module 101 is used to acquire multimodal raw data in real time during the user's reading process. The multimodal raw data includes eye movement trajectory data, voice interaction data, body posture data and biometric data.
[0058] The extraction module 102 is connected to the acquisition module and is used to clean and standardize the multimodal raw data, extract the deep features of the multimodal raw data, integrate heterogeneous features through a multimodal fusion network to generate behavioral cognitive feature vectors, and calculate feature fusion coefficients.
[0059] The evaluation module 103, connected to the extraction module, is used to input the behavioral cognitive feature vector into a pre-trained deep learning model to evaluate the user's reading comprehension level, reading fluency, vocabulary mastery, concentration and emotional state, and output the evaluation results and evaluation confidence.
[0060] The adaptation module 104, which is connected to the evaluation module, is used to perform deep semantic analysis on the reading material and construct a knowledge graph, and calculate the semantic association strength; based on the evaluation results, evaluation confidence and semantic association strength, it generates multi-type and multi-level adaptive difficulty questions.
[0061] Recommendation module 105 is connected to the evaluation module and is used to construct a user ability profile based on the evaluation results, combine user interests and preferences and knowledge graph coverage, calculate interest matching degree, and recommend personalized books, learning tasks and supplementary materials.
[0062] The intervention module 106 is connected to the evaluation module and the recommendation module respectively. It is used to monitor the key state parameters in the evaluation results in real time, determine whether intervention is needed based on the preset intervention trigger threshold, and if intervention is needed, it will perform real-time intervention through tactile feedback, voice prompts or display parameter adjustments, and record the intervention effect data.
[0063] In some embodiments, the extraction of depth features from the multimodal raw data includes: extracting eye movement features from eye movement trajectory data, the eye movement features including fixation duration, scan rate, and pupil change amplitude; extracting speech features from voice interaction data, the speech features including speech rate, pause interval, and pronunciation accuracy; extracting posture features from limb posture data, the posture features including sitting stability, page turning frequency, and gestures; and extracting physiological features from biometric data, the physiological features including heart rate variability and skin conductance response. The multimodal fusion network integrates the depth features of each modality through a weighted summation method. The feature fusion coefficient is calculated based on the signal-to-noise ratio of each modality data. The value range of the feature fusion coefficient is 0-1, and the higher the signal-to-noise ratio, the larger the corresponding feature fusion coefficient. The dimension of the behavioral cognitive feature vector is consistent with the total dimension of the depth features of each modality. According to embodiments of the present invention, learning status is monitored in real time from multiple dimensions to comprehensively assess attention, cognitive load, and emotional state; objective indicators are quantified to reduce subjective reporting bias; high-quality modal data is highlighted through weighted fusion to improve the reliability of feature representation; fine-grained data captures micro-behavioral changes during the learning process, facilitating timely adjustment of learning strategies and thereby improving the user's learning efficiency.
[0064] For example, in eye movement feature extraction, for a 30-second eye movement trajectory data, the calculated fixation duration is 285 milliseconds per fixation, the saccade rate is 4.2 effective saccades per second, and the pupil dilation amplitude is an average of 0.4 mm relative to the baseline value. In speech feature extraction, for a 10-second reading speech data, the calculated speech rate is 3.5 Chinese characters per second, the pause interval is an average of one pause exceeding 0.5 seconds every 2.1 seconds, and the pronunciation accuracy is [percentage missing] for 50 target Chinese characters, with correct pronunciation. The proportion was 94%; in posture feature extraction, for a 2-minute period of limb posture data, the sitting stability was calculated as a swaying frequency of more than 10 cm of body center of gravity shifting by 12 times per minute, the page-turning frequency was an average of once every 45 seconds, and the hand gestures were recorded as 5 specific hand gestures for raising hands to ask questions; in physiological feature extraction, for a 1-minute period of skin conductance response data, the average skin conductance level was calculated as 5.5 microSiemens; for synchronized heart rate data, the heart rate variability was calculated as 42 milliseconds.
[0065] In some embodiments, the pre-trained deep learning model includes a feature mapping layer, a classification prediction layer, and a confidence calculation layer. The classification prediction layer outputs preliminary scores for each evaluation dimension, including reading comprehension level score, reading fluency score, vocabulary mastery score, focus score, and emotional state score. The confidence calculation layer calculates the evaluation confidence based on the prediction error of each evaluation dimension and the completeness of the multimodal raw data. The evaluation confidence ranges from 0.5 to 1.0; the smaller the prediction error and the higher the data completeness, the greater the evaluation confidence. According to embodiments of the present invention, multi-dimensional evaluation provides comprehensive learning status feedback, helping users accurately identify weaknesses. Confidence calculation, combining prediction error and data completeness, improves the reliability of the scores, making users more confident in the evaluation results. Quantitative indicators intuitively present the credibility of the evaluation, assisting users in prioritizing high-confidence scores. Evaluation based on the completeness of multimodal data reduces misjudgments caused by missing data, ensures the relevance of suggestions, and thus improves the user's learning efficiency.
[0066] In some embodiments, the process of constructing a knowledge graph by the adaptation module includes: segmenting the reading material into words, recognizing entities, and extracting relationships to determine core knowledge points and the relationships between them; calculating the semantic association strength between adjacent knowledge points, whereby the semantic association strength is determined based on the co-occurrence frequency of knowledge points, semantic similarity, and logical dependencies, with a value range of 0-1; and constructing a hierarchical knowledge graph by using the core knowledge points and semantic association strength as node and edge weights, whereby the knowledge graph includes a basic knowledge point layer, an applied knowledge point layer, and an extended knowledge point layer. According to embodiments of the present invention, by presenting a structured knowledge system, it helps users establish a systematic cognitive framework; it reveals the relationships between knowledge points, highlighting logical connections through semantic association strength to promote understanding and memory; it organizes content hierarchically, conforming to a gradual learning pattern; it quantifies association strength to assist in priority judgment and focus; and it automates extraction and association, reducing manual processing costs and quickly generating personalized knowledge graphs, thereby improving users' learning efficiency.
[0067] For example, taking middle school mathematics reading materials related to "linear functions" as an example, the process of constructing a knowledge graph using the adaptation module is as follows: After word segmentation, entity recognition, and relation extraction, 12 core knowledge points are identified, including 4 basic knowledge points (variable definition, constant definition, basic concepts of functions, and direct proportion), 5 applied knowledge points (linear function expression, range of independent variables, function graph drawing, function value calculation, and simple practical problem modeling), and 3 extended knowledge points (the connection between linear functions and equations, the connection between linear functions and inequalities, and comprehensive application of multivariable linear functions). The semantic association strength between adjacent knowledge points is calculated: the co-occurrence frequency of variable definition and constant definition is 8 times / thousand characters, the semantic similarity is 0.72, the logical dependency is parallel, and the semantic association strength is 0.68; the co-occurrence frequency of linear function expression and function graph drawing is... The semantic similarity between the basic knowledge points and linear function expressions is 0.85, with a causal logical dependency and a semantic association strength of 0.83. The co-occurrence frequency of the basic knowledge points and linear function expressions is 10 times per thousand words, with a semantic similarity of 0.78, a progressive logical dependency, and a semantic association strength of 0.75. The co-occurrence frequency of the association between linear functions and equations and the comprehensive application of multivariate linear functions is 5 times per thousand words, with a semantic similarity of 0.63, an extended logical dependency, and a semantic association strength of 0.52. When constructing a hierarchical knowledge graph, 12 core knowledge points are used as nodes, and the above semantic association strengths are used as corresponding edge weights, forming a three-layer structure. The average edge weight between the four nodes in the basic knowledge point layer is 0.65, the average edge weight between the basic knowledge point layer and the applied knowledge point layer is 0.73, and the average edge weight between the applied knowledge point layer and the extended knowledge point layer is 0.58.
[0068] In some embodiments, the process of the adaptation module generating adaptive difficulty questions includes: if the assessment confidence level is greater than or equal to a first confidence threshold, then generating inference, critical, and multimodal questions based on the reading comprehension level score and semantic association strength; if the assessment confidence level is less than the first confidence threshold but greater than or equal to a second confidence threshold, then generating factual and general questions based on the vocabulary mastery score and semantic association at the basic knowledge point level; if the assessment confidence level is less than the second confidence threshold, generating basic memory-based questions; the first confidence threshold is greater than the second confidence threshold. According to embodiments of the present invention, by matching question types according to assessment confidence level, questions are avoided from being too difficult or too easy; corresponding question types are generated for users' weaknesses (such as vocabulary and reasoning ability) to accurately fill ability gaps; confidence level is positively correlated with question difficulty, and semantic association strength is negatively correlated with difficulty, ensuring smooth changes in difficulty; factual and memory-based questions are provided to consolidate the foundation at low confidence levels, while increasing thinking challenges at high confidence levels, adapting to different learning stages, thereby improving users' learning efficiency.
[0069] For example, the first confidence threshold is set to 0.8, and the second confidence threshold is set to 0.6; the difficulty coefficient is calculated as: Difficulty Coefficient = Assessment Confidence × 1.2 - Semantic Relevance Strength × 0.3; when the assessment confidence is 0.85 (≥ 0.8), the reading comprehension score is 85, and the semantic relevance strength is 0.6, 5 questions are generated, including 2 inference questions, 2 critical questions, and 1 multimodal question, with difficulty coefficients of 0.83, 0.85, 0.82, 0.86, and 0.84 respectively; when the assessment confidence is 0.72 ( When the confidence level is 0.6 ≤ 0.72 < 0.8, the vocabulary mastery score is 70, and the semantic association strength of the basic knowledge point layer is 0.75, 4 questions are generated, including 3 factual questions and 1 summary question, with difficulty coefficients of 0.63, 0.64, 0.62, and 0.65 respectively; when the evaluation confidence level is 0.55 (< 0.6), and the semantic association strength of the basic knowledge point layer is 0.8, 6 basic memory questions are generated, with difficulty coefficients of 0.41, 0.43, 0.40, 0.42, 0.44, and 0.43 respectively.
[0070] In some embodiments, the recommendation process of the recommendation module includes: constructing a user capability profile based on the scores of each dimension in the evaluation results and the user's historical reading records. The user capability profile includes capability weakness dimensions, strength dimensions, and interest tendency tags; calculating the interest matching degree between recommended books and the user capability profile, the interest matching degree being determined based on the fit between the book's theme and interest tendency tags, and the complementarity between the book's knowledge points and capability weakness dimensions; calculating the knowledge graph coverage, the knowledge graph coverage being the proportion of knowledge points covered by the recommended content to knowledge points that the user has not mastered; if the interest matching degree is greater than or equal to the matching degree threshold and the knowledge graph coverage is greater than or equal to the coverage threshold, then the book and its accompanying learning tasks are recommended first, and the frequency of recommendation is positively correlated with the interest matching degree. According to the embodiments of the present invention, by accurately matching interests, learning motivation and focus are improved; knowledge points are supplemented for capability weaknesses to achieve personalized gap filling; content with high coverage of unmastered knowledge is prioritized to improve time utilization efficiency; learning tasks and capability development are dynamically linked to strengthen the systematic nature of learning; and recommendation quality and adaptability are ensured through threshold screening and frequency adjustment to reduce information overload, thereby improving the user's learning efficiency.
[0071] For example, the user's reading comprehension score is 75, reading fluency is 88, vocabulary mastery is 62, focus is 80, and emotional state is 85. Their reading history over the past three months includes 7 books read: 5 in mathematics (3 related to linear functions and 2 related to geometry) and 2 in physics; no reading records in Chinese or English. They have a list of 4 knowledge points they haven't mastered: the connection between linear functions and equations, comprehensive application of multivariable linear functions, 10 core vocabulary words from junior high school, and problem-solving techniques for function reading comprehension questions. The matching threshold is set at 0.6, and the coverage threshold at 0.5. The user's ability profile shows weaknesses in the following dimensions: vocabulary mastery (62, below the 70 threshold), reading comprehension (75, below the 80 threshold), reading fluency (88, above the 85 threshold), and emotional state (85, equal to the 85 threshold). Interest tags: mathematics (interest weight 0.7), physics (interest weight 0.3).
[0072] Book Title Theme relevance Knowledge complementarity Interest matching degree (= theme fit degree × 0.6 + knowledge point complementarity degree × 0.4) Knowledge graph coverage (= number of unmastered knowledge points covered / 4) Linear Functions and Reading Comprehension Enhancement 0.7 0.75 0.7×0.6 + 0.75×0.4 = 0.72 3 / 4 = 0.75 Comprehensive Training on Linear Functions with Multiple Variables 0.7 0.5 0.7×0.6 + 0.5×0.4 = 0.62 2 / 4 = 0.5 "Quick Memorization of Junior High School Vocabulary and Physics Experiments" 0.3 0.5 0.3×0.6 + 0.5×0.4 = 0.38 2 / 4 = 0.5
[0073] Priority books recommended are "Linear Functions and Reading Ability Enhancement" and "Comprehensive Training of Multivariable Linear Functions" (both satisfying interest matching degree ≥ 0.6 and knowledge graph coverage ≥ 0.5); "Linear Functions and Reading Ability Enhancement" (interest matching degree 0.72) is recommended twice a week; "Comprehensive Training of Multivariable Linear Functions" (interest matching degree 0.62) is recommended once a week; each recommended book is paired with 3 corresponding knowledge point practice tasks, and the number of tasks is positively correlated with the knowledge graph coverage.
[0074] In some embodiments, the intervention process of the intervention module includes: real-time monitoring of the focus score and reading fluency score in the evaluation results; comparing the focus score with a focus threshold and the reading fluency score with a fluency threshold; if the focus score is less than the focus threshold and the duration is greater than a preset duration, or the reading fluency score is less than the fluency threshold and the pronunciation error rate is greater than the error rate threshold, then it is determined that real-time intervention is required; the intervention trigger threshold includes the focus threshold, fluency threshold, and error rate threshold, and the selection of the intervention method is determined based on the emotional state score. If the emotional state score is negative, voice prompt intervention is preferred. According to the embodiments of the present invention, real-time monitoring and threshold comparison can immediately detect a decline in learning status and avoid the accumulation of problems; combining multi-dimensional indicators (focus, fluency, error rate) and duration determination reduces misjudgment; selecting the intervention method according to the emotional state (such as using voice prompts when negative) avoids aggravating negative emotions, improves intervention acceptance, and thus improves the user's learning efficiency.
[0075] For example, the following parameters are set: focus threshold 70, fluency threshold 75, error rate threshold 8%, preset duration 60 seconds, emotional state score < 60 (negative), 60 ≤ emotional state score ≤ 80 (neutral), and emotional state score > 80 (positive). When a user's focus score is 63 (lasting 68 seconds) and emotional state score is 57, real-time intervention is deemed necessary because the focus score (63) < the focus threshold (70) and the duration (68 seconds) > the preset duration (60 seconds). Since the emotional state score (57) is negative, voice prompts are used for intervention. The voice prompt volume is 60 decibels, each prompt lasts 1.5 seconds, repeated every 4 seconds, for a total of 3 times. When a user's reading fluency score is 71 (10.2% pronunciation error rate) and emotional state score is 76, real-time intervention is deemed necessary because the reading fluency score is negative. The user's attention score (71 points) was less than the fluency threshold (75 points) and the pronunciation error rate (10.2%) was greater than the error rate threshold (8%), indicating that real-time intervention was required. Since the emotional state score (76 points) was neutral, tactile feedback intervention was used, with a tactile vibration intensity of 0.25N, a vibration frequency of 3Hz, each vibration lasting 0.8 seconds, and a 6-second interval between vibrations, for a total of 2 vibrations. When the user's attention score was 67 points, the reading fluency score was 73 points, the pronunciation error rate was 8.5%, and the emotional state score was 83 points. Since the reading fluency score (73 points) was less than the fluency threshold (75 points) and the pronunciation error rate (8.5%) was greater than the error rate threshold (8%), real-time intervention was required. Since the emotional state score (83 points) was positive, display parameter adjustment intervention was used, increasing the screen brightness by 15% and enlarging the font size by 2, with the adjustment lasting for 30 seconds.
[0076] In some embodiments, the calculation process of the feature fusion coefficient includes: calculating the signal-to-noise ratio (SNR) of the original data for each modality, wherein the SNR is the ratio of the effective signal strength to the noise signal strength of each modality; normalizing the SNR of each modality to obtain a normalized SNR value; and calculating the feature fusion coefficient using a weighted average method based on the normalized SNR value. The feature fusion coefficient is used to adjust the weights of the deep features of each modality in the integration process to optimize the representation accuracy of the behavioral cognitive feature vector. According to embodiments of the present invention, the SNR calculation ensures that high SNR modal data receives higher weights, making the fused features more reliable; the normalization process balances the dimensional differences of the SNR of different modalities, avoiding the dominance of a single modality; the weighted average method dynamically allocates weights based on the SNR, enhancing effective information, suppressing noise, improving the representation accuracy of the feature vector, and thus improving the user's learning efficiency.
[0077] In some embodiments, the calculation process of the assessment confidence score includes: calculating the prediction error of the preliminary scores for each assessment dimension, wherein the prediction error is the average of the absolute differences between the model's predicted values and the manually labeled values; calculating the completeness of the original multimodal data, wherein the completeness is the proportion of effective data to the total collected data; performing inverse normalization on the prediction error to obtain an error correction coefficient, and the assessment confidence score is the product of the error correction coefficient and the data completeness. According to embodiments of the present invention, the accuracy of model scoring is measured by prediction error to ensure the reliability of the assessment basis; high-quality multimodal data is screened by data completeness to improve the effectiveness of input information; error and completeness are integrated into a confidence score to quantify the credibility of the assessment; and the confidence score is used to dynamically adjust the difficulty of questions and recommended content to achieve a personalized learning path that matches the user's actual level, thereby improving the user's learning efficiency.
[0078] In some embodiments, the dynamic adjustment process of the intervention trigger threshold includes: recording effect feedback data after each intervention, the effect feedback data including the improvement in focus, the degree of fluency recovery, and user feedback scores after intervention; calculating a comprehensive intervention effect index, the comprehensive intervention effect index being a weighted sum of the improvement, the degree of recovery, and the user feedback scores; if the comprehensive intervention effect index is greater than the effect threshold, the corresponding intervention trigger threshold is lowered by a preset percentage; if the comprehensive intervention effect index is less than or equal to the effect threshold, the corresponding intervention trigger threshold is increased by a preset percentage; the adjustment range of the intervention trigger threshold is 50%-150% of the initial threshold to ensure the rationality of the intervention timing and the compatibility with the user experience. According to embodiments of the present invention, by dynamically adjusting the trigger threshold based on the actual intervention effect, the intervention timing is made more consistent with the user's current state, avoiding ineffective or frequent interference; combining focus, fluency, and subjective scores, the intervention effect is comprehensively quantified, improving the scientific nature of the adjustment; setting an adjustment range prevents extreme changes in the threshold, balancing the timeliness of intervention and the user experience; lowering the threshold when the effect is good (early intervention), and raising the threshold when the effect is poor (delayed intervention), achieving personalized adaptation through continuous optimization, thereby improving the user's learning efficiency.
[0079] For example, set the focus threshold T1=70 points, fluency threshold T2=75 points, and error rate threshold T3=8%; the effect threshold S=65; the preset adjustment ratio r=10%; the weight allocation of the comprehensive intervention effect index is: focus improvement weight 2.5, fluency recovery weight 2.5, user feedback score weight 0.5; the pre-intervention state data is: focus 62 points (<T1=70 points), fluency 69 points (<T2=75 points), pronunciation error rate 8.5% (>T3=8%), triggering the intervention; the post-intervention effect feedback data is: focus 76 points, fluency 78 points, user feedback score 82 points (out of 100); focus improvement ΔF=76-62=14 points, fluency recovery ΔL=78-69=9 points, comprehensive intervention effect index I=14×2.5 + 9×2.5 + 82×0.5=35+22.5+41=98.5; Since I=98.5>S=65, the thresholds are lowered by 10%, and the new thresholds are T1=70×(1-10%)=63 points, T2=75×(1-10%)=67.5 points, and T3=8%×(1-10%)=7.2%.
[0080] It should be understood that the various processes described above can be used to rearrange, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.
[0081] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.
Claims
1. An intelligent reading control system, characterized in that, include: The acquisition module is used to collect multimodal raw data in real time during the user's reading process. The multimodal raw data includes eye movement trajectory data, voice interaction data, body posture data, and biometric data. An extraction module, connected to the acquisition module, is used to clean and standardize the multimodal raw data, extract the deep features of the multimodal raw data, integrate heterogeneous features through a multimodal fusion network to generate behavioral cognitive feature vectors, and calculate feature fusion coefficients. An evaluation module, connected to the extraction module, is used to input the behavioral cognitive feature vector into a pre-trained deep learning model to evaluate the user's reading comprehension level, reading fluency, vocabulary mastery, concentration, and emotional state, and output the evaluation results and evaluation confidence. An adaptation module, connected to the evaluation module, is used to perform deep semantic analysis on the reading material and construct a knowledge graph, and calculate the semantic association strength; based on the evaluation results, evaluation confidence and semantic association strength, it generates multi-type and multi-level adaptive difficulty questions. The recommendation module, connected to the evaluation module, is used to construct a user capability profile based on the evaluation results, combine user interests and preferences with knowledge graph coverage, calculate interest matching degree, and recommend personalized books, learning tasks and supplementary materials. The intervention module is connected to the evaluation module and the recommendation module respectively. It is used to monitor the key status parameters in the evaluation results in real time, determine whether intervention is needed based on the preset intervention trigger threshold, and if intervention is needed, it will perform real-time intervention through tactile feedback, voice prompts or display parameter adjustments, and record the intervention effect data.
2. The intelligent reading control system according to claim 1, characterized in that, The extraction of deep features from the multimodal raw data includes: Eye movement features are extracted from eye movement trajectory data, including fixation duration, scan rate, and pupil change amplitude. Extract speech features from voice interaction data, including speech rate, pause interval, and pronunciation accuracy. Posture features are extracted from limb posture data, including sitting stability, page turning frequency, and hand gestures. Physiological features are extracted from biometric data, including heart rate variability and skin conductance. The multimodal fusion network integrates the deep features of each modality through a weighted summation method. The feature fusion coefficient is calculated based on the signal-to-noise ratio of each modality data. The value range of the feature fusion coefficient is 0-1. The higher the signal-to-noise ratio, the larger the corresponding feature fusion coefficient. The dimension of the behavioral cognition feature vector is consistent with the total dimension of the deep features of each modality.
3. The intelligent reading control system according to claim 2, characterized in that: The pre-trained deep learning model includes a feature mapping layer, a classification prediction layer, and a confidence calculation layer. The classification prediction layer outputs preliminary scores for each assessment dimension, which includes reading comprehension level score, reading fluency score, vocabulary mastery score, concentration score, and emotional state score. The confidence calculation layer calculates the evaluation confidence based on the prediction error of each evaluation dimension and the integrity of the original multimodal data. The evaluation confidence ranges from 0.5 to 1.
0. The smaller the prediction error and the higher the data integrity, the greater the evaluation confidence.
4. The intelligent reading control system according to claim 3, characterized in that, The process of constructing the knowledge graph by the adaptation module includes: The reading material is segmented into words, identified as entities, and its relationships are extracted to determine the core knowledge points and the relationships between them. Calculate the semantic association strength between adjacent knowledge points. The semantic association strength is determined based on the co-occurrence frequency of knowledge points, semantic similarity, and logical dependency, and the value range is 0-1. A hierarchical knowledge graph is constructed by using core knowledge points and semantic association strength as node and edge weights. The knowledge graph includes a basic knowledge point layer, an applied knowledge point layer, and an extended knowledge point layer.
5. The intelligent reading control system according to claim 4, characterized in that, The process by which the adaptation module generates adaptive difficulty questions includes: If the assessment confidence level is greater than or equal to the first confidence threshold, then based on the reading comprehension level score and semantic association strength, inference, critical and multimodal questions are generated; If the assessment confidence level is less than the first confidence threshold but greater than or equal to the second confidence threshold, then factual and general questions are generated based on the vocabulary mastery score and the semantic association of the basic knowledge point layer. If the assessment confidence level is less than the second confidence threshold, then basic memory-based questions are generated; the first confidence threshold is greater than the second confidence threshold.
6. The intelligent reading control system according to claim 5, characterized in that, The recommendation process of the recommendation module includes: Based on the scores of each dimension in the evaluation results and the user's historical reading records, the user capability profile is constructed, which includes capability weakness dimension, strength dimension and interest tendency tag; Calculate the interest matching degree between the recommended books and the user's ability profile. The interest matching degree is determined based on the fit between the book's theme and interest tendency tags, and the complementarity between the book's knowledge points and ability shortcomings. Calculate the knowledge graph coverage, which is the proportion of knowledge points covered by the recommended content to knowledge points that the user has not mastered; If the interest matching degree is greater than or equal to the matching degree threshold and the knowledge graph coverage is greater than or equal to the coverage threshold, then the book and its accompanying learning tasks will be recommended first, and the frequency of recommendation is positively correlated with the interest matching degree.
7. The intelligent reading control system according to claim 6, characterized in that, The intervention process of the intervention module includes: The focus score and reading fluency score in the evaluation results are monitored in real time, and the focus score is compared with the focus threshold, and the reading fluency score is compared with the fluency threshold. If the focus score is less than the focus threshold and the duration is greater than the preset duration, or if the reading fluency score is less than the fluency threshold and the pronunciation error rate is greater than the error rate threshold, then it is determined that real-time intervention is required. The intervention trigger thresholds include the focus threshold, fluency threshold, and error rate threshold, and the selection of the intervention method is determined based on the emotional state score. If the emotional state score is negative, voice prompts are used as the preferred intervention method.
8. The intelligent reading control system according to claim 7, characterized in that, The calculation process of the feature fusion coefficient includes: Calculate the signal-to-noise ratio (SNR) of the raw data for each mode, where the SNR is the ratio of the effective signal strength to the noise signal strength for each mode. The signal-to-noise ratio of each mode is normalized to obtain the normalized signal-to-noise ratio value; Based on the normalized signal-to-noise ratio value, the feature fusion coefficient is calculated using a weighted average method. The feature fusion coefficient is used to adjust the weights of each modality's deep features during the integration process, so as to optimize the representation accuracy of the behavioral cognitive feature vector.
9. The intelligent reading control system according to claim 8, characterized in that, The calculation process for the assessment confidence level includes: Calculate the prediction error of the preliminary score for each evaluation dimension, where the prediction error is the average of the absolute differences between the model prediction value and the manually labeled value; Calculate the integrity of the multimodal raw data, where integrity is the proportion of valid data to the total collected data. The prediction error is reverse normalized to obtain the error correction coefficient, and the evaluation confidence level is the product of the error correction coefficient and the data integrity.
10. The intelligent reading control system according to claim 9, characterized in that, The dynamic adjustment process of the intervention trigger threshold includes: Record the effect feedback data after each intervention, including the improvement in focus, the degree of fluency recovery, and user feedback ratings after the intervention. Calculate the comprehensive intervention effect index, which is a weighted sum of the improvement, recovery degree, and user feedback rating; If the comprehensive index of the intervention effect is greater than the effect threshold, the corresponding intervention trigger threshold will be lowered by a preset ratio; If the comprehensive index of the intervention effect is less than or equal to the effect threshold, the corresponding intervention trigger threshold will be increased by a preset ratio. The intervention trigger threshold is adjusted within the range of 50%-150% of the initial threshold to ensure the rationality of the intervention timing and compatibility with the user experience.