A terminal based on adaptive learning path planning and resource pushing

Abstract

The application discloses a kind of based on adaptive learning path planning and resource pushing terminal, including workbench, touch interactive screen is installed on the upper end surface of workbench, voice module and projection component, workbench lower end is fixed with machine case, and control cabinet is installed in the inner chamber of machine case, wherein control cabinet includes multimodal data acquisition module, for real-time acquisition learner learning behavior data and at least two kinds of physiological state data, physiological state data includes eye movement data.In the application, by deploying the deep reinforcement learning model compressed by knowledge distillation and mixed precision quantization on the local terminal, the edge intelligent planning module is used to process the multimodal data in real time and generate the learning path, which breaks away from the network delay and bandwidth dependence of traditional cloud architecture.Even in offline environment, the terminal can still respond to the state change of the learner within milliseconds, dynamically adjust the learning content, ensure the continuity and smoothness of the learning process, and significantly improve the real-time performance and reliability of personalized learning.

Description

Technical Field

[0001] This invention relates to the field of intelligent education technology, and in particular to a terminal based on adaptive learning path planning and resource delivery.

[0002] In the field of intelligent education, personalized learning path planning and precise resource delivery are the core elements for improving learners' learning efficiency and experience. Due to significant individual differences among learners, including factors such as knowledge base, cognitive style, attention state, and emotional fluctuations, learning outcomes are directly affected. Traditional teaching often uses fixed course outlines or manual recommendations, which are difficult to adapt to the dynamic changes of learners in real time, resulting in a disconnect between learning content and individual needs, and creating a "one-size-fits-all" teaching bottleneck.

[0003] With the development of online education, cloud-based adaptive learning systems have emerged. These systems collect log data such as learners' answer records and click behaviors, and use recommendation algorithms to generate personalized learning resource lists. However, when applied to high-precision personalized learning scenarios, these systems still face several technical bottlenecks. First, they are highly dependent on network connectivity, cloud processing suffers from latency and cannot operate offline, limiting the ability to adjust learning paths in real time. Second, data collection is limited to a single dimension; relying solely on behavioral logs makes it difficult to capture learners' physiological states, leading to rigid recommendation strategies and an inability to achieve true contextual awareness. Furthermore, the need to upload user learning data to the cloud poses a privacy risk, restricting their widespread application in sensitive educational scenarios. Additionally, existing control logic is mostly based on single-dimensional collaborative filtering or knowledge graph rules, ignoring the strong nonlinear relationship between multimodal physiological characteristics and learning outcomes, making it difficult to accurately track and intervene in learners' cognitive states in real time. Therefore, there is an urgent need for an adaptive learning terminal that can integrate multimodal physiological perception and edge intelligent decision-making while eliminating network dependence and privacy risks. Summary of the Invention

[0004] The purpose of this invention is to provide a terminal based on adaptive learning path planning and resource push to solve the problems of airflow disturbance affecting weighing accuracy and multi-environment parameter coupling control mentioned in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A terminal based on adaptive learning path planning and resource push includes a workbench. A touchscreen, a voice module, and a projection component are mounted on the upper surface of the workbench. A chassis is fixed to the lower end of the workbench, and a control cabinet is installed inside the chassis. The control cabinet includes: A multimodal data acquisition module is used to collect learners' learning behavior data and at least two kinds of physiological state data in real time. The physiological state data includes at least two of the following: eye movement data, heart rate data, skin conductance data, and facial expression data. The local heterogeneous storage module is used to store dynamic knowledge graph libraries, multi-format learning resource libraries, and learner profiles containing learner knowledge mastery, cognitive styles, and historical paths. The edge intelligent planning module is deployed locally on the terminal. It has a built-in deep reinforcement learning model that has undergone knowledge distillation and mixed precision quantization compression. It takes the current learning state vector output by the multimodal data acquisition module as input, outputs the optimal next knowledge point identifier in real time, and dynamically generates a personalized learning path. The context-aware push module is used to identify the learner's real-time attention level and emotional state based on the physiological state information in the current learning state vector, and adjust the presentation format, difficulty level and interaction method of the resources to be pushed accordingly, and retrieve the appropriate resources from the local heterogeneous storage module or from the cloud resource library through the communication module. The privacy-preserving federated learning module is used to accumulate learning and interaction experience locally on the terminal, only uploading encrypted model gradients or desensitized features to the cloud server to participate in the federated aggregation of the global model, and receiving updated model parameters to update the local deep reinforcement learning model. An interactive presentation module is used to present learning resources in a two-dimensional or three-dimensional manner, collect learners' interactive feedback, and transmit the feedback data to the multimodal data acquisition module. The multi-mode communication module supports Wi-Fi, 5G / 6G and near-field communication, and is used to synchronize model parameters, learning resources and de-identified data with the cloud server.

[0006] Preferably, the projection assembly includes a movable seat fixed to the upper surface of the workbench, a lead screw rotatably mounted in the inner cavity of the movable seat, a movable seat threaded onto the surface of the lead screw and sleeved in the inner cavity of the movable seat, a motor fixed to one side of the movable seat and driven by the lead screw, a rotating seat mounted on the upper surface of the movable seat, and a projector mounted on the rotating end of the rotating seat.

[0007] Preferably, the multimodal data acquisition module includes: The behavior acquisition unit collects learners' click sequences, correct and incorrect answers, and voice interaction data through a touch screen, keyboard, and microphone. The visual physiological acquisition unit includes at least one high-definition camera and an infrared camera, used to acquire learners’ facial videos and eye movement trajectories, and extract gaze focus, pupil diameter, blink frequency and facial expression codes through a local lightweight neural network. The contact-based physiological data acquisition unit includes wearable sensors integrated into the terminal housing or connected wirelessly, used to collect heart rate variability and skin conductance response to calculate the learner's cognitive load and emotional arousal. The environmental sensing unit, including a light sensor and a microphone array, is used to collect ambient light intensity and background noise decibels as supplementary contextual information.

[0008] Preferably, the dynamic knowledge graph library stored in the local heterogeneous storage module is organized in the form of a graph database. Each knowledge point node includes a mastery threshold and a recommended learning duration. The directed edges between nodes represent the predecessor and successor relationships and the strength of the association. The association strength is dynamically adjusted based on the group path data of historical learners and is updated periodically from the cloud via the communication module.

[0009] Preferably, the edge intelligent planning module includes: The state feature fusion unit is used to perform time alignment and normalization on the heterogeneous data (including behavioral features, physiological features, and environmental features) output by the multimodal data acquisition module, and splice them into a state vector of fixed dimensions. The lightweight reinforcement learning inference unit has a built-in deep Q-network (DQN) or proximal policy optimization (PPO) model with 8-bit quantization and head pruning. It takes the state vector as input, outputs the Q value or action probability of each candidate knowledge point, and selects the next knowledge point using an ε-greedy or sampling strategy. The path memory unit is used to record the sequence of visited knowledge points and the corresponding changes in mastery. Combined with the pre-constraints in the knowledge graph, the effectiveness of the candidate actions output by the model is verified to ensure that the generated learning path conforms to cognitive laws. The local experience replay buffer is used to store the most recent N learning experience tuples (s, a, r, s'). When the buffer is full, it triggers local model fine-tuning or uploads the data to the privacy-preserving federated learning module.

[0010] Preferably, the lightweight reinforcement learning model is obtained by distilling knowledge from a large model in the cloud using knowledge distillation technology. The teacher model is a deep network based on Transformer, and the student model is a 3-5 layer fully connected network. Attention transfer loss is introduced during the distillation process to enable the student model to simulate the intermediate layer features of the teacher model, thereby improving the representation ability of the local model.

[0011] Preferably, the context-aware push module includes: The real-time state parsing unit outputs an attention score (0-1) and an emotion category (confused, tired, happy, neutral) through a pre-trained lightweight convolutional neural network based on eye movements and facial expressions in the current physiological state data. The resource adaptation decision unit, based on the attention score and emotion category, combined with the learner's cognitive style preferences, decides on the resource recommendation strategy: if the attention score is below the threshold, the video resource is switched to audio playback or a break is prompted; if confusion is detected, the difficulty level is reduced and explanations of prior knowledge points are recommended; if a pleasant emotion is detected, challenging questions are added or extended resources are recommended. The multimodal resource retrieval unit retrieves matching resources from the local heterogeneous storage module based on the decision results and recommended knowledge point identifiers. It supports multiple formats such as video, audio, text, interactive exercises, and AR / VR scenes, and performs format conversion according to the terminal capabilities. The preloading and caching management unit preloads relevant resources from the cloud to the local machine based on the predicted next most likely knowledge points, reducing waiting latency.

[0012] Preferably, the privacy-preserving federated learning module includes: The local differential privacy processor adds Laplacian noise or uses a random response mechanism before uploading model gradients or features to ensure that a single learning record cannot be reversed. Gradient compression unit, which uses sparsification or quantization techniques to reduce the amount of uploaded data; The federated aggregation client is responsible for communicating with the cloud coordination server in multiple rounds, participating in global model updates, and receiving aggregated model parameters to replace the local model. The secure storage unit is used to encrypt and store the learner's raw physiological data, which is processed locally and not uploaded to the cloud.

[0013] Preferably, the interactive presentation module is a mobile terminal integrating a high refresh rate touch screen, stereo speakers, and a linear motor, or augmented reality glasses or a virtual reality headset, supporting multiple natural interaction methods such as gesture recognition, eye tracking, and voice control, and can adaptively switch display modes according to the type of pushed resources.

[0014] Preferably, the multi-mode communication module supports offline mode, where the edge intelligent planning module works entirely on the local model and cache resources, and the logs generated during the learning process are temporarily stored locally. Once the network is restored, the privacy-preserving federated learning module will resume transmission and synchronization from where interruption occurred.

[0015] The beneficial effects of this invention are as follows: 1. In this invention, by deploying a deep reinforcement learning model that has undergone knowledge distillation and mixed precision quantization compression on the local terminal, the edge intelligent planning module processes multimodal data in real time and generates learning paths, thus eliminating the network latency and bandwidth dependence of traditional cloud architecture. Even in an offline environment, the terminal can still respond to the learner's state changes in milliseconds and dynamically adjust the learning content to ensure the continuity and smoothness of the learning process, significantly improving the real-time performance and reliability of personalized learning. 2. In this invention, by integrating multimodal physiological data such as eye tracking, heart rate variability, skin conductance response, and facial expression coding, a comprehensive cognitive state profile of learners is constructed. The context-aware push module can identify emotional fluctuations such as inattention, cognitive fatigue, confusion, or pleasure in real time, and dynamically adjust the presentation format, difficulty level, and interaction method of resources accordingly, so that the learning content truly matches the learner's psychological and physiological state, and realizes personalized teaching with deep context awareness. 3. This invention introduces a privacy-preserving federated learning module. The original physiological and behavioral data always reside locally on the terminal. Only the encrypted model gradient, which has been subjected to differential privacy noise addition and sparse compression, is uploaded to participate in the global model aggregation in the cloud. The secure encryption chip ensures that sensitive information cannot be reverse-engineered. Even if the terminal is lost, privacy will not be leaked. While protecting the user's data sovereignty, the terminal model can still continuously evolve from collective intelligence, achieving a balance between privacy security and model performance. 4. In this invention, the projection assembly, consisting of a motor-driven lead screw and a rotating base, enables free adjustment of the projector's lateral movement and pitch rotation. Combined with automatic focus and keystone correction functions, it ensures that the image remains clear and square at all times. The laser light source supports short-throw projection, allowing for the projection of large-screen images in group discussions or collaborative scenarios. The projection assembly can be completely stored in a recessed area of ​​the workbench, satisfying both individual immersive learning needs and flexible switching to group collaboration modes, significantly improving the terminal's adaptability to different scenarios. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of a terminal based on adaptive learning path planning and resource push proposed in this invention; Figure 2 This is a schematic diagram of the projection component in this invention; Figure 3 This is a schematic diagram of the terminal hardware module connection in this invention; Figure 4 This is a schematic diagram of the internal functional modules and data flow of the terminal in this invention; Figure 5 This is a detailed structural diagram of the multimodal data acquisition module in this invention; Figure 6 This is a flowchart illustrating the internal processing of the edge intelligent planning module in this invention. Figure 7 This is a schematic diagram of the decision logic of the context-aware push module in this invention; Figure 8 This is a sequence diagram showing the interaction between the privacy-preserving federated learning module and the cloud in this invention.

[0017] In the diagram: 1. Workbench; 2. Touchscreen interactive screen; 3. Voice module; 4. Projection component; 401. Movable base; 402. Lead screw; 403. Movable base; 404. Motor; 405. Rotary base; 406. Projector; 5. Chassis; 6. Control cabinet. Detailed Implementation

[0018] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Reference Figure 1 - Figure 8 A terminal based on adaptive learning path planning and resource push includes a workbench 1. A touchscreen 2, a voice module 3, and a projection component 4 are mounted on the upper surface of the workbench 1. A chassis 5 is fixed to the lower end of the workbench 1, and a control cabinet 6 is installed inside the chassis 5. The control cabinet 6 includes: The multimodal data acquisition module is used to collect learners' learning behavior data and at least two kinds of physiological state data in real time. The physiological state data includes at least two of the following: eye movement data, heart rate data, skin conductance data, and facial expression data. The local heterogeneous storage module is used to store dynamic knowledge graph libraries, multi-format learning resource libraries, and learner profiles containing learner knowledge mastery, cognitive styles, and historical paths. The edge intelligent planning module is deployed locally on the terminal. It has a built-in deep reinforcement learning model that has undergone knowledge distillation and mixed precision quantization compression. It takes the current learning state vector output by the multimodal data acquisition module as input, outputs the optimal next knowledge point identifier in real time, and dynamically generates a personalized learning path. The context-aware push module is used to identify the learner's real-time attention level and emotional state based on the physiological state information in the current learning state vector, and adjust the presentation format, difficulty level and interaction method of the resources to be pushed accordingly, and retrieve the appropriate resources from the local heterogeneous storage module or from the cloud resource library through the communication module. The privacy-preserving federated learning module is used to accumulate learning and interaction experience locally on the terminal, only uploading encrypted model gradients or desensitized features to the cloud server to participate in the federated aggregation of the global model, and receiving updated model parameters to update the local deep reinforcement learning model. The interactive presentation module is used to present learning resources in two-dimensional or three-dimensional form, collect learners' interactive feedback, and transmit the feedback data to the multimodal data acquisition module. The multi-mode communication module supports Wi-Fi, 5G / 6G and near-field communication, and is used to synchronize model parameters, learning resources and de-identified data with the cloud server. In this embodiment, the workbench 1 adopts an ergonomic curved design and is covered with an anti-glare coating to adapt to long-term use under different lighting conditions. The touch screen 2 is an OLED screen that supports multi-touch and pressure feedback, with a resolution of 4K level, ensuring the fineness of the learning resources and the smoothness of the interaction. The voice module 3 includes a high-sensitivity microphone array and a directional speaker, which can realize far-field voice wake-up and noise reduction processing, making it convenient for learners to control the learning process by voice when their hands are occupied. The chassis 5 adopts a heat dissipation air duct design and has a shockproof bracket inside to ensure the stability and quietness of the control cabinet 6 during high-speed operation. The control cabinet 6 also integrates a backup power unit, which can maintain the normal operation of the terminal for at least 30 minutes in the event of a power outage to prevent the loss of learning data.

[0020] In this embodiment, the projection component 4 includes a movable seat 401 fixed to the upper surface of the workbench 1, a lead screw 402 rotatably mounted in the inner cavity of the movable seat 401, a movable seat 403 threadedly connected to the surface of the lead screw 402 and sleeved in the inner cavity of the movable seat 401, a motor 404 fixed on one side of the movable seat 401 and connected to the lead screw 402, a rotating seat 405 mounted on the upper surface of the movable seat 403, and a projector 406 mounted on the rotating end of the rotating seat 405. Motor 404 is a high-precision stepper motor. With the encoder, it can achieve micron-level displacement control of lead screw 402, thereby enabling the moving base 403 to move the projector 406 laterally on the workbench 1 to adapt to different learners' viewing positions. The rotating base 405 has two built-in motors with two degrees of freedom, which can realize the pitch and horizontal rotation of the projector 406. Combined with automatic focus and keystone correction functions, it ensures that the projected image is always clearly and squarely projected onto the wall or a special screen. The projector 406 uses a laser light source with a brightness of up to 3000 lumens and supports short-throw projection. Even near the workbench 1, it can project a large-size image, which is convenient for group discussions or multi-person collaborative learning scenarios. In addition, the projection component 4 can be completely stored in the groove on one side of the workbench 1, keeping the table surface clean when not in use.

[0021] In this embodiment, the multimodal data acquisition module includes: The behavior acquisition unit collects learners' click sequences, correct and incorrect answers, and voice interaction data through a touch screen, keyboard, and microphone. The visual physiological acquisition unit includes at least one high-definition camera and an infrared camera, used to acquire learners’ facial videos and eye movement trajectories, and extract gaze focus, pupil diameter, blink frequency and facial expression codes through a local lightweight neural network. The contact-based physiological data acquisition unit includes wearable sensors integrated into the terminal housing or connected wirelessly, used to collect heart rate variability and skin conductance response to calculate the learner's cognitive load and emotional arousal. The environmental sensing unit, including a light sensor and a microphone array, is used to collect ambient light intensity and background noise decibels as supplementary contextual information. In practical applications, the behavior acquisition unit not only records clicks and answers, but also performs semantic understanding and sentiment analysis on learners' voice commands. For example, it judges the learner's urgency or hesitation by the speech rate and tone. The visual physiological acquisition unit adopts a binocular camera design and uses infrared illumination to achieve eye tracking in low-light environments. Changes in pupil diameter can reflect fluctuations in cognitive load, while blinking frequency is negatively correlated with attention deficit. The contact physiological acquisition unit can be embedded in the grip area on both sides of the touch screen 2. When the learner's hands touch, it automatically collects heart rate and skin conductance, or obtains continuous data by connecting to a smart bracelet via Bluetooth. The ambient light sensor of the environmental perception unit can automatically adjust the screen brightness and projection brightness, while the microphone array monitors ambient noise in real time. When the noise exceeds the threshold, it automatically increases the speaker volume or prompts the learner to wear headphones to ensure a good learning experience.

[0022] In this embodiment, the dynamic knowledge graph library stored in the local heterogeneous storage module is organized in the form of a graph database. Each knowledge node includes the mastery threshold and recommended learning time. The directed edges between nodes represent the predecessor and successor relationships and the strength of the association. The association strength is dynamically adjusted based on the group path data of historical learners and is updated periodically from the cloud via the communication module. The dynamic knowledge graph database not only stores subject-specific knowledge points but also incorporates interdisciplinary connections, such as the potential link between "functions" in mathematics and "kinematics" in physics. The mastery threshold for each knowledge point node can be personalized based on the learner's age and grade level. The recommended learning time is dynamically adjusted based on group statistics and individual historical learning speed. The dynamic adjustment of the connection strength is based on millions of learning path data aggregated in the cloud. Through collaborative filtering algorithms, the jumping patterns of different learners between knowledge points are mined, thereby optimizing the rationality of the recommended path. Local storage also adopts a hot and cold data tiering strategy, with frequently accessed knowledge graph subgraphs residing in memory and low-frequency data compressed and stored in flash memory to improve retrieval efficiency.

[0023] In this embodiment, the edge intelligent planning module includes: The state feature fusion unit is used to perform time alignment and normalization on the heterogeneous data (including behavioral features, physiological features, and environmental features) output by the multimodal data acquisition module, and concatenate them into a state vector of fixed dimensions. The lightweight reinforcement learning inference unit has a built-in deep Q-network (DQN) or proximal policy optimization (PPO) model with 8-bit quantization and head pruning. It takes the state vector as input, outputs the Q value or action probability of each candidate knowledge point, and uses an ε-greedy or sampling strategy to select the next knowledge point. The path memory unit is used to record the sequence of visited knowledge points and the corresponding changes in mastery. Combined with the pre-constraints in the knowledge graph, the effectiveness of the candidate actions output by the model is verified to ensure that the generated learning path conforms to cognitive laws. The local experience replay buffer is used to store the N most recent learning experience tuples (s,a,r,s'). When the buffer is full, it triggers local model fine-tuning or uploads to the privacy-preserving federated learning module. The state feature fusion unit adopts a sliding window mechanism to align multimodal data from the past 5 seconds to the same timestamp. Missing values ​​are filled in by linear interpolation, and outliers are removed by median filtering. The lightweight reinforcement learning inference unit is accelerated by the terminal NPU, with a single inference time of less than 5ms, ensuring the real-time performance of path planning. The path memory unit not only verifies the pre-constraints but also dynamically adjusts the mastery decay of learned knowledge points according to the learner's forgetting curve. If a long period of no review is detected, a review node will be inserted into the path. The local experience replay buffer adopts a priority experience replay strategy, giving higher sampling weights to experiences that cause significant changes in learning performance (such as sudden changes in answer accuracy), thereby accelerating model convergence.

[0024] In this embodiment, the lightweight reinforcement learning model is obtained from a large model in the cloud using knowledge distillation technology. The teacher model is a deep network based on Transformer, and the student model is a 3-5 layer fully connected network. Attention transfer loss is introduced during the distillation process to make the student model simulate the intermediate layer features of the teacher model in order to improve the representation ability of the local model. Knowledge distillation is completed offline in the cloud. The teacher model is trained using massive learning logs to master complex decision-making patterns. The student model learns by minimizing the KL divergence of the output probability distribution of the teacher model and the mean square error of the intermediate layer features. Ultimately, while maintaining a small number of parameters, it achieves more than 90% of the performance of the teacher model. Before deployment, the model is also subjected to 8-bit weight quantization and activation value quantization to further reduce the model size and memory usage, enabling the terminal to run continuously with low power consumption. The distillation process is carried out once a quarter. After collecting the latest data in the cloud, the teacher model is updated, and a better student model is distilled and distributed to the terminal.

[0025] In this embodiment, the context-aware push module includes: The real-time state parsing unit outputs an attention score (0-1) and an emotion category (confused, tired, happy, neutral) through a pre-trained lightweight convolutional neural network based on eye movements and facial expressions in the current physiological state data. The resource adaptation decision unit determines the resource recommendation strategy based on attention score and emotion category, combined with the learner's cognitive style preferences: if the attention score is below the threshold, the video resource is switched to audio playback or a break is prompted; if confusion is detected, the difficulty level is reduced and explanations of prior knowledge points are recommended; if a pleasant emotion is detected, challenging questions are added or extended resources are recommended. The multimodal resource retrieval unit retrieves matching resources from the local heterogeneous storage module based on the decision results and recommended knowledge point identifiers. It supports multiple formats such as video, audio, text, interactive exercises, and AR / VR scenes, and performs format conversion according to the terminal capabilities. The preloading and caching management unit preloads relevant resources from the cloud to the local machine based on the predicted next most likely knowledge points, reducing waiting latency; The real-time state parsing unit uses a lightweight convolutional neural network based on the MobileNetV3 architecture. After fine-tuning with facial expression and eye-tracking datasets, it can output seven basic emotions and continuous attention values ​​in real time. The resource adaptation decision unit has a built-in rule engine, and rules can be updated via the cloud. For example, thresholds for learners of different age groups can be dynamically adjusted. The multimodal resource retrieval unit not only retrieves local resources but also requests them from the cloud via a communication module if they are missing locally, and simultaneously caches them locally. It also automatically selects resource resolution (such as video bitrate) based on the current network bandwidth. The preloading and caching management unit selects the three knowledge points with the highest probabilities based on the action probability distribution output by the reinforcement learning model for preloading. When cache space is insufficient, the LRU algorithm is used to evict the least used resources.

[0026] In this embodiment, the privacy-preserving federated learning module includes: The local differential privacy processor adds Laplacian noise or uses a random response mechanism before uploading model gradients or features to ensure that a single learning record cannot be reversed. Gradient compression unit, which uses sparsification or quantization techniques to reduce the amount of uploaded data; The federated aggregation client is responsible for communicating with the cloud coordination server in multiple rounds, participating in global model updates, and receiving aggregated model parameters to replace the local model. A secure storage unit is used to encrypt and store the learner's raw physiological data, which is processed locally only and not uploaded to the cloud. The local differential privacy processor sets different privacy budgets ε for different data types. For example, physiological data is set with a stricter ε (e.g., ε=1), while behavioral data can be appropriately relaxed (e.g., ε=3) to balance privacy protection and model performance. The gradient compression unit uses Top-k gradient sparsity, uploading only the 10% gradients with the largest absolute value, and quantizes them to 8 bits, reducing the amount of uploaded data by more than 90%. The federated aggregation client supports asynchronous federated learning and automatically participates in aggregation when the terminal is charging and connected to Wi-Fi to avoid affecting normal use. The secure storage unit uses a hardware encryption engine (e.g., TEE) to protect the original physiological data, so sensitive information cannot be read even if the terminal is lost.

[0027] In this embodiment, the interactive presentation module is a mobile terminal that integrates a high refresh rate touch screen, stereo speakers, and a linear motor, or augmented reality glasses or a virtual reality headset. It supports a variety of natural interaction methods such as gesture recognition, eye tracking, and voice control, and can adaptively switch display modes according to the type of pushed resources. When AR / VR resources are pushed, the interactive presentation module automatically switches to 3D mode. If a tablet terminal is currently being used, the learner will be prompted to connect a head-mounted display or enable naked-eye 3D effects (achieved through screen raster). Gesture recognition uses a depth camera (such as a ToF sensor) to capture 22 key points on the hand, supporting natural gestures such as clicking, swiping, and grasping. Virtual objects can be directly manipulated in AR scenes. Eye tracking is used for gaze-based rendering. In VR mode, the resolution of non-gaze areas is reduced to save power. Linear motors provide delicate tactile feedback, such as providing vibration encouragement when completing a question or simulating the feeling of pressing a button in a virtual experiment. Adaptive display mode switching also includes adjusting the screen color temperature according to ambient light and optimizing color saturation according to content type (video / text / charts) to improve visual comfort.

[0028] In this embodiment, the multi-mode communication module supports the edge intelligent planning module to work entirely on the local model and cache resources in offline mode. The logs generated during the learning process are temporarily stored locally and resumed and synchronized by the privacy-preserving federated learning module after the network is restored. In offline mode, the terminal can form an ad hoc network with other learning terminals via near-field communication (such as Bluetooth) to achieve resource sharing and path exchange within a small area. For example, terminals in the same classroom can exchange anonymized learning experiences to improve the diversity of recommendations when offline. When the network is restored, the multi-mode communication module automatically detects and switches to online mode. The privacy-preserving federated learning module first uploads the encrypted experience accumulated during offline, and then synchronizes the latest model parameters and resource updates. The breakpoint resume function ensures that the transmission of large files (such as AR scene packages) can continue after the transmission is interrupted, saving traffic. In addition, the communication module supports multi-link aggregation and can use Wi-Fi and 5G simultaneously to improve transmission speed and stability.

[0029] like Figure 3 As shown, the terminal in this embodiment includes an application processor, a neural network processing unit (NPU), memory, non-volatile memory, a touch screen, a front-facing infrared camera, a rear-facing high-definition camera, a heart rate sensor, a skin conductance sensor, a microphone array, a speaker, a light sensor, a Wi-Fi / 5G communication module, a Bluetooth module, a power management unit, and a security encryption chip. All components are connected via a high-speed bus. The non-volatile memory stores the operating system, a lightweight reinforcement learning model, a dynamic knowledge graph library, a learning resource cache, and a user profile database. The security encryption chip is used to store encryption keys and handle differential privacy noise additions in federated learning.

[0030] Figure 4 The functional module logic division and data flow of this terminal are shown. The multimodal data acquisition module receives raw signals from the camera heart rate sensor, skin conductance sensor, etc. After preprocessing and feature extraction, the feature vector is sent to the edge intelligent planning module. The planning module combines the knowledge graph and user profile in the local heterogeneous storage module, and generates the next knowledge point through a lightweight reinforcement learning model, which is then passed to the context-aware push module. The push module retrieves resources from the storage module or from the cloud through the communication module based on the physiological state analysis results. After adaptation, the interactive presentation module (touch screen, speaker, etc.) displays the information to the learner. The learner's feedback (such as answering questions, clicking) enters the acquisition module again, forming a closed loop. The privacy-protected federated learning module periodically uploads local experience or gradients to the cloud federated server through the communication module after secure encryption, and receives updated model parameters to update the local model.

[0031] like Figure 5 As shown, the multimodal data acquisition module includes: Behavior capture unit: Captures touch screen click coordinates, keyboard input, correctness of answers, video viewing duration, etc., to form a behavior sequence.

[0032] Visual physiological acquisition unit: It uses a front-facing infrared camera to capture facial video at 30fps, and uses a lightweight neural network (such as MobileNet) to run a local NPU to extract facial action units (AU) and eye movement features, including fixation point, saccade speed, pupil diameter change, and blink frequency. It uses a rear-facing high-definition camera to capture the environment in AR mode.

[0033] Contact-type physiological acquisition unit: acquires photoplethysmography (PPG) through heart rate sensor 208 to calculate heart rate variability (HRV); acquires skin conductance level (SCL) and phase response (SCR) through skin conductance sensor to reflect sympathetic nerve activity.

[0034] Environmental perception unit: Collects ambient illuminance and noise decibels through light sensors and microphone arrays, which are used to adjust the subsequent resource presentation method (such as automatically brightening the screen or switching to audio in low light).

[0035] The aforementioned multi-source data undergoes time synchronization (e.g., using a uniform 100ms time window) and normalization in the feature fusion unit, ultimately forming a 3D state vector containing behavioral, physiological, and environmental features, which is then fed into the planning module.

[0036] like Figure 6 As shown, the internal process of the edge intelligent planning module is as follows: Step S1: The state feature fusion unit receives the latest state vector and caches the states of the previous few time steps to capture timing information.

[0037] Step S2: The lightweight reinforcement learning inference unit loads an 8-bit quantized DQN model (34 nodes in the input layer, 128+64 nodes in the hidden layer, and the output layer corresponds to the number of candidate knowledge points in the knowledge graph). The model performs forward propagation to calculate the Q-value of each action.

[0038] Step S3: The path memory unit masks the Q value based on the pre-constraints in the knowledge graph (such as not recommending "integral" if "derivative" is not mastered) to block illegal actions.

[0039] Step S4: Use an ε-greedy strategy to select actions, exploring randomly with probability ε, otherwise select the action with the largest Q value.

[0040] Step S5: Return the selected knowledge point identifier to the push module, and store the experience (s,a,r,s') of this step into the local experience replay buffer (capacity 10,000 records). When the buffer is full, trigger local fine-tuning (update the model using mini-batch gradient descent) or package the experience for federated learning.

[0041] The design of the reward r takes into account the accuracy of answering questions (+1), learning efficiency (calculated based on the speed of improvement in mastery), and physiological state (if excessive fatigue is detected, a negative reward is given to encourage rest).

[0042] like Figure 7 As shown, the decision logic of the context-aware push module is as follows: Step S1: The real-time state analysis unit receives the current facial expression image and eye movement data from the acquisition module, and outputs the attention score (0~1) and emotion classification (confused, tired, happy, neutral) through a lightweight convolutional neural network (such as SqueezeNet).

[0043] Step S2: The resource adaptation decision unit obtains the attention score and sentiment classification, and combines them with the recommended knowledge point ID and learner preferences (such as whether they prefer videos or text / images) to formulate a push strategy. If the attention score is <0.3 and the emotion is fatigue: push an audio summary or prompt a 5-minute break, and reduce the difficulty; If the emotion is confusion: push out a micro-lesson on the preceding knowledge points or basic practice questions, and slow down the pace of the explanation; If the mood is pleasant: push extended reading or challenging exercises, or recommend AR interactive exploration; By default, videos or articles are pushed to learners based on their preferences.

[0044] Step S3: The multimodal resource retrieval unit searches for the corresponding format resource in the local resource cache based on the knowledge point ID and strategy. If the resource is not found locally, it requests it from the cloud through the communication module and caches it locally.

[0045] Step S4: The preloading and caching management unit retrieves resources from the cloud in advance based on the next few possible knowledge points predicted by the model (the first 3 in the model's output probability distribution) to achieve a smooth switch.

[0046] like Figure 8 As shown, the sequence of interactions between the privacy-preserving federated learning module and the cloud: T1: When the terminal's local accumulated experience reaches a threshold (e.g., 1000), a federated learning participation request is triggered.

[0047] T2: The terminal obtains the current global model version number and aggregation parameters from the cloud.

[0048] T3: The terminal uses local experience to calculate the model gradient, and then clips the gradient, adds Laplacian noise (to achieve ε-differential privacy), and then uses sparsification (retaining the top 10% of the largest gradients) for compression.

[0049] T4: Upload the encrypted and compressed gradient to the cloud federated aggregation server through a secure channel.

[0050] T5: After collecting gradients from enough terminals in the cloud, the FedAvg algorithm is executed to update the global model.

[0051] T6: The cloud sends the new model parameters to the terminal, which then uses them to replace the local model, completing one round of updates.

[0052] Throughout the process, the raw physiological and behavioral data remain locally on the terminal and are not uploaded to the cloud. Only the noise gradient, which is difficult to reverse, is uploaded, effectively protecting user privacy.

[0053] When the terminal detects that the network is unavailable, it automatically switches to offline mode: The multi-mode communication module stops making cloud requests, and the edge intelligent planning module relies entirely on local models and cached resources to operate.

[0054] New experiences generated during the learning process are temporarily stored in the local experience playback buffer and written to non-volatile memory.

[0055] The context-aware push module only uses the local resource library. If the resource is missing, it will downgrade to the default push format or prompt that it cannot be obtained.

[0056] Once the network is restored, the accumulated experience during the offline period will be automatically resumed and synchronized through the privacy-preserving federated learning module, and the local model and resource cache will be updated.

[0057] This invention provides a terminal based on adaptive learning path planning and resource push, whose working principle is based on a closed-loop intelligent learning paradigm of "collection, planning, push, and feedback," as detailed below: First, the learner starts the learning session through the touch screen 2 or voice module 3 on the workbench 1. At this time, the various functional modules in the control cabinet 6 work together to enter the working state. S1: Multimodal data acquisition and state perception. The multimodal data acquisition module activates all perception units in real time. The behavior acquisition unit records the learner's click sequence on the touch screen 2, the correctness of answers, and voice interaction data with the voice module 3. The visual physiological acquisition unit acquires the learner's facial video and eye movement trajectory at a frame rate of 30fps through the front infrared camera and the rear high-definition camera, and uses the local neural network processing unit (NPU) to run a lightweight neural network (such as MobileNet) to extract the gaze focus, pupil diameter change, blink frequency, and facial expression encoding in real time. The contact physiological acquisition unit acquires heart rate variability (HRV) and skin conductance response (SCR / SCL) through sensors embedded on both sides of the touch screen 2 or wirelessly connected wearable devices to calculate cognitive load and emotional arousal. The environmental perception unit acquires ambient light intensity and background noise decibels through light sensors and microphone arrays. The above multi-source heterogeneous data are time-aligned (e.g., 100ms time window), normalized, and outlier removed in the feature fusion unit, and finally form a fixed-dimensional current learning state vector, which is sent to the edge intelligent planning module. S2: Edge Intelligent Planning and Path Decision-Making. The edge intelligent planning module receives the current learning state vector. First, the state feature fusion unit uses a sliding window mechanism to integrate the temporal information of the past 5 seconds. Then, the lightweight reinforcement learning inference unit loads the deep reinforcement learning model after 8-bit quantization and head pruning, and uses the NPU to perform millisecond-level forward inference, outputting the Q-value or action probability of each candidate knowledge point in the knowledge graph. Next, the path memory unit performs validity verification on the model output based on the pre-constraints of the dynamic knowledge graph library stored in the local heterogeneous storage module (such as not recommending "integral" if "derivative" is not mastered), and blocks illegal actions to ensure that the learning path conforms to the cognitive law. Finally, the ε-greedy strategy is used to select the next optimal knowledge point identifier. At the same time, the learning experience (s, a, r, s') of this step is stored in the local experience replay buffer, where the reward r integrates the answer accuracy, learning efficiency and physiological state (such as giving negative rewards when fatigued). The knowledge point identifier generated by the decision is sent to the context-aware push module.

[0058] S3: Context-Aware Push and Resource Adaptation. The context-aware push module first performs in-depth analysis of facial expressions and eye movement information in the current physiological data through the real-time state analysis unit. It uses a lightweight convolutional neural network (such as SqueezeNet) to output attention scores (0-1) and emotion categories (confusion, fatigue, pleasure, neutral). The resource adaptation decision unit dynamically formulates push strategies based on these real-time states and the cognitive style preferences in the learner profile: if the attention score is below the threshold and the learner is fatigued, the video resource is switched to audio playback or a rest prompt is given; if confusion is detected, the difficulty level is reduced and prior knowledge points are explained; if pleasure is detected, challenging questions are added or AR extension resources are recommended. The multimodal resource retrieval unit first retrieves matching resources in the local heterogeneous storage module based on the decision results and knowledge point identifiers. If the resources are missing locally, they are requested from the cloud through the multimodal communication module, and the resolution is automatically selected for downloading and caching based on the current network bandwidth. The preloading and caching management unit retrieves resources from the cloud in advance based on the next few most likely knowledge points predicted by the reinforcement learning model, achieving seamless switching.

[0059] S4: Interactive presentation and feedback loop. The interactive presentation module (including touch screen 2, projection component 4, speaker, etc.) adaptively switches the display mode according to the type of pushed resources: two-dimensional resources are displayed on touch screen 2 in 4K quality, while three-dimensional AR / VR resources are projected or connected to the head-mounted display device through the projection component. Learners interact with resources through touch, voice, gestures, etc. The interaction results (such as correct or incorrect answers, dwell time, voice commands) are captured in real time by the multimodal data acquisition module, forming a new state vector, driving the next round of planning, and forming a complete learning loop.

[0060] S5: Privacy-Preserving Federated Learning and Model Evolution. The privacy-preserving federated learning module accumulates learning experience locally on the terminal. When the experience reaches a threshold (e.g., 1000 data points) and the terminal is charging or connected to Wi-Fi, the federated learning process is triggered. The local differential privacy processor adds Laplacian noise to the calculated model gradients to achieve ε-differential privacy protection. The gradient compression unit uses Top-k sparsity (retaining the first 10% of gradients) and 8-bit quantization, significantly reducing the amount of data uploaded. The encrypted and compressed gradients are uploaded to the cloud federated server through a secure channel. After collecting enough gradients from multiple terminals, the cloud executes a federated averaging algorithm to update the global model and sends the new model parameters to the terminal to replace the local model. Throughout the process, the original physiological data remains in the terminal's secure storage unit (protected by a secure encryption chip) and is not uploaded to the cloud, effectively protecting user privacy.

[0061] S6: Offline operation and breakpoint resume. When the network is unavailable, the multi-mode communication module automatically detects and switches to offline mode. At this time, the edge intelligent planning module relies entirely on the local model and cached resources to work. The learning log is temporarily stored in the local non-volatile memory. The context-aware push module only uses the local resource library. If resources are missing, it will push in a downgraded manner. When the network is restored, the terminal will automatically resume and synchronize the encrypted experience accumulated during the offline period, and receive the latest model parameters and resource updates to ensure the continuity of the learning process and the integrity of the data.

[0062] Through the collaborative work of the above six steps, this terminal achieves end-to-end adaptive learning, from physiological perception, real-time planning, context adaptation to privacy evolution, truly realizing an intelligent learning experience tailored to individual needs.

[0063] The above provides a detailed description of the adaptive learning path planning and resource push terminal provided by this invention. Specific embodiments have been used to illustrate the principles and implementation methods of this invention. These embodiments are merely illustrative and are intended to help understand the method and core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims of this invention.

Claims

1. A terminal based on adaptive learning path planning and resource push, characterized in that, The system includes a workbench (1), on the upper surface of which is mounted a touch screen (2), a voice module (3), and a projection component (4). A chassis (5) is fixed to the lower end of the workbench (1), and a control cabinet (6) is installed inside the chassis (5). The control cabinet (6) includes: A multimodal data acquisition module is used to collect learners' learning behavior data and at least two kinds of physiological state data in real time. The physiological state data includes at least two of the following: eye movement data, heart rate data, skin conductance data, and facial expression data. The local heterogeneous storage module is used to store dynamic knowledge graph libraries, multi-format learning resource libraries, and learner profiles containing learner knowledge mastery, cognitive styles, and historical paths. The edge intelligent planning module is deployed locally on the terminal. It has a built-in deep reinforcement learning model that has undergone knowledge distillation and mixed precision quantization compression. It takes the current learning state vector output by the multimodal data acquisition module as input, outputs the optimal next knowledge point identifier in real time, and dynamically generates a personalized learning path. The context-aware push module is used to identify the learner's real-time attention level and emotional state based on the physiological state information in the current learning state vector, and adjust the presentation format, difficulty level and interaction method of the resources to be pushed accordingly, and retrieve the appropriate resources from the local heterogeneous storage module or from the cloud resource library through the communication module. The privacy-preserving federated learning module is used to accumulate learning and interaction experience locally on the terminal, only uploading encrypted model gradients or desensitized features to the cloud server to participate in the federated aggregation of the global model, and receiving updated model parameters to update the local deep reinforcement learning model. An interactive presentation module is used to present learning resources in a two-dimensional or three-dimensional manner, collect learners' interactive feedback, and transmit the feedback data to the multimodal data acquisition module. The multi-mode communication module supports Wi-Fi, 5G / 6G and near-field communication, and is used to synchronize model parameters, learning resources and de-identified data with the cloud server.

2. The terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The projection assembly (4) includes a movable seat (401) fixed to the upper surface of the workbench (1), a lead screw (402) is rotatably installed in the inner cavity of the movable seat (401), a movable seat (403) is threadedly connected to the surface of the lead screw (402) and sleeved in the inner cavity of the movable seat (401), a motor (404) is fixed on one side of the movable seat (401) and is drivenly connected to the lead screw (402), a rotating seat (405) is installed on the upper surface of the movable seat (403), and a projector (406) is installed on the rotating end of the rotating seat (405).

3. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The multimodal data acquisition module includes: The behavior data collection unit collects learners' click sequences, correct and incorrect answers, and voice interaction data through a touch screen, keyboard, and microphone. The visual physiological acquisition unit includes at least one high-definition camera and an infrared camera, used to acquire learners’ facial videos and eye movement trajectories, and extract gaze focus, pupil diameter, blink frequency and facial expression codes through a local lightweight neural network. The contact-based physiological data acquisition unit includes wearable sensors integrated into the terminal housing or connected wirelessly, used to collect heart rate variability and skin conductance response to calculate the learner's cognitive load and emotional arousal. The environmental sensing unit, including a light sensor and a microphone array, is used to collect ambient light intensity and background noise decibels as supplementary contextual information.

4. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The dynamic knowledge graph library stored in the local heterogeneous storage module is organized in the form of a graph database. Each knowledge node contains the mastery threshold and recommended learning time. The directed edges between nodes represent the predecessor and successor relationships and the strength of the association. The association strength is dynamically adjusted based on the group path data of historical learners and is updated periodically from the cloud via the communication module.

5. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The edge intelligent planning module includes: The state feature fusion unit is used to perform time alignment and normalization on the heterogeneous data (including behavioral features, physiological features, and environmental features) output by the multimodal data acquisition module, and splice them into a state vector of fixed dimensions. The lightweight reinforcement learning inference unit has a built-in deep Q-network (DQN) or proximal policy optimization (PPO) model with 8-bit quantization and head pruning. It takes the state vector as input, outputs the Q value or action probability of each candidate knowledge point, and selects the next knowledge point using an ε-greedy or sampling strategy. The path memory unit is used to record the sequence of visited knowledge points and the corresponding changes in mastery. Combined with the pre-constraints in the knowledge graph, the effectiveness of the candidate actions output by the model is verified to ensure that the generated learning path conforms to cognitive laws. The local experience replay buffer is used to store the most recent N learning experience tuples (s, a, r, s'). When the buffer is full, it triggers local model fine-tuning or uploads the data to the privacy-preserving federated learning module.

6. A terminal based on adaptive learning path planning and resource push according to claim 5, characterized in that, The lightweight reinforcement learning model is obtained by distilling knowledge from a large model in the cloud using knowledge distillation technology. The teacher model is a deep network based on Transformer, and the student model is a 3-5 layer fully connected network. Attention transfer loss is introduced during the distillation process to make the student model simulate the intermediate layer features of the teacher model, thereby improving the representation ability of the local model.

7. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The context-aware push module includes: The real-time state parsing unit outputs an attention score (0-1) and an emotion category (confused, tired, happy, neutral) through a pre-trained lightweight convolutional neural network based on eye movements and facial expressions in the current physiological state data. The resource adaptation decision unit determines the resource recommendation strategy based on the attention score and emotion category, combined with the learner's cognitive style preferences: if the attention score is below the threshold, the video resource is switched to audio playback or a break is prompted; if confusion is detected, the difficulty level is reduced and explanations of prior knowledge points are recommended; if a pleasant emotion is detected, challenging questions are added or extended resources are recommended. The multimodal resource retrieval unit retrieves matching resources from the local heterogeneous storage module based on the decision results and recommended knowledge point identifiers. It supports multiple formats such as video, audio, text, interactive exercises, and AR / VR scenes, and performs format conversion according to the terminal capabilities. The preloading and caching management unit preloads relevant resources from the cloud to the local machine based on the predicted next most likely knowledge points, reducing waiting latency.

8. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The privacy-preserving federated learning module includes: The local differential privacy processor adds Laplacian noise or uses a random response mechanism before uploading model gradients or features to ensure that a single learning record cannot be reversed. Gradient compression unit, which uses sparsification or quantization techniques to reduce the amount of uploaded data; The federated aggregation client is responsible for communicating with the cloud coordination server in multiple rounds, participating in global model updates, and receiving aggregated model parameters to replace the local model. The secure storage unit is used to encrypt and store the learner's raw physiological data, which is processed locally and not uploaded to the cloud.

9. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The interactive presentation module is a mobile terminal that integrates a high refresh rate touch screen, stereo speakers, and a linear motor, or augmented reality glasses or a virtual reality headset. It supports a variety of natural interaction methods, including gesture recognition, eye tracking, and voice control, and can adaptively switch display modes according to the type of pushed resources.

10. A terminal based on adaptive learning path planning and resource push according to claim 1, characterized in that, The multi-mode communication module supports offline mode, where the edge intelligent planning module works entirely on local models and cached resources. Logs generated during the learning process are temporarily stored locally, and after the network is restored, the privacy-preserving federated learning module performs breakpoint resume and synchronization.

Images