Adaptive learning plan dynamic adjustment system based on reinforcement learning

Abstract

The application relates to a reinforcement learning-based adaptive learning plan dynamic adjustment system, which comprises a multi-modal data acquisition module for collecting interaction data from a terminal; a learner digital twin construction module for receiving the interaction data and outputting a digital twin state vector; a hierarchical reinforcement learning decision module for receiving the digital twin state vector and generating an activity instruction vector; and a learning plan generation and execution module for receiving the activity instruction vector and issuing a learning task. Full automation and high adaptability of learning plan generation and dynamic adjustment are realized. The learner state can be perceived in real time and responded to in real time, the matching accuracy and pushing timeliness of learning resources are significantly improved, the learning effect is ensured, and the cognitive load risk is effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of smart education technology, specifically to an adaptive learning plan dynamic adjustment system based on reinforcement learning. Background Technology

[0002] Existing adaptive learning systems are mostly based on knowledge tracking and heuristic rules, and can only recommend the next question or a single learning resource. They cannot comprehensively optimize long-term learning plans, including learning paths, time allocation, media type, review intervals, and cognitive load adjustment. A few solutions that incorporate reinforcement learning are limited to simple states (such as answer accuracy) and single actions (such as the next knowledge point), failing to consider the dynamic effects of forgetting patterns, cognitive load, and emotional states. Furthermore, online training suffers from security risks and slow cold-start issues. Therefore, there is an urgent need for a technical solution that can safely and efficiently generate and dynamically adjust comprehensive learning plans. Summary of the Invention

[0003] (a) Technical problems to be solved The technical problem to be solved by this invention is how to perform global, long-term and safe dynamic joint optimization of multi-granularity decision variables in an adaptive learning plan, so as to maximize long-term knowledge retention, improve learning efficiency and prevent cognitive overload.

[0004] (II) Technical Solution To address the above problems, this invention provides an adaptive learning plan dynamic adjustment system based on reinforcement learning, comprising: A multimodal data acquisition module is used to collect interactive data from the terminal; The learner digital twin construction module is used to receive the interaction data and output the digital twin state vector; A hierarchical reinforcement learning decision-making module is used to receive the digital twin state vector and generate an activity instruction vector. The learning plan generation and execution module is used to receive the activity instruction vector and issue learning tasks.

[0005] Furthermore, the interaction data includes interaction behavior data, physiological signal data, and learning scenario data.

[0006] Furthermore, the learner digital twin building block includes a cognitive diagnosis layer, a forgetting simulation layer, a cognitive load layer, and an emotional layer.

[0007] Furthermore, the cognitive diagnostic layer includes a knowledge point embedding module, an interactive feature encoding module, a knowledge state decoding module, and an explicit diagnostic projection module.

[0008] Furthermore, after receiving the digital twin state vector, the hierarchical reinforcement learning decision module follows a preset priority rule: emergency rest protection. Forgotten review New knowledge is learned, and activity instruction vectors are output.

[0009] Furthermore, digital twin state vectors From the knowledge state probability vector k t Extract the intensity vector r i Total cognitive load (CL) t Output valence V t Awakeness t The time interval vector Δt between the last review of each knowledge point review Current time period code c time It is pieced together; .

[0010] Furthermore, the activity command vector Index by knowledge points Activity Types Difficulty Level Study time Micro-rest sign It is pieced together; .

[0011] Furthermore, the terminal is equipped with a display screen; the interactive interface on the display screen includes: a personal information completion interface, a learning plan setting interface, and a learning mode selection interface; the personal information completion interface includes information on the subject to be selected, learning stage information, and recent exam results information; the learning plan setting interface includes information on recommended class hours to be selected, self-selected class hours information, textbook version information, and knowledge point chapter information; the learning mode selection interface includes information on the learning mode to be selected and learning difficulty information.

[0012] Furthermore, the interactive interface also includes a learning interface; the learning interface contains video recommendation information for course learning, post-class exercise recommendation information, and video segment information broken down from each of the video recommendation information.

[0013] (III) Beneficial Effects The above-described technical solution of the present invention has the following advantages: This invention constructs a learner digital twin comprising four layers: cognitive diagnosis, forgetting simulation, cognitive load, and emotion. This twin is then linked in a closed-loop manner with an adaptive decision-making module based on priority rules. This enables the real-time generation of learning activity instructions that balance long-term knowledge retention, immediate mastery, and cognitive-emotional states without relying on large amounts of training data. The system fully considers real-world learning bottlenecks such as memory decay, cognitive overload, and negative emotions. It can dynamically adjust learning content, difficulty, and pace, effectively improving learning efficiency, reducing the risk of cognitive overload, and enhancing learning motivation. The overall solution is simple in structure, responsive, and easy to deploy. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The terminal is equipped with a display screen; the interactive interface on the display screen includes: a personal information completion interface, a learning plan setting interface, and a learning mode selection interface; the personal information completion interface includes information on the subject to be selected, learning stage information, and recent exam scores; the learning plan setting interface includes information on recommended class hours, self-selected class hours, textbook version information, and knowledge point chapter information; the learning mode selection interface includes information on the learning mode to be selected and learning difficulty information. The interactive interface also includes a learning interface; the learning interface includes recommended video information for course learning, recommended after-class practice exercises, and video segment information broken down from each of the recommended video items.

[0016] In one alternative embodiment, the multimodal data acquisition module can collect three types of data from the terminal for subsequent processing: Interactive behavior data may preferably include the correctness of answers, response time, number of requests for prompts, learning resource type switching records, and operation records; operation records include pause and exit operations.

[0017] Physiological signal data, preferably obtained through cameras or wearable devices, includes eye movement patterns, pupil diameter changes, EEG attention index, and heart rate variability. Learning context data may preferably include learning time period (e.g., morning, late at night), device type, ambient noise level, and preceding activity tags (e.g., having just completed physical exercise).

[0018] After being anonymized and time-aligned, the aforementioned data is sent in chronological order to the learner's digital twin construction module and the hierarchical reinforcement learning decision-making module.

[0019] The learner digital twin building block is used to simulate, with high fidelity, the knowledge state, memory dynamics, cognitive load, and emotional changes of real learners in a virtual environment. It can construct a virtual student model in the cloud that evolves synchronously with the real learner, including a cognitive diagnosis layer, a forgetting simulation layer, a cognitive load layer, and an emotional layer. It outputs a digital twin state vector S in a unified format. t This provides the hierarchical reinforcement learning decision-making module with the environment state as input.

[0020] The cognitive diagnostic layer is used to transform learners’ explicit interactive behaviors into implicit knowledge state representations in real time.

[0021] It includes a knowledge point embedding module, an interactive feature encoding module, a knowledge state decoding module, and an explicit diagnostic projection module.

[0022] This system pre-defines a subject knowledge graph containing K knowledge points. The knowledge point embedding module generates a learnable knowledge point embedding vector e for each knowledge point i (i=1, ..., K) in the acquired subject knowledge graph. i ∈R 64 The dimension is set to 64. The knowledge point embedding vector e i Obtained through graph pre-training (such as the TransE algorithm), it preserves the structural relationships such as prerequisites and successors between knowledge points.

[0023] At time t, the learner completes an interactive activity and learns knowledge point k. The interaction feature encoding module encodes this interactive activity into an interaction feature vector x. t ∈R 71 The dimension is set to 71.

[0024] x t =Concat(e k,t a t r t d t ); in: Embedding target knowledge points e k,t ∈R 64 : The 64-dimensional vector of the target knowledge point k for this interactive activity.

[0025] Activity type unique hot code a t ∈R 5 The pre-set activities include five categories: passive video viewing, active text reading, low-level multiple-choice tests, high-level short-answer tests, and review of incorrect answers, i.e., a t It is a 5-dimensional vector.

[0026] scalar r of the answer result t ∈[0-1]: For test-type activities, the actual score rate is used; for viewing-type activities, the attention estimate is used. The attention estimate is derived from the physiological signal module; if no such module is available, 0.8 is used.

[0027] Duration feature d t =log(t duration +1), where t duration The duration in minutes of the activity.

[0028] Therefore, the concatenated interaction feature vector x t The dimensions are 64+5+1+1=71.

[0029] The knowledge state decoding module uses a gated recurrent unit (GRU) to decode the knowledge state hidden vector h from the previous time step at time t. t 1 and the interaction feature vector x of the current input t Calculate the knowledge state hidden vector h at the current time step. t ∈R 128 This 128-dimensional vector represents the "knowledge state" at time t.

[0030] Specifically, when the input interaction feature vector x t back: Get the reset gate vector R t =σ(W r ·[h t 1,x t ]+b r ); hide the knowledge state vector h from the previous time step. t 1. Interaction feature vector x with the current input t The vectors are concatenated along the row direction to form a mixed vector [h] of length 199. t 1,x t ]. R t The value range is (0,1), which controls the hidden vector h of the knowledge state at the previous time step. t 1. The degree of preservation of each dimension.

[0031] Get the updated gate vector Z t =σ(W z [h t 1,x t ]+b z Update the gate vector Z. t The final output is the hidden vector h of the knowledge state at the current moment.t At that time, what percentage of the knowledge state hidden vector h is directly inherited from the previous time step? t 1. What percentage of the vector is replaced with the new candidate state vector h̃? t .

[0032] Obtain the candidate state vector h̃ t =tanh(W h ·[R t ⊙h t 1,x t ]+b h Generate a new vector containing current new knowledge and filtered historical knowledge, with all values ​​compressed to the (-1, 1) interval by tanh.

[0033] The knowledge state hidden vector h at the current time step t =(1 Z t )⊙h t 1+Z t ⊙h̃ t When Z t When a certain dimension is close to 0, the state of that dimension remains almost unchanged; when it is close to 1, it is almost completely replaced by a new candidate state. This mechanism enables the network to retain important knowledge for a long time while being highly sensitive to relevant changes.

[0034] Where σ is the sigmoid function, tanh is the hyperbolic tangent function, and R... t To reset the gate vector, Z t To update the gate vector.

[0035] W r W z W h Here, h is the weight matrix, and h is the trainable parameter used for mixing. t 1 and x t [h] t 1,x t [] is the concatenation vector, which concatenates the 128-dimensional and 71-dimensional vectors at the end in the row direction.

[0036] B r b z b h It is a paranoia vector and also a trainable parameter.

[0037] This update mechanism allows h to automatically determine how much old knowledge to forget and how many new features to remember.

[0038] In the first interaction time (t=0), since there is no previous state, the system initializes h0 as a 128-dimensional vector with all zeros.

[0039] The explicit diagnostic projection module is used to hide the knowledge state vector h at time t. t This is reduced to a physically interpretable "probability of mastery of each knowledge point". The knowledge state hidden vector h at the current moment is... t Mapping to the knowledge point space via bilinear projection.

[0040] For each knowledge point i (i=1, ..., K) in the subject knowledge graph, calculate h. t Embedded with each knowledge point in a vector e i The matching degree between them, that is, the probability of mastering each knowledge point i: p i,t =Sigmoid(h·M·e i +b proj ) Where p i,t Let p be the probability of mastering the i-th knowledge point at time t. i,t A value close to 1 indicates that the student has likely mastered knowledge point i at this point. i,t A value close to 0 indicates that the student has most likely not mastered knowledge point i at this point.

[0041] M∈R 128×64 This is a bilinear projection matrix that maps a 128-dimensional knowledge state space to a 64-dimensional knowledge point embedding space. For example, a column of M might correspond to "algebraic computation ability," while the knowledge point embedding vector e... i A particular row may correspond to "the degree of algebraic operations required", and the inner product of the two precisely captures the match between the student's ability and the knowledge requirements.

[0042] b proj It is a scalar bias; h∈R 1×128 for h t Transpose of; Output vector: Traverse i=1,...,K to obtain the knowledge state probability vector k at time t. t =[p 1,t ,...,p K,t ].

[0043] As a preferred approach, the cognitive diagnostic layer needs to be trained in advance using offline historical data. After training is completed, all parameters are frozen, and only the forward inference function is retained.

[0044] The forgetting simulation layer is based on the knowledge state probability vector k output by the cognitive diagnostic layer. tIt simulates the dynamic changes in storage and retrieval strength in the human memory system and can predict the knowledge retention state at any future time, providing a basis for long-term reward calculation in reinforcement learning.

[0045] A two-component memory model is adopted. For each knowledge point i (i=1, ..., K), the following variables are defined: Storage strength si∈[0,+∞) Extraction intensity r i ∈[0,1] Initialize s i =0, r i =0.

[0046] When the system experiences a learning event at time t, this event is based on the target knowledge point set I. event and event intensity factor e=λ1·r t +λ2·1 正确测试 This is characterized by achieving a dual enhancement of "high score" and "successful extraction." Among them, r t ∈[0,1] represents the scalar of the answer result, λ1=0.8, λ2=0.2. For each knowledge point i∈I event Perform a synchronous update: s i ←s i +αe(1 s i / (1+si);makes s i The larger the value, the slower the growth rate, simulating the diminishing marginal returns of memory consolidation, while keeping si itself unbounded.

[0047] r i ←r i +βe(σ(s i ) r i ); where σ(s) i ) is the unbounded storage strength s i The "ideal retrieval strength" mapped to the (0,1) interval means that the stronger the storage, the higher the theoretically achievable upper limit of the retrieval strength. This indicates that r... i It will move toward this target value, with the step size controlled by the event intensity factor e and the extraction gain β.

[0048] Where α=0.15 is the storage learning rate, controlling the magnitude of the increase in storage strength per learning event. β=0.2 is the extraction gain, controlling the speed at which the extraction strength converges to the storage strength. σ(·) is the Sigmoid function.

[0049] Within a time interval Δt during which no learning events occur, the retrieval intensity of all knowledge points decays independently according to an exponential law, while the storage intensity remains unchanged: r i ←r i ·exp( γΔt(1 σ(s i ))) Where: γ = 0.03 / hour, is the base decay rate, controlling the rate of intensity exponential decay when no events occur. σ(s) i ) is σ(x) = 1 / (1 + e x e is the natural constant. σ(s) i The larger the value, the slower the decay, simulating the protection effect of high storage strength.

[0050] The actual knowledge availability of a knowledge point at the current moment (u) i Let p be the probability of mastering the i-th knowledge point at time t. i，t With extraction intensity r i Fusion: r i (t+Δt)=r i (t)·e γ·Δt·(1 σ(si)) The decay rate is determined by both the fundamental decay factor γ and the protective effect of the storage strength. σ(s) i When σ(s) approaches 1, the decay exponent approaches 0, and the decay is extremely slow; i When the value approaches 0, it decays at the maximum rate of γ. This demonstrates the protective effect of high storage intensity on extraction intensity.

[0051] The forgetting simulation layer receives the knowledge state probability vector k output by the cognitive diagnostic layer. t Perform calibration and synchronization. The actual knowledge availability of knowledge point i at time t (u) i for: u i (t)=κ·p i,t +(1 κ)·r i Where κ=0.7, it ensures automatic prediction of availability decay during long periods of no interaction. i,t Let be the probability of mastering the i-th knowledge point in the cognitive diagnostic layer at time t.

[0052] The cognitive load layer estimates the learner's total cognitive load (CL) in real time. t And determine whether it is overloaded.

[0053] The intrinsic workload (IL) is determined by both the information complexity of the current learning content and the learner's level of mastery.

[0054] in, =0.6, indicating that the intrinsic load is high when the average mastery probability is low.

[0055] External load (EL) is caused by the activity type. Preset base values: passive video viewing 0.2, active text reading 0.3, low-level multiple-choice test 0.4, high-level short-answer test 0.6, incorrect answer review 0.5. Additional values ​​of 0~0.2 can be added based on UI complexity and environmental interference (provided by the multimodal data acquisition module).

[0056] The relevant load GL is estimated from intrinsic motivation indices: GL=0.3·IM t Among them IM t It is a scalar of intrinsic motivation.

[0057] Dynamic total load: Original total load CL raw =IL+EL η GL, η=0.4 After time smoothing, accumulation and dissipation are simulated using differential equations.

[0058]

[0059] Where μ = 0.3, ν = 0.1, and Δt is the step size (minutes).

[0060] When CLt At a value of 0.8, cognitive overload is identified. This status signal will be used for reward / penalty and policy action masking.

[0061] The affective layer simulates the learner's emotional changes and outputs a valence Vt∈[ 1,1] and arousal At∈[0,1], and estimate intrinsic motivation IM. t The emotional layer employs a simplified linear dynamic system, updated based on prior emotional states and the impact of current events:

[0062]

[0063] in δV=δA=0.05 This represents the regression neutral trend coefficient. The event impact ΔV and ΔA are determined based on the current event type and outcome.

[0064] High accuracy test: ΔV = +0.1, ΔA = +0.05 Continuous errors: ΔV= 0.15, ΔA = 0.05 Long-term high load CL 0.7, lasting for more than 10 minutes: ΔV= 0.1; ΔA = +0.1 (frustration accompanied by anxiety). In common knowledge within this field, this is achieved through a predefined event impact mapping table, calibrated using extensive experimental data in educational psychology.

[0065] Intrinsic Motivation Scalar (IM) t Reflecting learners' curiosity and intrinsic motivation:

[0066] Among them, novelty t The average semantic distance between the embedding vectors of the current knowledge point and the knowledge points already mastered is w1=1.0, w2=0.5, w3=0.6.

[0067] The digital twin state vector S at time t t The outputs of the above layers are spliced ​​together and serve as the environmental input for the hierarchical reinforcement learning decision-making module. This fully describes the learner's immediate and historical state at the cognitive and emotional levels, enabling the policy network to make adaptive decisions with a long-term perspective.

[0068] Specifically, this includes: knowledge state probability vector k t Extract the intensity vector r i Total cognitive load (CL) t Output valence V t Awakeness t Δt is the vector of time intervals (in hours) since the last review of each knowledge point. review Current time period code c time (Encode the hours of the day using sine / cosine to preserve periodicity) The hierarchical reinforcement learning decision module obtains the digital twin state vector generated by the learner's digital twin construction module. As input vector, output activity command vector

[0069] Knowledge Point Index The values ​​{1, ..., K} represent the target knowledge points, where each knowledge point i (i=1, ..., K) in the subject knowledge graph. Activity type ={1,2,3,4,5}, which correspond to passive video viewing, active text reading, low-level multiple choice test, high-level short answer test, and review of incorrect questions, respectively; Difficulty level ={1,2,3,4,5}, corresponding to the expected matching probability {0.2,0.4,0.6,0.8,1.0}; Study time = {1,2,3,4}, which corresponds to {2,5,8,12} minutes.

[0070] Micro-rest sign ={0,1}, where 1 indicates that a 1-minute break will be inserted after the activity.

[0071] The decision is executed in order of priority. Once the condition is met, the corresponding action is output and no further judgments are made.

[0072] Priority 1 Emergency Rest Protection: When experiencing cognitive overload or severe negative emotions, force yourself to do 2 minutes of low-difficulty reading and rest, waiting for your condition to recover.

[0073] Condition: CL t 0.8 or V t 0.5 Output =1, =2, the rest will automatically take the default value.

[0074] Priority 2: Reviewing what has been forgotten Condition: There exists a knowledge point i, whose Δt review ·(1-r i ) 2.0 hours Output selects those that meet the conditions. (1-r) i The knowledge point with the highest value is used as

[0075] =3 Take p i,t Rounded to the nearest difficulty level =2+10(1-r i Round up to the nearest value in the {2, 5, 8, 12}. =0 Therefore, knowledge points with lower retrieval intensity and longer intervals are reviewed first, and the review time increases with the degree of forgetting.

[0076] Priority 3: Learning new knowledge: When intrinsic motivation is high, use tests to stimulate the desire for challenge; when intrinsic motivation is low, use videos to lower the cognitive threshold, thus starting with the weakest knowledge.

[0077] Condition: None of the above are met Output: Select p i,t Focus on mastering the knowledge points with the lowest values, especially the weakest ones.

[0078] If IMt 0.5, Choose to actively read the text, take a low-level multiple-choice test, or take a high-level short-answer test; otherwise, choose to passively watch the video.

[0079] Take p i,t Round to the nearest difficulty level, but not exceeding 0.6, to avoid making it too difficult.

[0080] =2+6(IMt) rounded up to the nearest value in {2,5,8,12}; =0.

[0081] The learner digital twin construction module receives interactive data from the multimodal data acquisition module. After processing through the cognitive diagnosis layer, forgetting simulation layer, cognitive load layer, and emotional layer, it outputs the digital twin state vector S. t The hierarchical reinforcement learning decision-making module receives a digital twin state vector S. t According to the priority rules: emergency rest protection, forgetting review, and new knowledge learning, the activity instruction vector 'a' is directly calculated and output to the learning plan generation and execution module.

[0082] The learning plan generation and execution module receives activity instruction vector 'a', selects corresponding content from the resource library based on the target knowledge points in the instruction, and executes the activity according to the activity type specified in the instruction. Difficulty Level and study time The tasks are combined into specific learning assignments and then pushed to the learner's device. Activity Type Difficulty Level and study time The correspondence between learning tasks and learning content can be pre-set in the resource library by subject matter experts, teachers, and other personnel.

[0083] Through the collaborative work of the above modules, this system can dynamically schedule a complete schedule including learning, review, testing and rest based on each learner's real-time status, with the goal of maximizing long-term knowledge retention, while strictly preventing cognitive overload and negative emotions, thus achieving a truly adaptive dynamic adjustment of the learning plan.

[0084] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The storage medium embodiments for providing the program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network. Furthermore, it should be understood that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this specification, and are not intended to limit them. Although this specification has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this specification.

Claims

1. A dynamic adjustment system for adaptive learning plans based on reinforcement learning, characterized in that, include: A multimodal data acquisition module is used to collect interactive data from the terminal; The learner digital twin construction module is used to receive the interaction data and output the digital twin state vector; A hierarchical reinforcement learning decision-making module is used to receive the digital twin state vector and generate an activity instruction vector. The learning plan generation and execution module is used to receive the activity instruction vector and issue learning tasks.

2. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 1, characterized in that, The interaction data includes interaction behavior data, physiological signal data, and learning scenario data.

3. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 1, characterized in that, The learner digital twin building block includes a cognitive diagnosis layer, a forgetting simulation layer, a cognitive load layer, and an emotional layer.

4. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 3, characterized in that, The cognitive diagnostic layer includes a knowledge point embedding module, an interactive feature encoding module, a knowledge state decoding module, and an explicit diagnostic projection module.

5. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 1, characterized in that, After receiving the digital twin state vector, the hierarchical reinforcement learning decision module follows a preset priority rule: emergency rest protection. Forgotten review New knowledge is learned, and activity instruction vectors are output.

6. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 1, characterized in that, Digital twin state vector From the knowledge state probability vector k t Extract the intensity vector r i Total cognitive load (CL) t Output valence V t Awakeness t The time interval vector Δt between the last review of each knowledge point review Current time period code c time It is pieced together; 。 7. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 6, characterized in that, Activity command vector Index by knowledge points Activity Types Difficulty Level Study time Micro-rest sign It is pieced together; 。 8. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 1, characterized in that, The terminal is equipped with a display screen; the interactive interface on the display screen includes: a personal information completion interface, a learning plan setting interface, and a learning mode selection interface; the personal information completion interface includes information on the subject to be selected, learning stage information, and recent exam results information; the learning plan setting interface includes information on recommended class hours to be selected, self-selected class hours information, textbook version information, and knowledge point chapter information; the learning mode selection interface includes information on the learning mode to be selected and learning difficulty information.

9. The adaptive learning plan dynamic adjustment system based on reinforcement learning according to claim 8, characterized in that, The interactive interface also includes a learning interface; the learning interface contains video recommendation information for course learning, post-class exercise recommendation information, and video segment information broken down from each of the video recommendation information.

Images