Individualized language learning training platform fused with reinforcement learning
By integrating reinforcement learning into a personalized language learning training platform, multi-dimensional interactive data is collected and integrated in real time to generate personalized practice questions. This solves the problem of insufficient accuracy in learning path adaptation in existing systems, and achieves precise adaptation of learning paths and improved learning efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 晏萍
- Filing Date
- 2026-03-06
- Publication Date
- 2026-05-22
AI Technical Summary
Existing personalized language learning systems struggle to perceive learners' overall state during the language learning process in a real-time and comprehensive manner, resulting in insufficient accuracy in adapting learning paths.
A personalized language learning and training platform integrating reinforcement learning is constructed. The platform collects multi-dimensional interactive data in real time through a state awareness module, integrates the data using a state encoding module to generate personalized practice questions, optimizes the practice generation strategy through a reinforcement learning decision-making module, and updates the strategy in conjunction with a reward calculation module.
It enables a comprehensive perception of learners' real-time cognitive activities and emotional states, generating precise and personalized practice questions, thereby improving the accuracy of learning path adaptation and learning efficiency.
Smart Images
Figure CN122072935A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of learning systems technology, and more specifically, to a personalized language learning and training platform that integrates reinforcement learning. Background Technology
[0002] Currently, educational informatization has developed significantly, and the application of artificial intelligence technology in the field of learning assistance is becoming increasingly widespread. Personalized learning aims to provide different learners with learning content and paths tailored to their abilities and states, and is one of the key directions for improving learning efficiency. However, existing personalized learning technologies still have several shortcomings. A common deficiency is that systems often focus on mining learners' knowledge weaknesses from static historical data and then selecting and pushing learning resources accordingly, lacking dynamic perception and response to learners' real-time cognitive processes, emotional states, and metacognitive abilities.
[0003] Specifically, existing patent technologies, such as patent CN113190761A, disclose a personalized learning system based on deep reinforcement learning. This system uses reinforcement learning to recommend learning content, but its focus is on matching and filtering content from existing learning resource libraries. It fails to address the fundamental problem of how to dynamically synthesize new, personalized practice content based on the learner's complex real-time state (such as cognitive load, problem-solving strategies, and confidence levels). Another example is patent CN119830013A, which involves using reinforcement learning to optimize the generation of prompts for large language models. Although this method introduces personalized user information, its application scenarios and goals are limited to optimizing text prompts to obtain better responses from large language models. It does not involve modeling the learner's multidimensional state (especially non-textual interaction behavior and cognitive state), nor does it construct a complete, closed-loop language learning training process from state perception to content generation to effect feedback. Therefore, a personalized language learning and training platform integrating reinforcement learning is proposed to address the above issues. Summary of the Invention
[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a personalized language learning training platform that integrates reinforcement learning, in order to solve the problem that the prior art is unable to perceive the learner's overall state in the language learning process in real time and comprehensively, and automatically generate the most matching personalized practice questions based on this dynamic state, resulting in insufficient accuracy in the adaptation of the learning path.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a personalized language learning training platform integrating reinforcement learning, comprising: The state awareness module collects and outputs learners' interaction data in real time. The interaction data includes correct and incorrect answer indicators, answer response time, speech fluency features or text input fluency features, interface interaction behavior sequence, and confidence score, thereby obtaining raw information reflecting learners' performance and state. The state encoding module, connected to the state perception module, receives the interaction data and processes it through a model containing a first feature network, a second feature network, and a third feature network. The first feature network outputs a first state vector representing knowledge mastery, the second feature network outputs a second state vector reflecting skill and cognitive state, and the third feature network outputs a third state vector reflecting metacognitive state. The state encoding module also includes a fusion layer that performs weighted fusion of the first state vector, the second state vector, and the third state vector to output a learner state vector representing the learner's overall state. The reinforcement learning decision module is connected to the state encoding module, receives the learner's state vector, performs analysis and decision-making based on the internal policy network, and outputs action instructions for defining the attributes of the next exercise question. The action instructions include target knowledge point identifier, question type identifier, interference intensity level, and context topic identifier. The practice generation module is connected to the reinforcement learning decision module. It receives the action instruction and, based on the target knowledge point identifier, question type identifier, interference intensity level, and context theme identifier, retrieves matching content elements from the preset material library and template library and dynamically assembles them to generate and output a personalized language practice question that meets the instruction requirements. The reward calculation module, connected to the state perception module and the state encoding module, receives the new round of interaction data and the corresponding new learner state vector, calculates and outputs the reward value based on preset rules; the reward value calculation rules are as follows: a basic reward value is determined based on the correctness or incorrectness of the answers in the new round, a skill reward value is determined based on the load index contained in the second state vector of the new learner state vector, a metacognitive reward value is determined based on the calibration scalar contained in the third state vector of the new learner state vector, and the basic reward value, skill reward value and metacognitive reward value are added together to obtain a comprehensive reward value; The reinforcement learning decision module is also connected to the reward calculation module, and receives the reward value as a feedback signal for policy optimization, which is used to update the parameters of its internal policy network. The state perception module, state encoding module, reinforcement learning decision module, practice generation module, and reward calculation module are sequentially connected to form a closed loop, enabling the platform to continuously optimize the practice question generation strategy based on learners' real-time feedback.
[0006] Furthermore, the state awareness module collects the sequence of interface interaction behaviors, specifically including: recording cursor movement trajectory, focus switching events, and text selection and deselection operations, in order to capture the learner's refined interaction patterns during the question-answering process.
[0007] Furthermore, in the state encoding module, the input of the second feature network includes a load index, which is calculated based on the hovering time in the option area of the cursor movement trajectory, the number of times the focus switching event occurs, and the time interval between adjacent selection and deselection operations, and is used to quantify the cognitive load level faced by learners when answering questions.
[0008] Furthermore, the input to the second feature network also includes a strategy identifier, which is obtained by sequence matching of the interface interaction behavior sequence with a pre-stored strategy template, and is used to identify the problem-solving strategy type adopted by the learner.
[0009] Furthermore, the state perception module collects the speech fluency features specifically by: extracting the total duration of effective speech segments, the number of pauses, and the number of word repetitions from the learner's speech audio, and calculating a fluency coefficient based on the above extracted items to evaluate the fluency of spoken expression.
[0010] Furthermore, the state-aware module collects the text input fluency features specifically by recording the average character input rate and the frequency of backspace key operation, and calculates a text fluency coefficient based on the above-mentioned records to evaluate the fluency of written expression.
[0011] Furthermore, in the state encoding module, the input to the second feature network also includes the fluency coefficient or the text fluency coefficient, modeling expressive fluency as a component of skill state.
[0012] Furthermore, in the state encoding module, the first feature network is a deep knowledge tracking model, whose inputs include the correct / incorrect answer identifier, the multidimensional vector of associated knowledge points, and the normalized answer response time. This model is used to sequentially predict the learner's mastery probability of each knowledge point.
[0013] Furthermore, in the state coding module, the input of the third feature network includes the confidence value, and its output includes a calibration scalar. The calibration scalar is calculated based on multiple historical sets of the confidence values and the corresponding correct / incorrect answer labels, and is used to measure the accuracy of the learner's self-assessment.
[0014] Furthermore, the method of operating the platform includes the following steps: Step S1: Collect the learner's interaction data after completing the current exercise through the state awareness module; Step S2: Process the interaction data through the state encoding module, extract and fuse features to obtain the current learner state vector; Step S3: The reinforcement learning decision module analyzes the current learner state vector and outputs an action command. Step S4: The exercise generation module retrieves and assembles content according to the action instructions, generates new language practice questions, and pushes them to the learner. Step S5: After the learner completes the new exercise, a new round of interaction data is collected through the state awareness module; Step S6: Process the data collected in step S5 through the state coding module to obtain the new learner's state vector; Step S7: Calculate the reward value using the reward calculation module, based on the calculation rules described in claim 1, the data collected in step S5, and the new learner's state vector. Step S8: Use the current learner state vector, the action instruction, the reward value, and the new learner state vector to construct training data, update the internal policy network of the reinforcement learning decision module, and complete one policy optimization loop.
[0015] The technical effects and advantages of this invention are as follows: First, this invention, by constructing a multi-layered state-awareness module, can collect multi-dimensional data such as learners' correctness of answers, reaction time, interface interaction sequences, speech or text fluency characteristics, and confidence levels. This data collection method helps to more comprehensively depict learners' real-time cognitive activities and emotional states during language practice, providing a rich and objective data foundation for subsequent personalized decision-making, and overcoming the limitations of relying solely on a single dimension such as answer accuracy.
[0016] Second, the state encoding module designed in this invention employs a parallel feature network and attention fusion mechanism. This module can fuse collected heterogeneous data, such as the knowledge point mastery probability obtained through a deep knowledge tracking model, the cognitive load index and strategy identifier calculated through behavioral analysis, and the calibration degree reflecting metacognition, into a unified learner state vector. This fusion method not only preserves information from various features but also dynamically emphasizes the most important state dimension through attention weights, thereby constructing a feature representation that better reflects the learner's complex and dynamic internal state, providing a reliable basis for generating accurate practice instructions.
[0017] Third, this invention utilizes a reinforcement learning decision module, taking the learner's state vector as input and directly outputting structured action instructions (including knowledge point emphasis, question type, difficulty, and theme) to guide the generation of practice questions. This method models the decision-making process for generating practice questions as a sequential decision problem. The system instantiates instructions through the practice generation module, and after the learner completes the practice, the reward calculation module calculates a comprehensive reward based on feedback from multiple aspects, including the learner's workload level, correctness of answers, and self-calibration in the new round of interaction. This reward signal drives the update of the reinforcement learning strategy, enabling the system to autonomously learn and optimize its generation strategy through continuous interaction with the learner. Attached Figure Description
[0018] Figure 1 This is a flowchart illustrating the overall closed-loop workflow of the system according to the present invention.
[0019] Figure 2 This is a flowchart of the internal processing of the status coding module of the present invention.
[0020] Figure 3 This is a branch structure diagram of the multi-dimensional evaluation of the reward calculation module of the present invention.
[0021] Figure 4 This is a flowchart illustrating the strategy optimization closed loop and system evolution of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Example 1 As attached Figures 1 to 4 The implementation details of the personalized language learning training platform that integrates reinforcement learning, as shown, are as follows: 1. Implementation of the State Awareness Module The state-aware module captures learners' interaction data in real time, as follows: (1) Record the correct and incorrect answers. and reaction time . Values (Represents correctness) or (This represents an error). The time from when the question is presented to when the answer is submitted, in milliseconds (ms). The system... Perform preprocessing: If Operations shorter than 100ms are considered invalid and ignored; operations longer than 30000ms are truncated to 30000ms.
[0024] (2) For oral practice, speech fluency features are collected. Audio is processed using a speech activity detection model to extract: total duration of speech segments. (Unit: seconds), Number of effective pauses (Number of times a silent segment lasts longer than 300ms), Number of times words are repeated .
[0025] Calculate the speech fluency coefficient The calculation formula is as follows: ; in, and As a preset coefficient, in this embodiment , . The range of values is .
[0026] (3) For text input practice, collect text fluency features. Calculate: average input rate. (Unit: characters / second), backspace (or delete operation) frequency (The calculation formula is: the number of times the backspace key is pressed divided by the total number of characters entered).
[0027] Calculate the text fluency coefficient The calculation formula is as follows: ; in, As a preset coefficient, in this embodiment . The range of values is .
[0028] (4) Collect the sequence of interactive behaviors on the interface. Record: cursor (or touch point) movement trajectory. Focus switching events Text selection event Select Cancel Event Original trajectory After sampling and normalization, it is converted into a sequence of relative coordinates relative to the upper left corner of the answer interface.
[0029] (5) Collect confidence level values By using the slider (range) before submission. to ) to obtain, The value is a floating-point number within the interval [0, 1]. If the learner does not interact with the slider, the default value will be used. .
[0030] 2. Implementation of the Status Coding Module The state encoding module encodes the interaction data into learner state vectors. This module contains three parallel feature extraction subnetworks. , , and a fusion layer .
[0031] 2.1 First Feature Network Implementation of (knowledge state extraction). It is a deep knowledge tracking model that can receive historical learning sequences and current answer information, and output a prediction of the probability of mastering knowledge points.
[0032] At each time step, The input consists of three parts: the current correct or incorrect answer indicator. , indicating the knowledge points tested in the question. dimensional vector ( The total number of knowledge points defined in the system, for example, 128; (Using one-hot encoding or embedding representation) and normalized reaction time Its calculation formula is .
[0033] Internally maintain a hidden state It is used to remember historical sequence information. Based on the current input and the implicit state of the previous moment The current hidden state is updated through its internal sequence model (such as using LSTM or GRU units). and output the first state vector. . It is A dimensional vector, whose dimensional vector is the first dimensional vector. The value of the nth element is in the interval [0, 1], indicating that the model predicts the learner's response to the nth element. The degree of mastery of each knowledge point. For new learners, Initialize to a zero vector.
[0034] 2.2 Second Feature Network Implementation of (skill status extraction). Processing data related to learning skills and cognitive processes.
[0035] (1) Calculate the load index Based on the sequence of interface interaction behaviors Calculate three intermediate quantities: average hover time within the option area. (Unit: seconds), within a unit of time (e.g., 60 seconds) Number of times the event occurred , The event and its adjacent Average time interval between events (Unit: seconds). Load Index The calculation formula is: ; in, This represents the Sigmoid activation function; , , , The preset parameters are obtained by training a logistic regression model on a batch of sample data collected and labeled in a controlled experimental environment (labeling methods may include "high / low cognitive load" labels based on eye tracking, self-report, or expert observation). The range of values is A higher value indicates a higher cognitive load.
[0036] (2) Calculation strategy identifier The system predefines standard interactive event encoding sequences corresponding to several typical problem-solving strategies (such as "grammar-first" and "context-first"). , .
[0037] Current interaction sequence Transformed into the same form of event encoding sequence (For example, discretizing continuous trajectories into direction codes and mapping event types to numbers). Calculations are performed using a dynamic time warping algorithm. and , distance , Set distance threshold (e.g., 5.0), if and ,but ;like and ,but ;otherwise, .
[0038] (3) input vector Depend on , and fluency coefficient (or (Selected according to the question type) is pieced together. It is input into a multilayer perceptron.
[0039] The structure of this multilayer perceptron is as follows: Input layer (dimension determined by...) The multilayer perceptron consists of a first hidden layer (64-dimensional, using ReLU activation function), a second hidden layer (32-dimensional, using ReLU activation function), and an output layer (32-dimensional, linear output). The output of this multilayer perceptron is the second state vector. .
[0040] 2.3 Third Feature Network Implementation of (metacognitive state extraction). Process learners' metacognitive information.
[0041] (1) Maintain a fixed capacity (e.g., 10) historical data queue Used to store the most recent Data pairs. Each time a new data pair is generated, the oldest data pair will be used. Remove data from the queue and pair the new data. Add to the queue. The queue is empty at initialization.
[0042] (2) Calculate the calibration scalar If the queue If the queue is not empty, then retrieve all historical data from the queue, calculated using the following formula: ; in, This represents calculating the arithmetic mean. If the queue is empty, then let... . The range of its value is [0, 1].
[0043] (3) Determine the risk indicators of setbacks Definition rule: If continuous Next (for example) (times) practice satisfies ( (For example, a load threshold, such as 0.7) and If so, it is determined that there is a high risk of setbacks. Otherwise, let .
[0044] (4) input vector Depend on and It was pieced together. The input is fed into a multilayer perceptron. The structure of the multilayer perceptron is as follows: input layer ( The multilayer perceptron consists of a first hidden layer (16-dimensional, using the ReLU activation function), a second hidden layer (16-dimensional, using the ReLU activation function), and an output layer (16-dimensional, linear output). The output of this multilayer perceptron is the third state vector. .
[0045] 2.4 Fusion Layer The implementation of. Receive output vectors from three subnetworks , , First, concatenate the three elements into a single long vector. Then calculate the attention weights: ; in, Let be a learnable parameter matrix with dimension . ; The bias vector is a learnable vector with dimension . ; Function ensures .
[0046] Final learner state vector We get the result from the weighted sum: ; Dimensions and Same, for dimension.
[0047] 3. Strengthen the implementation of the learning decision-making module This module contains an agent employing a proximal policy optimization algorithm, whose policy network... The input is a state vector Policy Network Output action commands , It is a structured parameter vector used to guide the generation of the next exercise. It includes the following four parts: :one A weight vector of dimension 1, used to indicate which knowledge points the recommendation algorithm should focus on. Each element value is normalized using the softmax function, so that its sum is 1. .
[0048] : A discrete question type category identifier, for example, represented by one-hot encoding as one of [1, 0, 0] (multiple choice), [0, 1, 0] (fill in the blank), [0, 0, 1] (oral question).
[0049] : A floating-point number in the interval [0, 1], representing the expected level of interference or overall difficulty of the question.
[0050] : A discrete contextual topic identifier, for example, represented by one-hot encoding as [1, 0, 0] (daily life), [0, 1, 0] (academic discussion), [0, 0, 1] (business office).
[0051] Policy Network The structure is as follows: Input layer ( The system consists of a first hidden layer (256 dimensions, using the GELU activation function), a second hidden layer (128 dimensions, using the GELU activation function), and an output layer. The output layer is divided into four sub-parts, corresponding to... ( (Dimension, using the softmax activation function) ( (Dimension, using the softmax activation function) ( (dimensional, mapped to the [0, 1] interval using the Sigmoid activation function). ( (Use the softmax activation function).
[0052] The system according to and The output probability distribution is sampled to determine specific values; and In this implementation, the output value is used directly in the policy network. The parameters are initialized using a uniform distribution of Xavier parameters.
[0053] 4. Practice the implementation of the generation module This module receives action commands. Internally, it maintains two core databases: one is a richly annotated language resource database. (Each piece of material, such as a sentence, paragraph, or audio dialogue, is marked with a knowledge point vector.) Difficulty level Topic Vectors metadata, among which and The encoding method and the status encoding module and action commands (Consistent); the other is a question type template library. For each Fixed presentation and interaction logic has been defined. The implementation process is as follows: (1) Resource retrieval: based on In and In the language resource library Perform similarity retrieval within the context of [the system / process]. Calculate [the similarity]. Metadata vector of each material With the target vector Cosine similarity, return the top [number] with the highest similarity. Article (e.g.) (Materials as candidate set) And randomly select one as the base material. .
[0054] (2) Template instantiation: based on In From the template library Load the corresponding practice question template.
[0055] (3) Content assembly and difficulty adjustment: Assemble the materials Fill in the question stem or context section of the template. If The corresponding question type requires options (such as multiple choice questions), then according to Difficulty Filter out difficulty tags from the interference item library. The closest distractors are selected and, along with the correct answer, form a list of options. Finally, a complete, interactive exercise object is generated. And render it to the user interface.
[0056] 5. Implementation of the reward calculation module This module allows learners to complete newly generated practice questions. It is triggered after a new round of interaction data is generated. It can calculate a scalar reward value based on the data from this round. Used to evaluate decision-making actions The effect.
[0057] The input data required for the calculation includes: the correct / incorrect answer flags c' for the new round of questions, and the load index for the new round. (Depend on (Calculated based on new interactive data) New round of calibration scalar (Depend on Based on the updated queue (Calculated) Reward value The calculation formula is: ; in, , , These represent basic rewards, skill rewards, and metacognitive rewards, respectively. The specific calculations are as follows: ; ; ; In this embodiment, the coefficient ,coefficient Calculated It is sent to the reinforcement learning decision module.
[0058] 6. System closed-loop operation and strategy update process Based on the above modules, the platform forms a self-iterating reinforcement learning closed loop, and its single loop process is as follows: (1) Initial state: The learner completes a practice question, and the system collects the interaction data. .
[0059] (2) State awareness and coding: The state awareness module processes The status coding module is based on Calculate and output the current learner state vector. .
[0060] (3) Reinforcement learning decision-making: The policy network of the reinforcement learning decision-making module according to Sampling yields an action .
[0061] (4) Exercise generation and execution: The exercise generation module is based on... Generate new practice questions And present it to the learners.
[0062] (5) New feedback collection: learners complete The system collects new interactive data. .
[0063] (6) New state coding and reward calculation: State coding module processing Output the new learner state vector Simultaneously calculate and And pass it to the reward calculation module; the reward calculation module is based on c', , Calculate reward value .
[0064] (7) Experience storage and policy update: Store the experience tuples generated in this loop. Store it in a first-in-first-out (FIFO) experience playback buffer.
[0065] When the amount of data accumulated in the buffer reaches the set capacity (For example After that, the system will run at fixed time steps (or whenever a new...) One experience, for example Then, a random sample of size 1 from the buffer is taken. (For example Update strategies using small batches of data.
[0066] During updates, the policy network is updated using the proximal policy optimization algorithm. The parameters.
[0067] The key hyperparameter settings for the PPO algorithm are as follows: Discount factor Generalized dominance estimation parameters Cutting parameters Value function loss coefficient Entropy reward coefficient Learning rate .
[0068] This update process continues to maximize the expected value of future cumulative rewards. Training can terminate when the total number of training steps reaches a preset value (e.g., ...). (Step), or the policy performance converges on an independent validation set.
[0069] Through multiple iterations of the above-mentioned loop, the policy network Capable of adapting to the real-time status of different learners Generate practice questions that can improve long-term cumulative rewards. .
[0070] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A personalized language learning training platform integrating reinforcement learning, characterized in that, include: The state awareness module collects and outputs learner interaction data in real time. The interaction data includes correct and incorrect answer indicators, answer response time, speech fluency features or text input fluency features, interface interaction behavior sequence, and confidence score. The state encoding module is connected to the state perception module, receives the interaction data, and processes it through a model containing a first feature network, a second feature network, and a third feature network. The first feature network outputs a first state vector, the second feature network outputs a second state vector, and the third feature network outputs a third state vector. The state encoding module also includes a fusion layer, which fuses the first state vector, the second state vector, and the third state vector to output a learner state vector. The reinforcement learning decision module is connected to the state encoding module, receives the learner's state vector, and outputs action instructions based on the internal policy network. The action instructions include target knowledge point identifiers, question type identifiers, interference intensity levels, and context topic identifiers. The practice generation module is connected to the reinforcement learning decision module, receives the action instructions, and retrieves and assembles content from the preset material library and template library according to the target knowledge point identifier, question type identifier, interference intensity level and context theme identifier, and generates and outputs a language practice question. The reward calculation module, connected to the state perception module and the state encoding module, receives the new round of interaction data and the corresponding new learner state vector, calculates and outputs the reward value; the reward value is calculated as follows: a basic reward value is determined based on the correctness or incorrectness of the answer, a skill reward value is determined based on the load index contained in the second state vector, a metacognitive reward value is determined based on the calibration scalar contained in the third state vector, and the basic reward value, skill reward value and metacognitive reward value are added together to obtain the reward value; The reinforcement learning decision module is also connected to the reward calculation module, and receives the reward value to update its internal policy network. The state perception module, state encoding module, reinforcement learning decision module, practice generation module, and reward calculation module form a closed loop.
2. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, The state awareness module collects the sequence of interface interaction behaviors, specifically including: recording cursor movement trajectory, focus switching events, and text selection and deselection operations.
3. The personalized language learning training platform integrating reinforcement learning according to claim 2, characterized in that, In the state encoding module, the input of the second feature network includes a load index, which is calculated based on the hovering duration of the cursor in the option area in the cursor movement trajectory, the number of times the focus switching event occurs, and the time interval between adjacent selection and deselection operations.
4. The personalized language learning training platform integrating reinforcement learning according to claim 2 or 3, characterized in that, The input to the second feature network also includes a policy identifier, which is obtained by sequence matching of the interface interaction behavior sequence with a pre-stored policy template.
5. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, The state awareness module collects the speech fluency features specifically by extracting the total duration of effective speech segments, the number of pauses, and the number of word repetitions from the learner's speech audio, and calculating the fluency coefficient based on the above extracted items.
6. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, The state awareness module collects the text input fluency features specifically by recording the average character input rate and the backspace key operation frequency, and calculates the text fluency coefficient based on the above recorded items.
7. The personalized language learning training platform integrating reinforcement learning according to claim 5 or 6, characterized in that, In the state coding module, the input of the second feature network also includes the fluency coefficient or the text fluency coefficient.
8. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, In the state encoding module, the first feature network is a deep knowledge tracking model, and its inputs include the correct / incorrect answer identifier, the multi-dimensional vector of associated knowledge points, and the normalized answer response time.
9. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, In the state coding module, the input of the third feature network includes the confidence value, and its output includes a calibration scalar, which is calculated based on multiple historical sets of the confidence values and the corresponding correct / incorrect answer identifiers.
10. The personalized language learning training platform integrating reinforcement learning according to claim 1, characterized in that, The operation method of the platform includes the following steps: Step S1: Collect the learner's interaction data after completing the current exercise through the state awareness module; Step S2: Process the interaction data through the state encoding module to obtain the current learner state vector; Step S3: The reinforcement learning decision module outputs action instructions based on the current learner state vector; Step S4: The exercise generation module generates new language practice questions based on the action instructions and pushes them to the learner; Step S5: Collect interaction data of the learner after completing a new exercise through the state awareness module; Step S6: Process the data collected in step S5 through the state coding module to obtain the new learner's state vector; Step S7: Calculate the reward value using the reward calculation module, based on the calculation rules described in claim 1, the data collected in step S5, and the new learner's state vector. Step S8: Use the current learner state vector, the action instruction, the reward value, and the new learner state vector to construct training data and update the internal policy network of the reinforcement learning decision module.