An online learning system based on reinforcement learning runs a big data management system

Abstract

The present application relates to big data management technical field, specifically relates to a kind of big data management system based on reinforcement learning online learning system operation, including learning state acceptance fragment construction module parses learning behavior chain, identifies learning state turning point, and is compressed and reconstructed into learning state acceptance fragment set;Reinforcement learning data management decision module constructs fragment set into manageable environment, generates data management action and forms management section;Reinforcement learning strategy feedback optimization module is according to the effect of section reuse reinforcement or inhibition corresponding action, update strategy and be used for the next round of management.The present application can reduce the redundancy occupation of big data of online learning system operation on the basis of keeping learning state turning point traceable, improve the accuracy and dynamic optimization ability of learning state identification, learning path adjustment and system resource scheduling.

Description

Technical Field

[0001] This invention relates to the field of big data management technology, and in particular to a big data management system for online learning systems based on reinforcement learning. Background Technology

[0002] With the development of online education platforms, smart learning systems, and educational data analytics technologies, online learning systems continuously generate a large amount of operational data during various stages, including accessing learning resources, learning knowledge points, answering exercises, page dwell time, page navigation, and interactive feedback. This data reflects the learning process, state changes, and path adjustment needs of learners under different knowledge tasks, and is a crucial basis for learning state identification, personalized learning path recommendation, and system resource scheduling. Therefore, how to effectively organize, compress, retain, and dynamically manage the big data generated by online learning systems has become a key issue that needs to be addressed in the field of online education data management.

[0003] Currently, online learning systems typically focus on the unified collection, storage, and statistical analysis of all behavioral data during operation data management. They lack analysis of the state continuity relationships of continuous behavioral chains for the same learner under the same knowledge task, making it difficult to accurately identify the key behavioral nodes that truly cause changes in learning state judgment results. For learning processes with stable states, high repetition, or low data value, existing methods often retain them completely, easily leading to redundant operational data, increased storage consumption, and a heavier computational burden. Furthermore, for data segments that involve state transitions, path adjustment criteria, or resource scheduling impacts, there is a lack of dynamic feedback mechanisms based on reuse effects, potentially resulting in the loss of key transition information, deviations in learning path adjustments, or abnormal system resource scheduling. Summary of the Invention

[0004] This invention provides a big data management system for online learning systems based on reinforcement learning. By constructing continuous behavior chains, identifying learning state transition nodes, and generating sets of learning state transition segments, it transforms scattered learning behavior data into manageable fragmented data units. Furthermore, it utilizes reinforcement learning agents to generate data management actions based on fragment state characteristics, transition node distribution characteristics, and data value characteristics, forming a data management cross-section for online learning operations. This management cross-section is then reused in the processes of learning state identification, learning path adjustment, and system resource scheduling. Based on the reuse effect, data management actions are strengthened or suppressed, thereby reducing operational data consumption while maintaining the traceability of learning state transition nodes, and achieving continuous adaptive optimization of the big data management strategy for online learning systems.

[0005] A big data management system for online learning based on reinforcement learning includes a learning state segment construction module, a reinforcement learning data management decision module, and a reinforcement learning strategy feedback optimization module, wherein; During the execution of the learning task, the learning state transition segment construction module performs behavior chain analysis on the big data of the online learning system, identifies the continuous behavior chain formed by the same learning object under the same knowledge task, analyzes the state impact of each behavior node in the continuous behavior chain, filters the learning state transition nodes that cause changes in the learning state judgment result, compresses and reconstructs the continuous behavior chain based on the learning state transition nodes, and merges the continuous and stable behavior process into learning state transition segments to generate a set of learning state transition segments. The reinforcement learning data management decision module constructs the set of learning state segments into a manageable environment for the reinforcement learning agent. The reinforcement learning agent generates corresponding data management actions based on the state characteristics, turning point distribution characteristics, and data value characteristics of each learning state segment. Based on the execution results of each data management action, a corresponding online learning operation data management section is formed to achieve dynamic optimization management of the big data operation. The reinforcement learning strategy feedback optimization module reuses the online learning operation data management section for learning state identification, learning path adjustment, and system resource scheduling. It also provides reward feedback to the reinforcement learning agent based on the reuse effect. When the reuse result keeps learning state inflection points traceable and reduces operational data consumption, the corresponding data management action is strengthened. Conversely, when the reuse result leads to the loss of learning state inflection points, deviation in learning path adjustment, or abnormal resource scheduling, the corresponding data management action is suppressed, resulting in an updated reinforcement learning data management strategy. This updated strategy is then applied to the next round of online learning system operation big data management, achieving continuous adaptive optimization of the data management strategy.

[0006] Optionally, the learning state transition segment construction module includes: Continuous Behavior Chain Recognition: During the execution of learning tasks, big data on the operation of the online learning system is acquired, and the learning behavior data is correlated and analyzed according to the learning object identifier, knowledge task identifier, and the order of behavior occurrence. The continuous learning behavior process formed by the same learning object under the same knowledge task is identified. The behaviors of accessing learning resources, learning knowledge points, answering exercises, learning pauses, page jumps, and interactive feedback are connected according to the time sequence to construct the corresponding continuous behavior chain. Learning state transition node extraction: Receive the continuous behavior chain, perform state influence analysis on each behavior node in the continuous behavior chain, evaluate the degree of influence of each behavior node on the learning state judgment result, identify the key behavior nodes that cause changes in the learning state judgment result between adjacent behavior nodes, and determine the key behavior nodes that cause changes in the learning state judgment result as learning state transition nodes, and generate a set of learning state transition nodes. Learning state transition segment generation: Based on the set of learning state transition nodes, the continuous behavior chain is compressed and reconstructed. The continuous behavior process that maintains the same learning state characteristics between adjacent learning state transition nodes is used as a unified analysis unit. Multiple behavior nodes with stable states and consistent change trends are merged. The merged behavior process is constructed into the corresponding learning state transition segment. The state category, duration, behavior density and transition relationship of each learning state transition segment are recorded to generate a set of learning state transition segments.

[0007] Optionally, the continuous behavior chain identification includes: Behavioral data collection: During the execution of learning tasks, collect behavioral records generated by the online learning system, including access to learning resources, learning of knowledge points, answering of exercises, learning pauses, page jumps, and interactive feedback behaviors. Write the learning object identifier, knowledge task identifier, behavior type, behavior occurrence time, behavior duration, and behavior result for each behavior record to form a standardized learning behavior record. Behavior association filtering: Based on the learning object identifier and knowledge task identifier, standardized learning behavior records are association-filtered, and learning behavior records belonging to the same learning object and corresponding to the same knowledge task are grouped into the same behavior set; Continuous behavior determination: The set of behaviors is sorted in ascending order according to the time of occurrence of the behaviors, and it is determined whether the behaviors belong to the same continuous learning process based on the time interval between adjacent behaviors; Behavior chain construction: Learning resource access, knowledge point learning, exercise answering, learning pauses, page jumps, and interactive feedback behaviors determined by continuity are linked together in chronological order to form a continuous learning behavior chain for the same learning object under the same knowledge task, and the occurrence order, behavior type, duration, and behavior result of each behavior node are recorded.

[0008] Optionally, the extraction of learning state transition nodes includes: State feature extraction: Receive the continuous behavior chain, extract the learning state features corresponding to each behavior node according to the order of behavior occurrence, the learning state features include answer accuracy, learning dwell effectiveness, resource access completion, interaction feedback intensity and page jump stability, and normalize each learning state feature to generate a state feature vector for each behavior node; State impact assessment: Calculate the corresponding learning state score based on the state feature vector of each behavior node, and determine the corresponding learning state judgment result in combination with the state feature vector to obtain the learning state category, including low participation state, unmastered state, knowledge consolidation state, mastered state and abnormal learning state. Calculate the degree of influence of the current behavior node on the learning state judgment result based on the change amplitude of the state feature vector between adjacent behavior nodes to form the state impact value. Turning point determination: The state influence value is compared with the state turning point threshold, and combined with the learning state category change results, key behavioral nodes in which the learning state judgment results change between adjacent behavioral nodes are identified. These include the transition between mastery and non-mastery, low participation and high learning effectiveness, normal and abnormal learning, and non-mastery and knowledge consolidation. When the state influence value of a behavioral node reaches the state turning point threshold and the learning state judgment results before and after the behavioral node change, the behavioral node is determined as a learning state turning point, and a set of learning state turning points is generated.

[0009] Optionally, the generation of the learning state transition segment includes: Transition Boundary Division: Receive a continuous behavior chain and a set of learning state transition nodes, use the position of the learning state transition node in the continuous behavior chain as the segment division boundary, and divide the continuous behavior chain according to the order of behavior occurrence, so that the continuous behavior process between adjacent learning state transition nodes forms a candidate successor segment. Stable segment merging: A consistency analysis is performed on the learning state judgment results and learning state score change trends of each behavior node in the candidate segment to determine whether the candidate segment belongs to a continuous behavior process with stable state and consistent change trend. When the proportion of the main learning state in the candidate segment reaches the state consistency threshold and the change direction of adjacent learning state scores is consistent, multiple behavior nodes in the candidate segment are merged into a unified analysis unit, and a set of learning state segment is generated.

[0010] Optionally, the reinforcement learning data management decision module includes: Management environment construction: Receive the set of learning state segments, extract the management features corresponding to each learning state segment, including learning state category, duration, behavior density, transition relationship, state influence value and segment integrity value, and convert the management features into an environment state that can be recognized by the reinforcement learning agent. At the same time, construct the state space of online learning operation data based on the order between learning state segments and the distribution of transition nodes. Data management action generation: The reinforcement learning agent generates corresponding data management actions based on the state characteristics of the segments inherited by each learning state, the distribution characteristics of turning points, and the data value characteristics. These actions include retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Management section formation: Based on the data management actions, the learning state segment set is classified and processed. The data results after retention, merging, delayed archiving, retrieval, and reinforcement are reorganized to form an online learning operation data management section. The online learning operation data management section is used to characterize the management status, data retention range, compression and merging results, disputed archiving markers, invalid data retrieval results, and missing segment reinforcement results of each learning state segment in the current round, thereby realizing dynamic optimization management of big data in the online learning system operation.

[0011] Optionally, the management environment construction includes: Management Feature Extraction: Receive a set of learning state succession fragments, read each learning state succession fragment in the order of fragment generation, and extract the learning state category, duration, behavior density, transition relationship, state influence value, and fragment integrity information to form corresponding fragment management features; Environmental state transition: The fragment management features are converted into environmental states that can be recognized by the reinforcement learning agent. In this process, the state category and transition relationship are encoded, and the duration, behavior density, mean state influence and fragment integrity value are normalized. State space construction: Based on the sequential order of learning state segments, the environmental state vectors are arranged according to the time series, and the distribution of turning points between adjacent segments is combined to construct the state space of the online learning operation data.

[0012] Optionally, the data management action generation includes: Action decision feature construction: Receive the state space of online learning operation data, and read the first... Each learning state inherits the environmental state vector, the distribution labels of the preceding and following turning points, and the data value-related indicators of the segment, and integrates the segment state features, turning point distribution features, and data value features into the action decision features of the reinforcement learning agent. Action value assessment: The action decision features are input into the reinforcement learning agent, which calculates the action value corresponding to each candidate data management action. The candidate data management actions include retaining turning points, merging stable segments, delaying the archiving of disputed segments, retrieving invalid segments, and reinforcing missing segments. Management Action Determination: The reinforcement learning agent determines management actions based on the action value of each candidate data management action, combined with fragment integrity value, inflection node density, mean state influence, state consistency, trend consistency, data redundancy, and data value feature value. If the fragment integrity value is lower than the fragment integrity threshold, or if the state category of adjacent fragments changes but no corresponding inflection node is identified, a missing fragment reinforcement action is generated. If the inflection node density and mean state influence both reach the corresponding thresholds, an inflection node retention action is generated. If state consistency, trend consistency, and data redundancy all reach the corresponding thresholds, and the inflection node density is lower than the corresponding threshold, a stable fragment merging action is generated. If the data value feature value, mean state influence, and inflection node density are all lower than the corresponding thresholds, and the fragment integrity value reaches the corresponding threshold, an invalid fragment retrieval action is generated. If the action value difference value is lower than the action value difference threshold, or if state consistency and trend consistency do not reach the corresponding thresholds, a disputed fragment postponement archiving action is generated. When no data management action determination condition is met, the candidate data management action with the highest action value is selected as the final data management action.

[0013] Optionally, the formation of the management section includes: Segment Management Status Classification: Receive the set of learning state segments and the corresponding data management actions for each learning state segment. Classify the learning state segments according to the following criteria: retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Determine the management status of each learning state segment in the current round. Reorganization of management results: Based on the data management actions corresponding to each learning state transition segment, data retention range, compression and merging results, dispute archiving markers, invalid data retrieval results, and missing segment reinforcement results are generated for learning state transition segments under different management states. Specifically, for segments that perform the action of retaining turning points, their corresponding learning state turning points and related behavior nodes are retained; for segments that perform the action of merging stable segments, the merged summary data is retained; for segments that perform the action of delaying the archiving of disputed segments, dispute archiving markers are written; for segments that perform the action of retrieving invalid segments, retrieval results are written; and for segments that perform the action of reinforcing missing segments, missing segment reinforcement results are written. Management section generation: The data management results of each learning status segment in the current round are rearranged according to the segment generation order to form an online learning operation data management section.

[0014] Optionally, the reinforcement learning policy feedback optimization module includes: Cross-section reuse evaluation: The online learning operation data management cross-sections formed in the current round are reused in the learning status identification, learning path adjustment and system resource scheduling processes, and the retention status of turning points, changes in data occupation, learning path adjustment results and resource scheduling results are collected after reuse. The turning point traceability rate, data occupation reduction rate, learning path offset value and resource scheduling deviation value are calculated to evaluate the reuse effect of the current data management actions. Reward feedback generation: Based on the turning point tracing rate, data occupancy reduction rate, learning path offset value, and resource scheduling deviation value, a reward feedback value is generated for the data management action corresponding to the segment of the current learning state, and an action feedback judgment value is generated. Specifically, when the turning point tracing rate reaches the turning point tracing rate threshold, the data occupancy reduction rate reaches the data occupancy reduction threshold, and the learning path offset value and resource scheduling deviation value do not exceed the corresponding threshold, the data management action is strengthened. When the turning point tracing rate does not reach the tracing rate threshold, or the learning path offset value and resource scheduling deviation value exceed the corresponding threshold, the data management action is suppressed. Policy update application: Input the reward feedback value and action feedback judgment value into the reinforcement learning agent to update the current action value function, obtain the updated reinforcement learning data management policy, and use the reinforcement learning data management policy for the next round of online learning system operation big data management.

[0015] The beneficial effects of this invention are: This invention analyzes the behavioral chains of big data generated by an online learning system during the execution of learning tasks. It connects behaviors such as accessing learning resources, learning knowledge points, answering exercises, pausing in learning, page navigation, and interactive feedback generated by the same learner under the same knowledge task into a continuous behavioral chain. Furthermore, it identifies key learning state transition nodes that can cause changes in the learning state assessment results. This allows the system to extract crucial nodes that truly influence changes in learning state from a large amount of scattered and redundant learning behavior data, avoiding the indiscriminate retention of all operational data. This improves the structured management capabilities of online learning operational big data and the accuracy of learning state identification.

[0016] This invention compresses and reconstructs continuous behavioral chains based on learning state transition nodes, merging stable and consistent continuous behavioral processes into learning state transition segments. These learning state transition segments are then constructed into a manageable environment for the reinforcement learning agent. This allows the agent to combine segment state characteristics, transition node distribution characteristics, and data value characteristics to perform data management actions such as retaining transition nodes, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. This maintains the traceability of key learning state change information while reducing the storage and computing resource consumption of stable and low-value segments.

[0017] This invention utilizes online learning operation data management sections for learning state identification, learning path adjustment, and system resource scheduling. It also provides reward feedback to reinforcement learning agents based on turning point tracing, data usage changes, learning path deviations, and resource scheduling errors. This strengthens data management actions that effectively reduce data usage without affecting learning state turning point tracing, while suppressing data management actions that lead to lost turning points, path adjustment deviations, or resource scheduling anomalies. This achieves continuous adaptive optimization of reinforcement learning data management strategies, improving the stability, reliability, and dynamic optimization capabilities of big data management in online learning systems. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the system functional modules according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the reinforcement learning data management decision module according to an embodiment of the present invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. Those skilled in the art may employ other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0021] like Figures 1-2 As shown, an online learning system based on reinforcement learning operates a big data management system, including a learning state segment construction module, a reinforcement learning data management decision module, and a reinforcement learning strategy feedback optimization module, wherein; During the execution of learning tasks, the learning state transition segment construction module analyzes the behavior chain of the big data of the online learning system, identifies the continuous behavior chain formed by the same learning object under the same knowledge task, analyzes the state impact of each behavior node in the continuous behavior chain, filters the learning state transition nodes that cause changes in the learning state judgment results, compresses and reconstructs the continuous behavior chain based on the learning state transition nodes, and merges the continuous and stable behavior processes into learning state transition segments to generate a set of learning state transition segments. The reinforcement learning data management decision module constructs a manageable environment for the reinforcement learning agent by taking the set of learning state segments. The reinforcement learning agent generates corresponding data management actions based on the state characteristics, turning point distribution characteristics, and data value characteristics of each learning state segment. Based on the execution results of each data management action, a corresponding online learning operation data management section is formed, realizing dynamic optimization management of big data operation. The reinforcement learning strategy feedback optimization module reuses the online learning operation data management section for learning state identification, learning path adjustment, and system resource scheduling. It also provides reward feedback to the reinforcement learning agent based on the reuse effect. When the reuse result keeps the learning state inflection point traceable and reduces the operation data consumption, the corresponding data management action is strengthened. When the reuse result causes the learning state inflection point to be lost, the learning path adjustment to deviate, or the resource scheduling to be abnormal, the corresponding data management action is suppressed, resulting in an updated reinforcement learning data management strategy. The updated reinforcement learning data management strategy is then applied to the next round of online learning system operation big data management process, realizing continuous adaptive optimization of the data management strategy.

[0022] The learning state transition segment construction module includes: Continuous Behavior Chain Recognition: During the execution of learning tasks, big data on the operation of the online learning system is acquired, and the learning behavior data is correlated and analyzed according to the learning object identifier, knowledge task identifier, and the order of behavior occurrence. The continuous learning behavior process formed by the same learning object under the same knowledge task is identified. The behaviors of accessing learning resources, learning knowledge points, answering exercises, learning pauses, page jumps, and interactive feedback are connected according to the time sequence to construct the corresponding continuous behavior chain. Learning state transition node extraction: Receive a continuous chain of behaviors, perform state impact analysis on each behavior node in the continuous chain of behaviors, evaluate the degree of influence of each behavior node on the learning state judgment result, identify the key behavior nodes that cause changes in the learning state judgment result between adjacent behavior nodes, and determine the key behavior nodes that cause changes in the learning state judgment result as learning state transition nodes, and generate a set of learning state transition nodes. Learning state transition segment generation: Based on the set of learning state transition nodes, the continuous behavior chain is compressed and reconstructed. The continuous behavior process that maintains the same learning state characteristics between adjacent learning state transition nodes is used as a unified analysis unit. Multiple behavior nodes with stable states and consistent change trends are merged. The merged behavior process is constructed into the corresponding learning state transition segment. The state category, duration, behavior density and transition relationship of each learning state transition segment are recorded to generate a set of learning state transition segments.

[0023] Continuous behavior chain identification includes: Behavioral data collection: During the execution of learning tasks, behavioral records generated by the online learning system are collected, including access to learning resources, learning of knowledge points, answering exercises, learning pauses, page navigation, and interactive feedback behaviors. Each behavioral record is then written with a learning object identifier, knowledge task identifier, behavior type, behavior occurrence time, behavior duration, and behavior result, forming standardized learning behavior records, specifically including: (1) During the execution of the learning task, original behavioral events are collected from the resource access interface, knowledge point learning interface, exercise answering interface, page dwell interface, page jump interface, and interactive feedback interface of the online learning system. For each original behavioral event, the corresponding learning object identifier, knowledge task identifier, behavior trigger time, behavior end time, behavior source interface, and original behavior content are recorded to form a set of original behavioral events, represented as: ; in, For the first A primary behavioral event, Identify the learning objects. For knowledge task identification, For the behavior trigger time, The end time of the action. For the behavior source interface, This refers to the original behavioral content; (2) Perform rule matching on the source interface and content of the original behavior event to map the original behavior event to a unified behavior type. The unified behavior type includes learning resource access, knowledge point learning, exercise answering, learning dwell, page jump and interactive feedback, represented as: ; in, For the first The unified behavior type corresponding to each original behavior event For behavior type mapping functions, For the behavior source interface; (3) Calculate the duration of the behavior based on the trigger time and end time of the original behavior event, and fill in the missing end time for behavior events according to the trigger time of the next behavior or the end time of the task, so that each behavior has a duration that can be used for subsequent time series analysis, expressed as: ; in, For the first The duration of each behavioral event; (4) Extract the corresponding behavior result fields according to different behavior types. Among them, the behavior result of learning resource access behavior includes the resource access completion status, the behavior result of knowledge point learning behavior includes the knowledge point learning completion status, the behavior result of exercise answering behavior includes the correctness of the answer and the score, the behavior result of learning dwell behavior includes the effective dwell status, the behavior result of page jump behavior includes the jump to the target page, and the behavior result of interactive feedback behavior includes the feedback content or feedback level. Write all kinds of results into the behavior result field to generate standardized behavior results. ; (5) Encapsulate the learning object identifier, knowledge task identifier, unified behavior type, behavior trigger time, behavior duration, and behavior result into standardized learning behavior records, and summarize multiple standardized learning behavior records into a standardized learning behavior record set, represented as: ; in, For the first Standardized learning behavior records; Behavioral Association Filtering: Based on the learning object identifier and knowledge task identifier, standardized learning behavior records are filtered by association. Learning behavior records belonging to the same learning object and corresponding to the same knowledge task are grouped into the same behavior set, represented as: ; in, For learning objects In knowledge tasks The set of behaviors below , These are the learning object identifier and the knowledge task identifier corresponding to the behavior record, respectively. Continuous behavior determination: The behavior set is sorted in ascending order according to the occurrence time of the behavior, and the behavior is determined to belong to the same continuous learning process based on the time interval between adjacent behaviors, as represented as: ; in, For the continuity determination result, Indicates behavior record and Belonging to the same continuous learning process, , These represent the times when adjacent actions occurred. The threshold is the continuous interval. ; in, , These represent the mean and standard deviation of the time intervals between historical learning behaviors, respectively. This is the threshold adjustment coefficient; Behavior chain construction: Learning resource access, knowledge point learning, exercise answering, learning pauses, page jumps, and interactive feedback behaviors determined by continuity are linked together in chronological order to form a continuous learning behavior chain for the same learning object under the same knowledge task, and the occurrence order, behavior type, duration, and behavior result of each behavior node are recorded.

[0024] Learning state transition node extraction includes: State Feature Extraction: Receive a continuous chain of behaviors and extract the learning state features corresponding to each behavior node according to the order of behavior occurrence. These learning state features include answer accuracy, learning dwell time effectiveness, resource access completion rate, interaction feedback intensity, and page jump stability. Each learning state feature is then normalized to generate a state feature vector for each behavior node, specifically including: (1) Receive a continuous chain of behaviors, read the behavior type, duration, result, resource completion marker, response result, interaction feedback record and page jump record of each behavior node in the order of behavior occurrence, and bind the above fields with the corresponding behavior node to form a basic field set of the node; (2) Based on the node's basic field set, calculate the answer accuracy, learning dwell time effectiveness, resource access completion rate, interaction feedback intensity, and page jump stability for each behavior node, as follows: ; ; ; ; ; in, For the first The accuracy rate of answers for each behavior node. For the first The number of correct answers corresponding to each behavior node. For the first The total number of responses corresponding to each behavior node. To ensure the effectiveness of learning, For the first The actual dwell time of each behavior node To preset the effective learning time, For resource access completion, This represents the number of learning resources that have been accessed. To account for the total number of accessed learning resources, For the intensity of interactive feedback, For the first The number of interaction feedbacks or feedback level values ​​for each behavior node. To preset the maximum feedback intensity, For page redirection stability, For the first The number of non-target page redirects corresponding to each behavior node. The maximum number of redirects is preset. (3) Normalize and verify the calculated state features, limit feature values ​​exceeding the upper limit to 1, and limit feature values ​​below the lower limit to 0, and encapsulate them in the order of answer accuracy, learning dwell effectiveness, resource access completion, interaction feedback intensity, and page jump stability to generate the first... The state feature vector of each behavioral node is represented as: ; ; ; ; in, For the first The state feature vector of each behavioral node These are the normalized state eigenvalues. The lower limit of the value, This represents the upper limit of the possible values. These are the characteristic values ​​of the original state. , The first The mean and standard deviation of the state features in the historical samples. This is the upper limit adjustment coefficient. This is the lower limit adjustment coefficient; State Impact Assessment: Based on the state feature vectors of each behavioral node, the corresponding learning state score is calculated. Combined with the state feature vectors, the corresponding learning state judgment result is determined, resulting in a learning state category, including low participation, no mastery, knowledge consolidation, mastery, and abnormal learning states. Based on the magnitude of changes in the state feature vectors between adjacent behavioral nodes, the degree of influence of the current behavioral node on the learning state judgment result is calculated, forming a state impact value, specifically including: (1) Receive the state feature vector of each behavior node, and perform weighted calculation on the answer accuracy, learning dwell effectiveness, resource access completion, interaction feedback intensity, and page jump stability according to the preset feature weights to obtain the first... The learning state score of each behavioral node is represented as: ; in, For the first Learning status score of each behavioral node The number of state features, For the first The first behavior node Item state characteristic value, For the first Item state feature weights; (2) Scoring based on learning status Combined with low participation threshold Threshold not known Mastering the threshold and abnormal threshold Determine the first Learning state judgment results for each behavior node , represented as: ; ; ; ; ; in, , These represent the mean and standard deviation of the historical sample participation index, respectively. For low participation adjustment coefficient, , These are the mean and standard deviation of historical learning status scores, respectively. Because the adjustment coefficient is not known, To understand the adjustment coefficient, , These represent the average and standard deviation of historical page redirection stability, respectively. This is the abnormal adjustment coefficient; (3) After obtaining the learning state judgment results of adjacent behavior nodes, calculate the degree of influence of the current behavior node on the learning state judgment results based on the change amplitude of the state feature vector of adjacent behavior nodes, and form the state influence value, which is expressed as: ; in, For the first The state influence value of each behavior node. , The first , The first behavior node Item state characteristic value; Turning point identification: By comparing the state influence value with the state transition threshold and combining it with the results of changes in learning state categories, key behavioral nodes where the learning state judgment results change between adjacent behavioral nodes are identified. These include transitions between mastery and non-mastery states, low participation and high learning effectiveness states, normal and abnormal learning states, and non-mastery and knowledge consolidation states. When the state influence value of a behavioral node reaches the state transition threshold and the learning state judgment result changes before and after the behavioral node, the behavioral node is identified as a learning state turning point, generating a set of learning state turning points, specifically including: (1) Based on the distribution of state influence values ​​of each behavior node in the continuous behavior chain, calculate the state transition threshold to distinguish between ordinary state fluctuations and significant state changes, expressed as: ; in, This is the threshold for state transition. , These represent the average and standard deviation of the state influence values ​​of each behavior node within the same continuous behavior chain. This is the threshold adjustment coefficient; (2) Compare the learning state judgment result of the current behavior node with the learning state judgment result of the previous behavior node to identify whether there is a change in the learning state category between adjacent behavior nodes, and mark the corresponding state change type, including the transition between mastery state and non-mastery state, the transition between low participation state and high learning effectiveness state, the transition between normal learning state and abnormal learning state, and the transition between non-mastery state and knowledge consolidation state, represented as: ; in, For the first The state change judgment value of each behavior node. This indicates that the current action node has undergone a learning state change relative to its predecessor action node; (3) Compare the state influence value of the current action node with the state transition threshold, and determine whether the action node belongs to the learning state transition node by combining the state change judgment value. When the state influence value of the current action node reaches the state transition threshold, and the learning state judgment result between the current action node and the previous action node changes, the current action node is determined as the learning state transition node, and all learning state transition nodes are summarized to generate a learning state transition node set, which is represented as: ; in, For the first The turning point determination value of each behavior node. Indicates the first Each behavioral node represents a learning state transition node. Indicates the first This behavior node does not belong to the learning state transition node.

[0025] The generation of learning state transition segments includes: Transition Boundary Delineation: Receives a continuous behavior chain and a set of learning state transition nodes. The position of each learning state transition node within the continuous behavior chain is used as the segmentation boundary. The continuous behavior chain is then divided according to the order of behavior occurrence, creating candidate successor segments for continuous behavior processes between adjacent learning state transition nodes. Specifically, this includes: (1) Receive the completed continuous behavior chain, convert each standardized learning behavior record in the continuous behavior chain into a behavior node sequence according to the order of behavior occurrence, and write the node number, learning state judgment result, learning state score and turning point judgment value for each behavior node to form a continuous behavior node sequence, represented as: ; in, A continuous sequence of nodes. For the first in a continuous chain of actions Each behavior node , This represents the total number of behavior nodes in a continuous chain of behaviors. (2) Based on the turning point judgment value of each behavior node, extract the node index corresponding to the learning state turning point from the continuous behavior node sequence, and use the first and last indexes of the continuous behavior chain as supplementary boundaries to form a segment partitioning boundary sequence, which is expressed as: ; in, This is the set of indices of learning state transition nodes. For the first The sequence number of each learning state transition node in the continuous behavior node sequence. For the first The turning point determination value of each behavior node. This indicates that the behavior node belongs to the learning state transition node; (3) Divide the continuous behavior node sequence according to the boundary sequence, and group the continuous behavior nodes between two adjacent boundary numbers into the same candidate transition segment, so that each candidate transition segment corresponds to a continuous learning behavior process between adjacent learning state transition nodes, as shown in: ; in, For the first One candidate segment to continue. , The first , Each segment is divided into boundary numbers; Stable Segment Merging: A consistency analysis is performed on the learning state judgment results and learning state score change trends of each behavioral node within a candidate segment to determine whether the candidate segment belongs to a continuous behavioral process with stable states and consistent change trends. When the proportion of the main learning states within a candidate segment reaches the state consistency threshold, and the change directions of adjacent learning state scores are consistent, multiple behavioral nodes within that candidate segment are merged into a unified analysis unit, generating a set of learning state segment continuations, specifically including: (1) Receive candidate continuation segments The learning state judgment results of each behavior node within the candidate segment are statistically analyzed, and the learning state that appears most frequently is determined as the main learning state of the candidate segment. Simultaneously, the proportion of the main learning state is calculated to determine whether the learning states within the candidate segment are consistent, expressed as follows: ; ; in, For the first The main learning state of each candidate segment. This is a set of learning state categories, including low participation, lack of mastery, knowledge consolidation, mastery, and abnormal learning states. For the first The number of behavior nodes in each candidate segment is The number of behavioral nodes, For the first The number of behavioral nodes belonging to the primary learning state in each candidate segment. For the first The total number of behavioral nodes within each candidate segment. The percentage of students in the primary learning state; (2) Read the learning state scores of each behavior node in the candidate segment according to the order of behavior occurrence, calculate the direction of change of learning state scores between adjacent behavior nodes, and count the proportion of the same direction of change in the candidate segment to determine whether the candidate segment has a consistent trend of state change, expressed as: ; ; in, For the first The first candidate segment The behavior node and the first The direction of score change between each behavioral node For the first The first candidate segment Learning status score of each behavioral node This is a sign function; it takes the value 1 when the score increases, 0 when the score remains unchanged, and -1 when the score decreases. The direction of score change is The number of adjacent node pairs, For the rating trend consistency value, when At that time, directly Set to 1; (3) The proportion of the main learning state State Consistency Threshold Compare and ensure consistency of rating trends Consistency threshold with trend By comparison, when a candidate segment simultaneously meets the conditions of consistent learning state category and consistent scoring trend, multiple behavioral nodes within that candidate segment are merged into a unified analysis unit, and its main learning state, node range, average learning state score, and trend direction are written into it, represented as: ; in, For the first A unified analysis unit is formed by merging candidate continuation segments. This represents the average learning state score of each behavioral node within the candidate segment. This indicates the dominant direction of score change for the candidate segment. ; ; in, The lower limit of the state consistency threshold is set to 0.5, which is used to ensure that the main learning state occupies at least half of the inline nodes in the segment. This represents the average percentage of the dominant learning states in historically confirmed stable segments. The standard deviation of the proportion of the dominant learning states in historically confirmed stable segments. This is the state consistency threshold adjustment coefficient. The lower limit of the trend consistency threshold is set to 0.5, which is used to ensure that the main direction of rating change within a segment occupies at least half of the adjacent node pairs. This represents the average of the consistency values ​​of the scoring trend among historically confirmed stable segments. The standard deviation of the consistency values ​​of the rating trend in historically confirmed stable segments. This is the trend consistency threshold adjustment coefficient.

[0026] The reinforcement learning data management decision module includes: Management Environment Construction: Receive the set of learning state segments, extract the management features corresponding to each learning state segment, including learning state category, duration, behavior density, transition relationship, state influence value, and segment integrity value, and convert the management features into an environment state that the reinforcement learning agent can recognize. At the same time, based on the sequence of learning state segments and the distribution of transition nodes, construct the state space of online learning operation data, so that the reinforcement learning agent can identify which segments need to be retained, merged, archived, recycled, or reinforced in a manageable environment. Data management action generation: The reinforcement learning agent generates corresponding data management actions based on the state characteristics of the segments inherited by each learning state, the distribution characteristics of turning points, and the data value characteristics. These actions include retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Management section formation: Based on the data management actions, the learning status segment set is classified and processed. The data results after retention, merging, delayed archiving, retrieval and reinforcement are reorganized to form the online learning operation data management section. The online learning operation data management section is used to characterize the management status, data retention range, compression and merging results, disputed archiving marks, invalid data retrieval results and missing segment reinforcement results of each learning status segment in the current round, thereby realizing the dynamic optimization management of big data in the operation of the online learning system.

[0027] The management environment setup includes: Management Feature Extraction: Receive a set of learning state transition segments, read each learning state transition segment in the order of segment generation, and extract the learning state category, duration, behavior density, transition relationships, state impact value, and segment integrity information to form corresponding segment management features, specifically including: (1) Receive the set of learning state segments, read each learning state segment in the order of generation of segments in the continuous behavior chain, and extract the behavior node sequence, behavior occurrence time, learning state judgment result and state influence value contained in the segment to form the basic data unit of the segment. (2) Based on the basic data units of the segment, calculate the segment learning state category, duration, behavior density, state influence value, and segment integrity information respectively. Among them, the segment learning state category is determined based on the learning state judgment result that appears most frequently in the segment; the duration is determined based on the behavior time at the beginning and end of the segment; the behavior density is determined based on the number of behavior nodes and the duration; the state influence value is determined by averaging the state influence values ​​of each behavior node in the segment; and the segment integrity information is determined based on the number of effective fields and the number of required fields, expressed as follows: ; ; ; ; ; in, For the first The learning state categories that connect different learning state segments. For the first The learning state judgment result in each learning state segment is: The number of behavioral nodes, The duration of the segment. This refers to the time when the first action node in this segment occurs. This represents the end time of the last action node in the segment. For fragment behavior density, The mean is affected by the state of the fragment. For the first The first segment The state influence value of each behavior node. This is the fragment integrity value. This represents the number of valid data fields already present in the segment. The total number of data fields required for data management of this segment; (3) Based on the connection relationship between the transition nodes of the current learning state segment and the adjacent segments before and after it, determine the transition relationship of the segment, and encapsulate the segment learning state category, duration, behavior density, transition relationship, state influence value and segment integrity information in a unified manner to form the segment management feature corresponding to the learning state segment, which is expressed as: ; ; in, For the first The transitional relationships between segments of a learning state. This is the learning state transition node adjacent to the beginning of this segment. This is the learning state transition node adjacent to the end of this segment. The segment learning state category is the segment that follows the previous learning state. The segment learning state category that connects to the current learning state. The segment learning state category that connects to the next learning state. For the first The segment management characteristics of each learning state segment; Environment State Transition: This involves converting fragment management features into environment states recognizable by the reinforcement learning agent. This includes encoding state categories and transitional relationships, and normalizing duration, behavior density, mean state influence, and fragment integrity values. This ensures that different types of management features can be integrated into a unified reinforcement learning state representation space. Specifically, this includes: (1) Receive the first Each learning state inherits the segment management characteristics of the segment, and the segment learning state category is defined. Relationship with transition Encoding is performed to convert the discrete information, originally represented by textual categories or associations, into numerical features that the reinforcement learning agent can recognize, represented as: ; in, For the first The learning state transition segment in the first... Encoded values ​​in the learning state, For the first in the set of learning state categories Class state; (2) Duration in the segment management feature Behavioral density State affects the mean and fragment integrity value Normalization is performed to map continuous features of different dimensions to the same value range, thus avoiding the influence of dimensional differences on the state recognition process of the reinforcement learning agent. This is expressed as: ; ; ; in, The duration after normalization. , These are the minimum and maximum durations of the segments that connect historical learning states, respectively. The normalized behavior density, , These represent the minimum and maximum values ​​of the behavior density of the segments inherited from the historical learning states, respectively. The normalized state affects the mean. , These are the minimum and maximum values ​​of the average influence of the historical learning state on the segment state; (3) The segment learning state category encoding results, transition relationship encoding results, and normalized duration, behavior density, mean state influence, and segment integrity value are concatenated in a preset order to generate an environment state vector that the reinforcement learning agent can recognize. The environment state vectors of multiple learning states that inherit segments are used as the basis of the online learning operation data state space, as follows: ; in, For the first Each learning state segment corresponds to an environmental state vector. Encoding results for the state category of the segment learning. The encoding result for the transition relationship; State space construction: Based on the sequential order of the learning state segments, the environmental state vectors are arranged according to the time series, and combined with the distribution of transition nodes between adjacent segments, the state space of the online learning data is constructed. This enables the reinforcement learning agent to recognize the state succession relationship between the current segment and the preceding and following segments. Specifically, this includes: (1) Receive the environment state vector corresponding to each learning state segment, and sort the environment state vectors in ascending order according to the start time of the learning state segment, so that the reinforcement learning agent can read the running data state according to the actual occurrence order of the learning process, as shown in: ; ; in, This is the sorting result obtained by ascending order of the segment start time. For the first The start time of each learning state transition segment The total number of segments that support the learning state. For the first Each learning state segment corresponds to an environmental state vector. This is a sequence of environmental state vectors arranged in chronological order. (2) Based on the number of learning state transition nodes between adjacent learning state segments, the influence value of the transition node state, and the changes in the learning state category of adjacent segments, a transition node distribution marker is generated to characterize the intensity of state changes and the density of transitions between adjacent segments, as shown below: ; in, For the first The first learning state transition segment and the first Distribution markers for transition nodes between learning state segments. For the first The first learning state transition segment and the first The number of learning state transition nodes between learning state segments. , The first The first learning state transition segment and the first The duration of each learning state segment. To prevent extremely small positive numbers with a denominator of zero, For the first The first learning state transition segment and the first Between each learning state segment The state influence value of each learning state transition node. This is a function to determine state category changes; it takes a value of 1 when the two are different and a value of 0 when they are the same. If the state influence value of the turning point is 0, then the state influence value of the turning point is 0. (3) Combine the time-ordered environmental state vectors with the corresponding inflection point distribution markers so that each state not only contains the management characteristics of the current segment itself, but also the state succession relationship between it and adjacent segments, thus forming the state space of the online learning operation data, represented as: ; ; in, For the first Each learning state corresponds to a reinforcement learning environment state in the segment. This is the environmental state vector of the segment itself. Mark the distribution of transition nodes between this segment and the previous segment. The distribution of transition nodes between this segment and the next segment is marked. When there is no preceding or following segment, the corresponding transition node distribution mark is set to a zero vector. This is the state space for online learning operation data.

[0028] Data management action generation includes:

[0029] Action decision feature construction: Receive the state space of online learning operation data, and read the first... Each learning state inherits the environmental state vector corresponding to a segment, the distribution labels of the preceding and following turning points, and the data value-related indicators. The segment state features, turning point distribution features, and data value features are then fused into the action decision features of the reinforcement learning agent, represented as follows: ; ; ; ; ; ; in, For the first The data value feature value of each learning state segment. This represents the normalized density of inflection points. For the first The normalized value of the historical call frequency of each learning state segment in learning state identification, learning path adjustment, or system resource scheduling. For the first The data redundancy between each learning state segment and its adjacent segments. , , , , These are the corresponding data value weight coefficients. This refers to the number of times the learning state segment was used for learning state identification, learning path adjustment, or system resource scheduling during the historical management process. This represents the maximum number of historical calls within the learning state transition segments under the same learning task. For the repetition degree of behavior nodes, To reduce the redundancy of learning resources, For state feature similarity, , , These are the weight coefficients for behavioral node repetition, learning resource repetition, and state feature similarity, respectively. For the first A set of behavioral nodes in a learning state sequence segment. In order to be with the first Each learning state is a set of behavioral nodes in adjacent segments. For the first The set of learning resources involved in each learning state transition segment. The set of learning resources involved in adjacent segments. This represents the environmental state vector corresponding to adjacent segments; ; in, For the first Action decision characteristics of a learning state segment; Action value assessment: Action decision features are input into the reinforcement learning agent, which calculates the action value corresponding to each candidate data management action. Candidate data management actions include retaining inflection points, merging stable segments, delaying the archiving of controversial segments, retrieving invalid segments, and reinforcing missing segments. Specifically, these include: (1) Receive the first Action decision characteristics of a learning state segment Furthermore, a set of candidate data management actions is established. Retaining turning points, merging stable segments, postponing the archiving of controversial segments, retrieving invalid segments, and reinforcing missing segments are encoded as action vectors recognizable by the reinforcement learning agent. This ensures that each candidate action forms a state-action input pair with the action decision features of the current segment, represented as: ; ; in, For the set of candidate data management actions, To preserve the turning point, To merge stationary segments, To postpone the archiving of disputed footage, To reclaim invalid fragments, To supplement the missing segments, For the first Action encoding vectors for each candidate data management action. For the first The first learning state transition segment and the first A state-action input pair is formed by a candidate action; (2) The action value function of the state-action input to the reinforcement learning agent is used to calculate the action value of the current learning state segment when performing different candidate data management actions. This value is used to characterize the comprehensive expected effect of the action on maintaining the traceability of the learning state transition node, reducing the running data occupation, and avoiding the deviation of the learning path adjustment. It is expressed as: ; ; in, For the first Each learning state segment executes candidate data management actions. The value of action at that time For parameters The reinforcement learning action value function, , For the weight parameters of the value network, , These are the bias parameters of the value network. It is a non-linear activation function. Represents the set of value network parameters; (3) Calculate the action value corresponding to each of the five candidate actions according to the order of the candidate data management action set, and combine the action values ​​into an action value sequence for the current learning state segment, which serves as the basis for determining the target data management action, as follows: ; in, For the first Each learning state carries over to the action value sequence corresponding to a given segment. To preserve the value of turning point actions, To merge the motion value of steady segments, To postpone the archiving of controversial action footage, To recover the value of invalid action segments, To enhance the value of missing action segments; Management Action Determination: The reinforcement learning agent determines management actions based on the action value of each candidate data management action, combined with factors such as fragment integrity value, inflection node density, mean state influence, state consistency, trend consistency, data redundancy, and data value feature value. If the fragment integrity value is lower than the fragment integrity threshold, or if the state category of adjacent fragments changes but no corresponding inflection node is identified, a missing fragment reinforcement action is generated. If both the inflection node density and mean state influence reach their respective thresholds, an inflection node retention action is generated. If state consistency, trend consistency, and data redundancy all reach their respective thresholds, and the inflection node density is lower than its respective threshold, a stable fragment merging action is generated. If the data value feature value, mean state influence, and inflection node density are all lower than their respective thresholds, and the fragment integrity value reaches its respective threshold, an invalid fragment retrieval action is generated. If the action value difference value is lower than the action value difference threshold, or if state consistency and trend consistency do not reach their respective thresholds, a disputed fragment postponement action is generated. When no data management action determination condition is met, the candidate data management action with the highest action value is selected as the final data management action, expressed as: ; in, For the first The final data management action determined by each learning state segment The fragment integrity threshold. This is the value for determining the fracture at the turning point. The threshold for the density of turning points. The threshold for state influence. The threshold for state consistency. The trend consistency threshold, For data redundancy threshold, For data value threshold, The difference in action value. The threshold for the difference in action value; ; ; ; ; in, For the first The maximum action value corresponding to a learning state segment. For the first The value of the next major action corresponding to each learning state segment. Indicates the first The segment learning state category that follows each learning state segment. This indicates the segment learning state category that follows the next learning state. This represents the function for determining state category changes; it takes a value of 1 when the two states are different and a value of 0 when they are the same. This indicates the number of learning state transition nodes identified between adjacent segments; ; ; ; ; ; ; ; ; in, This is the lower limit of the fragment integrity threshold. This represents the average integrity value of historical data segments that can normally participate in data management. The standard deviation of the integrity values ​​of historical data segments that can normally participate in data management. This is the integrity threshold adjustment coefficient. The average normalized density of turning points in the historical learning state transition segment. The standard deviation of the normalized density of turning points in the historical learning state segment. This is the threshold adjustment coefficient for the density of turning points. This represents the average of the normalized state influence mean in the historical learning state transition segment. The standard deviation of the normalized state influence mean in the historical learning state continuation segment. The threshold adjustment coefficient is affected by the state. This is the lower limit of the state consistency threshold. This represents the average percentage of the dominant learning states within a historically stable segment. The standard deviation of the proportion of the dominant learning states in a historical stable segment. This is the state consistency threshold adjustment coefficient. This is the lower limit of the trend consistency threshold. This represents the average of the consistent rating trends across historically stable segments. The standard deviation of the consistent rating trends in historical stable segments. This is the trend consistency threshold adjustment coefficient. This is the lower limit of the data redundancy threshold. This represents the average repetition rate of fragment data from historically performed merged and smoothed fragment actions. The standard deviation of repeatability for historically merged and smoothed segment actions. This is the data redundancy threshold adjustment coefficient. The average value of the data feature values ​​for segments that connect historical learning states. The standard deviation of the data value feature value of the segment that carries over the historical learning state. This is the data value threshold adjustment coefficient. It is the average of the differences between the value of the largest move and the value of the second largest move in the historical sample. This represents the standard deviation of the difference between the value of the largest move and the value of the second largest move in the historical sample. This is the threshold adjustment coefficient for the difference in action value.

[0030] The formation of the management section includes: Segment Management Status Classification: Receive the set of learning state segments and the corresponding data management actions for each segment. Classify the learning state segments according to the following criteria: retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Determine the management status of each learning state segment in the current round, represented as: ; in, For the first A set of learning state segments corresponding to data management actions; Management Result Reorganization: Based on the data management actions corresponding to each learning state transition segment, data retention range, compression and merging results, dispute archiving markers, invalid data retrieval results, and missing segment reinforcement results are generated for learning state transition segments under different management states. Specifically, for segments that perform the "retain transition node" action, their corresponding learning state transition nodes and related behavior nodes are retained; for segments that perform the "merge stable segments" action, the merged summary data is retained; for segments that perform the "delay archiving disputed segments" action, a dispute archiving marker is written; for segments that perform the "retrieve invalid segments" action, a retrieval result is written; and for segments that perform the "reinforce missing segments" action, a missing segment reinforcement result is written. This is represented as follows: ; in, For the first Data management results for each learning state segment. For the first The management status of each learning state segment. To retain the range of data, To compress the merged results, For dispute archiving, This is the result of invalid data retrieval. The result of augmenting missing segments; Management Section Generation: The data management results of each learning state segment in the current round are rearranged according to the segment generation order to form an online learning operation data management section. This management section can simultaneously represent the management status of each segment, the data retention range, the compression and merging results, the disputed archiving markers, the invalid data retrieval results, and the missing segment reinforcement results, as shown below: ; in, This is a data management section for the online learning operation formed in the current round. The total number of segments received during the learning phase.

[0031] The reinforcement learning strategy feedback optimization module includes: Cross-section reuse evaluation: The online learning operation data management cross-sections formed in the current round are reused in the learning status identification, learning path adjustment, and system resource scheduling processes. The retention status of turning points, changes in data usage, learning path adjustment results, and resource scheduling results after reuse are collected. The turning point traceability rate, data usage reduction rate, learning path offset value, and resource scheduling deviation value are calculated to evaluate the reuse effect of the current data management actions, expressed as follows: ; ; ; ; in, For the first The traceability rate of transition nodes in each learning state segment. This represents the number of learning state transition points that can still be traced after reuse. This represents the original number of learning state transition nodes contained in the segment. To reduce data usage rate, The amount of data occupied before performing data management actions. The amount of data used after performing data management actions. This is the learning path offset value. The learning path adjustment results are based on the management section. This is the result of adjusting the baseline learning path based on the original running data. This represents the resource scheduling deviation value. This is a vector of resource scheduling results obtained based on the management section. This is a vector of baseline resource scheduling results obtained based on the original operational data; Reward Feedback Generation: Based on the turning point tracing rate, data usage reduction rate, learning path offset value, and resource scheduling deviation value, reward feedback values ​​are generated for the data management actions corresponding to the current learning state segment, and action feedback judgment values ​​are generated. Specifically, when the turning point tracing rate reaches the turning point tracing rate threshold, the data usage reduction rate reaches the data usage reduction threshold, and the learning path offset value and resource scheduling deviation value do not exceed their corresponding thresholds, the data management action is strengthened. When the turning point tracing rate does not reach the tracing rate threshold, or the learning path offset value and resource scheduling deviation value exceed their corresponding thresholds, the data management action is suppressed. This is expressed as follows: ; ; in, For the first The reward value for the data management action corresponding to each learning state segment. , , , , These are the corresponding reward weight coefficients. For action feedback judgment value, when This indicates that corresponding data management actions are being strengthened. When indicates the suppression of the corresponding data management action, This indicates that the current action feedback state is maintained. The threshold for traceability rate at turning points. For the data occupancy reduction rate threshold, The learning path offset threshold. This is the threshold for resource scheduling deviation. ; ; ; ; in, This is the minimum allowable threshold for the traceability rate of turning points. , These represent the mean and standard deviation of the traceability rate at turning points in the historical normal reuse sample, respectively. This is the adjustment coefficient for the traceability rate threshold. The minimum allowable threshold for data occupancy reduction rate. , These represent the mean and standard deviation of the data occupancy reduction rate in the historical normal reuse sample, respectively. This is the threshold adjustment coefficient for the occupancy reduction rate. , These represent the mean and standard deviation of the learning path offset values ​​in the historical normal reuse samples, respectively. This is the learning path offset threshold adjustment coefficient. , These represent the mean and standard deviation of resource scheduling deviations in the historical normal reuse samples, respectively. This is the resource scheduling deviation threshold adjustment coefficient; Policy Update Application: The reward feedback value and action feedback decision value are input into the reinforcement learning agent to update the current action value function. This updates the data management policy so that reinforced data management actions have a higher selection probability under the same or similar conditions, while suppressed data management actions have a lower selection probability under the same or similar conditions. The updated reinforcement learning data management policy is then used for the next round of online learning system big data management, represented as: ; ; in, For the updated action value function, The action value function before the update. For the first The action decision characteristics of each learning state segment This is the data management action currently being performed in this segment. For learning rate, To reward feedback values, This is the adjustment coefficient for the motion feedback judgment value. This is the action feedback judgment value. Discount factor for future returns For the action decision characteristics of the corresponding segment in the next round of management, For the set of candidate data management actions, This is an updated reinforcement learning data management strategy.

[0032] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0033] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A big data management system for online learning systems based on reinforcement learning, characterized in that, It includes a learning state transition segment construction module, a reinforcement learning data management and decision-making module, and a reinforcement learning strategy feedback and optimization module, among which; During the execution of the learning task, the learning state transition segment construction module performs behavior chain analysis on the big data of the online learning system, identifies the continuous behavior chain formed by the same learning object under the same knowledge task, analyzes the state impact of each behavior node in the continuous behavior chain, filters the learning state transition nodes that cause changes in the learning state judgment result, compresses and reconstructs the continuous behavior chain based on the learning state transition nodes, and merges the continuous and stable behavior process into learning state transition segments to generate a set of learning state transition segments. The reinforcement learning data management decision module constructs the set of learning state segments into a manageable environment for the reinforcement learning agent. The reinforcement learning agent generates corresponding data management actions based on the state characteristics, turning point distribution characteristics, and data value characteristics of each learning state segment. Based on the execution results of each data management action, a corresponding online learning operation data management section is formed to achieve dynamic optimization management of the big data operation. The reinforcement learning strategy feedback optimization module reuses the online learning operation data management section for learning state identification, learning path adjustment, and system resource scheduling. It also provides reward feedback to the reinforcement learning agent based on the reuse effect. When the reuse result keeps learning state inflection points traceable and reduces operational data consumption, the corresponding data management action is strengthened. Conversely, when the reuse result leads to the loss of learning state inflection points, deviation in learning path adjustment, or abnormal resource scheduling, the corresponding data management action is suppressed, resulting in an updated reinforcement learning data management strategy. This updated strategy is then applied to the next round of online learning system operation big data management, achieving continuous adaptive optimization of the data management strategy.

2. The system of claim 1, wherein the system is configured to operate a big data management system. The learning state transition segment construction module includes: Continuous Behavior Chain Recognition: During the execution of learning tasks, big data on the operation of the online learning system is acquired, and the learning behavior data is correlated and analyzed according to the learning object identifier, knowledge task identifier, and the order of behavior occurrence. The continuous learning behavior process formed by the same learning object under the same knowledge task is identified. The behaviors of accessing learning resources, learning knowledge points, answering exercises, learning pauses, page jumps, and interactive feedback are connected according to the time sequence to construct the corresponding continuous behavior chain. Learning state transition node extraction: Receive the continuous behavior chain, perform state influence analysis on each behavior node in the continuous behavior chain, evaluate the degree of influence of each behavior node on the learning state judgment result, identify the key behavior nodes that cause changes in the learning state judgment result between adjacent behavior nodes, and determine the key behavior nodes that cause changes in the learning state judgment result as learning state transition nodes, and generate a set of learning state transition nodes. Learning state transition segment generation: Based on the set of learning state transition nodes, the continuous behavior chain is compressed and reconstructed. The continuous behavior process that maintains the same learning state characteristics between adjacent learning state transition nodes is used as a unified analysis unit. Multiple behavior nodes with stable states and consistent change trends are merged. The merged behavior process is constructed into the corresponding learning state transition segment. The state category, duration, behavior density and transition relationship of each learning state transition segment are recorded to generate a set of learning state transition segments.

3. The system of claim 2, wherein the system is configured to run a big data management system. The continuous behavior chain identification includes: Behavioral data collection: During the execution of learning tasks, collect behavioral records generated by the online learning system, including access to learning resources, learning of knowledge points, answering of exercises, learning pauses, page jumps, and interactive feedback behaviors. Write the learning object identifier, knowledge task identifier, behavior type, behavior occurrence time, behavior duration, and behavior result for each behavior record to form a standardized learning behavior record. Behavior association filtering: Based on the learning object identifier and knowledge task identifier, standardized learning behavior records are association-filtered, and learning behavior records belonging to the same learning object and corresponding to the same knowledge task are grouped into the same behavior set; Continuous behavior determination: The set of behaviors is sorted in ascending order according to the time of occurrence of the behaviors, and it is determined whether the behaviors belong to the same continuous learning process based on the time interval between adjacent behaviors; Behavior chain construction: Learning resource access, knowledge point learning, exercise answering, learning pauses, page jumps, and interactive feedback behaviors determined by continuity are linked together in chronological order to form a continuous learning behavior chain for the same learning object under the same knowledge task, and the occurrence order, behavior type, duration, and behavior result of each behavior node are recorded.

4. The system of claim 3, wherein the system is configured to run a big data management system. The extraction of learning state transition nodes includes: State feature extraction: Receive the continuous behavior chain, extract the learning state features corresponding to each behavior node according to the order of behavior occurrence, the learning state features include answer accuracy, learning dwell effectiveness, resource access completion, interaction feedback intensity and page jump stability, and normalize each learning state feature to generate a state feature vector for each behavior node; State impact assessment: Calculate the corresponding learning state score based on the state feature vector of each behavior node, and determine the corresponding learning state judgment result in combination with the state feature vector to obtain the learning state category, including low participation state, unmastered state, knowledge consolidation state, mastered state and abnormal learning state. Calculate the degree of influence of the current behavior node on the learning state judgment result based on the change amplitude of the state feature vector between adjacent behavior nodes to form the state impact value. Turning point determination: The state influence value is compared with the state turning point threshold, and combined with the learning state category change results, key behavioral nodes in which the learning state judgment results change between adjacent behavioral nodes are identified. These include the transition between mastery and non-mastery, low participation and high learning effectiveness, normal and abnormal learning, and non-mastery and knowledge consolidation. When the state influence value of a behavioral node reaches the state turning point threshold and the learning state judgment results before and after the behavioral node change, the behavioral node is determined as a learning state turning point, and a set of learning state turning points is generated.

5. The system of claim 4, wherein the system is configured to operate as a big data management system. The generation of the learning state continuation segment includes: Transition Boundary Division: Receive a continuous behavior chain and a set of learning state transition nodes, use the position of the learning state transition node in the continuous behavior chain as the segment division boundary, and divide the continuous behavior chain according to the order of behavior occurrence, so that the continuous behavior process between adjacent learning state transition nodes forms a candidate successor segment. Stable segment merging: A consistency analysis is performed on the learning state judgment results and learning state score change trends of each behavior node in the candidate segment to determine whether the candidate segment belongs to a continuous behavior process with stable state and consistent change trend. When the proportion of the main learning state in the candidate segment reaches the state consistency threshold and the change direction of adjacent learning state scores is consistent, multiple behavior nodes in the candidate segment are merged into a unified analysis unit, and a set of learning state segment is generated.

6. A big data management system for online learning systems based on reinforcement learning according to claim 1, characterized in that, The reinforcement learning data management decision module includes: Management environment construction: Receive the set of learning state segments, extract the management features corresponding to each learning state segment, including learning state category, duration, behavior density, transition relationship, state influence value and segment integrity value, and convert the management features into an environment state that can be recognized by the reinforcement learning agent. At the same time, construct the state space of online learning operation data based on the order between learning state segments and the distribution of transition nodes. Data management action generation: The reinforcement learning agent generates corresponding data management actions based on the state characteristics of the segments inherited by each learning state, the distribution characteristics of turning points, and the data value characteristics. These actions include retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Management section formation: Based on the data management actions, the learning state segment set is classified and processed. The data results after retention, merging, delayed archiving, retrieval, and reinforcement are reorganized to form an online learning operation data management section. The online learning operation data management section is used to characterize the management status, data retention range, compression and merging results, disputed archiving markers, invalid data retrieval results, and missing segment reinforcement results of each learning state segment in the current round, thereby realizing dynamic optimization management of big data in the online learning system operation.

7. The system of claim 6, wherein the system is configured to run a big data management system. The management environment setup includes: Management Feature Extraction: Receive a set of learning state succession fragments, read each learning state succession fragment in the order of fragment generation, and extract the learning state category, duration, behavior density, transition relationship, state influence value, and fragment integrity information to form corresponding fragment management features; Environmental state transition: The fragment management features are converted into environmental states that can be recognized by the reinforcement learning agent. In this process, the state category and transition relationship are encoded, and the duration, behavior density, mean state influence and fragment integrity value are normalized. State space construction: Based on the sequential order of learning state segments, the environmental state vectors are arranged according to the time series, and the distribution of turning points between adjacent segments is combined to construct the state space of the online learning operation data.

8. The online learning system based on reinforcement learning for running big data management system of claim 7, wherein, The data management action generation includes: Action decision feature construction: receive the state space of online learning running data, read the environment state vector corresponding to the first learning state, the before and after turning node distribution label and the data value related index, and fuse the segment state feature, the turning node distribution feature and the data value feature into the action decision feature of the reinforcement learning intelligent agent; ​ Action value assessment: The action decision features are input into the reinforcement learning agent, which calculates the action value corresponding to each candidate data management action. The candidate data management actions include retaining turning points, merging stable segments, delaying the archiving of disputed segments, retrieving invalid segments, and reinforcing missing segments. Management Action Determination: The reinforcement learning agent determines management actions based on the action value of each candidate data management action, combined with fragment integrity value, inflection node density, mean state influence, state consistency, trend consistency, data redundancy, and data value feature value. If the fragment integrity value is lower than the fragment integrity threshold, or if the state category of adjacent fragments changes but no corresponding inflection node is identified, a missing fragment reinforcement action is generated. If the inflection node density and mean state influence both reach the corresponding thresholds, an inflection node retention action is generated. If state consistency, trend consistency, and data redundancy all reach the corresponding thresholds, and the inflection node density is lower than the corresponding threshold, a stable fragment merging action is generated. If the data value feature value, mean state influence, and inflection node density are all lower than the corresponding thresholds, and the fragment integrity value reaches the corresponding threshold, an invalid fragment retrieval action is generated. If the action value difference value is lower than the action value difference threshold, or if state consistency and trend consistency do not reach the corresponding thresholds, a disputed fragment postponement archiving action is generated. When no data management action determination condition is met, the candidate data management action with the highest action value is selected as the final data management action.

9. A big data management system for online learning systems based on reinforcement learning according to claim 8, characterized in that, The formation of the management section includes: Segment Management Status Classification: Receive the set of learning state segments and the corresponding data management actions for each learning state segment. Classify the learning state segments according to the following criteria: retaining turning points, merging stable segments, postponing the archiving of disputed segments, reclaiming invalid segments, and reinforcing missing segments. Determine the management status of each learning state segment in the current round. Reorganization of management results: Based on the data management actions corresponding to each learning state transition segment, data retention range, compression and merging results, dispute archiving markers, invalid data retrieval results, and missing segment reinforcement results are generated for learning state transition segments under different management states. Specifically, for segments that perform the action of retaining turning points, their corresponding learning state turning points and related behavior nodes are retained; for segments that perform the action of merging stable segments, the merged summary data is retained; for segments that perform the action of delaying the archiving of disputed segments, dispute archiving markers are written; for segments that perform the action of retrieving invalid segments, retrieval results are written; and for segments that perform the action of reinforcing missing segments, missing segment reinforcement results are written. Management section generation: The data management results of each learning state segment in the current round are rearranged according to the segment generation order to form an online learning operation data management section.

10. The system of claim 9, wherein the system is configured to run a big data management system. The reinforcement learning strategy feedback optimization module includes: Cross-section reuse evaluation: The online learning operation data management cross-sections formed in the current round are reused in the learning status identification, learning path adjustment and system resource scheduling processes, and the retention status of turning points, changes in data occupation, learning path adjustment results and resource scheduling results are collected after reuse. The turning point traceability rate, data occupation reduction rate, learning path offset value and resource scheduling deviation value are calculated to evaluate the reuse effect of the current data management actions. Reward feedback generation: Based on the turning point tracing rate, data occupancy reduction rate, learning path offset value, and resource scheduling deviation value, a reward feedback value is generated for the data management action corresponding to the segment of the current learning state, and an action feedback judgment value is generated. Specifically, when the turning point tracing rate reaches the turning point tracing rate threshold, the data occupancy reduction rate reaches the data occupancy reduction threshold, and the learning path offset value and resource scheduling deviation value do not exceed the corresponding threshold, the data management action is strengthened. When the turning point tracing rate does not reach the tracing rate threshold, or the learning path offset value and resource scheduling deviation value exceed the corresponding threshold, the data management action is suppressed. Policy update application: Input the reward feedback value and action feedback judgment value into the reinforcement learning agent to update the current action value function, obtain the updated reinforcement learning data management policy, and use the reinforcement learning data management policy for the next round of online learning system operation big data management.

Images