A reinforcement learning-based intelligent evaluation model dynamic optimization method

Abstract

The application relates to the technical field of intelligent evaluation, and particularly discloses a dynamic optimization method of an intelligent evaluation model based on reinforcement learning, which fine-tunes a basic intelligent evaluation model and general parameters thereof according to historical interaction data of a student object, performs prediction on current interaction data of the student object by using updated model parameters, calculates a benchmark prediction error between a benchmark prediction result and actual interaction, integrates recent performance statistical characteristics of the student object and current individualized adjustment parameters as a current reinforcement learning (RL) state, selects an adjustment action by using an RL Agent policy network, and calculates and updates the individualized adjustment parameters of the student object based on the adjustment action. The method performs individualized adjustment on the real-time learning state of each student, guarantees the stability and generalization ability of the basic model, and realizes rapid response and targeted adjustment on the state change of the student.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent assessment, and more specifically, to a dynamic optimization method of an intelligent assessment model based on reinforcement learning. BACKGROUND

[0002] In the process of education intelligence, the intelligent assessment model, as the core tool for evaluating the ability of learners and providing personalized guidance, can effectively evaluate the knowledge level, skill mastery or ability state of individuals. However, the learning process and ability of individuals are dynamically changing, and their performance will continue to evolve over time, learning experience, forgetting and other factors. Traditional intelligent assessment models usually use static parameter configuration, which is difficult to adapt to the dynamic evolution of students' knowledge mastery in the learning process, especially when facing a learning group with significant cognitive ability differences. The fixed model parameters often lead to evaluation results lagging behind the actual learning progress, and the prediction accuracy and evaluation accuracy of the model may decrease over time, which cannot provide continuous and accurate personalized evaluation or feedback.

[0003] Currently, the mainstream parameter optimization method of intelligent assessment models relies on periodic batch updating, which collects students' learning data regularly and uses offline analysis to retrain and adjust the model parameters. However, in the education scene, the continuous interaction data generated by students contains the changing patterns of learning behavior. This offline optimization mechanism cannot respond to changes in individual learning states in a timely manner, has problems such as long update cycle and parameter optimization lag, and cannot guarantee the timeliness and accuracy of the evaluation results.

[0004] Therefore, an optimized dynamic optimization method and system of an intelligent assessment model based on reinforcement learning are expected. SUMMARY

[0005] To solve the above technical problems, the present application is proposed. The embodiments of the present application provide a dynamic optimization method of an intelligent assessment model based on reinforcement learning, which fine-tunes the basic intelligent assessment model and its general parameters according to the historical interaction data of a student object, predicts the current interaction data of the student object with the updated model parameters, and then calculates the benchmark prediction error between the benchmark prediction result and the real interaction, integrates the recent performance statistical characteristics of the student object and the current personalized adjustment parameters as the current reinforcement learning (RL) state, uses the RL Agent policy network to select adjustment actions, and updates the personalized adjustment parameters of the student object based on the adjustment actions. This method, based on the core parameters of the basic assessment model, adjusts the real-time learning state of each student individually, ensuring the stability and generalization ability of the basic model, and achieving rapid response and targeted adjustment to changes in the state of individual students.

[0006] According to an aspect of the present application, a reinforcement learning-based intelligent assessment model dynamic optimization method is provided, which comprises:

[0007] obtaining current interaction data of a first student object and a historical interaction sequence of the first student object;

[0008] using a basic intelligent assessment model and its general parameters, predicting the current interaction data based on the historical interaction sequence of the first student object to obtain a baseline prediction result;

[0009] calculating a baseline prediction error between the baseline prediction result and the real interaction, and integrating the baseline prediction error, recent performance statistical features and current personalized adjustment parameters to obtain a current RL state;

[0010] inputting the current RL state into an RL Agent policy network to obtain a selected adjustment action;

[0011] based on the selected adjustment action, calculating new personalized adjustment parameters.

[0012] Compared with the prior art, the reinforcement learning-based intelligent assessment model dynamic optimization method provided by the present application fine-tunes the basic intelligent assessment model and its general parameters according to the historical interaction data of the student object, predicts the current interaction data of the student object with the updated model parameters, and then calculates the baseline prediction error between the baseline prediction result and the real interaction, and integrates the recent performance statistical features of the student object and the current personalized adjustment parameters as the current reinforcement learning (RL) state, uses the RL Agent policy network to select the adjustment action, and calculates and updates the personalized adjustment parameters of the student object based on the adjustment action. This method, based on the core parameters of the basic assessment model, makes personalized adjustments according to the real-time learning state of each student individual, which not only ensures the stability and generalization ability of the basic model, but also realizes the rapid response and targeted adjustment of the state changes of the student individuals. BRIEF DESCRIPTION OF DRAWINGS

[0013] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description of embodiments of the present application, taken in conjunction with the accompanying drawings. The drawings provided in the specification and the embodiments of the present application together serve to provide a further understanding that enables one of ordinary skill in the art to make and use the present application. The drawings provided are for illustrative purposes and are not intended to limit the present application in any way. In the drawings, the same reference numerals generally refer to the same components or steps throughout the drawings.

[0014] Figure 1 A flowchart of the reinforcement learning-based intelligent assessment model dynamic optimization method according to the embodiments of the present application.

[0015] Figure 2Data flow diagram for the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application.

[0016] Figure 3 Flowchart of sub-step S2 of the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application.

[0017] Figure 4 Flowchart of sub-step S5 of the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application.

[0018] Figure 5 Flowchart of sub-step S52 of the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application.

[0019] Figure 6 Flowchart of sub-step S522 of the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application.

[0020] Figure 7 Flowchart of sub-step S5223 of the method of dynamic optimization of an intelligent evaluation model based on reinforcement learning according to an embodiment of the application. DETAILED DESCRIPTION

[0021] As used in this application and the appended claims, the articles "a," "an," and "the" include both the singular and plural, unless the context dictates otherwise. The term "comprising" and variations thereof as used in this application and the appended claims is used generically and includes both the open- and closed-ended inclusion of the steps and elements identified in the specification, and allows for additional steps and elements to be present.

[0022] Although the present application makes various references to certain modules in the system according to embodiments of the application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0023] Flowcharts have been used in the present application to illustrate the operations performed by the system according to embodiments of the application. It should be understood that the preceding or following operations are not necessarily performed in the exact order. Rather, various steps can be processed in reverse order, or at the same time, as desired. Other operations can also be added to, or removed from, these processes, or one or more operations can be removed from these processes.

[0024] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part but not all of the embodiments of the present application, and the present application can be implemented in many different ways. Therefore, the contents described should be considered as illustrative rather than limiting the present application.

[0025] It is worth noting that in this application, all actions of obtaining data are carried out in accordance with the data protection regulations policy of the place country and with the authorization given by the corresponding device owner.

[0026] In view of the technical problems described in the above background art, the present application proposes a reinforcement learning-based intelligent evaluation model dynamic optimization method, which fine-tunes the basic intelligent evaluation model and its general parameters according to the historical interaction data of a student object, predicts the current interaction data of the student object with the updated model parameters, and then calculates the benchmark prediction error between the benchmark prediction result and the real interaction, and integrates the recent performance statistical characteristics and the current individualized adjustment parameters as the current reinforcement learning (RL) state, uses the RL Agent policy network to select the adjustment action, and calculates and updates the individualized adjustment parameters of the student object based on the adjustment action. This method, based on the core parameters of the basic evaluation model, makes individualized adjustments according to the real-time learning state of each student, which not only ensures the stability and generalization ability of the basic model, but also realizes the rapid response and targeted adjustment to the changes of the individual state of the student.

[0027] Figure 1 A flowchart of the reinforcement learning-based intelligent evaluation model dynamic optimization method according to an embodiment of the present application. Figure 2 A data flow diagram of the reinforcement learning-based intelligent evaluation model dynamic optimization method according to an embodiment of the present application. As shown in Figure 1 and Figure 2 The reinforcement learning-based intelligent evaluation model dynamic optimization method comprises the following steps: S1, obtaining the current interaction data of a first student object and the historical interaction sequence of the first student object; S2, using a basic intelligent evaluation model and its general parameters, predicting the current interaction data based on the historical interaction sequence of the first student object to obtain a benchmark prediction result; S3, calculating the benchmark prediction error between the benchmark prediction result and the real interaction, and integrating the benchmark prediction error, the recent performance statistical characteristics and the current individualized adjustment parameters to obtain a current RL state; S4, inputting the current RL state into the RL Agent policy network to obtain a selected adjustment action; S5, calculating new individualized adjustment parameters based on the selected adjustment action.

[0028] In the above intelligent evaluation model dynamic optimization method based on reinforcement learning, the step S1 acquires the current interaction data of the first student object and the historical interaction sequence of the first student object. It should be understood that, since the learning behavior of a student has a time sequence correlation, and the cognitive state evolution is deeply influenced by the historical learning trajectory. Therefore, in order to capture the long-term dependence in the dynamic learning mode of the student, the current interaction data (such as the answer record, the interaction time length, the knowledge point staying time) and the historical interaction sequence (containing the behavior features of multiple learning stages in the past) of the student are collected in real time to construct a continuous learning state representation. Specifically, the interaction data stream generated by the student on the intelligent evaluation platform can be acquired in real time through a data interface, and the structured data containing the time stamp, the knowledge point label, the behavior type, the answer result and the like dimensions are extracted by using a sliding window mechanism, and the historical interaction sequence of the student is retrieved in combination with a distributed storage engine. In this way, a multi-granularity data basis covering short-term behavior features and long-term learning rules can be established to provide time sequence context support for subsequent evaluation model dynamic optimization.

[0029] Specifically, first of all, a real-time data pipeline capable of continuously collecting and processing student interaction behaviors needs to be constructed. This data pipeline is usually composed of a front-end application interface, a back-end service interface and a distributed data storage system. When the student performs operations such as answering questions, watching teaching videos and participating in interactive exercises on the intelligent evaluation platform, these behaviors are recorded in real time and uploaded to the background server through a unified data collection interface. The collected content includes not only the answer results, the answer time and the knowledge point label of the student, but also fine-grained behavior features such as page staying time, click frequency and jump path. These information constitutes the core content of the current interaction data, reflecting the learning state and cognitive performance of the student at a specific moment.

[0030] At the same time, in order to more comprehensively understand the learning mode of the student, it is also necessary to effectively integrate the historical interaction sequence of the student. The historical interaction sequence refers to the interaction behavior records accumulated by the student in multiple learning stages in the past, including but not limited to previous answer conditions, stage test results, knowledge mastery curve trends and learning path evolution rules. These data are usually stored in a structured or semi-structured form in a database, and may be distributed at multiple time nodes, covering learning performance under different knowledge points, different question types and different difficulty levels. By calling these historical data, a complete graph of the long-term learning behavior of the student can be established, thereby providing a basis for personalized modeling.

[0031] To ensure the effective fusion of current interaction data and historical interaction sequences, a sliding window mechanism is often used within the system to extract multi-scale time series features. Specifically, the sliding window can capture a continuous learning behavior segment in the time dimension, such as all interaction records in the past 24 hours or relevant learning activities for a specific knowledge point in the past week. This mechanism helps capture the changing trends of students' short-term behavior patterns while also reflecting the stability of their long-term learning habits. In addition, the window size can be dynamically adjusted according to actual needs to adapt to different types of learning tasks and evaluation scenarios.

[0032] During data collection and integration, it is also necessary to combine the knowledge point label system to semantically annotate students' behavior data. For example, in a math problem, whether the student answers correctly, the time taken to answer, whether multiple attempts are made to modify the answer, and other behaviors can be labeled with corresponding knowledge point labels (such as "quadratic function" and "trigonometric identity transformation"), forming structured learning behavior representations. This labeling method allows the system to more accurately identify students' mastery of specific knowledge points in subsequent analysis and determine the direction of their cognitive state changes accordingly.

[0033] In addition to the knowledge point dimension, the organization of students' interaction data also needs to consider information in multiple dimensions such as behavior type, timestamp, device type, and network environment. For example, the same problem answered at different times may reflect different student abilities; learning behavior on mobile devices and desktop devices may also exhibit different attention distribution characteristics. Through comprehensive analysis of these auxiliary dimensions, the data representation capability and prediction value can be further improved, providing more abundant input signals for subsequent model optimization.

[0034] To ensure data integrity and timeliness, the system usually uses a distributed data storage engine to manage students' historical interaction sequences. Such storage systems have high concurrent read-write capability and flexible data scalability, supporting fast retrieval and update of massive user behavior logs. In actual deployment, an indexing mechanism can be established to categorize and organize each student's learning records, allowing quick access to relevant historical data when needed. In addition, a caching strategy can be introduced to pre-load frequently accessed learning behavior segments into memory, thereby speeding up data retrieval and reducing system response delay.

[0035] In the process of implementation, data quality control is also an important step that cannot be ignored. Because students may make mistakes, abnormal interruptions, repeated submissions, and other situations during the use of the platform, it is necessary to set up corresponding data cleaning rules to eliminate invalid or incorrect interaction records. For example, for the behavior of repeatedly answering questions in a short period of time, the system can automatically identify and filter out the part that does not conform to the normal learning logic; for the residual data of users who have not logged in for a long time, it can be cleaned or archived regularly according to the preset strategy. These measures help to improve the accuracy and consistency of the data, and avoid the deviation of the model prediction caused by noise interference.

[0036] It is worth noting that the collection and integration of student interaction data not only involves technical operations, but also involves data privacy and security protection issues. Therefore, the system needs to comply with relevant data compliance requirements, such as GDPR or other national and regional information security standards, from the beginning of design. Specific measures include desensitizing sensitive fields, limiting unauthorized access, using encrypted transmission protocols, etc., to ensure that students' learning behavior data is not leaked or misused during the flow process.

[0037] In the above-mentioned dynamic optimization method of the intelligent evaluation model based on reinforcement learning, the step S2 uses the basic intelligent evaluation model and its general parameters to predict the current interaction data based on the historical interaction sequence of the first student object to obtain the baseline prediction result. Specifically, since directly using a personalized model may have the problem of cold start or insufficient generalization. Therefore, in the initial stage, the application first uses the basic intelligent evaluation model and its general parameters which have been fully trained, and on this basis, fine-tunes the general parameters according to the historical interaction sequence of the student object to obtain fine-tuned model parameters that adapt to the current learning state of the student object, and then uses the updated model parameters to predict the current interaction data to obtain a more accurate baseline prediction result. In this way, the stability and generalization ability of the basic model are utilized, and the adaptability of the model to the current student state is improved through personalized fine-tuning. Among them, Figure 3 The flowchart of the sub-step S2 of the dynamic optimization method of the intelligent evaluation model based on reinforcement learning according to the embodiment of the application. As shown in Figure 3 The step S2 includes the steps of: S21, inputting each interaction data in the historical interaction sequence of the first student object into the basic intelligent evaluation model with general parameters in turn to obtain updated output weight and bias definitions as updated general parameters; S22, inputting the current interaction data into the basic intelligent evaluation model with updated general parameters to obtain the baseline prediction result.

[0038] Specifically, the step S21 inputs each interaction data in the historical interaction sequence of the first student object into the basic intelligent assessment model with general parameters in sequence to obtain updated output weight and bias definitions as updated general parameters. Specifically, since the learning track of a student individual has time sequence continuity, and the evolution of the knowledge state depends on the implicit patterns in the historical interaction behavior. Therefore, in order to realize preliminary personalized adaptation under the premise of maintaining the generalization ability of the model, the application uses the historical interaction sequence of the first student object to fine-tune the general parameters of the basic model online based on the dynamic context perception principle. In an embodiment of the application, the basic intelligent assessment model is based on the LSTM (Long Short-Term Memory Network) architecture, and in the initial stage, the pre-trained general parameters are loaded to form the general knowledge reasoning ability across student groups; when processing the target student, the historical interaction sequence (such as the time-ordered answering results, knowledge point interaction labels and stay time, etc.) of the student is input into the basic intelligent assessment model with general parameters in sequence according to the time steps, and through the time sequence memory characteristics of the LSTM network, in the forward propagation process at each time step, the network internal representation is dynamically adjusted through the hidden state transmission mechanism (cell state update of the LSTM network), so that the model gradually learns the learning mode and knowledge mastery trend specific to the student, and the weight and bias parameters of the model output layer are updated using the back propagation algorithm (set the learning rate of other layers to 0), forming a context perception fine-tuned model for the student. In this way, the basic intelligent assessment model retains the global knowledge structure while capturing the long-term dependencies in individual historical behavior, thereby establishing a dynamically evolving reasoning benchmark for the prediction of current interaction data.

[0039] Specifically, the step S22 inputs the current interaction data into the basic intelligent assessment model with updated general parameters to obtain the benchmark prediction result. That is, the updated output layer weight and bias are locked as the temporary reasoning parameters of the basic intelligent assessment model, and the current interaction data (such as the latest knowledge point interaction event) is input into the model. At this time, the forward calculation process of the basic intelligent assessment model with updated general parameters will utilize two types of knowledge: the cross-group regularity knowledge in the basic general parameters (hidden in the hidden layer weight), and the individual short-term behavior characteristics contained in the output layer parameters fine-tuned by the historical interaction data, and finally output the benchmark prediction result (such as the answer correctness rate prediction value) that reflects both the group commonality regularity and the individual historical behavior mode.

[0040] In the above intelligent evaluation model dynamic optimization method based on reinforcement learning, in step S3, the baseline prediction error between the baseline prediction result and the real interaction is calculated, and the baseline prediction error, the recent performance statistical characteristics and the current personalized adjustment parameters are integrated to obtain the current RL state. It should be understood that the present application takes into account that there may be differences between the baseline prediction result predicted by the basic intelligent evaluation model and the real interaction result of the student, which reflects the prediction accuracy of the basic intelligent evaluation model for the current performance of the student. Therefore, in order to enable the reinforcement learning agent to fully perceive the current situation and make reasonable decisions, the present application takes the baseline prediction error between the baseline prediction result and the real interaction as the core, quantifies the immediate deviation of the current parameter configuration, combines the recent performance statistical characteristics of the student (such as the correct answer rate change trend of the past N interactions, high-frequency error knowledge points, knowledge point coverage entropy, etc.) and the current personalized adjustment parameters (such as the result of the previous RL Agent decision, if it is the first time, it is defaulted to be empty), after feature splicing and dimension alignment, the current reinforcement learning (RL) state is constructed, so that the reinforcement learning agent can perceive the immediate error, the medium-term trend and the long-term strategy influence at the same time, and provide comprehensive state information for the subsequent reinforcement learning strategy.

[0041] In the above intelligent evaluation model dynamic optimization method based on reinforcement learning, the step S4 inputs the current RL state into the RL Agent policy network to obtain a selected adjustment action. It should be understood that, in order to realize the autonomous evolution of the dynamic optimization strategy, the present application realizes multi-modal adjustment decision through the action space design of the policy network based on the principle of deep reinforcement learning. Specifically, the RL Agent policy network adopts a double-branch architecture: the value function branch is used to evaluate the state value, and the policy branch is used to output the probability distribution of four types of actions (directly output new parameters, output parameter adjustment amount, select a predefined adjustment strategy, and maintain the status quo). The network is trained by the proximal policy optimization (PPO) algorithm, and the design of the reward function comprehensively considers the short-term error reduction rate, the long-term learning goal achievement degree, and the parameter adjustment amplitude penalty term. In the inference stage, the RL Agent policy network calculates the probability distribution of each action according to the current state vector, and selects the optimal action through Gumbel-Softmax sampling. In this way, the RL Agent can intelligently weigh the pros and cons of different adjustment strategies and select an action that can quickly reduce the prediction error while maintaining the stability and sustainable optimization of the model. For example, when the prediction error is large and the student's recent performance fluctuates significantly, the policy network may tend to select the direct output of new individualized adjustment parameters to quickly adjust the model state; when the model prediction is relatively stable and the student's knowledge mastery trend is relatively stable, it may choose to fine-tune the existing parameters or maintain the status quo to avoid the risk of model instability caused by excessive adjustment. Finally, the selected adjustment action will be output as the decision of the reinforcement learning agent, guiding the further individualized adjustment of the basic intelligent evaluation model, and forming a closed-loop intelligent evaluation and optimization process.

[0042] In the above intelligent evaluation model dynamic optimization method based on reinforcement learning, the step S5 calculates new individualized adjustment parameters based on the selected adjustment action. Wherein, Figure 4 The flowchart of the sub-step S5 of the intelligent evaluation model dynamic optimization method based on reinforcement learning according to the embodiment of the present application. As shown in Figure 4 The step S5 includes the steps of: S51, in response to the selected adjustment action, outputting an individualized adjustment parameter adjustment amount; S52, fusing the output individualized adjustment parameter adjustment amount with the current individualized adjustment parameter to obtain the new individualized adjustment parameter.

[0043] Specifically, the step S51, in response to the selected adjustment action, outputs an individualized adjustment parameter adjustment amount. That is, according to the adjustment action selected by the RL Agent policy network, a new individualized adjustment parameter is calculated and determined. If the new individualized adjustment parameter is directly output, the new individualized adjustment parameter is directly applied; if the individualized adjustment parameter adjustment amount is output, the parameter is adjusted based on the original parameter to obtain the new individualized adjustment parameter; if the pre-defined adjustment strategy is selected, the corresponding pre-defined strategy is called to update the parameter (such as increasing the learning rate, reducing the adjustment step, etc.); if no adjustment is selected, the current individualized adjustment parameter remains unchanged. The new individualized adjustment parameter will be used for the prediction of the next interaction data, and this cycle iteration is repeated to dynamically adjust the evaluation model according to the learning progress and performance of the student, so as to achieve more effective evaluation and optimization of the student's learning state.

[0044] Specifically, the step S52, the output individualized adjustment parameter adjustment amount is fused with the current individualized adjustment parameter to obtain the new individualized adjustment parameter. In particular, when the selected adjustment action is to output the individualized adjustment parameter adjustment amount, the high-dimensional characteristics of the parameter adjustment space are easy to cause the adjustment action to conflict with the existing configuration, that is, directly adding the values may cause parameter space conflict or semantic distortion. Therefore, in order to capture the implicit association between the individualized adjustment parameter adjustment amount and the current individualized adjustment parameter and maintain the stability of the adjustment process, the deep learning algorithm is further introduced to deeply interact and fuse the output individualized adjustment parameter adjustment amount and the current individualized adjustment parameter, so as to generate a new individualized adjustment parameter, so that the generated individualized adjustment parameter responds to the current RL state and conforms to the internal parameter distribution law of the model. Wherein, Figure 5 The flow chart of the sub-step S52 of the intelligent evaluation model dynamic optimization method based on reinforcement learning according to the embodiment of the present application. As shown in Figure 5 The step S52 includes the steps of: S521, structurally encoding the output individualized adjustment parameter adjustment amount and the current individualized adjustment parameter to obtain a parameter adjustment amount structured encoding vector and a current adjustment parameter structured encoding vector; S522, multi-scale progressive interaction is performed on the parameter adjustment amount structured encoding vector and the current adjustment parameter structured encoding vector to obtain an updated individualized parameter high-dimensional representation; S523, feature decoding is performed on the updated individualized parameter high-dimensional representation to obtain the new individualized adjustment parameter.

[0045] More specifically, the step S521, the output personalized adjustment parameter adjustment amount and the current personalized adjustment parameter are structured coded to obtain the parameter adjustment amount structured coding vector and the current adjustment parameter structured coding vector. It should be understood that considering that the model parameters have high dimension and complex topological relationship, therefore, the present application is based on the principle of graph representation learning, and the parameter system is mapped to a resolvable topological space by structured coding. Specifically, first, the current personalized adjustment parameter (such as the model weight matrix θ_t) and the adjustment amount Δθ output by the reinforcement learning are respectively modeled as graph structure data: each graph structure node corresponds to a weight or bias item of a network layer in the model, the node feature includes parameter value, gradient history and other meta information, and the edge weight is determined by the correlation between nodes, such as the gradient correlation in back propagation. Subsequently, the graph convolution network (GCN) is used to perform hierarchical aggregation on the two parameter graphs to generate the parameter adjustment amount structured coding vector and the current adjustment parameter structured coding vector. In this way, not only the numerical characteristics of the parameters are retained, but also the functional coupling relationship between the parameters is deeply mined, which provides a semantically aligned vector space for subsequent deep interaction fusion.

[0046] More specifically, the step S522, the parameter adjustment amount structured coding vector and the current adjustment parameter structured coding vector are processed by a hierarchical feature interaction mechanism to realize multi-granularity knowledge fusion, and a composite feature expression considering global stability and local sensitivity, that is, the update personalized parameter high-dimensional representation is formed. Wherein, Figure 6 The flow chart of the sub-step S522 of the intelligent evaluation model dynamic optimization method based on reinforcement learning according to the embodiment of the present application. As shown in FIG. 6, the step S522 includes steps S5221-S5223. Figure 6As shown, the step S522 includes steps of: S5221, performing low-level feature interaction on the parameter adjustment structured coding vector and the current adjustment parameter structured coding vector to obtain a parameter adjustment-current adjustment parameter low-level joint perception feature vector; S5222, performing multi-level high-order feature interaction on the parameter adjustment structured coding vector and the current adjustment parameter structured coding vector to obtain a parameter adjustment-current adjustment parameter middle-level joint perception feature vector and a parameter adjustment-current adjustment parameter deep-level joint perception feature vector; and S5223, performing progressive interaction fusion on the parameter adjustment-current adjustment parameter low-level joint perception feature vector, the parameter adjustment-current adjustment parameter middle-level joint perception feature vector and the parameter adjustment-current adjustment parameter deep-level joint perception feature vector to obtain the updated personalized parameter high-dimensional representation.

[0047] In one specific example of the present application, the step S5221 is expressed by a formula as follows:

[0048]

[0049] wherein, represents position point addition, represents a multi-layer perception machine model, represents a parameter adjustment-current adjustment parameter low-level joint perception feature vector.

[0050] That is, by directly performing low-level interaction on the parameter adjustment structured coding vector and the current adjustment parameter structured coding vector, high-frequency detail information and accurate correspondence are retained, and detail loss that may occur in the deep network layer-by-layer abstraction process is avoided, so as to capture the association of the parameter adjustment and the current adjustment parameter in the fine dimension such as the pixel level, the timestamp level or the word level, and the obtained parameter adjustment-current adjustment parameter low-level joint perception feature vector contains fine association information in the original feature that is not abstracted and lost, thereby providing more detailed support for subsequent parameter optimization.

[0051] In one specific example of the present application, the step S5222 includes: first, performing multi-level implicit feature extraction on the parameter adjustment structured coding vector and the current adjustment parameter structured coding vector to obtain a parameter adjustment middle-level implicit feature coding vector, a current adjustment parameter middle-level implicit feature coding vector, a parameter adjustment deep-level implicit feature coding vector and a current adjustment parameter deep-level implicit feature coding vector, which is expressed by a formula as follows:

[0052]

[0053]

[0054]

[0055]

[0056] wherein, denotes a parameter adjustment amount structured encoding vector, denotes a current adjustment parameter structured encoding vector, and denote a weight matrix and a bias term of a middle-level implicit feature encoding vector extraction network respectively, denotes a ReLU activation function, denotes a middle-level implicit feature encoding vector of a parameter adjustment amount, denotes a middle-level implicit feature encoding vector of a current adjustment parameter, and denote a weight matrix and a bias term of a deep-level implicit feature encoding vector extraction network respectively, denotes a Sigmoid activation function, and denote a deep-level implicit feature encoding vector of a parameter adjustment amount and a deep-level implicit feature encoding vector of a current adjustment parameter respectively.

[0057] That is, the parameter adjustment amount structured encoding vector and the current adjustment parameter structured encoding vector are independently processed by using the nonlinear transformation and hierarchical feature learning ability of the deep neural network, so as to extract implicit features of different abstract granularity, to provide multi-dimensional information support for subsequent cross-feature interaction, so that the model can understand the association between parameter adjustment and current state from different levels. The parameter adjustment amount middle-level implicit feature encoding vector, the current adjustment parameter middle-level implicit feature encoding vector, the parameter adjustment amount deep-level implicit feature encoding vector and the current adjustment parameter deep-level implicit feature encoding vector generated in this way represent the internal relationship between the parameter adjustment amount and the current adjustment parameter at the structured information level and the global semantic level respectively, which not only retains the structural features with stability in the parameter adjustment process, but also extracts the core semantic information related to the dynamic change of the learning state, thereby providing a hierarchical information basis for subsequent middle-level and deep-level feature interaction.

[0058] Then, the middle-level feature interaction is performed on the parameter adjustment amount middle-level implicit feature encoding vector and the current adjustment parameter middle-level implicit feature encoding vector to obtain a parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector, which is expressed by the formula as follows:

[0059]

[0060]

[0061] wherein, is an attention fusion network, This indicates dot product by position. This represents the parameter adjustment amount – the joint sensing feature vector of the current adjustment parameters at different levels. for The feature scale value, Represents the transpose of a vector. It is a normalized exponential function.

[0062] In other words, by performing mid-level interaction between the latent feature encoding vectors of the parameter adjustment amount and the current adjustment parameter, and filtering out redundant information while retaining stable structural patterns, the dependency relationship between the two at the abstract structural level is modeled. This captures a more complex cross-source relationship than the low-level joint sensing feature vector of the parameter adjustment amount and the current adjustment parameter, providing moderately complex structural information support for subsequent parameter optimization. This allows the model to understand the adaptability of parameter adjustment to the current learning state from a structured correlation perspective. Based on this, the generated mid-level joint sensing feature vector of the parameter adjustment amount and the current adjustment parameter effectively represents the interaction relationship between the parameter adjustment amount and the current adjustment parameter at the structural level, eliminating noise interference in the original features and encoding invariant structural patterns. This enables the model to dynamically adjust based on moderately complex structural dependencies when updating personalized parameters, improving the structural level response accuracy of the evaluation model to changes in individual learning states.

[0063] Finally, deep-level feature interaction is performed on the deep latent feature encoding vector of the parameter adjustment amount and the deep latent feature encoding vector of the current adjustment parameter to obtain the deep-level joint perceptual feature vector of the parameter adjustment amount and the current adjustment parameter, which is expressed by the formula:

[0064]

[0065]

[0066] in, and For different low-rank projection matrices, This represents the GELU activation function. This represents the LayerNorm normalization function. This is the weight matrix of a deep feature fusion network. This represents the parameter adjustment amount - the current adjustment parameter linear projection gating interactive feature. This represents the parameter adjustment amount - the deep-level joint sensing feature vector of the currently adjusted parameters.

[0067] That is, by deep interaction between the parameter adjustment amount deep implicit feature encoding vector and the current adjustment parameter deep implicit feature encoding vector, the synergistic association of both in the core semantic, global learning state context or optimization target level is captured, the characteristics of high information density and dimension reduction of deep features are used to realize reasoning and alignment at the semantic level, and abstract information support about learning state evolution trend and parameter adjustment global significance is provided for personalized parameter update, so that the model can understand the adaptation relationship between parameter adjustment and student overall learning progress from the global semantic perspective. Based on this, the generated parameter adjustment amount-current adjustment parameter deep level joint perception feature vector extracts the core concept of learning state change in the form of low-dimensional and high-density representation, providing a semantic level decision basis for subsequent cross-level feature fusion.

[0068] Figure 7 The flowchart of sub-step S5223 of the intelligent evaluation model dynamic optimization method based on reinforcement learning according to the embodiment of the present application is shown. As shown in Figure 7 the step S5223, it includes the steps of: S52231, complementary interaction fusion based on a gating mechanism is performed on the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector to obtain a parameter adjustment amount-current adjustment parameter middle-low-level joint perception feature vector; S52232, cross-level interaction based on a cross-attention mechanism is performed on the parameter adjustment amount-current adjustment parameter middle-low-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter deep-level joint perception feature vector to obtain the updated personalized parameter high-dimensional representation.

[0069] In one specific example of the present application, the step S52231 is expressed by the formula:

[0070]

[0071]

[0072] wherein, represents feature concatenation, represents the weight matrix of the progressive complementary perception network, represents and the gating interaction weight vector between and represents the parameter adjustment amount-current adjustment parameter middle-low-level joint perception feature vector.

[0073] That is, through the gating mechanism, the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector are dynamically weighted, the complementary fusion of the two in the details and the structured mode is realized, and the generated parameter adjustment amount-current adjustment parameter middle-low level joint perception feature vector can adaptively fuse the detail accuracy of the low-level feature and the structural stability of the middle-level feature, and through the gating mechanism, the redundant information is filtered and the key association is strengthened, so that the model can simultaneously consider the subtle changes and structural evolution trend of learning behavior when updating the personalized parameters, and lay a feature foundation with details and structure for deep-level semantic fusion.

[0074] In particular, in one preferred example of the present application, the step S52232 comprises: first, obtaining a query embedding matrix, a key embedding matrix and a value embedding matrix, and performing multi-objective weight collaborative allocation on the query embedding matrix, the key embedding matrix and the value embedding matrix to obtain an optimized query embedding matrix, an optimized key embedding matrix and an optimized value embedding matrix. Here, considering that when the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector are complementarily interacted and fused based on the gating mechanism, the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector substantially generate a gating interaction weight vector, and then the gating interaction weight vector is fused based on the phase synchronization complement, and further constrained in a structured measure space via the query embedding matrix, the key embedding matrix and the value embedding matrix to realize interaction. Due to the instability of the deconstruction of the structured measure space constraint, the interaction efficiency is limited. Based on this, the present application further calculates the field decomposition integral representation of the query embedding matrix, the key embedding matrix and the value embedding matrix, and suppresses the non-integrable accumulation of the constraint association of the structured measure space through the integral expression of each decomposition field of the structured measure space, to avoid the formation of the geometric phase accumulation effect caused by the suppression of the local field structure feedback.

[0075] Specifically, first, for the pre-trained matrices 、 and , the decomposition field path integral representation of each matrix is calculated with the combination of the three as a coupled order parameter topological domain:

[0076]

[0077]

[0078]

[0079] wherein, , and denote a query embedding matrix, a key embedding matrix and a value embedding matrix, respectively, , and denote an optimized query embedding matrix, an optimized key embedding matrix and an optimized value embedding matrix, respectively.

[0080] Then, a field state balance alignment loss function is defined as:

[0081]

[0082] wherein, denotes a kernel norm, i.e., a sum of eigenvalues of a matrix, denotes a modulation hyperparameter, denotes a field state balance alignment loss function value.

[0083] In this way, the deconstruction stability regulation can be constrained by the field state balance mapping relationship of the sum of eigenvalues of the coupling order parameter topological domain of the embedding matrix, so as to realize the interaction maintenance of the query embedding matrix, the key embedding matrix and the value embedding matrix, thereby generating the optimized query embedding matrix, the optimized key embedding matrix and the optimized value embedding matrix.

[0084] Then, the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector is embedded and encoded using the optimized query embedding matrix to obtain a query vector, and the parameter adjustment amount-current adjustment parameter deep-level joint perception feature vector is embedded and encoded using the optimized key embedding matrix and the optimized value embedding matrix to obtain a key vector and a value vector, which are represented by formulas as:

[0085]

[0086]

[0087]

[0088] wherein, , and denote a query vector, a key vector and a value vector, respectively.

[0089] That is, by optimizing the query embedding matrix, optimizing the key embedding matrix and optimizing the value embedding matrix, the space mapping of different hierarchical feature vectors is performed, the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector is converted into a query vector focusing on detailed and structural information, and the parameter adjustment amount-current adjustment parameter deep-level joint perception feature vector is respectively mapped into a key vector and a value vector carrying global semantic association, thereby constructing a feature representation space suitable for cross-level interaction for subsequent cross-attention mechanism, enabling the model to search for key details in the middle-low level features based on deep-level semantic information, and realizing semantic alignment and weight allocation of multi-scale features.

[0090] Finally, the query vector, the key vector and the value vector are subjected to cross-level interaction based on the transformer architecture to obtain the updated personalized parameter high-dimensional representation, which is expressed by the formula:

[0091]

[0092] Among them, represents the updated personalized parameter high-dimensional representation.

[0093] That is, the query vector, the key vector and the value vector are subjected to progressive cross-level interaction to dynamically integrate complementary information between different abstract granularity features, thereby generating an updated personalized parameter high-dimensional representation that integrates multi-scale interaction perception, providing a comprehensive feature basis with detailed accuracy, structural stability and semantic consistency for personalized parameter updating, and ensuring that the subsequent parameter adjustment result meets the overall goal of subtle evolution of learning behavior patterns and cognitive development.

[0094] More specifically, in one specific example of the present application, the step S523 comprises: inputting the updated personalized parameter high-dimensional representation into a multi-layer perceptron-based feature decoder to obtain the new personalized adjustment parameter. Specifically, to ensure that the output parameter meets the model architecture constraints and maintains numerical stability, the present application further implements the inverse mapping of the feature space to the parameter space through a constraint decoding network. In one embodiment of the present application, the multi-layer perceptron (MLP)-based feature decoder adopts a phased decoding strategy: the first layer of network projects the updated personalized parameter high-dimensional representation to an intermediate space of the same dimension as the original parameter, and applies L2 regularization constraint to prevent numerical overflow; the second layer introduces a parameter grouping normalization mechanism, which performs grouping standardization according to parameter types (such as weights, biases) or functional modules (such as attention layers, fully connected layers); the final output layer adopts a Tanh activation function to constrain the numerical range to the interval [-1, 1], and then restores it to the actual parameter range through a learnable scaling coefficient. In addition, a parameter relationship constraint loss function is embedded in the decoding process to maintain the internal structure of the parameter system by minimizing the topological difference between the new and old parameter graphs (such as the KL divergence of the similarity matrix between nodes). In this way, the new personalized adjustment parameter can effectively integrate the optimization direction of the reinforcement learning strategy, while strictly following the inherent distribution rules of the model parameters, ensuring that the evaluation model always maintains stability and functional integrity during dynamic adjustment.

[0095] In summary, the reinforcement learning-based intelligent evaluation model dynamic optimization method based on the embodiments of the present application is illustrated, which fine-tunes the basic intelligent evaluation model and its general parameters according to the historical interaction data of the student object, predicts the current interaction data of the student object with the updated model parameters, and then calculates the benchmark prediction error between the benchmark prediction result and the real interaction, integrates the recent performance statistical characteristics of the student object and the current personalized adjustment parameter as the current reinforcement learning (RL) state, uses the RL Agent policy network to select the adjustment action, and updates the personalized adjustment parameter of the student object based on the adjustment action. This method, based on the core parameters of the basic evaluation model, adjusts the real-time learning state of each student individually, ensuring the stability and generalization ability of the basic model, and achieving rapid response and targeted adjustment to the changes in the state of individual students.

[0096] The basic principles of the present application are described above in combination with specific embodiments, but it should be noted that the advantages, advantages, effects, etc. mentioned in the present application are only examples and not limitations, and these advantages, advantages, effects, etc. cannot be considered as the must-have of each embodiment of the present application. In addition, the specific details of the above embodiments are only for the purpose of example and understanding, and are not limited to the above specific details. The present application can be realized without the above specific details.

[0097] In the above described embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments. In the several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the system embodiments described above are merely schematic, for example, the unit division is only a logical function division, and other division manners can be used in actual implementation. The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. In actual implementation, some or all of the units can be selected according to the actual needs to achieve the purposes of the embodiments.

[0098] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and range of equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be considered as limiting the claims.

[0099] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units stated in the system claim can also be implemented by one unit through software or hardware.

[0100] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for dynamic optimization of an intelligent assessment model based on reinforcement learning, characterized in that, The method comprises: obtaining current interaction data of a first student object and a historical interaction sequence of the first student object; using a basic intelligent assessment model and its general parameters, predicting the current interaction data based on the historical interaction sequence of the first student object to obtain a baseline prediction result; calculating a baseline prediction error between the baseline prediction result and real interaction, and integrating the baseline prediction error, recent performance statistical features and current personalized adjustment parameters to obtain a current RL state; inputting the current RL state into an RL Agent policy network to obtain a selected adjustment action; based on the selected adjustment action, calculating a new personalized adjustment parameter; based on the selected adjustment action, calculating a new personalized adjustment parameter, comprising: outputting a personalized adjustment parameter adjustment amount in response to the selected adjustment action; structurally encoding the output personalized adjustment parameter adjustment amount and the current personalized adjustment parameter to obtain a parameter adjustment amount structured encoding vector and a current adjustment parameter structured encoding vector; performing low-level feature interaction on the parameter adjustment amount structured encoding vector and the current adjustment parameter structured encoding vector to obtain a parameter adjustment amount-current adjustment parameter low-level joint perception feature vector; performing multi-level implicit feature extraction on the parameter adjustment amount structured encoding vector and the current adjustment parameter structured encoding vector to obtain a parameter adjustment amount middle-level implicit feature encoding vector, a current adjustment parameter middle-level implicit feature encoding vector, a parameter adjustment amount deep-level implicit feature encoding vector and a current adjustment parameter deep-level implicit feature encoding vector; performing middle-level feature interaction on the parameter adjustment amount middle-level implicit feature encoding vector and the current adjustment parameter middle-level implicit feature encoding vector to obtain a parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector; performing deep-level feature interaction on the parameter adjustment amount deep-level implicit feature encoding vector and the current adjustment parameter deep-level implicit feature encoding vector to obtain a parameter adjustment amount-current adjustment parameter deep-level joint perception feature vector; performing progressive interaction fusion on the parameter adjustment amount-current adjustment parameter low-level joint perception feature vector, the parameter adjustment amount-current adjustment parameter middle-level joint perception feature vector and the parameter adjustment amount-current adjustment parameter deep-level joint perception feature vector to obtain an updated personalized parameter high-dimensional representation; performing feature decoding on the updated personalized parameter high-dimensional representation to obtain a new personalized adjustment parameter.

2. The method of claim 1, wherein the method further comprises: The selected adjustment action comprises directly outputting a new personalized adjustment parameter, outputting a personalized adjustment parameter adjustment amount, selecting a predefined adjustment strategy and not adjusting.

3. The method of claim 1, wherein the method further comprises: using a basic intelligent assessment model and its general parameters, predicting the current interaction data based on the historical interaction sequence of the first student object to obtain a baseline prediction result, comprising: inputting each interaction data in the historical interaction sequence of the first student object into the basic intelligent assessment model with general parameters in turn to obtain updated output weight and bias definitions as updated general parameters; inputting the current interaction data into a basic intelligent evaluation model with updated general parameters to obtain the benchmark prediction result.

4. The method of claim 3, wherein the method further comprises: progressively interactively fusing the parameter adjustment-current adjustment parameter low-level joint perception feature vector, the parameter adjustment-current adjustment parameter middle-level joint perception feature vector, and the parameter adjustment-current adjustment parameter deep-level joint perception feature vector to obtain the updated personalized parameter high-dimensional representation, including: complementarily interactively fusing the parameter adjustment-current adjustment parameter low-level joint perception feature vector and the parameter adjustment-current adjustment parameter middle-level joint perception feature vector based on a gating mechanism to obtain a parameter adjustment-current adjustment parameter middle-low-level joint perception feature vector; cross-level interacting the parameter adjustment-current adjustment parameter middle-low-level joint perception feature vector and the parameter adjustment-current adjustment parameter deep-level joint perception feature vector based on a cross-attention mechanism to obtain the updated personalized parameter high-dimensional representation.

5. The method of claim 4, wherein the method further comprises: cross-level interacting the parameter adjustment-current adjustment parameter middle-low-level joint perception feature vector and the parameter adjustment-current adjustment parameter deep-level joint perception feature vector based on a cross-attention mechanism to obtain the updated personalized parameter high-dimensional representation, including: obtaining a query embedding matrix, a key embedding matrix, and a value embedding matrix, and performing multi-objective weight collaborative allocation on the query embedding matrix, the key embedding matrix, and the value embedding matrix to obtain an optimized query embedding matrix, an optimized key embedding matrix, and an optimized value embedding matrix; using the optimized query embedding matrix to perform embedding encoding on the parameter adjustment-current adjustment parameter middle-low-level joint perception feature vector to obtain a query vector, and using the optimized key embedding matrix and the optimized value embedding matrix to perform embedding encoding on the parameter adjustment-current adjustment parameter deep-level joint perception feature vector to obtain a key vector and a value vector, respectively; cross-level interacting the query vector, the key vector, and the value vector based on a transformer architecture to obtain the updated personalized parameter high-dimensional representation.

6. The method of claim 5, wherein the method further comprises: performing feature decoding on the updated personalized parameter high-dimensional representation to obtain the new personalized adjustment parameter, including: inputting the updated personalized parameter high-dimensional representation into a feature decoder based on a multi-layer perception machine to obtain the new personalized adjustment parameter.

Images