A service quality dynamic evaluation method for an online teaching platform

Abstract

The present application relates to the technical field of teaching quality evaluation, in particular to a service quality dynamic evaluation method for an online teaching platform, comprising: collecting learning behavior data of an online education platform based on the scene of teaching task access; decomposing the obtained learning behavior data in multiple modal scenes to determine the scene feature matrix under each scene; integrating real-time learning behavior data based on the scene feature matrix under each scene to determine the real-time interaction degree of learning behavior, and constructing a learning behavior portrait according to the response time under real-time interaction; calculating the group response time of each teaching task based on the obtained learning behavior portrait, and determining the interaction mode exhibited by the teaching task according to the task completion status corresponding to the teaching task; and based on the interaction mode exhibited by the teaching task, using the isolation forest algorithm for time series regression verification to determine the evaluation score of each user. The efficiency and accuracy of data processing are improved.

Description

Technical Field

[0001] This invention relates to the field of teaching quality assessment technology, specifically a dynamic evaluation method for the service quality of online teaching platforms. Background Technology

[0002] With the development of intelligent teaching platforms, the service quality of these platforms has gradually become a criterion for evaluating student behavior and teaching quality, and a result-oriented indicator that directly determines the stability of the teaching order and the fairness of the educational process. Existing technologies primarily focus on content recommendation and learning assessment, analyzing student learning behavior, answer records, and knowledge mastery to recommend learning paths and adapt content. However, considering the actual learning behaviors in various scenarios, problems such as unclear scenario segmentation and limited analysis of learning content can easily arise.

[0003] For example, Chinese Patent Publication No. CN120542747A discloses an AI-enabled smart teaching personalized service method and system, which relates to the field of smart teaching technology. The method includes: periodically collecting multimodal data of current students; performing correlation analysis on learning data and behavioral data in the multimodal data to generate analysis results; constructing a weight matrix based on group labels and ability prediction models in the analysis results, and dividing all knowledge nodes into hierarchical courses based on the weight matrix; determining the proportion of core knowledge and the proportion of extended knowledge based on the ability prediction model and learning style vector, and then fusing them to generate a fusion ratio; selecting reinforcement training sequences and extended content sequences from the core modules and extended modules respectively, and cross-arranging the reinforcement training sequences and extended content sequences according to the fusion ratio to generate a dynamic learning path.

[0004] For example, Chinese Patent Publication No. CN120146682A discloses a teaching evaluation method, device, storage medium, and program product, which relates to the field of teaching quality evaluation technology. The method includes: receiving a first user sending a first distribution instruction through a first server in the teaching classroom to send a first classroom questionnaire set to a second user group and a second server group in the teaching classroom; obtaining a second classroom questionnaire set uploaded by the second server group; receiving a second distribution instruction from the first user through the first server to send a first extracurricular questionnaire set to a second user group and a third server group outside the teaching classroom; obtaining a second extracurricular questionnaire set uploaded by the third server group; fusing the second classroom questionnaire set and the second extracurricular questionnaire set to obtain a target archive dataset; and evaluating the teaching of the target user group using the target archive dataset.

[0005] Existing technologies assess teaching quality by group tags and the degree of group mastery of knowledge points; or by using questionnaires and processing the data to characterize teaching quality. However, these technologies tend to focus on a single dimension of assessment, centered on learning behavior and teaching outcomes, neglecting the impact of online platform services on the teaching process. They cannot distinguish whether user learning abnormalities stem from their own abilities or platform service defects, resulting in a lack of scenario-based assessment in service quality evaluation and making the teaching service experience unpredictable for different user groups. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention provides a method for dynamic evaluation of service quality for online teaching platforms, comprising: collecting learning behavior data from the online education platform based on the scenario of accessing teaching tasks.

[0007] The acquired learning behavior data is decomposed into multimodal scenarios to determine the scene feature matrix for each scenario.

[0008] Based on the scene feature matrix of each scenario, real-time learning behavior data is integrated to determine the real-time interaction level of learning behavior, and a learning behavior profile under real-time interaction is constructed according to the response time under real-time interaction.

[0009] Based on the acquired learning behavior profiles, the group response time for each teaching task is calculated, and the interaction mode for the teaching task is determined according to the task completion status.

[0010] Based on the interactive mode of the teaching task presentation, the isolated forest algorithm is used to perform time series regression verification to determine the evaluation score of each user during learning.

[0011] The beneficial effects of this invention are as follows: First, this invention, based on the teaching task access scenario, constructs a task distribution vector according to the teaching task distribution time point, identifies the target user group in the corresponding scenario, and finally collects a learning behavior dataset composed of user experience data and business result data. This avoids ignoring specific teaching scenarios and users during data analysis, which would lead to missing data processing and insufficient scenario analysis in subsequent processes.

[0012] Second, this invention performs hierarchical clustering on learning behavior data to form scenario clusters that match the teaching business hierarchy. Based on the feature center vector of the scenario cluster, it performs initial screening using business semantic association rules, and then uses the ratio of the feature coverage in the current scenario cluster to the average coverage of all other scenario clusters as the quantification separation condition to separate single scenario features from candidate shared features. This achieves hierarchical segmentation of teaching scenarios, and by separating single scenario features from shared features, it clarifies the main analysis content in the current scenario, improving the accuracy of subsequent data analysis.

[0013] Third, this invention configures initial weighting factors for real-time interaction based on a scene feature matrix. These initial weighting factors are adjusted using metrics such as the percentage of effective interactions, average interaction duration, and the rate of change in interaction frequency. Ultimately, the frequency of these adjustments quantifies the level of real-time interaction in the current time period. Simultaneously, corresponding response times are extracted from real-time learning behavior data to perform parallel trend analysis of learning behavior. Finally, a combination of parallel trend data and real-time interaction levels is used to construct learning behavior profiles for different scenarios. This clarifies the level of user participation in learning and the platform's service adaptation to interactive behavior, characterizing the relationship between service quality and user learning behavior in a relatively parallel manner.

[0014] IV. This invention extracts response times corresponding to the current teaching task from learning behavior profiles and aggregates them into group response times. It then performs differentiated association verification based on the teaching task execution status: during normal execution, the quantile and average values ​​of the group response times are used as association features; during abnormal execution, the dispersion coefficient of the group response times and abnormal task completion data are used as association features, ultimately outputting the association results. Based on the association results, it retrieves and matches the interaction patterns of the corresponding teaching tasks. By demonstrating the differences in service experience among user groups, the overall interaction patterns are quantitatively displayed, and the interaction situations in different scenarios are explained simultaneously. This provides a traceable data combination for platform service analysis, improving the efficiency and accuracy of data processing.

[0015] V. This invention constructs an isolated forest based on the interaction mode of the current teaching task and corresponding data. Using the average path length of the isolated forest as a metric, it calculates the isolated forest anomaly score for each user. Combining this with time-series data from similar teaching tasks, it selects core features with the goal of minimizing the mean squared error, ultimately calculating the evaluation score for each user's learning. By identifying anomalous learning behaviors of individual users through the isolated forest algorithm and combining it with time-series regression verification, it eliminates the interference of short-term technical fluctuations and platform service anomalies on the evaluation results, achieving fairness and objectivity in the evaluation of individual user learning. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a flowchart illustrating a dynamic evaluation method for service quality of online teaching platforms.

[0018] Figure 2 This is a flowchart illustrating step S1 of a dynamic evaluation method for service quality of online teaching platforms.

[0019] Figure 3 This is a flowchart illustrating step S2 of a dynamic evaluation method for service quality of online teaching platforms.

[0020] Figure 4 This is a flowchart illustrating step S3 of a dynamic evaluation method for service quality of online teaching platforms.

[0021] Figure 5 This is a flowchart illustrating step S4 of a dynamic evaluation method for service quality of online teaching platforms.

[0022] Figure 6 This is a flowchart illustrating step S5 of a dynamic evaluation method for service quality of online teaching platforms. Detailed Implementation

[0023] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Where specific techniques or conditions are not specified in the embodiments, they shall be performed in accordance with the techniques or conditions described in the literature in the art or in accordance with the product manual.

[0024] See Figure 1 A method for dynamic evaluation of service quality for online teaching platforms includes: S1, collecting learning behavior data of the online education platform based on the scenario of accessing teaching tasks.

[0025] S2, perform multimodal scene decomposition on the acquired learning behavior data to determine the scene feature matrix under each scene.

[0026] S3 integrates real-time learning behavior data based on the scene feature matrix of each scenario, determines the real-time interaction level of learning behavior, and constructs a learning behavior profile under real-time interaction according to the response time under real-time interaction.

[0027] S4. Based on the acquired learning behavior profile, calculate the group response time for each teaching task, and determine the interaction mode for the teaching task according to the task completion status.

[0028] S5, based on the interactive mode of teaching task presentation, uses the isolated forest algorithm to perform time series regression verification to determine the evaluation score of each user during learning.

[0029] like Figure 2 As shown, when collecting learning behavior data from the online education platform in step S1, the implementation method includes: S11, for each teaching task, using the time point of the teaching task distribution as the basis for collection, constructing a task distribution vector, which consists of the distribution time point, scene identifier, actual number of participating users, and task ID.

[0030] Understandably, the teaching tasks in this solution specifically refer to specific tasks initiated by the online teaching platform / teacher. These tasks can be specific to the question-and-answer interaction format on the live stream, assignment distribution, or test question answering assessment. The service quality is evaluated by adapting the teaching task scenario to the scenario configured for each teaching task.

[0031] S12, based on the user groups configured by the task distribution vector, determine the target user groups in each teaching task segmentation scenario; the target user groups will be used as analysis data in the real-time interaction scenario, and the course access retention of the target user groups will be used as the business result of learning behavior.

[0032] S13 is composed of learning behavior data based on user experience data and business result data of the target user group.

[0033] Specifically, when users engage in online teaching, data is typically presented on the user experience side, including video entry, video exit, interface refresh, question response, and answer upload, as well as business-side results data such as the ratio of registered users to actual attendance, interaction frequency, interaction duration, exam pass rate, and user monthly active user retention rate. Using user experience data and business-side results data as learning behavior data can reflect the data distribution of the target user group in the corresponding scenario, thereby determining the distribution ratio of different learning task behaviors under online teaching.

[0034] In one embodiment of the present invention, in step S2, modal decomposition is performed according to the scenario in which the learning behavior is located. After clarifying the operation and maintenance processing and specific resources under each teaching task scenario, the adaptability of the online teaching platform to each scenario is explained.

[0035] like Figure 3 As shown, the implementation method of determining the scene feature matrix in each scenario in step S2 includes: S21, performing hierarchical clustering based on the scenario where the learning behavior data is located to form multiple scene clusters; wherein, during hierarchical clustering, the teaching scenario is taken as its clustering level, and the normalized feature vector of the data such as the interaction frequency and interaction duration of the learning behavior is taken as the clustering index value; at the same time, the K-Means algorithm can be combined to determine the initial cluster center, and the DBSCAN algorithm can be combined to remove noisy data to form scene clusters under multiple modalities; and when the clustering reaches the maximum number of iterations, the clustering iteration is stopped, and the maximum number of iterations can be set to 100 times as the termination condition for scene clustering.

[0036] Specifically, each scenario cluster represents a specific online teaching scenario. By representing the different forms of user online interaction, the course type and course scenario corresponding to each scenario cluster are labeled.

[0037] S22, using the feature center vector of each scene cluster as a basis, separates the single scene features and candidate shared features of each scene cluster through the business semantic association of each scene; where the feature center vector represents the feature value of the cluster center of the scene cluster.

[0038] Specifically, after clustering, the feature center vector of each scenario cluster represents the combination form of that cluster. For each scenario cluster's feature center vector, it is matched against a business semantic association database, selecting the associated business semantics for that scenario from the database, such as the interaction frequency and duration associated with live classes. Each scenario is then broken down into multiple single-scenario features that conform to the business semantic association. These single-scenario features are used to verify shared data across different scenarios, thus achieving preliminary data construction for multi-scenario learning interactions.

[0039] The implementation methods for separating the single-scene features of each scene cluster include: using the feature center vector of each scene cluster as the query condition, incorporating data that conforms to business semantic association into the single-scene feature candidate set; each feature in the single-scene feature candidate set represents a set of data that conforms to business semantic association.

[0040] For each feature in a single scene feature candidate set, the ratio of the coverage of that feature in the current scene cluster to the average coverage of all other scene clusters is used as the separation criterion to select the single scene feature and candidate shared features of the current scene cluster.

[0041] The coverage rate of a scene cluster represents the proportion of the corresponding feature combination within the scene cluster. The average coverage rate of all other scene clusters is calculated based on the currently separated scene clusters. This indicates whether the currently distinguished feature can be used as a feature form to distinguish this scene from other scenes, and outputs it as a single scene feature.

[0042] Secondly, when configuring the separation conditions, a preset threshold is set based on the ratio of scene cluster coverage. The portion that exceeds the preset threshold is considered to meet the separation conditions and is selected as a single scene feature, while other features are considered as candidate shared features. At the same time, the preset threshold of scene cluster coverage is the average scene cluster coverage calculated from any feature in historical data, in order to distinguish relatively unique single scene features and realize the data processing for scene division.

[0043] S23 aggregates the single scene features and candidate shared features according to the corresponding scene levels, and uses the aggregated single scene features and candidate shared features as the output scene feature matrix. Here, the aggregation of scene levels represents the level when clustering scene clusters, and the single scene features and candidate shared features under each level are used as the data combination for the current scene output.

[0044] It is understandable that in different scenarios, such as live classes and recorded classes, the frequency and duration of user interactions will be represented by different forms of data. The selected single scenario feature is used to highlight the uniqueness of a certain teaching scenario, and the candidate shared features are used to illustrate the similarities in different scenarios, so as to serve as a feature cluster of scenario components.

[0045] In one embodiment of the present invention, in step S3, a scene feature matrix is ​​introduced as the basis for quantization, a profile of real-time learning behavior is constructed, and the constructed learning behavior profile is used as the basis for data interaction in the teaching scenario.

[0046] The implementation method for determining the real-time interaction level of learning behavior in step S3 includes: using the scene feature matrix to perform scene analysis on the real-time learning behavior data and configuring the initial weight factor under real-time interaction; wherein, the initial weight factor is established based on the matching relationship between the real-time learning behavior data and the scene feature matrix. Specifically, the cosine similarity is calculated between the real-time learning behavior data and a single scene feature of the scene feature matrix. After selecting the scene cluster with the highest similarity, the cosine similarity value is used as the base value of the initial weight factor. Then, the base value is corrected according to the hierarchy of the scene cluster to finally obtain the initial weight factor for the current time period.

[0047] Secondly, the hierarchy of the scene cluster will determine the correction value of the initial weight factor; the more specific the scene, the larger the correction value. For example, if the scene cluster is considered as a multi-level hierarchical distribution of nodes, the root node is the teaching task, the first-level child nodes are scenarios such as live teaching and online quizzes, and the second-level child nodes are scenarios such as live teaching-real-time interaction. Here, adjustments can be made in real time according to the hierarchy corresponding to the scene cluster. Specifically, the correction value of the initial weight factor is expressed as the product of the current hierarchy number of the scene cluster and the base value. The base value can be set to 0.1 or a smaller value to reduce or increase the influence of the current hierarchy on the initial weight factor. The initial weight factor is adjusted by the same proportion as the correction value to complete the setting of the initial weight factor for each scene.

[0048] Then, by introducing the number of user interactions and interaction duration within each time period, and using the data ratio of effective interactions as the basis for quantifying the degree of real-time interaction, the value of the initial weight factor is adjusted.

[0049] Generally, if we know the number of interactions, interaction duration, and percentage of effective interactions within the current time period (such as a 5-minute sliding window), we can further verify the user's willingness to interact and learn in the current scenario, or the interaction between bullet comments / questions and course content, based on the effective interactions. This reflects whether users will give positive feedback to online teaching. Therefore, the initial weighting factors can be adaptively adjusted to characterize the interaction situation of online teaching in each time period.

[0050] Furthermore, data representing effective interactions refers to data on completed interactive responses. For example, in a course Q&A session, there is response data within a specified time, and the questions and answers relate to the current course content. After identifying the corresponding data, the proportion of the number of interactions corresponding to this data to the total number of interactions in the corresponding course is considered as the percentage of effective interactions. This, in turn, represents the interaction status of different users in the course within each time period.

[0051] Specifically, when adjusting the value of the initial weight factor, the implementation methods include: using the proportion of effective interaction times within multiple time periods as the adjustment direction of the initial weight factor, and statistically analyzing the average interaction duration and interaction frequency change rate corresponding to each behavior; since the bullet comments or Q&A during classroom Q&A or live interaction are mostly concentrated in specific time periods, if user interaction response to the course is considered, the effective interaction data of multiple courses in the corresponding time periods can be used as the user's behavioral profile under a specific course.

[0052] The adjustment value of the initial weight factor is determined by the range of values ​​for the average interaction duration and the rate of change of the number of interactions. The average interaction duration represents the average duration under effective interaction, and its value is determined according to the average interaction duration of historical data for this scenario. Specifically, the scenario is divided into three intervals: (0, 0.8), [0.8, 1.2], and (1.2, +∞), based on 0.8 and 1.2 times the average interaction duration of historical data. The initial weight factor within each of these three intervals is adjusted sequentially from smallest to largest by decreasing by 0.1, remaining unchanged, and increasing by 0.15. The intervals and adjustment values ​​for the initial weight factor can be adjusted using other values ​​selected based on historical data.

[0053] Secondly, the rate of change of interaction frequency can be divided into three intervals: (-∞, -20%), [-20%, 50%], and (50%, +∞). The initial weight factor is adjusted with the same value as the average interaction time, and then the adjustment of the initial weight factor in different time periods is statistically analyzed.

[0054] Understandably, the higher the frequency of adjustment of the initial weight factor, the faster the interaction frequency and the greater the change under the corresponding teaching class or teaching task distribution. It can characterize the activity level and fluctuation of user interaction behavior in the current time period. Combined with the proportion of effective interaction times, it can quantify the user's willingness to interact and the degree of adaptation of the platform service.

[0055] The real-time interaction level for the current time period is set based on the adjustment frequency of the initial weight factor. Specifically, the real-time interaction level is set using a weighted average based on the adjustment frequency of the initial weight factor within the current time period. First, a normalized weighted average of the number of valid interactions per line, the initial weight factor, and the average interaction duration per line is calculated (this weighting can be calculated by directly multiplying the normalized values ​​of the three, or by multiplying the normalized number of interactions and the average interaction duration by the initial weight factor, then summing the results). Then, according to the adjustment frequency of the initial weight factor, the average of the normalized weighted averages at multiple frequencies is calculated. The ratio of the current weighted average to the historical weighted average for this scenario is considered the real-time interaction level for the current scenario.

[0056] like Figure 4 As shown, the implementation method of constructing a learning behavior profile under real-time interaction in step S3 includes: S31, based on the scenario of matching teaching tasks, extracting the response time corresponding to the learning behavior from the real-time learning behavior data; wherein, the response time includes, but is not limited to, the single question answering time, the question loading waiting time, the answer submission failure time, and the task submission timeout time, etc., which represent the response time required for the teaching task; the response time is used to describe the time interval between the user initiating a specific learning behavior request in the teaching scenario during interaction and the system completing the task processing and returning.

[0057] Specifically, when the online platform distributes questionnaires and quizzes, it sets a response time based on the time required for users to answer the questions. The response time is used to statistically analyze the degree of user response to the teaching courses, thereby reflecting the evaluation dimensions of service quality.

[0058] S32, Based on the response time corresponding to the learning behavior, perform parallel trend analysis on the learning behavior to determine the parallel trend data corresponding to the learning behavior.

[0059] Parallel trend analysis involves statistical processing of concurrent teaching task scenarios, solving for parallel trends based on the parallel states of candidate shared features in the scenario feature matrix across different scenarios, interpreting the allocation of concurrent teaching tasks in different scenarios, identifying the allocation of resources in the current teaching platform system, and ultimately constructing a learning behavior profile under real-time interaction with the aim of improving user interaction experience.

[0060] Specifically, the response time and the number of parallel teaching tasks are selected for all scenarios in terms of candidate shared features, and the competition between the two for system resources is used as the parallel trend for the current analysis. At the same time, the least squares method can be used to calculate the response time based on the response time and the number of teaching tasks, and the slope values ​​corresponding to the two can be used to characterize the correlation of response time under different numbers of teaching tasks.

[0061] S33 uses a combination of parallel trend data and real-time interaction levels to configure learning behavior profiles for different scenarios.

[0062] The final output learning behavior profile will be based on the actual interaction situation and the current system's concurrent response time, combining the relevant parallel trends and real-time interaction levels to form a learning behavior profile for the corresponding scenario.

[0063] In one embodiment of the present invention, in step S4, learning behavior profiles of different individualized users under the same teaching task are introduced to extract the response time of the data group; therefore, the group response time is bound to the learning behavior profiles of the same teaching task, the same scenario cluster and the same target user group, and the response time related to the teaching task is gradually extracted from the learning behavior profiles of individualized users, including the corresponding maximum, minimum, average and standard deviation and other quantitative values, as the group response time here.

[0064] like Figure 5 As shown, the implementation method of determining the interactive mode of the teaching task in step S4 includes: S41, according to the data structure of the current teaching task, extracting the response time corresponding to the current teaching task from the learning behavior profile, and determining the range of the response time as the group response time.

[0065] S42 cross-correlates the group response time with the completion status of the current teaching task to clarify the causal relationship between technical service quality and teaching business results, and determines the associated results of the current teaching task. Here, task completion status represents the retention of the teaching task before and after execution, such as task completion rate, service quality complaint rate, negative review rate, user retention rate, and mid-course dropout rate, emphasizing the teaching business results; group response time emphasizes the system resources' responsiveness to the current teaching task at the technical level.

[0066] S43. Based on the association results of the current teaching task, retrieve the interaction patterns corresponding to each teaching task. Specifically, the list of interaction patterns is pre-constructed based on teaching scenarios, task types, association result features, etc.; then, when using teaching tasks for retrieval, the association results of the current teaching task are matched with the features in the interaction pattern list using cosine similarity, and the interaction pattern with the highest matching degree is the one corresponding to that teaching task.

[0067] Furthermore, when determining the correlation results of the current teaching task, the implementation method includes: based on the task completion status of the current teaching task, if the current teaching task is executed normally, the quantile values ​​and average values ​​corresponding to the group response time are used as cross-correlation features, and the data corresponding to the quantile values ​​are regarded as the output correlation results. Here, the quantile value represents the data at the 25th, 50th, and 75th percentile positions after the task completion status is sorted in ascending order. This can characterize the system response under the same teaching task, and the 75th percentile value can represent the value range in most cases. Therefore, the current quantile value and average value can represent the normal response and processing of most users under the same teaching task. The quantile value and average value here are used as the correlation results to complete the scenario verification and display.

[0068] If there are anomalies in the current teaching task, the consistency of the group response is determined based on the coefficient of variation corresponding to the group response time. The data description corresponding to the consistency and the abnormal data corresponding to the task completion status are regarded as the correlation results of the current output. Among them, the coefficient of variation is the ratio of the standard deviation of the response time to the mean, which is an indicator to measure the relative dispersion of the data.

[0069] If we consider abnormal branches in the teaching task, we need to use numerical data representing task completion status, such as task completion rate, service quality complaint rate, negative review rate, user retention rate, and mid-course dropout rate, as prerequisites for the current abnormal branch. Specifically, we introduce the numerical part of the task completion status corresponding to the teaching task, and use the numerical value corresponding to the task completion status as input data to perform anomaly retrieval on each value. When any value is abnormal, we extract the abnormal value entry from the task completion status, and use the consistency description corresponding to the abnormal value entry and the dispersion coefficient as the output correlation result. Finally, based on the relevant data in the correlation result, we select the interaction mode relevant to the current scenario.

[0070] Regarding abnormal conditions for task completion, scenario thresholds can be set based on the average values ​​of corresponding indicators in the historical data for that scenario. Any of the following conditions will be considered an anomaly in the current teaching task: 1. Task completion rate < 95% of the scenario threshold; 2. Valid submission rate < 95% of the scenario threshold; 3. Percentage of mid-course exits > twice the scenario threshold; 4. Service quality complaint rate > three times the scenario threshold; 5. User retention rate < 90% of the scenario threshold. This section on judging abnormal task completion is for illustrative purposes only; the specific percentages can be adjusted based on historical data requirements in the corresponding scenario.

[0071] The consistency of the described group response is calibrated by the range of values ​​of the coefficient of variation, which is used to characterize the relative discreteness of the response time values ​​of multiple users under the current teaching task when anomalies occur. For example, using 0.15, 0.25, and 0.35 as the dividing interval standard, data description intervals of high consistency, basic consistency, slight inconsistency, and severe inconsistency are set from small to large. In this scenario, the larger the value of the coefficient of variation, the greater the difference in user experience, and the more it affects the service quality of the teaching itself.

[0072] In one embodiment of the present invention, in step S5, the data will be re-matched based on the data from the aforementioned steps, and the data in other scenarios will be evaluated using a time-series regression verification method to simultaneously generate the user's evaluation score.

[0073] like Figure 6 As shown, the implementation method for determining the evaluation score of each user during learning in step S5 includes: S51, based on the interaction mode of the current teaching task, constructing an isolation tree using the data corresponding to each user in the interaction mode; wherein, the construction method of the isolation tree is as follows: in the current interaction mode, randomly select a single feature with completely equal probability from the full feature set of all user features, randomly select a split point within the value range of the feature, make the left subtree less than the value of the split point, and the right subtree greater than or equal to the value of the split point, and gradually recursively until the subset size is 1 or the maximum depth is reached (this depth is usually chosen as the logarithm of the current split feature with base 2 as the value), and then record the path length of the isolation tree (the number of splits when the data is isolated), thus completing the construction of the isolation tree.

[0074] S52 uses the average path length of isolated trees as the evaluation metric to configure an isolated forest anomaly score for each user. When the path length of a single isolated tree is obtained, the score of a single user can be quantified based on the user corresponding to the isolated data point using the average path length. This score is used to identify individual users whose behavior differs significantly from that of the normal user group. The higher the score, the easier it is for them to be isolated.

[0075] The anomaly score in Isolation Forest is set according to the Isolation Forest algorithm. Its calculation formula is generally: a base-2 exponent, where the exponent is the negative of the ratio of the average path length to the baseline average path length, used to quantify the relative anomaly of each user under random forest segmentation. The baseline average path length is the average path length calculated from a large sample of historical data, serving as the basis for the current data segmentation.

[0076] S53, based on the isolated forest anomaly scores of each user, combined with time series data of similar teaching tasks, filters the features of each user in a way that minimizes the mean squared error, and calculates the evaluation score of each user based on the filtered features.

[0077] Since the isolated forest anomaly score is determined based on the input features, when the feature set corresponding to a single user includes, but is not limited to, various features such as real-time interaction level, response time, interaction duration, and number of interactions, an isolated anomaly score can be obtained based on each input feature. Here, the input features can be filtered through prediction using the LSTM time series regression model to select the feature set for a single user. Then, based on the filtered feature set, the isolated forest anomaly scores are weighted and summed according to the relative weight of each feature to obtain the evaluation score for different users.

[0078] Specifically, the LSTM time-series regression model executes steps at each time step and generates predicted values ​​for each input feature using linear regression based on the previous state and the current input state. The mean squared error (MSE) is then calculated based on the predicted values. If the calculated MSE is less than 1.2 times the minimum MSE in historical data, the corresponding feature is considered a filtered feature. The weight of this feature is then set based on the ratio of its value to the total values ​​for all users. Combined with the isolated forest anomaly score of this feature, the features of the current user are weighted and summed sequentially to obtain the final evaluation score for each user.

[0079] After time-series regression, the evaluation scores tend to represent the isolation of each user relative to the user group under time-series prediction. Under the premise of fairness and consistency in the overall processing, the scores of each user's learning interaction relative to the group are given.

[0080] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A method for dynamic evaluation of service quality for online teaching platforms, characterized in that, include: Based on the scenarios in which teaching tasks are integrated, learning behavior data is collected from online education platforms; The acquired learning behavior data is decomposed into multimodal scenarios to determine the scene feature matrix for each scenario; Based on the scene feature matrix of each scenario, the real-time learning behavior data is integrated to determine the real-time interaction level of the learning behavior, and a learning behavior profile under real-time interaction is constructed according to the response time under real-time interaction. Based on the acquired learning behavior profile, the group response time for each teaching task is calculated, and the interaction mode for the teaching task is determined according to the task completion status. Based on the interactive mode of teaching task presentation, the isolated forest algorithm is used to perform time series regression verification to determine the evaluation score of each user during learning. The methods for determining the assessment scores for each user during learning include: Based on the interaction mode of the current teaching task, an isolated tree is constructed using the data corresponding to each user in the interaction mode; The average path length of isolated trees is used as the evaluation metric to configure an isolated forest anomaly score for each user. Based on the isolated forest anomaly scores of each user, and combined with time series data of similar teaching tasks, features of each user are screened in a way that minimizes the mean squared error, and the evaluation score of each user is calculated using the screened features.

2. The method for dynamic evaluation of service quality for online teaching platforms according to claim 1, characterized in that, When collecting learning behavior data from online education platforms, the methods include: For each teaching task, a task distribution vector is constructed based on the time point of task distribution. Based on the user groups configured by the task distribution vector, the target user groups in each teaching task partitioning scenario are determined. Learning behavior data is composed of user experience data and business outcome data of the target user group.

3. The method for dynamic evaluation of service quality for online teaching platforms according to claim 1, characterized in that, The methods for determining the scene feature matrix for each scenario include: Based on the context in which the learning behavior data is located, hierarchical clustering is performed to form multiple context clusters; Based on the feature center vectors of each scenario cluster, the single scenario features and candidate shared features of each scenario cluster are separated through the business semantic association of each scenario. Single scene features and candidate shared features are aggregated according to the corresponding scene level, and the aggregated single scene features and candidate shared features are used as the output scene feature matrix.

4. The method for dynamic evaluation of service quality for online teaching platforms according to claim 3, characterized in that, Methods for separating individual scene features from each scene cluster include: Using the feature center vector of each scene cluster as the query condition, data that conforms to business semantic association is included in the single scene feature candidate set; For each feature in a single scene feature candidate set, the ratio of the coverage of that feature in the current scene cluster to the average coverage of all other scene clusters is used as the separation criterion to select the single scene feature and candidate shared features of the current scene cluster.

5. The method for dynamic evaluation of service quality for online teaching platforms according to claim 1, characterized in that, Methods for determining the real-time level of interaction in learning behavior include: Utilize the scene feature matrix to perform scene analysis on real-time learning behavior data and configure initial weight factors under real-time interaction; By introducing the number of user interactions and interaction duration within each time period, and using the proportion of effective interactions as the basis for quantifying the degree of real-time interaction, the value of the initial weighting factor is adjusted. The real-time interaction level for the current time period is set based on the adjustment frequency of the initial weight factor.

6. The method for dynamic evaluation of service quality for online teaching platforms according to claim 5, characterized in that, When adjusting the initial weight factor, the following methods can be used: The proportion of effective interactions within multiple time periods is used as the initial weighting factor adjustment direction, and the average interaction duration and interaction frequency change rate corresponding to each behavior are statistically analyzed. The adjustment value of the initial weighting factor is determined by the range of values ​​for the average interaction duration and the rate of change of the number of interactions.

7. The method for dynamic evaluation of service quality for online teaching platforms according to claim 1, characterized in that, The methods for constructing learning behavior profiles in real-time interaction include: Based on the scenario of matching teaching tasks, the response time corresponding to the learning behavior is extracted from real-time learning behavior data; Based on the response time corresponding to the learning behavior, perform parallel trend analysis on the learning behavior to determine the parallel trend data corresponding to the learning behavior; By combining parallel trend data and real-time interaction levels, learning behavior profiles can be configured for different scenarios.

8. The method for dynamic evaluation of service quality for online teaching platforms according to claim 1, characterized in that, The implementation methods for determining the interactive mode of presenting teaching tasks include: Based on the data structure of the current teaching task, the response time corresponding to the current teaching task is extracted from the learning behavior profile, and the range of values ​​for the response time is determined as the group response time. Cross-correlation is performed between the group response time and the completion status of the current teaching task to determine the correlation result of the current teaching task; Based on the association results of the current teaching task, retrieve the interaction mode corresponding to each teaching task.

9. The method for dynamic evaluation of service quality for online teaching platforms according to claim 8, characterized in that, When determining the associated results of the current teaching task, the implementation methods include: Based on the completion status of the current teaching task, if the current teaching task is executed normally, the quantile value and average value corresponding to the group response time are used as cross-correlation features, and the data corresponding to the quantile value are regarded as the output correlation results. If there are anomalies in the current teaching task, the consistency of the group response is determined based on the dispersion coefficient corresponding to the group response time, and the data description corresponding to the consistency and the abnormal data corresponding to the task completion status are regarded as the correlation results of the current output.

Images