Evaluation data processing method and system, medium and program product

Abstract

The invention relates to the technical field of data processing, in particular to an evaluation data processing method and system, a medium and a program product, and the method comprises the steps: obtaining a to-be-evaluated test paper and all test result information corresponding to the to-be-evaluated test paper; identifying test question information of the to-be-evaluated test paper, and determining a to-be-evaluated test paper model corresponding to the to-be-evaluated test paper based on the test question information and the whole test result information; obtaining multi-modal teaching information corresponding to the test paper to be evaluated, and determining a student knowledge model based on the multi-modal teaching information; and matching the to-be-evaluated test paper model with the student knowledge mastering model to obtain a model matching result, evaluating the to-be-evaluated test paper based on the model matching result, and generating a test paper optimization report based on an evaluation result. The test paper quality can be improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an evaluation data processing method, processing system, medium, and program product. Background Technology

[0002] With the breakthrough development of artificial intelligence technology, intelligent education has become a core direction of global educational reform. AI-powered smart teaching technology can automate test paper generation, intelligent grading, and data analysis, thereby improving teaching efficiency. However, in the process of generating test papers using AI-powered smart teaching technology, papers often deviate significantly from the actual teaching content due to a disconnect from real-world teaching scenarios or students' cognitive patterns. Using these biased test papers for knowledge reinforcement may negatively impact students' understanding and consolidation of the teaching content.

[0003] Traditional test paper generation methods mainly rely on the subjective experience of the relevant test setters to arrange the order of questions, the distribution of question types, and the difficulty gradient. The quality of the test paper is closely related to the personal subjective experience of the relevant test setters. Differences in experience between different test setters may lead to fluctuations in the difficulty and discrimination of the test paper, resulting in unstable test paper quality. Therefore, there is still a relatively high probability that test papers with teaching deviations will appear. Summary of the Invention

[0004] To improve the quality of exam papers, this application provides a method, system, medium, and program product for processing assessment data.

[0005] Firstly, this application provides a method for processing evaluation data, employing the following technical solution: A method for processing evaluation data, comprising: Obtain the test paper to be evaluated and the information on all examination results corresponding to the test paper to be evaluated; Identify the question information of the test paper to be evaluated, and determine the test paper model corresponding to the test paper to be evaluated based on the question information and the information of all examination results; Obtain the multimodal teaching information corresponding to the test paper to be evaluated, and determine the student's knowledge model based on the multimodal teaching information; The test paper model to be evaluated is matched with the student knowledge mastery model to obtain the model matching result. The test paper to be evaluated is evaluated based on the model matching result, and a test paper optimization report is generated based on the evaluation result.

[0006] By adopting the above technical solutions, analyzing the overall exam results corresponding to the test papers to be evaluated facilitates the extraction of information reflecting students' levels. Analyzing the test question information allows for a comprehensive understanding of the test paper's question type distribution, knowledge point coverage, and difficulty gradient—that is, understanding the objective attributes of the test paper. Analyzing multimodal teaching information facilitates understanding students' classroom performance when learning the corresponding teaching content of the test paper, and also reflects the different students' mastery of knowledge points. The integration of multimodal data makes the evaluation more comprehensive and accurate. Finally, by combining test question information, result information, and classroom performance, the fit between the objective attributes of the test paper and students' actual performance is comprehensively reflected. Based on this fit, the test paper is evaluated, and an optimization report is generated based on the evaluation results. This improves the matching degree between the test paper's knowledge point coverage and the teaching syllabus, and ensures that the test paper quality continuously improves as the teaching scenario changes and students' abilities improve, thus facilitating quality improvement in subsequent test paper generation processes.

[0007] In one possible implementation, when the multimodal teaching information includes teaching content data, student classroom performance data, and teaching behavior data, determining the student knowledge mastery model based on the multimodal teaching information includes: Identify the teaching knowledge points contained in the teaching content data and the teaching relationships between each teaching knowledge point, and construct a basic teaching model based on the teaching knowledge points and the teaching relationships between each teaching knowledge point; Identify classroom interaction records and in-class assignments included in the student classroom performance data, and determine a knowledge point mastery matrix based on the classroom interaction records and in-class assignments. The knowledge point mastery matrix includes the student's mastery level for each knowledge point. The teaching duration and teaching type of each teaching knowledge point are determined from the teaching behavior data. Based on the teaching duration, teaching type, and knowledge point mastery matrix of each teaching knowledge point, the basic teaching model is optimized to obtain the student knowledge mastery model.

[0008] By adopting the above technical solutions, a basic teaching model is constructed by identifying the core knowledge points and logical connections between them in the teaching content data. This facilitates the clarification of teaching focus and knowledge structure, providing a structured framework for subsequent model optimization. By converting classroom interaction records and in-class assignments into calculable indicators, a knowledge point mastery matrix is ​​generated, making it easier to track the changing trends of students' knowledge mastery. By extracting teaching duration and teaching type from teaching behavior data, the impact of different teaching behaviors on students' mastery can be quantified, providing a reference for subsequent optimization of the basic teaching model. By combining the knowledge point mastery matrix and teaching behavior characteristics, a real-time model iteration and feedback mechanism can be used to continuously improve teaching quality and enhance the accuracy and effectiveness of determining the student knowledge mastery model.

[0009] In one possible implementation, the method further includes: Obtain real-time online answering behavior information for each question in the test paper to be evaluated; Based on the real-time online answering behavior information and the question information of the test paper to be evaluated, a target dynamic feedback list corresponding to each question to be evaluated in the test paper to be evaluated is determined; Each dynamic feedback list is overlaid onto the corresponding question to be evaluated to obtain a real-time feedback test paper.

[0010] By adopting the above technical solution and integrating real-time online answering behavior information with test question attribute information, it is easy to construct a dynamic feedback mechanism. That is, by identifying real-time online answering behavior information, abnormal answering behavior can be detected in a timely manner, and a target dynamic feedback list based on abnormal answering behavior can be superimposed on the position of the corresponding test question to form a real-time feedback test paper. This allows relevant teachers to keep abreast of students' answering status and quickly locate students' knowledge gaps or problem-solving errors based on the information in the feedback list, thereby facilitating the provision of targeted guidance and assistance for the subsequent test paper compilation process.

[0011] In one possible implementation, based on the real-time online response behavior information and the question information of the test paper to be evaluated, a target dynamic feedback list corresponding to the questions to be evaluated is determined, including: A basic dynamic feedback list is determined based on the test question information of the test paper to be evaluated; The real-time online answering behavior information includes real-time behavior indicators and the behavior indicator value corresponding to each real-time behavior indicator. The real-time behavior indicators include real-time click rate, real-time dwell time, and real-time accuracy rate. Based on the real-time behavior indicators and the behavior indicator values ​​corresponding to each real-time behavior indicator, a real-time indicator distribution map is determined. The distribution area of ​​the indicators is determined based on the real-time indicator distribution map, and a preset indicator area threshold is determined based on the current response time period; When the distribution area of ​​the indicator is lower than the preset indicator area threshold, an abnormal indicator is determined from the real-time behavior indicators, and the basic dynamic feedback list is optimized based on the abnormal indicator to obtain the target dynamic feedback list corresponding to the test question to be evaluated.

[0012] By adopting the above technical solution, and by capturing behavioral indicators such as real-time click rate, real-time dwell time, and real-time accuracy rate of students during the exam, it is easy to quantify students' cognitive load and knowledge mastery status during the answering process. Furthermore, by analyzing the real-time indicator distribution map composed of various real-time behavioral indicators, it is possible to determine whether there are any abnormal indicators, rather than feeding back all real-time behavioral indicators. Using the indicator distribution area as an examination condition helps to improve the accuracy and effectiveness of identifying abnormal indicators.

[0013] In one possible implementation, after determining the abnormal indicators corresponding to the test item to be evaluated, the method further includes: The test items to be evaluated that contain the aforementioned abnormal indicators are marked as items of concern; Based on a pre-defined knowledge graph, determine the associated questions corresponding to the questions of interest and the association level between the associated questions and the questions of interest; Based on the association level, determine the association indicator area threshold corresponding to the associated test question, and based on the association indicator area threshold, determine the association anomaly indicator corresponding to the associated test question; Based on the focus questions, focus multimodal teaching features are determined from the multimodal teaching information, and multimodal feature fusion is performed based on the focus multimodal teaching features to obtain focus multimodal fusion features. The focus multimodal fusion features include student focus score, teaching quality score, and knowledge point adaptation score. Based on the aforementioned anomaly indicators, the associated anomaly indicators, and the attention-based multimodal fusion features, the anomaly source type is determined, which includes student-related reasons, teaching-related reasons, and test-related reasons.

[0014] By adopting the above technical solutions, marking the questions containing anomalous indicators facilitates the rapid identification of the core source of anomalies. Analyzing the associated anomalous indicators corresponding to the related questions facilitates the assessment of whether the anomalous indicators of the questions are accidental. Analyzing the multimodal teaching characteristics of the questions facilitates the consideration and analysis of students' classroom status, teachers' teaching quality, and the fit between the questions and knowledge points, thereby facilitating the assessment of the impact of teaching and questions on anomalous indicators. By comprehensively analyzing the impact of associated anomalous indicators and the multimodal fusion characteristics on anomalous indicators, the efficiency and accuracy of the anomaly tracing process can be improved.

[0015] In one possible implementation, when the anomaly attribution type is student-related, the method further includes: Identify the students to be monitored corresponding to the abnormal indicators, and determine the weak points of the students to be monitored based on the information of all exam results. The learning habits of the students being monitored are determined based on the multimodal teaching information. Based on the identified weaknesses and learning habits, remedial data is determined from the teaching content data included in the multimodal teaching information, and the remedial data is then fed back to the students being monitored.

[0016] By adopting the above technical solution, multimodal teaching information can be used to construct personalized learning profiles for students and identify their learning habits. Based on these habits, tutoring data on weak areas can be pushed to improve the efficiency of students' absorption of tutoring data.

[0017] Secondly, this application provides a processing system, which adopts the following technical solution: A processing system comprising: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform the above-described assessment data processing method.

[0018] Thirdly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program stored thereon that can be loaded by a processor and execute the above-described evaluation data processing method.

[0019] Fourthly, this application provides a computer program product, which adopts the following technical solution: A computer program product includes a computer program that, when executed by a processor, implements the above-mentioned evaluation data processing method.

[0020] In summary, this application includes at least one of the following beneficial technical effects: Analyzing the overall exam results for the test papers under evaluation provides a clear understanding of student levels. Analyzing the test questions allows for a comprehensive understanding of the question type distribution, knowledge point coverage, and difficulty level of the test papers—that is, understanding the objective attributes of the test papers. Analyzing multimodal teaching information helps understand students' classroom performance when learning the corresponding teaching content of the test papers, and also reflects the different levels of knowledge mastery among students. The integration of multimodal data makes the evaluation more comprehensive and accurate. Finally, combining test question information, overall exam results, and classroom performance allows for a comprehensive reflection of the fit between the objective attributes of the test papers and students' actual performance. Based on this fit, the test papers are evaluated, and an optimization report is generated based on the evaluation results. This improves the matching degree between the test paper's knowledge point coverage and the teaching syllabus, and ensures that the test paper quality continuously improves as the teaching scenario changes and students' abilities improve, thus facilitating quality improvement in subsequent test paper generation processes.

[0021] By capturing students' real-time click-through rate, real-time dwell time, and real-time accuracy during the exam, it is easy to quantify students' cognitive load and knowledge mastery during the test. Instead of providing feedback on all real-time behavioral indicators, the analysis of the real-time indicator distribution map composed of various real-time behavioral indicators helps to determine whether there are any abnormal indicators. By using the indicator distribution area as an examination condition, it is easier to improve the accuracy and effectiveness of identifying abnormal indicators. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an evaluation data processing method in an embodiment of this application; Figure 2 This is a schematic diagram illustrating the process of determining a target dynamic feedback list in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a processing system according to an embodiment of this application. Detailed Implementation

[0023] The following is in conjunction with the appendix Figures 1 to 3 This application will be described in further detail.

[0024] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the object's authorization and consent.

[0027] Specifically, this application provides an evaluation data processing method executed by a processing system. This processing system can be a server or a terminal device. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this connection.

[0028] refer to Figure 1 , Figure 1 This is a flowchart illustrating an evaluation data processing method according to an embodiment of this application. The method includes steps S110-S140, wherein: Step S110: Obtain the test paper to be evaluated and the information on all examination results corresponding to the test paper to be evaluated.

[0029] Specifically, the test papers to be evaluated can be automatically generated by the AI-powered smart teaching system, generated by relevant test-setting teachers based on their personal experience, or generated by relevant test-setting teachers in conjunction with the AI-powered smart teaching system. These are the test papers that require quality assessment. Although the method of generating test papers by combining the AI-powered smart education system with relevant teachers integrates technological efficiency and teaching experience, limitations in technological maturity, human-machine collaboration mechanisms, and the dynamic nature of teaching necessitate assessment to ensure test paper quality. Assessment is not only the last line of defense in quality control but also a crucial step in promoting the continuous optimization of the AI-powered smart teaching system.

[0030] The overall exam results corresponding to the test papers to be evaluated include, but are not limited to, individual student answer data and overall test paper statistics. Individual student answer data includes, but is not limited to, answer details (i.e., the student's answers to each question on the test paper to be evaluated, including multiple-choice options, fill-in-the-blank answers, and short-answer essays); scores (i.e., the student's score on each question and the total score of the test paper); and time taken (i.e., the time spent by the student to complete each question or the entire test paper to be evaluated). Overall test paper statistics include, but are not limited to, question difficulty (i.e., the difficulty coefficient of each question); question discrimination (i.e., the ability of each question to differentiate between high-achieving and low-achieving students, usually calculated by the difference in scores between high-achieving and low-achieving students on that question); and question reliability (i.e., the stability and reliability of the exam results).

[0031] When using an online examination model, students' individual answer data can be directly collected from their devices and uploaded to the processing system. The processing system can then integrate the individual answer data according to assessment needs to obtain the overall examination results. When using an offline examination model, relevant teachers can collect each student's answer sheet, upload the collection results and the examination video to the processing system, and obtain the overall examination results based on the collection results and the examination video. The method for determining the overall examination results is not specifically limited in this embodiment.

[0032] Step S120: Identify the question information of the test paper to be evaluated, and determine the test paper model corresponding to the test paper to be evaluated based on the question information and the overall examination results information.

[0033] Specifically, the test information includes, but is not limited to, question type, knowledge point, cognitive level, and test attribute. Question type can be multiple choice, fill-in-the-blank, short answer, or comprehensive response questions. Knowledge point refers to the knowledge point label for each question in the test paper to be evaluated; for example, the knowledge point label for question a could be "Mathematics - Algebra - Quadratic Function". Cognitive level includes, but is not limited to, memorization, understanding, application, analysis, evaluation, and creation. Test attribute can be a difficulty value label representing the difficulty level of the question. The test information of the test paper to be evaluated can be identified based on a preset feature recognition algorithm and a preset knowledge graph. The specific preset feature recognition algorithm and preset knowledge graph are not specifically limited in this embodiment, as long as they can identify the corresponding test information from the test paper to be evaluated.

[0034] The test paper model to be evaluated is used to reflect the rationality of the layout of the test paper in terms of knowledge point coverage, question type distribution, and difficulty level classification. When determining the test paper model corresponding to the test paper to be evaluated based on the test question information and the overall exam results information, a pre-set weighted scoring method or a pre-set machine learning model can be used to integrate and analyze the test question information and the overall exam results information. The pre-set weighted scoring method can set weights for dimensions such as knowledge point coverage, question type distribution, and difficulty level classification. The pre-set machine learning model can be random forest or support vector machine, etc. The specific pre-set weighted scoring method and machine learning model are not specifically limited in this application embodiment, as long as the obtained test paper to be evaluated can reflect the rationality of the layout of the test paper to be evaluated in terms of knowledge point coverage, question type distribution, and difficulty level classification.

[0035] Step S130: Obtain the multimodal teaching information corresponding to the test paper to be evaluated, and determine the student's knowledge model based on the multimodal teaching information.

[0036] Specifically, the multimodal teaching information corresponding to the test paper to be evaluated is the classroom performance of all students when learning the knowledge points corresponding to the test paper. For example, if the knowledge point corresponding to the test paper to be evaluated is "application of quadratic functions", the corresponding multimodal teaching information is the classroom performance of all students when the relevant teacher teaches "application of quadratic functions". This information can be collected by AI smart teaching equipment and uploaded to the processing system. Based on the multimodal teaching information, the student knowledge model is determined. That is, the corresponding student profile is constructed according to the multimodal teaching information to reflect the degree of mastery of "application of quadratic functions" by all students during the class.

[0037] Furthermore, when multimodal teaching information includes teaching content data, student classroom performance data, and teaching behavior data, the student knowledge mastery model determined based on multimodal teaching information can specifically include: The teaching content data is used to identify the teaching knowledge points and the teaching relationships between them. Based on these relationships, a basic teaching model is constructed. The student classroom performance data is also identified, including classroom interaction records and in-class assignments. Based on these records, a knowledge point mastery matrix is ​​determined, showing the students' level of mastery of each knowledge point. Finally, the teaching duration and type of each knowledge point are determined from the teaching behavior data. Based on these factors, the basic teaching model is optimized to obtain a student knowledge mastery model.

[0038] Specifically, the teaching content data includes, but is not limited to, course outlines, textbook texts, teaching PPTs, video lectures, etc. This data can be uploaded to the processing device by relevant teachers in advance. The processing device can also directly retrieve data from the AI ​​smart education device after establishing a connection with it. The specific retrieval method is not specifically limited in this embodiment. A preset text analysis model can be used to extract teaching knowledge points from the teaching content data. This preset text analysis model can be the BERT-wwm model, and the extracted teaching knowledge points can be, for example, "function continuity" or "photosynthesis process." Based on a preset knowledge graph, the teaching relationships between each teaching knowledge point can be determined. The identified teaching knowledge points are imported into the preset teaching model, and the teaching knowledge points are connected based on their relationships to obtain a basic teaching model.

[0039] Student classroom performance data includes classroom interaction records and in-class assignments. Classroom interaction records include, but are not limited to, the number of questions asked, the accuracy rate of answers, and attention span. The number of questions asked counts the number of times students ask questions on each knowledge point; the accuracy rate of answers calculates the percentage of students who answer correctly on each knowledge point; and attention span assesses students' level of focus on knowledge points based on the duration or frequency of their attention in class. In-class assignments include, but are not limited to, assignment grades, completeness of solution steps, and error types. Assignment grades record students' scores on assignments related to each knowledge point; the completeness of solution steps assesses the detail and logic of students' solution steps; and error type analysis identifies common error types students make on each knowledge point, such as errors in conceptual understanding and calculation. The features corresponding to classroom interaction records and in-class assignments are normalized and quantified. Then, a preset learning model is selected, and the optimal number of trees and the importance of each feature are chosen through cross-validation. Based on the model output, the students' mastery of each knowledge point is evaluated. Finally, the mastery of each student for each knowledge point is imported into a preset matrix to obtain a knowledge point mastery matrix. In the preset matrix, rows represent students, columns represent knowledge points, and elements in the matrix represent the students' mastery of the corresponding knowledge points, which can be represented by numerical values ​​or levels. For example, using numerical values ​​to represent mastery, 1-5 represent "not mastered" to "proficient mastery," respectively.

[0040] The teaching duration of each knowledge point in the teaching behavior data can be used to quantify the time spent on each knowledge point, reflecting the allocation of teaching focus. Teaching types include, but are not limited to, lecturing, discussion, and experiments, reflecting the diversity of teaching methods. When optimizing the basic teaching model based on the teaching duration, teaching type, and knowledge point mastery matrix of each knowledge point, a weighted fusion optimization method can be adopted, or a machine learning model optimization method can be used. For example, when using the weighted fusion optimization method, weights can be directly assigned to the teacher's teaching duration and teaching type for each knowledge point, as well as the student's mastery of each knowledge point. These features are then weighted and fused into the basic teaching model to adjust the connection strength or direction between knowledge points, resulting in a student knowledge mastery model. Specific optimization methods are not limited in this embodiment, as long as the actual teaching situation of relevant teachers and the actual classroom performance of students in the corresponding knowledge point learning stage can be integrated into the basic teaching model. By combining the knowledge point mastery matrix and teaching behavior features, it is convenient to continuously improve teaching quality and enhance the accuracy and effectiveness of determining the student knowledge mastery model through real-time model iteration and feedback mechanisms.

[0041] Step S140: Match the test paper model to be evaluated with the student knowledge mastery model to obtain the model matching result, evaluate the test paper to be evaluated based on the model matching result, and generate a test paper optimization report based on the evaluation result.

[0042] Specifically, the model of the test paper to be evaluated can be directly matched with the student's knowledge mastery model. The model matching result can be used to evaluate the fit between the objective attributes of the test paper to be evaluated and actual teaching and student classroom performance. When the model matching result is lower than the preset matching value, the generated test paper optimization report can be used to reflect the defects of the test paper to be evaluated, such as mismatch of knowledge point coverage, unsuitable question type difficulty, inaccurate difficulty level, and insufficient teaching relevance. Timely feedback of the test paper optimization report makes it easier for relevant test paper teachers to adjust or optimize the subsequent test paper assembly process in a timely manner.

[0043] In this embodiment of the application, analyzing the overall exam results information corresponding to the test paper to be evaluated facilitates a direct understanding of students' levels. Analyzing the test question information facilitates a comprehensive understanding of the test paper's question type distribution, knowledge point coverage, and difficulty gradient, thus revealing the objective attributes of the test paper. Analyzing multimodal teaching information facilitates understanding students' classroom performance when learning the corresponding teaching content of the test paper, and also reflects the different students' mastery of knowledge points. The integration of multimodal data makes the evaluation more comprehensive and accurate. Finally, by combining test question information, overall exam results information, and classroom performance, the applicability between the objective attributes of the test paper and students' actual performance can be comprehensively reflected. Based on this applicability, the test paper is evaluated, and an optimization report is generated based on the evaluation results. This improves the matching degree between the test paper's knowledge point coverage and the teaching syllabus, and ensures that the test paper quality continuously improves as the teaching scenario changes and students' abilities improve, thereby facilitating the improvement of test paper quality in subsequent test paper generation processes.

[0044] Furthermore, to facilitate targeted guidance and assistance for the subsequent test paper assembly process, the method provided in this application embodiment also includes: Obtain real-time online answering behavior information for each question in the test paper to be evaluated; based on the real-time online answering behavior information and the question information of the test paper to be evaluated, determine the target dynamic feedback list corresponding to each question in the test paper to be evaluated; overlay each dynamic feedback list onto the position of the corresponding question to be evaluated to obtain the real-time feedback test paper.

[0045] Specifically, when using an online examination model, JavaScript code can be embedded in the front-end page of the online examination system to capture students' online answering behavior information in real time. This real-time online answering behavior information includes, but is not limited to, the start time, end time, number of pauses, and number of answer modifications. Analyzing this information allows for a better understanding of each student's real-time answering status. When the real-time answering status is good, i.e., there is no abnormal answering behavior, a target dynamic feedback list can be generated for each question to be evaluated based on the question information of the test paper. When the real-time answering status is poor, i.e., there may be abnormal answering behavior, a target dynamic feedback list can be determined for each question to be evaluated based on the student's real-time online answering behavior information. This allows teachers to promptly grasp students' answering status and quickly locate students' knowledge gaps or problem-solving errors based on the information in the feedback list, thus facilitating targeted guidance and assistance in the subsequent test paper preparation process.

[0046] Furthermore, for any test question to be evaluated, when determining the target dynamic feedback list corresponding to the test question based on real-time online answering behavior information and the test paper information to be evaluated, it may specifically include steps S210-S2250, such as... Figure 2 As shown, where: Step S210: Determine the basic dynamic feedback list based on the test question information of the test paper to be evaluated.

[0047] Specifically, in the initial stage of answering questions, basic tags about the questions to be evaluated can be extracted from the question information and historical answer records of the test papers to be evaluated. The basic tags may include the knowledge points they belong to, historical accuracy rates, etc. The specific content is not specifically limited in this embodiment of the application, but can be set according to the actual needs of the relevant teachers. A basic dynamic feedback list is generated based on the basic tags.

[0048] Step S220: Identify the real-time behavior indicators and the corresponding behavior indicator values ​​for each real-time behavior indicator contained in the real-time online answering behavior information. The real-time behavior indicators include real-time click-through rate, real-time dwell time, and real-time accuracy rate.

[0049] Specifically, based on a preset feature recognition algorithm, the corresponding real-time behavior indicators and the behavior indicator values ​​corresponding to each real-time behavior indicator can be identified from the real-time online answering behavior information. The specific preset feature recognition algorithm is not specifically limited in this application embodiment. Since real-time click-through rate refers to the frequency of student interaction with the interface during the answering process, such as the number of times students switch questions, options, or function buttons, high-frequency clicks may indicate that students are not paying attention or are randomly selecting answers; real-time dwell time refers to the time students spend on the question to be assessed, the duration from entering the question to submitting the answer. Too short a dwell time may indicate that students are guessing answers or skipping difficult questions, while too long a dwell time may reflect that students are confused, distracted, or overthinking. If a student spends an unusually long time on the question to be assessed, it may indicate that the student has insufficient mastery of the relevant knowledge points of the question to be assessed and needs further reinforcement teaching; real-time accuracy rate refers to the accuracy rate calculated by students in real time during the answering process. The accuracy rate directly reflects the student's mastery of the corresponding knowledge points of the question to be assessed. Abnormal fluctuations in real-time accuracy rate may indicate that the student is cheating, not paying attention, or forgetting knowledge. Therefore, when analyzing whether students have abnormal answering behavior, it is necessary to consider and analyze real-time behavioral indicators such as real-time click-through rate, real-time dwell time, and real-time accuracy rate.

[0050] Step S230: Determine the real-time indicator distribution map based on the real-time behavior indicators and the corresponding behavior indicator values ​​for each real-time behavior indicator.

[0051] Specifically, after quantifying the behavior index values ​​corresponding to each real-time behavior index, each real-time behavior index and its corresponding behavior index value are imported into a preset coordinate system to obtain the index coordinate points corresponding to each real-time behavior index. Connecting the index coordinate points can yield a real-time index distribution map. The preset coordinate system contains multiple coordinate axes, each of which corresponds to a real-time behavior index. The specific method for determining the real-time index distribution map is not specifically limited in this application embodiment.

[0052] Step S240: Determine the distribution area of ​​indicators based on the real-time indicator distribution map, and determine the preset indicator area threshold based on the current answering time period.

[0053] Step S250: When the distribution area of ​​the indicator is lower than the preset indicator area threshold, abnormal indicators are identified from the real-time behavioral indicators, and the basic dynamic feedback list is optimized based on the abnormal indicators to obtain the target dynamic feedback list corresponding to the test question to be evaluated.

[0054] Specifically, the corresponding indicator distribution area can be directly calculated from the coordinate points of each real-time behavioral indicator in the real-time indicator distribution map. Different answering stages correspond to different preset indicator area thresholds. The current exam stage can be determined by the opening duration; for example, the first 10 minutes are the "adaptation period," and the last 30 minutes are the "sprint period." After determining the current exam stage, the preset indicator area threshold corresponding to the current exam stage can be determined based on the preset threshold mapping relationship. The preset threshold mapping relationship is the correspondence between the current exam stage and the preset indicator area threshold. Specific details are not limited in this embodiment and can be determined by relevant teachers based on historical experimental data and uploaded to the processing system.

[0055] When the area of ​​the indicator distribution is not lower than the preset indicator area threshold, it indicates that the relevant students did not exhibit abnormal answering behavior during the process of answering the questions to be assessed. In this case, the basic dynamic feedback list can be used. When the area of ​​the indicator distribution is lower than the preset indicator area threshold, it indicates that the relevant students exhibited abnormal answering behavior during the process of answering the questions to be assessed. In this case, the basic dynamic feedback list is no longer used. Instead, abnormal indicators need to be identified from the real-time behavioral indicators, and the basic dynamic feedback list is optimized based on the abnormal indicators. When identifying abnormal indicators from the real-time behavioral indicators, real-time behavioral indicators with behavioral indicator values ​​higher than the preset indicator threshold can be identified as abnormal indicators. Alternatively, real-time behavioral indicators with an interval distance between the indicator coordinate point and the center point of the real-time indicator distribution map greater than a preset distance threshold can also be identified as abnormal indicators. The specific number of abnormal indicators, the preset indicator threshold, and the preset distance threshold are not specifically limited in this application. When there are multiple abnormal indicators, the multiple abnormal indicators need to be sorted according to the behavioral indicator value of each abnormal indicator and then written into the basic dynamic feedback list to obtain the target dynamic feedback list.

[0056] In this embodiment of the application, by capturing behavioral indicators such as real-time click rate, real-time dwell time, and real-time accuracy rate of students during the examination process, it is easy to quantify the cognitive load and knowledge mastery status of students during the answering process. Instead of feeding back all real-time behavioral indicators, the distribution area of ​​the indicators is used as an examination condition to improve the accuracy and effectiveness of identifying abnormal indicators.

[0057] Furthermore, to facilitate tracing the source of abnormal indicators, the technical method provided in this application, after determining the abnormal indicators corresponding to the test items to be evaluated, also includes: The evaluation process involves marking test questions containing anomalous indicators as "questions of interest." Based on a pre-defined knowledge graph, related test questions and the association levels between them are determined. The association level determines the area threshold of the associated indicators for each related test question, and the associated anomalous indicators are then determined based on this threshold. Based on the test questions, multimodal teaching features are identified from multimodal teaching information, and multimodal feature fusion is performed to obtain multimodal fusion features, including student focus scores, teaching quality scores, and knowledge point fit scores. Based on the anomalous indicators, associated anomalous indicators, and multimodal fusion features, the anomaly source type is determined, including student-related causes, teaching-related causes, and test question-related causes.

[0058] Specifically, abnormal indicators can be seen as signals of potential problems and can be used to characterize atypical situations encountered by students during the answering process. A single question to be evaluated may correspond to multiple abnormal indicators. To facilitate targeted analysis and resolution of potential problems during the answering process, in this embodiment, questions containing abnormal indicators can be marked as "questions of interest." This can be achieved by overlaying an interest marker on the question to be evaluated. The specific interest marker is not specifically limited in this embodiment, but each question of interest must correspond to at least one abnormal indicator. The preset knowledge graph contains multiple knowledge points and the relationships between them. Based on the preset knowledge graph, related knowledge points that are related to the knowledge points corresponding to the questions of interest can be identified, and the questions to be evaluated corresponding to these related knowledge points are identified as related questions. Based on the relationship between the knowledge points of interest and the related knowledge points, the association level between the questions of interest and the related questions can be determined. The specific content of the preset knowledge graph is not specifically limited in this embodiment.

[0059] Different association levels correspond to different association indicator area thresholds. The association indicator area threshold is used to assess whether students have abnormal answering behavior when answering related questions. The higher the association level, the closer the relationship between the question of interest and the related questions. At this time, the corresponding association indicator area threshold is smaller. By setting a smaller association indicator area threshold for related questions with closer relationships, it is easier to detect abnormal answering behavior of students in the process of answering related questions in a timely manner, that is, the association abnormality indicator. Multimodal teaching features are used to reflect students' specific classroom performance during the stage of focusing on test questions. These features include, but are not limited to, data on focus on teaching content, data on student classroom performance, and data on focus on teaching behavior. Multimodal teaching features are fused using a pre-defined feature fusion method to obtain multimodal fused features. This method can be early fusion, late fusion, or mixed fusion. During the multimodal feature fusion process, student focus scores can be calculated by analyzing facial expressions, body movements, and eye movements. For example, a higher focus score is considered when students exhibit focused expressions, maintain a stable gaze, and make fewer irrelevant movements. Teaching quality scores can be calculated by analyzing the teacher's teaching content, teaching methods, and interaction with students. For example, a higher teaching quality score is considered when the teacher's teaching content is clear, teaching methods are diverse, and student interaction is frequent. Knowledge point fit scores can be calculated by analyzing the correlation between focus questions and teaching content, and student mastery. For example, a higher knowledge point fit score is considered when focus questions are closely related to the teaching content and students have a good grasp of the material.

[0060] After identifying the associated anomaly indicators, the associated anomaly indicators can be matched with the anomaly indicators to determine the overlapping indicators. Then, based on the overlapping indicators and the multimodal fusion features of concern, the corresponding anomaly tracing type can be determined. The anomaly tracing type includes, but is not limited to, student-related reasons, teaching-related reasons, and test-related reasons. Different anomaly tracing types correspond to different parameter combinations of overlapping indicators and multimodal fusion features of concern. Based on the preset anomaly tracing type mapping relationship, the anomaly tracing type corresponding to different parameter combinations can be determined. The specific content of the preset anomaly tracing type mapping relationship is not specifically limited in this application embodiment, but can be determined by relevant personnel based on historical experimental data and then uploaded to the processing system.

[0061] Furthermore, when the anomaly tracing type is student-related, the method provided in this application embodiment also includes: Identify students with abnormal indicators and their weaknesses based on overall exam results; determine their learning habits based on multimodal teaching information; and, based on their weaknesses and learning habits, determine remedial data from the teaching content data included in the multimodal teaching information, and then provide feedback on the remedial data to the students with concerns.

[0062] Specifically, the students whose abnormal indicators are identified are those who exhibit abnormal indicators. Based on a preset semantic recognition algorithm, weak test questions with a lower accuracy rate than the class average are identified from all test results. The knowledge points corresponding to these weak test questions are then identified as weak areas. The specific preset semantic recognition algorithm is not limited in this embodiment of the application. By identifying weak areas, the knowledge points that the students need to strengthen can be determined.

[0063] Based on a preset feature recognition algorithm, the system can identify the attention time characteristics, interaction characteristics, and homework completion characteristics of students from multimodal teaching information. These characteristics include, but are not limited to, the distribution of learning time periods, the duration of learning, and learning intervals; interaction characteristics include, but are not limited to, the frequency of interaction, the number of questions asked, and the number of answers received; and homework completion characteristics include, but are not limited to, the homework completion rate, the homework accuracy rate, and the homework submission time. By employing statistical analysis or machine learning algorithms to analyze these characteristics, the system can reveal the students' learning habits.

[0064] Learning habits include, but are not limited to, preferences for learning time, learning resources, and learning styles. Learning time preferences can include morning, afternoon, evening, etc. For example, if a student's preferred learning time is evening, tutoring data can be pushed to them in the evening to improve their learning efficiency and engagement. Learning resource preferences can include videos, text, audio recordings, etc. For example, if a student's preferred learning resource is video, tutoring data in text format can be converted to video format. Learning style preferences can include visual, auditory, and hands-on learning, etc. If a student's preferred learning style is visual, more charts, images, and video-based tutoring data can be recommended.

[0065] Initial tutoring data can be determined from the teaching content data contained in multimodal teaching information based on the weak points. Then, the format of the initial tutoring data can be adjusted according to the determined learning habits to obtain the final tutoring data. The final tutoring data is more suitable for the learning habits of the students being monitored. Therefore, after feeding the final tutoring data back to the students being monitored, the students' acceptance can be improved, thereby facilitating the improvement of the students' absorption efficiency of the tutoring data.

[0066] This application provides a processing system, such as... Figure 3 As shown, Figure 3The processing system 300 shown includes a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, for example, via a bus 302. Optionally, the processing system 300 may also include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one type, and the structure of this processing system 300 does not constitute a limitation on the embodiments of this application.

[0067] Processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0068] Bus 302 may include a pathway for transmitting information between the aforementioned components. Bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 302 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0069] The memory 303 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0070] The memory 303 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the content shown in the foregoing method embodiments.

[0071] The processing system includes, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. It can also include servers. Figure 3 The processing system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0072] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0073] This application provides a computer program product including a computer program that, when executed by a processor, implements the methods described in any of the above embodiments.

[0074] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0075] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing evaluation data, characterized in that, include: Obtain the test paper to be evaluated and the information on all examination results corresponding to the test paper to be evaluated; Identify the question information of the test paper to be evaluated, and determine the test paper model corresponding to the test paper to be evaluated based on the question information and the information of all examination results; Obtain the multimodal teaching information corresponding to the test paper to be evaluated, and determine the student's knowledge model based on the multimodal teaching information; The test paper model to be evaluated is matched with the student knowledge mastery model to obtain the model matching result. The test paper to be evaluated is evaluated based on the model matching result, and a test paper optimization report is generated based on the evaluation result.

2. The evaluation data processing method according to claim 1, characterized in that, When the multimodal teaching information includes teaching content data, student classroom performance data, and teaching behavior data, the step of determining the student knowledge mastery model based on the multimodal teaching information includes: Identify the teaching knowledge points contained in the teaching content data and the teaching relationships between each teaching knowledge point, and construct a basic teaching model based on the teaching knowledge points and the teaching relationships between each teaching knowledge point; Identify classroom interaction records and in-class assignments included in the student classroom performance data, and determine a knowledge point mastery matrix based on the classroom interaction records and in-class assignments. The knowledge point mastery matrix includes the student's mastery level for each knowledge point. The teaching duration and teaching type of each teaching knowledge point are determined from the teaching behavior data. Based on the teaching duration, teaching type, and knowledge point mastery matrix of each teaching knowledge point, the basic teaching model is optimized to obtain the student knowledge mastery model.

3. The evaluation data processing method according to claim 1, characterized in that, Also includes: Obtain real-time online answering behavior information for each question in the test paper to be evaluated; Based on the real-time online answering behavior information and the question information of the test paper to be evaluated, a target dynamic feedback list corresponding to each question to be evaluated in the test paper to be evaluated is determined; Each dynamic feedback list is overlaid onto the corresponding question to be evaluated to obtain a real-time feedback test paper.

4. The evaluation data processing method according to claim 3, characterized in that, Based on the real-time online response behavior information and the question information of the test paper to be evaluated, a target dynamic feedback list corresponding to the question to be evaluated is determined, including: A basic dynamic feedback list is determined based on the test question information of the test paper to be evaluated; The real-time online answering behavior information includes real-time behavior indicators and the behavior indicator value corresponding to each real-time behavior indicator. The real-time behavior indicators include real-time click rate, real-time dwell time, and real-time accuracy rate. Based on the real-time behavior indicators and the behavior indicator values ​​corresponding to each real-time behavior indicator, a real-time indicator distribution map is determined. The distribution area of ​​the indicators is determined based on the real-time indicator distribution map, and a preset indicator area threshold is determined based on the current response time period; When the distribution area of ​​the indicator is lower than the preset indicator area threshold, an abnormal indicator is determined from the real-time behavior indicators, and the basic dynamic feedback list is optimized based on the abnormal indicator to obtain the target dynamic feedback list corresponding to the test question to be evaluated.

5. The evaluation data processing method according to claim 4, characterized in that, After identifying the abnormal indicators corresponding to the test items to be evaluated, the following is also included: The test items to be evaluated that contain the aforementioned abnormal indicators are marked as items of concern; Based on a pre-defined knowledge graph, determine the associated questions corresponding to the questions of interest and the association level between the associated questions and the questions of interest; Based on the association level, determine the association indicator area threshold corresponding to the associated test question, and based on the association indicator area threshold, determine the association anomaly indicator corresponding to the associated test question; Based on the focus questions, focus multimodal teaching features are determined from the multimodal teaching information, and multimodal feature fusion is performed based on the focus multimodal teaching features to obtain focus multimodal fusion features. The focus multimodal fusion features include student focus score, teaching quality score, and knowledge point adaptation score. Based on the aforementioned anomaly indicators, the associated anomaly indicators, and the attention-based multimodal fusion features, the anomaly source type is determined, which includes student-related reasons, teaching-related reasons, and test-related reasons.

6. The evaluation data processing method according to claim 5, characterized in that, When the anomaly attribution type is student-related, it also includes: Identify the students to be monitored corresponding to the abnormal indicators, and determine the weak points of the students to be monitored based on the information of all exam results. The learning habits of the students being monitored are determined based on the multimodal teaching information. Based on the identified weaknesses and learning habits, remedial data is determined from the teaching content data included in the multimodal teaching information, and the remedial data is then fed back to the students being monitored.

7. A processing system, characterized in that, The processing system includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform an evaluation data processing method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, include: The computer program is stored that can be loaded by a processor and executed as any one of claims 1-6 for processing evaluation data.

9. A computer program product, characterized in that, It includes a computer program, which, when executed by a processor, implements the steps of the evaluation data processing method according to any one of claims 1-6.

Images