Student hierarchical comprehensive evaluation method and system based on multi-modal accompanying data

Abstract

This invention relates to the field of educational informatization technology and discloses a hierarchical comprehensive evaluation method and system for students based on multimodal accompanying data. The method includes: constructing an evaluation label system with a tree-like hierarchical structure; performing structured representation learning of the evaluation label nodes based on the tree-like hierarchical structure of the evaluation label system and the text descriptions of each evaluation label node to obtain a structured semantic representation of each evaluation label node; constructing a student performance representation to characterize the student's overall performance based on the multimodal accompanying data generated by the student during the learning process; matching and calculating the student performance representation with the structured semantic representations of each evaluation label node, and generating a hierarchical comprehensive evaluation result for the student from bottom to top according to the hierarchical structure of the label system; this invention can improve the accuracy, interpretability, and applicability of student evaluation results.

Description

Technical Field

[0001] This invention relates to the field of educational informatization technology, specifically to a hierarchical comprehensive evaluation method and system for students based on multimodal accompanying data. It can be applied to comprehensive student evaluation, learning analysis, and educational decision support in various teaching and management scenarios, including higher education, primary and secondary education, and online education. Background Technology

[0002] Student comprehensive evaluation is a crucial component of educational management and talent cultivation, with its results widely applied in academic diagnosis, personalized guidance, teaching improvement, and educational decision-making. With the development of educational informatization and smart education, teaching activities, learning processes, and student behaviors are increasingly being continuously recorded as data. The types and quantities of data generated by students during the learning process are constantly increasing, encompassing multiple modalities of information such as academic performance, learning behavior, platform interaction, text expression, and audio / video. Against this backdrop, data-driven student evaluation and learning analytics methods have emerged. Existing student comprehensive evaluation technologies mainly include methods based on human experience rules, methods based on feature engineering and statistical analysis, and evaluation methods based on machine learning models.

[0003] Among these methods, those based on human experience and rules typically involve education administrators or teachers developing evaluation criteria and scoring rules based on their teaching experience to assess student learning performance. These methods generally revolve around a limited number of indicators such as exam scores, homework completion, and attendance, using manually set thresholds or weights for weighted calculations to obtain student evaluation results. Some schemes also incorporate questionnaires or subjective teacher scoring as supplementary measures to characterize students' learning attitudes or overall performance. While this method is simple to implement and easy to understand, it heavily relies on human experience, resulting in highly subjective evaluation results and difficulty in handling large-scale student data and complex, multi-dimensional evaluation needs.

[0004] Feature engineering and statistical analysis-based methods typically extract and construct features from student learning process data, such as statistical indicators like average grades, fluctuations, number of assignment submissions, learning duration, and platform login frequency. These features are then used to evaluate or stratify students using linear regression, cluster analysis, or simple classification models. While these methods introduce data-driven thinking to some extent, their evaluation effectiveness heavily relies on manually designed features and rules. They have limited ability to model the relationships between evaluation dimensions and struggle to express the hierarchical structure and semantic connotations of multidimensional indicators in students' comprehensive development.

[0005] Evaluation methods based on machine learning models further incorporate supervised learning or deep learning techniques to model students' historical data and predict their grades, academic risks, or learning status. These methods typically use multimodal student data as input features, employing neural networks, sequence models, or attention mechanisms for feature learning, and outputting evaluation results or predicted scores. Commonly used models include multilayer perceptrons, recurrent neural networks, and attention-based models. While these methods improve modeling capabilities, they often treat evaluation labels as fixed output categories or values, lacking a systematic model of the semantic structure and hierarchical relationships of the evaluation labels themselves. This makes the evaluation results difficult to interpret and hinders the dynamic adjustment and expansion of the evaluation system.

[0006] These methods have improved the automation and efficiency of student assessment to some extent, but they still have significant limitations. First, existing student assessment methods mostly rely on fixed indicators or flat dimensions, lacking modeling of the hierarchical relationships and internal structures between assessment dimensions, making it difficult to reflect the correlation and progression between different ability dimensions in students' comprehensive development. Second, although some methods introduce multimodal data, they often simply fuse different modalities as feature inputs, lacking systematic modeling of the semantic connotations of the assessment labels themselves, making the assessment results difficult to interpret and hindering the expansion and maintenance of the assessment system. In addition, existing methods usually rely on manually defined rules in the assessment process, with limited semantic understanding of assessment dimensions, often requiring significant manual intervention when the assessment system is adjusted or the application scenario changes.

[0007] In practical educational applications, student evaluation not only needs to output results but also requires a clear explanation of the evaluation criteria and performance across each dimension to support instructional intervention and personalized guidance. This necessitates evaluation methods that, based on multimodal data and combined with a clear and structured evaluation label system, can achieve a hierarchical characterization and comprehensive analysis of students' multidimensional performance. However, current technologies lack a unified evaluation method that can simultaneously integrate the semantics of evaluation labels, hierarchical structural information, and students' multimodal data.

[0008] The technical problem this application aims to solve is: how to make full use of the multimodal accompanying data naturally generated by students in the process of learning and growth, and construct a comprehensive student evaluation method that can simultaneously express the semantic connotation of evaluation labels and hierarchical structural relationships, so as to achieve a hierarchical, continuous and interpretable comprehensive evaluation of students' multidimensional performance. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention provides a hierarchical comprehensive evaluation method and system for students based on multimodal accompanying data. By constructing a hierarchical evaluation label system with textual semantic descriptions, the evaluation labels themselves possess clear semantic definitions and structural relationships. Furthermore, based on this, structured representation learning is performed on the evaluation label nodes, enabling the evaluation labels to express their semantic features and hierarchical relationships in a computable manner. This effectively corresponds with the students' multimodal accompanying data, improving the accuracy, interpretability, and applicability of student evaluation results.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0011] In a first aspect, the present invention provides a hierarchical comprehensive evaluation method for students based on multimodal adjoint data, comprising: Construct an evaluation label system with a tree-like hierarchical structure, where each evaluation label node in the evaluation label system is configured with a corresponding text description; Based on the tree-like hierarchical structure of the evaluation label system and the text description of each evaluation label node, we perform structured representation learning of the evaluation label nodes to obtain the structured semantic representation of each evaluation label node. Based on the multimodal accompanying data generated by students during the learning process, a student performance representation is constructed to characterize students' overall performance. The system matches and calculates student performance representations with the structured semantic representations of each evaluation label node, and generates hierarchical comprehensive evaluation results for students from bottom to top based on the hierarchical structure of the label system.

[0012] In one embodiment, constructing an evaluation labeling system with a tree-like hierarchical structure specifically includes: The evaluation label system is organized in a tree-like hierarchical structure. The evaluation label nodes in the evaluation label system are divided into root nodes, intermediate nodes, and leaf nodes. The root node is used to represent the student's comprehensive evaluation goal, the intermediate nodes are used to represent different evaluation dimensions, and the leaf nodes are used to represent the finest-grained evaluation indicators.

[0013] In one embodiment, each evaluation tag node in the evaluation tag system is configured with a corresponding text description, specifically including: configuring a corresponding text description for each evaluation tag node in the evaluation tag system to characterize the semantic connotation, evaluation focus and role in the comprehensive evaluation of the evaluation tag node; the text description is derived from teaching management norms, training programs, course objectives or manually defined indicator descriptions.

[0014] In one embodiment, the structured representation learning of the evaluation label nodes based on the tree-like hierarchical structure of the evaluation label system and the text description of each evaluation label node, to obtain the structured semantic representation of each evaluation label node, specifically includes: Semantic encoding is performed on the text description of the evaluation label node to obtain the initial semantic vector of the evaluation label node; The tree-like hierarchical structure is treated as a graph structure. The initial semantic vectors of the evaluation label node itself and its neighboring evaluation label nodes are aggregated using a graph attention network to obtain a structured semantic representation that integrates textual semantics and hierarchical structure information.

[0015] In one embodiment, the initial semantic vectors of the evaluation label node itself and its neighboring evaluation label nodes are aggregated using a graph attention network to obtain a structured semantic representation that integrates textual semantics and hierarchical structure information, specifically including: Any rating tag node Structured semantic representation The calculation process is as follows: ; in, express In the graph structure corresponding to the tree hierarchy The set of adjacent nodes in the middle For linear mapping parameters, It is a nonlinear mapping function; To evaluate the tag node and adjacent rating label nodes Attention weights between them.

[0016] In one embodiment, the construction of a student performance representation based on multimodal accompanying data generated by students during the learning process to characterize students' overall performance specifically includes: The evaluation period is divided into continuous time windows. Feature extraction and modality encoding are performed on the multimodal adjoint data within each time window to obtain window-level multimodal performance vectors. Multimodal performance vectors within the same time window are fused to obtain window-level student performance vectors. The student performance vectors at each window level within the evaluation period are then aggregated along the time dimension to generate a student performance representation that includes average level, trend and fluctuation characteristics.

[0017] In one embodiment, dividing the evaluation period into continuous time windows and performing feature extraction and modality encoding on the multimodal adjoint data within each time window to obtain a window-level multimodal performance vector specifically includes: students During the evaluation period The multimodal adjoint data collected internally is represented as : ; For students The first generation generated in teaching and learning activities The adjoint data of each modality, For the first Index of each modality The total number of modes; Evaluation cycle Divided into A series of consecutive time windows are defined, and the adjoint data of each modality is mapped to the corresponding time window. Let the first time window be... The concomitant data sequence of a modality within the k-th time window for: ; in, Indicates the k-th time window. The number of adjoint data for each modality This represents the m-th mode within the k-th time window. Accompanying data; Regarding the first The k-th modality is aggregated using a statistical mapping function to obtain the _th_ adjoint data sequence within the k-th time window. The feature vector of a modality within the k-th time window : ; in, Indicates the first The average performance level of a modality within the k-th time window. Indicates the first The activity density of a mode in the k-th time window This indicates the amount of performance change between adjacent time windows; This represents a statistical feature mapping function defined for the m-th mode, used to map the adjoint data sequence within a time window into a fixed-length statistical feature vector; Will By transforming the modality using a modality coding mapping function, a modality representation vector with a uniform dimension is obtained: ; in, and In order to be with the first Mapping parameters corresponding to each mode For the first The modal representation vector of a modality in the k-th time window; It is a non-linear mapping function.

[0018] In one embodiment, the process of fusing multimodal performance vectors within the same time window to obtain a window-level student performance vector, and aggregating the student performance vectors at each window level within the evaluation period along the time dimension to generate a student performance representation that includes average level, trend, and fluctuation characteristics, specifically includes: Within the k-th time window, for students The modal performance vectors of each modality are weighted and fused to form the student performance vector for the k-th time window. : ; Corresponding fusion weights for: ; This represents a learnable weight parameter vector used to score the importance of each modality's performance vector. Indicates transpose. For the first The modal representation vector of a modality in the k-th time window. ; By summarizing the student performance vectors across different time windows within the evaluation period, a student performance representation can be obtained. : ; Among them, the average level of student performance vectors in each time window within the evaluation period. Trends in student performance vectors within adjacent time windows Fluctuation characteristics of student performance vectors relative to the average level in each time window .

[0019] In one embodiment, the step of matching and calculating the student performance representation with the structured semantic representation of each evaluation label node, and generating a hierarchical comprehensive evaluation result for the student from bottom to top based on the hierarchical structure of the label system, specifically includes: For any leaf node in a tree-like hierarchical structure, the evaluation label node is... By matching functions for students Student performance characteristics With rating tag nodes Structured semantic representation Perform matching calculations to obtain students In the evaluation label node Evaluation score : ; in, Indicates the matching function; Any evaluation label node that is a non-leaf node The evaluation score is determined by The evaluation scores of the child nodes are combined to generate the evaluation label node. The set of child nodes is Then evaluate the label node. The evaluation score is: ; Indicates that students are child nodes The evaluation score on the platform express The corresponding aggregate weights; By a hierarchical aggregation method, the final hierarchical comprehensive evaluation results of students are generated.

[0020] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0021] Compared with the prior art, the beneficial technical effects of the present invention are: This invention uses multimodal accompanying data continuously generated by students during their actual learning process as the evaluation basis, avoiding the information gaps caused by relying solely on outcome-based or stage-based data. This makes student evaluation more comprehensive and continuous, and can truly reflect students' learning status and development process.

[0022] This invention uses hierarchical and semantic modeling of the evaluation label system to explicitly express the relationships and hierarchical structure between evaluation dimensions. The evaluation results not only provide comprehensive conclusions but also clearly present the performance of each fine-grained dimension, significantly improving the interpretability and understandability of the evaluation results.

[0023] This invention combines the semantic representation of evaluation labels with students' multimodal accompanying data, giving the evaluation system good scalability and adaptability. When evaluation indicators are adjusted or application scenarios change, there is no need to reconstruct the overall model structure on a large scale, which reduces system maintenance costs and enhances the application value of the technical solution in actual educational scenarios. Attached Figure Description

[0024] Figure 1 This is a flowchart of the method of the present invention.

[0025] Figure 2 This invention provides a hierarchical evaluation label system with textual semantics.

[0026] Figure 3 This is a flowchart illustrating the modeling process for the structured semantic representation of the evaluation tag node in this invention.

[0027] Figure 4 This is a flowchart illustrating the calculation process for student performance representation in this invention.

[0028] Figure 5 This is a schematic diagram of the student comprehensive evaluation hierarchy aggregation of the present invention. Detailed Implementation

[0029] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] like Figure 1 As shown, this invention provides a hierarchical comprehensive evaluation method for students based on multimodal adjoint data, comprising the following steps: S1. Construct an evaluation label system with a tree-like hierarchical structure. Each evaluation label node in the evaluation label system is configured with a corresponding text description. S2, based on the tree-like hierarchical structure of the evaluation label system and the text description of each evaluation label node, perform structured representation learning of the evaluation label nodes to obtain the structured semantic representation of each evaluation label node; S3, based on the multimodal accompanying data generated by students during the learning process, constructs a student performance representation to characterize students' overall performance; S4 matches and calculates the student performance representation with the structured semantic representation of each evaluation label node, and generates the student's hierarchical comprehensive evaluation result from bottom to top according to the hierarchical structure of the label system.

[0031] The following is a detailed introduction in several parts.

[0032] I. Construct a hierarchical evaluation label system with textual semantics.

[0033] First, an evaluation label system is constructed based on the educational evaluation objectives, and organized into a tree-like hierarchical structure, including a root node, intermediate nodes, and leaf nodes. The root node represents the student's comprehensive evaluation objective, the intermediate nodes represent different evaluation dimensions, and the leaf nodes represent the finest-grained evaluation indicators. For each evaluation label node in the evaluation label system, a corresponding text description is configured to characterize the semantic connotation, evaluation basis, and scope of application of that evaluation dimension, thus enabling the evaluation labels to possess not only structural information but also clear semantic expression capabilities.

[0034] like Figure 2 As shown, the evaluation label system is represented as a tree-like directed graph structure. : ; in, To evaluate the set of label nodes, For the Nth evaluation label node, This is the set of parent-child hierarchy edges between evaluation label nodes. Any evaluation label node... It corresponds to at least one parent node or a child node, and any leaf node no longer contains a child node.

[0035] For each evaluation label node in the evaluation label system Each evaluation label is configured with corresponding text description information to characterize its semantic connotation, evaluation focus, and role in student comprehensive evaluation. The text description can originate from teaching management regulations, curriculum plans, course objectives, or manually defined indicator descriptions, and can be in the form of natural language text. An evaluation label node is defined. The text description is represented as This is used to characterize the semantic definition of the evaluation label.

[0036] In this way, the evaluation label system not only has a clear hierarchical structure, but also has a computable semantic description basis. This provides input conditions for subsequent representation modeling of evaluation labels based on structured representation learning methods, and lays a unified semantic and structural foundation for the matching calculation between student performance representation and evaluation labels.

[0037] II. Construction of Evaluation Tag Node Representation Based on Hierarchical Structure and Textual Semantics

[0038] After constructing the evaluation label system, the tree-like hierarchical structure is transformed into a graph structure representation, and the textual description of each evaluation label node is encoded to obtain the initial semantic vector of the evaluation label node. Based on this, a graph attention network is introduced to learn the structured representation of the evaluation label nodes. Through message passing and attention aggregation between evaluation label nodes, the representation of each evaluation label node can simultaneously integrate its own semantic information and its upstream and downstream relationships in the hierarchical structure, thereby obtaining the structured semantic representation of each evaluation label node.

[0039] like Figure 3 As shown, firstly, for any evaluation label node in the evaluation label system... Text description Semantic encoding is performed. The text description is vectorized using an encoding method based on a pre-trained language model. In a preferred embodiment, a bidirectional encoder representation model (BERT) can be used to encode the text description of the evaluation label node to obtain the initial semantic vector of the evaluation label node. : .

[0040] After obtaining the initial semantic vectors of the evaluation label nodes, the hierarchical structure of the evaluation label system is treated as a graph structure, and a structured representation learning is performed on the initial semantic vectors within this structure. By introducing a neighborhood information aggregation method based on an attention mechanism among the evaluation label nodes, the node representation can simultaneously integrate its own semantic information and information from neighboring nodes with which it has a hierarchical relationship. For any evaluation label node... Its structured semantic representation The calculation process is expressed as follows: .

[0041] in, Represents the rating tag node In graph structure The set of adjacent nodes in the middle For linear mapping parameters, For non-linear mapping functions, attention weights It can be calculated based on the similarity function between the initial semantic vectors of the evaluation tag nodes and then normalized.

[0042] The above process can be iterated multiple times on the structure graph to integrate higher or lower level structural semantic information layer by layer, so that each evaluation label node corresponds to a node representation vector that contains both textual semantic information and hierarchical structural information. This provides a unified and computable label representation basis for subsequent matching calculations between student multimodal adjoint performance vectors and evaluation label node representations.

[0043] III. Constructing a representation of the performance of students' multimodal adjoint data.

[0044] Collect multimodal accompanying data naturally generated by students during the learning process, including but not limited to learning behavior data, platform interaction data, academic process data, and text, voice, or audio-visual data. Preprocess, extract, and fuse these multimodal accompanying data to construct a unified student performance representation, used to characterize students' comprehensive performance features in real-world learning scenarios.

[0045] like Figure 4 As shown, let the student During the evaluation period The multimodal adjoint data collected internally is represented as ; The accompanying data for each modality are all derived from data records naturally generated by students during normal teaching and learning activities.

[0046] Evaluation cycle Divided into several consecutive time windows And map each modal data to its corresponding time window. The mode is in the k-th time window The adjoint data sequence within is represented as: ; in, Indicates the k-th time window. The number of adjoint data for each modality.

[0047] Regarding the first For each modality, a statistical mapping function is used to aggregate the record sequences within the window to obtain the modality's performance feature vector within that time window: ; in, Indicates the first The average performance level of a modality in the k-th time window is specifically the arithmetic mean of the adjoint data in that time window. Indicates the first The activity density of a modality in the k-th time window is specifically the frequency of occurrence recorded per unit time. This represents the change in performance between adjacent time windows, specifically the difference in the average performance level between adjacent time windows; when the modal record is in vector form, the above operation is performed according to the dimension.

[0048] To achieve representation of different modal representation features in a unified feature space, the modal representation feature vectors are transformed using a modal coding mapping function to obtain modal representation vectors of a unified dimension: ; in, and In order to be with the first Mapping parameters corresponding to each modality. For modal data that does not generate valid records within a certain time window, a modality availability factor is introduced to adjust its impact, thereby reducing the impact of missing data on the subsequent fusion process.

[0049] Within the same time window, the modal performance vectors of each modality are weighted and fused to form a window-level student performance vector. : .

[0050] The corresponding fusion weights are: .

[0051] By summarizing the student performance vectors corresponding to each time window within the evaluation period along the time dimension, and aggregating and mapping their average level, trend, and fluctuation characteristics, a student performance representation is obtained. .

[0052] in: ; ; ; This represents the average level of student performance vectors across different time windows within the evaluation period, used to characterize the overall performance of students. This represents the changing trend of student performance vectors within adjacent time windows, used to characterize the evolution direction of student performance; This represents the fluctuation characteristics of student performance vectors relative to the average level in each time window, used to characterize the stability of student performance. By combining and mapping the above features, a comprehensive characterization of students' multimodal co-occurring performance can be achieved in three aspects: overall level, trend of change, and stability.

[0053] IV. Generate hierarchical evaluation results based on node representation and evidence representation.

[0054] like Figure 5 As shown, this invention matches and calculates the student's performance representation with the structured semantic representation of the evaluation label node to obtain the student's evaluation score on each evaluation label node; and organizes and outputs the student's multi-dimensional, hierarchical comprehensive evaluation results according to the hierarchical structure of the evaluation label system, so that the evaluation results can simultaneously reflect the student's performance in fine-grained evaluation dimensions and the overall comprehensive development level.

[0055] The student's performance representation is matched with the structured semantic representation of the evaluation label node using a matching function to obtain the student's evaluation score at the corresponding evaluation label node. : ; in This represents a matching function used to measure the correlation between student performance vectors and evaluation label node representations. It can be implemented using vector similarity, inner product, or a scoring method based on linear mapping.

[0056] After obtaining the students' evaluation scores at each evaluation label node, the evaluation results are summarized layer by layer from the leaf nodes upwards according to the hierarchical structure of the evaluation label system. For any non-leaf evaluation label node, its evaluation score is generated by combining the evaluation scores of its subordinate child nodes. The combination method is implemented through weighted summation, where the weights corresponding to each child node are used to characterize the relative importance of different evaluation dimensions in the upper-level evaluation.

[0057] Set evaluation label nodes The set of child nodes is Then evaluate the label node. The evaluation score is expressed as follows: ; in, Indicates that the student is in the child node The evaluation score on the platform This represents the aggregate weight corresponding to the child node. The weight can be pre-configured, manually set, or learned from historical data.

[0058] By using the above-mentioned hierarchical aggregation method, the evaluation results can maintain the differentiation of fine-grained evaluation dimensions while gradually forming intermediate and comprehensive evaluation results, ultimately generating a hierarchical multidimensional comprehensive evaluation output for students.

[0059] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0060] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0061] In one embodiment, the present invention provides a computer system, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0064] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A hierarchical comprehensive evaluation method for students based on multimodal adjoint data, characterized in that, include: Construct an evaluation label system with a tree-like hierarchical structure, where each evaluation label node in the evaluation label system is configured with a corresponding text description; Based on the tree-like hierarchical structure of the evaluation label system and the text description of each evaluation label node, we perform structured representation learning of the evaluation label nodes to obtain the structured semantic representation of each evaluation label node. Based on the multimodal accompanying data generated by students during the learning process, a student performance representation is constructed to characterize students' overall performance. The system matches and calculates student performance representations with the structured semantic representations of each evaluation label node, and generates hierarchical comprehensive evaluation results for students from bottom to top based on the hierarchical structure of the label system.

2. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 1, characterized in that, The construction of the evaluation label system with a tree-like hierarchical structure specifically includes: The evaluation label system is organized in a tree-like hierarchical structure. The evaluation label nodes in the evaluation label system are divided into root nodes, intermediate nodes, and leaf nodes. The root node is used to represent the student's comprehensive evaluation goal, the intermediate nodes are used to represent different evaluation dimensions, and the leaf nodes are used to represent the finest-grained evaluation indicators.

3. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 1, characterized in that, Each evaluation tag node in the evaluation tag system is configured with a corresponding text description, specifically including: for each evaluation tag node in the evaluation tag system, a corresponding text description is configured to describe the semantic connotation, evaluation focus and role of the evaluation tag node in the student comprehensive evaluation; the text description comes from teaching management norms, training programs, course objectives or manually defined indicator descriptions.

4. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 1, characterized in that, The hierarchical tree structure based on the evaluation label system and the text description of each evaluation label node are used to perform structured representation learning of the evaluation label nodes, resulting in a structured semantic representation of each evaluation label node. Specifically, this includes: Semantic encoding is performed on the text description of the evaluation label node to obtain the initial semantic vector of the evaluation label node; The tree-like hierarchical structure is treated as a graph structure. The initial semantic vectors of the evaluation label node itself and its neighboring evaluation label nodes are aggregated using a graph attention network to obtain a structured semantic representation that integrates textual semantics and hierarchical structure information.

5. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 4, characterized in that, The method of using a graph attention network to aggregate the initial semantic vectors of the evaluation label node itself and its neighboring evaluation label nodes yields a structured semantic representation that integrates textual semantics and hierarchical structure information, specifically including: Any rating tag node Structured semantic representation The calculation process is as follows: ； in, express In the graph structure corresponding to the tree hierarchy The set of adjacent nodes in the middle For linear mapping parameters, It is a nonlinear mapping function; To evaluate the tag node and adjacent rating label nodes Attention weights between them.

6. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 1, characterized in that, The student performance representation, constructed based on multimodal accompanying data generated by students during the learning process, is used to characterize students' overall performance. Specifically, it includes: The evaluation period is divided into continuous time windows. Feature extraction and modality encoding are performed on the multimodal adjoint data within each time window to obtain window-level multimodal performance vectors. Multimodal performance vectors within the same time window are fused to obtain window-level student performance vectors. The student performance vectors at each window level within the evaluation period are then aggregated along the time dimension to generate a student performance representation that includes average level, trend and fluctuation characteristics.

7. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 6, characterized in that, The process of dividing the evaluation period into continuous time windows and performing feature extraction and modality encoding on the multimodal adjoint data within each time window to obtain a window-level multimodal performance vector specifically includes: students During the evaluation period The multimodal adjoint data collected internally is represented as : ; For students The first generation generated in teaching and learning activities The adjoint data of each modality, For the first Index of each modality The total number of modes; Evaluation cycle Divided into A series of consecutive time windows are defined, and the adjoint data of each modality is mapped to the corresponding time window. Let the first time window be... The concomitant data sequence of a modality within the k-th time window for: ； in, Indicates the k-th time window. The number of adjoint data for each modality This represents the m-th mode within the k-th time window. Accompanying data; Regarding the first The k-th modality is aggregated using a statistical mapping function to obtain the _th_ adjoint data sequence within the k-th time window. The feature vector of a modality within the k-th time window : ； in, Indicates the first The average performance level of a modality within the k-th time window. Indicates the first The activity density of a mode in the k-th time window This indicates the amount of performance change between adjacent time windows; This represents a statistical feature mapping function defined for the m-th mode, used to map the adjoint data sequence within a time window into a fixed-length statistical feature vector; Will By transforming the modality using a modality coding mapping function, a modality representation vector with a uniform dimension is obtained: ； in, and In order to be with the first Mapping parameters corresponding to each mode For the first The modal representation vector of a modality in the k-th time window; It is a non-linear mapping function.

8. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 7, characterized in that, The process involves fusing multimodal performance vectors within the same time window to obtain a window-level student performance vector. Then, the student performance vectors at each window level within the evaluation period are aggregated along the time dimension to generate a student performance representation that includes average level, trend, and fluctuation characteristics. Specifically, this includes: Within the k-th time window, for students The modal performance vectors of each modality are weighted and fused to form the student performance vector for the k-th time window. : ; Corresponding fusion weights for: ; This represents a learnable weight parameter vector, used to score the importance of each modality's performance vector; Indicates transpose. For the first The modal representation vector of a modality in the k-th time window. ; By summarizing the student performance vectors across different time windows within the evaluation period, a student performance representation can be obtained. : ； Among them, the average level of student performance vectors in each time window within the evaluation period. Trends in student performance vectors within adjacent time windows Fluctuation characteristics of student performance vectors relative to the average level in each time window .

9. The student hierarchical comprehensive evaluation method based on multimodal adjoint data according to claim 1, characterized in that, The process of matching and calculating student performance representations with the structured semantic representations of each evaluation label node, and generating a hierarchical comprehensive evaluation result for students from bottom to top based on the hierarchical structure of the label system, specifically includes: For any leaf node in a tree-like hierarchical structure, the evaluation label node is... By matching functions for students Student performance characteristics With rating tag nodes Structured semantic representation Perform matching calculations to obtain students In the evaluation label node Evaluation score : ； in, Indicates the matching function; Any evaluation label node that is a non-leaf node The evaluation score is determined by The evaluation scores of the child nodes are combined to generate the evaluation label node. The set of child nodes is Then evaluate the label node. The evaluation score is: ； Indicates that students are child nodes The evaluation score on the platform express The corresponding aggregate weights; By a hierarchical aggregation method, the final hierarchical comprehensive evaluation results of students are generated.

10. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

Images