Multi-dimensional learning situation graph construction method and system based on graph neural network

Abstract

The application discloses a kind of multi-dimensional learning situation graph construction method and system based on graph neural network, and the construction method includes: obtaining the multi-modal learning behavior data of student;Extract the dynamic multi-modal interaction features of behavior data, and determine the cognitive dependence relationship between knowledge points;Based on student, test question, three kinds of nodes of knowledge point, and cognitive dependence relationship, construct multi-level heterogeneous graph, the multi-level heterogeneous graph includes bottom interaction graph and upper cognitive graph;Multi-level heterogeneous graph is input into layered graph neural network, and the node embedding is updated in bottom interaction graph by multi-scale attention fusion feature, meta-path heterogeneous graph convolution, and knowledge is realized in upper cognitive graph Knowledge forward and backward propagation, generate learning situation graph based on updated node embedding, and the learning situation graph includes individual knowledge mastery state and group knowledge mastery distribution.The application can capture learning interaction and knowledge association features, improve the comprehensiveness of learning situation analysis, and significantly improve the accuracy of learning situation graph construction.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and in particular relates to a method and system for constructing multidimensional learning information maps based on graph neural networks. Background Technology

[0002] With the popularization of online education platforms and the in-depth advancement of smart campus construction, educational big data is experiencing rapid growth. How to extract students' knowledge mastery status from massive amounts of learning behavior data to achieve learning diagnosis and personalized teaching has become a research hotspot in the field of educational data mining.

[0003] Traditional learning assessment methods are mainly divided into two categories: one is based on cognitive diagnostic theory, such as item response theory (IRT) and deterministic input noise AND gate model (DINA), which assesses students' knowledge mastery level through preset cognitive parameters; the other is based on deep learning knowledge tracking model (DKT), which uses recurrent neural network (RNN) or long short-term memory network (LSTM) to model students' answer sequences and predict future answer performance.

[0004] In recent years, with the rapid development of graph neural networks (GNNs), researchers have begun to introduce heterogeneous graphs into the field of learning analysis. By constructing interactive relationships between various types of nodes such as students, test questions, and knowledge points, and utilizing the message passing mechanism of graph neural networks, it is possible to capture structured information in the learning process more comprehensively.

[0005] However, the existing technology still has the following technical problems:

[0006] (1) The existing learning status diagnosis methods have a single data source, which only uses students' answer records or cognitive level labels, resulting in insufficient accuracy and comprehensiveness in the diagnosis of knowledge status;

[0007] (2) The knowledge point relationship graphs constructed by existing methods are mostly undirected graphs based on statistical co-occurrence, which only reflect the surface relationship between knowledge points, making the knowledge transmission logic unable to fit the actual teaching cognition and the internal relationship of subject knowledge;

[0008] (3) The existing knowledge propagation mechanism of graph neural networks is mainly based on one-way aggregation, which only collects information from neighboring nodes to update its own embedding. It cannot simulate the bidirectional interaction between prior knowledge points and subsequent knowledge points in the knowledge learning process. At the same time, it lacks effective control over the feedback intensity when constructing backpropagation, making it difficult to guarantee the mathematical computability of knowledge state updates and model convergence. As a result, the knowledge propagation mechanism does not conform to the real educational cognition law. Summary of the Invention

[0009] The purpose of this invention is to provide a method and system for constructing a multidimensional learning information map based on graph neural networks, so as to solve the technical problems mentioned in the background.

[0010] To achieve the above objectives, the present invention provides the following technical solution.

[0011] According to an embodiment of the present invention, a method for constructing a multidimensional learning information map based on a graph neural network is provided, comprising the following steps:

[0012] Acquire multimodal learning behavior data from learners;

[0013] Extract dynamic multimodal interaction features from behavioral data and determine the cognitive dependencies between knowledge points;

[0014] Based on three types of nodes—students, test questions, and knowledge points—and the aforementioned cognitive dependencies, a multi-level heterogeneous graph is constructed. This multi-level heterogeneous graph includes a bottom-level interaction graph and an upper-level cognitive graph. The bottom-level interaction graph uses the relationship between students and test questions, and the relationship between test questions and knowledge points, as edges. The upper-level cognitive graph uses knowledge points as nodes and cognitive dependencies as directed edges.

[0015] Multi-level heterogeneous graphs are input into a hierarchical graph neural network for hierarchical representation learning, including:

[0016] On the underlying interaction graph, dynamic multimodal interaction features are adaptively fused through a multi-scale attention mechanism, and node embeddings are updated through heterogeneous graph convolution guided by meta-paths, aggregating only effective neighborhood information on meta-paths; during message passing, the fused multimodal interaction features and node embeddings are concatenated and encoded through the edge node interaction layer, and the updated node embeddings are output after nonlinear transformation.

[0017] On the upper-level cognitive map, forward propagation is carried out according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical reverse propagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points.

[0018] A learning map is generated based on the updated node embedding, which includes the individual knowledge mastery status and the group knowledge mastery distribution.

[0019] Furthermore, the dynamic multimodal interaction features include at least two of the following: emotional features, physiological features, behavioral features, and environmental features;

[0020] The cognitive dependency relationship is a directed weighted relationship, which represents the degree of prerequisite dependency between knowledge points, including strong prerequisite relationships, weak prerequisite relationships, and correlation relationships.

[0021] Furthermore, the multi-scale attention mechanism includes:

[0022] Intramodal self-attention is used to capture the temporal dynamic correlation of features in a single modality;

[0023] Intermodal cross-attention is used to dynamically adjust the contribution weights of different modalities based on the embedding states of student nodes and question nodes.

[0024] Furthermore, the step of adaptively fusing the dynamic multimodal interaction features on the underlying interaction graph through a multi-scale attention mechanism includes:

[0025] In intramodal self-attention, for each modal feature, the temporal sequence of the modal feature is input into the self-attention model for intramodal encoding. Based on the dynamic correlation of the modal features at different time steps, the enhanced feature vector after intramodal encoding is obtained.

[0026] In intermodal cross-attention, for the t-th interaction, the attention weights of each modal feature sub-vector are calculated based on the dynamic multimodal interaction feature vector, and are expressed as:

[0027]

[0028] In the formula, and These are the embedding representations of the current student node and the question node, respectively. , Represents the weight matrix. , Indicates the bias term; Represents a multimodal interaction feature vector; These represent different modal characteristics, namely emotional characteristics, physiological characteristics, behavioral characteristics, and environmental characteristics;

[0029] The multimodal features are weighted and fused according to the attention weights to obtain the fused features. , is represented as:

[0030]

[0031] In the formula, Indicates attention weights. Represents dynamic multimodal interaction feature vectors In the diagram, the feature vector corresponding to the i-th mode;

[0032] The fused features are used as attributes of the answer edges and participate in the message passing process of graph convolution.

[0033] Furthermore, in the forward propagation on the upper-level cognitive graph, the weights are dynamically generated based on the current embedding state of the knowledge point nodes, and the forward propagation is represented as:

[0034]

[0035] In the formula, This represents the set of all prerequisite knowledge points for knowledge point j. This represents the dependency strength weight between the prerequisite knowledge point p and the subsequent knowledge point j. This represents the pre-propagation weight matrix; This represents the weight matrix of its own state; Indicates the activation function; This indicates the embedded state of the prior knowledge point p before it is updated; This indicates the embedding state of the (l+1)th layer of the knowledge point j to be modified later;

[0036] In the hierarchical backpropagation, differentiated feedback coefficients are used based on the type of cognitive dependency. The backpropagation is expressed as follows:

[0037]

[0038] In the formula, This represents the set of all subsequent knowledge points for knowledge point p. This represents the dependency strength weight between prerequisite knowledge point p and subsequent knowledge point j. Indicates the feedback coefficient. This represents the embedded state after the prior knowledge point p is updated. This indicates the embedding state of the prior knowledge point p at the l-th layer.

[0039] Furthermore, the step of updating node embeddings via heterogeneous graph convolution guided by meta-paths, aggregating only valid neighborhood information on the meta-paths, includes:

[0040] Constructing meta-path sets Each metapath The metapath represents a sequence of semantic relationships between node types; the metapath includes at least one of student-question-student, student-question-knowledge point, question-knowledge point-question, and knowledge point-question-knowledge point.

[0041] For each metapath Based on the multi-level heterogeneous graph, construct the adjacency matrix corresponding to the meta-path. ,in This indicates that there exists a meta-path from node i to node j. The path does not conform to the metapath. Even if a node has a direct edge connection, it will not participate in subsequent message passing as a meta-path neighbor.

[0042] For the target node v, according to the metapath Obtain its meta-path neighbor set, calculate the attention weights between the target node v and its meta-path neighbor u, and fuse the multimodal interaction features. As a weighting factor, after being mapped through a linear transformation layer, it is multiplied element-wise with the embeddings of the meta-path neighbors; according to the attention weight, only the node embeddings in the meta-path neighbor set are weighted and aggregated to obtain the meta-path. Semantic-specific node embedding;

[0043] For the target node v, aggregate the semantically specific node embeddings of all metapaths to obtain the final updated node embedding. .

[0044] Furthermore, the multi-level heterogeneous graph also includes an upper-level state graph, which constructs a knowledge state evolution trajectory using students as nodes and preset time windows as sequences, including:

[0045] Divide the timeline into multiple time windows of equal length. In each time window Within the window, a corresponding node embedding snapshot is generated based on the interactive data within that window. Where N is the number of student nodes. This indicates that student n is within the time window. Knowledge state embedding within.

[0046] Furthermore, hierarchical graph neural networks also include temporal evolution layers, which are used to capture the patterns of knowledge state evolution over time through gated recurrent units or long short-term memory networks.

[0047] In the time-series evolution layer, the time axis is divided into multiple time windows of equal length. Within each time window, a corresponding node embedding snapshot is generated based on the interaction data within the window. The nodes are embedded into a snapshot sequence input gated recurrent unit or a long short-term memory network to capture long-term dependencies in the evolution of knowledge states over time.

[0048] The output of the gated recurrent unit or long short-term memory network is a dynamically updated node embedding, which integrates historical knowledge state and current interaction information.

[0049] According to another embodiment of the present invention, a multidimensional learning information mapping system based on graph neural networks is also provided, comprising the following modules:

[0050] The data acquisition module is used to acquire multimodal learning behavior data of trainees;

[0051] The feature extraction module is used to extract dynamic multimodal interaction features from behavioral data and determine the cognitive dependencies between knowledge points.

[0052] The graph construction module is used to construct a multi-level heterogeneous graph based on three types of nodes: students, test questions, and knowledge points, as well as the cognitive dependency relationship. The multi-level heterogeneous graph includes a bottom-level interaction graph and an upper-level cognitive graph. The bottom-level interaction graph uses the relationship between students and test questions and the relationship between test questions and knowledge points as edges. The upper-level cognitive graph uses knowledge points as nodes and cognitive dependency relationships as directed edges.

[0053] The graph neural network module is used to input multi-level heterogeneous graphs into a hierarchical graph neural network for hierarchical representation learning. This includes: on the bottom interaction graph, adaptively fusing dynamic multimodal interaction features through a multi-scale attention mechanism, updating node embeddings through heterogeneous graph convolution guided by meta-paths, and aggregating only effective neighborhood information on meta-paths; during message passing, concatenating and encoding the fused multimodal interaction features and node embeddings through an edge node interaction layer, and outputting the updated node embeddings after nonlinear transformation; on the upper cognitive graph, forward propagation is performed according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical backpropagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points.

[0054] The knowledge graph generation module is used to generate a learning knowledge graph based on the updated node embeddings. The learning knowledge graph includes individual knowledge mastery status and group knowledge mastery distribution.

[0055] According to another embodiment of the present invention, a storage medium is provided, the storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform the steps of the multidimensional learning information map construction method based on graph neural networks as described in the above embodiments.

[0056] Compared with existing technologies, the beneficial effects of the multidimensional learning information mapping method based on graph neural networks in this invention are:

[0057] This invention acquires students' multimodal learning behavior data, extracts dynamic multimodal interaction features, and determines the directed weighted cognitive dependency relationship between knowledge points based on knowledge space theory. This relationship represents the degree of prerequisite dependency of knowledge points and satisfies cognitive constraints such as transitivity and antisymmetry, providing a semantic basis that fits cognition for knowledge dissemination.

[0058] This invention introduces an attention mechanism into the underlying interaction graph to adaptively fuse dynamic multimodal interaction features, enabling the perception and diagnosis of learners' learning status. Simultaneously, based on nodes and cognitive dependencies such as learners, test questions, and knowledge points, a multi-layered heterogeneous graph is constructed, comprising an underlying interaction graph and an upper-level cognitive graph. Through hierarchical representation learning in a hierarchical graph neural network, node embeddings are updated in the underlying interaction graph using attention-fusion features. In the upper-level cognitive graph, a bidirectional propagation mechanism is used to achieve both positive knowledge transfer from prior knowledge points to subsequent knowledge points and dynamic feedback updates of prior knowledge point status based on the mastery of subsequent knowledge points through a backpropagation mechanism combined with feedback coefficients. This simulates the bidirectional interactive influence process of knowledge learning.

[0059] In summary, this invention achieves multi-dimensional representation of learning progress by constructing multi-level heterogeneous graphs and learning hierarchical graph neural networks, thereby improving the accuracy of learning progress analysis and providing a basis for personalized teaching recommendations and group learning progress diagnosis. Attached Figure Description

[0060] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0061] In the attached diagram:

[0062] Figure 1 This is a flowchart illustrating the implementation of the multidimensional learning information map construction method based on graph neural networks of the present invention.

[0063] Figure 2 This is a sub-flowchart of the multidimensional learning information map construction method based on graph neural networks of the present invention;

[0064] Figure 3 This is a structural block diagram of the multidimensional learning information map construction system based on graph neural networks of the present invention;

[0065] Figure 4 This is a structural block diagram of a computer device according to the present invention. Detailed Implementation

[0066] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0067] It should be noted that, in the embodiments of this application, the information collected is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with relevant laws, regulations and standards, take necessary protective measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0068] The specific implementation of this application will be described in detail below with reference to specific embodiments.

[0069] Please refer to Figure 1 In one embodiment of the present invention, a method for constructing a multidimensional learning information map based on a graph neural network is provided, comprising the following steps:

[0070] Step S101: Obtain multimodal learning behavior data of trainees;

[0071] Specifically, in step S101 of this invention, multimodal learning behavior data of learners is collected from the online learning platform, including question-answering interaction data, emotional data, physiological data, behavioral data, and environmental data. The question-answering interaction data can be records of students answering questions, including question number, answering time, and answer result (correct / incorrect). Emotional data is collected via facial video from a camera and voice data from a microphone. Physiological data is collected via eye-tracking devices to capture eye movement trajectories and via wearable devices to capture skin conductance response data. Behavioral data can include recorded answering time, number of modifications, mouse trajectory, etc. Environmental data includes recorded learning time periods, learning locations, and device types.

[0072] Please continue to refer to Figure 1 The multidimensional learning information mapping method based on graph neural networks provided by this invention also includes:

[0073] Step S102: Extract dynamic multimodal interaction features from behavioral data and determine the cognitive dependencies between knowledge points;

[0074] Among them, cognitive dependency relationship is a directed weighted relationship, which represents the degree of prerequisite dependency between knowledge points; cognitive dependency relationship includes strong prerequisite relationship, weak prerequisite relationship and correlation relationship, and different dependency strength weights are assigned to each relationship; the dependency strength weights are dynamically optimized based on knowledge base and group learning path data.

[0075] In one implementation of the present invention, the dynamic multimodal interaction features include at least two of the following: emotional features, physiological features, behavioral features, and environmental features.

[0076] The emotional features in this embodiment are obtained based on facial expressions and voice data. For example, by inputting facial video and voice data into a three-dimensional convolutional neural network, continuous value vectors of four emotional dimensions, namely focus, confusion, frustration and pleasure, are extracted.

[0077] The physiological features provided in this embodiment are obtained based on eye movement trajectory and skin conductance response data. For example, physiological feature sub-vectors can be obtained by encoding the time-series data of eye movement trajectory and skin conductance response.

[0078] The behavioral features provided in this embodiment of the invention are obtained based on the answer time and mouse trajectory data. For example, the answer time, number of modifications, and mouse trajectory are encoded to obtain behavioral feature sub-vectors.

[0079] The environmental features provided in this embodiment are obtained based on learning time period and learning location data. For example, by embedding the learning time period, learning location, and device type, an environmental feature sub-vector is obtained.

[0080] Furthermore, when determining the cognitive dependencies between knowledge points, this invention predefines initial cognitive dependencies based on the course syllabus and knowledge base. By analyzing the group learning path data of students, it dynamically updates the strength and direction of cognitive dependencies using Bayesian structure learning. For example, the cognitive dependencies provided in this embodiment are represented as a directed weighted graph. Directed weighted graph Transitivity constraint to be satisfied: For any knowledge points i, j, k, if and ,but And there is a weighted graph. Satisfying antisymmetry constraint: If ,but .

[0081] Please continue to refer to Figure 1 The multidimensional learning information map construction method based on graph neural networks in this embodiment of the invention further includes:

[0082] Step S103: Based on the three types of nodes—students, test questions, and knowledge points—and the cognitive dependency relationship, construct a multi-level heterogeneous graph, which includes a bottom-level interaction graph and an upper-level cognitive graph.

[0083] In step S103 of the present invention, the underlying interaction graph uses students, test questions, and knowledge points as nodes, and the relationship between students and test questions and the relationship between test questions and knowledge points as edges. The dynamic multimodal interaction feature vector extracted in step 102 is used as the edge attribute of the answer edge.

[0084] The upper-level cognitive graph provided by this invention uses knowledge points as nodes and the cognitive dependencies determined in step S102 as directed edges. The direction of the edge represents the direction of knowledge transmission, and the weight of the edge represents the strength of the dependency.

[0085] Please continue to refer to Figure 1 The multidimensional learning information map construction method based on graph neural networks in this embodiment of the invention further includes:

[0086] Step S104: Input the multi-level heterogeneous graph into the hierarchical graph neural network to perform hierarchical representation learning, including:

[0087] On the underlying interaction graph, dynamic multimodal interaction features are adaptively fused through a multi-scale attention mechanism, and node embeddings are updated through heterogeneous graph convolution guided by meta-paths, aggregating only effective neighborhood information on meta-paths; during message passing, the fused multimodal interaction features and node embeddings are concatenated and encoded through the edge node interaction layer, and the updated node embeddings are output after nonlinear transformation.

[0088] On the upper-level cognitive map, forward propagation is carried out according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical reverse propagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points.

[0089] In this embodiment of the invention, the hierarchical graph neural network includes a bottom-level graph convolutional layer, a middle-level cognitive propagation layer, and an upper-level temporal evolution layer. The invention uses the Adam optimization algorithm to train the hierarchical graph neural network as a whole, with the learning rate set to 0.001, the batch size to 32, and the number of training rounds to 100. An early stopping strategy is adopted during training, and training is stopped when the validation set loss does not decrease for 10 consecutive rounds.

[0090] In this embodiment of the invention, the bottom graph convolutional layer is the bottom interaction layer. The dynamic multimodal interaction features are adaptively fused on the bottom interaction graph through a multi-scale attention mechanism. This multi-scale attention mechanism includes intra-modal self-attention and inter-modal cross-attention, specifically including:

[0091] Intramodal self-attention is used to capture the temporal dynamic correlation of a single modal feature. For each modal feature, the temporal sequence of the modal feature is input into the self-attention model for intramodal encoding. Based on the dynamic correlation of the modal feature at different time steps, the enhanced feature vector after intramodal encoding is obtained.

[0092] Intermodal cross-attention is used to dynamically adjust the contribution weights of different modalities based on the embedding states of student and question nodes; specifically, such as... Figure 2 As shown, cross-modal attention is used to achieve the following steps:

[0093] Step S201: For the t-th interaction, calculate the attention weight of each modality feature sub-vector based on the dynamic multimodal interaction feature vector;

[0094] Among them, the weight matrix in the attention mechanism , Initialization is performed using the Xavier normal initialization method, with bias terms... , Initialize as a vector of zeros;

[0095] Specifically, the attention weight in step S201 of this embodiment is represented as follows:

[0096]

[0097] In the formula, and These are the embedding representations of the current student node and the question node, respectively. , Represents the weight matrix. , Indicates the bias term; Represents a multimodal interaction feature vector; These represent different modal features, namely, sentiment features. Physiological characteristics Behavioral characteristics and environmental characteristics ;

[0098] Step S202: Perform weighted fusion of the multimodal features according to the attention weights to obtain the fused edge features;

[0099] In step S202, edge features Represented as:

[0100]

[0101] In the formula, Indicates attention weights. Represents dynamic multimodal interaction feature vectors In the diagram, the feature vector corresponding to the i-th mode;

[0102] Step S203: Use the fused edge features as attributes of the answering edge to participate in the message passing process of graph convolution and output the primary embedding of the node.

[0103] Furthermore, in this embodiment of the invention, the directed propagation of the intermediate cognitive propagation layer includes forward propagation and backward propagation; wherein, the prior propagation weight matrix in forward propagation... Self-state weight matrix All are initialized using the Xavier normal initialization method, activation function Choose the ReLU activation function;

[0104] In the forward propagation on the upper-level cognitive graph, the weights are dynamically generated based on the current embedding state of the knowledge point node. The lower the probability of mastering a prior knowledge point, the smaller the forward propagation weight for subsequent knowledge points. Forward propagation is used to simulate knowledge transfer and is represented as follows:

[0105]

[0106] In the formula, This represents the set of all prerequisite knowledge points for knowledge point j. This represents the dependency strength weight between the prerequisite knowledge point p and the subsequent knowledge point j. This represents the pre-propagation weight matrix; This represents the weight matrix of its own state; Indicates the activation function; This indicates the embedded state of the prior knowledge point p before it is updated; This indicates the embedding state of the (l+1)th layer of the knowledge point j to be modified later;

[0107] In hierarchical backpropagation, differentiated feedback coefficients are used according to the type of cognitive dependency relationship. The feedback coefficient corresponding to a strong precedence relationship is greater than that of a weak precedence relationship, and the feedback coefficient corresponding to a weak precedence relationship is greater than that of a correlation relationship.

[0108] Backpropagation is used for learning feedback, and is represented as:

[0109]

[0110] In the formula, This represents the set of all subsequent knowledge points for knowledge point p. This represents the dependency strength weight between prerequisite knowledge point p and subsequent knowledge point j. This represents the embedded state after the prior knowledge point p is updated. This represents the embedding state of the prerequisite knowledge point p at the l-th layer. This represents the feedback coefficient, with a value of 0.3.

[0111] Furthermore, in one implementation of the present invention, the step of updating node embeddings through heterogeneous graph convolution guided by metapaths and aggregating only valid neighborhood information on the metapaths includes:

[0112] Constructing meta-path sets ;

[0113] Each metapath The metapath represents a sequence of semantic relationships between node types; the metapath includes at least one of student-question-student, student-question-knowledge point, question-knowledge point-question, and knowledge point-question-knowledge point.

[0114] For each metapath Based on a multi-level heterogeneous graph, construct the adjacency matrix corresponding to the meta-path. ,in This indicates that there exists a meta-path from node i to node j. The path does not conform to the metapath. Even if a node has a direct edge connection, it will not participate in subsequent message passing as a meta-path neighbor.

[0115] For the target node v, according to the metapath Obtain its meta-path neighbor set, calculate the attention weights between the target node v and its meta-path neighbor u, and combine the multimodal interaction features fused in step S202. As a weighting factor, after being mapped through a linear transformation layer, it is multiplied element-wise with the embeddings of the meta-path neighbors; according to the attention weight, only the node embeddings in the meta-path neighbor set are weighted and aggregated to obtain the meta-path. Semantic-specific node embedding;

[0116] For the target node v, aggregate the semantically specific node embeddings of all metapaths to obtain the final updated node embedding. .

[0117] In one implementation of the present invention, the multi-level heterogeneous graph further includes an upper-level state graph, which constructs a knowledge state evolution trajectory with students as nodes and preset time windows as sequences.

[0118] Specifically, in this embodiment of the invention, the upper-level state diagram uses students as nodes and preset time windows (such as daily or weekly) as sequences to construct the knowledge state evolution trajectory of students in different time windows;

[0119] In this embodiment of the invention, the edges of the upper-level state graph are the knowledge state evolution edges of the trainees, and the edge weights are the rate of change of knowledge states within adjacent time windows.

[0120] The node embedding of the upper-level state graph is obtained by aggregating the knowledge point node embedding of the upper-level cognitive graph based on the student's knowledge mastery. The aggregation method adopts weighted summation, and the weight is the student's mastery probability of each knowledge point.

[0121] In the upper-level state diagram, the time axis is divided into multiple time windows of equal length. In each time window Within the window, a corresponding node embedding snapshot is generated based on the interactive data within that window. Where N is the number of student nodes. This indicates that student n is within the time window. Knowledge state embedding within.

[0122] Furthermore, the hierarchical graph neural network also includes an upper temporal evolution layer, which is used to capture the evolution of knowledge state over time through gated recurrent units or long short-term memory networks, and output dynamically updated node embeddings.

[0123] Specifically, the temporal evolution layer of this invention models the knowledge state evolution trajectory of each student node separately. During the temporal evolution process, the fused multimodal interaction features are used as external input to enhance the accuracy of temporal modeling. Finally, all dynamic nodes in the time space are embedded and arranged in chronological order to obtain the evolution trajectory of the student's knowledge state.

[0124] In this embodiment, the weight parameters of the gated recurrent unit / long short-term memory network in the temporal evolution layer are initialized using He normality, the bias term is initialized to 0, and the hidden layer dimension is consistent with the node embedding dimension.

[0125] In this embodiment, the temporal evolution layer is linked with the upper state graph. The node embedding sequence of the upper state graph is used as the input of the temporal evolution layer. The sequence is encoded by a gated recurrent unit or a long short-term memory network to capture the long-term evolution law of knowledge state. The encoded output is the final node embedding of the upper state graph and is fed back to the lower interaction graph to realize the dynamic update of node embedding.

[0126] In the time-series evolution layer, the time axis is divided into multiple time windows of equal length. Within each time window, a corresponding node embedding snapshot is generated based on the interaction data within the window. The node embedding snapshot sequence is input into a gated recurrent unit or a long short-term memory network to capture the long-term dependencies of knowledge state evolution over time; the output of the gated recurrent unit or the long short-term memory network is the dynamically updated node embedding, which integrates historical knowledge state and current interaction information.

[0127] Please continue to refer to Figure 1 The multidimensional learning information map construction method based on graph neural networks in this embodiment of the invention further includes:

[0128] Step S105: Generate a learning learning map based on the updated node embedding, wherein the learning learning map includes individual knowledge mastery status and group knowledge mastery distribution;

[0129] The learning map provided in this embodiment of the invention includes individual knowledge mastery status, group knowledge mastery distribution, and attribution analysis information. The attribution analysis information is used to demonstrate the contribution of interaction features of different modalities to knowledge mastery status.

[0130] In one implementation, generating a learning information map also includes:

[0131] The contribution of each modality feature subvector to the final mastery probability is quantified by calculating the gradient or SHAP value.

[0132] In the learning learning map, this invention displays the contribution ratio of each modality feature sub-vector in a visual form and generates learning suggestions in text form. The learning suggestions include at least one of adjusting the learning status, optimizing the learning behavior, and improving the learning environment.

[0133] In one implementation of this invention, a multi-dimensional learning graph generated based on dynamically updated node embeddings includes an individual cognitive diagnostic graph, a group commonality analysis graph, and a cognitive dependency verification graph. The individual cognitive diagnostic graph displays the probability of an individual student mastering each knowledge point in the form of a knowledge graph (node ​​colors represent the degree of mastery, from red to green representing low to high), and uses a heatmap to label the contribution of different modal features to the mastery probability. The group commonality analysis graph uses a spectral clustering algorithm to cluster the student group, displaying the commonalities in knowledge structure, learning behavior patterns, and emotional evolution patterns of different student groups. The cognitive dependency verification graph dynamically updates and verifies the cognitive dependencies between knowledge points based on the students' group learning data, forming a dynamically optimized prior knowledge network.

[0134] Meanwhile, this embodiment of the invention obtains the contribution of each modality feature sub-vector to the final mastery probability by calculating gradient or SHAP value, displays it in the form of a circular progress bar in the learning map, and generates learning suggestions in text form.

[0135] Please continue to refer to Figure 3 In another embodiment of the present invention, a multidimensional learning information mapping system based on graph neural networks is provided, comprising the following modules:

[0136] Data acquisition module 301 is used to acquire multimodal learning behavior data of trainees;

[0137] The feature extraction module 302 is used to extract dynamic multimodal interaction features of behavioral data and determine the cognitive dependencies between knowledge points;

[0138] Graph construction module 303 is used to construct a multi-level heterogeneous graph based on three types of nodes: students, test questions, and knowledge points, as well as the cognitive dependency relationship. The multi-level heterogeneous graph includes a bottom-level interaction graph and an upper-level cognitive graph. The bottom-level interaction graph uses the relationship between students and test questions and the relationship between test questions and knowledge points as edges. The upper-level cognitive graph uses knowledge points as nodes and cognitive dependency relationships as directed edges.

[0139] Graph Neural Network Module 304 is used to input multi-level heterogeneous graphs into a hierarchical graph neural network for hierarchical representation learning, including: on the bottom interaction graph, adaptively fusing dynamic multimodal interaction features through a multi-scale attention mechanism, updating node embeddings through heterogeneous graph convolution guided by meta-paths, and aggregating only effective neighborhood information on meta-paths; during message passing, concatenating and encoding the fused multimodal interaction features and node embeddings through an edge node interaction layer, and outputting the updated node embeddings after nonlinear transformation; on the upper cognitive graph, forward propagation is performed according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical backpropagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points;

[0140] The knowledge graph generation module 305 is used to generate a learning knowledge graph based on the updated node embeddings. The learning knowledge graph includes individual knowledge mastery status and group knowledge mastery distribution.

[0141] Please refer to Figure 4 According to another embodiment of the present invention, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the multidimensional learning information map construction method based on graph neural networks as described in the above embodiments.

[0142] According to another embodiment of the present invention, a storage medium is provided, characterized in that the storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the storage medium is located to perform the steps of the multidimensional learning information map construction method based on graph neural networks as described in the above embodiments.

[0143] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0144] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0149] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for constructing a multidimensional learning information map based on graph neural networks, characterized in that, Includes the following steps: Acquire multimodal learning behavior data from learners; Extract dynamic multimodal interaction features from behavioral data and determine the cognitive dependencies between knowledge points; Based on three types of nodes—students, test questions, and knowledge points—and the aforementioned cognitive dependencies, a multi-level heterogeneous graph is constructed. This multi-level heterogeneous graph includes a bottom-level interaction graph and an upper-level cognitive graph. The bottom-level interaction graph uses the relationship between students and test questions, and the relationship between test questions and knowledge points, as edges. The upper-level cognitive graph uses knowledge points as nodes and cognitive dependencies as directed edges. Multi-layered heterogeneous graphs are input into a hierarchical graph neural network for hierarchical representation learning, including: on the bottom interaction graph, dynamic multimodal interaction features are adaptively fused through a multi-scale attention mechanism, and node embeddings are updated through heterogeneous graph convolution guided by meta-paths, aggregating only effective neighborhood information on meta-paths; during message passing, the fused multimodal interaction features and node embeddings are concatenated and encoded through an edge node interaction layer, and the updated node embeddings are output after nonlinear transformation; on the upper cognitive graph, forward propagation is performed according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical backpropagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points. A learning map is generated based on the updated node embedding, which includes the individual knowledge mastery status and the group knowledge mastery distribution.

2. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 1, characterized in that, The dynamic multimodal interaction features include at least two of the following: emotional features, physiological features, behavioral features, and environmental features; The cognitive dependency relationship is a directed weighted relationship, representing the degree of prerequisite dependency between knowledge points.

3. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 2, characterized in that, The multi-scale attention mechanism includes: Intramodal self-attention is used to capture the temporal dynamic correlation of features in a single modality; Intermodal cross-attention is used to dynamically adjust the contribution weights of different modalities based on the embedding states of student nodes and question nodes.

4. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 3, characterized in that, The steps of adaptively fusing the dynamic multimodal interaction features on the underlying interaction graph using a multi-scale attention mechanism include: In intramodal self-attention, for each modal feature, the temporal sequence of the modal feature is input into the self-attention model for intramodal encoding. Based on the dynamic correlation of the modal features at different time steps, an enhanced feature vector is obtained. In intermodal cross-attention, for the t-th interaction, the attention weights of each modal feature sub-vector are calculated based on the dynamic multimodal interaction feature vector, and are expressed as: In the formula, and These are the embedding representations of the current student node and the question node, respectively. , Represents the weight matrix. , Indicates the bias term; Represents a multimodal interaction feature vector; Representing different modal characteristics; Multimodal features are weighted and fused based on attention weights to obtain the fused features. , is represented as: In the formula, Indicates attention weights. Represents dynamic multimodal interaction feature vectors In the graph, the feature vector corresponding to the i-th modality is used as the attribute of the answer edge and participates in the message passing process of graph convolution.

5. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 4, characterized in that, In the forward propagation on the upper-level cognitive graph, the weights are dynamically generated based on the current embedding state of the knowledge point nodes. The forward propagation is represented as follows: In the formula, This represents the set of all prerequisite knowledge points for knowledge point j. This represents the dependency strength weight between the prerequisite knowledge point p and the subsequent knowledge point j. This represents the pre-propagation weight matrix; This represents the weight matrix of its own state; Indicates the activation function; This indicates the embedded state of the prior knowledge point p before it is updated; This indicates the embedding state of the (l+1)th layer of the knowledge point j to be modified later; In hierarchical backpropagation, the feedback coefficient is determined based on the type of cognitive dependency. Backpropagation is represented as follows: In the formula, This represents the set of all subsequent knowledge points for knowledge point p. This represents the dependency strength weight between prerequisite knowledge point p and subsequent knowledge point j. Indicates the feedback coefficient. This represents the embedded state after the prior knowledge point p is updated. This indicates the embedding state of the prior knowledge point p at the l-th layer.

6. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 5, characterized in that, The steps of updating node embeddings via heterogeneous graph convolution guided by metapaths, aggregating only valid neighborhood information along the metapaths, include: Constructing meta-path sets Each metapath This represents a sequence of semantic relationships between node types, for each metapath. Constructing the adjacency matrix corresponding to the meta-path based on a multi-level heterogeneous graph ; For the target node v, according to the metapath Obtain its meta-path neighbor set, calculate the attention weights between the target node v and its meta-path neighbor u, and then fuse the multimodal interaction features. As a weighting factor, after being mapped through a linear transformation layer, it is multiplied element-wise with the embeddings of the meta-path neighbors; according to the attention weight, only the node embeddings in the meta-path neighbor set are weighted and aggregated to obtain the meta-path. Semantic-specific node embedding; For the target node v, aggregate the semantically specific node embeddings of all metapaths to obtain the final updated node embedding. .

7. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 6, characterized in that, The multi-level heterogeneous graph also includes an upper-level state graph, which uses students as nodes and preset time windows as sequences to construct a knowledge state evolution trajectory, including: Divide the timeline into multiple time windows of equal length. In each time window Within the window, a corresponding node embedding snapshot is generated based on the interactive data within that window. Where N is the number of student nodes. This indicates that student n is within the time window. Knowledge state embedding within.

8. The method for constructing a multidimensional learning information map based on a graph neural network according to claim 7, characterized in that, Hierarchical graph neural networks also include a temporal evolution layer, which is used to capture the patterns of knowledge state evolution over time; In the temporal evolution layer, the time axis is divided into multiple time windows of equal length. Within each time window, a corresponding node embedding snapshot is generated based on the interaction data within the window. The node embedding snapshot sequence is input into the Long Short-Term Memory network, and the dynamically updated node embedding is output.

9. A construction system for implementing the multidimensional learning information map construction method based on graph neural networks as described in any one of claims 1 to 8, characterized in that, Includes the following modules: The data acquisition module is used to acquire multimodal learning behavior data of trainees; The feature extraction module is used to extract dynamic multimodal interaction features from behavioral data and determine the cognitive dependencies between knowledge points. The graph construction module is used to construct a multi-level heterogeneous graph based on three types of nodes: students, test questions, and knowledge points, as well as the cognitive dependency relationship. The multi-level heterogeneous graph includes a bottom-level interaction graph and an upper-level cognitive graph. The bottom-level interaction graph uses the relationship between students and test questions and the relationship between test questions and knowledge points as edges. The upper-level cognitive graph uses knowledge points as nodes and cognitive dependency relationships as directed edges. The graph neural network module is used to input multi-level heterogeneous graphs into a hierarchical graph neural network for hierarchical representation learning. This includes: on the bottom interaction graph, adaptively fusing dynamic multimodal interaction features through a multi-scale attention mechanism, updating node embeddings through heterogeneous graph convolution guided by meta-paths, and aggregating only effective neighborhood information on meta-paths; during message passing, concatenating and encoding the fused multimodal interaction features and node embeddings through an edge node interaction layer, and outputting the updated node embeddings after nonlinear transformation; on the upper cognitive graph, forward propagation is performed according to the direction and weight of cognitive dependencies to simulate the migration process of knowledge from prior knowledge points to subsequent knowledge points, and a hierarchical backpropagation mechanism is set up to update the status of prior knowledge points based on the mastery of subsequent knowledge points. The knowledge graph generation module is used to generate a learning knowledge graph based on the updated node embeddings. The learning knowledge graph includes individual knowledge mastery status and group knowledge mastery distribution.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to perform the steps of the multidimensional learning information map construction method based on graph neural networks as described in any one of claims 1 to 8.

Images