Programming score prediction method based on large language model and heterogeneous graph neural network

Abstract

The invention provides a programming score prediction method based on a large language model and a heterogeneous graph neural network, and the method comprises the steps: data collection: systematically collecting log information from an online platform, and storing student data, question data and submitted data generated by students on different questions into corresponding databases, subsequent efficient retrieval and use are facilitated; performing feature enhancement based on LLM: enriching representation of entity nodes by using features generated by the LLM; construction of a student question solving heterogeneous graph: constructing question doing records of the students in the online platform into a heterogeneous graph containing three node types of the students, submission records and questions; the three different types of nodes are associated with the edges belonging to the two types through submission; and student programming score prediction: predicting the programming score of the student by using an HGT model according to the heterogeneous attribute of the question doing network of the student. The accuracy of programming performance prediction is improved.

Description

Technical Field

[0001] This application relates to the technical field of performance prediction, and particularly to a programming performance prediction method based on large language models and heterogeneous graph neural networks. Background Art

[0002] During the online programming learning process, it is crucial to identify students with learning difficulties early to prevent learning failures, and predicting students' future programming performance is the main method for early identification of programming learning difficulties.

[0003] Existing methods for predicting students' performance can be divided into three categories: The first is the method based on static models. Static models refer to traditional machine learning or deep learning methods. Static models analyze students' historical learning behaviors and extract static information from them to predict students' future academic performance. The second is the method based on sequential models. Sequential models refer to a series of models represented by knowledge tracing and its variants. These models focus on the sequential relationships in learning materials, allow for dynamic knowledge updates, and can better capture the temporal changes in students' knowledge and the relationships with learning materials. The third is the method based on graph models. With the development of graph neural networks, graph structures provide an effective solution for predicting students' performance. It not only enhances the expression of process information but also gets rid of the limitations of time series. For example, the attention-based GCN model can analyze and train the complex knowledge evolution graph structure data presented in students' data, and then predict students' performance in future courses.

[0004] Although the above methods can, to a certain extent, perform intelligent scoring on students' programming performance, there are still certain drawbacks:

[0005] Traditional static models mainly focus on the static relationships between students' characteristics and question characteristics, and do not fully consider the process information of students answering questions. Therefore, the applicability of these static models in predicting students' programming performance is limited.

[0006] Sequential models represented by knowledge tracing can handle the behavioral sequences of answering questions, but they usually assume a fixed answering order. In the actual application scenarios of online programming platforms, students usually have the freedom to choose the order of answering questions, and this characteristic challenges the effectiveness of sequential models in programming performance prediction.

[0007] Although there have been attempts to use graph structure data to represent the problem-solving process, there is still a lack of effective representation of the programming process (especially the code submitted by students). Summary of the Invention

[0008] This application provides a programming performance prediction method based on large language models and heterogeneous graph neural networks, which can be used to solve the technical problem of insufficient utilization of existing information in the prior art.

[0009] This application provides a programming performance prediction method based on large language models and heterogeneous graph neural networks. The method includes:

[0010] Step 1, data collection: Collect log information from an online platform, and store student data, question data, and submission data generated by students on different questions in a corresponding database, providing basic data support for subsequent processing and student performance prediction;

[0011] Step 2, feature enhancement based on LLM: In the process of feature construction, select the student's problem-solving process data from the problem-solving records as entity features, and use the LLM to mine potential features to enhance the information representation of the entities;

[0012] Step 3, construction of the student problem-solving heterogeneous graph: Construct the problem-solving records of students on the online platform into a heterogeneous graph containing three node types: students, submission records, and questions; The three different types of nodes are interconnected through two types of edges: submission and belonging, restoring the complex relationships between different entities; The submission node is associated with the corresponding student node and question node, and can be connected to other submission nodes through students or questions, thus accurately reflecting the student's answering process and behavior patterns;

[0013] Step 4, student programming performance prediction: For the heterogeneous attributes of the student problem-solving network, use the HGT model to classify the submission nodes and predict the student's programming performance.

[0014] In order to fully exploit the huge educational data generated by current online programming platforms, this application proposes a student programming performance prediction method based on LLM feature enhancement and graph neural networks: HGT_Code_LLM, to achieve efficient utilization of this massive data. The main advantages of this technical solution are as follows:

[0015] (1) Aiming at the lack of process information in current student performance prediction methods, innovatively incorporate the code submitted by students into the programming performance prediction task, using it as the interaction medium between students and questions, providing a richer representation of process information for students' programming performance and improving the accuracy of programming performance prediction.

[0016] (2) Aiming at the problem that traditional methods cannot represent complex learning processes, heterogeneous network graphs are used to represent the complex relationships involved in the process of students answering questions, enabling it to capture diverse interactions and complex relationships in the programming practice process and form a complete representation of learning data. At the same time, the HGT model, which is specifically designed for heterogeneous graph data processing, is selected to ensure the performance of programming performance prediction.

[0017] (3) Aiming at the deficiency of feature representation in the current text attribute graph, this application adopts a node feature enhancement representation strategy of "LM + LLM" to construct the feature embeddings of different types of nodes. Based on the analysis and reasoning capabilities of the LLM, node features can fuse the shallow features from the surface of historical data and the deep features contained within historical data. This LLM-based feature enhancement method will further improve the performance of the GNN model in predicting students' programming performance.

[0018] (4) This application constructs a new student programming performance prediction technology, HGT_Code_LLM, in one step. By converting the student performance prediction task into a node classification task in the field of graph neural networks, the high-performance heterogeneous graph neural network model HGT is selected as the basic model, and the performance of the HGT model in the performance prediction task is enhanced by adding code mediators and deep features extracted by the LLM.

[0019] Through experimental verification, the performance of HGT_Code_LLM on real data is better than that of traditional machine learning methods and other graph neural network baseline models. Brief Description of the Drawings

[0020] Figure 1 It is the technical flow chart of HGT_Code_LLM provided by the embodiment of this application;

[0021] Figure 2 It is the student problem-solving flow chart provided by the embodiment of this application

[0022] Figure 3 It is the programming performance prediction model diagram based on HGT provided by the embodiment of this application;

[0023] Figure 4 It is the ablation experiment result diagram provided by the embodiment of this application. Detailed Embodiment

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following will further describe the embodiments of this application in detail with reference to the accompanying drawings.

[0025] To accurately reflect the complex relationships among multiple students, their submitted code, and different problems, this method uses graph-structured data to represent the students' problem-solving processes. The code submitted by students is used as independent submission nodes to connect student nodes and problem nodes, constructing a student problem-solving graph. At the same time, a Graph Neural Network (GNN) model is used as the main approach for predicting students' programming performance, and two methods are adopted to enhance the model's performance. Through the calculation of mutual attention, message passing, and message aggregation, the embedding of the submission nodes containing graph-level context information is finally obtained, and the prediction of students' programming performance is achieved.

[0026] First, the embodiments of the present application will be introduced in conjunction with the accompanying drawings.

[0027] Step 1, data collection: Collect log information from the online platform, and store the student data, problem data, and submission data generated by students on different problems in the corresponding database to provide basic data support for subsequent processing and student performance prediction.

[0028] Step 2, feature enhancement based on LLM: In the process of feature construction, select the student problem-solving process data from the problem-solving records as entity features, and use LLM to mine potential features to enhance the information representation of entities.

[0029] Step 21: The feature sources of the problem nodes include historical data and feature representations enhanced by LLM; historical data records are utilized, and three statistical features are selected from the student problem-solving process records to comprehensively describe the students' academic levels; the three features are: the total number of submissions, the problems done, and the average correct rate.

[0030] Step 211: Count the total number of submissions by the student within the current problem scope.

[0031] Step 212: Use an N-dimensional vector represented by one-hot encoding to represent the problems done by the student, where N is the number of problems within the current problem scope; each dimension corresponds to a problem, and if the student has submitted on the current problem, the value of the corresponding dimension is 1, otherwise it is 0.

[0032] Step 213: Calculate the average of the correct rates of the student on different problems as the average correct rate.

[0033] Step 214: Considering the statistical items of the total number of submissions, the problems done, and the average correct rate comprehensively, construct the feature representation of the student as an N+2-dimensional vector, where N represents the number of problems; such a feature representation can more comprehensively describe the students' academic levels and provide strong support for subsequent prediction of students' programming performance; the feature representation of the student node is shown in Table 1.

[0034] Table 1: Student Node Feature Table

[0035]

[0036] Step 22: The feature sources of the question nodes include historical data and feature representations enhanced by the LLM; the features selected by the question nodes from historical data include: question description, question difficulty, and pass rate; in addition, use the large language model to mine the knowledge points examined in each question and their proportions in the current question;

[0037] Step 221: Collect the specific semantic information of the question itself, including the description of the question, input and output requirements, and the difficulty level predefined by the administrators or teachers of the online programming platform when creating the question;

[0038] Step 222: Calculate the proportion of the number of submissions that passed the assessment in the current question to the total number of submissions as the pass rate of the question;

[0039] Step 223: Use the large language model to mine the knowledge points examined in the question and their proportions. The prompt template of the large model is as follows:

[0040] Input: [Question description text, input and output requirements]

[0041] Task objective: Your task is to extract the key knowledge points from programming questions for primary and secondary school programming education;

[0042] Special requirements:

[0043] Please provide your answer in the following JSON format:

[0044]

[0045] That is:

[0046]

[0047]

[0048] In the large language model, the name of each knowledge point is a word or a phrase with less than five words; the percentage sum of all keywords is 1; and only the first five knowledge points are extracted to avoid excessive knowledge point extraction;

[0049] Step 224: Use the prompt template in Step 223 to generate the knowledge points involved in each question and their proportions in the question through API requests;

[0050] Step 225: Use SBERT to generate semantic embeddings of the question descriptions. SBERT is a pre-trained sentence embedding model that can map text sentences to vector representations in a high-dimensional space to capture their semantic information. Incorporate the question difficulty and pass rate as additional features and fuse them with the semantic embeddings of the question descriptions to obtain historical features;

[0051] Step 226: Use SBERT to generate semantic embeddings for each knowledge point. Calculate the weighted average of the semantic embeddings of the knowledge points according to the weights as the knowledge embedding of the question node; Concatenate the semantic embedding and the knowledge embedding of the question node as the final feature representation of the question node; The feature representation of the question node is shown in Table 2;

[0052] Table 2: Question Node Feature Table

[0053]

[0054] Step 23: Use the code submitted by students in the historical data and the code syntax and semantic features extracted by the LLM to construct the features of the student submission node;

[0055] Step 231, CodeBERT is a pre-trained model specifically for code, optimized and trained on a large amount of code text based on the BERT architecture, and shows excellent performance in tasks such as code understanding and code generation. Generate semantic embeddings for each piece of student code through CodeBert;

[0056] Step 232: Use the large language model to further extract semantic and syntactic features from the student code to enhance the feature representation of the submission node. The prompt template used is as follows:

[0057] Input: [Specific code obtained from student submission data]

[0058] Task objective:

[0059] Your task is to extract syntactic and semantic features from the C++ code submitted by primary and middle school students;

[0060] Syntactic features (grammatical feature): Include the following four aspects:

[0061] Keywords and identifiers: Important keywords and identifiers that appear in the code.

[0062] Sentence structure: Important sentence structures that appear in the code.

[0063] Data type: Data types used in the code.

[0064] Code complexity: Analyze the time and space complexity of the code.

[0065] Semantic feature: Includes the following four aspects:

[0066] Logical process: Summarize the main logical processes in the code.

[0067] Data operation: The main data operations involved in the code.

[0068] Algorithms and functions: The main algorithms or functions used in the code and their meanings.

[0069] Coding style: The coding style of the student reflected in the code.

[0070] Special requirements:

[0071] Please provide the answer in the following JSON format:

[0072]

[0073] That is:

[0074]

[0075]

[0076] Among them, the answer to each sub-item must be a single word or a phrase of no more than ten words; all sub-items must generate answers, and no content can be skipped or added;

[0077] Step 233: Use the hint template in Step 232 to generate the feature and syntax features in the student code through API requests;

[0078] Step 234: Use SBERT to vectorize the extracted syntax features and semantic features respectively, and then add the two as the code enhancement features generated by the large model; Concatenate the features generated by the student code itself with the enhanced features extracted by the large model as the final feature representation of the submission node; The feature representation of the submission node is shown in Table 3;

[0079] Table 3: Submission Node Feature Table

[0080]

[0081] Step 3: Construction of the heterogeneous graph of students' problem-solving: This method adopts a more fine-grained approach to construct a heterogeneous graph containing three node types: students, submission records, and questions, based on the problem-solving records of students in the online platform; the three different types of nodes are interconnected through two types of edges: submission and belonging; thus, the submission nodes can be associated with the corresponding student nodes and question nodes, and can be related to other submission nodes through students or questions, accurately reflecting the students' answering processes and behavior patterns, such as Figure 2 as shown

[0082] Step 31: Use the "torch_geometric" package to create a heterogeneous graph of students' problem-solving processes, create student nodes, question nodes, and submission nodes, and assign the node features created in Step 2 to the corresponding nodes;

[0083] Step 32: According to the corresponding relationships in the dataset, establish the creation edge relationship between students and submissions, and the belonging edge relationship between submissions and questions;

[0084] Step 33: Calculate the scores of the submission nodes and add corresponding classification labels to the submission nodes according to the scores;

[0085] Step 34: Save the heterogeneous graph of students' problem-solving as the input for the subsequent prediction model.

[0086] Step 4: Prediction of students' programming scores: For the heterogeneous properties of the students' problem-solving network, use the HGT model dedicated to processing heterogeneous graphs to classify the submission nodes, and then predict the students' programming scores. The structure of the HGT model Figure 4 is shown

[0087] Step 41: For the heterogeneous subgraph of students' problem-solving in the constructed graph during each sampling, the HGT model extracts all linked node pairs, where the student node s and the question node p are linked to the submission node c through the edge e;

[0088] Step 42: HGT is used to aggregate information from students and questions to obtain the context representation of the submission node c. This information aggregation process is decomposed into three parts: heterogeneous mutual attention, heterogeneous message passing, and specific target information aggregation;

[0089] Step 421: Heterogeneous mutual attention calculation:

[0090] For a given submission node c and all its neighbor nodes n∈N(c), the neighbor nodes include student and question nodes. According to their meta-relation triples To calculate mutual attention; HGT refers to the architecture design of the Transformer model, maps the target node c to a query vector, maps the source node n to a key vector, and calculates the dot product of the two as the attention score; the multi-head attention calculation method used by HGT is as follows:

[0091]

[0092] For the i-th attention head ATT-head i (n, e, c), use a linear transformation to map the source node of type τ(n) to the i-th key vector K i (n), also use a linear transformation to map the target node c to the i-th query vector Q i (c); then use different weight matrices according to the type of φ(e) Calculate the similarity between the query vector Q i (c) and the key vector to capture the semantic relationship between different types of nodes in the heterogeneous graph.

[0093] Step 422: After completing the calculation of mutual attention, transfer the information of the student and question nodes to the submission node; during the transfer process, it is also necessary to use the meta-relationship between the edges to collect the messages of different types of nodes; the message passing calculation method used by HGT is as follows:

[0094]

[0095] For the i-th message head MSG-head i (n, e, c), project the source node of type τ(n) after linear transformation into the i-th message vector, and then use The weight matrix combines the messages on different types of edges, and finally connects h message heads to obtain the message representation MSG-head i (n, e, c);

[0096] Step 423: After completing the calculation of heterogeneous multi-head attention and messages, aggregate them from the student and question nodes to the submission node; use heterogeneous mutual attention as the weight for aggregating messages of different types of nodes to obtain the updated vector representation of the submission node, and the calculation method is as follows:

[0097]

[0098] Use a linear transformation to map the vector of the submission node c back to the original feature distribution, and perform a residual connection with the feature representation of the submission node output by the previous layer, and finally obtain the output H of the l-th HGT of the submission node c (l) [c], and the calculation method is as follows:

[0099]

[0100] Step 43: Performance Prediction: Through the HGT module of layer L, the submission node obtains information about different students, other submissions, and different question nodes; the context representation of the submission node is mapped into an N-dimensional vector after a linear transformation, where N represents the number of performance level labels. After Softmax calculation, the probability that the current submission node belongs to different performance levels is obtained, and then the prediction result of the student's programming performance is obtained.

[0101] Experimental Verification:

[0102] To verify the effectiveness of the technology proposed in this application, this application conducted comparative experiments and ablation experiments on the collected real dataset. In the comparative experiment, HGT_Code_LLM was compared with other graph neural network models and traditional machine learning methods. All models used Micro-F1 and Macro-F1 as evaluation metrics to comprehensively measure the prediction performance of the models. The results of the comparative experiment are shown in Table IV. To further verify the effectiveness of each module of HGT_Code_LLM in student performance prediction, this study conducted ablation experiments to compare the model performance under different conditions. Comparing the model performance in three different states, namely, only using the HGT model, the HGT model after integrating the student code features into the submission node, and the HGT model after integrating both the student code and the features of the large model, the evaluation metrics also used Micro-F1 and Macro-F1. The results of the ablation experiment are as Figure 4 shown.

[0103] Table IV: Results of Comparative Experiment

[0104] Model Name Micro-F1 Macro-F1 HGT_Code_LLM 0.806 0.727 GAT 0.644 0.567 R-GCN 0.684 0.591 HetGNN 0.702 0.617 LR 0.589 0.513 SVM 0.576 0.526

[0105] The results of the comparative experiments show that, compared with the traditional machine learning-based methods for predicting student performance, HGT_Code_LLM can achieve a performance improvement of about 20% in the student programming prediction task and also shows a performance advantage of more than 10% when compared with other graph neural network baseline models. The results of the ablation experiments show that with the addition of student code and LLM enhanced features, the HGT model demonstrates more powerful performance in downstream tasks. HGT_Code incorporates code features into the submission nodes. The real code feature representation can not only reflect the student's programming ability but also serve as an important basis for generating grade levels on different questions. The submission nodes play an important role in associating student and question nodes. Therefore, the addition of real code features enables the model to learn the procedural information of students' problem-solving processes, and the resulting context representation of the submission nodes will also be more in line with the actual situation, making the prediction of grade levels on the submission nodes more accurate compared to the basic HGT model. HGT_Code_LLM enables the student problem-solving network to contain deeper context information through the LLM-to-LM feature enhancement representation method. These information are not available in the historical data records themselves. The LLM-enhanced node feature representation also provides more learning space for the model during the training process, enabling the model to better capture the structural information and context representation in the graph, and further improving the effect of predicting students' programming grades based on HGT_Code.

[0106] This application innovatively incorporates the code submitted by students into the programming performance prediction task, using it as an interaction medium between students and questions, providing a richer procedural information representation for students' programming performance and improving the accuracy of programming performance prediction.

[0107] This application uses a heterogeneous network graph to represent the complex relationships involved in the process of students answering questions, enabling it to capture diverse interactions and complex relationships during the programming practice process and form a complete representation of learning data. At the same time, the HGT model dedicated to heterogeneous graph data processing is selected to ensure the performance of programming performance prediction.

[0108] This application adopts the "LM + LLM" node feature enhancement representation strategy to construct the feature embeddings of different types of nodes. Based on the analysis and reasoning capabilities of the LLM, the node features can fuse the shallow features from the surface of historical data and the deep features contained within the historical data. This LLM-based feature enhancement method will further improve the performance of the GNN model in predicting students' programming performance.

[0109] The above-described embodiments of this application do not constitute a limitation on the protection scope of this application.

Claims

1. A programming performance prediction method based on a large language model and a heterogeneous graph neural network, characterized in that: The method comprises: Step 1, data collection: collect log information from the online platform, store student data, question data and student submission data on different questions in the corresponding database, and provide basic data support for subsequent processing and student score prediction; Step 2: Feature enhancement based on LLM: In the process of feature construction, the student's test-taking process data is selected from the test-taking records as entity features, and LLM is used to mine potential features to enhance the information representation of the entity; Step 3: Construct a heterogeneous graph of student problem solving: Construct the student's problem solving records in the online platform into a heterogeneous graph containing three types of nodes: students, submission records, and questions. The three different types of nodes are interconnected through submissions and edges belonging to two types, restoring the complex relationship between different entities. The submission node is associated with the corresponding student node and question node, and can be connected to other submission nodes through students or questions. Step 4: Prediction of students’ programming scores: Based on the heterogeneous properties of the students’ problem-solving network, the HGT model is used to classify the submission nodes and predict the students’ programming scores.

2. The method according to claim 1, characterized in that Step 2: Feature enhancement based on LLM, including: Step 21: The feature sources of the question node include historical data and LLM enhanced feature representation; three statistical features are selected from the student's question-solving process records to comprehensively describe the student's academic level; the three features are: total submission number, questions done, and average correct rate; Step 22 uses a large language model to mine the knowledge points tested in each question and their proportion in the current question; Step 23: Use the student submitted code in the historical data and the code syntax and semantic features extracted by LLM to construct the features of the student submitted node.

3. The method according to claim 2, characterized in that Step 21: Using historical data records, three statistical features were selected to comprehensively describe the students’ academic level, including: Step 211: Count the total number of submissions by students within the current topic range; Step 212: Use an N-dimensional vector represented by one-hot encoding to represent the questions that the student has done, where N is the number of questions in the current question range; each dimension corresponds to a question, and if the student has submitted a question on the current question, the value of the corresponding dimension is 1, otherwise it is 0; Step 213: Calculate the average of the correct rates of students on different questions as the average correct rate; Step 214: Considering the total number of submissions, the questions done and the average correct rate, the feature representation of the student is constructed as an N+2-dimensional vector, where N represents the number of questions; the feature representation of the student node is shown in Table 1; Table 1: Student node feature table Table 1 is the student node feature table.

4. The method according to claim 2, characterized in that: Step 22: The feature sources of the question node include historical data and LLM-enhanced feature representation; the features selected from the historical data by the question node include: question description, question difficulty and pass rate; in addition, the large language model is used to generate the knowledge points involved in each question and their proportion in the current question; including: Step 221: Collecting specific semantic information of the question itself, including the description of the question, input and output requirements, and the difficulty level of the question pre-defined by the administrator or teacher of the online programming platform when creating the question; Step 222: Calculate the ratio of the number of submissions that pass the assessment to the total number of submissions in the current question as the passing rate of the question; Step 223: Use the large language model to mine the knowledge points tested in the question and their proportions. The prompt template using the large model is as follows: Input: [Topic description text, input and output requirements] Task Objective: Your task is to extract key knowledge points from programming problems for primary and secondary school programming education; Special Requests: Please provide your answer in the following JSON format: In the large language model, each knowledge point name is a word or a phrase of less than five words; the sum of the percentages of all keywords is 1; and only the first five knowledge points are extracted to avoid extracting too many knowledge points; Step 224: using the prompt template in step 223, generating the knowledge points involved in each question and the proportion they occupy in the question through API request; Step 225: Use SBERT to generate a semantic embedding of the question description, and use the question difficulty and pass rate as additional features to fuse with the semantic embedding of the question description to obtain historical features; Step 226: Use SBERT to generate the semantic embedding of each knowledge point, and calculate the weighted average of the semantic embedding of the knowledge point according to the weight as the knowledge embedding of the question node; concatenate the semantic embedding and knowledge embedding of the question node as the final feature representation of the question node; the feature representation of the question node is shown in Table 2; Table 2: Question node feature table Table 2 is the feature table of the topic nodes.

5. The method according to claim 2, characterized in that: Step 23: Use the code submitted by students in the historical data and the code syntax and semantic features extracted by LLM to construct the features of the student submission node; including: Step 231, generate semantic embedding of each student code through CodeBert; Step 232: Use the large language model to further extract semantic features and grammatical features in the student code to enhance the feature representation of the submission node. The prompt template used is as follows: Input: [Specific code obtained from student submissions] Mission objectives: Your task is to extract grammatical and semantic features from the C++ codes submitted by primary and secondary school students; Grammatical features: including the following four aspects: Keywords and identifiers: important keywords and identifiers that appear in the code; Statement structure: important statement structures appearing in the code; Data type: the data type used in the code; Code complexity: Analyze the time complexity and space complexity of the code; Semantic features: including the following four aspects: Logical process: summarize the main logical process in the code; Data operations: the main data operations involved in the code; Algorithms and functions: The main algorithms or functions used in the code and their significance; Coding style: The student's coding style as reflected in the code. Special Requests: Please provide your answer in the following JSON format: The answer to each sub-item must be a word or a phrase of no more than ten words. All sub-items must generate answers and cannot be skipped or added. Step 233: using the prompt template in step 232, generating features and grammatical features in the student code by way of API request; Step 234: Use SBERT to vectorize the extracted grammatical features and semantic features respectively, and then add the two together as the code enhancement features generated by the large model; concatenate the features generated by the student code itself and the enhancement features extracted by the large model as the final feature representation of the submission node; the feature representation of the submission node is shown in Table 3; Table 3: Submit node feature table Table 3 is the submission node feature table.

6. The method according to claim 1, characterized in that Step 3: Construction of heterogeneous graphs for students to solve problems: including: Step 31: Use the "torch_geometric" package to create a heterogeneous graph of the student's problem-solving process, create student nodes, problem nodes, and submission nodes, and assign the created node features to the corresponding nodes; Step 32: According to the corresponding relationship in the data set, establish a create edge relationship between student and submission, and a belong to edge relationship between submission and question; Step 33: Calculate the score of the submitted node, and add a corresponding classification label to the submitted node according to the score; Step 34: Save the heterogeneous graph of student problem solving as input for subsequent prediction models.

7. The method according to claim 1, characterized in that Step 4: Prediction of students’ programming scores: Based on the heterogeneous properties of students’ problem-solving networks, the HGT model is used to predict students’ programming scores, including: Step 41: For each sampled heterogeneous subgraph of student questions, the HGT model extracts all linked node pairs, where the student node s and the question node p are linked to the submission node c via the edge e; Step 42: HGT is used to aggregate information from students and questions to obtain the contextual representation of the submission node c. This information aggregation process is decomposed into three parts: heterogeneous mutual attention, heterogeneous message passing, and target-specific information aggregation; Step 421: Heterogeneous mutual attention calculation: For a given submission node c and all its neighbor nodes n∈N(c), the neighbor nodes include students and question nodes, according to their meta-relation triples To calculate mutual attention; map the target node c to the query vector, map the source node n to the key vector, and calculate the dot product of the two as the attention score; the multi-head attention calculation method used by HGT is as follows: For the i-th attention head ATT-head i (n,e,c), using a linear transformation to map the τ(n) type source node to the i-th key vector K i (n), also use linear transformation to map the target node c to the i-th query vector Q i (c); then use different weight matrices according to the type of φ(e) Calculate the query vector Q i (c) Similarity with the key vector; Step 422: After the calculation of mutual attention is completed, the information of the student and question nodes is transmitted to the submission node. In the transmission process, the meta-relationship between edges is also needed to collect messages of different types of nodes. The message transmission calculation method used by HGT is as follows: For the i-th message header MSG-head i (n,e,c), project the τ(n) type source node into the i-th message vector after linear transformation, and then use The weight matrix merges messages on different types of edges, and finally concatenates h message headers to obtain the message representation MSG-head for each node pair. i (n,e,c); Step 423: After the calculation of heterogeneous multi-head attention and messages is completed, they are aggregated from the student and question nodes to the submission node; heterogeneous mutual attention is used as the weight for the aggregation of messages of different types of nodes to obtain the vector representation of the updated submission node, which is calculated as follows: Use linear transformation to map the vector of the submitted node c back to the original feature distribution, and perform residual connection with the submitted node feature representation output by the previous layer, and finally obtain the output H of the lth HGT of the submitted node c (l) [c], calculated as follows: Step 43: Score prediction: Through the HGT module of the L layer, the submission node obtains the information of different students, other submissions and different question nodes; the context representation of the submission node is mapped to an N-dimensional vector after linear transformation, where N represents the number of performance level labels. After Softmax calculation, the probability that the current submission node belongs to different performance levels is obtained, and then the prediction result of the student's programming performance is obtained.

Images