A programming education intelligent management method and system based on neural network

Abstract

The present invention discloses a programming education intelligent management method and system based on neural network, the method comprising: extracting first semantic information of student source code text based on a first neural network; extracting second semantic information of the abstract syntax tree of the student source code based on a second neural network; extracting third semantic information of the variable program dependency graph of the student source code based on a third neural network; fusing the first semantic information, the second semantic information, and the third semantic information using a multimodal feature fusion model based on Transformer to obtain fused semantic features; inputting the fused semantic features into an output layer to obtain a student source code quality analysis result. The present invention utilizes multiple data types of source code to fully extract the semantic information of the source code, thereby more comprehensively capturing the semantic information of the source code.

Description

Technical Field

[0001] The present invention relates to the technical field of source code analysis, and in particular to a programming education intelligent management method and system based on neural network. Background Art

[0002] Due to the demand for information and communication technology (ICT) in modern society, the importance of programming is increasing. Artificial intelligence can provide more dynamic support, such as code search, code suggestions, algorithm identification and classification, etc. In this context, people are trying to introduce AI models and neural networks into smart programming education.

[0003] Chinese patent document CN201911338011.X provides "An artificial intelligence teaching system based on graphic programming", which includes a programming terminal, a cloud server, and an embedded device. The programming terminal transmits the basic parameters of the neural network to the cloud server. The cloud server receives the basic parameters of the neural network, processes them, and generates a trained network model. The embedded device receives and runs the trained network model through the Internet to obtain the corresponding operation results. The invention simplifies complex programming to achieve the purpose of quickly building a neural network, quickly running the model, and viewing the results of the neural network operation. Young students do not need to write or only need to write less program code to complete complex program design, thereby improving learning efficiency and learning fun. However, the model cannot complete the evaluation of the content of the code written by students, and cannot accurately provide students with high-quality programming education. Summary of the invention

[0004] In view of the problems existing in the above-mentioned prior art, the present invention provides a programming education intelligent management method and system based on neural network, which can accurately extract the semantic information of the student's code by extracting the text semantic information of the source code, the abstract syntax tree semantic information and the variable program dependency graph semantic information respectively and fusing the features, so as to evaluate and understand the code written by the students and evaluate the students' mastery of each knowledge point. The technical solution is as follows:

[0005] In a first aspect, a programming education intelligent management method based on a neural network is provided, including a source code semantic understanding model, wherein a data processing process of the source code semantic understanding model includes the following steps:

[0006] Extracting first semantic information of the student's source code text based on the first neural network;

[0007] Extracting second semantic information of the abstract syntax tree of the student's source code based on the second neural network;

[0008] Extracting third semantic information of a variable program dependency graph of a student's source code based on a third neural network;

[0009] The first semantic information, the second semantic information, and the third semantic information are fused using a Transformer-based multimodal feature fusion model to obtain a fused semantic feature;

[0010] The fused semantic features are input into the output layer to obtain the student source code quality analysis result.

[0011] In some embodiments, the programming education intelligent management method further includes:

[0012] Based on the programming question keywords input by the student user, the natural language semantic understanding model is input to obtain the user's question semantic features;

[0013] Based on the source code data stored in the database, a source code semantic understanding model is input to obtain source code semantic features;

[0014] The answer to the user's input question is obtained by matching the user's question semantic features with the source code semantic features.

[0015] In some implementations, extracting first semantic information of the student source code text based on the first neural network includes:

[0016] For the student source code text data, extract key character information and record it as token. Token includes keywords, variable names, and operators.

[0017] The extracted text tokens are embedded into word vectors using the neural bag-of-words model NBOW;

[0018] Input the token word vector of the source code text into BiLSTM to obtain the semantic information of the code function, which is recorded as the first semantic information x t .

[0019] In some implementations, extracting the second semantic information of the abstract syntax tree of the student source code based on the second neural network includes:

[0020] Extracting a basic path set vector for each function module in the abstract syntax tree of the source code;

[0021] The basic path set is input into the neural network model to automatically generate functional semantic features and obtain the second semantic information x p .

[0022] In some implementations, extracting a basic path set vector from each function module in an abstract syntax tree of the source code comprises the following steps:

[0023] (1) Obtain the control flow graph CFG of the function;

[0024] (2) Starting from the entry node of the function, for the entry node, obtain an edge starting from the entry node and a tail node connected to the edge;

[0025] (3) taking the tail node as a new target analysis node, and obtaining all edges starting from the target analysis node;

[0026] (4) If the target analysis node does not have an edge starting from the target analysis node, proceed to step (5); otherwise, proceed to step (6);

[0027] (5) The target analysis node is determined to be a boundary node; for the boundary node, a sequence of all nodes from the entry node to the boundary node forms a basic path;

[0028] (6) Obtain the tail node of each edge starting from the target analysis node. If all tail nodes have been visited, proceed to step (5); otherwise, proceed to step (7);

[0029] (7) Select one of the unvisited tail nodes as a new target analysis node and go to step (3).

[0030] In some embodiments, the step of inputting the basic path set into the neural network model to automatically generate functional semantic features and obtain the second semantic information includes:

[0031] Let the basic path set P = {p1,...,p i ...,p n}, n is the path p in the path set i The total number of , i = 1, 2, ..., n;

[0032] Encode the abstract syntax tree node corresponding to the path node into a token vector;

[0033] Use the word embedding algorithm to map the token vector to a real-valued vector to obtain a path vector c with a dimension of d. i ;

[0034] Use the n*n dimensional weight matrix W to transform the n path vectors c with dimension d into i Weighted fusion and activation by tanh function to obtain n combined context vectors with dimension d tanh is the hyperbolic tangent function; W is the learnable fully connected layer weight matrix;

[0035] The n combined context vectors corresponding to the n path vectors Input the attention layer, and output the attention score vector based on the attention vector a of dimension n. The attention vector a is a learnable parameter obtained during the network model training.

[0036] The attention score vector output by the attention layer is normalized into the attention weight vector α through the softmax function;

[0037] Based on the attention weight vector α, the n combined context vectors are fused to obtain the semantic feature vector x corresponding to the basic path set p .

[0038] In some implementations, the method for calculating the attention weight includes: the attention weight αi corresponding to the i-th path is: Here, a is a learnable attention parameter.

[0039] In some implementations, extracting third semantic information of a variable program dependency graph of a student source code based on a third neural network includes:

[0040] A variable program dependency graph VPDG is constructed based on a program dependency graph in which nodes represent statements and edges represent dependency relationships. The variable program dependency graph of program P is a ternary structure (G, S, Q), where G = (N, E) is obtained based on the program dependency graph PDG; N and E are the node set and directed edge set in G, respectively; S and Q are mappings from nodes to states and from nodes to conditional probability distributions, respectively;

[0041] Based on the variable program dependency graph VPDG, it is input into the gated graph convolutional network GGNN to obtain the third semantic information of the variable program dependency graph. The output layer of the gated graph convolutional network GGNN is connected to the attention layer to obtain the weights of different nodes, and the third semantic information is output after weighted fusion.

[0042] In some implementations, the fusing the first semantic information, the second semantic information, and the third semantic information using a Transformer feature fusion model to obtain a fused semantic feature includes:

[0043] A first similarity parameter is obtained based on the input first semantic information and second semantic information; and a first cross-modal semantic alignment feature is obtained by passing the two semantic information through a Transformer-based multi-head cross attention module;

[0044] A second similarity parameter is obtained based on the input first semantic information and third semantic information; and the two semantic information are passed through a Transformer-based multi-head cross attention module to obtain a second cross-modal semantic alignment feature;

[0045] The second semantic information and the third semantic information are input to obtain a third similarity parameter; and the two semantic information are passed through a Transformer-based multi-head cross attention module to obtain a third cross-modal semantic alignment feature;

[0046] The weight of the cross-modal semantic alignment feature is obtained by fusing the first cross-modal semantic alignment feature, the second cross-modal semantic alignment feature, and the third cross-modal semantic alignment feature through a fully connected layer and then passing through a softmax function;

[0047] The first similarity parameter, the second similarity parameter, and the third similarity parameter are weightedly fused based on the weights to obtain a fused semantic feature.

[0048] In a second aspect, a programming education intelligent management system based on a neural network is provided, including a source code semantic understanding model, wherein the source code semantic understanding model includes: a first neural network unit, a second neural network unit, a third neural network unit, a multimodal feature fusion model unit, and an output layer unit;

[0049] The first neural network unit is used to extract first semantic information of the student source code text;

[0050] The second neural network unit is used to extract second semantic information of the abstract syntax tree of the student source code;

[0051] The third neural network unit is used to extract third semantic information of the variable program dependency graph of the student source code;

[0052] The multimodal feature fusion model unit is used to fuse the first semantic information, the second semantic information, and the third semantic information to obtain a fused semantic feature;

[0053] The output layer unit is used to input the fused semantic features and output the student source code quality analysis results.

[0054] The present invention discloses a neural network-based programming education intelligent management method and system, which has the following beneficial effects: not only the text data of the source code is utilized, but also the abstract syntax tree of the source code is utilized to extract the functional semantics of the source code, and the variable program dependency graph is utilized to extract the third semantic information of the source code, and the multiple data types of the source code are utilized to fully extract the semantic information of the source code. The Transformer-based multimodal feature fusion model can be used to simultaneously combine the text representation of the source programming language code, the path set representation of the abstract syntax tree (AST), and the variable program dependency graph representation, so as to provide a semantic understanding method for students to write code, analyze the students' programming results, evaluate the students' mastery of each knowledge point, and provide a new tool for educators. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a framework diagram of a source code semantic understanding model of an embodiment of the present application;

[0056] Figure 2It is a technical framework diagram for providing a code search function for students in an embodiment of the present application;

[0057] Figure 3 is a flow chart of a method for extracting a basic path set vector based on a source code abstract syntax tree in an embodiment of the present application;

[0058] Figure 4 It is a structural block diagram of a programming education intelligent management system based on a neural network in an embodiment of the present application. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0060] The embodiment of the present application provides a programming education intelligent management method based on a neural network, including a source code semantic understanding model, the source code semantic understanding model is used to analyze source code semantics, the source code semantic understanding model includes: a first neural network, a second neural network, a third neural network, a multimodal feature fusion model, and an output layer. The data processing process of the source code semantic understanding model includes the following steps:

[0061] Step 1, extracting first semantic information of the student source code text based on the first neural network;

[0062] Step 2, extracting second semantic information of the abstract syntax tree of the student source code based on the second neural network;

[0063] Step 3, extracting third semantic information of the variable program dependency graph of the student source code based on the third neural network;

[0064] Step 4, fusing the first semantic information, the second semantic information, and the third semantic information using a Transformer-based multimodal feature fusion model to obtain a fused semantic feature;

[0065] Step 5: Input the fused semantic features into the output layer to obtain the student source code quality analysis results.

[0066] In the embodiment of the present application, not only the text data of the source code is used, but also the abstract syntax tree of the source code is used to extract the functional semantics of the source code, the variable program dependency graph is used to extract the third semantic information of the source code, and the multiple data types of the source code are used to fully extract the semantic information of the source code. The multimodal feature fusion model based on Transformer can be used to combine the text representation of the source programming language code, the path set representation of the abstract syntax tree (AST), and the variable program dependency graph representation, so as to more comprehensively capture the semantic information of the source code. It can be understood that the first neural network, the second neural network, the third neural network, the multimodal feature fusion model, and the output layer in the source code semantic understanding model in the embodiment of the present application are trained based on the source code data sample set. In the embodiment of the present application, QScored large code smell and quality metric-dataset (Code Smell) is used as a training set. First, the source code is preprocessed, including lexical analysis and syntax analysis, and the code is converted into an abstract syntax tree (AST), a control flow graph, and other forms suitable for neural network processing. Then the above-mentioned neural network is used to learn semantic representation from the preprocessed code. The training model is used to identify the patterns and structures in the code, thereby capturing the semantic information of the code. Using a large code base (Large Code Smell and Quality Metrics - Dataset (CodeSmell)) as training data, a neural network model is trained to learn the semantic information of the code. Finally, the system can be used to evaluate the code quality of students and give corresponding suggestions on their programming ability.

[0067] Furthermore, the programming education intelligent management method provided in the embodiment of the present application also includes:

[0068] Step 6: Based on the programming question keywords input by the student user, the natural language semantic understanding model is input to obtain the user's question semantic features;

[0069] Step 7, based on the source code data stored in the database, input the source code semantic understanding model to obtain source code semantic features;

[0070] Step 8: Matching is performed based on the semantic features of the user's question and the semantic features of the source code to obtain the answer to the question input by the user.

[0071] In the embodiment of the present application, the first neural network, the second neural network, and the third neural network based on the source code semantic understanding model fully extract the semantic information of the source code, and combine with the natural language semantic understanding model to obtain the semantic features of the user's question, and based on the matching of the two, obtain the programming answer that meets the user's programming question in the source code database. When students want to search for specific code, they can enter a natural language query into the system and obtain the corresponding recommended code results through the source code semantic understanding model and the natural language semantic understanding model.

[0072] In one embodiment, in the above step 1, extracting first semantic information of the student source code text based on the first neural network includes:

[0073] Step 11, extract key character information from the student source code text data and record it as a token, where the token includes keywords, variable names, and operators;

[0074] Step 12: The extracted text tokens are embedded into word vectors using the neural bag-of-words model (NBOW).

[0075] Step 13: Input the token word vector of the source code text into BiLSTM to obtain the semantic information of the code function, which is recorded as the first semantic information x t .

[0076] In the embodiment of the present application, BiLSTM is used to extract the semantic information of the entire function. The input of BiLSTM is the word embedding vector of each token. The BiLSTM model includes two LSTM layers in opposite directions, which can capture the previous and next context information in the source code. The forward LSTM and the backward LSTM can not only process long sequences and long dependencies, but also better capture the contextual relationship in the source code sequence through the bidirectional information flow.

[0077] In one embodiment, in the above step 2, extracting the second semantic information of the abstract syntax tree of the student source code based on the second neural network includes:

[0078] Extracting a basic path set vector for each function module in the abstract syntax tree of the source code;

[0079] Step 23: Input the basic path set into the neural network model to automatically generate functional semantic features and obtain the second semantic information x p .

[0080] Although code snippets have something in common with plain text, they should not be simply processed by text-based or symbol-based methods because they have richer and more explicit structural information. Therefore, in the embodiments of the present application, on the basis of extracting the text semantic information of the source code, a neural network is used to extract the functional semantic information in the grammatical structure such as the source code abstract syntax tree (AST). However, for large-scale software, the scale and depth of the abstract syntax tree corresponding to the source code are large, so the neural network model based on the abstract syntax tree is easily affected by the gradient vanishing problem, that is, the gradient becomes smaller and smaller during the training process.

[0081] Therefore, for abstract syntax trees with more complex structures, this system proposes a path-based automatic source code semantic feature extraction model. Unlike existing models that work on the entire AST, this model generates a basic path set for each function module in the source code and analyzes it on the abstract syntax tree node, which can effectively reduce the scale of analysis.

[0082] In one embodiment, the above step 22, extracting the basic path set vector for each function module in the abstract syntax tree of the source code, includes the following steps:

[0083] (1) Obtain the control flow graph CFG of the function;

[0084] (2) Starting from the entry node of the function, for the entry node, obtain an edge starting from the entry node and a tail node connected to the edge;

[0085] (3) taking the tail node as a new target analysis node, and obtaining all edges starting from the target analysis node;

[0086] (4) If the target analysis node does not have an edge starting from the target analysis node, proceed to step (5); otherwise, proceed to step (6);

[0087] (5) The target analysis node is determined to be a boundary node; for the boundary node, a sequence of all nodes from the entry node to the boundary node forms a basic path;

[0088] (6) Obtain the tail node of each edge starting from the target analysis node. If all tail nodes have been visited, proceed to step (5); otherwise, proceed to step (7);

[0089] (7) Select one of the unvisited tail nodes as a new target analysis node and go to step (3).

[0090] In an embodiment of the present application, all paths from the entry node (vex) of the function to the exit node or boundary node are searched on the control flow graph to obtain a path set. OldPath is the vex path from the entry node to a subsequent node on the control flow graph, and is a sequence of all nodes between the entry node and the point vex. And OldPaths is a collection of all oldPaths. The algorithm first iteratively searches for edges starting from the target point and stores them in the edgeHashTable. If there is no such edge, it indicates that the vex searched by DFS is a boundary node. And the oldPath from the entry node to the boundary node vex is added to the paths set. Otherwise, traverse the edgeHashTable of the target point to obtain the tail node endNode of each edge therein. If the node in oldPaths does not contain this endNode, it means that this node has not been visited before, then it will be added to oldPath. If all the tail nodes endNodes connected to a certain node vex are in oldPaths, that is, these tail nodes are already on a certain path, then in this case, this node vex is also considered to be a boundary node, and the oldPath containing it is added to the paths set. Each reachable path oldPath is composed of a series of adjacent nodes, and the reachable path set paths obtained by the algorithm is the collection of oldPath. This algorithm solves the problem of possible path explosion. First, it only finds one reachable path from the entry node to other nodes in the control flow graph, rather than all paths. In addition, for nodes that have been visited in the control flow graph (that is, there is a path that can reach this point), the algorithm will skip the analysis. So far, this algorithm has completed the extraction of reachable paths within the function.

[0091] In one embodiment, the above step 23, inputting the basic path set into the neural network model to automatically generate functional semantic features to obtain the second semantic information, includes:

[0092] Step 231, record the basic path set P = {p1, ..., p i ...,p n}, n is the path p in the path set i The total number of , i = 1, 2, ..., n;

[0093] Step 232, encoding the abstract syntax tree node corresponding to the path node into a token vector;

[0094] Step 233: Use the word embedding algorithm to map the token vector to a real-valued vector to obtain a path vector c with a dimension of d. i ;

[0095] Step 234, use the n*n dimensional weight matrix W to transform the n path vectors c with a dimension of d i Weighted fusion and activation by tanh function to obtain n combined context vectors with dimension d tanh is the hyperbolic tangent function; W is the learnable fully connected layer weight matrix;

[0096] Step 235: n combined context vectors corresponding to the n path vectors Input the attention layer, and output the attention score vector based on the attention vector a of dimension n. The attention vector a is a learnable parameter obtained during the network model training.

[0097] Step 236, normalize the attention score vector output by the attention layer into an attention weight vector α through a softmax function;

[0098] Step 237, based on the attention weight vector α, the n combined context vectors are fused to obtain the semantic feature vector x corresponding to the basic path set p .

[0099] In the embodiment of the present application, the fully connected layer is used to capture the intrinsic connection and pattern between the path vectors between the features, and the activation function tanh is further used to enable the network to learn more complex and abstract feature combinations. The input path vector set is mapped to a new feature space through the fully connected layer weighting and activation function. Furthermore, in the embodiment of the present application, the attention vector a of the attention layer is trained to focus on the function reachable path features that are more critical to the current task among the abstract features of the n path vectors, i.e., the n combined context vectors, and improve the semantic feature vector x of the basic path set. p extraction efficiency and accuracy.

[0100] Specifically, in step 236, the calculation of the attention weight includes: the attention weight α corresponding to the i-th path i for: Among them, a is a learnable attention parameter, which is randomly initialized and learned through network model training.

[0101] In one embodiment, in the above step 3, extracting the third semantic information of the variable program dependency graph of the student source code based on the third neural network includes:

[0102] Step 31, constructing a variable program dependency graph VPDG based on a program dependency graph in which nodes represent statements and edges represent dependency relationships, wherein the variable program dependency graph of program P is a ternary structure (G, S, Q), wherein G = (N, E) is obtained based on the program dependency graph PDG for program P; N and E are respectively the node set and the directed edge set in G; S and Q are respectively the mappings from nodes to states and from nodes to conditional probability distributions;

[0103] Step 32, based on the variable program dependency graph VPDG input into the gated graph convolutional network GGNN, the third semantic information of the variable program dependency graph is obtained, the output layer of the gated graph convolutional network GGNN is connected to the attention layer to obtain the weights of different nodes, and the third semantic information is output after weighted fusion.

[0104] In the embodiment of the present application, the acquisition of S and Q requires program instrumentation, executing the instrumented program and running the test case to obtain the program execution trajectory and test case execution information; scanning the control flow graph CFG and the program dependency graph PDG according to the program running trajectory and the test case execution information, obtaining the execution times of each state of the node, and calculating the execution state probability and conditional dependency probability of the node. Among them, the calculation method of the execution state probability of the node in S in the ternary structure (G, S, Q) is as follows:

[0105] According to the execution process of the program, it is mainly divided into three logical structures: sequential execution, branch selection and loop nesting. The execution state probability of the statement node in the execution process is calculated for each of the three logical structures. j The corresponding set Pa(X j ) is a directed edge (X i , X j ) of node X i The set of Pa(X j ) is node X j The set of all parent nodes of , the state probability distribution of the node is:

[0106] For sequential nodes: The probability distribution of the node execution state is: j Total number of executions / Pa(X j The total number of times all nodes in ) are executed * Pa (X j ) is the sum of the execution state probabilities of all nodes in;

[0107] For branch nodes: the execution state probability distribution of the node is: for the case of selecting the true branch, the state probability distribution of the node = the total number of times the node executes the true branch / the total number of times the node is executed * the execution state probability of the node; for the case of selecting the false branch, the state probability distribution of the node = the total number of times the node executes the false branch / the total number of times the node is executed * the execution state probability of the node; of course, the sum of the execution state probability of the node when the true branch is selected and the execution state probability of the node when the false branch is selected is 1;

[0108] For loop nested nodes, the state probability distribution of the node is similar to that of the branch node mentioned above. The difference is that in the calculation of the state probability distribution of the node, the execution state probability of the node is calculated based on the probability of its parent node.

[0109] In the embodiment of the present application, the calculation method of the conditional probability distribution of the nodes in Q in the ternary structure (G, S, Q) is as follows:

[0110] For node ni without a parent node, the conditional probability distribution of node ni = the total number of times node ni is in state xi / the total number of times node ni is executed;

[0111] For a node ni with a parent node, the conditional probability that the node ni is in the xi state and the parent node is in the Xi state is = the total number of times the node ni is in the xi state and the parent node is in the Xi state / the total number of times the node ni is in the xi state.

[0112] In one implementation, the above step 4, fusing the first semantic information, the second semantic information, and the third semantic information using a Transformer feature fusion model to obtain a fused semantic feature, includes:

[0113] Step 41, obtaining a first similarity parameter based on the input first semantic information and second semantic information; and obtaining a first cross-modal semantic alignment feature by applying the two semantic information to a Transformer-based multi-head cross attention module;

[0114] Step 42, obtaining a second similarity parameter based on the input first semantic information and third semantic information; and obtaining a second cross-modal semantic alignment feature by applying the two semantic information to a Transformer-based multi-head cross attention module;

[0115] Step 43, the input second semantic information and the third semantic information obtain a third similarity parameter; and the two semantic information are subjected to a Transformer-based multi-head cross attention module to obtain a third cross-modal semantic alignment feature;

[0116] Step 44, based on the first cross-modal semantic alignment feature, the second cross-modal semantic alignment feature, and the third cross-modal semantic alignment feature, after being fused through a fully connected layer, a softmax function is used to obtain the weight of the cross-modal semantic alignment feature;

[0117] Step 45 , weighted fusion is performed on the first similarity parameter, the second similarity parameter, and the third similarity parameter based on the weights to obtain a fused semantic feature.

[0118] In an embodiment of the present application, for the text semantics, abstract syntax tree semantics, and variable program dependency graph semantics of the source code, the cross-attention mechanism is used to obtain the cross-modal semantic alignment features of any two semantics. Furthermore, the weights of each cross-modal feature are obtained through the softmax function, and the fused semantic features of the three semantics are obtained after weighted fusion. In another embodiment, consider dividing the first semantic information, the second semantic information, and the third semantic information into several local semantic information, and calculating the cross-modal similarity parameters and cross-modal semantic alignment features at the level of local semantic information to determine the similarity between different parts of different semantic information. When obtaining the fused semantic features of the three semantic information, the similarities between different parts are weightedly fused to obtain the fused semantic features of the first semantic information, the second semantic information, and the third semantic information of fine-grained analysis. Specifically:

[0119] A first similarity parameter matrix is ​​obtained based on multiple local information vectors of the input first semantic information and multiple local information vectors of the second semantic information; and a first cross-modal semantic alignment feature matrix is ​​obtained by passing the multiple local information vectors of the two semantic information through a Transformer-based multi-head cross attention module;

[0120] A second similarity parameter matrix is ​​obtained based on multiple local information vectors of the input first semantic information and multiple local information vectors of the third semantic information; and a second cross-modal semantic alignment feature matrix is ​​obtained by passing the multiple local information vectors of the two semantic information through a Transformer-based multi-head cross attention module;

[0121] The input multiple local information vectors of the second semantic information and the multiple local information vectors of the third semantic information obtain a third similarity parameter matrix; and the multiple local information vectors of the two semantic information are passed through a Transformer-based multi-head cross attention module to obtain a third cross-modal semantic alignment feature matrix;

[0122] The weight of the cross-modal semantic alignment feature is obtained by concatenating the first cross-modal semantic alignment feature matrix, the second cross-modal semantic alignment feature matrix, and the third cross-modal semantic alignment feature matrix, and then fusing them through a fully connected layer and a softmax function;

[0123] The first similarity parameter matrix, the second similarity parameter matrix, and the third similarity parameter matrix are weightedly fused based on the weights to obtain fused semantic features. The similarity parameter matrices of the first similarity parameter matrix, the second similarity parameter matrix, and the third similarity parameter matrix are obtained by performing a dot multiplication of the transpose of a matrix composed of multiple local information vectors of the first semantic information and a matrix composed of multiple local information vectors of the second semantic information.

[0124] The embodiment of the present application provides a programming education intelligent management system based on a neural network, the programming education intelligent management system includes a source code semantic understanding model, the source code semantic understanding model includes: a first neural network unit, a second neural network unit, a third neural network unit, a multimodal feature fusion model unit, and an output layer unit;

[0125] The first neural network unit is used to extract first semantic information of the student source code text;

[0126] The second neural network unit is used to extract second semantic information of the abstract syntax tree of the student source code;

[0127] The third neural network unit is used to extract third semantic information of the variable program dependency graph of the student source code;

[0128] The multimodal feature fusion model unit is used to fuse the first semantic information, the second semantic information, and the third semantic information to obtain a fused semantic feature;

[0129] The output layer unit is used to input the fused semantic features and output the student source code quality analysis results.

[0130] Furthermore, the programming education intelligent management system also includes a natural language semantic understanding model, which is used to obtain the semantic features of the user's problem based on the programming problem keywords input by the student user. In the embodiment of the present application, the first neural network, the second neural network, and the third neural network based on the source code semantic understanding model fully extract the semantic information of the source code, and combine the natural language semantic understanding model to obtain the semantic features of the user's problem. Based on the matching of the two, the programming answer that meets the user's programming problem is obtained in the source code database. When a student wants to search for a specific code, he can enter a natural language query into the system, and obtain the corresponding recommended code results through the source code semantic understanding model and the natural language semantic understanding model.

[0131] For the specific limitations of the neural network-based programming education intelligent management system, please refer to the limitations of the neural network-based programming education intelligent management method above, which will not be repeated here. Each unit in the above-mentioned neural network-based programming education intelligent management system can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned units.

[0132] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A programming education intelligent management method based on neural network, characterized in that: It includes a source code semantic understanding model, and the data processing process of the source code semantic understanding model includes the following steps: Extracting first semantic information of the student's source code text based on the first neural network; Extracting second semantic information of the abstract syntax tree of the student's source code based on the second neural network; Extracting third semantic information of a variable program dependency graph of a student's source code based on a third neural network; The first semantic information, the second semantic information, and the third semantic information are fused using a Transformer-based multimodal feature fusion model to obtain a fused semantic feature; Input the fused semantic features into the output layer to obtain the student source code quality analysis result; The extracting of the third semantic information of the variable program dependency graph of the student source code based on the third neural network includes: A variable program dependency graph VPDG is constructed based on a program dependency graph in which nodes represent statements and edges represent dependency relationships. The variable program dependency graph of program P is a ternary structure (G, S, Q), where G = (N, E) is obtained based on the program dependency graph PDG; N and E are the node set and directed edge set in G, respectively; S and Q are mappings from nodes to states and from nodes to conditional probability distributions, respectively; Based on the variable program dependency graph VPDG, the variable program dependency graph is input into the gated graph convolutional network GGNN to obtain the third semantic information of the variable program dependency graph. The output layer of the gated graph convolutional network GGNN is connected to the attention layer to obtain the weights of different nodes, and the third semantic information is output after weighted fusion. The steps of obtaining S and Q include: Perform program instrumentation, execute the instrumented program and run the test case to obtain the program execution trajectory and test case execution information; scan the control flow graph CFG and the program dependency graph PDG according to the program running trajectory and test case execution information, obtain the execution times of each state of the node, and calculate the execution state probability and conditional dependency probability of the node; the calculation method of the execution state probability of the node in S in the ternary structure (G, S, Q) is as follows: According to the execution flow of the program, the calculation statement nodes are divided into sequential nodes, branch nodes and loop nesting nodes; node X j The corresponding set Pa(X j ) is a directed edge (X i , X j ) of node X i The set of Pa(X j ) is node X j The set of all parent nodes of , the state probability distribution of the node is: For sequential nodes: The probability distribution of the node execution state is: (node ​​X j Total number of executions / Pa(X j )*Pa(X j ) is the sum of the execution state probabilities of all nodes in; For branch nodes: the execution state probability distribution of the node is: for the case of selecting the true branch, the state probability distribution of the node = (the total number of times the node executes the true branch / the total number of times the node is executed) * the execution state probability of the node; for the case of selecting the false branch, the state probability distribution of the node = (the total number of times the node executes the false branch / the total number of times the node is executed) * the execution state probability of the node; where the sum of the execution state probability of the node when the true branch is selected and the execution state probability of the node when the false branch is selected is 1; For loop nested nodes, the execution state probability of the node is calculated based on the probability of its parent node; The calculation method of the conditional probability distribution of the nodes in Q in the ternary structure (G, S, Q) includes: For node ni without a parent node, the conditional probability distribution of node ni = the total number of times node ni is in state Ti / the total number of times node ni is executed; For a node ni with a parent node, the conditional probability that the node ni is in the Ti state and the parent node is in the Ti state = the total number of times the node ni is in the Ti state and the parent node is in the Ti state / the total number of times the node ni is in the Ti state.

2. The programming education intelligent management method according to claim 1, characterized in that: Also includes: Based on the programming question keywords input by the student user, the natural language semantic understanding model is input to obtain the user's question semantic features; Based on the source code data stored in the database, a source code semantic understanding model is input to obtain source code semantic features; The answer to the user's input question is obtained by matching the user's question semantic features with the source code semantic features.

3. The neural network-based programming education intelligent management method according to claim 1 is characterized in that: The extracting first semantic information of the student source code text based on the first neural network includes: For the student source code text data, extract key character information and record it as token. Token includes keywords, variable names, and operators. The extracted text tokens are embedded into word vectors using the neural bag-of-words model NBOW; Input the token word vector of the source code text into BiLSTM to obtain the semantic information of the code function, which is recorded as the first semantic information x t .

4. The neural network-based programming education intelligent management method according to claim 1, characterized in that: The second semantic information of the abstract syntax tree of the student source code extracted based on the second neural network includes: Extracting a basic path set vector for each function module in the abstract syntax tree of the source code; The basic path set is input into the neural network model to automatically generate functional semantic features and obtain the second semantic information x p .

5. The neural network-based programming education intelligent management method according to claim 4 is characterized in that: The step of extracting a basic path set vector from each function module in the abstract syntax tree of the source code comprises the following steps: (1) Obtain the control flow graph CFG of the function; (2) Starting from the entry node of the function, for the entry node, obtain an edge starting from the entry node and a tail node connected to the edge; (3) taking the tail node as a new target analysis node, and obtaining all edges starting from the target analysis node; (4) If the target analysis node does not have an edge starting from the target analysis node, proceed to step (5); otherwise, proceed to step (6); (5) The target analysis node is determined to be a boundary node; for the boundary node, a sequence of all nodes from the entry node to the boundary node forms a basic path; (6) Obtain the tail node of each edge starting from the target analysis node. If all tail nodes have been visited, proceed to step (5); otherwise, proceed to step (7); (7) Select one of the unvisited tail nodes as a new target analysis node and go to step (3).

6. The neural network-based programming education intelligent management method according to claim 4 is characterized in that: The step of inputting the basic path set into the neural network model to automatically generate functional semantic features and obtain the second semantic information includes: Let the basic path set P = {p1,...,p i ...,p n }, n is the path p in the path set i The total number of , i = 1, 2, ..., n; Encode the abstract syntax tree node corresponding to the path node into a token vector; Use the word embedding algorithm to map the token vector to a real-valued vector to obtain a path vector c with a dimension of d. i ; Use the n*n dimensional weight matrix W to transform the n path vectors c with dimension d into i Weighted fusion and activation by tanh function to obtain n combined context vectors with dimension d tanh is the hyperbolic tangent function; W is the learnable fully connected layer weight matrix; The n combined context vectors corresponding to the n path vectors Input the attention layer, and output the attention score vector based on the attention vector a of dimension n. The attention vector a is a learnable parameter obtained during the network model training. The attention score vector output by the attention layer is normalized into the attention weight vector α through the softmax function; Based on the attention weight vector α, the n combined context vectors are fused to obtain the semantic feature vector x corresponding to the basic path set p .

7. The neural network-based programming education intelligent management method according to claim 6 is characterized in that: The method for calculating the attention weight includes: the attention weight α corresponding to the i-th path i for: Here, b is a learnable attention parameter.

8. The neural network-based programming education intelligent management method according to claim 1, characterized in that: The first semantic information, the second semantic information, and the third semantic information are fused using a Transformer feature fusion model to obtain a fused semantic feature, including: A first similarity parameter is obtained based on the input first semantic information and second semantic information; and a first cross-modal semantic alignment feature is obtained by passing the two semantic information through a Transformer-based multi-head cross attention module; A second similarity parameter is obtained based on the input first semantic information and third semantic information; and the two semantic information are passed through a Transformer-based multi-head cross attention module to obtain a second cross-modal semantic alignment feature; The second semantic information and the third semantic information are input to obtain a third similarity parameter; and the two semantic information are passed through a Transformer-based multi-head cross attention module to obtain a third cross-modal semantic alignment feature; The weight of the cross-modal semantic alignment feature is obtained by fusing the first cross-modal semantic alignment feature, the second cross-modal semantic alignment feature, and the third cross-modal semantic alignment feature through a fully connected layer and then passing through a softmax function; The first similarity parameter, the second similarity parameter, and the third similarity parameter are weightedly fused based on the weights to obtain a fused semantic feature.

9. A programming education intelligent management system based on neural network, characterized in that: It includes a source code semantic understanding model, which includes: a first neural network unit, a second neural network unit, a third neural network unit, a multimodal feature fusion model unit, and an output layer unit; The first neural network unit is used to extract first semantic information of the student source code text; The second neural network unit is used to extract second semantic information of the abstract syntax tree of the student source code; The third neural network unit is used to extract third semantic information of the variable program dependency graph of the student source code; The multimodal feature fusion model unit is used to fuse the first semantic information, the second semantic information, and the third semantic information to obtain a fused semantic feature; The output layer unit is used to input the fused semantic features and output the student source code quality analysis results; The third neural network unit performs the following steps when extracting the third semantic information of the variable program dependency graph of the student source code: A variable program dependency graph VPDG is constructed based on a program dependency graph in which nodes represent statements and edges represent dependency relationships. The variable program dependency graph of program P is a ternary structure (G, S, Q), where G = (N, E) is obtained based on the program dependency graph PDG; N and E are the node set and directed edge set in G, respectively; S and Q are mappings from nodes to states and from nodes to conditional probability distributions, respectively; Based on the variable program dependency graph VPDG, the variable program dependency graph is input into the gated graph convolutional network GGNN to obtain the third semantic information of the variable program dependency graph. The output layer of the gated graph convolutional network GGNN is connected to the attention layer to obtain the weights of different nodes, and the third semantic information is output after weighted fusion. The steps of obtaining S and Q include: Perform program instrumentation, execute the instrumented program and run the test case to obtain the program execution trajectory and test case execution information; scan the control flow graph CFG and the program dependency graph PDG according to the program running trajectory and test case execution information, obtain the execution times of each state of the node, and calculate the execution state probability and conditional dependency probability of the node; the calculation method of the execution state probability of the node in S in the ternary structure (G, S, Q) is as follows: According to the execution flow of the program, the calculation statement nodes are divided into sequential nodes, branch nodes and loop nesting nodes; node X j The corresponding set Pa(X j ) is a directed edge (X i , X j ) of node X i The set of Pa(X j ) is node X j The set of all parent nodes of , the state probability distribution of the node is: For sequential nodes: The probability distribution of the node execution state is: (node ​​X j Total number of executions / Pa(X j )*Pa(X j ) is the sum of the execution state probabilities of all nodes in; For branch nodes: the execution state probability distribution of the node is: for the case of selecting the true branch, the state probability distribution of the node = (the total number of times the node executes the true branch / the total number of times the node is executed) * the execution state probability of the node; for the case of selecting the false branch, the state probability distribution of the node = (the total number of times the node executes the false branch / the total number of times the node is executed) * the execution state probability of the node; where the sum of the execution state probability of the node when the true branch is selected and the execution state probability of the node when the false branch is selected is 1; For loop nested nodes, the execution state probability of the node is calculated based on the probability of its parent node; The calculation method of the conditional probability distribution of the nodes in Q in the ternary structure (G, S, Q) includes: For node ni without a parent node, the conditional probability distribution of node ni = the total number of times node ni is in state Ti / the total number of times node ni is executed; For a node ni with a parent node, the conditional probability that the node ni is in the Ti state and the parent node is in the Ti state = the total number of times the node ni is in the Ti state and the parent node is in the Ti state / the total number of times the node ni is in the Ti state.

Images