Programming knowledge tracking method based on code prediction enhancement

Abstract

The invention belongs to the technical field of computers, and particularly discloses a programming knowledge tracking method based on code prediction enhancement, and the method comprises the steps: extracting embedded representations based on historical programming questions and corresponding historical programming answers, and obtaining historical question embedded representations and historical answer embedded representations; based on the embedded representation of the historical question, the embedded representation of the historical answer and the embedded representation of the target programming question, the embedded representation of a source code input by the tested object for the target programming question is predicted through a multi-head attention mechanism model, and the predicted embedded representation of the source code serves as a programming answer prediction result; on the basis of the embedded representation of the target programming question, the programming answer prediction result and the current knowledge state of the tested object, the correct answer probability of the tested object for the target programming question is predicted, and the current knowledge state is obtained by analyzing time sequence evolution of the knowledge state on the basis of historical programming answers of the tested object. Through the method, the programming performance of the learner can be accurately predicted.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and more specifically, relates to a programming knowledge tracing method based on code prediction enhancement. Background Art

[0002] Thanks to the development of deep learning, numerous deep knowledge tracing models have been proposed. However, most of these classic knowledge tracing (KT) methods are targeted at general disciplines and not specifically designed for programming performance prediction. This makes them difficult to capture the syntactic and logical information of code and fails to reflect the cognitive evolution of learners across multiple code submissions. This type of method, known as Result-Driven State-Based KT, models performance based solely on problem-solving results and predicts performance by constructing features of the learner's latent cognitive state. Subsequently, researchers have incorporated the semantic and structural information of code into programming problems, designing specialized neural network architectures to achieve more accurate performance prediction. This type of method is known as Semantic-Driven State-Based KT. Its core idea is to enable the model to perceive the semantic information of the code, thereby more accurately capturing the learner's cognitive state in programming. This method has achieved significantly better results than Result-Driven State-Based KT on many real-world programming datasets.

[0003] However, these methods are still essentially end-to-end model architectures, directly using problem-solving performance as input to construct the learner's latent cognitive state and use this to predict the student's next problem-solving performance. In programming, this model has significant shortcomings: whether a learner can correctly solve a programming problem depends primarily on whether the code they submit can pass the test case, rather than simply inferring their cognitive level based on their past problem-solving performance. The shortcomings of existing technologies lead to low prediction accuracy. How to accurately predict learners' programming performance is a technical problem that needs to be solved urgently in this field. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this application is to accurately predict the programming performance of learners.

[0005] To achieve the above objectives, in a first aspect, the present application provides a programming knowledge tracking method based on code prediction enhancement, the method comprising: Based on historical programming questions and corresponding historical programming answers, embedding representations are extracted to obtain historical question embedding representations and historical answer embedding representations. The historical programming answers are the source code input by the test subject (e.g., a student learning programming) in response to the historical programming questions. The source code is used to solve the programming questions. Based on the embedding representations of historical questions, historical answers, and the target programming question (the target programming question is different from the historical programming questions), a multi-head attention mechanism model (including a multi-head attention mechanism layer (MHA) and a fully connected layer) is used to predict the embedding representation of the source code input by the test subject for the target programming question. The predicted source code embedding representation is used as the programming answer prediction result corresponding to the target programming question. The probability of the subject answering the target programming question correctly is predicted based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The current knowledge state is obtained by analyzing the temporal evolution of the knowledge state based on the subject's historical programming answers.

[0006] In one possible implementation, extracting the embedded representation includes: Input historical programming questions into the large language model, extract the original embedding representation of the historical programming questions through the large language model, and input historical programming answers into the large language model, extract the original embedding representation of the historical programming answers through the large language model; Based on the original embedding representation of historical programming questions, dimensionality reduction is performed to obtain the embedding representation of historical questions. Based on the original embedding representation of historical programming answers, dimensionality reduction is performed to obtain the embedding representation of historical answers.

[0007] Optionally, the above-mentioned method of extracting embedded representations for historical programming problems can also be applied to target programming problems to extract embedded representations of the target programming problems.

[0008] In one possible implementation, the method of predicting the probability of the subject correctly answering the target programming problem based on the embedded representation of the target programming problem, the programming answer prediction result, and the current knowledge state of the subject includes: Based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge status of the subject, feature splicing is performed to obtain the spliced ​​features; Input the concatenated features into the multi-layer perceptron, perform dimension compression through the multi-layer perceptron, and obtain the dimension-compressed features output by the multi-layer perceptron; After the input dimension is compressed, the features are fed into the activation function to obtain the probability of the correct answer output by the activation function.

[0009] In a possible implementation, before predicting the probability of the subject's correct answer to the target programming question, the following steps are further included: Based on the correctness of the source code (the source code is also the programming answer) input by the subject for the programming question at the current time step, the embedded representation of the programming answer corresponding to the programming question is projected to obtain the correctness information at the current time step. ; Based on the knowledge state of the object being measured at the previous time step , the embedded representation of the programming problem at the current time step and the right and wrong information at the current time step , analyze the temporal evolution of the knowledge state through the long short-term memory network LSTM, obtain the knowledge state of the object under test at the current time step, and use it as the current knowledge state of the object under test .

[0010] In one possible implementation, the model used to extract the embedding representation is constructed based on a large language model and a linear dimensionality reduction module. The large language model is used to extract the original embedding representation, and the linear dimensionality reduction module is used to reduce the dimensionality of the original embedding representation. The linear dimensionality reduction module is trained through the following steps: Based on programming problem samples, programming answer samples, and sample labels, the evaluation simulator is trained by minimizing the binary cross entropy loss function; Among them, the sample label is used to indicate whether the programming answer sample can solve (pass) the programming question sample. The evaluation simulator is constructed by cascading linear dimensionality reduction modules and multi-layer perceptrons. The multi-layer perceptron is used to predict the probability that the programming answer can solve (pass) the programming question based on the embedding representation corresponding to the programming question and the embedding representation corresponding to the programming answer.

[0011] In one possible implementation, the multi-head attention mechanism model is trained by the following steps: Based on programming question samples, programming answer samples, and sample labels, a multi-head attention mechanism model is trained using the total loss function of code prediction. The total loss function for code prediction is determined by the following formula: ; ; ; ; in, represents the total loss function of code prediction, It is a hyperparameter. Different samples are numbered according to the time sequence of the answers. Indicates the +The embedding representation of the programming problem sample at 1 time step, Indicates the +The embedding representation of the programming answer sample at 1 time step, Indicates that based on the previous The embedding representation corresponding to the samples of time steps and , the programming answer prediction results predicted by the multi-head attention mechanism model, represents the sigmoid function, represents the inner product operation, express The corresponding correct code set item, express The corresponding error code set item, It is a triple A collection of represents the number of samples, Indicates the The labels corresponding to the samples at time steps, Indicates that the multilayer perceptron in the evaluation simulator is based on and , predicting the probability that the programming answer prediction result can solve the programming problem.

[0012] In one possible implementation, the prediction model used to predict the probability of the subject answering the target programming question correctly is trained through the following steps: Based on programming problem samples, programming answer samples, and sample labels, a long short-term memory (LSTM) network and a prediction model are trained using a binary cross-entropy loss function. The long short-term memory (LSTM) network is used to obtain the knowledge state of the object under test at the current time step based on the knowledge state of the object under test at the previous time step, the embedded representation of the programming problem at the current time step, and the right or wrong information at the current time step, as the current knowledge state of the object under test. The prediction model is used to predict the probability of the subject correctly answering the target programming question based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The target programming question is the programming question at the next time step; Binary cross entropy loss function used for training long short-term memory networks (LSTMs) and prediction models It is determined by the following formula: ; ; ; in, Indicates that the prediction model is for The prediction result given by the programming problem sample (target programming problem) under the time step (next time step), MLP represents multi-layer perceptron, Represents the current knowledge state of the object being tested given by the long short-term memory network LSTM, Represents a splicing operation.

[0013] In a second aspect, the present application provides a programming knowledge tracking device based on code prediction enhancement, comprising: An embedding representation extraction module is used to extract embedding representations based on historical programming questions and corresponding historical programming answers, obtaining the embedding representations of the historical questions and the embedding representations of the historical answers. The historical programming answers are the source code input by the test subject in response to the historical programming questions, and the source code is used to solve the programming questions. The code prediction module is used to predict the embedding representation of the source code input by the test subject for the target programming question based on the embedding representation of historical questions, the embedding representation of historical answers, and the embedding representation of the target programming question. The predicted source code embedding representation is used as the programming answer prediction result corresponding to the target programming question; The programming performance prediction module is used to predict the probability of the subject's correct answer to the target programming question based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The current knowledge state is based on the subject's historical programming answers and is obtained by analyzing the time-series evolution of the knowledge state.

[0014] In a third aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0016] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the existing technologies: This application decomposes the programming prediction task into code prediction (predicting the embedded representation of the source code input by the subject for the target programming question, that is, the programming answer prediction result) and performance prediction: first, the code information submitted historically by the learner (subject) is used to predict the current possible code features (programming answer prediction results); second, the learner's current cognitive state is constructed in combination with historical answering performance; finally, the programming answer prediction results, current cognitive state and question (topic) characteristics are combined to jointly predict the learner's performance, that is, predict whether the learner can solve (pass) the target programming question. The target programming question is used as a test case. By comprehensively considering the impact of the learner's past answering performance on the target programming question, and the probability that the programming answer prediction result can solve the target programming question, programming knowledge tracking based on code prediction enhancement is realized, which can accurately predict the learner's programming performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flowchart of a method for tracking programming knowledge based on enhanced code prediction provided by an embodiment of the present application; Figure 2 Schematic diagram of the structure of a programming knowledge tracking device based on code prediction enhancement provided by an embodiment of the present application; Figure 3 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0019] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0020] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more, for example, multiple processing units means two or more processing units, etc.; multiple elements means two or more elements, etc.

[0021] To achieve accurate programming performance prediction, a more reasonable approach is to decompose the programming prediction task into two parts: code prediction and performance prediction. First, the learner's previously submitted code is used to predict the current code's likely characteristics. Second, the learner's past performance is combined with past answering performance to construct the learner's cognitive state. Finally, these two components are combined with the question characteristics to jointly predict whether the learner can answer the question correctly. This type of approach can be called semantically driven code-based knowledge tracing (SCKT).

[0022] The core concept of SCKT focuses on code feature prediction. However, this task is inherently more challenging than performance prediction and requires considering both (A) individuality and (B) commonality. First, regarding (A) individuality, it is necessary to fully consider the student's individual programming skills and algorithmic thinking, which are hidden in historical code sequences. Dynamically extracting skill features relevant to the target problem from historical code sequences is a fundamental and challenging issue in code prediction. Second, regarding (B) commonality, similar programming problems often share common code patterns, and for a given problem, there are often only a limited number of mainstream correct solutions. If only considering the learner's individual programming patterns, when the learner's interaction data is too short, the prediction results are prone to significant deviations. Therefore, the SCKT method needs to balance the learner's individual programming characteristics with the common code patterns of the problem in code feature prediction to improve prediction accuracy and robustness.

[0023] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0024] like Figure 1 As shown, SCKT consists of three main modules, each corresponding to a training step.

[0025] In Step 1, a large language model was used to encode the question text and student code, obtaining high-dimensional semantic embeddings that fully captured the code's logical structure and the question's semantics. Based on this, an evaluation simulation model was trained and its internal dimensionality reduction module was used to extract features from the original embeddings generated by the large model, providing a unified, low-dimensional, and semantically preserved representation for subsequent stages.

[0026] Step 2: After obtaining the embedded representations of the problem and code, a code predictor is trained. This module comprehensively considers the student's individual programming habits and common problem-solving patterns in previous answers. This module predicts the student's likely performance on a new problem (in this application, the questions used to test students can also be called problems) based on the student's historical answer sequence.

[0027] Step 3: Model the student’s knowledge status based on the embedded representation of the student’s answer sequence provided in Step 1, and use the code predictor in Step 2 to generate the answer code for the question to be predicted, achieving accurate prediction based on the student’s possible answer pattern and knowledge status.

[0028] Finally, a student programming performance prediction model was trained based on the above modules.

[0029] The following is an expanded description of Step 1 to Step 3.

[0030] Step 1: Get Embedding.

[0031] The CodeQwen1.5-7B-chat model (a large language model dedicated to code) is selected to generate embedding representations of question descriptions and source code.

[0032] Given a tokenized input sequence (It can be a question description or student code). The model calculates the context-aware representation matrix of the sequence through a multi-layer self-attention mechanism: Where n represents the length of the sequence and D represents the hidden layer dimension of the Transformer model.

[0033] In order to extract a fixed-dimensional vector from the representation matrix H as the global semantic representation of the entire input sequence, an aggregation function is applied to all row vectors of H. In this application, the mean pooling strategy is used as the aggregation function To obtain the final sequence embedding .

[0034] Subsequently, an evaluation simulator was designed to perform feature extraction based on the high-dimensional semantic embedding of questions and codes extracted by the large language model (LLM), and to determine whether the target code can solve the corresponding programming problems.

[0035] This evaluation simulator mainly completes two tasks: first, the traditional online evaluation (Online Judge, OJ) system relies on predefined test cases to judge the correctness of the code, while this simulator analyzes the semantic embedding of problems and code to capture their similarities at the logical level and achieve code passability prediction under the condition of no test cases; second, Represents the input programming problem to LLM, the original embedding representation extracted by LLM, Represents the input of student programming answers to LLM, the original embedding representation extracted by LLM, and the original embedding representation generated by LLM There is often a large amount of redundant information. In order to improve prediction accuracy and computational efficiency, an independent dimensionality reduction network is constructed to extract more discriminative low-dimensional feature representations.

[0036] Specifically, we first designed a dual-channel linear dimensionality reduction module (one channel is used to reduce the dimensionality of programming questions, and the other channel is used to reduce the dimensionality of programming answers). The linear dimensionality reduction module is a tool that maps (projects) high-dimensional data into a low-dimensional space through mathematical transformation. It aims to reduce data complexity and computational cost while preserving the important information and structure of the original data as much as possible: ; Where, and is the projection matrix. The reduced-dimensional problem and code embedding are concatenated into a joint feature vector , and fed into a multi-layer perceptron (MLP) classifier for prediction: in, , is a learnable parameter, is the Sigmoid function, and Indicates bias. Prediction results Representation model code Can the prediction probability of the corresponding problem be passed? Training is performed by minimizing the binary cross entropy loss function: ; Different samples are numbered according to the time sequence of the answers, and the corner marks Indicates the number of time steps. Each time step corresponds to a programming question and a programming answer. Indicates the The joint feature vector corresponding to the samples in the time step (the samples in one time step are composed of programming question samples and corresponding programming answer samples), Indicates the The label corresponding to the sample at the time step indicates whether the programming answer can solve (pass) the programming question. This optimization process not only improves prediction performance but also effectively selects features that are highly relevant to code correctness judgment and suppresses redundant and noisy information.

[0037] During the training process of Step 1, the parameters of the dual-channel linear dimensionality reduction module and the multi-layer perceptron MLP are adjusted according to the loss function value.

[0038] After training, the dimensionality reduction matrix can be used to extract features from semantic embeddings of large models: .in, and is the problem and code embedding (dimension D) output by LLM, and and is the low-dimensional representation after extraction (dimension is d). Indicates the number of time steps. Each time step corresponds to a programming question and a programming answer. Indicates input Programming questions at time steps to LLM, and the original embedding representation extracted by LLM, Indicates input The programming answer under time steps is sent to LLM, and the original embedding representation extracted by LLM is For The embedding representation obtained after dimensionality reduction is For The embedded representation obtained by dimensionality reduction.

[0039] Step 2: Code predictor.

[0040] Based on the embedding of question text and code information in Step 1, a code predictor based on the dual constraints of individuality and commonality based on the attention mechanism is designed. It jointly models individual historical behavior and group statistical priors to achieve fine-grained code embedding prediction. Given a student's historical interaction sequence , predicting students’ answers to questions Embedded representation of the answer code : ; Code prediction uses a multi-head attention mechanism (MHA) to generate code embedding representations through the semantic association between the target question and historical interaction data. The model input includes: is the embedding of the target problem, , B represents the number of batches BatchSize, represents the embedding sequence of historical questions, Represents the corresponding historical code embedding sequence. The calculation can be expressed as: . Among them, h is the number of attention heads, is the output transformation matrix. Attention output It captures the weighted semantic representation of the target question and the historical code. The attention output is then mapped to the target code embedding space through a fully connected layer: in, and are the learnable parameters of the fully connected layer, Embed the predicted code. Construct a semantic alignment loss to maximize the semantic similarity between the predicted code of the target question and the actual answer code: ,in It is the sigmoid function, and the inner product operation <·,·> measures the semantic similarity of two embeddings.

[0041] The above personalized modeling only focuses on the historical programming behavior of individual students, and does not take into account the common answering characteristics that are significant in programming exercises. For example, most programming problems have only a limited number of mainstream correct solutions, and their common error patterns also have certain commonalities, such as incorrect boundary condition processing. Another problem is that when the interaction data of students' historical questions is sparse, relying solely on the personalized attention mechanism will lead to prediction bias due to insufficient historical information. Therefore, by designing the contrast loss of group perception, the prediction code Close to similar correct code patterns in embedding space , stay away from bad code patterns : .in is all possible triples A collection of; for each question q maintain a positive sample pool (Correct code set) and negative sample pool (Error code set); at each prediction point , randomly sample positive samples and negative samples Calculate their similarities respectively.

[0042] To evaluate the quality of the generated code predictions, concatenate the predicted code embeddings and the corresponding question embedding get , and input it into the evaluation simulation model for prediction. Its loss can directly reflect the quality of code embedding prediction. Therefore, the total loss of code prediction is designed to be: ; in is a hyperparameter that represents the weight of the loss function.

[0043] During the training process of Step 2, the parameters of the multi-head attention mechanism are adjusted according to the loss function value.

[0044] Step 3: Programming performance prediction.

[0045] In the programming practice scenario, the semantic features of the question (such as the problem description, input and output requirements) combined with the answer code can serve as a fine-grained knowledge state indicator. Therefore, this model constructs a knowledge state representation by using the semantic encoding of the question text and the answer code information to capture the student's understanding of the knowledge point. In order to better track the student's knowledge state, two projection matrices are used. and Project the code based on whether the answer is correct or not, and introduce the correct or incorrect information into the code embedding. First, project the code differently based on whether the answer is correct or not: ; Where, Indicates that the programming answer can solve (pass) the corresponding programming question, Indicates that the programming answer cannot solve (cannot pass) the corresponding programming problem. In order to better preserve the information in the original code embedding, the projection matrix is ​​used during the training process. and In order to capture the temporal evolution of the student's knowledge state, this application uses a long short-term memory network (LSTM) to model the student's knowledge state, with the following input features: The calculation of LSTM at each time step t is: ,in is the student’s current state of knowledge, is the student’s knowledge state at the previous time step.

[0046] In order to predict whether the student can pass the target question, the target question, the embedding representation of the code predicted by the code predictor in Step 2, and the student’s knowledge state are concatenated: Then the multi-layer perceptron MLP is used to predict the results: ,in It is a feature distiller consisting of a four-layer fully connected network, through progressive dimension compression, is the sigmoid activation function, which maps the feature to the probability of correct answer in the interval [0,1].

[0047] Here we choose binary cross entropy as the loss function of SCKT: ; in It is The true labels of samples, is the prediction result. N is the number of samples for prediction.

[0048] During the training process of Step 3, the parameters of LSTM and MLP are adjusted according to the loss function value.

[0049] To comprehensively evaluate the model performance, three types of baseline methods were compared on three real-world programming datasets: traditional knowledge tracing models, code-enhanced knowledge tracing models, and knowledge tracing methods based on pre-trained code large language models.

[0050] Classic knowledge tracking methods: Includes multiple classic knowledge tracking methods based on time series modeling: DKT models the evolution of knowledge states through RNN, DKVMN uses memory networks to store skill mastery, AKT uses self-attention mechanisms and adaptive knowledge retrieval respectively, and SimpleKT improves efficiency by simplifying interaction modeling. DTransformer builds a model from the problem level to the knowledge level to clearly diagnose the student's knowledge proficiency on each problem. LPKT monitors knowledge status by directly modeling the student's learning process. ReKT proposes a simplified structure by optimizing the complexity of existing methods while retaining strong predictive capabilities. extraKT focuses on expanding the context window to handle longer interaction sequences.

[0051] Programming knowledge tracing methods improve traditional knowledge tracing frameworks by integrating code structure information. CodeDKT, for the first time, combines Code2Vec with an attention mechanism to extract code features. ECKT builds on CodeDKT by using a large model to extract knowledge components from code. PST constructs a Code Information Graph and Code Tracing Graph to model code evolution. PDKT uses a fine-tuned CodeBERT to extract code features. SQKT automatically extracts skill information from student questions. However, SQKT, which lacks student question information in this dataset and only uses the underlying structure, falls short of its theoretical performance limit.

[0052] Pre-trained language model-based approach: Leveraging the powerful representational capabilities of pre-trained code models, combined with LSTM, we track students' knowledge status. We selected CodeBERT (CB), GraphCodeBERET (GCB), CodeQwen1.5-7B-Chat, and Qwen2.5-Coder-7B-Instruct as encoding tools. We embed the problem text and student code, then use LSTM to map the learner's detailed learning trajectory and model their knowledge status.

[0053] Table 1 below compares the performance of SCKT with 17 baseline models on three programming datasets (data collected from AIZU.org for Java, Python, and C programming languages). Experimental results show that SCKT achieves state-of-the-art performance on all datasets, with leading margins of 0.16% to 6.06% in AUC and ACC metrics, fully demonstrating its effectiveness and generalization capabilities in programming knowledge tracing tasks.

[0054] Table 1 Performance comparison table

[0055] The following describes the programming knowledge tracing device based on code prediction enhancement provided by the present application. The programming knowledge tracing device based on code prediction enhancement described below and the programming knowledge tracing method based on code prediction enhancement described above can refer to each other.

[0056] Figure 2 This is a schematic diagram of the structure of a programming knowledge tracking device based on code prediction enhancement provided by an embodiment of the present application. Figure 2 As shown, the apparatus comprises: an embedding representation extraction module 10, a code prediction module 20 and a programming performance prediction module 30. An embedded representation extraction module 10 is configured to extract embedded representations based on historical programming questions and corresponding historical programming answers, obtaining embedded representations of the historical questions and embedded representations of the historical answers, wherein the historical programming answers are source code input by the test subject in response to the historical programming questions, and the source code is used to solve the programming questions; A code prediction module 20 is configured to predict the embedded representation of the source code input by the test subject for the target programming question based on the embedded representation of the previous questions, the embedded representation of the previous answers, and the embedded representation of the target programming question, using a multi-head attention mechanism model. The predicted embedded representation of the source code is used as the programming answer prediction result corresponding to the target programming question; The programming performance prediction module 30 is used to predict the probability of the subject's correct answer to the target programming question based on the embedded representation of the target programming question, the programming answer prediction result and the current knowledge state of the subject. The current knowledge state is based on the subject's historical programming answers and is obtained by analyzing the time-series evolution of the knowledge state.

[0057] It is understandable that the detailed functional implementation of each of the above units / modules can be found in the introduction of the aforementioned method embodiment, and will not be repeated here.

[0058] It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment. The implementation principle and technical effect of the corresponding program module in the device are similar to those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be repeated here.

[0059] Based on the method in the above embodiment, an embodiment of the present application provides an electronic device, Figure 3 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 3As shown, the electronic device may include: a processor (Processor) 810, a communication interface (Communications Interface) 820, a memory (Memory) 830, and a communication bus 840. The processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the method in the above embodiment.

[0060] In addition, the logic instructions in the aforementioned memory 830 can be implemented in the form of a software functional unit and, when sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0061] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0062] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0063] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0064] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC.

[0065] The above embodiments can be implemented in whole or in part using software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions. When loaded and executed on a computer, the computer program instructions fully or partially produce the processes or functions described in the embodiments of this application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or semiconductor media (e.g., solid-state drive (SSD)).

[0066] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0067] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A programming knowledge tracing method based on code prediction enhancement, characterized in that: include: Based on the historical programming questions and the corresponding historical programming answers, extracting embedded representations to obtain the historical question embedding representation and the historical answer embedding representation. The historical programming answers are the source code input by the test subject in response to the historical programming questions, and the source code is used to solve the programming questions. Based on the embedding representations of historical questions, historical answers, and the target programming question, a multi-head attention mechanism model is used to predict the embedding representation of the source code input by the test subject for the target programming question. The predicted source code embedding representation is used as the programming answer prediction result corresponding to the target programming question. The probability of the subject answering the target programming question correctly is predicted based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The current knowledge state is obtained by analyzing the temporal evolution of the knowledge state based on the subject's historical programming answers.

2. The programming knowledge tracing method based on code prediction enhancement according to claim 1, characterized in that: The extracting of the embedded representation includes: Input historical programming questions into the large language model, extract the original embedding representation of the historical programming questions through the large language model, and input historical programming answers into the large language model, extract the original embedding representation of the historical programming answers through the large language model; Based on the original embedding representation of historical programming questions, dimensionality reduction is performed to obtain the embedding representation of historical questions. Based on the original embedding representation of historical programming answers, dimensionality reduction is performed to obtain the embedding representation of historical answers.

3. The programming knowledge tracing method based on code prediction enhancement according to claim 1, characterized in that: The method of predicting the probability of the subject answering the target programming question correctly based on the embedded representation of the target programming question, the programming answer prediction result, and the current knowledge state of the subject includes: Based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge status of the subject, feature splicing is performed to obtain the spliced ​​features; Input the concatenated features into the multi-layer perceptron, perform dimension compression through the multi-layer perceptron, and obtain the dimension-compressed features output by the multi-layer perceptron; After the input dimension is compressed, the features are fed into the activation function to obtain the probability of the correct answer output by the activation function.

4. The programming knowledge tracing method based on code prediction enhancement according to claim 1, characterized in that: Before predicting the probability of the subject's correct answer to the target programming question, it also includes: Based on the correctness of the source code input by the subject for the programming question at the current time step, the embedded representation of the programming answer corresponding to the programming question is projected to obtain the correctness information at the current time step; Based on the knowledge state of the tested object at the previous time step, the embedded representation of the programming problem at the current time step, and the right / wrong information at the current time step, the long short-term memory network (LSTM) is used to analyze the temporal evolution of the knowledge state and obtain the knowledge state of the tested object at the current time step as the current knowledge state of the tested object.

5. The programming knowledge tracing method based on code prediction enhancement according to any one of claims 1 to 4, characterized in that: The model used to extract the embedded representation is constructed based on a large language model and a linear dimensionality reduction module. The large language model is used to extract the original embedded representation, and the linear dimensionality reduction module is used to reduce the dimensionality of the original embedded representation. The linear dimensionality reduction module is trained through the following steps: Based on programming problem samples, programming answer samples, and sample labels, the evaluation simulator is trained by minimizing the binary cross entropy loss function; Among them, the sample label is used to indicate whether the programming answer sample can solve the programming question sample. The evaluation simulator is constructed by cascading linear dimensionality reduction modules and multi-layer perceptrons. The multi-layer perceptron is used to predict the probability that the programming answer can solve the programming question based on the embedding representation corresponding to the programming question and the embedding representation corresponding to the programming answer.

6. The programming knowledge tracing method based on code prediction enhancement according to claim 5, characterized in that: The multi-head attention mechanism model is trained by the following steps: Based on programming question samples, programming answer samples, and sample labels, a multi-head attention mechanism model is trained using the total loss function of code prediction. The total loss function for code prediction is determined by the following formula: ； ； ； ； in, represents the total loss function of code prediction, It is a hyperparameter. Different samples are numbered according to the time sequence of the answers. Indicates the +The embedding representation of the programming problem sample at 1 time step, Indicates the +The embedding representation of the programming answer sample at 1 time step, Indicates that based on the previous The embedding representation corresponding to the samples of time steps and , the programming answer prediction results predicted by the multi-head attention mechanism model, represents the sigmoid function, represents the inner product operation, express The corresponding correct code set item, express The corresponding error code set item, It is a triple A collection of represents the number of samples, Indicates the The labels corresponding to the samples at time steps, Indicates that the multilayer perceptron in the evaluation simulator is based on and , predicting the probability that the programming answer prediction result can solve the programming problem.

7. The programming knowledge tracing method based on code prediction enhancement according to claim 6, characterized in that: The prediction model used to predict the probability of the subject answering the target programming question correctly is obtained by training through the following steps: Based on programming problem samples, programming answer samples, and sample labels, a long short-term memory (LSTM) network and a prediction model are trained using a binary cross-entropy loss function. The long short-term memory (LSTM) network is used to obtain the knowledge state of the object under test at the current time step based on the knowledge state of the object under test at the previous time step, the embedded representation of the programming problem at the current time step, and the right or wrong information at the current time step, as the current knowledge state of the object under test. The prediction model is used to predict the probability of the subject correctly answering the target programming question based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The target programming question is the programming question at the next time step; Binary cross entropy loss function used for training long short-term memory networks (LSTMs) and prediction models It is determined by the following formula: ； ； ； in, Indicates that the prediction model is for The prediction results given by the programming problem samples under time steps, MLP represents multi-layer perceptron, Represents the current knowledge state of the object being tested given by the long short-term memory network LSTM, Represents a splicing operation.

8. A programming knowledge tracking device based on code prediction enhancement, characterized in that: include: An embedding representation extraction module is used to extract embedding representations based on historical programming questions and corresponding historical programming answers, obtaining the embedding representations of the historical questions and the embedding representations of the historical answers. The historical programming answers are the source code input by the test subject in response to the historical programming questions, and the source code is used to solve the programming questions. The code prediction module is used to predict the embedding representation of the source code input by the test subject for the target programming question based on the embedding representation of historical questions, the embedding representation of historical answers, and the embedding representation of the target programming question. The predicted source code embedding representation is used as the programming answer prediction result corresponding to the target programming question; The programming performance prediction module is used to predict the probability of the subject's correct answer to the target programming question based on the embedded representation of the target programming question, the programming answer prediction results, and the current knowledge state of the subject. The current knowledge state is based on the subject's historical programming answers and is obtained by analyzing the time-series evolution of the knowledge state.

9. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed on a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images