A Student Academic Early Warning and Prediction Method Based on Feature Interaction Neural Tree

Abstract

The student academic performance early warning prediction method based on feature interaction neural tree is as follows: First, various behavioral data generated by students in school are collected and organized from the school system to construct a student behavior dataset. The minority class samples are generated by improving the SMOTE algorithm to alleviate the problem of dataset imbalance. Second, the balanced dataset is encoded by multivariate Gaussian coding and standardized by layer normalization. Then, the processed data is input into the feature interaction neural tree module FINT Block to enhance the feature representation capability, and all DODT outputs are fused using a weighted average mechanism. Finally, multiple FINT Blocks are stacked by combining residual connections to obtain accurate and interpretable academic performance early warning prediction classification results.

Description

[0001] Technology Neighborhood

[0002] This invention relates to the field of student academic performance early warning and prediction technology, and in particular to a student academic performance early warning and prediction method based on feature interaction neural tree. Background Technology

[0003] The rapid innovation of information technology has spurred the accumulation of massive amounts of data, driving the vigorous development of data mining technology. As a crucial application area of ​​data mining, educational data mining has gradually become a research hotspot. How to effectively utilize data mining techniques to solve practical problems in student management and education has become an important issue in the field of education. Therefore, constructing a scientific and reasonable student academic performance early warning and prediction system not only occupies an important position in higher education teaching reform but also has significant academic value and practical guiding significance.

[0004] Traditional early warning methods often rely on a single performance indicator, making it difficult to comprehensively capture students' academic status. In addition, existing methods for preventing grade repetition only consider the relationship between students and courses when analyzing student behavior data, resulting in incomplete data analysis, failure to deeply explore the correlations and differences between different student samples, and class imbalance in the dataset.

[0005] By leveraging deep learning and data mining technologies, multi-dimensional behavioral data (such as course grades, attendance, and participation in classroom activities) are analyzed. Based on feature interactions, a precise model for predicting the risk of repeating a grade is established, thereby providing early intervention support for students and improving the quality of education and management efficiency in universities.

[0006] Patent application CN118505457A discloses a learning early warning method based on an improved random forest algorithm. This method collects student data, preprocesses outliers and missing values, removes redundant and irrelevant features based on measurement variance and Pearson correlation coefficient, then clusters the data to form N class centers, generating samples for training the model. These samples are divided into equal-sized subsets, and each decision tree is trained based on a different subset. N decision trees are generated from the sample set, and each tree is trained on a different subset, iteratively updated individually, and weighted predictions are made by combining information from the N trees. Incremental learning is introduced, enabling the performance prediction model to be gradually updated as new student performance data arrives, adapting to dynamic changes in academic performance. In each iteration, new samples are gradually introduced into each tree, simulating the gradual integration of new knowledge into the learning process. Based on the prediction results, alerts are sent to teachers and students. While this application uses a clustering method to generate class centers for training samples, this method is sensitive to the number of clusters, potentially leading to uneven distribution of class centers and affecting the quality of the training samples. Furthermore, artificial samples generated by clustering cannot guarantee the characteristics of the data and are difficult to reflect the interaction between various student behavioral data, which is not conducive to the model achieving good predictive ability in scenarios with complex and ever-changing academic performance. Summary of the Invention

[0007] To overcome the problems of the existing technology, the present invention proposes a college academic performance early warning prediction method based on campus behavior data. This method improves the SMOTE algorithm to generate minority class samples, alleviating the data imbalance problem at the data level. It utilizes a feature encoding module to divide the data into vectorized representations, solving the problems of low efficiency and difficulty in capturing global data patterns when the model faces heterogeneous features. A feature interaction decision module selects important features, completing the interaction of feature information between different features and different samples to achieve classification decisions. Finally, a residual fusion output module fuses the outputs of all decision trees through a weighted average mechanism and uses residual connections to fuse the feature outputs at each level. This method has the advantages of low complexity and high accuracy.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A student academic performance prediction method based on feature-interactive neural trees (FINTBlocks) is proposed. First, various behavioral feature data generated by students in school are collected and organized from the school system to construct a student behavioral feature dataset. Minority class samples are generated by improving the SMOTE algorithm to alleviate the dataset imbalance problem. Second, the balanced student behavioral feature data is encoded using multivariate Gaussian coding, and the encoded data is standardized using layer normalization. Then, the processed data is input into the FINTBlock feature interaction module. The FINTBlock strengthens the feature representation and fuses decisions through feature selection, self-attention, inter-sample attention, and differentiable decision trees (DODT), improving classification performance. Finally, multiple FINTBlocks are stacked together, and the output of each block is added to its input through residual connections to form the final feature representation. The stacked FINTBlocks constitute an end-to-end deep network architecture, ultimately yielding the academic performance prediction classification result.

[0010] A student academic performance early warning prediction method based on feature interaction neural tree, specifically including the following steps:

[0011] Step 1: Construct a student behavior feature dataset and use the improved SMOTE algorithm to generate minority class samples to balance the dataset;

[0012] Step 2: Encode the balanced dataset from Step 1 using multivariate Gaussian coding, and standardize the processed feature data using layer normalization.

[0013] Step 3: Input the standardized feature data obtained in Step 2 into the Feature Interaction Neural Tree Module FINTBlock. The Feature Interaction Neural Tree Module FINTBlock first selects features with high relevance to prediction through the feature selection module to reduce the interference of redundant features. Then, it inputs the selected feature vectors into the Feature Interaction Layer FIL, which uses self-attention to focus on the internal feature relationships of a single sample and inter-sample attention to focus on the feature relationships between different samples. Then, it completes the classification decision in parallel through multiple differentiable decision trees DODT. Each differentiable decision tree DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all differentiable decision trees DODT are integrated into the final classification result through a weighted fusion mechanism.

[0014] Step 4: Stack multiple feature interaction neural tree modules (FINT Blocks) together. The output of each block is added to its input through residual connections to form the final feature representation, and finally obtain the academic warning prediction classification prediction result.

[0015] Step 1 uses an improved SMOTE algorithm to generate minority class samples to balance the dataset, including:

[0016] Step 1.1 Calculate the relative neighborhood density of the samples: Let the given feature dataset be D, which contains minority class samples D min and majority class sample D maj For minority class samples x i ∈D min Calculate the relative neighborhood density:

[0017]

[0018] in, It is x i The set of natural neighbors for a sample x in the dataset. i and x j When x i It is x j One of the k nearest neighbors of x, and x j It is also x i If x is one of the k nearest neighbors, then x j It's x i Natural neighbors; NN r (x i ) represents sample x i The set of the r-th nearest neighbors; x represents i The number of natural neighbors, ψ(x) i D) represents the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x) i ,x j ) is x i and neighbor x j The distance between them; It is the relative neighborhood density, where m is x i The number of neighbors of the same class (minority class) of x, where n is the number of neighbors of x. i The number of out-of-class neighbors (majority class);

[0019] Step 1.2 Select minority class samples with a relative neighborhood density greater than a set threshold as center points x. center And based on the center point x center And its nearest neighbor samples, constructing a system based on x center A hypersphere centered on the data is used to generate new samples within the hypersphere through nonlinear transformations in order to balance the data distribution.

[0020] Let the selected sample be x. center Centered on the hypersphere, a point on the surface of the hypersphere is selected as x. surface :

[0021]

[0022] x new =x center +R·x truncate (6)

[0023] Where, x gen Random points are generated within a unit hypersphere, where v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; based on the truncation parameter α... trunc Adjust x gen The position is obtained by x truncate x / / It is x gen In x center To x surface Projection onto a unit vector; x new To make x truncate Translate to x center Centered on, radius R = |x center -x surface The final new sample is obtained within the hypersphere.

[0024] The specific method for step 2 is as follows:

[0025] Step 2: Utilize learnable multivariate Gaussian kernels, for each Gaussian kernel k... i The dataset balanced in step 1 is encoded using a formula, and the encoded feature data is standardized using layer normalization:

[0026]

[0027] Among them, u i The input is the calculated primary feature, and the input is the input feature data sample, μ. i Let Σ be the mean vector of the Gaussian kernel function. i μ is the covariance matrix of the Gaussian kernel function. norm_i The mean of the normalized layer, Let be the variance of the layer normalization, i represent the i-th primary feature vector, j represent the j-th feature dimension in the feature vector, and H represent the dimension of the primary feature vector. For u i The result after normalization of the j-th feature dimension, for u i After normalizing each feature dimension, we get

[0028] 5. The specific method for step 3 is as follows:

[0029] Step 3.1: Input the primary feature vector processed in Step 2 into the feature interaction neural tree module FINTBlock. In the FINTBlock module, a projection matrix P is first determined by the feature selection module. Each row of the projection matrix P corresponds to a one-hot vector. The projection matrix P is used to select from the primary feature vector... Select a subset from the set to form a new projected feature vector. We introduce a learnable matrix A and use the sparse function h(·) to transform matrix A into an approximately sparse projection matrix.

[0030] Achieve efficient approximation of sparse structures in continuous space;

[0031]

[0032] x p It is through a sparse matrix For input features Project the selected feature vectors and input them into the feature interaction layer FIL.

[0033] Step 3.2: First, the self-attention layer (FIL) in the FINTBlock feature interaction neural tree module captures the correlation between different features within a single data sample. By assigning different weights to different features, the output Z of the self-attention is obtained. self ;

[0034] Q = X p W Q K = X p W X V = X p W V (12)

[0035]

[0036] Among them, X p ∈R b×d For the input batch data, b is the batch size, and n is the feature dimension; A self For attention weights, Z self It is the feature representation output enhanced by the self-attention mechanism;

[0037] Next, we obtain the self-attention output Z. self Next, the intersample attention is calculated using the following formula:

[0038] Q = Z self W Q K = Z self WK V = Z self W V (14)

[0039]

[0040] X inter It is the feature representation after inter-sample attention processing;

[0041] Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree (DODT) of the feature interaction neural tree module FINTBlock. The DODT consists of two layers of neural networks, with each node using the sigmoid function instead of the traditional step split function; X inter The input features are split into DODTs using a learnable splitting threshold S:

[0042]

[0043] Wherein, t(X) inter ) represents the classification prediction result of the sample, and f(Z; Θ) is a two-layer neural network. b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network;

[0044] Step 3.3: In the Feature Interaction Neural Tree (FINT) Block, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism. Specifically, there are N differentiable decision trees (DODTs), and the output of each tree is t. i (X t If the final prediction is calculated using the following formula:

[0045]

[0046] The specific method for step 4 is as follows:

[0047] Step 4: By stacking multiple feature interaction neural tree modules (FINT Blocks) through residual connections, the vanishing gradient problem is alleviated and the network training process is simplified.

[0048] X t+1 =RELU(O(X) t )+X t (19)

[0049]

[0050] Where L represents L feature interaction neural tree modules (FINT Blocks) stacked together through residual connections, O(X tX is the result obtained by weighted averaging and fusion of multiple differentiable decision trees (DODT). t+1 is the output feature of the t-th layer, where output represents the final academic warning prediction classification result;

[0051] The parameters of the entire model are optimized using the Cross-Entropy Loss function, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the splitting threshold S in DODT, and the network parameters Θ of the two-layer neural network. Finally, the academic warning prediction classification result is output.

[0052]

[0053] Where N is the number of samples, C is the number of categories, and y ic It is the true label of the i-th sample in category c. It is the predicted probability of the i-th sample in category c.

[0054] The prediction system based on the above early warning method includes:

[0055] The few-sample generation module is used in step 1. By calculating the relative neighborhood density, it selects a small number of samples with higher density and synthesizes new samples to balance the data distribution by performing a nonlinear transformation inside a hypersphere centered on the selected samples and their nearest neighbors.

[0056] The feature encoding module, used in step 2, encodes the balanced dataset using multivariate Gaussian encoding, maps the feature space to a new space, and uses layer normalization to standardize the data.

[0057] The Feature Interaction Neural Tree module is used in step 3. It inputs the preprocessed data obtained in step 2 into the FINTBlock. The FINTBlock includes the following steps: First, the feature selection module selects the most useful features for prediction and inputs the selected feature vectors into the feature interaction layer. Self-attention and inter-sample attention focus on the internal feature relationship of a single sample and the relationship between different samples, respectively. The classification decision is completed in parallel by multiple DODTs. Each DODT performs independent feature splitting and prediction on the input features. Finally, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism.

[0058] The residual connection output module is used in step 4 to connect multiple FINT Blocks through residual connection, thereby forming the final feature representation and finally obtaining an accurate and interpretable classification prediction result.

[0059] The prediction device based on the above-mentioned early warning method includes: a memory and a processor, wherein the memory stores a computer program that causes the processor to execute the student academic early warning prediction method based on feature interaction neural tree.

[0060] A storage medium for receiving user input programs, wherein the stored computer programs, when executed by a processor, can obtain accurate academic warning prediction and classification results based on a feature-interactive neural tree-based student academic warning prediction method.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. This invention effectively solves the problem of high noise levels in newly synthesized samples that do not conform to the original data distribution, as addressed by few-sample generation algorithms. Compared to an existing learning warning method (CN118505457A) based on an improved random forest algorithm that uses the SMOTE (Synthetic Minority Over-sampling Technique) for data augmentation, this invention selects a minority of samples with higher density by calculating the relative neighborhood density in the few-sample generation module. Higher density indicates that the sample is located in a safer region of the data distribution, avoiding the noise impact of random sample selection in the SMOTE algorithm and effectively reducing the generation of noise and outlier samples. Furthermore, unlike the SMOTE algorithm which generates samples through linear interpolation, this invention utilizes nonlinear transformations within a hypersphere centered on the selected sample and its nearest neighbors, improving the diversity of synthesized samples and enhancing minority class classification performance.

[0063] 2. This invention features a highly efficient feature processing module: Compared with an existing learning warning method based on an improved random forest algorithm (CN118505457A) that uses measurement variance and Pearson correlation coefficient for feature screening, this invention encodes the data using multivariate Gaussian coding, mapping the feature space to a new space, and uses layer normalization to standardize the data, thus preserving all the features of the data while effectively capturing the potential structure and distribution characteristics of the data.

[0064] 3. This invention features a highly efficient feature interaction module: This invention captures nonlinear relationships between features and between samples through an attention mechanism. On one hand, the self-attention mechanism can uncover higher-order interactions between features, enhancing the expressive power of feature representations; on the other hand, the inter-sample attention mechanism, by considering the similarity between samples, captures the differences and commonalities between different samples, thereby improving the model's adaptability to complex data structures. This hybrid attention mechanism not only improves the model's nonlinear fitting ability but also enhances its generalization ability, especially showing stronger performance in heterogeneous and high-dimensional data.

[0065] 4. This invention possesses the high efficiency of residual network structures: The invention employs a residual connection network structure, which effectively alleviates the gradient vanishing problem by providing a direct propagation path for gradients. Furthermore, residual connections achieve efficient fusion of features at different levels by superimposing inputs and outputs, preserving more effective information and improving the overall performance of the model.

[0066] 5. This invention has broader application potential: By training on a constructed dataset of student behavior from our school, the model can learn multi-dimensional student behavioral characteristics. Through a feature interaction module, it effectively captures the complex relationship between student behavior and academic performance. Furthermore, by combining a self-attention mechanism to extract key behavioral patterns, it enhances the model's non-linear fitting ability. This provides rich information for accurately identifying potential academic risks, making it particularly suitable for academic early warning analysis in diverse contexts. This model boasts high accuracy and robustness, possessing broad application potential. It helps teachers and counselors effectively identify students with academic risks, enabling early detection, early warning, and early supervision, allowing for timely intervention and assistance to reduce their risk of repeating a grade.

[0067] This invention focuses on using student behavioral data for academic performance prediction through data mining and deep learning techniques. First, an improved SMOTE algorithm is used to synthesize minority class samples, balancing the dataset and mitigating classification inaccuracies caused by class imbalance. To address the heterogeneity of tabular data, multivariate Gaussian coding is used to encode the data, mapping the feature space to a new space, and layer normalization is applied to standardize the data. In the balanced dataset, a feature selection module identifies and selects the most relevant features, utilizing self-attention and inter-sample attention mechanisms to capture nonlinear relationships between features and samples, and a differentiable decision tree is used for classification decisions. Finally, residual connections are used to link different feature interactions and decision modules, achieving efficient fusion of features at different levels and improving the model's classification prediction performance.

[0068] In summary, this invention employs an optimized few-shot generation algorithm to alleviate the data imbalance problem. Through a feature-interactive neural tree model, combined with self-attention and inter-sample attention mechanisms, it effectively captures the nonlinear relationships between features and samples. By designing residual connections, the model can fully preserve information from different levels of input features while enhancing the modeling ability of complex relationships. This method demonstrates excellent generalization ability across different datasets without requiring personalized adjustments for specific datasets. By optimizing the association representation between features, it significantly improves the modeling ability for heterogeneous data, providing a more efficient and accurate solution for academic early warning tasks and showcasing broad application potential. Attached Figure Description

[0069] Figure 1 A flowchart for student academic performance early warning and prediction.

[0070] Figure 2 This is a diagram of the overall architecture of a student academic performance early warning and prediction model based on a feature-interactive neural tree.

[0071] Figure 3 This is a diagram of the optimized SMOTE algorithm framework.

[0072] Figure 4 The FINTBlock framework diagram is shown for the feature interaction neural tree module.

[0073] Figure 5 A framework diagram for calculating inter-sample attention. Detailed Implementation

[0074] This example demonstrates a student academic performance early warning and prediction method based on a feature-interactive neural tree. The process is as follows: Figure 1 The specific network structure framework it adopts is as follows: Figure 2 As shown:

[0075] Step 1: Construct a student behavior feature dataset. Use an improved SMOTE algorithm to generate minority class samples to balance the dataset. Figure 3 As shown;

[0076] Step 1.1 Calculate the relative neighborhood density of the samples: Let the given feature dataset be D, which contains minority class samples D min and majority class sample D maj For minority class samples x i ∈D min Calculate the relative neighborhood density:

[0077]

[0078]

[0079] in, It is x i The set of natural neighbors for a sample x in the dataset. i and x j When x i It is x j One of the k nearest neighbors of x, and x j It is also x i If x is one of the k nearest neighbors, then x j It's x i Natural neighbors; NN r (x i ) represents sample x i The set of the r-th nearest neighbors; x represents iThe number of natural neighbors, ψ(x) i D) represents the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x) i ,x j ) is x i and neighbor x j The distance between them; It is the relative neighborhood density, where m is x i The number of neighbors of the same class (minority class) of x, where n is the number of neighbors of x. i The number of out-of-class neighbors (majority class);

[0080] Step 1.2 Select minority class samples with a relative neighborhood density greater than a set threshold as center points x. center And based on the center point x center And its nearest neighbor samples, constructing a system based on x center A hypersphere centered on the data is used to generate new samples within the hypersphere through nonlinear transformations in order to balance the data distribution.

[0081] Let the selected sample be x. center Centered on the hypersphere, a point on the surface of the hypersphere is selected as x. surface :

[0082]

[0083] x new =x center +R·x truncate (6)

[0084] Where, x gen Random points are generated within a unit hypersphere, where v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; based on the truncation parameter α... trunc Adjust x gen The position is obtained by x truncate x / / It is x gen In x center To x surface Projection onto a unit vector; x new To make x truncate Translate to x center Centered on, radius R = |x center -x surface The final new sample is obtained within the hypersphere.

[0085] Step 2: Encode the balanced dataset from Step 1 using multivariate Gaussian coding, and standardize the processed feature data using layer normalization.

[0086] Step 2 utilizes learnable multivariate Gaussian kernels, for each Gaussian kernel k i The dataset balanced in step 1 is encoded using a formula, and the encoded feature data is standardized using layer normalization:

[0087]

[0088] Among them, u i The input is the calculated primary feature, and the input is the input feature data sample, μ. i Let Σ be the mean vector of the Gaussian kernel function. i μ is the covariance matrix of the Gaussian kernel function. norm_i The mean of the normalized layer, Let be the variance of the layer normalization, i represent the i-th primary feature vector, j represent the j-th feature dimension in the feature vector, and H represent the dimension of the primary feature vector. For u i The result after normalization of the j-th feature dimension, for u i After normalizing each feature dimension, we get

[0089] Step 3: Input the standardized feature data obtained in Step 2 into the Feature Interaction Neural Tree module FINTBlock, such as... Figure 4 As shown, the Feature Interaction Neural Tree (FINT) Block first selects features with high relevance to the prediction through a feature selection module to reduce the interference of redundant features. Then, the selected feature vectors are input into the Feature Interaction Layer (FIL), which uses self-attention to focus on the internal feature relationships of a single sample and inter-sample attention to focus on the feature relationships between different samples. Then, multiple Differentiable Decision Trees (DODTs) are used to complete the classification decision in parallel. Each DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism.

[0090] Step 3.1: Input the primary feature vector processed in Step 2 into the feature interaction neural tree module FINTBlock. In the FINTBlock module, a projection matrix P is first determined by the feature selection module. Each row of the projection matrix P corresponds to a one-hot vector. The projection matrix P is used to select from the primary feature vector... Select a subset from the set to form a new projected feature vector. We introduce a learnable matrix A and use the sparse function h(·) to transform matrix A into an approximately sparse projection matrix, thereby achieving an effective approximation of sparse structures in continuous space.

[0091]

[0092] x p It is through a sparse matrix For input features Project the selected feature vectors and input them into the feature interaction layer FIL.

[0093] Step 3.2: First, the self-attention layer (FIL) in the FINTBlock feature interaction neural tree module captures the correlation between different features within a single data sample. By assigning different weights to different features, the output Z of the self-attention is obtained. self ;

[0094] Q = X p W Q K = X p W X V = X p W V (12)

[0095]

[0096] Among them, X p ∈R b×d For the input batch data, b is the batch size, and n is the feature dimension; A self For attention weights, Z self It is the feature representation output enhanced by the self-attention mechanism;

[0097] Next, we obtain the self-attention output Z. self Next, the intersample attention is calculated using the following formula:

[0098] Q = Z self W Q K = Z self W K V = Z self W V (14)

[0099]

[0100] X inter It is the feature representation after inter-sample attention processing;

[0101] Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree (DODT) of the feature interaction neural tree module FINTBlock. The DODT consists of two layers of neural networks, with each node using the sigmoid function instead of the traditional step split function; Xinter The input features are split into DODTs using a learnable splitting threshold S:

[0102]

[0103] Wherein, t(X) inter ) represents the classification prediction result of the sample, and f(Z; Θ) is a two-layer neural network. b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network;

[0104] Step 3.3: In the Feature Interaction Neural Tree (FINT) Block, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism. Specifically, there are N differentiable decision trees (DODTs), and the output of each tree is t. i (X t If the final prediction is calculated using the following formula:

[0105]

[0106] Step 4: Stack multiple FINT Blocks together. The output of each Block is added to its input through a residual connection to form the final feature representation, ultimately obtaining the academic warning prediction classification result.

[0107] By stacking multiple feature-interacting neural tree modules (FINT Blocks) using residual connections, the vanishing gradient problem is mitigated and the network training process is simplified.

[0108] X t+1 =RELU(O(X) t )+X t (19)

[0109]

[0110] Where L represents L feature interaction neural tree modules (FINT Blocks) stacked together through residual connections, O(X t X is the result obtained by weighted averaging and fusion of multiple differentiable decision trees (DODT). t+1 is the output feature of the t-th layer, where output represents the final academic warning prediction classification result;

[0111] The parameters of the entire model are optimized using the Cross-Entropy Loss function, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the splitting threshold S in DODT, and the network parameters Θ of the two-layer neural network. Finally, the academic warning prediction classification result is output.

[0112]

[0113] Where N is the number of samples, C is the number of categories, and y ic It is the true label of the i-th sample in category c. It is the predicted probability of the i-th sample in category c.

[0114] Experimental analysis was conducted using a constructed dataset of behavioral data from all undergraduate students in the 2022 cohort. To address the imbalance problem in the dataset, a few-shot generation algorithm was employed to synthesize minority class samples to balance the data distribution. The balanced dataset was then feature-encoded using multivariate Gaussian encoding, followed by layer normalization to standardize the preprocessed tabular data. The processed data was then input into the Feature Interaction Neural Tree (FINT) Block. The FINT Block enhances feature representation and fuses decisions through feature selection, self-attention, inter-sample attention, and Differentiable Decision Tree (DODT), improving classification performance. Finally, to further enhance model performance, multiple FINT Blocks were stacked together. The output of each block was added to its input via residual connections to form the final feature representation. These stacked FINT Blocks constitute an end-to-end deep network architecture, achieving efficient and accurate classification prediction.

[0115] The operating system used in the experiment was Ubuntu 22.04.1, and the deep learning framework used was PyTorch. The specific configurations involved in the experiment are shown in Table 1.

[0116] Table 1 Experimental Configuration Table

[0117]

[0118] Through a series of detailed ablation experiments, the effectiveness of each innovative sub-module proposed in this invention has been fully verified, thereby significantly enhancing the feasibility and performance of this invention. The following is an overview of these key experimental results:

[0119] On the original student behavior dataset, the study compared classic Decision Tree models with deep learning models such as DeepFM, TabNet, DANet, ARM-Net, MambaTab, and NCART. The specific comparison results are shown in Table 2, with the best-performing experimental results highlighted in bold.

[0120] Table 2. Comparison Experiment Results of Base Networks

[0121]

[0122] Table 3 lists the different performances of the balanced data from different few-shot generation algorithms on the base model NCART. The experimental results show that the improved SMOTE algorithm performs well in the few-shot generation algorithm experiment. It achieves the best performance in the three key indicators of precision, recall and accuracy. In particular, compared with CTGAN, the accuracy is improved by about 27%, which shows the effectiveness of the improved SMOTE algorithm in improving the classification performance of the academic warning prediction model.

[0123] Table 3 compares the experimental results of different few-shot generation algorithms.

[0124]

[0125]

[0126] Table 3 lists the impact of each module added in steps 2 and 3 on model performance. Through ablation analysis, the model combining the improved SMOTE algorithm, Gaussian coding module, and feature interaction module achieved optimal performance across the four key metrics of Precision, Recall, F1, and Accuracy. The improved SMOTE algorithm alleviates the class imbalance problem, preventing the model from neglecting to learn features from minority class samples; the Gaussian coding module encapsulates data into feature vectors, avoiding interference from heterogeneous data; the feature interaction module utilizes self-attention to focus on the feature interactions of individual samples and combines inter-sample attention for cross-sample information fusion, enhancing feature representation capabilities. Experimental results confirm the effectiveness and necessity of these modules in improving model performance.

[0127] Table 4 Ablation Experiment Results

[0128]

[0129] As can be seen from the simulation experiments and tables above, compared with the existing technology, the present invention demonstrates significant advantages and innovations in the direction of student academic performance early warning prediction through the innovative application of the few-sample generation algorithm, multivariate Gaussian encoder, and feature interaction module, and the performance comparison with the latest model.

[0130] Validation experiments and performance advantages: Extensive ablation experiments demonstrate the effectiveness of the proposed innovative submodule, reinforcing the feasibility and performance of the invention. Comparative experiments with existing classification models such as TabNet-based, DeepFM-based, and DANet-based methods showcase the performance advantages on the constructed student behavior feature dataset, implying that this model achieves better results in predicting student academic performance.

Claims

1. A student academic performance early warning and prediction method based on feature-interactive neural trees, characterized in that, Specifically, the following steps are included: Step 1: Construct a student behavior feature dataset, including class attendance rate, academic performance, online time, and number of times entering the library. Use the improved SMOTE algorithm to generate minority class samples to balance the dataset. Step 1.1 Calculate the relative neighborhood density of the sample; Step 1.2 Select minority class samples with a relative neighborhood density greater than a set threshold as center points x. center And based on the center point x center And its nearest neighbor samples, constructing a system based on x center A new sample is generated inside a hypersphere centered on the supersphere through nonlinear transformation; Step 2: Encode the balanced dataset from Step 1 using multivariate Gaussian coding, and standardize the processed feature data using layer normalization. Step 3: The feature data obtained from the standardization process in Step 2 is input into the Feature Interaction Neural Tree Module (FINT Block). The FINT Block first selects features with high relevance to the prediction through the feature selection module to reduce the interference of redundant features. Then, the selected feature vectors are input into the Feature Interaction Layer (FIL). Self-attention is used to focus on the internal feature relationships of a single sample, and inter-sample attention is used to focus on the feature relationships between different samples. Then, multiple Differentiable Decision Trees (DODTs) are used to complete the classification decision in parallel. Each DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism. Step 4 involves stacking multiple feature interaction neural tree modules (FINT Blocks) together. The output of each block is added to its input via residual connections to form the final feature representation, ultimately yielding the academic warning prediction classification result. Specifically, this involves stacking multiple feature interaction neural tree modules (FINT Blocks) via residual connections to alleviate the vanishing gradient problem and simplify the network training process. X t+1 =CLOCK(O(X t )+X t ) Where L represents L feature interaction neural tree modules (FINT Blocks) stacked together through residual connections, O(X t X is the result obtained by weighted averaging and fusion of multiple differentiable decision trees (DODT). t+1 is the output feature of the t-th layer, where output represents the final academic warning prediction classification result; The parameters of the entire model are optimized using the Cross-Entropy Loss function, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the splitting threshold S in DODT, and the network parameters Θ of the two-layer neural network. Finally, the academic warning prediction classification result is output. Where N is the number of samples, C is the number of categories, and y ic It is the true label of the i-th sample in category c. It is the predicted probability of the i-th sample in category c.

2. The student academic performance early warning and prediction method based on feature-interactive neural trees according to claim 1, characterized in that, Step 1 uses an improved SMOTE algorithm to generate minority class samples to balance the dataset, including: Step 1.1 Calculate the relative neighborhood density of the samples: Let the given feature dataset be D, which contains minority class samples D min and majority class sample D maj For minority class samples x i ∈D min Calculate the relative neighborhood density: in, It is x i The set of natural neighbors for a sample x in the dataset. i and x j When x i It is x j One of the k nearest neighbors of x, and x j It is also x i If x is one of the k nearest neighbors, then x j It's x i Natural neighbors; NN r (x i ) represents sample x i The set of the r-th nearest neighbors; x represents i The number of natural neighbors, ψ(x) i D) represents the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x) i ,x j ) is x i and neighbor x j The distance between them; It is the relative neighborhood density, where m is x i The number of neighbors of the same class, i.e., the minority class, where n is the number of neighbors of x. i The number of dissimilar neighbors, i.e., the number of majority class neighbors; Step 1.2 Select minority class samples with a relative neighborhood density greater than a set threshold as center points x. center And based on the center point x center And its nearest neighbor samples, constructing a system based on x center A hypersphere centered on the data is used to generate new samples within the hypersphere through nonlinear transformations in order to balance the data distribution. Let the selected sample be x. center Centered on the hypersphere, a point on the surface of the hypersphere is selected as x. surface : x new =x center +R·x truncate Where, x gen Random points are generated within a unit hypersphere, where v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; based on the truncation parameter α... trunc Adjust x gen The position is obtained by x truncate x / / It is x gen In x center To x surface Projection onto a unit vector; x new To make x truncate Translate to x center Centered on, radius R = |x center -x surface The final new sample is obtained within the hypersphere.

3. The student academic performance early warning and prediction method based on feature-interactive neural trees according to claim 1, characterized in that, The specific method for step 2 is as follows: Using learnable multivariate Gaussian kernels, for each Gaussian kernel k i The dataset balanced in step 1 is encoded using a formula, and the encoded feature data is standardized using layer normalization: Among them, u i The input is the calculated primary feature, and the input is the input feature data sample, μ. i Let Σ be the mean vector of the Gaussian kernel function. i μ is the covariance matrix of the Gaussian kernel function. norm_i The mean of the normalized layer. Let be the variance of the layer normalization, i represent the i-th primary feature vector, j represent the j-th feature dimension in the feature vector, and H represent the dimension of the primary feature vector.

4. The student academic performance early warning and prediction method based on feature-interactive neural trees according to claim 1, characterized in that, The specific method for step 3 is as follows: Step 3.1: Input the primary feature vector processed in Step 2 into the Feature Interaction Neural Tree Module (FINT Block). In the FINT Block, a projection matrix P is first determined by the feature selection module. Each row of the projection matrix P corresponds to a one-hot vector. The projection matrix P is used to select from the primary feature vector... Select a subset from the set to form a new projected feature vector. For u i After normalizing each feature dimension, we get We introduce a learnable matrix A and use the sparse function h(·) to transform matrix A into an approximately sparse projection matrix, thereby achieving an effective approximation of sparse structures in continuous space. x p It is through a sparse matrix For input features Project the selected feature vectors and input them into the feature interaction layer FIL. Step 3.2: First, the self-attention layer (FIL) in the FINTBlock feature interaction neural tree module captures the correlation between different features within a single data sample. By assigning different weights to different features, the output Z of the self-attention is obtained. self ; Q=X p W Q ,K=X p W k ,V=X p W V Among them, X p ∈R b×d For the input batch data, b is the batch size, and d is the feature dimension; A self For attention weights, Z self It is the feature representation output enhanced by the self-attention mechanism; Next, we obtain the self-attention output Z. self Next, the intersample attention is calculated using the following formula: Q=Z self W Q ,K=Z self W K ,V=Z self W V X inter It is the feature representation after inter-sample attention processing; Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree (DODT) of the feature interaction neural tree module FINT Block. The DODT consists of two layers of neural networks, with each node using the sigmoid function σ instead of the traditional step splitting function; X inter The input features are split into DODTs using a learnable splitting threshold S: Wherein, t(X) inter ) represents the classification prediction result of the sample, and f(Z; Θ) is a two-layer neural network. b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network; Step 3.3: In the Feature Interaction Neural Tree (FINT) Block, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism. Specifically, there are N differentiable decision trees (DODTs), and the output of each tree is t. i (X t If the final prediction is calculated using the following formula:

5. A prediction system based on the method of any one of claims 1 to 4, characterized in that, include: The few-sample generation module is used in step 1. By calculating the relative neighborhood density, it selects a small number of samples with higher density and synthesizes new samples to balance the data distribution by performing a nonlinear transformation inside a hypersphere centered on the selected samples and their nearest neighbors. The feature encoding module, used in step 2, encodes the balanced dataset using multivariate Gaussian encoding, maps the feature space to a new space, and uses layer normalization to standardize the data. The Feature Interaction Neural Tree module is used in step 3. It inputs the preprocessed data obtained in step 2 into the FINT Block. The FINT Block includes the following steps: First, the feature selection module selects the most useful features for prediction and inputs the selected feature vectors into the feature interaction layer. Self-attention and inter-sample attention focus on the internal feature relationship of a single sample and the relationship between different samples, respectively. The classification decision is completed in parallel by multiple DODTs. Each DODT performs independent feature splitting and prediction on the input features. Finally, the outputs of all DODTs are integrated into the final classification result through a weighted fusion mechanism. The residual connection output module is used in step 4 to connect multiple FINT Blocks through residual connection, thereby forming the final feature representation and finally obtaining the accurate academic warning prediction classification prediction result.

6. A prediction device based on the method of any one of claims 1 to 4, characterized in that, include: A memory and a processor, the memory storing a computer program that causes the processor to execute the student academic performance early warning and prediction method based on feature-interactive neural trees as described in any one of claims 1 to 4.

7. A user input program storage medium, wherein the stored computer program, when executed by a processor, is capable of obtaining accurate academic warning prediction classification results based on the student academic warning prediction method based on feature interaction neural tree as described in any one of claims 1 to 4.

Images