Student academic early warning prediction method based on feature interaction neural tree

Abstract

A student academic early warning and prediction method based on a feature interaction neural tree comprises the steps that firstly, various behavior data generated by students in a school are collected and arranged from a school system, a student behavior data set is constructed, minority class samples are generated through an improved SMOTE algorithm, and the problem of data set imbalance is relieved; secondly, coding the balanced data set through multivariate Gaussian coding, and standardizing the balanced data through layer normalization; inputting the processed data into a feature interaction neural tree module FINT Block, enhancing the feature representation capability, and fusing all DODT outputs by using a weighted average mechanism; and finally, in combination with residual connection, stacking a plurality of FINT Blocks to obtain an accurate and interpretable academic early warning prediction classification result.

Description

[0001] Technology Neighborhood

[0002] The present invention relates to the technical field of student academic early warning prediction, and in particular to a student academic early warning prediction method based on a feature interactive neural tree. Background Art

[0003] The rapid innovation of information technology has led to the accumulation of massive data and promoted the vigorous development of data mining technology. As one of the important application areas of data mining, educational data mining has gradually become a research hotspot. How to effectively use data mining technology to solve practical problems in student management and education and teaching has become an important topic in the field of education. Therefore, building a scientific and reasonable student academic early warning and prediction system not only occupies an important position in the teaching reform of colleges and universities, but also has significant academic value and practical guiding significance.

[0004] Traditional warning methods often rely on a single performance indicator, which makes it difficult to fully capture students' academic status. Except for the first time, existing warning methods for grade retention only consider the connection between students and courses when analyzing student behavior data. The analysis data is not comprehensive enough, and there is no in-depth exploration of the correlation and differences between different student samples. There are also problems such as class imbalance in the data set.

[0005] With the help of deep learning and data mining technology, multi-dimensional behavioral data (such as course grades, attendance rate, number of times participated in classroom activities, etc.) are analyzed, and an accurate risk prediction model for grade retention is established based on feature interaction, so as to provide students with early intervention support and improve the education quality and management efficiency of colleges and universities.

[0006] The patent application is a learning warning method based on an improved random forest algorithm (CN118505457A), which collects student data, pre-processes outliers and missing values, removes redundant and irrelevant features based on the Pearson correlation coefficient of the measurement variance, and then clusters to form N class centers, generates samples for training the model, and divides them into sample subsets of equal size. Each decision tree is trained based on a different sample subset, and N decision trees are generated from the sample set. Each decision tree is trained based on a different sample subset, iteratively updated separately, and voting weighted prediction is performed based on the information in the N trees; incremental learning is introduced to enable the performance prediction model to be gradually updated when new student performance data arrives, adapting to the dynamic changes in academic performance; in each round of iteration, new samples are gradually introduced into each tree to simulate the gradual integration of new knowledge into the learning process; based on the prediction results, the triggered warning is sent to teachers and students. This application uses a clustering method to generate class centers for the generation of training samples, but this method is sensitive to the selection of the number of clustering categories, which may lead to uneven distribution of class centers, thereby affecting the quality of training samples. In addition, the artificial samples generated by clustering cannot guarantee the characteristics of the data, and it is difficult to reflect the interaction between students' various behavioral data, which is not conducive to the model to achieve good predictive ability in scenarios with complex and changeable academic performance. Summary of the invention

[0007] In order to overcome the problems of the above-mentioned prior art, the purpose of the present invention is to propose a college academic early warning prediction method for campus behavior data, generate minority class samples by improving the SMOTE algorithm, alleviate the problem of data imbalance from the data level, use the feature encoding module to package the data into vectorized representations, solve the problem that the model is inefficient and difficult to capture global data patterns when facing heterogeneous features, use the feature interaction decision module to select important features, complete the feature information interaction between different features and different samples, realize classification decisions, use the residual fusion output module, fuse the outputs of all decision trees through the weighted average mechanism, and use the residual connection to fuse the feature outputs of each level, which has the advantages of low complexity and high accuracy.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] A student academic early warning prediction method based on feature interaction neural tree, firstly collects and organizes various behavioral feature data generated by students in school from the school system, constructs a student behavioral feature data set, generates minority class samples by improving the SMOTE algorithm, and alleviates the problem of unbalanced data set; secondly, encodes the balanced student behavioral feature data by multivariate Gaussian coding, and standardizes the encoded data by layer normalization; then inputs the processed data into the feature interaction neural tree module FINT Block, and the feature interaction neural tree module FINT Block strengthens the feature representation and fuses the decision through feature selection, self-attention, inter-sample attention and differentiable decision tree DODT to improve the classification performance; finally, multiple feature interaction neural tree modules FINT Block are stacked together, and the output of each Block is added to its input through residual connection to form the final feature representation. The stacked feature interaction neural tree modules FINTBlocks constitute an end-to-end deep network architecture, and finally obtain the academic early warning prediction classification result.

[0010] A student academic early warning prediction method based on a feature interactive neural tree specifically includes the following steps:

[0011] Step 1: construct a student behavior feature dataset and use the improved SMOTE algorithm to generate minority class samples to balance the dataset;

[0012] Step 2, encoding the data set balanced in step 1 by multivariate Gaussian coding, and standardizing the processed feature data by layer normalization;

[0013] Step 3: Input the feature data obtained by the standardized processing in step 2 into the feature interaction neural tree module FINTBlock. The feature interaction neural tree module FINT Block first selects features with high correlation with prediction through the feature selection module to reduce the interference of redundant features. Secondly, the selected feature vector is input into the feature interaction layer FIL, and self-attention is used to focus on the internal feature relationship of a single sample, and inter-sample attention is used to focus on the feature relationship between different samples. Then, classification decisions are completed in parallel through multiple differentiable decision trees DODT. Each differentiable decision tree DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all differentiable decision trees DODT are integrated into the final classification result through the weighted fusion mechanism.

[0014] Step 4: stack multiple feature interaction neural tree modules FINT Block together, and add the output of each Block to its input through residual connection to form the final feature representation, and finally obtain the academic warning prediction classification prediction result.

[0015] The step 1 uses the improved SMOTE algorithm to generate minority class samples to balance the data set, including:

[0016] Step 1.1 Calculate the relative neighborhood density of the sample: Assume that the given feature data set is D, which contains minority class samples D min and majority class samples D maj ; For the minority class sample x i ∈D min , calculate the relative neighborhood density:

[0017]

[0018] in, is x i The natural neighbor set of x in the data set i and x j , when x i is x j One of the k nearest neighbors of x j It is also x i One of the k nearest neighbors, then x j It's x i Natural neighbor of NN r (x i ) represents the sample x i The rth nearest neighbor set of ; Represents x i The number of natural neighbors, ψ(x i ,D) is the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x i ,x j ) is x i and neighbor x j The distance between is the relative neighborhood density, m is x i The number of similar neighbors (minority class) of x, n is i The number of heterogeneous neighbors (majority class) of ;

[0019] Step 1.2 Select the minority class samples whose relative neighborhood density is greater than the set threshold as the center point x center , and according to the center point x center and its neighboring samples, construct center A hypersphere is centered on , and new samples are generated inside the hypersphere through nonlinear transformation to balance the data distribution;

[0020] Assume that the selected sample x center is the center point, and the selected hypersphere surface point is x surface :

[0021]

[0022] x new =x center +R·x truncate (6)

[0023] Among them, x gen is to generate random points in the unit hypersphere, v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; according to the truncation parameter α trunc Adjust x gen The position of x truncate , x / / is x gen In x center to x surface Projection onto the unit vector; x new To x truncate Shift to x center As the center, the radius R = |x center -x surface The final generated new sample is obtained within the hypersphere of |.

[0024] The specific method of step 2 is:

[0025] Step 2: Using the learnable multivariate Gaussian kernel, for each Gaussian kernel k i , the balanced data set in step 1 is encoded by the formula, and the encoded feature data is standardized by layer normalization:

[0026]

[0027] Among them, u i is the calculated primary feature, input is the input feature data sample, μ i is the mean vector of the Gaussian kernel function, Σ i is the covariance matrix of the Gaussian kernel function; μ norm_i is the layer normalized mean, is the variance of layer normalization, i represents the i-th primary feature vector, j represents the j-th feature dimension in the feature vector, H represents the dimension of the primary feature vector, for u i The result after normalization of the j-th feature dimension of u i After normalizing each feature dimension of

[0028] 5. The specific method of step 3 is:

[0029] Step 3.1, input the primary feature vector processed in step 2 into the feature interaction neural tree module FINTBlock, in which a projection matrix P is first determined by the feature selection module, and each row of the projection matrix P corresponds to a unique hot vector; the projection matrix P is used to extract the primary feature vector Select a subset from the eigenvectors to form a new projected eigenvector Introduce a learnable matrix A and use the sparse function h(·) to transform the matrix A into an approximately sparse projection matrix.

[0030] Achieve effective approximation of sparse structures in continuous space;

[0031]

[0032] x p It is a sparse matrix For input features Projection is performed to obtain the selected feature vector, which is input into the feature interaction layer FIL;

[0033] Step 3.2, firstly, the self-attention of the feature interaction layer FIL in the feature interaction neural tree module FINTBlock is used to capture the correlation between different features within a single data sample, and the output Z of the self-attention is obtained by assigning different weights to different features. self ;

[0034] Q=X p W Q ,K=X p W X ,V=X p W V (12)

[0035]

[0036] Among them, X p ∈R b×d is the input batch data, b is the batch size, and n is the feature dimension; A self is the attention weight, Z self It is the feature representation output enhanced by the self-attention mechanism;

[0037] Next, we get the output Z of self-attention. self After that, the Intersample Attention is calculated, and the calculation formula is as follows:

[0038] Q=Z self W Q ,K=Z self WK ,V=Z self W V (14)

[0039]

[0040] X inter It is the feature representation after the attention processing between samples;

[0041] Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree DODT of the feature interaction neural tree module FINTBlock. The differentiable decision tree DODT consists of two layers of neural networks, and each node uses the Sigmoid function instead of the traditional step splitting function. inter Input into DODT, and split the input features through the learnable split threshold S:

[0042]

[0043] Among them, t(X inter ) is the classification prediction result of the sample, f(Z; Θ) is a two-layer neural network, b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network;

[0044] Step 3.3, in the feature interaction neural tree module FINT Block, the outputs of all DODTs are integrated into the final classification result through the weighted fusion mechanism; specifically, suppose there are N differentiable decision trees DODT, and the output of each tree is t i (X t ), the final prediction is expressed by the following formula:

[0045]

[0046] The specific method of step 4 is:

[0047] Step 4: Stack multiple feature interaction neural tree modules FINT Block through residual connections to alleviate the gradient disappearance problem and simplify the network training process:

[0048] X t+1 =RELU(O(X t )+X t ) (19)

[0049]

[0050] Among them, L means that there are L feature interaction neural tree modules FINT Block stacked together through residual connections, O(X t) is the result of weighted average fusion of multiple differentiable decision trees DODT, X t+1 is the output feature of the tth layer, and output represents the final academic warning prediction classification result;

[0051] The Cross-Entropy Loss function is used to optimize the parameters of the entire model, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the split threshold S in DODT, and the network parameters Θ of the two-layer neural network, and finally output the academic warning prediction classification results;

[0052]

[0053] Where N is the number of samples, C is the number of categories, and y ic is the true label of the i-th sample in category c, is the predicted probability of the i-th sample in category c.

[0054] The prediction system based on the above-mentioned early warning method includes:

[0055] The few-sample generation module is used in step 1 to select a few samples with larger density by calculating the relative neighborhood density, and synthesize new samples to balance the data distribution by performing nonlinear transformation inside a hypersphere centered on the selected sample and its neighbors;

[0056] The feature encoding module is used in step 2 to encode the balanced dataset through multivariate Gaussian encoding, map the feature space to a new space, and standardize the data using layer normalization;

[0057] The feature interaction neural tree module is used in step 3. The preprocessed data obtained in step 2 is input into FINTBlock. FINT Block includes the following steps: first, the most useful features for prediction are selected through the feature selection module, and the selected feature vectors are input into the feature interaction layer. Self-attention and inter-sample attention focus on the internal feature relationship of a single sample and the relationship between different samples respectively. The classification decision is completed in parallel through multiple DODTs. Each DODT performs independent feature splitting and prediction on the input features. Finally, the outputs of all DODTs are integrated into the final classification result through the weighted fusion mechanism.

[0058] The residual connection output module is used in step 4 to connect multiple FINT Blocks through residual connections to form the final feature representation, and finally obtain accurate and interpretable classification prediction results.

[0059] The prediction device based on the above-mentioned early warning method includes: a memory and a processor, wherein the memory stores a computer program so that the processor executes the student academic early warning prediction method based on the feature interactive neural tree.

[0060] A storage medium for receiving a program input by a user, wherein when the stored computer program is executed by a processor, a method for predicting student academic early warning based on a feature interactive neural tree can be used to obtain accurate academic early warning prediction classification results.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. The present invention effectively solves the problem that the noise of new samples synthesized by the few-sample generation algorithm is too large to meet the distribution of the original data. Compared with the existing learning warning method based on the improved random forest algorithm (CN118505457A) that uses the SMOTE (Synthetic Minority Over-sampling Technique) method for data enhancement, the present invention selects a few samples with larger density by calculating the relative neighborhood density in the few-sample generation module. The higher the density, the safer the data distribution area. This avoids the noise impact caused by the random selection of samples in the SMOTE algorithm and effectively reduces the generation of noise and abnormal samples. In addition, unlike the SMOTE algorithm that generates samples through linear interpolation, the present invention uses nonlinear transformation inside a hypersphere centered on the selected sample and its neighbors to increase the diversity of synthesized samples and improve the classification performance of minority classes.

[0063] 2. The present invention has an efficient feature processing module: compared with the existing learning warning method based on the improved random forest algorithm (CN118505457A) which is based on the feature screening process of measurement variance and Pearson correlation coefficient, the present invention encodes the data through multivariate Gaussian coding, maps the feature space to a new space, and uses layer normalization to standardize the data, which not only retains all the features of the data, but also can effectively capture the potential structure and distribution characteristics of the data.

[0064] 3. The present invention has an efficient feature interaction module: The present invention captures the nonlinear relationship between features and samples through the attention mechanism. On the one hand, the self-attention mechanism can mine the high-order interaction relationship between features and enhance the expressiveness of feature representation; on the other hand, the inter-sample attention mechanism captures the differences and commonalities between different samples by considering the similarities between samples, thereby improving the model's adaptability to complex data structures. This hybrid attention mechanism not only improves the nonlinear fitting ability of the model, but also enhances the generalization of the model, especially in heterogeneous and high-dimensional data.

[0065] 4. The present invention has the high efficiency of residual network structure: The present invention adopts the residual connection network structure, which effectively alleviates the gradient vanishing problem by providing a direct propagation path for the gradient. In addition, the residual connection achieves efficient fusion of features at different levels by superimposing the input and output, retaining more effective information and improving the overall performance of the model.

[0066] 5. The present invention has a wider range of application potential: by training on the constructed student behavior data set of the school, the model can learn multi-dimensional student behavior characteristics, effectively capture the complex relationship between student behavior and academic performance through the feature interaction module, and extract key behavior patterns in combination with the self-attention mechanism, which improves the nonlinear fitting ability of the model, which provides rich information for accurately identifying potential academic risks, and is particularly suitable for academic early warning analysis in a diverse context. This model has the advantages of high accuracy and robustness, and it has a wide range of application potential, helping teachers and counselors to effectively identify students with academic risks, and can achieve early detection, early warning, and early supervision, and timely intervention and assistance for such students to reduce their risk of repeating a grade.

[0067] The present invention focuses on using students' behavioral data to make academic early warning predictions through data mining and deep learning technology. First, the improved SMOTE algorithm is used to synthesize minority class samples and balance the data set, thereby alleviating the problem of inaccurate classification caused by class imbalance. In view of the heterogeneity of tabular data, the data is encoded by multivariate Gaussian coding, the feature space is mapped to a new space, and the data is standardized by layer normalization. For the balanced data set, the most relevant features are identified and selected by the feature selection module, and the nonlinear relationships between features and samples are captured by self-attention and inter-sample attention mechanisms, and classification decisions are made through differentiable decision trees. Finally, different feature interactions and decision modules are connected by residual connections, which realizes the efficient fusion of features at different levels and improves the classification prediction performance of the model.

[0068] In summary, the present invention adopts an optimized few-sample generation algorithm to alleviate the problem of data imbalance. The present invention effectively captures the nonlinear relationship between features and samples through a feature interaction neural tree model combined with self-attention and inter-sample attention mechanisms. Through the design of residual connections, the model can fully retain the information of different levels of input features while enhancing the modeling ability of complex relationships. This method shows excellent generalization ability in different data sets, without the need for personalized adjustments for specific data sets. By optimizing the correlation representation between features, the modeling ability of heterogeneous data is significantly improved, providing a more efficient and accurate solution for academic warning tasks, showing broad application potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A flowchart for academic early warning prediction for students.

[0070] Figure 2 This is the overall architecture diagram of the student academic warning prediction model based on feature interaction neural tree.

[0071] Figure 3 This is the optimized SMOTE algorithm framework diagram.

[0072] Figure 4 This is the framework diagram of the feature interaction neural tree module FINTBlock.

[0073] Figure 5 Diagram of the inter-sample attention computation framework. DETAILED DESCRIPTION

[0074] This example presents a student academic early warning prediction method based on feature interaction neural tree. The process is shown in Figure 1 The specific network structure framework adopted is as follows Figure 2 As shown:

[0075] Step 1: Construct a student behavior feature dataset and use the improved SMOTE algorithm to generate minority class samples to balance the dataset. Figure 3 As shown;

[0076] Step 1.1 Calculate the relative neighborhood density of the sample: Assume that the given feature data set is D, which contains minority class samples D min and majority class samples D maj ; For the minority class sample x i ∈D min , calculate the relative neighborhood density:

[0077]

[0078]

[0079] in, is x i The natural neighbor set of x in the data set i and x j , when x i is x j One of the k nearest neighbors of x j It is also x i One of the k nearest neighbors, then x j It's x i Natural neighbor of NN r (x i ) represents the sample x i The rth nearest neighbor set of ; Represents x iThe number of natural neighbors, ψ(x i ,D) is the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x i ,x j ) is x i and neighbor x j The distance between is the relative neighborhood density, m is x i The number of similar neighbors (minority class) of x, n is i The number of heterogeneous neighbors (majority class) of ;

[0080] Step 1.2 Select the minority class samples whose relative neighborhood density is greater than the set threshold as the center point x center , and according to the center point x center and its neighboring samples, construct center A hypersphere is centered on , and new samples are generated inside the hypersphere through nonlinear transformation to balance the data distribution;

[0081] Assume that the selected sample x center is the center point, and the selected hypersphere surface point is x surface :

[0082]

[0083] x new =x center +R·x truncate (6)

[0084] Among them, x gen is to generate random points in the unit hypersphere, v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; according to the truncation parameter α trunc Adjust x gen The position of x truncate , x / / is x gen In x center to x surface Projection onto the unit vector; x new To x truncate Shift to x center As the center, the radius R = |x center -x surface The final generated new sample is obtained within the hypersphere of |.

[0085] Step 2, encoding the data set balanced in step 1 by multivariate Gaussian coding, and standardizing the processed feature data by layer normalization;

[0086] Step 2: Use the learnable multivariate Gaussian kernel to calculate the optimal Gaussian kernel k for each Gaussian kernel. i , the balanced data set in step 1 is encoded by the formula, and the encoded feature data is standardized by layer normalization:

[0087]

[0088] Among them, u i is the calculated primary feature, input is the input feature data sample, μ i is the mean vector of the Gaussian kernel function, Σ i is the covariance matrix of the Gaussian kernel function; μ norm_i is the layer normalized mean, is the variance of layer normalization, i represents the i-th primary feature vector, j represents the j-th feature dimension in the feature vector, H represents the dimension of the primary feature vector, for u i The result after normalization of the j-th feature dimension of u i After normalizing each feature dimension of

[0089] Step 3: Input the feature data obtained by the standardized processing in step 2 into the feature interaction neural tree module FINTBlock, as shown in Figure 4 As shown in the figure, the feature interaction neural tree module FINT Block first selects the features with high correlation with prediction through the feature selection module to reduce the interference of redundant features. Secondly, the selected feature vector is input into the feature interaction layer FIL, and the self-attention is used to focus on the internal feature relationship of a single sample, and the inter-sample attention is used to focus on the feature relationship between different samples. Then, the classification decision is completed in parallel through multiple differentiable decision trees DODT. Each differentiable decision tree DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all differentiable decision trees DODT are integrated into the final classification result through the weighted fusion mechanism.

[0090] Step 3.1, input the primary feature vector processed in step 2 into the feature interaction neural tree module FINTBlock, in which a projection matrix P is first determined by the feature selection module, and each row of the projection matrix P corresponds to a unique hot vector; the projection matrix P is used to extract the primary feature vector Select a subset from the eigenvectors to form a new projected eigenvector A learnable matrix A is introduced, and the sparse function h(·) is used to transform the matrix A into an approximately sparse projection matrix, so as to effectively approximate the sparse structure in the continuous space.

[0091]

[0092] x p It is a sparse matrix For input features Projection is performed to obtain the selected feature vector, which is input into the feature interaction layer FIL;

[0093] Step 3.2, firstly, the self-attention of the feature interaction layer FIL in the feature interaction neural tree module FINTBlock is used to capture the correlation between different features within a single data sample, and the output Z of the self-attention is obtained by assigning different weights to different features. self ;

[0094] Q=X p W Q ,K=X p W X ,V=X p W V (12)

[0095]

[0096] Among them, X p ∈R b×d is the input batch data, b is the batch size, and n is the feature dimension; A self is the attention weight, Z self It is the feature representation output enhanced by the self-attention mechanism;

[0097] Next, we get the output Z of self-attention. self After that, the Intersample Attention is calculated, and the calculation formula is as follows:

[0098] Q=Z self W Q ,K=Z self W K ,V=Z self W V (14)

[0099]

[0100] X inter It is the feature representation after the attention processing between samples;

[0101] Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree DODT of the feature interaction neural tree module FINTBlock. The differentiable decision tree DODT consists of two layers of neural networks. Each node uses the Sigmoid function instead of the traditional step splitting function.inter Input into DODT, and split the input features through the learnable split threshold S:

[0102]

[0103] Among them, t(X inter ) is the classification prediction result of the sample, f(Z; Θ) is a two-layer neural network, b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network;

[0104] Step 3.3, in the feature interaction neural tree module FINT Block, the outputs of all DODTs are integrated into the final classification result through the weighted fusion mechanism; specifically, suppose there are N differentiable decision trees DODT, and the output of each tree is t i (X t ), the final prediction is expressed by the following formula:

[0105]

[0106] Step 4: stack multiple FINT Blocks together, and add the output of each Block to its input through a residual connection to form the final feature representation, and finally obtain the academic warning prediction classification prediction result:

[0107] By stacking multiple feature interaction neural tree modules FINT Block through residual connections, the gradient disappearance problem is alleviated and the network training process is simplified:

[0108] X t+1 =RELU(O(X t )+X t ) (19)

[0109]

[0110] Among them, L means that there are L feature interaction neural tree modules FINT Block stacked together through residual connections, O(X t ) is the result of weighted average fusion of multiple differentiable decision trees DODT, X t+1 is the output feature of the tth layer, and output represents the final academic warning prediction classification result;

[0111] The Cross-Entropy Loss function is used to optimize the parameters of the entire model, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the split threshold S in DODT, and the network parameters Θ of the two-layer neural network, and finally output the academic warning prediction classification results;

[0112]

[0113] Where N is the number of samples, C is the number of categories, and y ic is the true label of the i-th sample in category c, is the predicted probability of the i-th sample in category c.

[0114] Through experimental analysis, a 22-grade undergraduate behavior dataset was constructed. In order to solve the imbalance problem of the dataset, a few-sample generation algorithm was used to synthesize minority class samples to balance the data distribution. The balanced dataset was feature encoded by multivariate Gaussian coding, and then the preprocessed tabular data was standardized by layer normalization. The processed data was then input into the feature interaction neural tree module FINT Block. FINT Block strengthened the feature representation and integrated the decision through feature selection, self-attention, inter-sample attention and differentiable decision tree DODT to improve the classification performance. Finally, in order to further improve the model performance, multiple FINT Blocks were stacked together, and the output of each Block was added to its input through residual connection to form the final feature representation. The stacked FINT Blocks formed an end-to-end deep network architecture to achieve efficient and accurate classification prediction.

[0115] The operating system used in the experiment is Ubuntu 22.04.1, and the deep learning framework used is PyTorch. The specific configurations involved in the experiment are shown in Table 1.

[0116] Table 1 Experimental configuration table

[0117]

[0118] Through a series of detailed ablation experiments, the effectiveness of each innovative submodule proposed in the present invention has been fully verified, thereby significantly enhancing the feasibility and performance of the present invention. The following is an overview of these key experimental results:

[0119] On the original student behavior dataset, the study compared the classic Decision Tree model, DeepFM, TabNet, DANet, ARM-Net, MambaTab, NCART and other models in deep learning. The specific comparison results are shown in Table 2, where the best experimental results are bolded.

[0120] Table 2. Base network comparison experimental results

[0121]

[0122] Table 3 lists the different performances of the balanced data of different few-sample generation algorithms on the base model NCART. From the experimental results, it can be seen that the improved SMOTE algorithm performs well in the experiment of the few-sample generation algorithm. It has achieved the best performance in the three key indicators of Precision, Recall and Accuracy. In particular, compared with CTGAN, the accuracy rate has increased by about 27%, indicating the effectiveness of the improved SMOTE algorithm in improving the classification performance of the academic warning prediction model.

[0123] Table 3 Comparison of experimental results of different few-sample generation algorithms

[0124]

[0125]

[0126] Table 3 lists the impact of each module added in step 2 and step 3 on the model performance. Through ablation experimental analysis, the model combining the improved SMOTE algorithm, Gaussian coding module, and feature interaction module achieved the best performance in the four key indicators of Precision, Recall, F1, and Accuracy. The improved SMOTE algorithm alleviates the class imbalance problem and prevents the model from ignoring the features of minority class samples; the Gaussian coding module encapsulates the data into feature vectors to avoid the interference of heterogeneous data on the model; the feature interaction module uses self-attention to focus on the feature interaction of a single sample, and combines the attention between samples to perform cross-sample information fusion to enhance the feature representation capability. The experimental results confirm the effectiveness and necessity of these modules in improving model performance.

[0127] Table 4 Ablation experiment results

[0128]

[0129] It can be seen from the above simulation experiments and tables that compared with the prior art, the present invention demonstrates significant advantages and innovations in the direction of student academic early warning prediction through the innovative application of a few-sample generation algorithm, a multivariate Gaussian encoder, and a feature interaction module, and performance comparison with the latest model.

[0130] Verification experiments and performance advantages: Through sufficient ablation experiments, the effectiveness of the proposed innovative submodules is proved, which strengthens the feasibility and performance of the invention. Through comparative experiments with existing classification models such as TabNet-based methods, DeepFM-based methods, and DANet-based methods, the present invention demonstrates performance advantages on the constructed student behavior feature dataset, which means that the model achieves better results in student academic early warning prediction.

Claims

1. A student academic early warning prediction method based on feature interactive neural tree, characterized in that: Firstly, various behavioral characteristic data generated by students in school are collected and sorted from the school system to construct a student behavioral characteristic data set. The minority class samples are generated by improving the SMOTE algorithm to alleviate the problem of unbalanced data set. Secondly, the balanced student behavioral characteristic data are encoded by multivariate Gaussian coding, and the encoded data are standardized by layer normalization. Then the processed data are input into the feature interaction neural tree module FINT Block. The feature interaction neural tree module FINT Block strengthens the feature representation and fuses the decision through feature selection, self-attention, inter-sample attention and differentiable decision tree DODT to improve the classification performance. Finally, multiple feature interaction neural tree modules FINT Block are stacked together, and the output of each Block is added to its input through residual connection to form the final feature representation. The stacked feature interaction neural tree modules FINT Blocks constitute an end-to-end deep network architecture, and finally the academic warning prediction classification results are obtained.

2. According to claim 1, a student academic early warning prediction method based on feature interactive neural tree is characterized in that: The specific steps include: Step 1: construct a student behavior feature dataset and use the improved SMOTE algorithm to generate minority class samples to balance the dataset; Step 2, encoding the data set balanced in step 1 by multivariate Gaussian coding, and standardizing the processed feature data by layer normalization; Step 3: Input the feature data obtained by the standardized processing in step 2 into the feature interaction neural tree module FINT Block. The feature interaction neural tree module FINT Block first selects features with high correlation with prediction through the feature selection module to reduce the interference of redundant features. Secondly, the selected feature vector is input into the feature interaction layer FIL, and self-attention is used to focus on the internal feature relationship of a single sample, and inter-sample attention is used to focus on the feature relationship between different samples. Then, classification decisions are completed in parallel through multiple differentiable decision trees DODT. Each differentiable decision tree DODT performs feature splitting and prediction on the input features separately. Finally, the outputs of all differentiable decision trees DODT are integrated into the final classification result through the weighted fusion mechanism. Step 4: stack multiple feature interaction neural tree modules FINTBlock together, and add the output of each Block to its input through a residual connection to form the final feature representation, and finally obtain the academic warning prediction classification prediction result.

3. The student academic early warning prediction method based on feature interactive neural tree according to claim 2 is characterized in that: The step 1 uses the improved SMOTE algorithm to generate minority class samples to balance the data set, including: Step 1.1 Calculate the relative neighborhood density of the sample: Assume that the given feature data set is D, which contains minority class samples D min and majority class samples D maj ; For the minority class sample x i ∈D min , calculate the relative neighborhood density: in, is x i The natural neighbor set of x in the data set i and x j , when x i is x j One of the k nearest neighbors of x j It is also x i One of the k nearest neighbors, then x j It's x i Natural neighbor of NN r (x i ) represents the sample x i The rth nearest neighbor set of ; Represents x i The number of natural neighbors, ψ(x i ,D) is the absolute natural density, reflecting the sample x i The density in the natural neighborhood, d(x i ,x j ) is x i and neighbor x j The distance between is the relative neighborhood density, m is x i The number of neighbors of the same type (minority class), n is x i The number of heterogeneous neighbors (majority class) of ; Step 1.2 Select the minority class samples whose relative neighborhood density is greater than the set threshold as the center point x center , and according to the center point x center and its neighboring samples, construct center A hypersphere is centered on , and new samples are generated inside the hypersphere through nonlinear transformation to balance the data distribution; Assume that the selected sample x center is the center point, and the selected hypersphere surface point is x surface : Among them, x gen is to generate random points in the unit hypersphere, v is a vector sampled from a normal distribution, r is a random number sampled from a uniform distribution, and p is the dimension of the feature space; according to the truncation parameter α trunc Adjust x gen The position of x truncate , x / / is x gen In x center to x surface Projection onto the unit vector; x new To x truncate Shift to x center As the center, the radius R = |x center -x surface The final generated new sample is obtained within the hypersphere of |.

4. The student academic early warning prediction method based on feature interactive neural tree according to claim 2 is characterized in that: The specific method of step 2 is: Using learnable multivariate Gaussian kernels, for each Gaussian kernel k i , the balanced data set in step 1 is encoded by the formula, and the encoded feature data is standardized by layer normalization: Among them, u i is the calculated primary feature, input is the input feature data sample, μ i is the mean vector of the Gaussian kernel function, Σ i is the covariance matrix of the Gaussian kernel function; μ norm_i is the layer normalized mean, is the variance of layer normalization, i represents the i-th primary feature vector, j represents the j-th feature dimension in the feature vector, H represents the dimension of the primary feature vector, for u i The result after normalization of the j-th feature dimension of u i After normalizing each feature dimension of 5. The student academic early warning prediction method based on feature interactive neural tree according to claim 2 is characterized in that: The specific method of step 3 is: Step 3.1, input the primary feature vector processed in step 2 into the feature interaction neural tree module FINT Block, in which a projection matrix P is first determined by the feature selection module, and each row of the projection matrix P corresponds to a unique hot vector; the projection matrix P is used to extract the primary feature vector Select a subset from the eigenvectors to form a new projected eigenvector Introduce a learnable matrix A and use sparse functions Convert the matrix A into an approximately sparse projection matrix to achieve effective approximation of the sparse structure in the continuous space; x p It is a sparse matrix For input features Projection is performed to obtain the selected feature vector, which is input into the feature interaction layer FIL; Step 3.2, firstly, the self-attention of the feature interaction layer FIL in the feature interaction neural tree module FINTBlock is used to capture the correlation between different features within a single data sample, and the output Z of the self-attention is obtained by assigning different weights to different features. self ; Q=X p W Q ,K=X p W X ,V=X p W V (12) Among them, X p ∈R b×d is the input batch data, b is the batch size, and n is the feature dimension; A self is the attention weight, Z self It is the feature representation output enhanced by the self-attention mechanism; Next, we get the output Z of self-attention. self After that, the Intersample Attention is calculated, and the calculation formula is as follows: Q=Z slf W Q ,K=Z self W K ,V=Z self W V (14) X inter It is the feature representation after the attention processing between samples; Finally, the feature representation enhanced by the self-attention mechanism is input into the differentiable decision tree DODT of the feature interaction neural tree module FINT Block. The differentiable decision tree DODT consists of two layers of neural networks, and each node uses the Sigmoid function instead of the traditional step splitting function. inter Input into DODT, and split the input features through the learnable split threshold S: Among them, t(X inter ) is the classification prediction result of the sample, f(Z; Θ) is a two-layer neural network, b1, b2 are all learnable network parameters, and ReLU is the activation function of the neural network; Step 3.3, in the feature interaction neural tree module FINT Block, the outputs of all DODTs are integrated into the final classification result through the weighted fusion mechanism; specifically, suppose there are N differentiable decision trees DODT, and the output of each tree is t i (X t ), the final prediction is expressed by the following formula:

6. The student academic early warning prediction method based on feature interactive neural tree according to claim 2 is characterized in that: The specific method of step 4 is: By stacking multiple feature interaction neural tree modules FINT Block through residual connections, the gradient disappearance problem is alleviated and the network training process is simplified: X t+1 =CLOCK(O(X t )+X t ) ( 19 Among them, L means that there are L feature interaction neural tree modules FINT Block stacked together through residual connections, O(X t ) is the result of weighted average fusion of multiple differentiable decision trees DODT, X t+1 is the output feature of the tth layer, and output represents the final academic warning prediction classification result; The Cross-Entropy Loss function is used to optimize the parameters of the entire model, including the feature selection matrix P, the query of the attention mechanism in FIL, the parameters of the key and value matrices, the split threshold S in DODT, and the network parameters Θ of the two-layer neural network, and finally output the academic warning prediction classification results; Where N is the number of samples, C is the number of categories, and y ic is the true label of the i-th sample in category c, is the predicted probability of the i-th sample in category c.

7. A prediction system based on the method according to any one of claims 1 to 6, characterized in that: include: The few-sample generation module is used in step 1 to select a few samples with larger density by calculating the relative neighborhood density, and synthesize new samples to balance the data distribution by performing nonlinear transformation inside a hypersphere centered on the selected sample and its neighbors; The feature encoding module is used in step 2 to encode the balanced dataset through multivariate Gaussian encoding, map the feature space to a new space, and standardize the data using layer normalization; The feature interaction neural tree module is used in step 3 to input the preprocessed data obtained in step 2 into the FINT Block. The FINT Block includes the following steps: first, the most useful features for prediction are selected through the feature selection module, and the selected feature vectors are input into the feature interaction layer. Self-attention and inter-sample attention focus on the internal feature relationship of a single sample and the relationship between different samples respectively, and the classification decision is completed in parallel through multiple DODTs. Each DODT performs independent feature splitting and prediction on the input features. Finally, the outputs of all DODTs are integrated into the final classification result through the weighted fusion mechanism. The residual connection output module is used in step 4 to connect multiple FINT Blocks through residual connection to form the final feature representation, and finally accurately obtain the academic warning prediction classification prediction results.

8. A prediction device based on the method according to any one of claims 1 to 6, characterized in that: include: A memory and a processor, wherein the memory stores a computer program so that the processor executes the student academic warning prediction method based on a feature interactive neural tree as described in any one of claims 1 to 7.

9. A storage medium for receiving user input programs, wherein when the stored computer program is executed by a processor, it is able to obtain accurate academic warning prediction classification results based on the student academic warning prediction method based on the feature interactive neural tree as described in any one of claims 1 to 6.

Images