Mama state space model-based student performance prediction method and system
By employing a lightweight sequence modeling method based on the Mamba state-space model, the problem of balancing model performance and computational efficiency in student performance prediction under small data scenarios is solved, achieving efficient and robust student performance prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TIANHE COLLEGE GUANGDONG POLYTECHNIC NORMAL UNIV
- Filing Date
- 2025-12-26
- Publication Date
- 2026-05-08
AI Technical Summary
Existing technologies struggle to balance model performance and computational efficiency in small data scenarios. Linear models are unable to capture nonlinear interactions, tree models are prone to overfitting, and deep models have redundant parameters and high computational complexity, leading to unstable student performance predictions and wasted computational resources.
A lightweight sequence modeling method based on the Mamba state space model is adopted. Through feature embedding and serialization, the Mamba state space encoding module is used to capture feature interactions, and a fully connected neural network is combined for regression prediction to build a lightweight student performance prediction system.
It improves prediction accuracy on small datasets, reduces parameter size and computational complexity, avoids overfitting, and achieves robust training and efficient student performance prediction.
Smart Images
Figure CN121997291A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and more specifically, to a method and system for predicting student performance based on the Mamba state-space model. Background Technology
[0002] Existing methods for predicting student performance mainly fall into two categories: traditional machine learning models and deep learning models. Traditional methods, such as linear regression and logistic regression, as well as tree models like decision trees and random forests, have been widely adopted. These methods are easy to implement on small to medium-sized datasets and have a certain degree of interpretability, but they have limitations when modeling complex relationships. In recent years, some research has also attempted to apply deep models (including models based on self-attention mechanisms such as Transformer) to student performance prediction tasks using tabular data, hoping to leverage the powerful representation learning capabilities of deep learning to improve prediction accuracy. However, the data available in the education field is often limited in scale and feature dimensions, resulting in a small number of training samples. Faced with this "small data" scenario, the existing methods mentioned above all have shortcomings: linear models, due to their simple assumptions, are difficult to capture the non-linear interaction between features, resulting in limited prediction performance; tree models are prone to overfitting when the amount of data is small, and the increase in model complexity also leads to increased computational overhead; while large deep models such as Transformer perform excellently in big data tasks, their large parameter scale makes them prone to overfitting on small datasets, resulting in parameter redundancy and unstable training process. At the same time, due to their high computational complexity, they are difficult to achieve performance improvements commensurate with their overhead in tasks such as educational prediction.
[0003] Therefore, how to balance model performance and computational efficiency in small data scenarios has become a pressing technical problem to be solved in student performance prediction. Summary of the Invention
[0004] Therefore, in order to overcome the problem of balancing model performance and computational efficiency in small data scenarios, the present invention provides a student performance prediction method and system based on the Mamba state-space model, the specific technical solution of which is as follows: A student performance prediction method based on the Mamba state-space model includes the following steps: The system acquires original feature data of students from multiple sources, preprocesses and vectorizes the original feature data, maps different types of features into feature vectors of a unified dimension based on a feature type awareness strategy, and obtains feature sequences by combining learnable positional encoding weights. Construct a Mamba state space model, input the feature sequence into the Mamba state space model, process the feature sequence through the Mamba state space model, and obtain the hidden state vector containing comprehensive information of all features. Construct a regression prediction model, input the hidden state vector into the regression prediction model, and obtain the student performance prediction results.
[0005] The student performance prediction method based on the Mamba state-space model effectively captures the complex interactions between features through sequence modeling, thus achieving better prediction accuracy. It significantly reduces parameter size and computational complexity, making the model less prone to overfitting on small datasets and more robust in training. This overcomes the problem of balancing model performance and computational efficiency in small data scenarios faced by existing technologies.
[0006] Preferably, the specific method for processing the feature sequence includes the following steps: The Mamba state-space encoding module performs hidden state updates and information aggregation on the feature sequence step by step, generating a hidden state vector containing comprehensive information of all features at the end of the feature sequence. The Mamba state space model includes a Mamba state space encoding module.
[0007] Preferably, the acquisition of the feature sequence specifically includes calculating the feature representation of the fused location information based on the following formula. :
[0008] Where k is the position index of the feature in the sequence. Let be the embedding vector of the k-th feature. The position encoding vector for the k-th position. These are learnable weight parameters used to dynamically adjust the contribution of positional encoding to feature representation; Feature representation of all fused location information The feature sequence is constructed by arranging the features in a predetermined order.
[0009] Preferably, the specific method for obtaining student performance prediction results includes the following steps: A fully connected neural network layer is used as the output prediction head. The hidden state vector is input into the fully connected neural network layer, and the student performance prediction result is obtained through activation function or linear transformation.
[0010] Preferably, the specific method for obtaining the feature sequence includes the following steps: Select the appropriate encoding method based on the type of student original feature data from multiple sources, and perform feature encoding preprocessing. Obtain the embedding weights and embedding biases, and map the preprocessed multi-source student original feature data according to the embedding weights and embedding biases to obtain the feature sequence.
[0011] A student performance prediction system based on the Mamba state-space model, used to implement the aforementioned student performance prediction method based on the Mamba state-space model, includes: The feature embedding and serialization module is used to acquire original feature data of students from multiple sources, and to preprocess and vectorize the original feature data of students from multiple sources. Based on the feature type awareness strategy, different types of features are mapped into feature vectors of a unified dimension, and feature sequences are obtained by combining learnable positional encoding weights. The Mamba state-space model is used to process feature sequences and obtain hidden state vectors containing comprehensive information about all features. A regression prediction model is used to obtain student performance prediction results based on the hidden state vector.
[0012] Preferably, the Mamba state-space model includes: The Mamba state-space encoding module is used to progressively update the hidden state and aggregate information in the feature sequence, generating a hidden state vector containing comprehensive information about all features at the end of the feature sequence.
[0013] Preferably, the feature embedding and serialization module includes: The embedding module is used to embed the original feature data of students from multiple sources, converting different types of features into vector representations of a unified dimension. The permutation module is used to arrange all features in a predetermined order to construct a feature sequence.
[0014] Preferably, the regression prediction model includes: The fully connected neural network layer is used to normalize the hidden state vector to obtain the student performance prediction results. Attached Figure Description
[0015] The invention will be further understood from the following description taken in conjunction with the accompanying drawings. The components in the drawings are not necessarily drawn to scale, but rather the emphasis is on illustrating the principles of the embodiments. In different views, the same reference numerals designate corresponding parts.
[0016] Figure 1 This is a schematic diagram of the overall process of a student performance prediction method based on the Mamba state-space model in one embodiment of the present invention; Figure 2 This is a schematic diagram showing an overview of the Mambular-EDM model structure in one embodiment of the present invention; Figure 3 This is a flowchart illustrating a feature sequence construction method in one embodiment of the present invention; Figure 4 This is a schematic diagram of the internal structure of the Mamba module in one embodiment of the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to its embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention.
[0018] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly attached to the other element or there may be an intervening element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only possible implementation.
[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0020] In this invention, "first" and "second" do not represent a specific quantity or order, but are merely used to distinguish names.
[0021] Before describing the specific embodiments of the present invention, a brief introduction to the prior art will be given first.
[0022] With the development of educational informatization and educational data mining (EDM), student performance prediction has become an important research direction in the field of education. Researchers analyze multi-source information such as students' historical grades, learning behaviors, and online learning data, and use data mining and machine learning techniques to predict students' future academic performance or learning outcomes. These predictive models help to identify academic risks early and provide personalized teaching interventions, thus having wide application value in educational assessment and teaching management.
[0023] Existing methods for predicting student performance mainly fall into two categories: traditional machine learning models and deep learning models. Traditional methods, such as linear regression and logistic regression, as well as tree models like decision trees and random forests, have been widely adopted. These methods are easy to implement on small to medium-sized datasets and have a certain degree of interpretability, but they have limitations when modeling complex relationships. In recent years, some research has also attempted to apply deep models (including models based on self-attention mechanisms such as Transformer) to student performance prediction tasks using tabular data, hoping to leverage the powerful representation learning capabilities of deep learning to improve prediction accuracy. However, the data available in the education field is often limited in scale and feature dimensions, resulting in a small number of training samples. Faced with this "small data" scenario, the existing methods mentioned above all have shortcomings: linear models, due to their simple assumptions, are difficult to capture the non-linear interaction between features, resulting in limited prediction performance; tree models are prone to overfitting when the amount of data is small, and the increase in model complexity also leads to increased computational overhead; while large deep models such as Transformer perform excellently in big data tasks, their large parameter scale makes them prone to overfitting on small datasets, resulting in parameter redundancy and unstable training process. At the same time, due to their high computational complexity, they are difficult to achieve performance improvements commensurate with their overhead in tasks such as educational prediction.
[0024] In summary, the main shortcomings and technical challenges of existing technologies include the following: 1. How to balance model performance and computational efficiency in small-data scenarios. In other words, with limited data scale, it is necessary to maximize the accuracy of student performance prediction while avoiding overly complex models that lead to low training and inference efficiency.
[0025] 2. Insufficient Modeling of Inter-Feature Interactions: Existing methods are inadequate in modeling complex relationships between features. Linear models can only express linear combinations and are insufficient to characterize potential non-linear dependencies between features; even tree models or simple feature combination methods cannot fully capture high-order feature interactions, limiting the model's ability to uncover the intrinsic patterns in educational data.
[0026] 3. Complex models are unstable when trained on educational data: Data sets in the education field are usually limited in number and have a lot of noise. Complex models such as deep neural networks are prone to problems such as non-convergence, overfitting or unstable gradients when trained on such small data, resulting in large fluctuations in model performance and difficulty in achieving stable improvement.
[0027] 4. The Transformer architecture has low cost-effectiveness in educational tasks: Self-attention models such as the Transformer have high computational complexity and require massive amounts of data to fully realize their advantages. In small-scale data tasks such as predicting student performance, directly using the Transformer is often costly with limited performance improvement, making it difficult to demonstrate high cost-effectiveness.
[0028] Based on this, this invention proposes a Mambular-EDM framework, introducing a novel Mamba state-space model into the student performance prediction task. This framework, through a lightweight sequence modeling structure, effectively balances model prediction performance and computational efficiency under conditions of small sample educational data. Specific technical means and methods are as follows: like Figure 1 As shown, a student performance prediction method based on the Mamba state-space model in one embodiment of the present invention includes the following steps: S1. Obtain original feature data of students from multiple sources, and preprocess and vectorize the original feature data of students from multiple sources. Based on the feature type awareness strategy, map different types of features into feature vectors of the same dimension, and obtain feature sequences by combining learnable positional encoding weights.
[0029] As a preferred technical solution, the specific method for obtaining feature sequences includes the following steps: embedding the original feature data of students from multiple sources, converting different types of features into vector representations of a unified dimension, arranging all features in a predetermined order, and constructing a feature sequence.
[0030] As a preferred technical solution, the acquisition of the feature sequence specifically includes calculating the feature representation of the fused location information based on the following formula. :
[0031] Where k is the position index of the feature in the sequence. Let be the embedding vector of the k-th feature. The position encoding vector for the k-th position. These are learnable weight parameters used to dynamically adjust the contribution of location encoding to feature representation; they fuse all feature representations containing location information. The feature sequence is constructed by arranging the features in a predetermined order.
[0032] Specifically, this invention first embeds the students' original feature data, converting different types of features into vector representations of a unified dimension. For example, student grades, homework completion rates, class attendance rates, and participation in extracurricular activities are mapped into fixed-length vectors through an embedding layer. Then, all features are arranged in a predetermined order to construct a feature sequence input model. By treating features as a sequence, the model can process originally independent multidimensional features from a sequence modeling perspective, thus providing a suitable input format for the subsequent state-space model. This embedding and serialization strategy helps unify the representation of numerical and categorical features and creates conditions for capturing the sequential relationships and mutual influences between features.
[0033] S2. Construct a Mamba state space model. Input the feature sequence into the Mamba state space model and process the feature sequence through the Mamba state space model to obtain a hidden state vector containing comprehensive information about all features.
[0034] As a preferred technical solution, the specific method for processing the feature sequence includes the following steps: the feature sequence is progressively updated with hidden states and information is aggregated through the Mamba state space encoding module, and a hidden state vector containing comprehensive information of all features is generated at the end of the feature sequence; wherein, the Mamba state space model includes the Mamba state space encoding module.
[0035] Specifically, after the feature sequence is prepared, the core Mamba state-space model is used to encode the sequence. This invention utilizes Mamba as a sequence modeling module to progressively update the state and aggregate information of the input feature sequence.
[0036] Mamba is a Selective State-Space Model (SSL) that enables recursive processing of the entire sequence with time complexity increasing linearly with sequence length. Specifically, the Mamba state-space encoding module receives the current feature vector and the previous state at each sequence step, using its internal state selection mechanism to determine how to update the internal state representation. This mechanism allows the model to selectively retain or forget previous state information based on the current input, thus transmitting key information throughout the feature sequence. Unlike the Transformer, which contains a large number of parameters, the Mamba state-space encoding module does not rely on self-attention or deep multilayer perceptrons; it achieves efficient modeling of long sequences solely through state-space recursion, resulting in a more streamlined model structure.
[0037] For student performance prediction tasks, the Mamba state-space encoding module can capture the temporal dependencies and interactions between features, while avoiding the instability of training large models on small data. Its linear time complexity and parameter efficiency ensure the model's efficiency in processing long feature sequences, enabling the method of this invention to run efficiently in educational scenarios with limited computing resources.
[0038] S3. Construct a regression prediction model by inputting the hidden state vector into the regression prediction model to obtain the student performance prediction results.
[0039] As a preferred technical solution, the specific method for obtaining student performance prediction results includes the following steps: using a fully connected neural network layer as the output prediction head, inputting the hidden state vector into the fully connected neural network layer, and obtaining the student performance prediction results through an activation function or linear transformation.
[0040] Specifically, after encoding by the Mamba state-space encoding module, a hidden state vector containing comprehensive feature information is generated at the end of the sequence. This invention sets a lightweight regression prediction output layer after the Mamba encoding layer to map the hidden state to the desired student performance prediction result. In practice, a fully connected neural network layer can be used as the output prediction head, inputting the final hidden state and outputting a real value representing the prediction of the student's target performance (e.g., predicted score or pass probability) through an activation function or linear transformation.
[0041] Because the entire model is an end-to-end design, the parameters of the embedding layer, Mamba encoding layer, and regression prediction output layer are jointly optimized using training data. During training, historical student data is used as a basis to minimize the error (such as mean squared error) between the predicted and true values, thereby adjusting the model parameters. Through this simple and efficient output structure, this invention can directly use the deep feature representation obtained by Mamba encoding for regression prediction, achieving accurate estimation of student performance.
[0042] The Mambular-EDM model of this invention exhibits excellent scalability and adaptability in its structural design. On the one hand, compared to traditional linear and tree models, this method effectively captures the complex interactions between features through sequence modeling, thus achieving better performance in prediction accuracy. On the other hand, compared to deep models based on large networks such as Transformers, this invention significantly reduces parameter size and computational complexity, making the model less prone to overfitting and more robust in training on small datasets. Simultaneously, this framework has the advantage of modular design, making it easy to extend according to needs: for example, if more student features are introduced in the future, this method can accommodate them by extending the feature sequence or increasing the embedding dimension; if the data scale increases, the model capacity can also be increased by increasing the number of Mamba layers or the state dimension. Therefore, Mambular-EDM performs exceptionally well in small-sample educational data tasks, but can also smoothly scale to larger-scale data scenarios, possessing broad application potential and flexibility.
[0043] Through the aforementioned techniques and methods, this invention constructs an efficient student performance prediction method based on the Mamba state-space model, overcoming the challenge of balancing model performance and computational efficiency in small-data scenarios faced by existing technologies. This method is simple in structure, easy to implement, and possesses good generalization and scalability, making it highly suitable for small-scale data prediction tasks in the education field.
[0044] In summary, the student performance prediction method based on the Mamba state-space model effectively captures the complex interactions between features through sequence modeling, thus achieving better prediction accuracy. It significantly reduces parameter size and computational complexity, making the model less prone to overfitting on small datasets and more robust in training. This overcomes the problem of balancing model performance and computational efficiency in existing technologies in small data scenarios.
[0045] In one embodiment, other state-space models, such as the S4 structured state-space model, can be used instead of the Mamba algorithm.
[0046] S4 and other similar models also demonstrate high efficiency in long sequence modeling, exhibiting similar hidden-state recursive structures and linear time complexity. When a mature state-space model already exists in a specific application, or when the model needs to be adjusted for specific data characteristics, replacing Mamba with S4 or other subsequently improved state-space models can be considered. This replacement will not change the overall framework, but may require re-tuning parameters to suit the characteristics of the new model, achieving performance comparable to that of Mamba.
[0047] For feature sequence input, a direct feature input method using a conventional multilayer perceptron (MLP) can be used instead.
[0048] When the sequential correlation between features is weak, or for the purpose of further simplifying the model structure, features can be treated as a sequence instead of a sequence, and all features can be directly input as a vector into a traditional MLP network for prediction. This alternative eliminates the sequence modeling process, resulting in a simpler model structure. However, this may result in the loss of information about the sequential interactions between features. Therefore, this approach is suitable for situations where features are independent or their sequential relationship can be ignored, and can be considered as an alternative when a minimalist model is required.
[0049] In addition, structural parameters such as the number of layers and the dimension of hidden states in the model can be flexibly adjusted according to task requirements.
[0050] The framework of this invention allows for adjustments to model complexity to adapt to different data sizes and problem complexities. For example, when student features are few and the dataset is extremely small, a single-layer Mamba algorithm and low-dimensional hidden states can be used to reduce model complexity and prevent overfitting. Conversely, with larger datasets or feature sets, multiple Mamba encoding layers can be stacked or the state dimension of each layer can be increased to enhance the model's expressive power. By adjusting these structural parameters, support for different application scenarios can be achieved without changing the core architecture, ensuring an optimal balance between performance and efficiency.
[0051] In one embodiment, the specific method for obtaining the feature sequence includes the following steps: selecting the appropriate encoding method according to the type of the original feature data of multi-source students, and performing feature encoding preprocessing; obtaining the embedding weights and embedding bias terms. According to the embedding weight And the embedded bias term is used to map the preprocessed multi-source student original feature data to obtain feature sequences.
[0052] For example, according to the function Obtain the feature sequence. Among them, . This represents the k-th student feature value in the original input (e.g., math score = 85, subject category = "science"). This represents the feature preprocessing result, which is the result of preprocessing the same encoding of features after dynamically selecting the corresponding encoding method according to the feature type. The learnable embedding weights have a dimension equal to the original feature dimension multiplied by the embedding dimension, and are used to map features to a unified vector space. Embedding bias term. Used to enhance the expressive power of the model. This is a feature type-aware function, typically a dynamic non-linear activation function, used to dynamically adjust the normalization strategy. Let be the representation of the k-th feature in the embedding space.
[0053] Specifically, for numerical features (such as grades and attendance), the min-max normalization method is used for unified feature encoding to eliminate differences in units and retain relative size information; for categorical features (such as subjects and classes), one-hot encoding is used to convert them into sparse vectors to avoid spurious order relationships caused by numerical conversion.
[0054] In educational data, the same feature can have different importance in different scenarios (e.g., the predictive contribution of "number of extracurricular activities" differs for arts and science students). If the input is a numerical feature, Apply differentiable quantization functions: This enhances the model's sensitivity to small changes in feature values; if the input is a categorical feature, Applying piecewise linear mappings: Automatically learns the decision boundaries between categories. The learnable threshold.
[0055] Thus, through Normalized features can be mapped to a semantic space of uniform dimension (e.g., d=64) to eliminate differences in feature dimensions. In the embedding space, the model can learn implicit relationships between features, such as the implicit association between "attendance rate" and "job quality".
[0056] For small sample scenarios, weight sharing is used. It can reduce the number of parameters and prevent overfitting; for high cardinality category features (such as school names), embedding representation can compress information dimensions.
[0057] In general, functions By employing a dynamic discretization mechanism and a feature type-aware function, the core contradiction between feature heterogeneity and dynamic importance in educational data is resolved, laying the foundation for efficient modeling of the Mamba state space in the future.
[0058] In one embodiment, the sequence position encoding function is expressed as follows: , .in, , .
[0059] k represents the position index of the current feature in the sequence, starting from 1. For example, in a sequence consisting of multiple student features such as grades and attendance, the position of the first feature is k=1, the position of the second feature is k=2, and so on. d represents the dimension of the embedding vector, that is, the length of the vector represented by each feature embedding. d is a preset hyperparameter, usually an even number (such as 64, 128, etc.).
[0060] Let be the vector representation of the k-th feature after the embedding layer, which contains the feature's type and numerical information. Let i represent the dimension index of the positional encoding, ranging from 0 to (d / 2)-1. Each i generates two components, a sine value and a cosine value, which together fill the d-dimensional vector.
[0061] The position encoding vector representing the k-th position is composed of d / 2 sine components and d / 2 cosine components interleaved. Specifically, it is generated as follows: for each i (from 0 to (d / 2)-1), the sine value (corresponding to dimension 2i) and the cosine value (corresponding to dimension 2i+1) are calculated.
[0062] denominator Used to control sine / cosine waves of different frequencies, ensuring that the wavelength period of different dimensions i corresponds to a geometrically increasing sequence. These are learnable weight parameters used to dynamically adjust the contribution of positional encoding to feature representation. This parameter is optimized through backpropagation during training, enabling the model to adaptively determine the importance of feature positional information.
[0063] Weight parameters It can be generated through a feature-aware neural network, for example, .in, These represent the learnable weight matrix and the preset bias, respectively, and σ is the Sigmoid activation function.
[0064] Traditional Transformer positional encoding uses a fixed formula to generate position vectors to distinguish elements at different positions in a sequence. However, in educational feature sequences, there is no strict temporal order between features; for example, the order of "grades" and "attendance" has no natural temporal relationship. Therefore, traditional positional encoding may introduce spurious order dependencies. This embodiment introduces learnable weight parameters... This allows the model to dynamically adjust the weights of location information based on actual training data. For example, if the order of certain features has little impact on the prediction result, such as the relative positions of "gender" and "home address" being irrelevant, then α will be learned to a value close to 0, thereby suppressing the influence of location encoding. When the feature order is indeed important, such as "previous semester grades" should precede "current semester grades" in the sequence, It will be learned as a positive value, reinforcing the role of positional encoding. That is, for time-sensitive features such as performance trends, positional encoding is enhanced, making... →1; For static features such as gender, suppress positional interference, making →0.
[0065] Embed the feature vector The weighted sum of the location code and the feature value is used to obtain the final representation that integrates the feature value and location information. Addition preserves the original information of the feature embedding while introducing the relative positional relationship between features through positional encoding. This addition method is similar to superimposing a positional offset on the feature vector, enabling the model to distinguish scenarios where the same feature appears in different positions. For example, placing "math score" at the beginning or end of a sequence will result in different representations due to different positional encodings.
[0066] In educational data, the order of features may vary depending on the data collection method (e.g., different schools record student features in different orders). Through a learnable α parameter, the model can automatically ignore irrelevant positional noise and focus on truly important order relationships. The computation of positional encoding is deterministic (requiring no learnable parameters other than α) and is completed only once during data preprocessing, without adding extra burden to model training. This makes this method suitable for resource-constrained educational information environments.
[0067] In summary, this embodiment is the first to incorporate learnable weight parameters. By introducing feature sequence modeling of educational data, the problem of mismatch between position encoding and the inherent order of features in traditional sequence models is solved, providing a structured input representation for efficient feature aggregation in subsequent Mamba state space models.
[0068] like Figure 2 As shown, an embodiment of the present invention provides a student performance prediction system based on the Mamba state space model, used to implement the student performance prediction method based on the Mamba state space model, which includes a feature embedding and serialization module, a Mamba state space model, and a regression prediction model.
[0069] The feature embedding and serialization module is used to acquire original feature data of students from multiple sources, and to preprocess and vectorize the original feature data of students from multiple sources. Based on the feature type awareness strategy, different types of features are mapped to feature vectors of a unified dimension, and feature sequences are obtained by combining learnable positional encoding weights. The Mamba state space model is used to process the feature sequences and obtain hidden state vectors containing comprehensive information of all features. The regression prediction model is used to obtain student performance prediction results based on the hidden state vectors.
[0070] exist Figure 2 In this model, the feature embedding and serialization module corresponds to the feature embedding and serialization layer, the Mamba state space model corresponds to the Mamba state space encoding layer, and the regression prediction model corresponds to the regression prediction output layer.
[0071] As a preferred technical solution, the Mamba state space model includes a Mamba state space encoding module. The Mamba state space encoding module is used to progressively update the hidden states and aggregate information on the feature sequence, generating a hidden state vector containing comprehensive information about all features at the end of the feature sequence.
[0072] The feature embedding and serialization module includes an embedding module and an arrangement module. The embedding module is used to embed the original feature data of students from multiple sources, converting different types of features into vector representations of a uniform dimension; the arrangement module is used to arrange all features in a predetermined order to construct a feature sequence.
[0073] The regression prediction model includes a fully connected neural network layer. This fully connected neural network layer is used to normalize the hidden state vectors to obtain the student performance prediction results.
[0074] Specifically, this invention introduces the advanced Mamba state-space model into the student performance prediction task and creatively transforms the originally static and independent set of student features into an ordered sequence for processing. By utilizing the selective information propagation mechanism of the state-space model, it effectively captures the dependencies and sequence patterns between features, thus achieving efficient feature sequence modeling.
[0075] First, such as Figure 3 As shown, the original feature data of students in various dimensions are embedded and represented, and arranged in a predetermined order to form a feature sequence; then, this feature sequence is input into the Mamba state-space encoding module for step-by-step processing. Figure 4 As shown, the Mamba state-space encoding module, through its internal state-selective mechanism, sequentially updates the hidden states and aggregates information on the input sequence, ultimately generating a hidden representation containing comprehensive information about all features at the end of the sequence. Finally, this hidden representation is used to predict the student's target performance index through a regression output layer. The above process of embedding features into a sequence and encoding them via the Mamba state-space encoding module fully leverages the advantages of the Mamba model in sequence dependency modeling and efficient computation.
[0076] To address the limitations of limited training data and sparse sample sizes in the education field, this invention specifically designs a lightweight student performance prediction model structure. While ensuring prediction accuracy, the model composition is simplified as much as possible, reducing the number of parameters to lower the risk of overfitting and computational resource consumption. This model architecture design approach, optimized for small sample sizes, provides a novel solution for educational data scenarios.
[0077] The model of this invention consists of three parts: a feature embedding and serialization layer, a Mamba state-space encoding layer, and a streamlined regression prediction output layer, without introducing redundant deep networks or complex attention mechanisms. By reducing the number of network layers and the size of control parameters, the model is more stable when trained on small datasets and is insensitive to data noise and outliers. Even with limited samples, this lightweight structure can still learn and generalize effectively, avoiding the performance degradation of large models due to insufficient data. This lightweight prediction structure, tailored for small-sample scenarios, ensures the model's high efficiency and reliability.
[0078] The Mambular-EDM model architecture proposed in this invention organically combines feature embedding representation, state-space sequence encoding, and regression prediction output to form an end-to-end solution for student performance prediction. This complete architecture is unprecedented in existing technologies, achieving efficient modeling and prediction of educational data through innovative combinations of modules.
[0079] First, the feature embedding and serialization layer preprocesses and vectorizes the multi-source student feature data, enabling different types of features to be expressed in a unified dimensional space. Second, the sorted feature vector sequence is input into the Mamba state-space encoding layer, where state recursion and selection mechanisms are used to extract and integrate the correlation information between features. Finally, the output prediction head performs regression operations on the encoded hidden representation to obtain the final prediction result of student performance. These functional modules are closely integrated, forming the unique novel model structure of this invention, achieving efficient end-to-end modeling from feature input to prediction output.
[0080] In summary, this invention, by introducing the Mamba state-space model and employing a lightweight sequence modeling architecture, achieves significant technical results in predicting student performance in small data scenarios, overcoming the main shortcomings of existing technologies. Specific technical effects are as follows: 1. Enabled efficient and lightweight student performance prediction. Leveraging the linear complexity of the Mamba state-space model's sequence processing capabilities and its streamlined network structure, this invention's model boasts high inference speed and low resource consumption. Compared to complex deep models, this method significantly reduces unnecessary computational overhead, enabling faster prediction of student performance. This efficient and lightweight nature makes the model suitable for real-time deployment in educational settings or operation in environments with limited computing resources, ensuring its feasibility in practical applications.
[0081] 2. Reduced model parameters and computational overhead. The overall architecture of this invention significantly reduces the number of model parameters. By replacing high-parameter components such as self-attention with the Mamba state-space encoding module, the number of parameters that need to be trained and stored in the model is significantly reduced, resulting in a corresponding decrease in computational and memory overhead. The time required for model training and inference is reduced, improving efficiency. This reduction in parameters and computational load means that this method can operate smoothly even on ordinary teaching equipment or under limited hardware conditions, thus lowering the barrier to practical application.
[0082] 3. Robust training and superior performance were achieved in small-sample table tasks. Because of its streamlined model structure and optimization for small datasets, this invention achieves stable convergence even with limited training samples and is less prone to overfitting. Compared to complex models with redundant parameters, the Mambular-EDM framework demonstrates stronger generalization capabilities. On typical small-sample educational datasets, the method of this invention achieves prediction accuracy superior to traditional linear models and large deep models. By effectively modeling feature interactions and avoiding excessive model complexity, this approach achieves robust training and excellent performance under small-data conditions, validating its applicability to educational data scenarios.
[0083] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0084] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.
Claims
1. A student performance prediction method based on the Mamba state-space model, characterized in that, The student performance prediction method based on the Mamba state-space model includes the following steps: The system acquires original feature data of students from multiple sources, preprocesses and vectorizes the original feature data, maps different types of features into feature vectors of a unified dimension based on a feature type awareness strategy, and obtains feature sequences by combining learnable positional encoding weights. Construct a Mamba state space model, input the feature sequence into the Mamba state space model, process the feature sequence through the Mamba state space model, and obtain the hidden state vector containing comprehensive information of all features. Construct a regression prediction model, input the hidden state vector into the regression prediction model, and obtain the student performance prediction results.
2. The student performance prediction method based on the Mamba state-space model as described in claim 1, characterized in that, The specific methods for processing feature sequences include the following steps: The Mamba state-space encoding module performs hidden state updates and information aggregation on the feature sequence step by step, generating a hidden state vector containing comprehensive information of all features at the end of the feature sequence. The Mamba state space model includes a Mamba state space encoding module.
3. The student performance prediction method based on the Mamba state-space model as described in claim 2, characterized in that, The acquisition of the feature sequence specifically includes calculating the feature representation of the fused location information based on the following formula. : Where k is the position index of the feature in the sequence. Let be the embedding vector of the k-th feature. The position encoding vector for the k-th position. These are learnable weight parameters used to dynamically adjust the contribution of positional encoding to feature representation; Feature representation of all fused location information The feature sequence is constructed by arranging the features in a predetermined order.
4. The student performance prediction method based on the Mamba state-space model as described in claim 3, characterized in that, The specific methods for obtaining student performance prediction results include the following steps: A fully connected neural network layer is used as the output prediction head. The hidden state vector is input into the fully connected neural network layer, and the student performance prediction result is obtained through activation function or linear transformation.
5. The student performance prediction method based on the Mamba state-space model as described in claim 1, characterized in that, The specific methods for obtaining feature sequences include the following steps: Select the appropriate encoding method based on the type of student original feature data from multiple sources, and perform feature encoding preprocessing. Obtain the embedding weights and embedding biases, and map the preprocessed multi-source student original feature data according to the embedding weights and embedding biases to obtain the feature sequence. For numerical features, normalization is used for encoding; for categorical features, one-hot encoding is used. A feature type-aware function is constructed, and the mapping strategy is dynamically adjusted according to the feature type. Through learnable embedding weights and embedding bias terms, the preprocessed feature data is mapped into a feature embedding vector of uniform dimension.
6. A student performance prediction system based on the Mamba state-space model, used to implement the student performance prediction method based on the Mamba state-space model as described in any one of claims 1-5, characterized in that, The student performance prediction system based on the Mamba state-space model includes: The feature embedding and serialization module is used to acquire original feature data of students from multiple sources, and to preprocess and vectorize the original feature data of students from multiple sources. Based on the feature type awareness strategy, different types of features are mapped into feature vectors of a unified dimension, and feature sequences are obtained by combining learnable positional encoding weights. The Mamba state-space model is used to process feature sequences and obtain hidden state vectors containing comprehensive information about all features. A regression prediction model is used to obtain student performance prediction results based on the hidden state vector.
7. A student performance prediction system based on the Mamba state-space model as described in claim 6, characterized in that, The Mamba state-space model includes: The Mamba state-space encoding module is used to progressively update the hidden state and aggregate information in the feature sequence, generating a hidden state vector containing comprehensive information about all features at the end of the feature sequence.
8. The student performance prediction system based on the Mamba state-space model as described in claim 7, characterized in that, The feature embedding and serialization module includes: The embedding module is used to embed the original feature data of students from multiple sources, converting different types of features into vector representations of a unified dimension. The permutation module is used to arrange all features in a predetermined order to construct a feature sequence.
9. A student performance prediction system based on the Mamba state-space model as described in claim 8, characterized in that, The regression prediction model includes: The fully connected neural network layer is used to normalize the hidden state vector to obtain the student performance prediction results.