Compressive Transformer for Long Learning Record Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing transformer series score prediction models are limited by high operation costs proportional to the square of input time series data length, restricting them to analyzing only 100 question-solving records, which is insufficient for understanding a user's overall learning level.
Innovation Solution
A method using a compressive transformer to predict test scores, where training data is embedded through layers, with an upper triangular attention mask matrix for pre-training, allowing the model to effectively process and analyze long learning records by using a compressive transformer architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a transformer series model is used for score prediction, then the model can process sequential data effectively, but the operation cost increases proportionally to the square of the input time series data length, limiting the analysis to only 100 question-solving records
Solution Approach 1:
The patent divides the input sequence into fixed-size chunks (e.g., 100 records each) and processes them separately through multiple encoding passes. Instead of processing all records at once, the model segments the long sequence into manageable portions, reducing the computational burden from O(n²) to O(k²) where k is the chunk size, while still analyzing the entire learning history through iterative processing
Solution Approach 2:
The patent performs preliminary encoding of individual question-solving records into embedding vectors before processing them through the transformer model. This pre-processing step transforms the raw sequential data into a compact representation that reduces the dimensionality and complexity of subsequent processing, enabling efficient handling of long learning records
2Measurement precision
If only 100 question-solving records are analyzed, then the operation cost remains manageable, but the data is insufficient to understand the overall learning level of the user
Solution Approach 1:
The patent performs preliminary encoding of individual question-solving records into embedding vectors that capture essential features before processing. This pre-processing transforms each record into a compact representation, allowing the model to efficiently process and retain information from a larger number of records without proportionally increasing computational cost
Solution Approach 2:
The patent divides the long sequence of question-solving records into multiple fixed-size chunks and processes them through iterative encoding passes. Each chunk is processed independently but contributes to the overall understanding, allowing the model to analyze the entire learning history (e.g., 500+ records) by combining insights from multiple segments
Data Source
AI summary
A method for predicting a test score of a user through an artificial intelligence model by a terminal, includes: delivering training data of the user to a first layer for embedding; embedding the training data through the first layer; delivering an embedding vector from the first layer to a second layer including a compressive transformer; delivering an output value from the second layer to a third layer for predicting the test score; and outputting a prediction value for predicting the test score from the third layer.