Hierarchical Attention Time-Series Model for Explainable Behavior Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for predicting user behavior are inefficient and produce suboptimal results due to the lack of contextual understanding of user actions, often operating as 'black boxes' that fail to provide explainable predictions.
Innovation Solution
A hierarchical attention time-series (HAT) model is employed, which processes time-stamped user activity data in parallel, determining attention scores for both individual actions and sessions, enabling accurate and explainable predictions by contextualizing user behavior through attention layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional machine learning models evaluate each user action independently in a serial manner, then the model structure is simple, but the processing efficiency is low and prediction accuracy is suboptimal
Solution Approach 1:
The patent segments user activity data into meaningful units (actions, sessions, and time-series sequences) and processes them through a hierarchical architecture. The HAT model divides the prediction task into multiple layers: action-level attention, session-level attention, and time-series prediction, allowing parallel processing of different data granularities simultaneously, thus improving processing efficiency while maintaining manageable complexity through structured organization
Solution Approach 2:
The patent introduces hierarchical dimensions to the model architecture, adding action-level and session-level attention layers above the base time-series model. This multi-dimensional approach allows the system to process user actions independently at the action level while simultaneously considering contextual relationships at the session level, enabling parallel computation paths that improve overall processing efficiency without overwhelming complexity
2Measurement precision
If conventional machine learning models give inordinate weight to certain user actions, then the model is easy to interpret, but the prediction accuracy is suboptimal due to lack of contextual understanding
Solution Approach 1:
The patent introduces attention mechanisms as intermediary layers between the input data and the prediction output. These attention layers compute weighted importance scores for different user actions and sessions, acting as mediators that dynamically determine which actions deserve more weight based on their contextual relevance. This intermediary processing enables the model to accurately capture contextual relationships without requiring overly complex architectural changes
Solution Approach 2:
The patent applies local quality by allowing different parts of the model to specialize in different aspects of contextual understanding. The action-level attention layer focuses on local relationships between individual actions, while the session-level attention layer captures broader contextual patterns. This localized specialization enables accurate contextual processing by breaking down the complex task into manageable, focused sub-tasks
3Loss of information
If conventional machine learning models operate as black boxes, then the model structure is simple, but the explainability of predictions is poor
Solution Approach 1:
The patent implements feedback mechanisms where the attention layers compute and return importance scores that explain which actions and sessions influenced the prediction. The model architecture includes explicit feedback paths that propagate attention weights from the prediction layer back to the input features, providing interpretable information about the decision-making process. This feedback structure enables explainable predictions without requiring fundamentally complex architectural changes
Data Source
AI summary
Aspects of the present disclosure provide techniques for behavior prediction. Embodiments include receiving activity data of a user, identifying user sessions comprising sets of time-stamped actions in the activity data, and segmenting the activity data into subsets corresponding to the user sessions. Embodiments include providing the subsets as inputs to a hierarchical attention time-series (HAT) model comprising: a first layer that determines attention scores for respective time-stamped actions in the subsets; and a second layer that determines attention scores for the subsets based on aggregations of the attention scores for the respective time-stamped actions. Embodiments include receiving, as outputs from the HAT model in response to the inputs: a prediction based on the subsets, the attention scores for the respective time-stamped actions, and the attention scores for the subsets; and explanatory information based on the attention scores for the respective time-stamped actions and the attention scores for the subsets.