Neural Network Model Augmentation for Sequential Recommendation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data sparsity and noise in sequences hinder the performance of neural network models in sequential recommendation systems, as traditional methods fail to adequately train complex sequential models and reveal actual item correlations.
Innovation Solution
The implementation of contrastive self-supervised learning (SSL) with model augmentation, which constructs view pairs using methods such as neuron masking, layer dropping, and encoder complementing to improve data augmentation and incorporate comprehensive self-supervised signals, thereby enhancing the performance of neural network models in sequential recommendation tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional sequence modeling is used to characterize item relationships, then the model structure can be implemented, but data sparsity and noise undermine the performance due to insufficient training and inability to reveal actual item correlations
Solution Approach 1:
The patent applies preliminary action by performing data augmentation on the input sequence before processing it through the neural network model. Multiple augmented views of the same sequence are generated in advance, each processed independently to produce predictions that are then aggregated. This preliminary transformation of the data helps mitigate data sparsity by creating additional training samples from limited input data, thereby improving model performance without requiring more original data.
2Adaptability or versatility
If complex sequential model structure is used, then item correlations can be modeled, but insufficient training data hinders adequate training
Solution Approach 1:
The patent employs copying by creating multiple augmented views of the input sequence through transformations such as masking, shuffling, or filtering. Each augmented view serves as a copy that preserves the essential item relationships while presenting different perspectives. These copied views are processed through the complex sequential model to generate multiple predictions, which are then combined to improve the overall prediction accuracy. This copying strategy enables adequate training of complex models by effectively increasing the quantity of training data.
3Productivity
If noisy item sequences are used for training, then the model can be trained on available data, but the recommendation ability is impeded because noisy sequences are unable to reveal actual item correlations
Solution Approach 1:
The patent implements feedback by generating multiple predictions from different augmented views of the same noisy sequence and then aggregating these predictions through voting or averaging mechanisms. The feedback loop involves comparing the aggregated prediction with the actual target, computing the loss, and updating the model parameters accordingly. This feedback mechanism helps filter out noise by leveraging the consensus across multiple augmented views, thereby improving measurement precision in detecting actual item correlations while maintaining training productivity.
Data Source
AI summary
A method for providing a neural network system includes performing contrastive learning to the neural network system to generate a trained neural network system. The performing the contrastive learning includes performing first model augmentation to a first encoder of the neural network system to generate a first embedding of a sample, performing second model augmentation to the first encoder to generate a second embedding of the sample, and optimizing the first encoder using a contrastive loss based on the first embedding and the second embedding. The trained neural network system is provided to perform a task.