Personalized Saliency Model Adaptation for Mixed Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing saliency models, trained on global data, fail to accurately reflect individual users' personalized views on salient content, leading to inaccuracies in determining relevant image portions in mixed reality environments.
Innovation Solution
The development of personalized saliency models that update based on individual user reactions to captured images, allowing users to generate and share their personalized models, which can then refine global saliency models, using multi-dimensional saliency maps and supervised learning techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If global saliency models are used, then data coverage and generalizability are improved, but accuracy for individual users deteriorates
Solution Approach 1:
The patent segments the saliency model into two components: a global saliency model trained on diverse user data and a personalized saliency model trained on individual user data. This segmentation allows the system to maintain broad data coverage through the global model while achieving individual user accuracy through the personalized model, directly resolving the contradiction between generalizability and individual accuracy.
Solution Approach 2:
The patent merges the global saliency model and personalized saliency model into a unified system where the personalized model is trained using the global model as initialization and then refined with individual user feedback. This combination allows the system to leverage both the broad data coverage of global models and the individual accuracy of personalized models simultaneously.
2Measurement precision
If personalized saliency models are developed, then accuracy for individual users is improved, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-training a global saliency model on diverse user data before personalization. This pre-trained global model serves as a foundation that reduces the complexity of individual personalization, as the personalized model only needs to adapt from this existing foundation rather than training from scratch, thus managing system complexity while maintaining accuracy.
Solution Approach 2:
The system implements self-service through automated feedback collection from user interactions (eye tracking, device usage patterns) that automatically retrains and updates personalized models without requiring manual user intervention. This automation reduces operational complexity while maintaining high individual accuracy through continuous adaptation.
3Measurement precision
If user feedback is collected continuously, then model personalization accuracy is improved, but data processing requirements increase
Solution Approach 1:
The patent extracts only the most relevant features from user feedback data (eye movement patterns, device interaction behaviors) rather than processing all raw data. This selective extraction maintains high personalization accuracy by focusing on discriminative features while significantly reducing the volume of data that needs to be processed and stored.
Solution Approach 2:
The system applies partial action by collecting and processing only a subset of potentially useful feedback data - specifically focusing on eye tracking and device usage patterns that are most indicative of user attention and interest. This partial data collection achieves effective personalization without the excessive data processing requirements of comprehensive data collection.
Data Source
AI summary
Methods, systems, and computer readable media for providing personalized saliency models, e.g., for use in mixed reality environments, are disclosed herein, comprising: obtaining, from a server, a first saliency model for the characterization of captured images, wherein the first saliency model represents a global saliency model; capturing a first plurality of images by a first device; obtaining information indicative of a reaction of a first user of the first device to the capture of one or more images of the first plurality images; updating the first saliency model based, at least in part, on the obtained information to form a personalized, second saliency model; and transmitting at least a portion of the second saliency model to the server for inclusion into the global saliency model. In some embodiments, a user's personalized (i.e., updated) saliency model may be used to modify one or more characteristics of at least one subsequently captured image.