Saliency Map Prediction via Probability Distribution Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current saliency detection methods in computer vision, particularly for visual attention prediction, face challenges in accurately modeling the stochastic nature of visual attention using loss functions suited for semantic tasks, leading to suboptimal performance in predicting saliency maps.
Innovation Solution
A method involving a deep neural network trained with a loss function based on probability distance measures, such as the Bhattacharyya distance, paired with a softmax activation function, to optimize the prediction of saliency maps as probability distributions, effectively addressing the limitations of existing methods by better aligning with the stochastic nature of visual attention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard regression loss functions are used for training saliency prediction models, then the models can be trained using conventional semantic task approaches, but the prediction accuracy for saliency maps deteriorates due to mismatch with the stochastic nature of visual attention
Solution Approach 1:
The patent changes the parameter representation by modeling saliency maps as probability distributions rather than continuous values. This involves transforming the output space from real numbers to probability space (0 to 1 with sum constraint), and changing the loss function from regression-based (L2, L1) to probability-based (cross-entropy, KL divergence), which fundamentally alters how the model learns to predict saliency while matching the stochastic nature of visual attention
Solution Approach 2:
The patent substitutes the mechanical regression loss mechanism with a probabilistic loss mechanism. Instead of minimizing pixel-wise squared errors or absolute differences, the system uses cross-entropy or KL divergence to measure the difference between predicted and ground truth probability distributions, which is mathematically more appropriate for modeling uncertain, stochastic phenomena like human visual attention
2Measurement precision
If deep neural networks are trained with probability distribution-based loss functions, then the prediction accuracy of saliency maps improves, but the computational complexity of training increases
Solution Approach 1:
The patent applies preliminary action by pre-computing the ground truth probability distributions from attention maps before training. The attention maps are processed offline to create target probability distributions that are then used during training, separating the computationally intensive ground truth generation from the iterative training process and enabling efficient optimization
Solution Approach 2:
The patent uses softmax activation in the final layer which inherently normalizes outputs to form a valid probability distribution. This partial action of normalization built into the activation function reduces the computational burden during training iterations, as the probability constraints are automatically satisfied without requiring additional normalization steps in the loss computation
3Reliability
If saliency maps are modeled as probability distributions, then the stochastic nature of visual attention is better captured, but the distance computation between distributions becomes more complex than simple pixel-wise comparison
Solution Approach 1:
The patent employs feedback mechanisms through gradient-based optimization where the computed distance (cross-entropy or KL divergence) and its gradient with respect to network parameters are fed back during backpropagation. This allows the complex probability distribution comparison to be integrated seamlessly into the standard deep learning training loop, making the complexity manageable through automated differentiation and iterative optimization
Solution Approach 2:
The patent introduces the softmax function as an intermediary that transforms raw network outputs into valid probability distributions. This intermediary layer ensures that the output always satisfies probability constraints (non-negative and sum to 1), simplifying the distance computation by guaranteeing valid input to the loss function without requiring additional constraint enforcement mechanisms
Data Source
AI summary
A method for generating a system for predicting saliency in an image and method of use of the prediction system are described. Attention maps for each of a set of training images are used to train the system. The training includes passing the training images though a neural network and optimizing an objective function over the training set which is based on a distance measure computed between a first probability distribution computed for a saliency map output by the neural network and a second probability distribution computed for the attention map for the respective training image. The trained neural network is suited to generation of saliency maps for new images.