Image Saliency Analysis Using Low- and High-Order Features

The patent combines bottom-up low-order image feature amounts with top-down high-order image feature amounts (including eye tracking data and deep learning models) to calculate saliency. This merging allows the system to maintain the computational efficiency of bottom-up methods while incorporating the line-of-sight predictability and human visual behavior understanding from top-down methods, thereby resolving the contradiction between evaluation accuracy and line-of-sight predictability.

The patent uses a composite approach by integrating multiple types of feature amounts (low-order image features, high-order image features, eye tracking data, and deep learning predictions) into a unified saliency calculation framework. This composite material approach enables the system to leverage the strengths of different methods while mitigating their individual weaknesses, achieving both accurate saliency evaluation and reliable line-of-sight prediction.

The patent segments the saliency calculation into distinct components: low-order image feature amounts, high-order image feature amounts, eye tracking data, and deep learning predictions. By segmenting the complex deep learning process into interpretable components, the system maintains line-of-sight predictability while providing factor transparency, allowing designers to understand which factors contribute most to saliency evaluation results.

The patent introduces an intermediary layer that translates complex deep learning predictions into interpretable feature amounts. This intermediary mechanism preserves the predictive accuracy of deep learning models while making the decision factors transparent and actionable for designers, effectively mediating between model accuracy and interpretability.