Prototype-Based Image Inference for Explainable Similarity Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models, particularly deep learning models, lack transparency in their decision-making processes, making it difficult to explain the basis for inference results, especially in applications like image retrieval where the target image may belong to an unclassified class.
Innovation Solution
A convolutional neural network-based learning model is trained using metric learning to generate feature maps, prototype vectors, and integrated similarity vectors, allowing for improved explainability by bringing prototype vectors closer to relevant pixel vectors based on distributed prototype belongingness, enabling clearer decision-making basis in image retrieval processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning models use high complexity learning models for accurate inference, then inference accuracy is improved, but explainability of decision-making basis deteriorates
Solution Approach 1:
The patent segments the learning model into distinct components: a convolutional neural network for feature extraction and a separate prototype vector module for interpretation. The CNN processes input images through multiple layers to generate feature maps, while prototype vectors independently represent interpretable image concepts. This segmentation allows the complex CNN to maintain high inference accuracy while the separate prototype module provides explainable decision-making basis through similarity calculations.
2Loss of information
If prototype vectors are brought closer to all pixel vectors to improve explainability, then transparency is improved, but computational complexity increases
Solution Approach 1:
The patent applies local quality by calculating similarity between prototype vectors and pixel vectors selectively rather than uniformly across all combinations. The system identifies relevant pixel vectors based on spatial proximity and feature similarity, then focuses computational resources on calculating prototype-pixel similarities only for these relevant regions. This localized approach maintains transparency by showing which specific image regions contribute to the inference decision while reducing overall computational complexity.
3Reliability
If distributed prototype belongingness is used to train multiple images per class, then learning robustness is improved, but training time increases
Solution Approach 1:
The patent implements preliminary action by pre-computing and storing prototype vectors that represent each image class before the actual training process. These prototype vectors are generated in advance based on class labels and stored for efficient retrieval during training. When training multiple images per class with distributed prototype belongingness, the system references these pre-computed prototypes rather than computing similarities from scratch, significantly reducing training time while maintaining learning robustness through consistent prototype-based supervision.
Data Source
AI summary
A learning model generates a plurality of prototype vectors and generates an integrated similarity vector that indicates similarity between an input image and each prototype for a plurality of prototypes in accordance with similarity between one prototype vector and each pixel vector in a feature map acquired from an CNN. An image processing apparatus obtains prototype belongingness (distributed prototype belongingness) for each image by distributing prototype belongingness of a belonged prototype of each class to each of two or more images that belong to one class. Then, the learning model is subjected to machine learning in accordance with the distributed prototype belongingness of each prototype vector for each image so that each prototype vector is brought closer to any pixel vector in the feature map corresponding to each image.