Object Classification Using Local Feature Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning methods for image classification are opaque, brittle, computationally intensive, and require large datasets, making them unsuitable for applications needing high accuracy, robustness, and explainability, especially in contexts with varying object classes and limited data availability.
Innovation Solution
A computer-implemented method using machine-learned local feature identification models to classify objects in images by comparing identified local features with masked images of candidate objects, based on overlap scores and confidence scores, allowing for robustness and explainability, and enabling low-shot learning by generating 3D models and segmented image masks for new object classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning methods are used for image classification, then classification accuracy can be improved, but the computational cost and training complexity increase significantly
Solution Approach 1:
The patent segments the object recognition task into multiple components: detecting local features (edges, corners, textures), describing them with invariant descriptors, and matching them against template objects. This divides the complex deep learning problem into manageable stages that can be solved with simpler, more interpretable algorithms at each step.
Solution Approach 2:
The patent introduces local feature descriptors as intermediary representations between the raw image data and the final classification decision. These descriptors serve as intermediate features that capture essential object characteristics while being computationally efficient and interpretable, bridging the gap between simple feature detection and complex classification.
2Measurement precision
If deep learning methods are used for image classification, then object detection capability can be improved, but the robustness to adversarial attacks and scene variance decreases
Solution Approach 1:
The patent focuses on detecting and describing local features (edges, corners, textures) rather than relying on global image patterns. Each local feature is described with invariant descriptors that capture its essential properties independently of the overall image context, making the system more robust to adversarial perturbations that affect global patterns.
Solution Approach 2:
Instead of training a model to recognize objects directly from images (forward approach), the patent inverts the approach by detecting local features first, describing them with invariant properties, and then matching these descriptions against known object templates. This inversion makes the system more interpretable and resistant to adversarial attacks.
3Measurement precision
If deep learning methods are used for image classification, then classification performance can be improved, but the explainability and interpretability of the model decreases
Solution Approach 1:
The patent segments the classification process into interpretable stages: local feature detection, feature description with invariant descriptors, and template matching. Each stage produces intermediate results that can be visualized and understood, providing clear explanations for why a particular classification decision was made, unlike opaque deep learning models.
Solution Approach 2:
The patent replaces the complex, opaque neural network mechanisms with more interpretable computational approaches: traditional image processing operators for feature detection, mathematical descriptors for feature characterization, and explicit template matching algorithms. These mechanical/computational processes are inherently more explainable than deep learning black boxes.
4Measurement precision
If deep learning methods are used for image classification, then accuracy on large datasets can be improved, but the requirement for large training datasets increases
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing invariant local feature descriptors for template objects before the actual classification task. These pre-computed descriptors serve as reusable references that can be matched against detected features in new images without requiring retraining, enabling the system to generalize from fewer training examples.
Solution Approach 2:
The patent uses copying by creating and storing template representations of objects with their characteristic local feature descriptors. These templates can be copied and reused across multiple classification tasks, eliminating the need to retrain on large datasets for each new object class and enabling efficient low-shot learning.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to image classification, in particular to object recognition/detection in images. According to an aspect of the present invention, there is provided a computer implemented method of classifying an object in an image. The method comprises: identifying, using one or more machine-learned local feature identification models, a plurality of local features of an object in the image, each local feature comprising a respective feature identity and a respective feature area; comparing the plurality of identified local features to a plurality of sets of masked images of candidate objects, wherein: each set of masked images corresponds to a respective candidate object and comprises a plurality of masked images of the respective candidate object from a plurality of viewing angles; each masked image comprises a plurality of feasible search zones for local features of the candidate object; and the comparison of the plurality of local features to a masked image of a candidate object is based on an overlap between the identified local features and feasible search zones of said candidate object in the masked image; and classifying the object in the image as one of the candidate objects based on the comparison of identified local features to a plurality of sets of masked images of candidate objects.