Image Classification Combining Visual and Contextual Tensors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) face significant accuracy reduction when classifying images that deviate from the norm, such as those with poor quality or unusual environments, as they rely solely on visual features without considering contextual information.
Innovation Solution
A method that combines image data and metadata using a combinator, such as a concatenation layer or a transformer module, to provide additional context for the neural network, improving classification accuracy and robustness, especially in low-quality image scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If CNNs are used for image classification based solely on visual features, then the classification process is simple and fast, but the accuracy significantly reduces when images deviate from the norm (poor quality, unusual environments)
Solution Approach 1:
The patent merges visual features extracted from images with contextual information from metadata by concatenating their respective tensors. This combination allows the neural network to leverage both appearance-based and context-based information, improving classification accuracy for images that deviate from the norm while maintaining a relatively straightforward architectural integration.
Solution Approach 2:
The patent introduces metadata as an intermediary source of contextual information that bridges the gap between visual features and classification decisions. By processing metadata separately and then combining it with visual features, the system gains additional information about the image context without directly modifying the core image processing pipeline.
2Reliability
If metadata is processed and combined with image data, then classification accuracy improves, but processing time and computational complexity increase
Solution Approach 1:
The patent applies preliminary action by processing metadata and extracting visual features in parallel before combining them. This allows both processing streams to occur simultaneously rather than sequentially, reducing overall processing time while still integrating both information sources for improved accuracy.
Solution Approach 2:
The patent segments the classification task into two independent processing streams: one for extracting visual features from images and another for processing metadata. This segmentation allows both streams to be processed independently and efficiently, then combined at the tensor level without creating computational bottlenecks.
Data Source
Figure 1~3
Figure 2
Figure 4
AI summary
The disclosure relates to a method implemented by computer means for classifying images and to a corresponding computer system and a corresponding computer program. The method comprises: - using an image preprocessor, extracting image data representing at least one image feature of an image to obtain a first tensor representing the extracted image data, - using a metadata preprocessor, processing metadata representing a contextual information of the image to obtain a second tensor representing the processed metadata, - using a combinator, combining the first tensor and the second tensor to obtain a combination result, and - providing the combination result to an image classification layer of an artificial neural network to obtain a classification result.