Dual Neural Network Image Retrieval with Local and Global Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image retrieval techniques face challenges due to the 'semantic gap' between low-level visual features and high-level semantics, leading to unsatisfactory results, especially when features change with viewing angle, distance, or illumination, causing recognition errors and overfitting.
Innovation Solution
A method using two neural networks, one for local feature extraction and another for global feature extraction, with specific convolutional layer configurations and loss functions, to process images at different resolutions and capture diverse features, improving accuracy by combining local and global features and using spatial transformer networks for spatial invariance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image retrieval techniques using underlying visual features are used, then the retrieval process is simple, but the retrieval accuracy is unsatisfactory due to semantic gap
Solution Approach 1:
The patent divides feature extraction into two separate neural networks: one dedicated to local feature extraction and another to global feature extraction. This segmentation allows each network to specialize in specific aspects of image analysis, improving overall retrieval accuracy while maintaining manageable system complexity through modular architecture
Solution Approach 2:
The patent combines local features and global features to form a composite feature representation for image retrieval. By integrating multiple types of features (local details and global context) into a unified retrieval system, the patent overcomes the semantic gap problem while achieving high accuracy without excessive complexity
2Measurement precision
If deep learning-based image retrieval with CNN is used, then the feature extraction ability is powerful, but the system complexity increases
Solution Approach 1:
The patent segments the deep learning system into two distinct but complementary neural networks with different architectures and functions. The first network focuses on local features with specific convolutional layers, while the second handles global features differently. This segmentation makes the complex deep learning system more manageable and interpretable
Solution Approach 2:
The patent applies different network configurations and processing methods to local and global features respectively. Local features are extracted with one set of convolutional parameters and processing steps, while global features use different parameters and methods, allowing each part of the system to be optimized for its specific function
3Adaptability or versatility
If images are processed at different resolutions, then the adaptability to various viewing conditions improves, but the computational complexity increases
Solution Approach 1:
The patent performs resolution adaptation in advance by training the neural networks on images of multiple resolutions before actual retrieval operations. This preliminary training enables the networks to handle various viewing conditions without requiring real-time resolution transformation during retrieval, reducing computational complexity during operation
Data Source
AI summary
According to the embodiments of the present application, there are proposed a method and electronic device for retrieving an image, and computer readable storage medium. The method includes: processing an image to be retrieved using a first neural network to determine a local feature vector of the image to be retrieved; processing the image to be retrieved using a second neural network to determine a global feature vector of the image to be retrieved; and determining, based on the local feature vector and the global feature vector, an image having a similarity to the image to be retrieved which is higher than a similarity threshold.