Object Search Device Using Neural Network Feature Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object search methods using images lack information on three-dimensional shapes and irregularities, leading to erroneous results, while three-dimensional data-based searches require costly depth cameras and lengthy feature extraction processes.
Innovation Solution
An object search device and method that uses a first neural network for image feature extraction and a second neural network for three-dimensional data feature extraction, updating parameters to reduce differences between image and three-dimensional features, allowing for accurate searches based on image data alone.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three-dimensional data is used for feature extraction, then search accuracy is improved, but installation cost and processing time increase
Solution Approach 1:
The patent creates a virtual three-dimensional model as a copy of the actual three-dimensional object, allowing feature extraction from this virtual model instead of requiring physical depth cameras. The virtual model is generated by synthesizing multiple two-dimensional images taken from different angles, creating a three-dimensional representation that can be processed without expensive depth-sensing hardware.
Solution Approach 2:
The patent replaces the mechanical/optical depth camera system with a computational approach using standard two-dimensional cameras and image processing algorithms. Instead of using specialized hardware to capture three-dimensional data directly, the system uses software to synthesize three-dimensional information from multiple two-dimensional images, eliminating the need for depth cameras.
2Measurement precision
If three-dimensional data is used for feature extraction, then search accuracy is improved, but processing time increases
Solution Approach 1:
The patent performs feature extraction on the virtual three-dimensional model in advance, before the actual search operation. The system pre-processes the synthesized three-dimensional data to extract shape and irregularity features, storing these extracted features for rapid retrieval during search operations. This preliminary feature extraction eliminates the need to perform time-consuming three-dimensional processing during real-time searches.
Solution Approach 2:
The patent extracts only the essential shape and irregularity features from the virtual three-dimensional model, rather than processing the entire three-dimensional data set during search operations. By extracting and storing only the relevant geometric features in advance, the system reduces the computational burden during actual searches, achieving fast retrieval without sacrificing accuracy.
3Productivity
If two-dimensional images are used for search, then processing time is reduced, but search accuracy deteriorates due to lack of three-dimensional information
Solution Approach 1:
The patent transforms two-dimensional image data into a three-dimensional virtual model, adding the third dimension to the data without requiring three-dimensional cameras. By synthesizing multiple two-dimensional images taken from different angles and positions, the system creates a three-dimensional representation that preserves shape and irregularity information, enabling accurate shape-based search while maintaining compatibility with standard two-dimensional imaging hardware.
Data Source
AI summary
An object of the invention is to configure an object search device capable of expressing information on shapes and irregularities as features only by images, in a search for an object that is characteristic in shape or irregularity, and performing an accurate search.The object search device includes: an image feature extraction unit that is configured with a first neural network, and is configured to input an image to extract an image feature; a three-dimensional data feature extraction unit that is configured with a second neural network, and is configured to input three-dimensional data to extract a three-dimensional data feature; a learning unit that is configured to extract an image feature and a three-dimensional data feature from an image and three-dimensional data of an object obtained from a same individual, respectively, and update an image feature extraction parameter so as to reduce a difference between the image feature and the three-dimensional data feature; and a search unit that is configured to extract image features of a query image and a gallery image of the object by the image feature extraction unit using the updated image feature extraction parameter, and calculate a similarity between the image features of both images to search for the object.