3D Visual Search Using LiDAR and CAD Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing visual search technologies struggle to accurately match products in complex scenes without requiring user input beyond pointing the camera, and they lack effective utilization of 3D information for precise product sizing and matching.
Innovation Solution
The integration of 3D sensors, such as LiDAR, with CAD models and advanced object detection and tracking techniques allows for real-time, 'tap-less' visual search. This system captures 3D image information and combines it with 2D data to provide accurate product matching and sizing, using cloud-based visual search engines for complex algorithmic processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional visual search technologies are used, then the system is simple to operate, but the product matching accuracy is poor in complex scenes
Solution Approach 1:
The patent combines 2D image data from cameras with 3D depth information from LiDAR sensors to create a multi-modal visual search system. This merging of different data types enables accurate product matching in complex scenes by leveraging both texture/color information and geometric structure, resolving the contradiction between matching accuracy and system complexity
Solution Approach 2:
The patent transitions from traditional 2D visual search to 3D-enhanced visual search by incorporating depth information. This dimensional upgrade allows the system to understand product geometry, spatial relationships, and scene structure, significantly improving matching accuracy while the modular architecture manages the increased complexity
2Measurement precision
If 3D sensors and CAD models are integrated, then product sizing precision is improved, but device complexity increases
Solution Approach 1:
The patent uses CAD models as digital copies of products with precise dimensional information. By comparing captured 3D point clouds against these digital twins, the system achieves accurate product sizing and identification without requiring physical measurement tools, resolving the contradiction between sizing precision and device complexity
Solution Approach 2:
The patent introduces point cloud processing and feature matching algorithms as intermediaries between the 3D sensor data and product identification. These computational mediaries translate raw sensor data into meaningful product attributes, enabling precise sizing while managing the complexity of sensor integration through standardized processing pipelines
3Power
If cloud-based visual search engines are used, then processing capability is improved, but response time increases
Solution Approach 1:
The patent pre-processes and structures product data in the cloud before search operations, organizing CAD models, 3D features, and product attributes in advance. This preliminary preparation enables faster matching during actual search operations, resolving the contradiction between processing capability and response time by shifting work to beforehand
Solution Approach 2:
The patent divides the visual search process into distinct stages: local 3D feature extraction on-device, cloud-based matching and ranking, and result refinement. This segmentation allows computationally intensive tasks to be distributed, with time-critical operations performed locally and complex matching done in the cloud, balancing processing capability with response time
4Ease of operation
If tap-less visual search is implemented, then ease of operation is improved, but reliability of search results decreases
Solution Approach 1:
The patent implements automatic object detection, tracking, and selection without requiring user input. The system autonomously identifies products in the scene, extracts features, and performs matching, providing tap-less operation. Combined with 3D verification mechanisms, this maintains reliability while improving ease of operation
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables seamless product matching and sizing in crowded scenes with minimal user interaction, providing accurate search results by leveraging both 2D and 3D visual data, thus enhancing the shopping experience and improving product discovery efficiency.
Implementation Method 1
Light Detection And Ranging (LiDAR) is a known sensing method usable to measure and extract an exact distance of an object/surface from a device. Generally, the LiDAR process sends pulses of light and calculates the time it takes for the pulses of light to return to the LiDAR source.
Implementation Method 2
the LiDAR process sends pulses of light and calculates the time it takes for the pulses of light to return to the LiDAR source. The calculated time is used to determine the distance of the object from the device.
Data Source
AI summary
A system and method determines that an object within an image frame being captured via use of an imaging system is an object of interest. The determined object of interest is used to extract from a three-dimensional information obtained via use of a three-dimensional (3D) data obtaining component of the imaging system a 3D information for the object of interest. At least a part of the 3D information for the object of interest is caused to be provided to a cloud-based visual search process for the purpose of locating one or more matching products from within a product database for the object of interest with the located one or more matching products being returned to a customer as a product search result.