CGR Object Instance Detection for Detailed Scene Understanding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for detecting objects and understanding physical environments in computer-generated reality (CGR) environments are inefficient and inaccurate, particularly in identifying specific instances of objects and incorporating their characteristics for enhanced CGR environments.
Innovation Solution
A method combining object detection with instance detection using machine learning models to identify object types and specific instances, accessing databases for characteristics, and integrating these characteristics into CGR environments for improved scene understanding and simulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing object detection techniques are used in CGR environments, then basic object identification can be achieved, but accuracy in identifying specific instances and incorporating their characteristics is insufficient
Solution Approach 1:
The patent segments the object detection process into two distinct stages: (1) object type detection using a first neural network to identify general categories, and (2) instance detection using a second neural network to identify specific instances and extract their characteristics. This segmentation allows each network to specialize in different aspects of detection, improving overall accuracy while preserving detailed object information.
Solution Approach 2:
The patent introduces an intermediary database that stores characteristics of detected object instances. The second neural network queries this database to retrieve detailed characteristics (dimensions, materials, colors, etc.) of identified object instances. This intermediary structure enables accurate retrieval and incorporation of object characteristics into the CGR environment without requiring the neural network to directly memorize all possible object details.
2Measurement precision
If instance detection with database lookup is implemented, then precise object characteristics can be obtained, but system complexity increases
Solution Approach 1:
The patent creates a digital copy of object characteristics by storing them in a database indexed by object instance identifiers. Instead of complex real-time analysis of object features, the system copies pre-stored characteristic data from the database into the CGR environment. This copying approach simplifies the detection system while maintaining high precision in instance identification and characteristic retrieval.
3Reliability
If detailed object characteristics are integrated into CGR environment, then quality and realism of virtual content improve, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary action by pre-processing and storing object characteristics in a database before they are needed in the CGR environment. When an object instance is detected, its characteristics are already prepared and readily accessible in the database, eliminating the need for time-consuming real-time analysis during CGR rendering. This preliminary preparation significantly reduces processing time while maintaining high CGR quality.
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that determine a particular object instance in CGR environments. In some implementations, an object type of an object depicted in an image of a physical environment is identified. Then, a particular instance is determined based on the object type and the image. In some implementations, objects of the particular instance have a set of characteristics that differs from sets of characteristics associated with other instances of the object type. Then, the set of characteristics of the particular instance of the object depicted in the physical environment is obtained.