Dual Descriptor Data for Object Recognition Across Imaging Modalities
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing home monitoring systems face challenges in accurately recognizing and tracking objects during transitional periods when cameras adjust imaging modalities due to changing ambient lighting conditions, as object features can vary significantly between different imaging modalities.
Innovation Solution
The use of dual descriptor data, which represents associations of object features in multiple imaging modalities, enables improved object recognition and tracking by allowing cameras to identify and associate object attributes across different imaging modalities, such as RGB and IR data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the camera switches between RGB and IR imaging modalities to adapt to lighting conditions, then the system can capture data across different lighting environments, but object recognition accuracy deteriorates during transitional periods when the camera is adjusting capture modes
Solution Approach 1:
The patent introduces dual descriptor data as an intermediary mechanism that bridges RGB and IR imaging modalities. This descriptor data includes both RGB descriptors and IR descriptors for the same object, allowing the system to maintain consistent object recognition across modalities during transitions. The dual descriptor acts as a mediator that enables accurate tracking even when the camera switches between capture modes.
Solution Approach 2:
The system performs preliminary actions by pre-computing and storing dual descriptor data for objects before actual recognition occurs. By preparing both RGB and IR descriptors in advance and storing their associations, the system can quickly retrieve and match descriptors during transitions without delay, maintaining recognition accuracy while adapting to lighting changes.
2Measurement precision
If the system stores separate object features for each imaging modality, then the system can accurately represent objects in different modalities, but the system complexity and data storage requirements increase
Solution Approach 1:
The patent merges RGB and IR object feature representations into a unified dual descriptor structure. Instead of storing completely separate feature sets, the system combines both modalities' descriptors for each object into a single associated structure, reducing redundancy while maintaining the ability to accurately represent objects in either modality.
Solution Approach 2:
The dual descriptor data structure serves multiple functions simultaneously: it stores RGB descriptors for color-based recognition, stores IR descriptors for low-light recognition, and maintains the association between both modalities. This multi-functional design eliminates the need for separate storage systems while preserving accuracy for different imaging conditions.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using dual descriptor data. One of the methods includes: detecting, using a first set of descriptor features included in dual descriptor data, a first representation within first image data collected by a camera; determining a change to an imaging modality of the camera; detecting, using a second set of features included in the dual descriptor data, a second representation within second image data collected by the camera; classifying the first representation and the second representation as associated with a same object using the dual descriptor data; and in response to classifying the first representation and the second representation as associated with the same object using the dual descriptor data, transmitting operational instructions to one or more appliances connected to the system.