Real-Time Object Detection via Continual Learning Feature Fingerprinting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in detecting new objects that were not present during their training, leading to inefficient object detection, as existing CNN-based systems require costly updates and parameter transfers.
Innovation Solution
An object detecting system that updates in real-time by generating feature maps and query vectors from input images, comparing them to feature fingerprint databases, and using short-term and long-term update units to refine and expand the database entries, allowing for the detection of untrained classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If CNN parameters are updated regularly to detect new objects, then object detection capability is improved, but learning cost and parameter transfer cost increase
Solution Approach 1:
The system enables autonomous vehicles to perform self-learning by continuously generating feature maps and query vectors from captured images, comparing them against stored feature fingerprints, and automatically updating their detection database without external intervention or costly retraining processes
Solution Approach 2:
The patent implements continuous learning where the object detection system operates indefinitely, constantly updating its knowledge base with new object types encountered during operation, rather than requiring periodic complete retraining cycles
2Adaptability or versatility
If CNN parameters are updated regularly to detect new objects, then object detection capability is improved, but parameter transfer cost increases
Solution Approach 1:
The system extracts only the essential feature representations (query vectors) from full images and stores compact feature fingerprints in the database, transferring only these condensed representations rather than complete model parameters, significantly reducing data transfer requirements
Solution Approach 2:
Instead of transferring complete CNN parameters, the system creates and transfers simplified copies of object features in the form of query vectors and feature fingerprints, which capture essential object characteristics without the complexity of full model parameters
3Reliability
If CNN is trained on fixed training data sets, then initial object detection performance is achieved, but inability to detect new objects arises over time
Solution Approach 1:
The system transitions from a static, fixed training model to a dynamic, evolving detection system that continuously adapts to new object types by learning from images captured during operation, updating its feature database in real-time
Solution Approach 2:
The system uses feedback from captured images and their corresponding feature maps to continuously refine and expand its detection capabilities, comparing new object features against existing fingerprints and updating the database based on observed patterns
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for updating an object detecting system to detect objects with untrained classes in real-time is provided. The method includes steps of: (a) the object detecting system, if at least one input image is acquired, instructing a recognizer included therein to generate a specific feature map, and to generate a specific query vector; (b) the object detecting system instructing a similarity determining unit (i) to compare the specific query vector to data vectors, to thereby calculate each of first similarity scores between the specific query vector and each of the data vectors, and (ii) to add a specific partial image to an unknown image DB, if a specific first similarity score is smaller than a first threshold value; (c) the object detecting system, if specific class information is acquired, instructing a short-term update unit to generate a specific short-term update vector, and update the feature fingerprint DB.