DVS Object Recognition Using Color Conversion and Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human posture detection methods trained on RGB images exhibit high error rates when applied to dynamic vision sensor (DVS) images due to significant differences between DVS and RGB images, leading to high noise, low resolution, and poor signal quality, limiting the application scope of DVS-related products.
Innovation Solution
An object recognition method that converts DVS images into color images using an image conversion model, extracts feature maps from both images, fuses them, and performs object recognition using an object recognition model to improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing human posture detection methods trained on RGB images are applied to DVS images, then the method can be used with DVS sensors, but the error value increases significantly (20% to 30%)
Solution Approach 1:
The patent introduces an image conversion model as an intermediary that transforms DVS images into RGB-compatible format. This mediator enables the existing RGB-based human posture detection model to process DVS input effectively, bridging the compatibility gap between different sensor types while maintaining detection accuracy
Solution Approach 2:
The patent changes the parameter representation of DVS images by converting them from event-based temporal data into spatial RGB image format. This parameter transformation allows the input data to match the expected format of existing RGB-based models, resolving the incompatibility issue
2Reliability
If DVS image data is used directly for human posture detection, then privacy protection is maintained, but the error value is much higher (20% to 30%) due to high noise and low resolution
Solution Approach 1:
The image conversion model serves as a mediator that processes raw DVS data into enhanced RGB-compatible images. This intermediary processing step removes noise and improves resolution while preserving the underlying event data's privacy-protecting characteristics
Solution Approach 2:
The patent creates a converted copy of the DVS image in RGB format that preserves the essential visual information needed for accurate posture detection while eliminating the noise and resolution limitations of the original DVS data
3Measurement precision
If a large amount of DVS image data is collected and marked to train a dedicated DVS posture detection model, then the model can be optimized for DVS images, but the process is time-consuming and resource-intensive
Solution Approach 1:
Instead of creating a new DVS-specific model through extensive training, the patent uses an image conversion model to create RGB-compatible copies of DVS images, allowing the use of existing pre-trained RGB models. This approach saves significant training time and resources while achieving comparable accuracy
Solution Approach 2:
The image conversion model enables a single RGB-based human posture detection model to handle both RGB and DVS inputs universally. This multi-functionality eliminates the need for separate DVS-specific model training, reducing development time and resource requirements
Data Source
AI summary
An object recognition method and an object recognition device are provided. The method includes: obtaining a dynamic vision sensor (DVS) image, and converting a DVS image into a color image using an image conversion model; extracting a first feature map of the DVS image, and extracting a second feature map of the color image; fusing the first feature map and the second feature map into a third feature map; and performing an object recognition operation on the third feature map using an object recognition model to obtain an object recognition result corresponding to the DVS image.