Image Sensor Motion ROI Processing for On-Sensor AI Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image sensors in user terminals, such as smartphones, face challenges in performing AI operations due to memory and computation requirements, limiting their ability to efficiently detect and process image frames for tasks like face detection and QR code recognition.
Innovation Solution
An image sensor system with a motion detection circuit and a micro control unit (MCU) that adjusts target frames based on detected motion, converts operation modes, and uses a neural network to detect objects, including brightness and rotation corrections, to optimize processing and reduce power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If AI algorithms are performed through standalone servers or cloud services due to high memory and computation requirements, then processing capability is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent extracts the AI processing function from external servers and embeds it directly into the image sensor device. A neural network processor is integrated within the image sensor to perform AI algorithms locally, eliminating the need for complex cloud service architecture while maintaining processing capability.
Solution Approach 2:
The patent implements a nested architecture where the neural network processor is embedded within the image sensor module, which itself is part of the user terminal. This nested structure allows AI processing to be integrated at multiple levels, reducing overall system complexity while enabling local processing.
2Measurement precision
If the entire image frame is processed for object detection, then detection accuracy is improved, but power consumption and processing time increase
Solution Approach 1:
The patent segments the image frame into multiple regions of interest (ROIs) based on motion detection results. Instead of processing the entire frame, only the segmented ROI containing the target object is subjected to AI-based object detection, significantly reducing power consumption while maintaining detection accuracy.
Solution Approach 2:
The patent applies partial action by performing comprehensive AI processing only on the necessary portion of the image (the ROI). Motion detection is performed on the full frame to identify ROIs, then AI object detection is applied selectively to these partial regions, avoiding excessive processing of the entire image.
3Reliability
If motion detection and object detection are performed separately, then detection reliability is improved, but processing time and power consumption increase
Solution Approach 1:
The patent merges motion detection and object detection into an integrated processing pipeline. Motion detection results are directly used to define ROIs for object detection, and the neural network processor performs both functions in a coordinated manner, reducing processing time while maintaining the reliability of separate detection mechanisms.
Solution Approach 2:
The patent performs motion detection as a preliminary action before object detection. By first identifying regions with motion and using these as ROIs for subsequent object detection, the system prepares the data in advance, reducing the time required for the more computationally intensive object detection phase.
Data Source
AI summary
An image sensor includes: a motion detection circuit configured to detect a motion in image frames; and a micro control unit (MCU) configured to adjust at least a portion of a target frame among the image frames based on whether the motion is detected, and detect whether a target object is present based on the adjusted portion of the target frame.