Vision Pipe Architecture for Parallel Image Scaling and Computer Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image signal processing pipelines face performance sacrifices when processing complex algorithms, leading to increased CPU bandwidth consumption, power usage, and device size due to the need for additional hardware components.
Innovation Solution
The proposed architecture performs resizing and computer vision operations in parallel using a vision pipe architecture with a front-end pre-processor and back-end portion, including a resizer and computer vision components, allowing for simultaneous scaling and computer vision operations on luminance images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If additional hardware components are added to increase processing performance, then image processing capability is improved, but device size increases
Solution Approach 1:
The patent combines scaling operations and computer vision operations into a single integrated hardware pipeline. The vision pipe architecture merges the resizer component and computer vision processing components into one unified structure that processes image data through multiple stages without requiring separate hardware blocks, thereby improving processing capability while maintaining compact device size.
Solution Approach 2:
The vision pipe architecture is designed as a multi-functional processing pipeline that can perform both scaling operations and various computer vision operations (such as object detection, face recognition, and image enhancement) within the same hardware structure. This universal design eliminates the need for dedicated separate hardware for each function, optimizing device size while maintaining high processing performance.
2Productivity
If additional hardware components are added to increase processing performance, then image processing capability is improved, but power consumption increases
Solution Approach 1:
The patent combines scaling operations and computer vision operations into a single integrated hardware pipeline. The vision pipe architecture merges the resizer component and computer vision processing components into one unified structure that processes image data through multiple stages without requiring separate hardware blocks, thereby improving processing capability while maintaining compact device size.
Solution Approach 2:
The vision pipe architecture enables continuous processing of image data through a pipeline structure where data flows sequentially through multiple processing stages. This continuous action eliminates idle time between operations and ensures that each hardware component is constantly utilized, improving overall processing efficiency while optimizing power consumption by avoiding the need for multiple discrete processing cycles.
3Productivity
If additional hardware components are added to increase processing performance, then image processing capability is improved, but CPU bandwidth consumption increases
Solution Approach 1:
The patent extracts image processing functions (scaling and computer vision operations) from the CPU and implements them as dedicated hardware components in the form of a vision pipe architecture. This extraction transfers the processing burden from the CPU to specialized hardware, thereby improving image processing performance while reducing CPU bandwidth consumption as the CPU no longer needs to handle these computationally intensive tasks.
Solution Approach 2:
The patent replaces software-based image processing executed on the CPU with hardware-based processing using dedicated vision pipe components. This substitution of mechanical/computational approach (hardware circuitry) for software execution significantly improves processing performance while reducing the burden on CPU bandwidth, as hardware operations can be performed in parallel and with greater efficiency.
4Measurement precision
If resizing and computer vision operations are performed sequentially, then processing accuracy is maintained, but processing time increases
Solution Approach 1:
The patent segments the image processing task into multiple distinct stages within the vision pipe architecture, including a resizer stage and separate computer vision processing stages. Each stage processes specific aspects of the image data independently, allowing for optimized processing at each step while maintaining overall accuracy. This segmentation enables parallel execution of different processing operations on different portions of the data stream.
Solution Approach 2:
The vision pipe architecture performs preliminary scaling operations on the image data before it enters the computer vision processing stages. By pre-processing the image data with appropriate scaling and formatting in the initial stages of the pipeline, subsequent computer vision operations can be performed more efficiently on already-prepared data, reducing overall processing time while maintaining the accuracy required for reliable object detection and recognition.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments relate to an architecture of a vision pipe included in an image signal processor. The architecture includes a front-end portion that includes a pair of image signal pipelines that generate an updated luminance image data. A back-end portion of the vision pipe architecture receives the updated luminance images from the front-end portion and performs, in parallel, scaling and various computer vision operations on the updated luminance image data. The back-end portion may repeatedly perform this parallel operation of computer vision operations on successively scaled luminance images to generate a pyramid image.