Multi-Stream Vision Processing via Tagged Synchronization Manager
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern advanced vision processing algorithms, such as deep learning neural networks, introduce high computation and memory workloads that limit the number of vision streams that can be simultaneously processed by a controller, leading to inefficient use of processing capacity and increased costs due to unsynchronized computations across multiple CPUs and accelerators in advanced autonomous vehicle and ADAS applications.
Innovation Solution
A system and method that improves synchronization of computations by using a controller to read and process multiple perception streams with unique identification tags, applying a predetermined policy for scheduling and processing, and dispatching processed streams to appropriate destinations, allowing for efficient use of shared hardware resources and pipelining of processing algorithms across multiple devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple perception streams are processed simultaneously using multiple CPUs and accelerators, then processing capacity increases, but computation synchronization becomes poor resulting in lost processing capacity
Solution Approach 1:
A synchronization manager is introduced as an intermediary component that coordinates computations across multiple CPUs and accelerators. The synchronization manager receives computation requests, determines synchronization requirements, and manages the synchronization process to ensure proper coordination while maintaining processing capacity.
Solution Approach 2:
The system segments computation management into distinct functional components: a synchronization manager that handles coordination logic, computation units that execute processing, and a queue manager that handles request prioritization. This segmentation allows each component to specialize in its function while maintaining overall system synchronization.
2Productivity
If advanced vision processing algorithms are used, then processing capability improves, but computation and memory workloads increase limiting the number of vision streams
Solution Approach 1:
The system applies partial processing by breaking down vision stream processing into discrete computation requests that can be handled incrementally. The queue manager processes requests in priority order, allowing the system to handle multiple streams with varying computational demands without requiring all resources to be allocated simultaneously, thus reducing peak memory workload.
Solution Approach 2:
The system dynamically adjusts resource allocation based on real-time computation requirements. The synchronization manager and queue manager work together to allocate computation and memory resources dynamically to different vision streams based on their current priority and workload, allowing efficient handling of multiple streams without static over-provisioning.
3Productivity
If multiple CPUs and accelerators are used for multiple perception streams, then processing throughput increases, but computation synchronization deteriorates leading to higher costs
Solution Approach 1:
The synchronization manager serves multiple functions: it acts as a request receiver, a synchronization coordinator, a priority manager, and a resource allocation controller. This universal component manages all synchronization aspects across different CPU and accelerator types, reducing overall system complexity despite the diversity of processing units.
Solution Approach 2:
The system implements feedback mechanisms where the synchronization manager monitors computation progress and synchronization status across multiple processors. Based on this feedback, the manager dynamically adjusts scheduling and resource allocation to maintain proper synchronization, enabling complex multi-processor operations with simplified coordination logic.
Data Source
AI summary
A method in a multiprocessor system for processing multiple perception streams is disclosed. The method comprises: reading data from a plurality of perception streams according to a reading schedule determined by a predetermined policy, each perception stream comprising perception data from a different perception sensor; assigning a unique identification tag to each perception stream; writing each perception stream with its unique identification tag to a server input queue based on the predetermined policy; and processing the tagged perception streams using a server. The processing includes: retrieving tagged perception streams from the server input queue; applying a processing algorithm to process the retrieved tagged perception streams; and outputting the processed perception streams to a server output queue. The method further includes retrieving the processed perception streams from the server output queue and dispatching each retrieved processed perception stream to an appropriate post-processing destination based on the tag associated with the retrieved processed perception stream.