Instruction Stream Manager for Multi-Stream Vision Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In advanced autonomous vehicle and driver-assistance systems, the lack of synchronization between CPUs and accelerators when processing multiple perception streams leads to lost processing capacity and increased costs due to inefficient computation distribution.
Innovation Solution
An instruction stream manager is implemented to distribute computations across multiple CPUs and accelerators by partitioning instruction streams, assigning release times and deadlines, and using scheduling queues to optimize synchronization, ensuring efficient processing and resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple perception streams are processed by shared CPUs and accelerators, then processing capacity is increased, but synchronization between devices deteriorates leading to lost processing capacity
Solution Approach 1:
The patent segments the processing system into multiple independent scheduling queues, each dedicated to a specific processing device (CPU or accelerator). Each queue manages its own stream of computations, allowing devices to operate independently without synchronization conflicts. This segmentation resolves the contradiction by enabling high processing capacity through parallel operation while maintaining reliability through device-specific queue isolation.
Solution Approach 2:
The patent introduces scheduling queues as intermediary structures between the perception streams and processing devices. These queues act as buffers and organizers that decouple the input streams from the processing devices, allowing each device to consume computations at its own pace without requiring synchronization with other devices. This intermediary mechanism enables both high productivity and reliable operation.
2Productivity
If multiple perception streams are processed by shared CPUs and accelerators, then processing capacity is increased, but computation distribution efficiency deteriorates
Solution Approach 1:
The patent divides the computation distribution system into multiple device-specific scheduling queues, where each queue is dedicated to a particular processing device. This segmentation allows computations to be distributed efficiently to the appropriate device without complex coordination overhead. Each queue independently manages its computations, simplifying the distribution process while maintaining high processing capacity through parallel operation of multiple queues.
3Reliability
If synchronization is enforced between CPUs and accelerators, then computation coordination is improved, but overhead increases
Solution Approach 1:
The patent extracts the synchronization requirement from the system by eliminating the need for inter-device synchronization. Each processing device operates independently with its own dedicated scheduling queue, so no synchronization between devices is required. This extraction of the synchronization problem completely eliminates synchronization overhead while maintaining computation coordination through the independent queue structure.
Solution Approach 2:
The scheduling queues serve as intermediaries that completely decouple the processing devices from each other. By introducing these queue structures, the patent eliminates direct device-to-device interactions and synchronization requirements. Each device only interacts with its own queue, removing all synchronization overhead while maintaining reliable computation coordination through the queue mediation layer.
Data Source
AI summary
A stream manager for managing the distribution of instructions to a plurality of processing devices includes a dispatcher module configured to: receive multiple instruction streams, wherein each instruction stream includes a plurality of requested computations for processing perception data from a perception data source; partition each instruction stream into a plurality of partitions based on type of device to perform a requested computation from the instruction stream; assign a release time and deadline to each partition, and dispatch partition computations to a plurality of scheduling queues to distribute processing of the partition computations amongst the plurality of processing devices. The plurality of scheduling queues include: a plurality of CPU schedulers, wherein each CPU scheduler is assigned to a specific CPU and a specific scheduling queue; and a plurality of accelerator schedulers, wherein each accelerator scheduler is assigned to a specific scheduling queue and a specific type of accelerator.