Vision Pipeline Memory Synchronization for Low-Latency Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision systems face latency issues due to memory allocation and synchronization challenges across multiple processors, which are critical in mission-critical operations like autonomous vehicles.
Innovation Solution
A vision data structure that allocates and maintains memory ranges efficiently across processors, using non-blocking operations and asynchronous data synchronization, allowing synchronization only when necessary, and supporting multiple data formats like NHWC and NCHW to reduce conversion operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory allocation and synchronization operations are performed across multiple processors, then data consistency is maintained, but processing latency increases
Solution Approach 1:
The system pre-allocates memory ranges for vision data in both CPU memory and GPU memory before processing begins. This preliminary action ensures that when data needs to be transferred or synchronized, the destination memory locations are already prepared, eliminating allocation delays during critical processing moments and reducing overall latency while maintaining data consistency.
Solution Approach 2:
The system implements synchronization operations that monitor and coordinate data states between CPU and GPU memories. By using feedback mechanisms to detect when data has been modified and when synchronization is actually needed, the system maintains data consistency across processors while avoiding unnecessary synchronization operations that would increase latency.
2Measurement precision
If vision data is synchronized between multiple processors, then data accuracy is improved, but processing speed decreases
Solution Approach 1:
The system performs synchronization operations selectively rather than continuously. By using partial action - synchronizing only when data modifications occur or when specific conditions are met - the system maintains data accuracy between CPU and GPU memories without the performance penalty of constant full synchronization, thus preserving processing speed.
3Speed
If memory ranges are allocated for each processor, then data access efficiency is improved, but memory management complexity increases
Solution Approach 1:
The vision data structure is designed to be universal and processor-agnostic, managing memory ranges for multiple processors (CPU, GPU, and other vision processors) through a single unified interface. This multi-functionality allows the same data structure to serve multiple processors simultaneously, improving data access efficiency for all processors while hiding the underlying memory management complexity from individual processor operations.
Data Source
AI summary
Techniques for maintaining and synchronizing data is a processing pipeline data between multiple processing units to improve a system latency are described herein. For example, the techniques may include determining, in response to an invocation of vision processing on first vision data stored in a first memory range in a first memory associated with a central processing unit (CPU), that second vision data stored in a second memory range in a second memory associated with a graphic processing unit (GPU) is a modified copy of the first vision data. The second vision data may be obtained using a non-blocking operation from the second memory range. The first vision data stored in the first memory range may be replaced with the second vision data obtained from the second memory range. The vision processing may then be performed using the second vision data stored in the first memory.