Vision Data Architecture With Async CPU-GPU Memory Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision techniques in automated systems face latency issues due to memory allocation and synchronization challenges across multiple processors, which are critical in mission-critical operations like autonomous vehicles.
Innovation Solution
A vision data structure that allocates and maintains memory ranges efficiently across processors, using non-blocking operations and asynchronous data synchronization, allowing synchronization only when necessary, and supporting multiple data formats like NHWC and NCHW to reduce conversion operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If memory allocation and synchronization operations are performed across multiple processors, then data consistency is maintained, but processing latency increases
Solution Approach 1:
The system pre-allocates memory ranges for vision data in both CPU memory and GPU memory before processing begins. This preliminary action ensures that when synchronization is needed, data can be copied directly to pre-prepared memory locations without allocation delays, maintaining consistency while reducing latency.
Solution Approach 2:
The patent extracts the synchronization operation from the critical processing path by using non-blocking copy operations. The synchronization is performed asynchronously in the background, allowing the main processing to continue without waiting for data consistency to be established, thus reducing perceived latency while maintaining reliability.
2Reliability
If vision data is synchronized between CPU and GPU memories, then data consistency is ensured, but processing speed decreases
Solution Approach 1:
The patent introduces a buffer memory as an intermediary between CPU and GPU. Vision data can be copied to the buffer asynchronously, and processing can proceed independently once data is available. This mediator decouples the synchronization from the processing speed, ensuring consistency without blocking the processing pipeline.
Solution Approach 2:
The system maintains continuous processing by using non-blocking synchronization operations. While data is being synchronized in the background, the processing pipeline continues to execute other operations or processes available data, ensuring that useful actions are never interrupted by synchronization delays.
3Ease of manufacture
If memory ranges are reallocated for each vision processing operation, then memory management is simple, but processing efficiency decreases
Solution Approach 1:
Memory ranges are allocated in advance during system initialization or before batches of processing operations. This preliminary allocation eliminates the need for repeated allocation and deallocation during processing, maintaining simple memory management while significantly improving processing efficiency through reduced overhead.
Solution Approach 2:
The patent creates universal memory ranges that can be reused across multiple processing operations. Instead of allocating dedicated memory for each operation, the same memory ranges serve multiple purposes and multiple operations, simplifying management while boosting productivity through efficient resource utilization.
4Adaptability or versatility
If data format conversion operations are performed, then compatibility between processors is achieved, but processing time increases
Solution Approach 1:
The system performs data format conversion in advance, before the actual processing operation begins. By converting data to the required format beforehand and storing it in appropriately formatted memory regions, the patent achieves format compatibility without adding conversion time to the critical processing path.
Data Source
AI summary
Techniques for maintaining and synchronizing data is a processing pipeline data between multiple processing units to improve a system latency are described herein. For example, the techniques may include determining, in response to an invocation of vision processing on first vision data stored in a first memory range in a first memory associated with a central processing unit (CPU), that second vision data stored in a second memory range in a second memory associated with a graphic processing unit (GPU) is a modified copy of the first vision data. The second vision data may be obtained using a non-blocking operation from the second memory range. The first vision data stored in the first memory range may be replaced with the second vision data obtained from the second memory range. The vision processing may then be performed using the second vision data stored in the first memory.