Dynamic Memory Pool Allocation for GPU Behavioral Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
GPU memory allocation is a synchronizing event that suspends other processes, and hardware restrictions limit memory transfers between the host CPU and device GPU, hindering the timely analysis of large data streams in behavioral recognition systems.
Innovation Solution
Implementing dynamic memory management using memory pools in both host CPU and GPU, allowing for variable-sized memory allocation and reducing memory transfers by compositing data from multiple sources before transfer, thereby avoiding synchronization events and optimizing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If GPU memory allocation is used for processing video data, then parallel processing capability is improved, but synchronization events occur that suspend other GPU processes
Solution Approach 1:
The system pre-allocates memory pools in both host and device memory before processing begins. This preliminary action ensures that memory is already available when needed, eliminating the need for synchronization events during actual processing operations.
Solution Approach 2:
The memory management system is divided into separate host memory pools and device memory pools, each independently managed. This segmentation allows parallel operations to occur in device memory without blocking host memory operations, resolving the synchronization bottleneck.
2Productivity
If memory transfers between host CPU and device GPU are performed, then data processing is enabled, but transfer rate is limited by hardware restrictions
Solution Approach 1:
The system composites multiple input data streams into a single consolidated data structure before transfer. By merging multiple small transfers into one larger transfer operation, the system maximizes the utilization of available transfer bandwidth and reduces the overhead of multiple transfer initiations.
Solution Approach 2:
Data from multiple sources is pre-composed and consolidated in host memory before the transfer to device memory begins. This preliminary composition ensures that the transfer operation is optimized and that no additional processing is needed during the transfer itself.
3Device complexity
If fixed memory allocation is used in GPU, then memory management is simplified, but adaptability to variable-sized data streams is reduced
Solution Approach 1:
The system implements dynamic memory pools that can adapt their allocation to match the actual size requirements of incoming data streams. Memory pools are initialized with a base allocation but can dynamically adjust to accommodate variable-sized inputs without requiring fixed predetermined sizes.
Solution Approach 2:
The memory allocation parameters are made changeable based on the actual data stream characteristics. Instead of fixed allocation, the system adjusts memory pool sizes and configurations to match the varying requirements of different video inputs and processing stages.
Data Source
AI summary
Techniques are disclosed for dynamic memory allocation in a behavioral recognition system. According to one embodiment of the disclosure, one or more variable-sized chunks of memory is allocated from a device memory for a memory pool. An application allocates at least one of the chunks of memory from the memory pool for processing a plurality of input data streams in real-time. A request to allocate memory from the memory pool for input data is received. Upon determining that one of the chunks is available in the memory pool to store the input data, the chunk is allocated from the memory pool in response to the request.