Storage Controller Accelerates Big Data Analytics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current datacenter infrastructure faces significant performance bottlenecks due to network latency and congestion caused by numerous storage access requests across data-center networks, which slows down critical software applications like Big Data Analytics.
Innovation Solution
The implementation of an improved storage controller that enables application software portions to run directly on storage devices, reducing the need for network transactions by positioning search and query algorithms close to storage media, thereby minimizing network round trips and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If application software runs on servers accessing storage over network, then data processing capability is maintained, but network latency and congestion increase significantly
Solution Approach 1:
The patent introduces storage devices with embedded processors as intermediaries between servers and storage media. These intermediate processing units execute search and query algorithms directly at the storage location, eliminating the need for servers to repeatedly access storage over the network for data processing operations. This intermediary layer resolves the contradiction by maintaining data processing capability while removing the network latency bottleneck.
Solution Approach 2:
The patent shifts the computation dimension from centralized server processing to distributed storage-device processing. By embedding processors within storage devices and enabling local execution of data access algorithms, the system creates a new dimensional approach where data processing occurs in parallel at multiple storage nodes rather than sequentially through network-accessed servers. This dimensional shift eliminates network latency while preserving processing capability.
2Speed
If search algorithms are positioned close to storage media, then data access speed improves, but device complexity increases
Solution Approach 1:
The patent makes storage devices multi-functional by equipping them with embedded processors that can execute various search and query algorithms. Instead of requiring separate dedicated hardware for each function, the universal processor architecture within storage devices can perform multiple data processing tasks locally. This universality resolves the contradiction by enabling fast data access through local processing while avoiding the complexity of specialized hardware for each operation.
Solution Approach 2:
The patent enables storage devices to serve themselves by executing data access algorithms locally without requiring constant server intervention. The embedded processors allow storage devices to independently perform search, filter, and query operations on their own data, reducing the need for complex external control systems and minimizing network dependency. This self-service capability achieves fast data access while simplifying the overall system architecture.
Data Source
AI summary
An improved data storage system and apparatus including an improved storage controller that provides storage compute functionality that enables the acceleration of datacenter software, and that enables easier deployment of application software portions onto storage devices, in a manner that supports runtime performance acceleration of network-latency-throttled applications. Mechanisms and methods are provided for server hosted applications to initiate deployment of, initiate execution of, and interoperate with a multitude of softwares on a multitude of storage devices, where these softwares execute proximate to storage contents on the storage devices.


