Neural Network Dataset Selection With Storage-Aware Tiling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require exponentially increasing memory to process large datasets, leading to poor performance or failure due to memory constraints, especially as the length of input text increases, and current approaches do not efficiently manage memory usage.
Innovation Solution
The system divides input data into subsets based on available storage, using a dynamic self-attention kernel to optimize memory usage by dynamically choosing tile sizes and performing tiled matrix multiplications, reducing memory requirements and avoiding costly memory reads and writes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural networks process large datasets using conventional approaches, then processing capability is improved, but memory requirements increase exponentially causing poor performance or failure
Solution Approach 1:
The patent segments the input dataset into multiple smaller subsets that can be processed individually. Each subset is processed through the neural network separately, and results are aggregated to produce the final output. This segmentation allows the system to handle large datasets without requiring exponential memory growth, as each subset fits within available memory constraints.
Solution Approach 2:
The patent introduces a temporal dimension to the processing approach by processing data subsets sequentially over time rather than attempting to load the entire dataset into memory simultaneously. This transforms the memory-intensive parallel processing problem into a time-efficient sequential processing solution, allowing large datasets to be handled within limited memory boundaries.
2Productivity
If neural networks increase the length of input text to improve processing capability, then productivity is improved, but memory requirements increase causing runtime failures
Solution Approach 1:
The patent divides the input text into smaller segments or chunks that can be processed individually. Each segment is processed through the neural network separately, avoiding the memory exhaustion that would occur if the entire long text were loaded at once. This segmentation maintains reliability while preserving the ability to process long inputs.
Solution Approach 2:
The patent performs preliminary processing steps to prepare and segment the input data before it enters the neural network. By pre-processing and dividing the input text into manageable portions, the system prevents memory exhaustion during runtime while maintaining the capability to handle long input sequences.
3Productivity
If conventional neural network approaches are used to process large datasets, then processing capability is improved, but computing time and resources increase
Solution Approach 1:
The patent segments the processing task into smaller independent units that can be handled more efficiently. By processing smaller subsets of data, each requiring less computational resources, the overall computing time is reduced compared to attempting to process the entire large dataset at once with conventional approaches.
Data Source
AI summary
Apparatuses, systems, and techniques to perform a neural network to select a dataset. In at least one embodiment, for example, a neural network calculates an expected relevance of data of a dataset subset, where said subset is portioned based, at least in part, on an amount of available storage. In at least one embodiment, as another example, a processor is to cause one or more neural networks to identify one or more first portions of first information to be used by one or more neural networks to generate second information, wherein one or more neural networks are to identify one or more first portions based, at least in part, on an amount of available storage.