Sliding Window Memory Layout for Large Time-Series Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Overlapping sliding window techniques for training time-series foundation models lead to excessive memory usage due to data redundancy, causing out-of-memory issues with current techniques that copy batched time series data from the central processing unit to the graphical processing unit.
Innovation Solution
Implement a method that senses a raw data sequence using a central processing unit, computes the transfer data size based on the comparison between the raw data sequence and the graphics processing unit's memory size, and transfers the data using zero-copy operations, allowing the graphics processing unit to execute the sliding window algorithm with memory pointers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If overlapping sliding window techniques are used to increase training data volume, then training accuracy is improved, but memory footprint explodes causing out-of-memory issues
Solution Approach 1:
The patent segments the time-series data into non-overlapping chunks and processes them individually, avoiding the creation of multiple overlapping window copies. Each chunk is processed once and fed into the foundation model sequentially, reducing memory requirements while maintaining training effectiveness through proper chunk selection and processing strategy.
Solution Approach 2:
The patent eliminates unnecessary data copying by using zero-copy operations where the foundation model processes data directly from the source buffer without creating intermediate copies. This is achieved through direct memory access and pointer manipulation, preventing the memory footprint explosion that occurs with traditional overlapping window techniques that copy data multiple times.
2Productivity
If batched time series data is copied from CPU to GPU, then data processing capability is improved, but memory usage increases due to data redundancy
Solution Approach 1:
The patent implements zero-copy data transfer from CPU to GPU, eliminating redundant data copies. Instead of copying batched time series data multiple times between different memory spaces, the system uses direct memory access and shared memory buffers that allow both CPU and GPU to access the same data without duplication, significantly reducing memory usage while maintaining processing capability.
Solution Approach 2:
The patent nests data structures efficiently by organizing time-series data in a hierarchical manner where smaller data chunks are stored within larger buffers, and memory pointers reference these nested structures. This allows the system to process large volumes of data without allocating separate memory spaces for each data level, reducing overall memory requirements.
Data Source
AI summary
An embodiment senses a raw data sequence by a central processing unit, responsive to the raw data sequence, computes by the central processing unit a transfer data size of the raw data sequence based at least in part on the comparison of a data size of the raw data sequence to a memory size of a graphics processing unit. The embodiment transfers by the central processing unit of the raw data sequence to the graphics processing unit based on the transfer data size. The embodiment trains a foundation model on the raw data sequence where a sliding window algorithm is executed on the raw data sequence by the graphics processing unit, where generating a window of the sliding window algorithm is based on a memory pointer to the raw data sequence.


