Data supply acceleration system and method for single-machine to small-scale multi-GPU (Graphics Processing Unit) training scene

Abstract

The invention provides a data supply acceleration system and method for single-machine to small-scale multi-GPU (Graphic Processing Unit) training scenes, and the system adopts an off-line learning and on-line inference architecture. The system comprises an offline training module, an online portrait generation module, an adaptive strategy selection module, an income inference and replacement execution module, a training cooperative scheduling module and a strategy switching control module. In the off-line stage, the system learns'retention revenue 'of data entries based on historical logs and solidifies a revenue prediction model and a strategy selector. In the online stage, the system generates a system state portrait in real time, an optimal strategy is adaptively selected according to the system state portrait, cache replacement is driven based on profit prediction, and GPU idling is reduced through a collaborative scheduling mechanism. According to the method, the model parameters are updated in an off-line manner, the operation overhead is low, the I / O waiting time and the tail time delay in the deep learning training process can be effectively reduced, and the GPU utilization rate and the overall training throughput rate are improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning training system optimization, specifically to a data supply acceleration system and method based on offline profit prediction and adaptive strategy selection, for single-machine to small-scale multi-GPU training scenarios. Background Technology

[0002] In the field of deep learning technology, training based on single machines to small-scale multi-GPU clusters is the mainstream choice for model development in both academia and industry. With the rapid increase in GPU computing power, the bottleneck in the training process is increasingly shifting from computation itself to data supply. In training tasks such as image recognition and natural language processing, especially when the dataset size reaches the terabyte level, the path of loading data from storage media to GPU memory faces severe challenges: massive small files result in huge overhead for random reads from the storage system; the speed of data preprocessing (such as decoding and enhancement) on the CPU side is difficult to match the computing speed of the GPU; and data movement between buses in a multi-GPU environment further erodes effective computation time. These problems ultimately manifest as limited training sample throughput, a significant decrease in GPU utilization during the data loading phase, and data waiting becoming the dominant bottleneck in the end-to-end training process.

[0003] Existing optimization techniques have significant shortcomings in this scenario. One type is heuristic prefetching and caching strategies, such as those relying on fixed LRU / LFU algorithms, sequential block reading, or static quotas. These are prone to mismatch in heterogeneous storage and variable access patterns (interleaved random and sequential access) scenarios involving single-machine multi-GPU training. Another type is computationally important sampling methods, which can reduce the computational cost of low-value samples, but still require reading the entire dataset from disk into memory for evaluation. Their optimization goal is to align FLOPs savings rather than directly reduce I / O latency, thus yielding minimal results when I / O is already a bottleneck. Furthermore, purely online learning solutions introduce additional system overhead and the risk of training instability.

[0004] A patent search revealed an invention patent with publication number CN115904666A, which discloses a deep learning training task scheduling system for GPU clusters. The system includes an offline performance characterization module and an online task scheduling and deployment module. The offline performance characterization module predicts the similarity of different training tasks to determine the set of hardware metrics to be collected for newly submitted tasks and the performance degradation caused by the mixed deployment of two training tasks, i.e., performance interference. It also collects the computation time of training tasks in dedicated GPU mode with different batch sizes offline to fit the batch size-computation time relationship. The online task scheduling and deployment module retrieves tasks to be scheduled from the task queue and generates GPU affinity and batch size distribution strategies for the tasks to be scheduled based on the batch size-computation time relationship and performance interference. It allocates suitable GPUs based on the user-submitted training tasks and the number of GPUs specified by the user, i.e., the GPUs with the least performance interference after mixed deployment, thereby minimizing the average job completion time and improving cluster resource utilization. This completely solves the problem of maximizing overall system throughput and improving cluster resource utilization in GPU clusters where some training tasks cannot fully utilize GPU resources independently and need to adhere to group scheduling principles. This patent lacks caching and I / O optimization mechanisms, failing to address the waiting issue at the data supply level; it also lacks counterfactual correction methods, making offline performance prediction accuracy susceptible to data bias; and it only focuses on GPU load balancing, neglecting to ensure training sampling coverage, which may affect model training accuracy.

[0005] In summary, given the problems of the existing technologies, researching a data supply acceleration system and method for single-machine to small-scale multi-GPU training scenarios has become a critical task that urgently needs to be addressed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a data supply acceleration system and method for single-machine to small-scale multi-GPU training scenarios.

[0007] A data supply acceleration system for single-machine to small-scale multi-GPU training scenarios provided by the present invention includes: The offline training module is used to collect and analyze the running logs, and to learn the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows offline, and output a fixed profit prediction model and a fixed policy selector. The profile generation module is used to collect system operation indicators at the training window granularity during online training, and to standardize and handle outliers of the system operation indicators to form profile vectors. The adaptive policy selection module is used to input the profile vector into the fixed policy selector and obtain the policy parameter set corresponding to the current training window. The revenue inference and replacement execution module is used to perform replacement operations on candidate cache entries to be replaced when the cache capacity is limited or when the preset eviction trigger condition is reached. The training collaborative scheduling module is used to perform scheduling at the candidate batch level to ensure that the sample coverage in each round remains unchanged and does not increase the total computational workload. The strategy switching control module is used to construct a cache index or queue corresponding to the new strategy at the boundary of the training window, based on the new strategy defined by the strategy parameter group and using a shadow data structure.

[0008] Preferably, the offline training module performs the following steps: Step S1.1: Collect runtime logs containing training window-level metrics and cache entry-level event information; Step S1.2: Based on the running log, construct a benefit-related label for each cached entry in each training window. The benefit-related label includes the waiting time saved per hit and the expected number of hits in the next window. Step S1.3: Based on the running logs and revenue-related labels, train the revenue prediction model and policy selector using the offline context bandit method, and solidify the model parameters of the revenue prediction model and policy selector.

[0009] Preferably, in step S1.2, the waiting time saved per hit is obtained by weighting the difference in waiting time between the missed batch and the hit batch in the same training batch according to the entry size and decoding overhead, and stabilizing it using an exponential moving average; the expected number of hits in the next window is statistically estimated based on the sampling weight and the window length; for the waiting time that does not actually occur in the current window due to the substitution event, a bucket-based counterfactual approximation is used, and the round-end correction is performed with the measured delay during subsequent actual loading to form the benefit target.

[0010] Preferably, in step S1.2, a revenue target value is constructed for each cached entry based on the following relationship:

[0011] in, The savings per unit hit are obtained by weighting the difference between the waiting time of the missed batch and the waiting time of the hit batch in the same training batch according to the data size of the entry and the decoding cost, and then stabilizing it using an exponential moving average. The expected number of hits for the next window is estimated based on sampling weights and window length statistics. For the waiting time that does not actually occur in the current window due to alternative access, a bucket counterfactual approximation based on media type and packet granularity is used, and the actual waiting time is corrected during subsequent actual loading.

[0012] Preferably, the image generation module is also used to calculate the image drift index, which includes a comprehensive measure based on the rank correlation coefficient and the rate of change of distribution entropy.

[0013] Preferably, the adaptive strategy selection module triggers strategy switching or increases exploration parameters based on the profile drift index, and only outputs the strategy parameter group when the preset minimum cooldown time and benefit advantage threshold are met.

[0014] Preferably, the revenue inference and replacement execution module performs the following operations: Step S4.1: Within each training window, construct a candidate set for all potentially eliminated items; and collect item-level feature sets; Step S4.2: Call the fixed revenue prediction model to perform forward inference on each item in the candidate set to obtain the retained revenue score; Step S4.3: When the cache reaches the replacement condition, the entries are eliminated in order of retention benefit score from low to high.

[0015] Preferably, the training collaborative scheduling module performs the following operations: Step S5.1: If there are high-priority samples that are not ready in the candidate batch, postpone the entire candidate batch and prioritize scheduling batches where all samples are ready. Step S5.2: If the candidate batch contains only unread low-quality samples, select unvisited low-quality samples from the cache to perform equivalent replacement on the unread low-quality samples, and record the replaced samples to the missing set. Step S5.3: At the end of the current round window, perform merged training based on the missing set.

[0016] Preferably, the strategy switching control module performs the following operations: Step S6.1: At the end of each training window, perform batch reconstruction of the shadow heap or shadow index based on the current cache entries and the new strategy; Step S6.2: After the shadow structure is constructed, a seamless switch between the old and new strategy structures is achieved through pointer replacement or equivalent atomic operations. Step S6.3: Set a maximum rate or a progressive threshold in the first few eliminations after the switch to avoid buffer jitter and ensure stable convergence.

[0017] This invention also provides a data supply acceleration method for single-machine to small-scale multi-GPU training scenarios, based on the aforementioned data supply acceleration system for single-machine to small-scale multi-GPU training scenarios, comprising the following steps: Step S1: Based on historical running logs or pre-experiment logs, perform offline learning on the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows, and output a fixed profit prediction model and a fixed policy selector. Step S2: During online training, system operation indicators are collected at the training window granularity, and the system operation indicators are standardized and outlier processing is performed to form a profile vector. Step S3: Input the image vector into the fixed policy selector to obtain the policy parameter group corresponding to the current training window; Step S4: When the cache capacity is limited or the preset eviction trigger condition is reached, perform a replacement operation on the candidate cache entries to be evicted. Step S5: Perform scheduling at the candidate batch level to ensure that the sample coverage remains unchanged in the current round and does not increase the total computational workload. Step S6: At the boundary of the training window, according to the new policy defined by the policy parameter group, construct the cache index or queue corresponding to the new policy using the shadow data structure.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention addresses the problem that I / O waiting is the dominant bottleneck in single-machine to small-scale multi-GPU training scenarios. It uses "wait time saved per hit" as the core benefit indicator, closely aligning cache eviction decisions with the goal of directly reducing I / O waiting costs. Compared to indirect optimization methods that aim to save computational resources, this invention can more effectively solve the training stagnation problem in I / O-constrained scenarios.

[0019] 2. This invention adopts an architecture that combines offline learning with online lightweight inference, avoiding the additional overhead and convergence risk brought about by online model training. In the online stage, mechanisms such as shadow structure, atomic switching and hysteresis control are used to ensure smooth policy updates, which meets the requirements of high throughput and stability for multi-GPU training.

[0020] 3. In heterogeneous storage and random access scenarios of single-machine multi-GPU training, this invention effectively reduces GPU idle time and tail latency caused by data not being ready in multi-GPU training through a collaborative scheduling mechanism of "readiness priority" and "equivalent substitution training", thereby improving training efficiency and hardware utilization.

[0021] 4. This invention enables the system to adapt to heterogeneous storage, changing data access patterns and loads in a single-machine multi-GPU environment by dynamically driving strategy selection through online profiling, thus maintaining stable performance under various media and data distributions.

[0022] 5. This system is designed in a plug-in format, which can be easily connected to mainstream deep learning frameworks and is compatible with existing training technology stacks, thus reducing the engineering threshold for deployment and use in real multi-GPU training environments.

[0023] 6. The offline learning model of this invention has cross-task reuse capability, supports horizontal scaling and multi-task concurrency on small-scale multi-GPU clusters, and avoids the cost of repeated deep optimization for each new task.

[0024] This invention can reduce training pauses and tail latency caused by data supply in complex training scenarios with heterogeneous storage and limited bandwidth, improve end-to-end training efficiency, and at the same time ensure model accuracy and system stability. Attached Figure Description

[0025] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a data supply acceleration system architecture diagram for a single-machine to small-scale multi-GPU training scenario according to an embodiment of the present invention; Figure 2 This is a flowchart of a data supply acceleration method for single-machine to small-scale multi-GPU training scenarios according to an embodiment of the present invention; Figure 3 This is an offline training pipeline diagram in an embodiment of the present invention; Figure 4 This is a flowchart of the online window-level processing in an embodiment of the present invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0027] This invention discloses a data supply acceleration system and method for single-machine to small-scale multi-GPU training scenarios. Integrated into a deep learning training framework, this invention aims to solve the technical problem of GPU idleness due to data I / O waiting in single-machine to small-scale multi-GPU training scenarios. Specifically, the system adopts an "offline learning, online inference" architecture, including an offline training module, an online profile generation module, an adaptive policy selection module, a profit inference and replacement execution module, a training collaborative scheduling module, and a policy switching control module. In the offline phase, the system learns the "retained profit" of data entries based on historical logs and solidifies the profit prediction model and policy selector. In the online phase, the system generates a system state profile in real time, adaptively selects the optimal policy accordingly, and drives cache replacement based on profit prediction, while reducing GPU idle time through a collaborative scheduling mechanism. This invention does not update model parameters online, has low operating overhead, and can effectively reduce I / O waiting time and tail latency during deep learning training, improving GPU utilization and overall training throughput.

[0028] Example 1: Figure 1 This is a data supply acceleration system architecture diagram for a single-machine to small-scale multi-GPU training scenario according to an embodiment of the present invention.

[0029] like Figure 1 As shown, this embodiment provides a data supply acceleration system for single-machine to small-scale multi-GPU training scenarios. The system adopts an "offline learning, online inference" architecture and includes: The offline training module is used to collect and analyze runtime logs, and to learn the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows offline, outputting a fixed profit prediction model and a fixed policy selector.

[0030] The runtime logs are derived from historical runtime logs of the same or similar tasks, or from pre-experiment logs specifically executed for system initialization. In this embodiment, the offline training module operates only before the training task starts or during intervals. Its purpose is to generate and solidify the profit prediction model and policy selector to be used in the subsequent online phase, providing low-overhead inference capabilities for online operation. Once the online phase begins, the offline training module itself is no longer invoked.

[0031] Specifically, the offline training module performs the following steps: Step S1.1: Collect runtime logs containing training window-level metrics and cache entry-level event information.

[0032] Specifically, training window-level metrics include: average / quantile data waiting time, hit / miss rate, object (vector / sample) size distribution, decoding time statistics, sequential access ratio, importance distribution drift metric, cache pressure and prefetch waste ratio; cache entry-level event information includes: each hit / miss event, the size and media type of the evicted entry, decoding / copyback latency, replacement events and subsequent actual access.

[0033] Step S1.2: Based on the running log, construct a benefit-related label for each cached entry in each training window. The benefit-related label includes the waiting time saved per hit and the expected number of hits in the next window. Specifically, the waiting time saved per hit is based on the difference in waiting time between the missed batch and the hit batch in the same training batch, weighted according to the entry size and decoding overhead, and stabilized using an exponential moving average; the expected number of hits in the next window is statistically estimated based on the sampling weight and window length; for the waiting time that does not actually occur in the current window due to substitution events, a bucket-based counterfactual approximation is used, and the round-end correction is performed using the measured latency during subsequent actual loading to form the profit target.

[0034] Further, in step S1.2, a revenue target value is constructed for each cached entry based on the following relationship:

[0035] in, The savings per unit hit are obtained by weighting the difference between the waiting time of the missed batch and the waiting time of the hit batch in the same training batch according to the data size of the entry and the decoding cost, and then stabilizing it using an exponential moving average. The expected number of hits for the next window is estimated based on sampling weights and window length statistics. For the waiting time that does not actually occur in the current window due to alternative access, a bucket counterfactual approximation based on media type and packet granularity is used, and the actual waiting time is corrected during subsequent actual loading.

[0036] Step S1.3: Based on the running logs and revenue-related labels and representation features (including sample importance, object size and decoding cost statistics, hit / miss records, etc.), the offline context bandit method is used to train the revenue prediction model (outputting retained revenue scores) and the policy selector (outputting policy type and parameters), and the model parameters of the revenue prediction model and the policy selector are fixed for online inference (gradient updates are no longer performed in the online stage).

[0037] The profile generation module is used to collect system operation indicators at the training window granularity during online training, and to standardize and handle outliers of the system operation indicators to form profile vectors.

[0038] In this embodiment, the profile vector is generated based on the metrics collected within each online time window or step window. The system operation metrics include: average / quantile waiting time, hit rate / miss rate, cache usage, replacement / hit features, object (vector / sample) size distribution, decoding time distribution, sequential access ratio (seq_ratio), prefetch waste rate, sample importance distribution drift, and phased load changes.

[0039] Specifically, the portrait generation module is also used to calculate portrait drift metrics, which include a comprehensive measure based on the rank correlation coefficient and the rate of change of distribution entropy, used to characterize the non-stationarity of the access / importance distribution across windows.

[0040] The adaptive strategy selection module is used to input the profile vector into the fixed strategy selector, obtain the strategy parameter group corresponding to the current training window, and drive the specific execution behavior of the data plane.

[0041] In this embodiment, the strategies defined by the strategy parameter group include: cache replacement criteria, cache space quota, packing granularity, prefetch throttling threshold, and batch scheduling mode.

[0042] Specifically, the adaptive strategy selection module triggers strategy switching or increases exploration parameters based on the profile drift index, and only outputs the strategy parameter group when the preset minimum cooldown time and benefit advantage threshold are met.

[0043] The revenue inference and replacement execution module is used to perform replacement operations on candidate cache entries to be replaced when the cache capacity is limited or when the preset eviction trigger condition is reached.

[0044] In other words, when cache capacity is limited or an eviction threshold is triggered, the benefit inference and replacement execution module performs benefit-driven priority ranking and replacement decisions on candidate items, directly aligning with the optimization goal of "reducing data waiting".

[0045] Specifically, the revenue inference and replacement execution module performs the following operations: Step S4.1: Within each training window, construct a candidate set for all potentially eliminated items (such as low-quality hotspots exhausted or expired items); and collect item-level feature sets.

[0046] The feature set includes: importance score of recursive derivation, hit history statistics, object size, media type, decoding / copyback cost estimate, pack popularity and number of remaining alternative samples.

[0047] Step S4.2: Call the fixed revenue prediction model to perform forward inference on each item in the candidate set to obtain the retained revenue score; Step S4.3: When the cache reaches the replacement condition, entries are evicted in ascending order of retention benefit score to achieve benefit alignment (i.e., prioritizing the retention of entries that minimize future waiting costs). During the online phase, only inference is performed; no model parameter updates are made. The training collaborative scheduling module is used to perform scheduling at the candidate batch level (batch generated by training sampling) to ensure that the sample coverage remains unchanged in the current round and does not increase the total computational workload.

[0048] In other words, this module links caching / prefetching strategies with training scheduling at the batch execution level, balancing low latency and stable data distribution.

[0049] Specifically, the training collaborative scheduling module performs the following operations: Step S5.1: If there are high-priority samples that are not ready in the candidate batch, the candidate batch is postponed as a whole, and the batches with all samples ready are scheduled first to avoid GPU idle time.

[0050] Step S5.2: If the candidate batch contains only unread low-quality samples, select unaccessed low-quality samples from the cache (L-cache) to perform equivalent replacements on the unread low-quality samples, and record the replaced samples to the missing set. Step S5.3: At the end of the current round window, perform merged training based on the missing set.

[0051] The strategy switching control module is used to construct a cache index or queue (heapify / index building) corresponding to the new strategy at the boundary of the training window, based on the new strategy defined by the strategy parameter group and using shadow data structures (including but not limited to shadow heap or shadow index).

[0052] During the online phase, no model parameter updates are performed; forward inference and control are only based on the revenue prediction model and the strategy selector.

[0053] Specifically, the strategy switching control module performs the following operations to implement the shadow data structure mechanism: Step S6.1: At the end of each training window, perform batch reconstruction (heapify / rebuild index) of the shadow heap or shadow index based on the current cache entries and the new strategy. Step S6.2: After the shadow structure is constructed, a seamless switch between the old and new strategy structures is achieved through pointer replacement or equivalent atomic operations. Step S6.3: Set a maximum rate or a progressive threshold in the first few eliminations after the switch to avoid buffer jitter and ensure stable convergence.

[0054] Example 2: This embodiment provides a data supply acceleration method for single-machine to small-scale multi-GPU training scenarios, which is implemented on the data supply acceleration system for single-machine to small-scale multi-GPU training scenarios in the above embodiment. That is, those skilled in the art can understand the data supply acceleration method for single-machine to small-scale multi-GPU training scenarios as the operation mode of the data supply acceleration system for single-machine to small-scale multi-GPU training scenarios.

[0055] Figure 2 This is a flowchart of a data supply acceleration method for single-machine to small-scale multi-GPU training scenarios according to an embodiment of the present invention. Figure 4 This is a flowchart of the online window-level processing in an embodiment of the present invention.

[0056] like Figure 2 , Figure 4 As shown, the data provisioning acceleration method for single-machine to small-scale multi-GPU training scenarios includes the following steps: Step S1: Based on historical running logs or pre-experiment logs, perform offline learning on the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows, and output a fixed profit prediction model and a fixed policy selector. Step S2: During online training, system operation indicators are collected at the training window granularity, and the system operation indicators are standardized and outlier processing is performed to form a profile vector. Step S3: Input the image vector into the fixed policy selector to obtain the policy parameter group corresponding to the current training window; Step S4: When the cache capacity is limited or the preset eviction trigger condition is reached, perform a replacement operation on the candidate cache entries to be evicted. Step S5: Perform scheduling at the candidate batch level to ensure that the sample coverage remains unchanged in the current round and does not increase the total computational workload. Step S6: At the boundary of the training window, according to the new policy defined by the policy parameter group, construct the cache index or queue corresponding to the new policy using the shadow data structure.

[0057] Figure 3 This is the offline training pipeline diagram (data acquisition → feature engineering → label construction → model training → solidification) in an embodiment of the present invention.

[0058] like Figure 3 As shown, step S1 performs the following steps: Step S1.1: Collect runtime logs containing training window-level metrics and cache entry-level event information.

[0059] Specifically, training window-level metrics include: average / quantile data waiting time, hit / miss rate, object (vector / sample) size distribution, decoding time statistics, sequential access ratio, importance distribution drift metric, cache pressure and prefetch waste ratio; cache entry-level event information includes: each hit / miss event, the size and media type of the evicted entry, decoding / copyback latency, replacement events and subsequent actual access.

[0060] Step S1.2: Based on the running log, construct a benefit-related label for each cached entry in each training window. The benefit-related label includes the waiting time saved per hit and the expected number of hits in the next window. Specifically, the waiting time saved per hit is based on the difference in waiting time between the missed batch and the hit batch in the same training batch, weighted according to the entry size and decoding overhead, and stabilized using an exponential moving average; the expected number of hits in the next window is statistically estimated based on the sampling weight and window length; for the waiting time that does not actually occur in the current window due to substitution events, a bucket-based counterfactual approximation is used, and the round-end correction is performed using the measured latency during subsequent actual loading to form the profit target.

[0061] Further, in step S1.2, a revenue target value is constructed for each cached entry based on the following relationship:

[0062] in, The savings per unit hit are obtained by weighting the difference between the waiting time of the missed batch and the waiting time of the hit batch in the same training batch according to the data size of the entry and the decoding cost, and then stabilizing it using an exponential moving average. The expected number of hits for the next window is estimated based on sampling weights and window length statistics. For the waiting time that does not actually occur in the current window due to alternative access, a bucket counterfactual approximation based on media type and packet granularity is used, and the actual waiting time is corrected during subsequent actual loading.

[0063] Step S1.3: Based on the running logs and revenue-related labels and representation features (including sample importance, object size and decoding cost statistics, hit / miss records, etc.), the offline context bandit method is used to train the revenue prediction model (outputting retained revenue scores) and the policy selector (outputting policy type and parameters), and the model parameters of the revenue prediction model and the policy selector are fixed for online inference (gradient updates are no longer performed in the online stage).

[0064] Specifically, step S4 performs the following operations: Step S4.1: Within each training window, construct a candidate set for all potentially eliminated items (such as low-quality hotspots exhausted or expired items); and collect item-level feature sets.

[0065] The feature set includes: importance score of recursive derivation, hit history statistics, object size, media type, decoding / copyback cost estimate, pack popularity and number of remaining alternative samples.

[0066] Step S4.2: Call the fixed revenue prediction model to perform forward inference on each item in the candidate set to obtain the retained revenue score; Step S4.3: When the cache reaches the replacement condition, entries are evicted in ascending order of retention benefit score to achieve benefit alignment (i.e., prioritizing the retention of entries that minimize future waiting costs). During the online phase, only inference is performed; no model parameter updates are made. Specifically, step S5 performs the following operations: Step S5.1: If there are high-priority samples that are not ready in the candidate batch, the candidate batch is postponed as a whole, and the batches with all samples ready are scheduled first to avoid GPU idle time.

[0067] Step S5.2: If the candidate batch contains only unread low-quality samples, select unaccessed low-quality samples from the cache (L-cache) to perform equivalent replacements on the unread low-quality samples, and record the replaced samples to the missing set. Step S5.3: At the end of the current round window, perform merged training based on the missing set.

[0068] Specifically, step S6 performs the following operations to implement the mechanism of the shadow data structure: Step S6.1: At the end of each training window, perform batch reconstruction (heapify / rebuild index) of the shadow heap or shadow index based on the current cache entries and the new strategy. Step S6.2: After the shadow structure is constructed, a seamless switch between the old and new strategy structures is achieved through pointer replacement or equivalent atomic operations. Step S6.3: Set a maximum rate or a progressive threshold in the first few eliminations after the switch to avoid buffer jitter and ensure stable convergence.

[0069] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0070] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A data supply acceleration system for single-machine to small-scale multi-GPU training scenarios, characterized in that, include: The offline training module is used to collect and analyze the running logs, and to learn the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows offline, and output a fixed profit prediction model and a fixed policy selector. The profile generation module is used to collect system operation indicators at the training window granularity during online training, and to standardize and handle outliers of the system operation indicators to form profile vectors. An adaptive strategy selection module is used to input the profile vector into the fixed strategy selector to obtain the strategy parameter group corresponding to the current training window; The revenue inference and replacement execution module is used to perform replacement operations on candidate cache entries to be replaced when the cache capacity is limited or when the preset eviction trigger condition is reached. The training collaborative scheduling module is used to perform scheduling at the candidate batch level to ensure that the sample coverage in each round remains unchanged and does not increase the total computational workload. The strategy switching control module is used to construct a cache index or queue corresponding to the new strategy at the boundary of the training window, based on the new strategy defined by the strategy parameter group and using a shadow data structure.

2. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The offline training module performs the following operations: Step S1.1: Collect runtime logs containing training window-level metrics and cache entry-level event information; Step S1.2: Based on the running log, construct a benefit-related label for each cached entry in each training window. The benefit-related label includes the waiting time saved per hit and the expected number of hits in the next window. Step S1.3: Based on the running logs and the revenue-related tags, train the revenue prediction model and the policy selector using the offline context bandit method, and solidify the model parameters of the revenue prediction model and the policy selector.

3. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 2, characterized in that, In step S1.2, the waiting time saved by a unit hit is based on the difference in waiting time between the missing batch and the hit batch in the same training batch, weighted according to the entry size and decoding overhead, and stabilized using an exponential moving average. The expected number of hits in the next window is statistically estimated based on the sampling weight and window length. For the waiting time that does not actually occur in the current window due to the substitution event, a bucket-style counterfactual approximation is adopted, and the end-of-cycle correction is performed with the measured delay during subsequent actual loading to form the revenue target.

4. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 3, characterized in that, In step S1.2, a revenue target value is constructed for each cache entry based on the following relationship: in, The savings per unit hit are obtained by weighting the difference between the waiting time of the missed batch and the waiting time of the hit batch in the same training batch according to the size of the entry data and the decoding cost, and then stabilizing it using an exponential moving average. The expected number of hits for the next window is estimated based on sampling weights and window length statistics. For the waiting time that does not actually occur in the current window due to alternative access, a bucket counterfactual approximation based on media type and packet granularity is used, and the actual waiting time is corrected during subsequent actual loading.

5. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The image generation module is also used to calculate the image drift index, which includes a comprehensive measure based on the rank correlation coefficient and the rate of change of distribution entropy.

6. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The adaptive strategy selection module triggers strategy switching or increases exploration parameters based on the profile drift index, and only outputs the strategy parameter group when the preset minimum cooldown time and benefit advantage threshold are met.

7. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The revenue inference and replacement execution module performs the following operations: Step S4.1: Within each training window, construct a candidate set for all potentially eliminated items; and collect item-level feature sets; Step S4.2: Call the fixed revenue prediction model to perform forward inference on each item in the candidate set to obtain the retention revenue score; Step S4.3: When the cache reaches the replacement condition, the entries are eliminated in order of retention benefit score from low to high.

8. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The training collaborative scheduling module performs the following operations: Step S5.1: If there are high-priority samples that are not ready in the candidate batch, the candidate batch is postponed as a whole, and the batches with all samples ready are scheduled first. Step S5.2: If the candidate batch contains only unread low-quality samples, select unvisited low-quality samples from the cache to perform equivalent replacement on the unread low-quality samples, and record the replaced samples to the missing set. Step S5.3: At the end of the current window, perform merged training based on the missing set.

9. The data supply acceleration system for single-machine to small-scale multi-GPU training scenarios as described in claim 1, characterized in that, The strategy switching control module performs the following operations: Step S6.1: At the end of each training window, perform batch reconstruction of the shadow heap or shadow index based on the current cached entries and the new strategy; Step S6.2: After the shadow structure is constructed, a seamless switch between the old and new strategy structures is achieved through pointer replacement or equivalent atomic operations. Step S6.3: Set a maximum rate or a progressive threshold in the first few eliminations after the switch to avoid buffer jitter and ensure stable convergence.

10. A method for accelerating data provisioning in single-machine to small-scale multi-GPU training scenarios, based on the data provisioning acceleration system for single-machine to small-scale multi-GPU training scenarios as described in any one of claims 1-9, characterized in that, Includes the following steps: Step S1: Based on historical running logs or pre-experiment logs, perform offline learning on the I / O and caching behavior of single-machine to small-scale multi-GPU training tasks within several preset training windows, and output a fixed profit prediction model and a fixed policy selector. Step S2: During online training, system operation indicators are collected at the training window granularity, and the system operation indicators are standardized and outlier processing is performed to form a profile vector. Step S3: Input the image vector into the fixed policy selector to obtain the policy parameter group corresponding to the current training window; Step S4: When the cache capacity is limited or the preset eviction trigger condition is reached, perform a replacement operation on the candidate cache entries to be evicted. Step S5: Perform scheduling at the candidate batch level to ensure that the sample coverage remains unchanged in the current round and does not increase the total computational workload. Step S6: At the boundary of the training window, according to the new policy defined by the policy parameter group, construct the cache index or queue corresponding to the new policy using the shadow data structure.

Images