Data loading methods for distributed system computing nodes and learning models, and corresponding systems

Abstract

This specification provides embodiments of a distributed system computing node, a data loading method for a learning model, and a corresponding system. The data loading method for the machine learning model includes: determining the data access sequence for each iteration of the machine learning model; in a first processing step, preloading sample data corresponding to the data access sequence of each iteration into a cache pool according to the order of the iterations; in a second processing step, reading the corresponding sample data from the cache pool according to the order of the iterations; wherein the first and second processing steps are executed in parallel; and in a third processing step, inputting the sample data into the machine learning model for iterative training according to the order of the iterations; wherein the third processing step is executed in parallel with the second processing step.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data loading method for distributed system computing nodes and learning models. Background Technology

[0002] With the continuous development of artificial intelligence technology, machine learning models are constantly expanding, placing increasingly higher demands on computing, storage, and I / O performance. To meet these high-performance requirements, traditional data loading schemes cache recently accessed data. Frequently accessed data remains in the cache indefinitely, while infrequently accessed data is evicted through a timeout mechanism. This approach is effective in scenarios with localized hotspots, where most data can be read from the cache without accessing remote storage. However, in scenarios without localized hotspots, model data loading is slow, and simply optimizing the backend data resource layer is no longer sufficient to meet performance requirements. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a data loading method for distributed system computing nodes and learning models. One or more embodiments of this specification also relate to a data loading device for a machine learning model, a distributed machine learning model training system, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data loading method for a distributed system computing node is provided, comprising: pre-registering local memory resources as memory based on RDMA transmission to establish a cache pool; determining the data access sequence of multiple iterations of a machine learning model; in a first processing step, preloading sample data corresponding to the data access sequence of each of the multiple iterations into the cache pool according to the order of the multiple iterations; in a second processing step, reading the corresponding sample data from the cache pool according to the order of the multiple iterations, wherein the first processing step and the second processing step are executed in parallel; and in a third processing step, inputting the sample data into the machine learning model for iterative training according to the order of the multiple iterations, wherein the third processing step and the second processing step are executed in parallel.

[0005] According to a second aspect of the embodiments of this specification, a data loading method for a machine learning model is provided, comprising: determining a data access sequence for each of multiple iterations of the machine learning model; in a first processing step, preloading sample data corresponding to the data access sequence for each of the multiple iterations into a cache pool according to the order of the multiple iterations; in a second processing step, reading the corresponding sample data from the cache pool according to the order of the multiple iterations, wherein the first processing step and the second processing step are executed in parallel; and in a third processing step, inputting the sample data into the machine learning model for iterative training according to the order of the multiple iterations, wherein the third processing step and the second processing step are executed in parallel.

[0006] According to a third aspect of the embodiments of this specification, a data loading apparatus for a machine learning model is provided, comprising: a sequence determination module configured to determine a data access sequence for each of multiple iterations of the machine learning model; a data preloading module configured to, in a first processing step, preload sample data corresponding to the data access sequences of each of the multiple iterations into a cache pool according to the chronological order of the multiple iterations; a data pre-reading module configured to, in a second processing step, read the corresponding sample data from the cache pool and send it to the data loading module according to the chronological order of the multiple iterations, wherein the first processing step and the second processing step are executed in parallel; and a data loading module configured to, in a third processing step, input the received sample data into the machine learning model according to the chronological order of the multiple iterations, so that the machine learning model is trained iteratively, wherein the third processing step and the second processing step are executed in parallel.

[0007] According to a fourth aspect of the embodiments of this specification, a distributed machine learning model training system is provided, comprising: backend storage, a cache pool, and multiple computing nodes for parallel training of a machine learning model. The backend storage is configured to store the training dataset of the machine learning model. The computing nodes are configured to determine a data access sequence for multiple iterations of the machine learning model; in a first processing step, sample data corresponding to the data access sequence of each iteration is preloaded into the cache pool according to the order of the multiple iterations; in a second processing step, the corresponding sample data is read from the cache pool according to the order of the multiple iterations; wherein the first processing step and the second processing step are executed in parallel; and in a third processing step, the sample data is input into the machine learning model for iterative training according to the order of the multiple iterations; wherein the third processing step and the second processing step are executed in parallel.

[0008] According to a fifth aspect of the embodiments of this specification, a computing device is provided, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the computer-executable instructions are executed by the processor, the steps of the data loading method for a distributed system computing node or the data loading method for a machine learning model described in any embodiment of this specification are implemented.

[0009] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data loading method for a distributed system computing node or the data loading method for a machine learning model described in any embodiment of this specification.

[0010] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the data loading method for the distributed system computing node or the data loading method for the machine learning model described above.

[0011] This specification provides an embodiment of a data loading method for a machine learning model. Since this method determines the data access sequence for each iteration of the machine learning model, in a first processing step, sample data corresponding to the data access sequence of each iteration is preloaded into a cache pool according to the order of the iterations. In a second processing step, the corresponding sample data is read from the cache pool and sent to the data loading module according to the order of the iterations. The first and second processing steps are executed in parallel. In a third processing step, the data loading module iteratively inputs the received sample data into the machine learning model according to the order of the iterations, enabling the machine learning model to be trained iteratively. The third processing step and the second processing step are executed in parallel. The parallel execution of the processing demonstrates that this method leverages the predictability of data access sequences for sample data in machine learning model training tasks. It implements a two-stage pipeline for proactive data loading. The first stage proactively preloads the sample data required for multiple iterations of the machine learning model into a cache pool in the order of iterations through a first processing step. The second stage, concurrent with the first stage, reads the pre-fetched sample data from the cache pool. Then, through a third processing step running in parallel with the second stage, the sample data is iteratively input into the machine learning model. This overlaps the data pre-fetching process with the model training process, minimizing the model's data waiting time, resolving the data I / O bottleneck, improving data loading speed, and meeting the performance requirements of ever-expanding machine learning models. Attached Figure Description

[0012] Figure 1 This is a schematic diagram illustrating an application scenario of the data loading method for a machine learning model provided in one embodiment of this specification on a single machine.

[0013] Figure 2 This is a schematic diagram illustrating an application scenario of the data loading method for a machine learning model provided in one embodiment of this specification in a distributed system.

[0014] Figure 3 This is a flowchart illustrating a data loading method for a machine learning model according to one embodiment of this specification;

[0015] Figure 4 This is a spacetime diagram provided in one embodiment of this specification;

[0016] Figure 5 This is a spacetime diagram provided in another embodiment of this specification;

[0017] Figure 6 This is a flowchart illustrating the data loading process of a machine learning model according to one embodiment of this specification.

[0018] Figure 7 This is a schematic diagram of the structure of a data loading device for a machine learning model provided in one embodiment of this specification;

[0019] Figure 8 This is a schematic diagram of the structure of a distributed machine model training system provided in one embodiment of this specification;

[0020] Figure 9 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0021] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0022] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0023] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0024] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0025] RDMA (Remote Direct Memory Access) is a technology that allows user applications to directly access (read or write) remote memory without memory copying or kernel interference.

[0026] High-performance AI cluster: A group of independent computing nodes interconnected by a high-performance network.

[0027] DLT: Deep Learning Task.

[0028] DNN: Deep Neural Networks.

[0029] With the continuous development of artificial intelligence technology, machine learning models are constantly expanding, placing increasingly higher demands on computing and storage performance. While multi-machine, multi-GPU training clusters provide solutions to the computing power requirements, data I / O has become a key bottleneck affecting training time in many scenarios. In scenarios without localized hotspots, the computing cluster-side cache cannot improve performance by preloading the entire dataset. Therefore, simply optimizing the backend data resource layer (such as storage or database) is no longer sufficient to meet the high-concurrency data I / O demands. In view of this, this specification provides a data loading method for machine learning models to achieve proactive data loading acceleration for AI (Artificial Intelligence) deep learning, thereby accelerating I / O performance, avoiding waste of computing resources, and improving computing efficiency.

[0030] To make the methods provided in the embodiments of this specification easier to understand, firstly, in conjunction with Figure 1 and Figure 2 The illustrated application scenario diagram provides a schematic description of the application scenarios involved in the methods provided in the embodiments of this specification.

[0031] Figure 1The diagram illustrates an application scenario of the data loading method for a machine learning model provided in one embodiment of this specification, applied on a single machine. Figure 1 As shown in the embodiments of this specification, the application scenarios provided include: server 120.

[0032] Server 120 may include servers providing various services. For example, server 120 may provide AI services to clients, such as image recognition, speech recognition, and text prediction; it may also be a server supporting the background training of machine learning models used on clients or other servers; or it may be a server processing data sent by clients. Specifically, server 120 may be a server in a distributed system's server cluster, or a server integrated with blockchain. Server 120 may also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; or it may be an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0033] It should be noted that server 120 can be a standalone server or any server in a server cluster. For example... Figure 1 As shown, server 120 may include: a data preloading module, a data loading module, a cache pool, and a storage device. The data loading module, based on the predictability of the data access sequences of sample data for the machine learning model training task, first predicts the data access sequences for each iteration of the machine learning model and sends these sequences to the data preloading module. In the first processing step, the data preloading module preloads the sample data corresponding to the data access sequences of each iteration into the cache pool according to the order of the iterations. For example, the data preloading module can send a preloading instruction to the storage device to preload the corresponding sample data into the cache pool. In the second processing step, the data preloading module pre-reads the corresponding sample data from the cache pool according to the order of the iterations and sends it to the data loading module. The first and second processing steps are executed in parallel. In the third processing step, the data loading module inputs the received sample data into the machine learning model according to the order of the iterations, allowing the machine learning model to be trained iteratively. The third processing step is executed in parallel with the second processing step. Figure 1As shown in the processing diagram, during the training of the machine learning model in the current iteration, the data preloading module can simultaneously pre-read the sample data required for the next iteration from the cache pool in the second processing step and provide it to the data loading module. Thus, when the training of the machine learning model in the current iteration ends, the data loading module can promptly input the sample data required for the next iteration into the machine learning model for the next iteration of training. The machine learning model does not need to wait for data, thereby reducing the model data waiting time and improving the data loading speed.

[0034] Figure 2 The diagram illustrates an application scenario of a data loading method for a machine learning model provided in another embodiment of this specification, used in a distributed system. Figure 2 As illustrated in the application scenario, this specification also provides a data loading method for a distributed system computing node, comprising: pre-registering local memory resources as RDMA-based memory to establish a cache pool; determining the data access sequence for each iteration of a machine learning model; in a first processing step, preloading sample data corresponding to the data access sequence of each iteration into the cache pool according to the order of the iterations; in a second processing step, reading the corresponding sample data from the cache pool according to the order of the iterations, wherein the first processing step and the second processing step are executed in parallel; in a third processing step, inputting the sample data into the machine learning model for iterative training according to the order of the iterations, wherein the third processing step and the second processing step are executed in parallel. Specifically, as shown in the example... Figure 2 As shown in the embodiments of this specification, the distributed system includes multiple computing nodes, from DLT Worker0 to DLT WorkerN, wherein each computing node is equipped with an active data loading acceleration device. The active data loading acceleration device includes a data loading module (DataLoader) and a data preloading module (Preloader). Multiple computing nodes can execute training tasks in parallel, and different devices communicate with each other via a network. Each computing node registers its local memory resources as memory that can perform both inter-process communication and RDMA-based transfer, resulting in a globally unified, RDMA-based distributed resource pool, i.e., a cache pool, presented to the user.

[0035] The data loading module can run on compute nodes, providing the sample data required for computation of the machine learning model (DNNModel) for upper-layer deep learning tasks. At the beginning of each epoch (an entire dataset traversed once), the data loading module on each compute node determines a data access sequence for multiple iterations and passes this sequence to the data preloading module of that node. Different compute nodes are assigned different data access sequences; therefore, each compute node processes its own data access sequence based on a similar processing flow.

[0036] In the first processing step, the data preloading module preloads the sample data corresponding to the data access sequence of each of the multiple iterations from the backend storage to the cache pool according to the order of the multiple iterations. In other words, the deep learning working dataset is stream-cached into the distributed resource pool according to the predicted sequence.

[0037] In the second processing step, the data loading module reads the corresponding sample data from the cache pool according to the order of the multiple iterations, either through memory copying or direct RDMA access. Compared to remote storage, the data loading module can achieve faster data loading I / O transmission speed. Of course, the data loading module can also utilize the data preloading module to execute the second processing step to obtain sample data; the method provided in this specification does not limit this approach.

[0038] In the third processing step, the data loading module iteratively inputs the received sample data into the machine learning model according to the order of the multiple iterations, enabling the machine learning model to be trained iteratively. This third processing step is executed in parallel with the second processing step. Therefore, at the beginning of each iteration, the data loading module can promptly load the sample data pre-fetched in the second processing step, corresponding to the data access sequence of the current iteration, into the machine learning model of that computing node for training the current iteration.

[0039] Understandably, since the third processing step is executed in parallel with the second processing step, during the current iteration of training, the data loading module can simultaneously read the sample data required for the next iteration from the cache pool during the second processing step. At the end of the current iteration, the sample data required for the next iteration can be promptly input into the machine learning model for training, thus reducing model data waiting time. Furthermore, the parallel training of the machine learning model by multiple computing nodes further improves the model training speed.

[0040] The parallelism among the first, second, and third processing steps can be achieved through multiple parallel threads or multiple parallel processes. In the process-based implementation, the processes do not interfere with each other, allowing the parallel execution steps to overlap spatially. This meets the high-concurrency data I / O requirements of large-scale distributed AI training cluster platforms, provides optimized data loading performance, and greatly improves the performance of large-scale, high-concurrency AI.

[0041] The data loading method, data loading device, distributed machine learning model training system, computing device, and computer-readable storage medium for the machine learning model provided in this specification will be described in detail in the following embodiments.

[0042] See Figure 3 , Figure 3 A flowchart is shown of a data loading method for a machine learning model according to an embodiment of this specification, which specifically includes the following steps.

[0043] Step 302: Determine the data access sequence for each iteration of the machine learning model.

[0044] The machine learning model described herein is an expression of an artificial intelligence (AI) algorithm that searches for patterns or makes predictions by analyzing massive amounts of data. During iterative training, the machine learning model continuously adjusts its parameters (such as weights and biases) until it achieves the weights and biases that minimize potential loss, resulting in a machine learning model suitable for practical application. The specific application areas of the machine learning model in the embodiments of this specification are not limited. For example, the machine learning model can be used for various AI tasks such as image recognition, speech recognition, and text prediction, and can be, for example, a machine learning model for deep neural networks or deep learning tasks.

[0045] The data access sequence refers to the serialized representation of the sample data input into the machine learning model during iteration. One sample corresponds to one data access sequence, and several samples correspond to several data access sequences. Specifically, the data access sequence can be represented by sequential numbers or random numbers, or any other symbols that can distinguish each sample.

[0046] The multiple iterations refer to any number of iterations the machine learning model undergoes from the start to the end of training. In AI training, training all samples once is called an epoch. Since the amount of sample data in an epoch is very large, an epoch is divided into multiple batches, each containing several samples, and each sample corresponds to a data access sequence. The machine learning model is trained batch by batch, and the training of one batch is called one iteration.

[0047] For example, suppose a complete dataset for a visual machine learning model contains 16,000 images. It's generally impossible to train on all the images at once; instead, training is done in batches. Assuming a batch size of 100, then one iteration corresponds to 100 images. If the visual machine learning model is trained on a single machine with a single GPU, it will require a total of 160 iterations. If the model is trained in parallel using multiple GPUs, such as an eight-GPU machine, it will require 20 iterations. Alternatively, with two machines and sixteen GPUs, it will require 10 iterations. Here, a GPU can be understood as a processor, and one GPU corresponds to one computing node.

[0048] Combination Figure 2For example, the data loading module can determine the data access sequences for multiple iterations of the machine learning model and send these sequences to the data preloading module. Assuming the machine learning model training is performed by a GPU (graphics processing unit) card in the computing node, one GPU card constitutes one computing node. Each GPU card can be assigned a corresponding data loading module. The data loading module can assign a number to each sample in the complete training dataset, thus obtaining the data access sequence for each sample. For example, the data access sequences for 16,000 images could range from 0 to 15,999. To facilitate the determination of data access sequences, in one embodiment, a data sampler can generate batches of data access sequences. For example, assuming a batch size of batch_size = 100, each batch generates 100 data access sequences, corresponding to 100 images. Assuming the data sampler generates the numbers sequentially, the corresponding data access sequence numbers are: 0-99, 100-199, etc. Of course, the data sampler can also generate random data access sequences. That is, within the same epoch, the data sampler will not generate the same data access sequences. The data sampler assigns the sequence numbers of the data access sequences for 16,000 images to each data loading module, ensuring that the GPUs corresponding to these data loading modules train on different images. At the beginning of each epoch, the data sampler generates data access sequences for each sample in multiple iterations and assigns them to the data loading modules. Therefore, the data loading modules can determine their respective data access sequences for multiple iterations. For example, assuming there are two devices with eight GPUs each, there are a total of 16 compute nodes. Assuming a batch size of 100, the 16,000 images are divided into 10 iterations. Assuming the data access sequences for the 16,000 images are: 0-99, 100-199, ..., the data access sequences for the machine learning model of one compute node over 10 iterations can be determined as: 0-99, 100-199, ..., 900-999, and so on for the machine learning models of other compute nodes.

[0049] Step 304: In the first processing, according to the order of the multiple iterations, the sample data corresponding to the data access sequence of each of the multiple iterations is preloaded into the cache pool.

[0050] The methods provided in the embodiments of this specification can be applied to a single machine or a distributed cluster. For example, when the method is applied to a single machine, the cache pool can be represented as the local memory space of the single machine. As another example, when the method is applied to a distributed cluster, the cache pool can be represented as RDMA memory with globally unified addressing presented to users in the cluster. Specifically, it can utilize the local memory resources of each computing node in the cluster, registering the local memory of each computing node as memory capable of both inter-process communication and RDMA transfer.

[0051] By combining an RDMA-based distributed resource pool implementation and leveraging the predictability of AI training access sequences, a separate process is launched in the background to preload sample data accurately based on the predicted access sequences. This preloads the sample data from the slow backend storage into the RDMA-based distributed resource pool in an orderly manner. Then, high-performance RDMA transmission is used to launch another separate process to perform high-performance pre-fetching from the distributed resource pool to the data loading module. Through the combination of these two approaches, near-end caching of data and high-performance data transmission are achieved, which greatly shortens the idle time of the GPU waiting for data, improves GPU utilization, and accelerates the performance of AI training.

[0052] Using the example of 16,000 images above, the multiple iterations refer to 10 iterations. The data access sequence for the machine learning model on one computing node, from beginning to end, is: 0-99, 100-199, ... 900-999. Therefore, during the preloading process, the corresponding images are preloaded into the cache pool in the order of 0-99, 100-199, ... 900-999. This ensures that data from the earlier iterations enters the cache pool first, and data flows into the cache pool in an orderly manner according to training needs.

[0053] The actual data corresponding to the data access sequence can be stored on any storage device. For example, such as Figure 2 The distributed system shown has its actual data stored in the backend storage devices. The preloading specifically refers to retrieving the actual data from the backend storage and placing it into a cache pool.

[0054] See Figure 4 The spatiotemporal diagram shown can provide a more vivid understanding of the first processing step, such as... Figure 4 The process shown in the spatiotemporal diagram, where backend data is preloaded into the RDMA distributed resource pool of the computing cluster, is the first processing step.

[0055] It should be noted that in step 304, the sample data is preloaded in the order of multiple iterations. This serves two purposes: firstly, to ensure that sample data from earlier iterations is read first and trained first; and secondly, to maintain a first-in, first-out (FIFO) principle, as sample data placed in the cache pool early on and used for training is rarely used again in the current training. Therefore, when cache space is insufficient, the cache space can be released after the earlier data is used. For example, in one embodiment, to facilitate data reading and space release, the data access sequence numbering order is consistent with the iteration order. Therefore, sample data can be read and the cache space for trained data can be released according to the data access sequence number in the iteration order.

[0056] Step 306: In the second processing, the corresponding sample data is read from the cache pool according to the order of the multiple iterations, wherein the first processing and the second processing are executed in parallel.

[0057] The term "parallelism" refers to the fact that the preloading process is independent of and does not interfere with the process of reading sample data from the cache pool, and can be executed in parallel. For example, the first processing process can be an independent process, and the second processing process can be another independent process. Furthermore, to facilitate timely retrieval of the sample data required for the current iteration from the cache pool, the preloading process can be monitored. Based on the monitoring results, it can be determined whether the data required for the current iteration has entered the cache pool, and the timing for the second processing process to begin parallel reading can be determined in conjunction with the actual needs of model training.

[0058] The sample data read from the cache pool can be temporarily loaded into the local CPU memory or GPU memory of the computing node. On one hand, temporarily loading into local CPU memory or GPU memory allows the sample data to be input into the machine learning model closer to the processor, improving the I / O of the machine learning model. On the other hand, temporarily loading into local CPU memory or GPU memory also facilitates the computing node to perform relevant data preprocessing on the sample data. It should be noted that in the methods provided in the embodiments of this specification, the computing node can be implemented using a GPU card processor or a CPU processor; this specification does not impose any limitations on this. In application scenarios using GPU cards, the sample data can be read into GPU memory, resulting in better performance acceleration. In application scenarios without GPU cards, or in application scenarios where it is not possible to directly load the sample data into GPU memory, the sample data can be loaded into local CPU memory.

[0059] In addition, after reading the sample data from the cache pool, the read sample data can be preprocessed according to preset data preprocessing rules. Therefore, before the sample data is input into the machine learning model, the method further includes: preprocessing the received sample data according to preset data preprocessing rules during the second processing step.

[0060] The preset data preprocessing rules can be set according to the needs of different application scenarios. For example, in the application scenario of training machine learning models for image recognition, the preset data preprocessing rules may include: image resizing rules to adjust images of different sizes to the same pixel size, image normalization rules, standardization rules, binarization rules, missing value imputation rules, etc. As another example, in the application scenario of training machine learning models for text prediction, the preset data preprocessing rules may include: text redundancy deduplication rules to remove duplicate and redundant text information, text segmentation rules to divide words in sentences, etc.

[0061] See Figure 4 The spacetime diagram shown can more vividly illustrate the meaning of parallelism. For example... Figure 4 As shown, the second processing step involves the computing cluster's RDMA distributed resource pool being loaded into the data loading module, Dataloader. During this second processing step, the sample data can be preprocessed by Dataloader. In practical applications, if the sample data does not require preprocessing, the preprocessing step can be omitted; the method provided in this specification does not impose any limitations on this. Figure 4 As shown, the process of reading the data required for the current iteration from the cache pool overlaps with the preloading process in time and space, and is executed in parallel, which can effectively eliminate data waiting time and improve efficiency.

[0062] Step 308: In the third processing step, the sample data is input into the machine learning model in the order of the multiple iterations for iterative training, wherein the third processing step is executed in parallel with the second processing step.

[0063] Using the example of image recognition using a visual machine learning model, let's assume that the data access sequence for the first iteration in a series of iterations consists of images numbered 0 to 99. In this case, images numbered 0 to 99 can be input into the visual machine learning model during the first iteration to train it. Similarly, if the data access sequence for the second iteration consists of images numbered 100 to 99, images numbered 100 to 199 can be input into the model during the second iteration to train it. This process continues until the final iteration of the series begins the training process.

[0064] It is understandable that, since the third processing step and the second processing step are executed in parallel, during the training of the machine learning model, the second processing step can simultaneously retrieve the sample data required for the next iteration from the cache pool. This ensures that when the training of the current iteration in the third processing step ends, the sample data for the next iteration is promptly input into the machine learning model for the next iteration of training. Therefore, the parallelism mentioned in this step refers to the process of inputting sample data into the machine learning model for the current iteration of training being independent of and does not interfere with the process of retrieving the sample data required for the next iteration from the cache pool; they can be executed in parallel.

[0065] In addition, after reading the sample data required for the next iteration from the cache pool, the read sample data can be preprocessed according to the preset data preprocessing rules. For details, please refer to the description in step 306, which will not be elaborated here.

[0066] For example, such as Figure 4 The spatiotemporal diagram shown indicates that the "computation" in the third processing step is the model training process. Through... Figure 4 It is evident that the computation process of model training and the process of retrieving the sample data required for the next iteration from the cache pool overlap in time and space, and are executed in parallel. That is, when a "computation", i.e. the training of the current iteration, ends, the sample data required for the next iteration has already been pre-fetched from the cache pool. Therefore, the sample data required for the next iteration can be input into the machine learning model in a timely manner for the next iteration of training, thereby effectively eliminating the data waiting time of the machine learning model, improving efficiency, and greatly enhancing the performance of large-scale high-concurrency model training.

[0067] Since this method determines the data access sequence for each iteration of the machine learning model, in the first processing step, the sample data corresponding to the data access sequence of each iteration is preloaded into a cache pool according to the order of the iterations. In the second processing step, the corresponding sample data is read from the cache pool and sent to the data loading module according to the order of the iterations. The first and second processing steps are executed in parallel. In the third processing step, the data loading module iteratively inputs the received sample data into the machine learning model according to the order of the iterations, allowing the machine learning model to be trained iteratively. The third processing step is executed in parallel with the second processing step. Therefore, this method... This method leverages the predictability of data access sequences for sample data used in machine learning model training tasks. It implements a two-stage pipeline for proactive data loading. The first stage proactively preloads the sample data required for multiple iterations of the machine learning model into a cache pool in the order of iterations through a first processing step. The second stage, concurrent with the first stage, reads the sample data required for iterations from the cache pool through a second processing step. Then, through a third processing step that runs in parallel with the second process, the sample data is iteratively input into the machine learning model. This overlaps the data pre-reading process with the model training process, minimizing the model's data waiting time, resolving the data I / O bottleneck, improving data loading speed, and meeting the performance requirements of ever-expanding machine learning models.

[0068] For example, Figure 4 The spacetime diagram shown is Figure 5 As can be seen from the comparison with another spacetime diagram shown, such as Figure 5 In the scenario where data I / O becomes a performance bottleneck, data waiting time exists between model calculations, causing the expensive GPU to remain idle. However, the method implemented according to the embodiments of this specification... Figure 4 The spatiotemporal diagram shown illustrates how parallel pipelines of data preloading and data prefetching streams overlap the data preloading and training computation processes, eliminating data waiting time, thereby improving GPU utilization and reducing AI training time.

[0069] The following example of image recognition using a visual machine learning model illustrates the data loading process of the machine learning model. In this embodiment, the first, second, and third processing steps are respectively represented as the first process, the second process, and the third process.

[0070] Specifically, suppose a computing node determines the data access sequence corresponding to each of the 10 iterations of a machine learning model as follows: images numbered 0 to 99, images numbered 100 to 199, images numbered 20 to 299, ... images numbered 900 to 999.

[0071] In the first process, images numbered 0-99, 100-199, 20-299, ... 900-999 are preloaded into the cache pool in sequence from the backend storage.

[0072] In parallel with the first process, the second process reads images 0-99 from the cache pool and preprocesses the images 0-99.

[0073] In parallel with the second process, in the third process, images from 0 to 99 are input into the machine learning model for training in the current iteration.

[0074] In parallel with the third process, the second process reads images 100-199 from the cache pool, preprocesses the images 100-199, and then passes the processed images to the third process.

[0075] When the training of images 0-99 in the third process is completed, images 100-199 are input into the machine learning model in the third process to train the model with images 100-199.

[0076] In parallel with the third process, the second process reads images 200-299 from the cache pool, preprocesses the images 200-299, and then passes the processed images to the third process.

[0077] When the training of images 100-199 is completed in the third process, images 200-299 are input into the machine learning model in the third process to train the model with images 200-199.

[0078] ...

[0079] This process continues until the third process has trained on 900 to 999 images, at which point the loop stops.

[0080] As can be seen from the above processing, the first process and the second process are executed in parallel and overlapping manner in time and space, and the second process and the third process are executed in parallel and overlapping manner in time and space. This can provide optimized data loading performance, effectively reduce data loading time, eliminate data waiting time for the model, and eliminate data waiting time for model calculation, thus greatly improving the performance of large-scale high-concurrency AI.

[0081] It should be noted that the examples of the first, second, and third processes described above are merely illustrative of the methods provided in the embodiments of this specification. In practical applications, the first, second, and third processing processes can be implemented using various methods such as threads and processes, and the steps executed can be split, merged, or called between program modules as needed.

[0082] The following is in conjunction with the appendix Figure 6 Taking the application of the data loading method for the machine learning model provided in this specification across multiple epochs as an example, the data loading method for the machine learning model will be further explained. In this embodiment, multiple epochs mean multiple complete training datasets. Determining the data access sequence for each iteration of the machine learning model may include: dividing a complete training dataset into multiple batches, wherein each batch contains several data access sequences, and each batch corresponds to one iteration of the machine learning model. Accordingly, the method further includes: when the machine learning model completes training on the complete dataset, for the next complete training dataset, returning to the step of dividing a complete training dataset into multiple batches, until the training of the machine learning model ends. In this embodiment, the training of a machine learning model includes multiple epochs of training, each epoch contains multiple iterations, and each epoch repeats the entire process from determining the data access sequence for each iteration of the machine learning model to the completion of training in the last iteration. Since the sample data preloading process and the sample data prefetching process overlap in time and space in each epoch, and the sample data prefetching process overlaps in time and space with the model training process, the performance of large-scale, high-concurrency model training is improved.

[0083] in, Figure 6 The present specification illustrates a process flowchart of a data loading method for a machine learning model according to an embodiment of this specification, which specifically includes the following steps.

[0084] Step 602: Process initialization.

[0085] To enable the three processes—preloading, reading sample data from the cache pool, and model training—to overlap and run in parallel in time and space, multiple parallel processes are first initialized. Specifically, in step 602, the data preloading process, data prefetching process, and model training process can be initialized by the data loading module set up on the computing node.

[0086] Alternatively, in step 602, an RDMA-based distributed cache resource can be requested to obtain the available distributed cache resource.

[0087] Step 604: Obtain the data access sequence for the current epoch of training.

[0088] In step 604, the complete training dataset for the current epoch is divided into multiple batches, with one batch corresponding to one iteration, and each iteration containing several data access sequences. Specifically, for example, referring to the sampler example mentioned above, the data loading module can obtain the data access sequences for the current epoch from the sampler and pass the data access sequences to the data preloading process.

[0089] Step 606: The data preloading process actively performs data preloading.

[0090] In step 606, the data preloading process sequentially preloads sample data from the backend storage into the RDMA-based distributed resource pool according to the received data access sequence. It is understood that since the sample data is no longer needed after model training, sample data that has already been trained in previous iterations can be dynamically deleted from the cache pool to accommodate sample data for subsequent iterations' data access sequences. This data replacement process efficiently utilizes the resources of the distributed resource pool.

[0091] Step 608: The data prefetching process actively performs data prefetching.

[0092] In step 608, the data prefetching process reads the sample data of the current iteration from the distributed resource pool of RDMA according to the order of the data access sequence in multiple iterations, and performs the corresponding data preprocessing operations. It can be understood that since step 608 and step 606 are performed independently by two separate processes, these two steps overlap and run in parallel in time and space.

[0093] Step 610: Model training process involves model computation.

[0094] In step 610, the model training process inputs the sample data of the current iteration, retrieved by the data prefetching process, into the machine learning model for training computation of the current iteration. Furthermore, during model computation, the data prefetching process simultaneously retrieves sample data for the next iteration from the RDMA distributed resource pool and performs corresponding data preprocessing, thus iterating until training is completed after M iterations. This parallel processing of multiple processes eliminates data waiting time, significantly improving the performance of large-scale, high-concurrency AI.

[0095] Once the current epoch has been completed, the next epoch is taken as the current epoch, and the process returns to step 604. This process repeats steps 604-610 for each epoch until the machine learning model has been trained for M iterations across N epochs, ultimately yielding the machine learning model used in artificial intelligence applications.

[0096] Corresponding to the above method embodiments, this specification also provides embodiments of a data loading device for machine learning models. Figure 7 A schematic diagram of a data loading device for a machine learning model according to one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0097] The sequence determination module 702 can be configured to determine the respective data access sequences for multiple iterations of a machine learning model.

[0098] The data preloading module 704 can be configured to preload sample data corresponding to the data access sequence of each of the multiple iterations into the cache pool in the first processing step, according to the order of the multiple iterations.

[0099] The data pre-reading module 706 can be configured to read the corresponding sample data from the cache pool and send it to the data loading module in the order of the multiple iterations during the second processing process, wherein the first processing process and the second processing process are executed in parallel.

[0100] The data loading module 708 can be configured to input the received sample data into the machine learning model in the order of the multiple iterations during the third processing, so that the machine learning model can be trained iteratively. The third processing is executed in parallel with the second processing.

[0101] Because the device determines the data access sequence for each iteration of the machine learning model, in the first processing step, sample data corresponding to the data access sequence of each iteration is preloaded into a cache pool according to the order of the iterations. In the second processing step, the corresponding sample data is read from the cache pool and sent to the data loading module according to the order of the iterations. The first and second processing steps are executed in parallel. In the third processing step, the data loading module iteratively inputs the received sample data into the machine learning model according to the order of the iterations, allowing the machine learning model to be trained iteratively. The third processing step is executed in parallel with the second processing step. Therefore, this... This approach leverages the predictability of data access sequences for sample data used in machine learning model training tasks. It implements a two-stage pipeline for proactive data loading. The first stage proactively preloads the sample data required for multiple iterations of the machine learning model into a cache pool in the order of iterations through a first processing step. The second stage, concurrent with the first stage, reads the pre-fetched sample data from the cache pool. Then, through a third processing step running in parallel with the second stage, the sample data is iteratively input into the machine learning model. This overlaps the data pre-fetching process with the model training process, minimizing the model's data waiting time, resolving the data I / O bottleneck, improving data loading speed, and meeting the performance requirements of ever-expanding machine learning models.

[0102] In one or more embodiments of this specification, the first processing process, the second processing process, and the third processing process are each an independent process.

[0103] In one or more embodiments of this specification, the data pre-reading module may also be configured to preprocess the received sample data according to a preset data preprocessing rule during the second processing.

[0104] In one or more embodiments of this specification, the apparatus can be configured on any one or more computing nodes of a distributed system. The cache pool is obtained by registering the local memory resources of the computing nodes in the distributed system as memory based on RDMA transfer.

[0105] In one or more embodiments of this specification, multiple computing nodes of the distributed system can be used to perform the training of the machine learning model in parallel.

[0106] In one or more embodiments of this specification, the sequence determination module 702 can be configured to divide a complete training dataset into multiple batches, wherein a batch contains several data access sequences, and a batch corresponds to one iteration of the machine learning model. Accordingly, the apparatus may further include a loop execution module, which can be configured to, when the machine learning model completes training on the complete dataset, trigger the sequence determination module to return to the step of dividing a complete training dataset into multiple batches for the next complete training dataset, until the training of the machine learning model ends.

[0107] The above is a schematic scheme of a data loading device for a machine learning model according to this embodiment. It should be noted that the technical solution of this data loading device for a machine learning model and the technical solution of the data loading method for a machine learning model described above belong to the same concept. For details not described in detail in the technical solution of the data loading device for a machine learning model, please refer to the description of the technical solution of the data loading method for a machine learning model described above.

[0108] Corresponding to the above method embodiments, this specification also provides embodiments of a distributed machine model training system. Figure 8 A schematic diagram of the structure of a distributed machine model training system provided in one embodiment of this specification is shown. Figure 8 As shown, the system includes: backend storage 802, cache pool 804, and multiple computing nodes 806 for parallel training of machine learning models. Figure 8 In the diagram, compute node 806A and compute node 806B are used to illustrate multiple compute nodes 806.

[0109] The backend storage 802 can be configured to store the training dataset of the machine learning model.

[0110] The computing node 806 can be configured to determine the data access sequence for each iteration of the machine learning model. In the first processing step, the sample data corresponding to the data access sequence of each iteration is preloaded into a cache pool according to the order of the iterations. In the second processing step, the corresponding sample data is read from the cache pool according to the order of the iterations. The first and second processing steps are executed in parallel. In the third processing step, the sample data is input into the machine learning model for iterative training according to the order of the iterations. The third processing step is executed in parallel with the second processing step.

[0111] It is understandable that different computing nodes are assigned different data access sequences. Therefore, each computing node processes sample data of different data access sequences based on a similar processing flow.

[0112] In practical applications, the functionality of the computing node 806 can be implemented in multiple modules according to the needs of the scenario. For example, the computing node 806 can be equipped with a data loading module and a data preloading module.

[0113] The data loading module is configured to determine the data access sequence for each iteration of the machine learning model and send the data access sequence to the data preloading module.

[0114] The data preloading module is configured to, in the first processing step, preload the sample data corresponding to the data access sequence of each of the multiple iterations into the cache pool 804 according to the order of the multiple iterations, and in the second processing step, read the corresponding sample data from the cache pool 804 and send it to the data loading module according to the order of the multiple iterations. The first processing step and the second processing step are executed in parallel.

[0115] The data loading module is further configured to input the received sample data into the machine learning model in the order of the multiple iterations during the third processing, so that the machine learning model is trained iteratively. The third processing is executed in parallel with the second processing.

[0116] The data preloading module can be set outside or inside the computing node depending on the application scenario, and this specification does not limit this.

[0117] The above is an illustrative scheme of a distributed machine learning model training system according to this embodiment. It should be noted that the technical solution of this distributed machine learning model training system and the technical solution of the aforementioned data loading method for machine learning models belong to the same concept. Details not described in detail in the technical solution of the distributed machine learning model training system can be found in the description of the technical solution of the aforementioned data loading method for machine learning models.

[0118] Figure 9 A structural block diagram of a computing device 900 according to one embodiment of this specification is shown. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0119] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0120] In one embodiment of this specification, the aforementioned components of the computing device 900 and Figure 9 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 9 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0121] The computing device 900 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 900 can also be a mobile or stationary server.

[0122] The processor 920 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the data loading method for the above-mentioned machine learning model.

[0123] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data loading method for the machine learning model described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data loading method for the machine learning model described above.

[0124] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the data loading method for the machine learning model described above.

[0125] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data loading method for the machine learning model described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data loading method for the machine learning model described above.

[0126] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the data loading method for the machine learning model described above.

[0127] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the data loading method for the machine learning model described above belong to the same concept. Details not described in detail in the computer program's technical solution can be found in the description of the technical solution of the data loading method for the machine learning model described above.

[0128] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0129] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0130] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0131] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0132] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for loading data onto a computing node in a distributed system, comprising: Pre-register local memory resources as RDMA-based memory to establish a cache pool; Determine the data access sequence for each iteration of the machine learning model, wherein the data access sequence refers to the serialized representation of the sample data input to the machine learning model during the iteration process; In the first processing step, the sample data corresponding to the data access sequence of each of the multiple iterations is preloaded into the cache pool according to the order of the multiple iterations. In the second processing step, the corresponding sample data is read from the cache pool and temporarily loaded into the local CPU memory or GPU memory of the computing node according to the order of the multiple iterations. The first processing step and the second processing step are executed in parallel. In the third processing step, the sample data is input into the machine learning model in the order of the multiple iterations for iterative training. The third processing step is executed in parallel with the second processing step.

2. A data loading method for a machine learning model, comprising: Determine the data access sequence for each iteration of the machine learning model, wherein the data access sequence refers to the serialized representation of the sample data input to the machine learning model during the iteration process; In the first processing step, the sample data corresponding to the data access sequence of each of the multiple iterations is preloaded into the cache pool according to the order of the multiple iterations. In the second processing step, the corresponding sample data is read from the cache pool and temporarily loaded into the local CPU memory or GPU memory of the computing node according to the order of the multiple iterations. The first processing step and the second processing step are executed in parallel. In the third processing step, the sample data is input into the machine learning model in the order of the multiple iterations for iterative training. The third processing step is executed in parallel with the second processing step.

3. The method according to claim 2, wherein the first processing step, the second processing step, and the third processing step are each an independent process.

4. The method according to claim 2, further comprising, before the sample data is input into the machine learning model: In the second processing step, the received sample data is preprocessed according to preset data preprocessing rules.

5. The method according to claim 2, wherein the method is applied to any one or more computing nodes in a distributed system; The cache pool is obtained by registering the local memory resources of the computing nodes in the distributed system as memory based on RDMA transmission.

6. The method according to claim 5, wherein multiple computing nodes of the distributed system are used to perform training of the machine learning model in parallel.

7. The method according to claim 2, wherein determining the respective data access sequences of multiple iterations of the machine learning model comprises: A complete training dataset is divided into multiple batches, where each batch contains several data access sequences and each batch corresponds to one iteration of the machine learning model. The method further includes: When the machine learning model completes the training of the complete training dataset, for the next complete training dataset, return to the step of dividing a complete training dataset into multiple batches, until the training of the machine learning model ends.

8. A data loading device for a machine learning model, comprising: The sequence determination module is configured to determine the data access sequence for each of the multiple iterations of the machine learning model, wherein the data access sequence refers to the serialized representation of the sample data input to the machine learning model during the iteration process; The data preloading module is configured to preload the sample data corresponding to the data access sequence of each of the multiple iterations into the cache pool in the first processing step, according to the order of the multiple iterations. The data pre-fetching module is configured to, in the second processing process, read the corresponding sample data from the cache pool according to the order of the multiple iterations, temporarily load it into the local CPU memory or GPU memory of the computing node, and send it to the data loading module, wherein the first processing process and the second processing process are executed in parallel. The data loading module is configured to input the received sample data into the machine learning model in the order of the multiple iterations during the third processing step, so that the machine learning model can be trained iteratively. The third processing step is executed in parallel with the second processing step.

9. A distributed machine model training system, comprising: Backend storage, cache pool, and multiple compute nodes for parallel training of machine learning models; The backend storage is configured to store the training dataset of the machine learning model; The computing node is configured to determine the data access sequence for each iteration of the machine learning model, wherein the data access sequence refers to the serialized representation of the sample data input to the machine learning model during the iteration. In the first processing step, the sample data corresponding to the data access sequence of each iteration is preloaded into a cache pool according to the order of the multiple iterations. In the second processing step, the corresponding sample data is read from the cache pool and temporarily loaded into the local CPU memory or GPU memory of the computing node according to the order of the multiple iterations. The first processing step and the second processing step are executed in parallel. In the third processing step, the sample data is input into the machine learning model for iterative training according to the order of the multiple iterations. The third processing step and the second processing step are executed in parallel.

10. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the data loading method of the distributed system computing node according to claim 1, or implement the steps of the data loading method of the machine learning model according to any one of claims 2 to 7.

11. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the data loading method for a distributed system computing node as described in claim 1, or implement the steps of the data loading method for a machine learning model as described in any one of claims 2 to 7.

Images