Method and apparatus for training machine learning model using parameter server

Abstract

Provided are a training method and device for a machine learning model using a parameter server. The training method comprises: receiving a parameter pre-reading request sent by a computing node when the computing node is training a plurality of first parameters corresponding to a first batch of training samples, the parameter pre-reading request being used to request pre-reading of a plurality of second parameters corresponding to a second batch of training samples, reading the plurality of second parameters from a parameter storage area of the parameter server to a pre-reading cache area of the parameter server, the pre-reading cache area storing a plurality of parameters corresponding to the training samples of the first batch to the second batch; determining, from the plurality of parameters stored in the pre-reading cache area, whether the plurality of second parameters have been pre-read before the second batch within the range of the first batch to the second batch; and in the case where there are second parameters in the plurality of second parameters that have not been pre-read before the second batch within the range of the first batch to the second batch, sending the second parameters that have not been pre-read to the computing node.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a method and apparatus for training a machine learning model using a parameter server, as well as the parameter server itself. Background Technology

[0002] With the rapid growth of data volume, machine learning is widely used in various fields to extract value from data. Machine learning models are trained offline using massive amounts of training samples, and offline training modes are divided into synchronous training and asynchronous training. Models trained using synchronous training have high accuracy, and some applications with high accuracy requirements, such as ad click applications, require synchronous training. However, synchronous training requires a lot of waiting time, resulting in low training efficiency. Asynchronous training is faster, but its accuracy is lower compared to synchronous training. Summary of the Invention

[0003] The exemplary embodiments disclosed herein may at least solve the above-described problems, or may not solve the above-described problems.

[0004] According to a first aspect of this disclosure, a method for training a machine learning model using a parameter server is provided, comprising: receiving a parameter prefetch request sent by the computing node while a computing node is training multiple first parameters corresponding to a first batch of training samples, wherein the parameter prefetch request is used to request prefetching multiple second parameters corresponding to a second batch of training samples, the second batch being after the first batch in training order and separated from the first batch by a predetermined number of batches, and the multiple parameters corresponding to each batch of training samples representing multiple parameters of the machine learning model trained using that batch of training samples; reading the multiple second parameters from the parameter storage area of ​​the parameter server into a prefetch cache area of ​​the parameter server, wherein the prefetch cache area stores multiple parameters corresponding to training samples from the first batch to the second batch; determining from the multiple parameters stored in the prefetch cache whether the multiple second parameters have been prefetched before the second batch within the range from the first batch to the second batch; and if there are second parameters among the multiple second parameters that have not been prefetched before the second batch within the range from the first batch to the second batch, sending the unprefetched second parameters to the computing node.

[0005] Optionally, the training method may further include: if among the plurality of second parameters there is a second parameter that was pre-read before the second batch within the range from the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM, then the pre-read second parameter is read and cached in the DRAM of the parameter server.

[0006] Optionally, the training method may further include: determining whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training; and if it is determined that there is an ultra-high frequency parameter among the pre-read second parameters, sending the ultra-high frequency parameter to the cache of each computing node using the ultra-high frequency parameter.

[0007] Optionally, determining whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training may include: determining whether the pre-read second parameter is an ultra-high frequency parameter based on the distance between the pre-read second parameter in the second batch and the most recently pre-read batch before the second batch, wherein when the distance of a certain second parameter among the pre-read second parameters is less than a predetermined threshold, the second parameter is determined to be an ultra-high frequency parameter.

[0008] Optionally, the training method may further include: when the computing node completes the training of the plurality of first parameters, receiving the updated parameter values ​​of the plurality of first parameters from the computing node and caching the updated parameter values ​​of the plurality of first parameters in the pre-read cache area, and releasing the space storing the plurality of first parameters from the pre-read cache area.

[0009] Optionally, releasing the space storing the plurality of first parameters from the prefetch cache may include: determining from the plurality of parameters stored in the prefetch cache whether the plurality of first parameters have been prefetched after the first batch within the range of the first batch to the second batch; if there is a first parameter among the plurality of first parameters that has not been prefetched after the first batch within the range of the first batch to the second batch, flushing the updated parameter value of the unprefetched first parameter to the parameter storage area and clearing it from the prefetch cache; if there is a first parameter among the plurality of first parameters that has been prefetched after the first batch within the range of the first batch to the second batch, flushing the updated parameter value of the prefetched first parameter to the parameter storage area, updating the first parameter of the most recently prefetched batch after the first batch, and sending it to the computing node that requested to prefetch the first parameter of the most recently prefetched batch after the first batch.

[0010] Optionally, the training method may further include: determining whether the pre-read first parameter is an ultra-high frequency parameter based on the frequency of its use in training; and if it is determined that there is an ultra-high frequency parameter among the pre-read first parameters, skipping the step of sending the ultra-high frequency parameter to the computation node of the first parameter in the most recent pre-read batch after requesting pre-reading the first batch.

[0011] Optionally, the pre-read cache may include a first area and a second area, wherein the first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list stores the parameter names of hot parameters, and the cold parameter list stores the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use during training. The training method may further include: when a computing node is training multiple first parameters corresponding to the first batch of training samples, if the cold parameter list does not contain the parameter name of a second parameter that has not been pre-read before the second batch within the range from the first batch to the second batch, then... The parameter names of the second parameters that have not been pre-read are stored in the cold parameter list; and if there are no parameter names of the second parameters that have been pre-read before the second batch within the range from the first batch to the second batch in the hot parameter list, the parameter names of the pre-read second parameters are stored in the hot parameter list; wherein, the training method may further include: when the computing node completes the training of the plurality of first parameters, if there are parameter names of the first parameters that have not been pre-read after the first batch within the range from the first batch to the second batch in the cold parameter list, the parameter names of the unpre-read first parameters are removed from the cold parameter list.

[0012] Optionally, the parameter storage area can be implemented by persistent memory, and the prefetch cache area can be implemented by DRAM.

[0013] According to a second aspect of this disclosure, a training apparatus for a machine learning model executed using a parameter server is provided, comprising: a request receiving unit configured to: receive a parameter prefetch request sent by a computing node while a computing node is training multiple first parameters corresponding to a first batch of training samples, wherein the parameter prefetch request is used to request prefetching multiple second parameters corresponding to a second batch of training samples, the second batch being after the first batch in training order and separated from the first batch by a predetermined number of batches, and the multiple parameters corresponding to each batch of training samples representing multiple parameters of the machine learning model trained using that batch of training samples; and a parameter reading unit configured to: read the parameters... Multiple second parameters are read from the parameter storage area of ​​the parameter server into the pre-read cache area of ​​the parameter server, wherein the pre-read cache area stores multiple parameters corresponding to the training samples from the first batch to the second batch; the pre-read determination unit is configured to determine from the multiple parameters stored in the pre-read cache whether the multiple second parameters have been pre-read before the second batch within the range from the first batch to the second batch; the parameter sending unit is configured to send the unpre-read second parameter to the computing node if there is a second parameter among the multiple second parameters that has not been pre-read before the second batch within the range from the first batch to the second batch.

[0014] Optionally, the training method may further include: a parameter caching unit configured to: read and cache the pre-read second parameter in the DRAM of the parameter server if, among the plurality of second parameters, there is a second parameter that has been pre-read before the second batch within the range of the first batch to the second batch and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM.

[0015] Optionally, the pre-read determination unit may also be configured to: determine whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training; the parameter sending unit may also be configured to: send the ultra-high frequency parameter to the cache of each computing node using the ultra-high frequency parameter if it is determined that there is an ultra-high frequency parameter among the pre-read second parameters.

[0016] Optionally, the pre-read determination unit can be configured to: determine whether the pre-read second parameter is an UHF parameter based on the distance between the second batch and the most recently pre-read batch before the second batch, wherein when the distance of a certain second parameter among the pre-read second parameters is less than a predetermined threshold, the second parameter is determined to be an UHF parameter.

[0017] Optionally, the training device may further include: a pre-read management unit, configured to: when the computing node completes training of the plurality of first parameters, receive updated parameter values ​​of the plurality of first parameters from the computing node, cache the updated parameter values ​​of the plurality of first parameters in the pre-read cache, and release the space storing the plurality of first parameters from the pre-read cache.

[0018] Optionally, the prefetch management unit can be configured to: determine from the plurality of parameters stored in the prefetch cache whether the plurality of first parameters have been prefetched after the first batch within the range of the first batch to the second batch; if there is a first parameter among the plurality of first parameters that has not been prefetched after the first batch within the range of the first batch to the second batch, flush the updated parameter value of the unprefetched first parameter to the parameter storage area and clear it from the prefetch cache; if there is a first parameter among the plurality of first parameters that has been prefetched after the first batch within the range of the first batch to the second batch, flush the updated parameter value of the prefetched first parameter to the parameter storage area, update the first parameter of the most recently prefetched batch after the first batch, and send it to the computing node that requested to prefetch the first parameter of the most recently prefetched batch after the first batch.

[0019] Optionally, the pre-read management unit may also be configured to: determine whether the pre-read first parameter is an ultra-high frequency parameter based on the frequency of its use in training; and if it is determined that there is an ultra-high frequency parameter among the pre-read first parameters, skip the step of sending the ultra-high frequency parameter to the computation node of the first parameter in the most recent pre-read batch after requesting pre-reading the first batch.

[0020] Optionally, the pre-read cache may include a first area and a second area, wherein the first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list stores the parameter names of hot parameters, and the cold parameter list stores the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use in training. The training device may further include a list management unit. When a computing node is training multiple first parameters corresponding to the first batch of training samples, the list management unit is configured to ensure that the cold parameter list does not contain any second parameters that have not been pre-read before the second batch within the range from the first batch to the second batch. In the case of multiple first parameters, the parameter names of the unpreread second parameters are stored in the cold parameter list; and if there is no parameter name of the second parameter that has been preread before the second batch within the range from the first batch to the second batch in the hot parameter list, the parameter name of the preread second parameter is stored in the hot parameter list; when the computing node completes the training of the multiple first parameters, the list management unit is configured to: remove the parameter name of the unpreread first parameter from the cold parameter list if there is a parameter name of the first parameter that has not been preread after the first batch within the range from the first batch to the second batch in the cold parameter list.

[0021] Optionally, the parameter storage area can be implemented by persistent memory, and the prefetch cache area can be implemented by DRAM.

[0022] According to a third aspect of this disclosure, a parameter server is provided, comprising: a parameter storage area configured to store parameters of a machine learning model; and a pre-read cache area configured to store parameters of the machine learning model requested for pre-read by a computing node; wherein, while the computing node is training multiple first parameters corresponding to a first batch of training samples, a parameter pre-read request is received from the computing node, wherein the parameter pre-read request is used to request pre-reading multiple second parameters corresponding to a second batch of training samples, the second batch being after the first batch in training order and separated from the first batch by a predetermined number of batches, and the multiple parameters corresponding to each batch of training samples represent the parameters obtained using... The machine learning model trained by the batch of training samples has multiple parameters; the multiple second parameters are read from the parameter storage area into the pre-read cache area, wherein the pre-read cache area stores multiple parameters corresponding to the first batch to the second batch of training samples; from the multiple parameters stored in the pre-read cache area, it is determined whether the multiple second parameters have been pre-read before the second batch within the range of the first batch to the second batch; if there is a second parameter among the multiple second parameters that has not been pre-read before the second batch within the range of the first batch to the second batch, the unpre-read second parameter is sent to the computing node.

[0023] Optionally, if among the plurality of second parameters there is a second parameter that was pre-read before the second batch within the range of the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM, the pre-read second parameter can be read and cached in the DRAM of the parameter server.

[0024] Optionally, based on the frequency with which the pre-read second parameter is used in training, it is determined whether the pre-read second parameter is an ultra-high frequency parameter; if it is determined that there is an ultra-high frequency parameter among the pre-read second parameters, the ultra-high frequency parameter is sent to the cache of each computing node using the ultra-high frequency parameter.

[0025] Optionally, determining whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training may include: determining whether the pre-read second parameter is an ultra-high frequency parameter based on the distance between the pre-read second parameter in the second batch and the most recently pre-read batch before the second batch, wherein when the distance of a certain second parameter among the pre-read second parameters is less than a predetermined threshold, the second parameter is determined to be an ultra-high frequency parameter.

[0026] Optionally, when the computing node completes the training of the plurality of first parameters, the updated parameter values ​​of the plurality of first parameters are received from the computing node and cached in the pre-read cache area, and the space for storing the plurality of first parameters is released from the pre-read cache area.

[0027] Optionally, releasing the space storing the plurality of first parameters from the prefetch cache may include: determining from the plurality of parameters stored in the prefetch cache whether the plurality of first parameters have been prefetched after the first batch within the range of the first batch to the second batch; if there is a first parameter among the plurality of first parameters that has not been prefetched after the first batch within the range of the first batch to the second batch, flushing the updated parameter value of the unprefetched first parameter to the parameter storage area and clearing it from the prefetch cache; if there is a first parameter among the plurality of first parameters that has been prefetched after the first batch within the range of the first batch to the second batch, flushing the updated parameter value of the prefetched first parameter to the parameter storage area, updating the first parameter of the most recently prefetched batch after the first batch, and sending it to the computing node that requested to prefetch the first parameter of the most recently prefetched batch after the first batch.

[0028] Optionally, based on the frequency with which the pre-read first parameter is used in training, it is determined whether the pre-read first parameter is an ultra-high frequency parameter; if it is determined that there is an ultra-high frequency parameter among the pre-read first parameters, the step of sending the ultra-high frequency parameter to the computation node of the first parameter in the most recent pre-read batch after requesting the pre-read of the first batch is skipped.

[0029] Optionally, the pre-read cache may include a first area and a second area, wherein the first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list stores the parameter names of hot parameters, and the cold parameter list stores the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use during training. Specifically, when a computing node is training multiple first parameters corresponding to the first batch of training samples, if the cold parameter list does not contain the parameter name of a second parameter that has not been pre-read before the second batch within the range from the first batch to the second batch, the pre-read cache will be... The parameter names of second parameters that have not been pre-read are stored in the cold parameter list; and if there is no parameter name of a second parameter that has been pre-read before the second batch within the range from the first batch to the second batch in the hot parameter list, the parameter name of the pre-read second parameter is stored in the hot parameter list; wherein, when the computing node completes the training of the plurality of first parameters, if there is a parameter name of a first parameter that has not been pre-read after the first batch within the range from the first batch to the second batch in the cold parameter list, the parameter name of the unpre-read first parameter is removed from the cold parameter list.

[0030] Optionally, the parameter storage area can be implemented by persistent memory, and the prefetch cache area can be implemented by DRAM.

[0031] According to a fourth aspect of this disclosure, a computer-readable storage medium for storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device causes the at least one computing device to perform a training method for a machine learning model executed using a parameter server according to this disclosure.

[0032] According to a fifth aspect of this disclosure, a system is provided that includes at least one computing device and at least one storage device for storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform a training method for a machine learning model executed using a parameter server according to this disclosure.

[0033] According to the training method and apparatus for a machine learning model using a parameter server disclosed herein, and the parameter server itself, by querying whether the parameters of the currently pre-read batch have been pre-read between the currently pre-read batch and the batch being trained, the parameter values ​​of parameters that have not been pre-read are sent back in advance to the computing node that requested the pre-read, thereby saving the overhead of subsequent PULL and PUSH operations and achieving the goal of accelerating training. Furthermore, this disclosure also designs and implements a hotspot awareness technique, which distinguishes and caches hot parameters by whether a parameter is repeatedly accessed by multiple batches during the pre-read process, caching the hot parameters in the DRAM of the parameter server, thereby accelerating PULL and PUSH operations and achieving the goal of accelerating training. In addition, this disclosure also sends the hot parameters back in advance to the computing node that most recently requested the pre-read after the training of the batch being trained is completed, thereby saving the overhead of subsequent PULL and PUSH operations and achieving the goal of accelerating training. Attached Figure Description

[0034] These and / or other aspects and advantages of the invention will become clear and more readily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, wherein:

[0035] Figure 1 This is a schematic diagram illustrating the sparse characteristics of ultra-high dimensional space.

[0036] Figure 2 This is a schematic diagram illustrating the distributed training process.

[0037] Figure 3 This is a schematic diagram illustrating the high-frequency and low-frequency features of ultra-high-dimensional sparse features.

[0038] Figure 4 This is a schematic diagram showing the distribution of characteristic parameters and access frequency.

[0039] Figure 5 This is a schematic diagram illustrating the synchronous training mode.

[0040] Figure 6 This is a schematic diagram illustrating the asynchronous training mode.

[0041] Figure 7 This is a schematic diagram illustrating the general approach of training a machine learning model using a parameter server according to an exemplary embodiment of the present disclosure.

[0042] Figure 8 This is a flowchart illustrating a method for training a machine learning model using a parameter server according to an exemplary embodiment of the present disclosure.

[0043] Figure 9 This is an exemplary prefetch flowchart illustrating an exemplary embodiment of the present disclosure.

[0044] Figure 10 This is an exemplary training completion flowchart illustrating an exemplary embodiment of the present disclosure.

[0045] Figure 11 This is a block diagram illustrating a training apparatus for a machine learning model executed using a parameter server according to an exemplary embodiment of the present disclosure.

[0046] Figure 12 This is a block diagram illustrating a parameter server according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0047] The following description, provided with reference to the accompanying drawings, is intended to aid in a full understanding of embodiments of the invention as defined by the claims and their equivalents. Various specific details are included to aid understanding, but these details are to be considered exemplary only. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Furthermore, for clarity and brevity, descriptions of well-known functions and structures are omitted.

[0048] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. As another example, "performing at least one of step one and step two" indicates the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0049] Machine learning models are trained using massive amounts of training samples before being deployed. For example, ultra-high-dimensional sparse models are trained with massive amounts of training samples to train high-dimensional parameters. Ultra-high-dimensional sparse models are a type of machine learning model that differs from traditional machine learning models (such as Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs), and Recurrent Neural Networks (RNNs)). The input to ultra-high-dimensional sparse models is ultra-high-dimensional sparse features. Figure 1 This is a schematic diagram illustrating the sparsity characteristics of ultra-high dimensional dimensions. (Refer to...) Figure 1Ultra-high-dimensional sparse features have two main characteristics. First, the range of values ​​for the input features is enormous. Ultra-high-dimensional sparse features are divided into multiple fields, each containing multiple possible values. The total number of these different types of values ​​is called the dimension of the field. For example, Field 1 is date (Day), with values ​​ranging from Monday to Sunday, so Field 1 has a dimension of 7; Field 2 is gender, with values ​​ranging from male to female, so Field 2 has a dimension of 2; Field 3 is items. Assuming there are 2 million items (e.g., Carlsberg beer, Tiger beer, etc.), Field 3 has a dimension of 2 million. The total dimension of ultra-high-dimensional sparse features can often reach hundreds of millions or even billions of possibilities. Second, each training sample contains only a limited number of features. Each field in the training data contains only one value. For example, a training sample may contain only three feature values: Tuesday, Male, and Tiger beer.

[0050] Due to the enormous dimensionality of ultra-high-dimensional sparse features, the training process of ultra-high-dimensional sparse models uses all feature values ​​(e.g., Monday to Sunday, Male, Female, Tiger beer, etc.) as keys, and the corresponding feature embeddings as values, stored in a parameter server cluster. A parameter server is a programming framework for writing distributed parallel programs, focusing on supporting distributed storage and collaboration of large-scale parameters. Offline parameter servers are primarily used during the training process, storing model parameters and providing high-concurrency parameter query and update operations.

[0051] Ultra-high dimensional sparse models often require multiple computing nodes (e.g., CPU nodes, GPU nodes, etc.) for training. Figure 2 This is a schematic diagram illustrating the distributed training process. (For example...) Figure 2As shown, training of ultra-high-dimensional sparse models generally involves four steps. Step (1) Prefetching and distributing training data. The amount of training data is large, and it is usually stored on a slow disk. Before each training session, a batch of data is prefetched and distributed to multiple computing nodes for training. For example, a batch contains 6 training data (S1 to S6), and two computing nodes (GPU A and GPU B) are used for training. Since each training data includes features from multiple fields, the usual practice is to distribute the 6 training data in a batch equally to the two computing nodes (GPU A and GPU B) in the order of prefetching. For example, training data S1 to S3 are distributed to computing node GPU A for training, and training data S4 to S6 are distributed to computing node GPU B for training. Step (2) The two computing nodes query the embedding values ​​of the training data features from the PS Embedding table (encoding table) stored in the parameter server. Here, the encoding table is used to map the features contained in the training data to features suitable for training machine learning models (i.e., embedding values). Step (3): The two computing nodes train the ultra-high-dimensional sparse model based on the feature embedding values ​​to obtain the parameter gradients of the ultra-high-dimensional sparse model. Step (4): The two computing nodes push the new gradients of the parameters of the ultra-high-dimensional sparse model obtained after training back to the parameter server.

[0052] Different fields have different ranges of feature values, and the frequency of these features appearing in the training samples will also differ. Figure 3 This is a schematic diagram illustrating the high-frequency and low-frequency features of ultra-high-dimensional sparse features. (Refer to...) Figure 3 Since Field 1 and Field 2 have relatively small value ranges, each feature in Field 1 and Field 2 has a relatively high probability of appearing in each training sample, and can be identified as high-frequency features. Field 3, on the other hand, contains millions of item types, but each training sample only contains one value from Field 3 (e.g., Tiger beer). Therefore, each feature in Field 3 has a relatively low probability of appearing in each training sample, and can be identified as low-frequency features. Figure 4 This is a schematic diagram showing the distribution of characteristic parameters and access frequencies. For example... Figure 4 As shown, the ad click load was statistically analyzed. Only a small portion of the parameters were accessed frequently, while 79% of the parameters were accessed only once during the entire training process.

[0053] The following reference Figure 5 and Figure 6Describe the working principles of synchronous and asynchronous training modes.

[0054] Figure 5 This is a schematic diagram illustrating the synchronous training mode. (Refer to...) Figure 5 A batch containing multiple data points is often divided into multiple sub-batches and executed by multiple computing nodes (workers). Each sub-batch may contain samples with the same parameter. For example, a batch may be divided into two sub-batches; taking batch 0 as an example, both computing nodes 1 and 2 contain parameter A. Synchronous training requires that the gradients of all computing nodes within the same batch with respect to A be updated to A before generating A1. Batch 1 must then continue training based on A1 to ensure accuracy. The advantage of synchronous training is high accuracy, but because traditional parameter servers only provide simple PULL and PUSH functions, to achieve the requirements of synchronous training, the subsequent batch, i.e., ... Figure 5 In the training process, batch 1 can only begin acquiring the parameters needed for training a new batch after all the parameter gradients of all computation nodes in batch 0 have been transmitted to the parameter server and all parameters involved in batch 0 have been updated. This creates an invisible barrier between batches, resulting in unnecessary waiting time during training.

[0055] Figure 6 This is a schematic diagram illustrating the asynchronous training mode. (Refer to...) Figure 6 Unlike synchronous mode, asynchronous mode allows each computing node to immediately start pushing the parameters needed for the next batch of training after completing the parameter training for its assigned sub-batch and pushing it back to the parameter server. Compared to synchronous mode, asynchronous mode reduces the time spent waiting for parameter updates within a batch, significantly improving the parallelism of parameter querying and updating, thereby increasing training efficiency. However, asynchronous mode may lose some training gradients, such as... Figure 6 As shown, Batch 1 should have continued training parameter A based on A1. However, because asynchronous mode does not wait for parameter A to be updated from A0 to A1, the updates to A from Batch 0 are lost in asynchronous mode. Even worse, for frequently accessed parameters, this accumulation of lost gradients can severely impact the final accuracy.

[0056] Currently, applications in the industry that require high model accuracy, such as ad click applications, all require synchronous training. In synchronous training, parameters are currently pre-read or updated at the batch level. All computing nodes pull and push parameters at the beginning and end of each batch, resulting in I / O spikes and causing system bottlenecks.

[0057] To address the aforementioned technical issues, this disclosure proposes a fine-grained pre-reading mechanism that is aware of access hotspots. This mechanism solves the performance problems of synchronous training on offline parameter servers, maintaining the same accuracy as synchronous training while achieving a training speed close to that of asynchronous training, thus significantly improving training speed. Specifically, since not all parameters used in a batch need to be obtained until the current batch training is completed, such as... Figure 5 The E and F parameters in Batch 1 were not updated in Batch 0. Therefore, these parameters can be pre-read to the local compute node during Batch 0 training without affecting correctness. This disclosure checks whether the parameters of the currently pre-read batch have been pre-read between the current pre-read batch and the batch currently being trained. It then sends the parameter values ​​of unpre-read parameters back to the compute node that requested the pre-read in advance, saving the overhead of subsequent pull and push operations and accelerating training. Furthermore, this disclosure also implements a hotspot awareness technique. By identifying whether a parameter is repeatedly accessed by multiple batches during the pre-read process, it distinguishes and caches hot parameters, storing them in the parameter server's DRAM to accelerate pull and push operations, thereby accelerating training. Additionally, after the training of the batch currently being trained is completed, this disclosure also sends the hot parameters back to the compute node that most recently requested the pre-read in advance, saving the overhead of subsequent pull and push operations and accelerating training.

[0058] Below, we will refer to Figures 7 to 12 The present disclosure provides a detailed description of a method and apparatus for training a machine learning model using a parameter server, as well as a parameter server, according to exemplary embodiments thereof.

[0059] Figure 7 This is a schematic diagram illustrating the general approach of training a machine learning model using a parameter server according to an exemplary embodiment of the present disclosure.

[0060] Reference Figure 7The parameter server may include a parameter storage area and a read-ahead cache. Parameters stored in the parameter storage area and the read-ahead cache can be stored in key-value pairs of parameter names (e.g., A, B, C…) and parameter values ​​(e.g., Data). The parameter storage area may be implemented using at least one of fast dynamic random access memory (DRAM), persistent memory (PMEM), solid-state drive (SSD), hard disk drive (HDD), etc., and the read-ahead cache may be implemented using at least one of DRAM, PMEM, etc. According to an exemplary embodiment of this disclosure, the parameter storage area is used to store model parameters, and the read-ahead cache is used to PULL and PUSH parameters with each computing node. Therefore, the parameter storage area may be implemented using non-volatile memory (NVM) capable of permanently or semi-permanently storing data, and the read-ahead cache may be implemented using volatile memory with high access speed. NVM generally refers to memory that can store data even when power is off. Flash memory is a widely known type of NVM, also known as first-generation NVM. Newer generations of NVM (such as, but not limited to, STT-RAM, PCM, ReRAM, and 3D xPoint) are also called persistent memory. The essential difference between PMEM (Power Memory) and the first-generation NVM is that: (1) its performance is greatly improved, and it is closer to DRAM used for computer main memory; (2) it is free from the physical limitation that it must be addressed in multiples of a fixed number of bytes (such as Flash, which is usually 4KB (4*1024 bytes)) when accessing, and can be addressed in single-byte (Byte-addressable). PMEM has a speed similar to DRAM memory, and the data stored on PMEM still exists after the system is powered off. It is applied to the field of model parameter backup technology, which can save model parameters and improve access speed, and has great advantages. In addition, since DRAM has a fast access speed, in order to improve training efficiency, each computing node usually pulls and pushes parameters with DRAM in the parameter server. Therefore, according to the exemplary embodiment of this disclosure, the parameter storage area can be implemented by PMEM, and the pre-read cache area is implemented by DRAM.

[0061] For ease of understanding, Figure 7 The example provided assumes the parameter server is currently training the parameters for batch N-3. Simultaneously, compute nodes can pre-read the parameters needed for the next few batches based on the received training data. Compute nodes pre-read in ascending order of batch size; in this example, the order is: first Batch N-3, then Batch N-2, then Batch N-1, and finally Batch N. The specific parameters pre-read for each batch are as follows... Figure 7As shown in the upper right corner, the parameters of the training batch N-3 include E, W, Z, and A; the parameters of the pre-read batch N-2 include Q, F, G, and R; the parameters of the pre-read batch N-1 include A, C, T, and U; and the parameters of the pre-read batch N include B, I, V, and A. The parameter server's pre-read cache stores the parameters from batch N-3 to batch N. Assuming that the compute node is currently pre-reading batch N, the parameter server first checks the pre-read cache for the parameters B, I, V, and A to be pre-read in batch N to see if these parameters have been pre-read between the currently pre-read batch (batch N) and the currently training batch (batch N-3) (i.e., between batch N-3 and batch N-1). Since parameters B, I, and V have not been accessed in the previous batches, this means that the values ​​of parameters B, I, and V stored on the parameter server are the latest and will not be modified before Batch N. Therefore, the values ​​of B, I, and V can be sent back to the computing node that requested the pre-reading in advance, instead of waiting until the computing node starts training Batch N, thereby saving I / O time and improving training efficiency.

[0062] Furthermore, if Batch N has parameters, such as parameter A, that have been pre-read in previous batches, this indicates that parameter A will be modified before Batch N. Therefore, the value of parameter A currently stored on the parameter server is not accurate and up-to-date in Batch N. In this case, parameter A can be considered a hot parameter. If the value of parameter A is stored on a slower storage device than DRAM (such as PMEM, SSD, HDD, etc.), the value of parameter A can be read and cached in DRAM to improve the efficiency of subsequent pull and push operations.

[0063] Furthermore, whether parameter A is an ultra-high frequency parameter can be determined based on its frequency of use during training. For example, the distance between parameter A in the currently pre-read batch (e.g., Batch N) and the batch that most recently pre-read parameter A (e.g., Batch N-1) before the current batch can be used to determine if parameter A is an ultra-high frequency parameter. If this distance is less than a predetermined threshold, parameter A is determined to be an ultra-high frequency parameter. If parameter A is determined to be an ultra-high frequency parameter, the parameter server can synchronously store parameter A in the cache (e.g., GPU cache) of each computing node that will use parameter A, thereby accelerating access to ultra-high frequency parameters more quickly.

[0064] In addition, such as Figure 7As shown, the parameter server's prefetch cache can include a batch transaction zone for recording parameters requested for each batch, and a feature index zone. The feature index zone can store two lists: a hot parameter list and a cold parameter list. In the example above, the parameter name of parameter A can be stored in the hot parameter list, while parameters B, I, and V can be stored in the cold parameter list. This better distinguishes and maintains hot parameters (also known as high-frequency parameters) and cold parameters (also known as low-frequency parameters), thus accelerating training.

[0065] Furthermore, when training Batch N-3 is complete, the prefetch cache can release the storage space for the parameters of Batch N-3. Specifically, the fine-grained prefetch mechanism checks whether there are hot parameters in Batch N-3, for example, by querying the hot parameter list in the feature index area. If a parameter is not a hot parameter, such as E, W, or Z, it means that no computation node has requested prefetching for this parameter between Batch N-3 and the latest prefetch batch (i.e., Batch N). After the value of this parameter is updated, it can be directly removed from the batch interaction area and stored in the parameter storage area of ​​the parameter server. In addition, if the parameter names of parameters E, W, and Z are in the cold parameter list, the parameter names of parameters E, W, and Z can also be removed from the cold parameter list. If there are hot parameters in Batch N-3, such as parameter A, the most recently requested batch (e.g., Batch N-1) and compute node after Batch N-3 can be found. The value of parameter A updated by Batch N-3 is then updated in the batch interaction area to store parameter A in Batch N-1, and the updated value of parameter A is returned to the compute node that requested the pre-reading. This avoids waiting until the compute node that requested the pre-reading starts training Batch N-1 before sending the value back to the requesting compute node, thus saving I / O time and improving training efficiency. Furthermore, it can detect whether parameter A is a very high-frequency parameter and whether it is cached in the cache of the compute node using parameter A (e.g., GPU cache). If so, it means that parameter A will be synchronized between GPUs without needing to pull and push through the parameter server, thus accelerating access to very high-frequency parameters and improving training speed.

[0066] Figure 8 This is a flowchart illustrating a method for training a machine learning model using a parameter server according to an exemplary embodiment of the present disclosure.

[0067] Reference Figure 8In step 801, while the computing node is training multiple first parameters corresponding to the first batch of training samples, it can receive a parameter prefetch request sent by the computing node. This parameter prefetch request is used to request the prefetching of multiple second parameters corresponding to the second batch of training samples. Here, the second batch is trained after the first batch and is separated from the first batch by a predetermined number of batches. The multiple parameters corresponding to each batch of training samples represent multiple parameters of the machine learning model trained using that batch of training samples. For example, in... Figure 7 In the example, the multiple first parameters corresponding to the first batch of training samples can be the parameters E, W, Z, and A of Batch N-3, and the multiple second parameters corresponding to the second batch of training samples can be the parameters B, I, V, and A of Batch N. Furthermore, the distance between the first and second batches (i.e., a predetermined number of batches) can be preset, for example, but not limited to, 50 batches. This disclosure does not limit the number of batches requested for parameter prefetching, as long as it reaches the second batch. For example, assuming the distance between the first and second batches is preset to 50 batches, when training is being performed on the first batch of training samples, the parameter prefetching request can request to prefetch the parameters of the second to fifty-first batches of training samples; subsequently, when training is being performed on the second batch of training samples, the parameter prefetching request can request to prefetch the parameters of the fifty-second batch of training samples. For example, assuming the distance between the first batch and the second batch is preset to 50 batches, when training is being performed on the Nth batch of training samples and the parameters of the N+48th batch of training samples have been read, the parameter prefetch request can request to prefetch the parameters of the N+49th and N+50th batches of training samples.

[0068] In step 802, multiple second parameters can be read from the parameter storage area of ​​the parameter server into the pre-read cache area of ​​the parameter server. The pre-read cache area stores multiple parameters corresponding to the first and second batches of training samples. Here, the parameters stored in the parameter storage area and the pre-read cache area can be stored in the form of key-value pairs of parameter names (also called feature names) and parameter values ​​(also called embedding values).

[0069] According to exemplary embodiments of the present disclosure, the parameter storage area may be implemented by at least one of DRAM, PMEM, SSD, HDD, etc., and the prefetch cache area may be implemented by at least one of DRAM, PMEM, etc.

[0070] According to exemplary embodiments of this disclosure, the parameter storage area may be implemented by non-volatile memory (NVM) capable of permanently or semi-permanently storing data, and the read-ahead cache area may be implemented by volatile memory with fast access speed.

[0071] According to an exemplary embodiment of this disclosure, the parameter storage area may be implemented by PMEM, and the prefetch cache area may be implemented by DRAM.

[0072] In step 803, from the multiple parameters stored in the pre-read cache, it is determined whether multiple second parameters, within the range from the first batch to the second batch, have been pre-read before the second batch. For example, in Figure 7 In the example, you can query the prefetch cache to see if the parameters of Batch N have been prefetched in Batch N-3 to Batch N-1.

[0073] In step 804, if among the multiple second parameters there is a second parameter that has not been pre-read before the second batch within the range from the first batch to the second batch, the unpre-read second parameter is sent to the computing node. For example, in Figure 7 In the example, the second parameter that has not been pre-read in Batch N-3 to Batch N-1 may include parameters B, I, and V.

[0074] According to an exemplary embodiment of this disclosure, in the case where, among a plurality of second parameters, there exists a second parameter that was pre-read before the second batch within the range of the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM (e.g., PMEM), the pre-read second parameter is read and cached in the DRAM of the parameter server. For example, in Figure 7 In the example, the second parameter that is pre-read in Batch N-3 to Batch N-1 may include parameter A.

[0075] According to an exemplary embodiment of this disclosure, whether a pre-read second parameter is an ultra-high frequency parameter can be determined based on the frequency with which it is used in training; if an ultra-high frequency parameter is determined to exist among the pre-read second parameters, the ultra-high frequency parameter is sent to the cache of each computing node using the ultra-high frequency parameter. For example, whether a pre-read second parameter is an ultra-high frequency parameter can be determined based on the distance between the second batch and the most recently pre-read batch before the second batch, wherein when the distance of a certain second parameter among the pre-read second parameters is less than a predetermined threshold, the second parameter is determined to be an ultra-high frequency parameter. Of course, this disclosure is not limited to the above-described method for determining ultra-high frequency parameters, and ultra-high frequency parameters can also be determined by any other feasible means, such as by counting access times, by machine learning, etc.

[0076] According to an exemplary embodiment of this disclosure, when a computing node completes training on multiple first parameters, updated parameter values ​​of the multiple first parameters are received from the computing node and cached in a pre-read cache, and the space storing the multiple first parameters is released from the pre-read cache. For example, from the multiple parameters stored in the pre-read cache, it is determined whether the multiple first parameters have been pre-read after the first batch within the range of the first batch to the second batch; if there is a first parameter among the multiple first parameters that has not been pre-read after the first batch within the range of the first batch to the second batch, the updated parameter value of the unpre-read first parameter is flushed to the parameter storage area and cleared from the pre-read cache; if there is a first parameter among the multiple first parameters that has been pre-read after the first batch within the range of the first batch to the second batch, the updated parameter value of the pre-read first parameter is flushed to the parameter storage area, updating the first parameter of the most recently pre-read batch after the first batch, and sent to the computing node requesting the pre-read of the first parameter of the most recently pre-read batch after the first batch. For example, in Figure 7 As shown in the diagram, after Batch N-3 training is completed, the parameter server's prefetch cache can receive updated parameters E, W, Z, and A from the compute nodes. It can then flush the updated parameters E, W, and Z to the parameter storage area and clear them from the prefetch cache. It can also flush the updated parameter A to the parameter storage area, update its updated value to parameter A in Batch N-1, and send its updated value to each compute node that requested the prefetch of parameter A in Batch N-1.

[0077] According to an exemplary embodiment of this disclosure, it is determined whether the pre-read first parameter is an ultra-high frequency parameter based on the frequency with which it is used in training; if it is determined that an ultra-high frequency parameter exists among the pre-read parameters, the step of sending the ultra-high frequency parameter to the computing node of the first parameter in the most recently pre-read batch after the first batch requested for pre-reading is skipped. This is because the ultra-high frequency parameter can be stored in the cache of the computing node and updated synchronously between computing nodes through ring-reduce, without the need for PULL and PUSH through the parameter server.

[0078] According to an exemplary embodiment of this disclosure, the prefetch cache of the parameter server can store not only prefetched parameters but also cold / hot parameter lists. Therefore, the prefetch cache may include a first area and a second area, wherein the first area is used to store prefetched parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list stores the parameter names of hot parameters, and the cold parameter list stores the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency with which the parameters are used during training.

[0079] According to an exemplary embodiment of this disclosure, when a computing node is training multiple first parameters corresponding to the first batch of training samples, if the parameter name of a second parameter that has not been pre-read before the second batch does not exist in the cold parameter list within the range from the first batch to the second batch, the parameter name of the second parameter that has not been pre-read is stored in the cold parameter list; and if the parameter name of a second parameter that has been pre-read before the second batch does not exist in the hot parameter list within the range from the first batch to the second batch, the parameter name of the pre-read second parameter is stored in the hot parameter list. For example, in Figure 7 As shown in the diagram, when training Batch N-3 and pre-reading Batch N, parameters B, I, and V in Batch N can be stored in the cold parameter list, and parameter A in Batch N can be stored in the hot parameter list.

[0080] According to an exemplary embodiment of this disclosure, when a computing node completes training on multiple first parameters, hot parameters can be determined through a hot parameter list, and the hot parameters are sent to the computing node that most recently requested pre-reading after the currently training batch. Additionally, if the cold parameter list contains parameter names of first parameters that have not been pre-read after the first batch (ranging from the first batch to the second batch), the parameter names of the unpre-read first parameters are removed from the cold parameter list. For example, in... Figure 7 In the illustration, parameter A can be sent to the computation node that requests pre-reading in Batch N-1, and if parameter names E, W, and Z exist in the cold parameter list, parameter names E, W, and Z can be removed from the cold parameter list.

[0081] Figure 9 This is an exemplary prefetch flowchart illustrating an exemplary embodiment of the present disclosure.

[0082] Reference Figure 9 In step 901, it is possible to query from the parameters involved in the training batch (e.g., the first batch) to the previous batch that is being pre-read (e.g., the second batch) whether the parameters involved in the pre-read batch have been pre-read.

[0083] In step 902, if there are parameters in the batch being pre-read that have not been pre-read between the training batch and the previous batch, the unpre-read parameters are returned to the computation node that requested to pre-read these parameters.

[0084] In step 903, if there are parameters among the parameters involved in the pre-read batch that have been pre-read between the training batch and the previous batch, the parameter name of the pre-read parameter is stored in the hot parameter list, and the parameter value of the pre-read parameter is cached in DRAM (in the case where the parameter value of the pre-read parameter is stored in a storage device with a slower read speed than DRAM (e.g., PMEM)).

[0085] In step 904, it can be queried whether the distance between the pre-read parameters in the current pre-read batch and the most recently pre-read batch before the current pre-read batch is less than a predetermined threshold.

[0086] In step 905, if the distance is determined to be less than a predetermined threshold, the pre-read parameters can be identified as ultra-high frequency (UHF) parameters, and these UHF parameters can be sent to the caches of each computing node using them. This allows for the synchronization of UHF parameter updates across computing nodes without going through a parameter server. For example, a GPU hot parameter list can be maintained in the parameter server to store the names of the UHF parameters. The parameters stored in the GPU hot parameter list can be stored in the GPU caches of each computing node and updated synchronously without going through the parameter server.

[0087] Figure 10 This is an exemplary training completion flowchart illustrating an exemplary embodiment of the present disclosure.

[0088] Reference Figure 10 In step 1001, after the current training batch has been trained, it is possible to query whether the parameter names of the parameters involved in the current training batch exist in the cold parameter list.

[0089] In step 1002, if there are parameters in the current training batch that have not been pre-read within the range from the current training batch to the current pre-read batch and exist in the cold parameter list, then remove the parameter names of the unpre-read parameters from the cold parameter list.

[0090] In step 1003, for parameters that are not in the cold parameter list but have been pre-read after the current training batch, it can be determined whether the pre-read parameter is an ultra-high frequency parameter. For example, it can be queried whether the parameter name of the pre-read parameter is in the GPU hot parameter list.

[0091] In step 1004, if the pre-read parameter is an ultra-high frequency parameter, no operation is performed. This is because ultra-high frequency parameters can be stored in the cache of the computing nodes and updated synchronously between computing nodes through ring-reduce, without having to PULL and PUSH through the parameter server.

[0092] In step 1005, if the pre-read parameter is not an ultra-high frequency parameter, the pre-read parameter is sent to the computation node of the pre-read parameter in the most recent pre-read batch after the current training batch.

[0093] In step 1006, space for storing parameters involved in the current training batch can be released from the pre-read cache.

[0094] Figure 11 This is a block diagram illustrating a training apparatus for a machine learning model executed using a parameter server according to an exemplary embodiment of the present disclosure.

[0095] Reference Figure 11 The training apparatus 1100 for a machine learning model executed using a parameter server according to an exemplary embodiment of the present disclosure may include a request receiving unit 1101, a parameter reading unit 1102, a pre-read determination unit 1103, and a parameter sending unit 1104.

[0096] While the computing node is training multiple first parameters corresponding to the first batch of training samples, the request receiving unit 1101 can receive a parameter pre-read request sent by the computing node. This parameter pre-read request is used to request the pre-reading of multiple second parameters corresponding to the second batch of training samples. Here, the second batch is trained after the first batch and is a predetermined number of batches apart from the first batch. The multiple parameters corresponding to each batch of training samples represent multiple parameters of the machine learning model trained using that batch of training samples.

[0097] The parameter reading unit 1102 can read multiple second parameters from the parameter storage area of ​​the parameter server to the pre-read cache area of ​​the parameter server. The pre-read cache area stores multiple parameters corresponding to the first and second batches of training samples. Here, the parameters stored in the parameter storage area and the pre-read cache area can be stored in the form of key-value pairs of parameter names (also called feature names) and parameter values ​​(also called embedding values).

[0098] According to exemplary embodiments of the present disclosure, the parameter storage area may be implemented by at least one of DRAM, PMEM, SSD, HDD, etc., and the prefetch cache area may be implemented by at least one of DRAM, PMEM, etc.

[0099] According to exemplary embodiments of this disclosure, the parameter storage area may be implemented by non-volatile memory (NVM) capable of permanently or semi-permanently storing data, and the read-ahead cache area may be implemented by volatile memory with fast access speed.

[0100] According to an exemplary embodiment of this disclosure, the parameter storage area may be implemented by PMEM, and the prefetch cache area may be implemented by DRAM.

[0101] The pre-read determination unit 1103 can determine from the multiple parameters stored in the pre-read cache whether multiple second parameters have been pre-read before the second batch within the range of the first batch to the second batch.

[0102] The parameter sending unit 1104 can send the unread second parameter to the computing node if there is a second parameter that has not been pre-read before the second batch in the range of the first batch to the second batch.

[0103] According to an exemplary embodiment of this disclosure, the training apparatus 1100 may further include a parameter cache unit (not shown). In cases where, among a plurality of second parameters, there exists a second parameter that has been pre-read prior to the second batch within the range of the first to second batches, and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM (e.g., PMEM), the parameter cache unit may read and cache the pre-read second parameter onto the DRAM of the parameter server.

[0104] According to an exemplary embodiment of this disclosure, the pre-read determination unit 1103 can determine whether a pre-read second parameter is an ultra-high frequency parameter based on the frequency with which it is used in training; if it is determined that an ultra-high frequency parameter exists among the pre-read second parameters, the ultra-high frequency parameter is sent to the cache of each computing node using the ultra-high frequency parameter. For example, the pre-read determination unit 1103 can determine whether a pre-read second parameter is an ultra-high frequency parameter based on the distance between the second batch and the most recently pre-read batch before the second batch, wherein when the distance of a certain second parameter among the pre-read second parameters is less than a predetermined threshold, the second parameter is determined to be an ultra-high frequency parameter. Of course, this disclosure is not limited to the above-described method for determining ultra-high frequency parameters, and ultra-high frequency parameters can also be determined by any other feasible means, such as by counting access times, by machine learning, etc.

[0105] According to an exemplary embodiment of this disclosure, the training apparatus 1100 may further include a pre-read management unit (not shown). When the computing node completes training on the plurality of first parameters, the pre-read management unit receives the updated parameter values ​​of the plurality of first parameters from the computing node, caches the updated parameter values ​​of the plurality of first parameters in a pre-read cache, and releases the space storing the plurality of first parameters from the pre-read cache. For example, the prefetch management unit determines from the multiple parameters stored in the prefetch cache whether multiple first parameters have been prefetched after the first batch within the range from the first batch to the second batch; if there is a first parameter among the multiple first parameters that has not been prefetched after the first batch within the range from the first batch to the second batch, the updated parameter value of the first parameter that has not been prefetched is flushed to the parameter storage area and cleared from the prefetch cache; if there is a first parameter among the multiple first parameters that has been prefetched after the first batch within the range from the first batch to the second batch, the updated parameter value of the prefetched first parameter is flushed to the parameter storage area, updated to the first parameter of the most recently prefetched batch after the first batch, and sent to the computing node that requested the prefetching of the first parameter of the most recently prefetched batch after the first batch.

[0106] According to an exemplary embodiment of this disclosure, the prefetch management unit can determine whether the prefetched first parameter is an ultra-high frequency parameter based on the frequency with which it is used in training; if it is determined that an ultra-high frequency parameter exists among the prefetched parameters, the step of sending the ultra-high frequency parameter to the computing node of the first parameter in the most recently prefetched batch after the first batch of requests for prefetching is skipped. This is because the ultra-high frequency parameter can be stored in the cache of the computing node and updated synchronously between computing nodes through ring-reduce, without the need for PULL and PUSH through the parameter server.

[0107] According to an exemplary embodiment of this disclosure, the prefetch cache of the parameter server can store not only prefetched parameters but also cold / hot parameter lists. Therefore, the prefetch cache may include a first area and a second area, wherein the first area is used to store prefetched parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list stores the parameter names of hot parameters, and the cold parameter list stores the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency with which the parameters are used during training.

[0108] According to an exemplary embodiment of this disclosure, the training apparatus 1100 may further include a list management unit (not shown). When a computing node is training multiple first parameters corresponding to the first batch of training samples, if there is no parameter name of a second parameter that has not been pre-read before the second batch within the range from the first batch to the second batch in the cold parameter list, the list management unit stores the parameter name of the second parameter that has not been pre-read into the cold parameter list; and if there is no parameter name of a second parameter that has been pre-read before the second batch within the range from the first batch to the second batch in the hot parameter list, the list management unit stores the parameter name of the second parameter that has been pre-read into the hot parameter list.

[0109] According to an exemplary embodiment of this disclosure, when a computing node completes training on multiple first parameters, hot parameters can be determined through a hot parameter list, and the hot parameters can be sent to the computing node that most recently requested pre-reading after the currently training batch. Furthermore, if the cold parameter list contains parameter names of first parameters that have not been pre-read after the first batch (ranging from the first batch to the second batch), the list management unit removes the parameter names of the unpre-read first parameters from the cold parameter list.

[0110] Figure 12 This is a block diagram illustrating a parameter server according to an exemplary embodiment of the present disclosure.

[0111] Reference Figure 12 According to an exemplary embodiment of the present disclosure, the parameter server 1200 may include a parameter storage area 1201 and a prefetch cache 1202. The parameter storage area 1201 may be used to store parameters of a machine learning model. The prefetch cache 1202 may be used to store parameters of the machine learning model requested for prefetching by a computing node. Parameters stored in the parameter storage area 1201 and the prefetch cache 1202 may be stored in the form of key-value pairs of parameter names (e.g., A, B, C…) and parameter values ​​(e.g., Data).

[0112] According to an exemplary embodiment of the present disclosure, the parameter storage area 1201 may be implemented by at least one of DRAM, PMEM, SSD, HDD, etc., and the prefetch cache area 1202 may be implemented by at least one of DRAM, PMEM, etc.

[0113] According to an exemplary embodiment of the present disclosure, parameter storage area 1201 may be implemented by non-volatile memory (NVM) capable of permanently or semi-permanently storing data, and read-ahead cache area 1202 may be implemented by volatile memory with fast access speed.

[0114] According to an exemplary embodiment of the present disclosure, the parameter storage area 1201 may be implemented by PMEM, and the prefetch cache area 1202 may be implemented by DRAM.

[0115] According to an exemplary embodiment of this disclosure, the prefetch cache may include a first region 1202-1 (e.g., Figure 7 (in the batch interaction area) and the second area 1202-2 (for example, Figure 7 The feature index area is defined as follows: the first area 1202-1 is used to store pre-read parameters, and the second area 1202-2 is used to store hot parameter lists and cold parameter lists. The hot parameter list is used to store the parameter names of hot parameters, and the cold parameter list is used to store the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use in training.

[0116] The training method and apparatus for a machine learning model using a parameter server according to this disclosure can be based on, for example... Figure 12 The parameters shown are executed by server 1200.

[0117] The above has been referred to Figures 1 to 12 A method and apparatus for training a machine learning model using a parameter server, according to exemplary embodiments of the present disclosure, are described.

[0118] Figure 11 The units in the training apparatus for the machine learning model executed by the parameter server shown can be configured as software, hardware, firmware, or any combination thereof to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, pure software code, or a module combining software and hardware. Furthermore, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, client, or server).

[0119] In addition, refer to Figure 8 The described method for training a machine learning model using a parameter server can be implemented by a program (or instructions) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of this disclosure, a computer-readable storage medium storing instructions, wherein when the instructions are executed by at least one computing device, they cause the at least one computing device to execute the method for training a machine learning model using a parameter server according to this disclosure.

[0120] The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, and servers. It should be noted that the computer program can also be used to perform additional steps beyond those described above, or to perform more specific processing while performing the above steps. The details of these additional steps and further processing are already described in the reference... Figure 8 The relevant methods were mentioned in the description of the process, so they will not be repeated here to avoid repetition.

[0121] It should be noted that each unit in the training apparatus for a machine learning model executed by a parameter server according to an exemplary embodiment of the present disclosure can rely entirely on the operation of a computer program to achieve its corresponding function. That is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called by a specialized software package (e.g., a lib library) to achieve its corresponding function.

[0122] on the other hand, Figure 11 The units shown can also be implemented using hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment used to perform the corresponding operation can be stored in a computer-readable medium such as a storage medium, so that the processor can perform the corresponding operation by reading and running the corresponding program code or code segment.

[0123] For example, exemplary embodiments of the present disclosure can also be implemented as a computing device including a storage component and a processor, wherein the storage component stores a set of computer-executable instructions, and when the set of computer-executable instructions is executed by the processor, a training method for a machine learning model executed using a parameter server according to exemplary embodiments of the present disclosure is performed.

[0124] Specifically, the computing device can be deployed on a server or client, or on node devices in a distributed network environment. Furthermore, the computing device can be a PC, tablet, personal digital assistant, smartphone, web application, or other device capable of executing the aforementioned set of instructions.

[0125] Here, the computing device is not necessarily a single computing device, but can be any collection of devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. The computing device can also be part of an integrated control system or system manager, or can be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0126] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0127] Some operations described in the training method of a machine learning model executed using a parameter server according to exemplary embodiments of the present disclosure can be implemented in software, some operations can be implemented in hardware, and some operations can be implemented in a combination of software and hardware.

[0128] The processor can execute instructions or code stored in one of the storage components, which can also store data. Instructions and data can also be sent and received over a network via a network interface device, which can employ any known transport protocol.

[0129] Storage components can be integrated with the processor, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, storage components can include separate devices, such as external disk drives, storage arrays, or other storage devices that can be used by any database system. Storage components and the processor can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor to read files stored in the storage component.

[0130] In addition, the computing device may include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the computing device may be interconnected via a bus and / or network.

[0131] The training method for a machine learning model executed using a parameter server according to exemplary embodiments of this disclosure can be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams can be equally integrated into a single logic device or operate according to non-precise boundaries.

[0132] Therefore, refer to Figure 8 The described method for training a machine learning model using a parameter server can be implemented through a system comprising at least one computing device and at least one storage device containing storage instructions.

[0133] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a training method of a machine learning model executed using a parameter server, according to an exemplary embodiment of the present disclosure. A storage device stores a set of computer-executable instructions, which, when executed by the at least one computing device, execute a method for training a machine learning model using a parameter server. Figure 8 The described method utilizes a parameter server to train a machine learning model.

[0134] The foregoing has described various exemplary embodiments of this disclosure. It should be understood that the foregoing description is exemplary only and not exhaustive, and this disclosure is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A method for training a machine learning model using a parameter server, comprising: While the computing node is training multiple first parameters corresponding to the first batch of training samples, a parameter prefetch request is received from the computing node. The parameter prefetch request is used to request the prefetching of multiple second parameters corresponding to the second batch of training samples. The second batch is after the first batch in the training order and is separated from the first batch by a predetermined number of batches. The multiple parameters corresponding to each batch of training samples represent multiple parameters of the machine learning model trained using that batch of training samples. The plurality of second parameters are read from the parameter storage area of ​​the parameter server into the pre-read cache area of ​​the parameter server, wherein the pre-read cache area stores the plurality of parameters corresponding to the first batch to the second batch of training samples; From the plurality of parameters stored in the pre-read cache, determine whether the plurality of second parameters have been pre-read before the second batch within the range of the first batch to the second batch; If, among the plurality of second parameters, there is a second parameter that has not been pre-read before the second batch within the range of the first batch to the second batch, the unpre-read second parameter is sent to the computing node.

2. The training method as described in claim 1, wherein, Also includes: If, among the plurality of second parameters, there exists a second parameter that was pre-read before the second batch within the range of the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a read speed slower than DRAM, the pre-read second parameter is read and cached in the DRAM of the parameter server.

3. The training method as described in claim 2, wherein, Also includes: Based on the frequency of the pre-read second parameter used in training, determine whether the pre-read second parameter is an ultra-high frequency parameter; If it is determined that there is an ultra-high frequency parameter among the pre-read second parameters, the ultra-high frequency parameter is sent to the cache of each computing node that uses the ultra-high frequency parameter.

4. The training method as described in claim 3, wherein, The step of determining whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training includes: Based on the distance between the pre-read second parameter and the most recently pre-read batch before the second batch, it is determined whether the pre-read second parameter is an UHF parameter. When the distance of a certain pre-read second parameter is less than a predetermined threshold, the second parameter is determined to be an UHF parameter.

5. The training method as described in claim 1, further comprising: When the computing node completes the training of the plurality of first parameters, the updated parameter values ​​of the plurality of first parameters are received from the computing node and cached in the pre-read cache area, and the space for storing the plurality of first parameters is released from the pre-read cache area.

6. The training method as described in claim 5, wherein, Releasing space from the prefetch cache that stores the plurality of first parameters includes: From the plurality of parameters stored in the pre-read cache, determine whether the plurality of first parameters have been pre-read after the first batch within the range of the first batch to the second batch; If, among the plurality of first parameters, there is a first parameter that has not been pre-read after the first batch within the range from the first batch to the second batch, the updated parameter value of the first parameter that has not been pre-read is flushed to the parameter storage area and cleared from the pre-read cache area; If, among the plurality of first parameters, there exists a first parameter that has been pre-read after the first batch within the range of the first batch to the second batch, the updated parameter value of the pre-read first parameter is flushed down to the parameter storage area, updating the first parameter of the most recently pre-read batch after the first batch, and sent to the computing node that requested to pre-read the first parameter of the most recently pre-read batch after the first batch.

7. The training method as described in claim 6, further comprising: Based on the frequency of the pre-read first parameter used in training, determine whether the pre-read first parameter is an ultra-high frequency parameter; If it is determined that there is an ultra-high frequency parameter among the first parameters that have been pre-read, the step of sending the ultra-high frequency parameter to the computing node of the first parameter in the most recent pre-read batch after the request to pre-read the first batch is skipped.

8. The training method according to any one of claims 1 to 7, wherein, The pre-read cache area includes a first area and a second area. The first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list is used to store the parameter names of hot parameters, and the cold parameter list is used to store the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use in training. The training method further includes: When a computing node is training multiple first parameters corresponding to the first batch of training samples, if the parameter name of a second parameter that has not been pre-read before the second batch does not exist in the cold parameter list within the range from the first batch to the second batch, the parameter name of the second parameter that has not been pre-read is stored in the cold parameter list; and if the parameter name of a second parameter that has been pre-read before the second batch does not exist in the hot parameter list within the range from the first batch to the second batch, the parameter name of the second parameter that has been pre-read is stored in the hot parameter list. The training method further includes: When the computing node completes the training of the plurality of first parameters, if there are parameter names of first parameters that have not been pre-read after the first batch in the range from the first batch to the second batch in the cold parameter list, the parameter names of the first parameters that have not been pre-read are removed from the cold parameter list.

9. The training method as described in claim 1, wherein, The parameter storage area is implemented using persistent memory, and the prefetch cache area is implemented using DRAM.

10. A training apparatus for a machine learning model executed using a parameter server, comprising: The request receiving unit is configured to receive a parameter pre-read request sent by the computing node when the computing node is training multiple first parameters corresponding to the first batch of training samples. The parameter pre-read request is used to request the pre-reading of multiple second parameters corresponding to the second batch of training samples. The second batch is after the first batch in the training order and is separated from the first batch by a predetermined number of batches. The multiple parameters corresponding to each batch of training samples represent multiple parameters of the machine learning model trained using that batch of training samples. The parameter reading unit is configured to read the plurality of second parameters from the parameter storage area of ​​the parameter server to the pre-read cache area of ​​the parameter server, wherein the pre-read cache area stores the plurality of parameters corresponding to the first batch to the second batch of training samples; The pre-read determination unit is configured to: determine from the plurality of parameters stored in the pre-read cache whether the plurality of second parameters have been pre-read before the second batch within the range of the first batch to the second batch; The parameter sending unit is configured to send the unread second parameter to the computing node if, among the plurality of second parameters, there is a second parameter that has not been pre-read before the second batch within the range of the first batch to the second batch.

11. The training apparatus of claim 10, wherein, Also includes: The parameter caching unit is configured to read and cache the pre-read second parameter in the DRAM of the parameter server if, among the plurality of second parameters, there is a second parameter that was pre-read before the second batch within the range of the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a slower read speed than DRAM.

12. The training apparatus of claim 11, wherein, The pre-read determination unit is also configured as follows: Based on the frequency of the pre-read second parameter used in training, determine whether the pre-read second parameter is an ultra-high frequency parameter; The parameter sending unit is also configured to send the ultra-high frequency parameter to the cache of each computing node that uses the ultra-high frequency parameter if it is determined that there is an ultra-high frequency parameter among the pre-read second parameters.

13. The training apparatus of claim 12, wherein, The pre-read determination unit is configured as follows: Based on the distance between the pre-read second parameter and the most recently pre-read batch before the second batch, it is determined whether the pre-read second parameter is an UHF parameter. When the distance of a certain pre-read second parameter is less than a predetermined threshold, the second parameter is determined to be an UHF parameter.

14. The training apparatus of claim 10, further comprising: The pre-read management unit is configured to: when the computing node completes the training of the plurality of first parameters, receive the updated parameter values ​​of the plurality of first parameters from the computing node, cache the updated parameter values ​​of the plurality of first parameters in the pre-read cache, and release the space storing the plurality of first parameters from the pre-read cache.

15. The training apparatus of claim 14, wherein, The prefetch management unit is configured to: determine from the plurality of parameters stored in the prefetch cache whether the plurality of first parameters have been prefetched after the first batch within the range of the first batch to the second batch; If, among the plurality of first parameters, there is a first parameter that has not been pre-read after the first batch within the range from the first batch to the second batch, the updated parameter value of the first parameter that has not been pre-read is flushed to the parameter storage area and cleared from the pre-read cache area; If, among the plurality of first parameters, there exists a first parameter that has been pre-read after the first batch within the range of the first batch to the second batch, the updated parameter value of the pre-read first parameter is flushed down to the parameter storage area, updating the first parameter of the most recently pre-read batch after the first batch, and sent to the computing node that requested to pre-read the first parameter of the most recently pre-read batch after the first batch.

16. The training apparatus of claim 15, wherein the pre-read management unit is further configured to: Based on the frequency of the pre-read first parameter used in training, determine whether the pre-read first parameter is an ultra-high frequency parameter; If it is determined that there is an ultra-high frequency parameter among the first parameters that have been pre-read, the step of sending the ultra-high frequency parameter to the computing node of the first parameter in the most recent pre-read batch after the request to pre-read the first batch is skipped.

17. The training apparatus according to any one of claims 10 to 16, wherein, The pre-read cache area includes a first area and a second area. The first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list is used to store the parameter names of hot parameters, and the cold parameter list is used to store the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use in training. The training device also includes a list management unit; When a computing node is training multiple first parameters corresponding to the first batch of training samples, the list management unit is configured to: if there is no parameter name of a second parameter that has not been pre-read before the second batch within the range from the first batch to the second batch in the cold parameter list, store the parameter name of the second parameter that has not been pre-read into the cold parameter list; and if there is no parameter name of a second parameter that has been pre-read before the second batch within the range from the first batch to the second batch in the hot parameter list, store the parameter name of the second parameter that has been pre-read into the hot parameter list. When the computing node completes the training of the plurality of first parameters, the list management unit is configured to remove the parameter name of the first parameter that has not been pre-read after the first batch from the cold parameter list if there is a parameter name of the first parameter that has not been pre-read within the range of the first batch to the second batch.

18. The training apparatus of claim 10, wherein, The parameter storage area is implemented using persistent memory, and the prefetch cache area is implemented using DRAM.

19. A parameter server, comprising: The parameter storage area is configured to store the parameters of the machine learning model; A prefetch cache is configured to store parameters of the machine learning model that are requested to be prefetched by the compute nodes; When the computing node is training multiple first parameters corresponding to the first batch of training samples, a parameter pre-read request sent by the computing node is received. The parameter pre-read request is used to request the pre-read of multiple second parameters corresponding to the second batch of training samples. The second batch is after the first batch in the training order and is separated from the first batch by a predetermined number of batches. The multiple parameters corresponding to each batch of training samples represent multiple parameters of the machine learning model trained using that batch of training samples. The plurality of second parameters are read from the parameter storage area into the pre-read cache area, wherein the pre-read cache area stores the plurality of parameters corresponding to the first batch to the second batch of training samples; From the plurality of parameters stored in the pre-read cache, determine whether the plurality of second parameters have been pre-read before the second batch within the range of the first batch to the second batch; If, among the plurality of second parameters, there is a second parameter that has not been pre-read before the second batch within the range of the first batch to the second batch, the unpre-read second parameter is sent to the computing node.

20. The parameter server as described in claim 19, wherein, If, among the plurality of second parameters, there exists a second parameter that was pre-read before the second batch within the range of the first batch to the second batch, and the pre-read second parameter is stored in a storage device with a read speed slower than DRAM, the pre-read second parameter is read and cached in the DRAM of the parameter server.

21. The parameter server as described in claim 20, wherein, Based on the frequency of the pre-read second parameter used in training, determine whether the pre-read second parameter is an ultra-high frequency parameter; If it is determined that there is an ultra-high frequency parameter among the pre-read second parameters, the ultra-high frequency parameter is sent to the cache of each computing node that uses the ultra-high frequency parameter.

22. The parameter server as described in claim 21, wherein, The step of determining whether the pre-read second parameter is an ultra-high frequency parameter based on the frequency of its use in training includes: Based on the distance between the pre-read second parameter and the most recently pre-read batch before the second batch, it is determined whether the pre-read second parameter is an UHF parameter. When the distance of a certain pre-read second parameter is less than a predetermined threshold, the second parameter is determined to be an UHF parameter.

23. The parameter server as described in claim 19, wherein, When the computing node completes the training of the plurality of first parameters, the updated parameter values ​​of the plurality of first parameters are received from the computing node and cached in the pre-read cache area, and the space for storing the plurality of first parameters is released from the pre-read cache area.

24. The parameter server as described in claim 23, wherein, Releasing space from the prefetch cache that stores the plurality of first parameters includes: From the plurality of parameters stored in the pre-read cache, determine whether the plurality of first parameters have been pre-read after the first batch within the range of the first batch to the second batch; If, among the plurality of first parameters, there is a first parameter that has not been pre-read after the first batch within the range from the first batch to the second batch, the updated parameter value of the first parameter that has not been pre-read is flushed to the parameter storage area and cleared from the pre-read cache area; If, among the plurality of first parameters, there exists a first parameter that has been pre-read after the first batch within the range of the first batch to the second batch, the updated parameter value of the pre-read first parameter is flushed down to the parameter storage area, updating the first parameter of the most recently pre-read batch after the first batch, and sent to the computing node that requested to pre-read the first parameter of the most recently pre-read batch after the first batch.

25. The parameter server as described in claim 24, wherein, Based on the frequency of the pre-read first parameter used in training, determine whether the pre-read first parameter is an ultra-high frequency parameter; If it is determined that there is an ultra-high frequency parameter among the first parameters that have been pre-read, the step of sending the ultra-high frequency parameter to the computing node of the first parameter in the most recent pre-read batch after the request to pre-read the first batch is skipped.

26. The parameter server as claimed in any one of claims 19 to 25, wherein, The pre-read cache area includes a first area and a second area. The first area is used to store pre-read parameters, and the second area is used to store a hot parameter list and a cold parameter list. The hot parameter list is used to store the parameter names of hot parameters, and the cold parameter list is used to store the parameter names of cold parameters. The hot parameters and the cold parameters are divided according to the frequency of their use in training. Specifically, when a computing node is training multiple first parameters corresponding to the first batch of training samples, if the parameter name of a second parameter that has not been pre-read before the second batch does not exist in the cold parameter list within the range from the first batch to the second batch, the parameter name of the second parameter that has not been pre-read is stored in the cold parameter list; and if the parameter name of a second parameter that has been pre-read before the second batch does not exist in the hot parameter list within the range from the first batch to the second batch, the parameter name of the second parameter that has been pre-read is stored in the hot parameter list. When the computing node completes the training of the plurality of first parameters, if there are parameter names of first parameters that have not been pre-read after the first batch within the range of the first batch to the second batch in the cold parameter list, the parameter names of the first parameters that have not been pre-read are removed from the cold parameter list.

27. The parameter server as described in claim 19, wherein, The parameter storage area is implemented using persistent memory, and the prefetch cache area is implemented using DRAM.

28. A computer-readable storage medium for storing instructions, wherein, When the instruction is executed by at least one computing device, it causes the at least one computing device to perform the training method of the machine learning model using a parameter server as described in any one of claims 1 to 9.

29. A system comprising at least one computing device and at least one storage device for storing instructions, wherein, When the instruction is executed by the at least one computing device, it causes the at least one computing device to perform the training method of the machine learning model using a parameter server as described in any one of claims 1 to 9.

Images