A neural network model training method, device and computing device cluster

Abstract

A neural network model training method, the method comprises: in the training process of the current training step, obtaining a sample feature vector and first state data, wherein the sample feature vector is obtained based on a training sample related to the current training step, the first state data is state data from a remote memory pool and obtained by a previous training step, and the current training step is a training step subsequent to the previous training step in the training process of the neural network model; training the current training step based on the sample feature vector and the state data; and unloading second state data to the remote memory pool, wherein the second state data is state data obtained based on the current training step, and at least part of the second state data is used for training of a next training step, and the next training step is a training step subsequent to the current training step in the training process of the neural network model. Thus, in the event of a fault, only the operators involved in the current task are calculated, increasing the availability of the training cluster.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to a neural network model training method, apparatus, and computing device cluster. Background Technology

[0002] Training large models typically involves numerous weight parameters and massive training corpora, thus demanding significant computing power. It often requires training clusters comprised of tens of thousands of AI accelerators for extended periods, lasting months. Furthermore, large model training generally employs a multi-GPU synchronization mechanism, meaning that each training iteration must aggregate weight gradients from all AI accelerators and update the weight parameters before proceeding to the next step. This means that even if only one AI accelerator fails, the entire training cluster must revert to the most recent checkpoint, resulting in significant waste of computing resources and training delays. Therefore, minimizing time losses due to training failures remains a critical technical challenge. Summary of the Invention

[0003] This application provides a neural network model training method, apparatus, computing device cluster, computer storage medium, and computer product, which can perform only operator-level rollback when the AI ​​accelerator fails, increasing the availability of the training cluster.

[0004] Firstly, this application provides a method for training a neural network model. The method includes: during the training process of the current training step, acquiring sample feature vectors and first state data, wherein the first state data is state data obtained in the previous training step from a remote memory pool, wherein the sample feature vectors are obtained based on training samples related to the current training step, and the current training step is a training step following the previous training step in the training process of the neural network model; training the current training step based on the sample feature vectors and state data; and unloading second state data obtained in the current training step into the remote memory pool, wherein at least a portion of the second state data is used for training the next training step, which is a training step following the current training step in the training process of the neural network model.

[0005] For example, state data includes one or more of the following: weights, gradients, activations, or optimizers. Here, the state data obtained in the previous training step is called the first state data, and the state data obtained in the current training step is called the second state data. Unloading refers to transferring data from the AI ​​accelerator to a remote memory pool and simultaneously deleting the data from the AI ​​accelerator. For example, the second state data is stored in the remote memory pool before the next training step begins. For example, at least a portion of the second state data used for training in the next training step may be data such as weights obtained in the current training step, and some data in the second state data may also be used for training in the current training step, such as activation data and gradients obtained in the current training step.

[0006] In this way, by offloading the second-state data to a remote memory pool during a training step, the AI ​​accelerator does not store state data for extended periods during neural network model training. This ensures that the second-state data already obtained in the current training step is not lost in the event of an AI accelerator failure or data loss. In this case, the AI ​​accelerator can then retrieve the first-state data and the already obtained second-state data from the remote memory pool to recalculate the operators involved in the current task, thereby increasing the availability of the training cluster.

[0007] In one possible implementation, training for the current training step is performed based on the sample feature vector and the first state data, including: obtaining the first activation based on the sample feature vector, wherein the first activation refers to the activation data obtained during the forward propagation of the sample feature vector in the current training step; and reading a portion of the weight data from the first state data from the remote memory pool when calculating the first activation.

[0008] For example, the calculation of the first activation can be divided into multiple parts, so that while calculating one part, some weight data is read from a remote memory pool. Alternatively, the weights related to the matrix multiplication operators to be calculated later can be read while calculating the first activation.

[0009] In this way, data stored in a remote memory pool can be read simultaneously while the first activation is being calculated, thus masking the calculation and reading processes and reducing the adverse impact of the remote memory pool's read speed on the AI ​​accelerator's calculation speed.

[0010] In one possible implementation, the first state data includes a first weight. When calculating the first activation, partial weight data from the first state data is read from a remote memory pool. This includes: when calculating the first activation is performed by executing a matrix multiplication operator, performing multiple rounds of matrix multiplication based on the first activation and the first weight associated with the current matrix multiplication operator, so that a partial result corresponding to the sub-weight is obtained in any round of matrix multiplication. The first weight contains multiple sub-weights, and the combination of each sub-weight is equivalent to the first weight. Based on the partial results corresponding to each sub-weight, a calculation result is obtained, which refers to the result of matrix multiplication between the first activation and the first weight. Specifically, in the current round of matrix multiplication, matrix multiplication is performed based on the first activation and the sub-weights to obtain a partial result corresponding to the sub-weight, and at least one of the unread sub-weights is read from the remote memory pool. At least one of the unread sub-weights is partial weight data from the first state data.

[0011] For example, the AI ​​accelerator has a caching structure, including L1 / L2 cache and L3 cache. During data preparation, a portion of the weights W is read into the L1 / L2 cache, and the computation core can then perform a matrix multiplication of that portion of the weights with the activation X. Furthermore, while performing the matrix multiplication on this portion of the weights, the remaining portion of the weights W can be read.

[0012] In this way, the time spent activating X and calculating some of the weights masks the time spent reading the remaining weights from the remote memory pool. This masking refers to the effective concealment of the latency in loading data from the slower remote memory pool into the cache during the computation process through the scheduling of data read and computation operations. This means that the observable total computation time does not significantly increase due to read latency. Therefore, the computation of matrix multiplication operators can be optimized so that the computational performance of the computational core is not hampered by the bandwidth of the remote memory pool.

[0013] In one possible implementation, the first state data includes a first weight. When calculating the first activation, a portion of the weight data in the first state data is read from a remote memory pool. This includes: when the calculation of the first activation is a computation task with a matrix multiplication operator, obtaining a computation result related to the first weight based on the first activation and the first weight associated with the current computation task, wherein the current computation task contains at least one matrix multiplication operator, and the computation task is used to represent the smallest execution unit obtained by partitioning the computation graph based on a pre-set granularity; when performing the operation of the current computation task, reading the weights associated with the matrix multiplication operator in the next computation task from the remote memory pool, wherein the next computation task is used to represent a computation task located after the current computation task and containing a matrix multiplication operator, and the weights associated with the matrix multiplication operator in the next computation task are a portion of the weight data in the first state data.

[0014] For example, the computational task can vary depending on the choice of granularity; the computational task can include network layers or operators.

[0015] Thus, when the computation task is at the network layer, since the network layer contains many operators, and the data required for most operator computations is already in the cache, the overhead of data retrieval is very small and can be ignored. Therefore, when these operators are being computed, they only transfer data between caches (e.g., reading from L1 / L2 cache to L3 cache), so data can still be continuously read from the remote memory pool into the cache during this period. Therefore, when the computation task is at the network layer, the negative impact of the remote memory pool bandwidth on the data computation of the computing core can be reduced by reading the weights related to the matrix multiplication operator in the next network layer while performing the computation of the current network layer. On the other hand, when the computation task is at the operator level, the weights related to the next matrix multiplication operator can be read while performing the matrix multiplication operator computation, thereby masking the computation and reading and increasing computational efficiency.

[0016] In one possible implementation, the AI ​​accelerator includes a cache structure, and the method further includes: after obtaining the first activation, the first activation is stored in the cache structure of the AI ​​accelerator until the computation of the first activation is completed, and then the first activation is unloaded from the cache structure to a remote memory pool.

[0017] This allows state data to be stored in a remote memory pool in real time, and when the AI ​​accelerator fails, the stored state data avoids recalculation from the checkpoint. Furthermore, storing the first activation in the AI ​​accelerator's cache structure after it is obtained reduces the number of interactions between the AI ​​accelerator and the remote memory pool in subsequent calculations, thus improving computational efficiency.

[0018] In one possible implementation, the neural network model includes a large model.

[0019] For example, the number of rows and columns activated in the first instance is greater than 1000.

[0020] Thus, when the neural network model is large, the number of characters (tokens) in a batch is generally large, resulting in a large activation value X. Therefore, the time spent on matrix multiplication of activation value X is also relatively long. Consequently, the time spent reading sub-weights from a remote memory pool can be masked by computation.

[0021] In one possible implementation, the L1 / L2 cache is based on static random access memory (SRAM), and the L3 cache is based on dynamic random access memory (DRAM).

[0022] For example, the L3 cache has a capacity of more than 10GB.

[0023] In this way, the main memory (also known as video memory) originally located in the AI ​​accelerator is used as a cache, and the data originally stored in the AI ​​accelerator is moved to a remote memory pool. This means that if the AI ​​accelerator fails and data is lost, it is unnecessary to retrain using checkpointing; instead, training can be performed using the data stored in the remote memory pool. This increases the availability of the training cluster.

[0024] In one possible implementation, the method further includes: when the failure of the first AI accelerator is recovered, obtaining first state data and second state data related to the first task from a remote memory pool, wherein the first task is a task whose execution was interrupted due to the failure, determined according to the second state data related to the first AI accelerator in the remote memory pool; and executing the first task according to the first state data and second state data related to the first task.

[0025] For example, the first task could be computation on network layers, computation on operators, or computation on the current training step.

[0026] In this way, by storing both the first and second state data in a remote memory pool, a rollback of at most one training step can be performed when the AI ​​accelerator recovers from a failure, thus avoiding repeated calculations of a large number of training steps when calculating from the most recent checkpoint. Furthermore, depending on the pre-set parameters, the first task can either be a recalculation of the current operator or simply a rollback of a few operators before continuing the calculation.

[0027] In one possible implementation, the method further includes: if second state data for executing the first task is missing, retrieving first state data and second state data corresponding to the second task from a remote memory pool, wherein the second task is determined based on the missing second state data; executing the second task according to the first state data and second state data corresponding to the second task to obtain the missing second state data; and executing the first task based on the missing second state data and the first state data and second state data related to the first task.

[0028] For example, the second task could be computation of network layers or computation of operators.

[0029] In this way, when the AI ​​accelerator recovers from a failure, it can roll back to the computation of only a certain network layer or operator in the current training step, thereby improving the availability of the training cluster.

[0030] Secondly, this application provides an intent recognition device, which includes an acquisition module and a processing module. The acquisition module is used to acquire sample feature vectors and first state data during the training process of the current training step. The sample feature vectors are obtained based on training samples related to the current training step, and the first state data are state data obtained from the previous training step from a remote memory pool. The current training step is a training step that follows the previous training step in the training process of the neural network model.

[0031] The processing module is used to train the current training step based on the sample feature vector and the first state data; and to unload the second state data to the remote memory pool, wherein the second state data is the state data obtained based on the current training step, and at least a portion of the second state data is used for training the next training step, which is the training step that follows the current training step in the training process of the neural network model.

[0032] In one possible implementation, the processing module is further configured to: obtain a first activation based on the sample feature vector and the first state data during the training of the current training step; and, when calculating the first activation, read a portion of the weight data from the first state data from the remote memory pool.

[0033] In one possible implementation, the first state data includes a first weight. The processing module is further configured to read partial weight data from the first state data from a remote memory pool when calculating the first activation. During this process: when the calculation of the first activation is performed by executing a matrix multiplication operator, multiple rounds of matrix multiplication are performed based on the first activation and the first weight associated with the current matrix multiplication operator, so that a partial result corresponding to the sub-weight is obtained in any round of matrix multiplication. The first weight contains multiple sub-weights, and the combination of each sub-weight is equivalent to the first weight. Based on the partial results corresponding to each sub-weight, a calculation result is obtained, which refers to the result of matrix multiplication of the first activation and the first weight. Specifically, during the current round of matrix multiplication, matrix multiplication is performed based on the first activation and the sub-weights to obtain a partial result corresponding to the sub-weights, and at least one of the unread sub-weights is read from the remote memory pool. At least one of the unread sub-weights is partial weight data from the first state data.

[0034] In one possible implementation, the first state data includes a first weight. The processing module is further configured to read a portion of the weight data from the first state data from a remote memory pool when calculating the first activation. During this process: when the calculation of the first activation is a computation task with a matrix multiplication operator, the calculation result related to the first weight is obtained based on the first activation and the first weight associated with the current computation task. The current computation task contains at least one matrix multiplication operator, and the computation task is used to represent the smallest execution unit obtained by partitioning the computation graph based on a pre-set granularity. When performing the operation of the current computation task, the weights associated with the matrix multiplication operator in the next computation task are read from the remote memory pool. The next computation task is used to represent a computation task located after the current computation task and containing a matrix multiplication operator. The weights associated with the matrix multiplication operator in the next computation task are a portion of the weight data from the first state data.

[0035] In one possible implementation, the AI ​​accelerator includes a cache structure, and the processing module is further configured to store the first activation in the cache structure of the AI ​​accelerator after the first activation is obtained, until the calculation of the first activation is completed, and then unload the first activation from the cache structure to a remote memory pool.

[0036] In one possible implementation, the neural network model includes a large model.

[0037] In one possible implementation, the state data includes one or more of the following: weights, gradients, activations, or optimizers.

[0038] Thirdly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0039] Fourthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the computing device cluster may include one or more computing devices.

[0040] Fifthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0041] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0042] Figure 1 and Figure 2 This is a timing diagram of an existing checkpoint-based fault detection method;

[0043] Figure 3 This is a schematic diagram of the architecture of a large model training system provided in an embodiment of this application;

[0044] Figure 4 This is a schematic diagram of the architecture of another large model training system provided in the embodiments of this application;

[0045] Figure 5 This is a timing diagram illustrating fault resolution when an AI accelerator malfunctions, provided in an embodiment of this application.

[0046] Figure 6 This is a schematic diagram illustrating a technical concept for performing calculations on matrix multiplication operators, provided in an embodiment of this application.

[0047] Figure 7 This is a timing diagram illustrating the computation of a matrix multiplication operator provided in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram illustrating matrix multiplication and subweights performed according to an embodiment of this application;

[0049] Figure 9 This is a schematic diagram illustrating a technical concept for calculating matrix multiplication operators in a graph model with granular operators, as provided in an embodiment of this application.

[0050] Figure 10 This is a schematic diagram of a computation graph with a granularity of network layer provided in an embodiment of this application;

[0051] Figure 11 This is a timing diagram of a matrix multiplication operator in a graph pattern with a granularity of network layers, provided in an embodiment of this application.

[0052] Figure 12 This is a flowchart illustrating a neural network training method provided in an embodiment of this application;

[0053] Figure 13 This is a schematic diagram illustrating the relationship between the training process and training steps of a neural network according to an embodiment of this application;

[0054] Figure 14 This is a schematic diagram illustrating retraining when an AI accelerator malfunctions, provided in an embodiment of this application.

[0055] Figure 15 This is a schematic diagram of the structure of a neural network model training device provided in an embodiment of this application;

[0056] Figure 16 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0057] Figure 17 This is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0058] Figure 18 This is a schematic diagram of another computing device cluster structure provided in an embodiment of this application. Detailed Implementation

[0059] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0060] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0061] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0062] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0063] First, the relevant technical terms involved in the technical solution provided in this application will be introduced.

[0064] Large models: Large models refer to deep learning models with billions or even tens of billions of parameters. They are trained with massive amounts of data and powerful computing resources to achieve strong generalization and multi-task processing capabilities. Large models typically employ the Transformer architecture. Common examples of large models include large language models and multimodal models composed of large language models, which are models with an enormous number of parameters. Today, large models are widely used in fields such as natural language processing and image generation.

[0065] Training: Neural network models typically involve numerous matrix multiplications and corresponding weight parameters. Training generally refers to the entire computational process of obtaining the effective weight parameters of the model, with a basic unit being a training step. The entire training process consists of many training steps sequentially.

[0066] A training step refers to the number of times a model's parameters are updated during the training phase in deep learning. Each training step typically includes two processes: forward propagation and backpropagation. In a training step, forward propagation is usually performed first to obtain the model's output and parameters such as the activation values ​​of each layer. Next, the loss is calculated based on the model's output. Then, backpropagation is used to calculate the gradient of each layer based on the loss value. Finally, based on the calculated gradients, an optimization algorithm is used to update the weights.

[0067] A network layer (layer) is a structural unit composed of several neurons in a neural network. It is responsible for performing specific mathematical operations or transformations on the input data (such as linear transformations, activation functions, convolutions, etc.). A neural network typically contains multiple network layers. The output of each network layer serves as the input to the next network layer. Through continuous mapping between network layers, data features are gradually extracted to achieve the final goal of the neural network model (such as classification or regression).

[0068] Forward propagation refers to the unidirectional flow of data from the input layer to the output layer in a neural network, where information is calculated and transmitted layer by layer to ultimately obtain the prediction result.

[0069] Backpropagation refers to the process of calculating the loss based on the output result and then propagating the gradient from the output layer back to the input layer using the chain rule, in order to update and optimize network parameters.

[0070] A checkpoint (CkPt) is a snapshot of model parameters saved during the training process of machine learning and deep learning. Checkpoints typically include data such as weights, gradients, and optimizer states. Using checkpoints, training can continue directly from the saved snapshot data without starting from scratch.

[0071] AI accelerators are hardware accelerators designed to accelerate AI neural networks, deep learning, and machine learning. AI accelerators include various types, such as neural processing units (NPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs).

[0072] High-bandwidth memory (HBM) is a high-performance memory technology based on 3D stacking technology. HBM is a type of dynamic random access memory (DRAM). HBM is often used as GPU video memory, offering significantly higher bandwidth compared to traditional video memory. To meet the demands of high-performance computing, modern HBM is often designed to be tens of gigabytes in size.

[0073] Unloading: In deep learning, unloading refers to moving data from one device or component (such as processor memory) to another location (such as hard drive) and deleting it from the original location at the same time, in order to free up storage space on the original device.

[0074] Operators: Deep learning algorithms are composed of basic computational units called operators. In neural network models, operators correspond to the fundamental computational logic within a layer. Operators come in different types, such as operators for convolution and operators for matrix multiplication. For example, the output data after data passes through a convolution operator indicates that the data has undergone a convolution operation. Similarly, two vectors passing through a matrix multiplication operator indicates that a matrix multiplication calculation will be performed between the two vectors.

[0075] Single-operator mode (eager execution): also known as "dynamic graph mode," is a commonly used framework in deep learning. In this mode, during neural network training, the code within the neural network is executed line by line. Typically, each operator in single-operator mode executes independently.

[0076] Graph execution, also known as static graph execution, is another commonly used framework in deep learning. In graph execution, multiple operators are typically combined into a computation graph, then optimized, compiled, and executed as a whole. Intuitively, graph execution doesn't execute operators initially; instead, a computation graph is generated, optimized, and then executed based on the optimized graph. For example, the logic of layer A in a neural network is: calculate M based on data A, calculate N based on data B, and finally perform matrix multiplication of M and N, where data A and B are unrelated. In single-operator execution, this would be done sequentially according to the code, calculating M, N, and then performing the matrix multiplication. However, in graph execution, choosing an appropriate granularity allows for optimization of layer A's computational logic. One possible approach is to treat operators as granularity, transform the layer into a computation graph, and then optimize that graph. If optimization reveals that calculating M and N are unrelated, the calculations for M and N can be performed in parallel using two threads, saving overall computation time. Understandably, a neural network model can be constructed using either a single-operator pattern or a graph pattern, or it can be constructed using a hybrid pattern that combines both single-operator and graph patterns.

[0077] Granularity: In graph modeling, granularity determines the smallest unit considered during optimization. For example, when the granularity is at the layer level, optimization no longer considers how multiple operators within a layer perform optimization. When the granularity is at the operator level, optimization considers how different operators within the same layer perform optimization. It's understandable that smaller granularity leads to a more complex optimization task, while larger granularity makes the optimization task relatively simpler.

[0078] Next, the technical solution provided in this application will be introduced.

[0079] During the training of large models, training clusters are typically used. A training cluster contains tens of thousands of AI accelerators, creating a cluster effect. Generally, the probability of an AI accelerator failing is low, but due to the cluster effect, accelerator failures are frequent during the training of large models. A large model often requires hundreds of thousands to millions of training steps to complete. During training, the data used is usually stored in the GPU memory of the AI ​​accelerators. When one AI accelerator fails, the data stored in its GPU memory may be lost, forcing the large model to be trained from scratch due to data loss.

[0080] To avoid the increased overall training time caused by training from scratch, a checkpoint-based fault recovery method has been proposed in the field of model training. This method takes a snapshot of the model after a training step, obtaining snapshot data of the model at the time of that step. When a fault occurs during subsequent training, the snapshot data from the most recent checkpoint can be read, and training can begin from the next step corresponding to that checkpoint, avoiding training from scratch and thus reducing the time spent on model training. For example, a checkpoint can be saved after each training round, or after a certain number of steps.

[0081] like Figure 1 As shown in (A), checkpoints can be saved every 1000 steps, i.e., when step = 1000, 2000, 3000... checkpoints are saved. Figure 1 In (B), a fault occurred at step = 1600. At this point, the checkpoint at step = 1000 can be read (the process of reading the checkpoint is also called "fault recovery"), and training at step = 1001 can be performed based on the read snapshot data. However, as... Figure 1 As shown in (B), even if checkpoints are set and fault recovery is performed, the process from step = 1001 to step = 1600 will still be recalculated. The time taken for this recalculation can be called the checkpoint rollback recalculation cost. If we denote the checkpoint rollback recalculation cost as t, then in Figure 1 In (B), it took at least 2t to reach the pre-failure progress, which undoubtedly reduced the overall availability of the training cluster. It's easy to see that the more frequently checkpoints are saved, the lower the overhead of checkpoint rollback and recalculation will be when a failure occurs. However, as... Figure 2 As shown, to prevent some weight parameters from being updated during the saving process, model training is typically stopped during checkpoint saving and fault recovery. Therefore, as the frequency of checkpoint saving increases, the overall time required to complete model training may actually increase, reducing the availability of the training cluster.

[0082] As can be seen, while checkpoint-based fault recovery methods can resume training from the most recently saved checkpoint instead of starting from scratch in the event of a failure, the steps of checkpoint rollback and recalculation, checkpoint saving, and fault recovery still incur significant time costs, reducing the availability of the training cluster. Although bandwidth pressure and transmission latency during checkpoint loading can be alleviated through single-card loading and multi-card transfer, checkpoint loading and the resulting restart of the training process still incur additional overhead. Furthermore, training interruptions caused by these overheads also reduce the overall availability of the system.

[0083] In view of this, this application proposes a neural network training system. By using all storage units on the AI ​​accelerator as caches and setting up a remote memory pool to store state data, when the AI ​​accelerator in the training cluster fails, only operator-level rollback is required, which increases the availability of the system and shortens the training time.

[0084] For example, Figure 3 This illustration shows a schematic diagram of the technical concept of a neural network training system provided in an embodiment of this application. Figure 3 In this system, the neural network training system comprises multiple AI accelerators and a remote memory pool, with the AI ​​accelerators connected to the remote memory pool. The remote memory pool primarily stores the state data of the neural network model, including one or more of the following: weights, gradients, activations, and optimizer data. Located outside the AI ​​accelerators, the remote memory pool has a capacity significantly larger than that of a multi-level cache structure, but its bandwidth is less than that of the multi-level cache structure. For example, the remote memory pool could be a storage pool built using DDR memory. AI accelerators can connect directly to the remote memory pool, or a network switch can be used to coordinate access to the remote memory pool from each AI accelerator; the specific approach depends on the actual situation and is not limited here.

[0085] A single AI accelerator consists of a compute die and a multi-level cache structure. This multi-level cache structure includes L1 / L2 caches and an L3 cache. In this system, the AI ​​accelerator does not hold the state data of the neural network model for extended periods. Instead, it dynamically retrieves relevant state data from a remote memory pool based on the current computation task. After the task is completed, the newly calculated state data is unloaded to the remote memory pool. Data that will not be used again in the short term can also be directly unloaded or destroyed. It should be noted that, to avoid frequent data transfers from the remote memory pool, the AI ​​accelerator can also store state data that will be used in subsequent computation tasks in the cache for a period of time, and then unload it to the remote memory pool when the data will not be used again in the short term. When executing the current computation task, the compute die can simultaneously read the required data from both the local cache and the remote memory pool. For example, assuming the current computation task is to perform the multiplication of matrix A and matrix B, matrix A can be pre-stored in the L3 cache, while matrix B is stored in the remote memory pool. When executing the current computational task, the computing core can read data from the L3 cache and the remote memory pool respectively to read matrix A and matrix B into the L1 / L2 cache, and then perform the computation for the current task based on matrix A and matrix B. Furthermore, the computing core can integrate multiple AI computing units, each specifically designed for the AI ​​task and containing multiple matrix multiplication units to improve computational parallelism and efficiency. The specific structural design of the computing core can be determined according to the actual situation and is not limited here. As one possible approach, the AI ​​accelerator is equipped with a high-speed input / output interface (IO interface), through which the AI ​​accelerator connects to the remote memory pool.

[0086] L1 / L2 caches store data related to the computational task currently being executed by the compute core and provide this data directly to the compute core. Additionally, for data unrelated to the current computational task, the L1 / L2 cache can offload this data to the L3 cache or destroy it. For example, the L1 cache can be a private cache located within each streaming multiprocessor (SM), while the L2 cache is a cache shared by all streaming multiprocessors (SMs). As one possible implementation, the L1 / L2 cache can be implemented using static random access memory (SRAM).

[0087] L3 cache is primarily used to provide a larger cache than L1 / L2 cache. L3 cache has less bandwidth than L1 / L2 cache, but its capacity is greater. Furthermore, like L1 / L2 cache, L3 cache does not store state data; instead, it serves as an extension of L1 / L2 cache to reduce the overhead of computing cores accessing remote storage. As one possible implementation, L3 cache can be implemented using HBM or other types of DRAM. Alternatively, L3 cache can be derived from main memory modifications within AI accelerators. For example, L3 cache size is typically 10GB or larger.

[0088] From another perspective, both L1 / L2 and L3 caches are used to provide cached data for the computing core. Their main function is to cache frequently used data, ensuring that during the current computational task, the core only interacts with the higher-bandwidth L1 / L2 and L3 caches, rather than frequently interacting with the remote memory pool. Therefore, the L3 cache can be viewed as a larger cache layer than the L1 / L2 cache, but with a slightly slower data transfer rate, located between the computing core and the remote memory pool. The L3 cache can be used to store data unrelated to the current operator execution or data that the L1 / L2 cache cannot temporarily store.

[0089] In this system, due to the clustering effect during neural network training, AI accelerators are prone to frequent failures. In this embodiment, by setting up a remote memory pool to store state data, only operator-level rollback can be performed when an AI accelerator fails. Specifically, when an AI accelerator fails, the data stored on each storage medium on the AI ​​accelerator is lost. Figure 2In previous systems, storing state data generated during neural network model training in the GPU memory of the AI ​​accelerator necessitated using the previously described checkpoint-based fault recovery method for training progress recovery, resulting in significant overhead. However, in this system, the L3 cache is used as a cache, and various state data are not stored in the L3 cache but rather in a remote memory pool less prone to failure. Therefore, in this system, when the AI ​​accelerator fails and the data in the L3 cache disappears, the L3 cache only needs to reread the data related to the current computation from the remote memory pool to continue computation. Thus, this system can significantly reduce the overhead of fault recovery and checkpoint rollback recalculation. Furthermore, the lower failure rate of the remote memory pool in this system reduces the probability of state data loss, thereby reducing the frequency of checkpoint saving and consequently lowering the overhead of checkpoint saving during training. In summary, by using this system, the overhead of fault recovery, checkpoint saving, and checkpoint rollback recalculation in the training cluster can be reduced, allowing more time to be devoted to training and improving the availability of the training cluster.

[0090] It's important to note that during model training, AI accelerator failures and GPU memory failures are the most common types of faults. Typically, the training cluster stores state data in GPU memory because its bandwidth is greater than that of the remote memory pool. In other words, the time it takes to transfer the same data from GPU memory to the L1 / L2 cache is less than the time it takes to transfer it from the remote memory pool to the L1 / L2 cache. Therefore, even if GPU memory capacity is insufficient, the remote memory pool will only be used as an extension of GPU memory to store data that cannot be stored in GPU memory; this approach still exists. Figure 2 The problem lies in the fact that the state data of the neural network model in this system can be stored in a remote memory pool, while the data stored in the L1 / L2 cache and L3 cache is the data required for the computation of the computing core, thus improving the availability of the training cluster.

[0091] On the other hand, in this system, because a remote memory pool is used to store state data, the capacity requirement for the GPU memory on the AI ​​accelerator is reduced. Currently, most GPU memory on AI accelerators is HBM memory. HBM uses a 3D stacking process; the larger the HBM memory, the more layers it has. As the number of HBM layers increases, its failure rate also increases. In this system, because a remote memory pool is used to store state data, the HBM memory capacity requirement is reduced. This allows the use of smaller capacity HBM with fewer stacking layers, thereby reducing the probability of GPU memory failure in the training cluster and improving the system's availability during training.

[0092] Within the aforementioned system framework, a technical concept for fault resolution when an AI accelerator malfunctions is proposed, which involves only rolling back a small number of operators when the AI ​​accelerator fails. For example... Figure 5 As shown, Figure 5 This is a timing diagram illustrating task recovery during fault resolution. For ease of description, the AI ​​accelerator experiencing this fault will be referred to as the First AI Accelerator. Figure 4 As shown, the technical concept includes a task localization module and a task execution module. The task localization module is used to determine the first task interrupted due to the failure when the first AI accelerator recovers from a failure, based on the second state data related to the first AI accelerator in the remote memory pool. When an AI accelerator fails, all its stored data is lost. In this case, a separate task localization module can be provided, which can determine the task interrupted by the first AI accelerator during the failure based on the second state data received from the remote memory pool. As one possible implementation, the task localization module can obtain the most recently stored second state data of the first AI accelerator in the remote memory pool. Subsequently, the subsequent computation task can be determined based on the most recently stored second state data. For example, if the most recently stored data is found to be inter-layer activation data between layer 1 and layer 2, it can be inferred that the next computation will be the inter-layer activation data between layer 2 and layer 3. Alternatively, when determining the first task, the subsequent first task can also be determined based on the most recently stored second state data in the remote memory pool, combined with the computation graph or the executed code.

[0093] The task execution module is used to retrieve first state data and second state data related to the first task from a remote memory pool, and execute the first task based on the first state data and second state data related to the first task. For example, the first task can be a computation of a network layer or a computation of an operator.

[0094] Furthermore, when the AI ​​accelerator malfunctions, some second-state data may not yet be saved to the remote memory pool. For example, the activation data for calculating the first task might remain in the AI ​​accelerator's cache after the activation data is obtained, only being unloaded to the remote memory pool after the first task is completed. Therefore, when the AI ​​accelerator malfunctions, the activation data related to the first task will also disappear. Consequently, when calculating the first task, it will be found that the relevant activation data is not stored in the second-state data. In the current scenario, the activation data related to the first task can be referred to as the missing second-state data.

[0095] If the second state data for executing the first task is found to be missing, the first task cannot be executed temporarily. The missing second state data can be calculated first. For example, if the second state data for executing the first task is missing, a second task can be determined based on the missing second state data. This second task is used to retrieve the missing second state data. Subsequently, the first and second state data corresponding to the second task are retrieved, and the second task is executed based on these corresponding first and second state data to retrieve the missing second state data. Thus, the second state data required to execute the first task is prepared. Based on the missing second state data and the first and second state data related to the first task retrieved from the remote memory pool, the first task is executed. Therefore, the faulty AI accelerator can now operate normally.

[0096] Within the aforementioned system framework, to reduce the impact of remote memory pool bandwidth limitations on the computational performance of the AI ​​accelerator, such as... Figure 6 As shown, this application also proposes a technical concept for performing matrix multiplication operations to maximize overall computational efficiency by efficiently coordinating the allocation of computational tasks with the remote memory pool during neural network model training within the aforementioned system framework. A matrix multiplication operator is a commonly used operator in large model operations; one matrix multiplication operator performs one matrix multiplication operation. Typically, matrix multiplication calculations constitute a large portion of the overall computation in large models. For example, the calculation process of a matrix multiplication operator can be represented as Y = X * W. Here, X is the activation, W is the weight used to update the weights of activation X, and Y is the calculation result.

[0097] Under this technical concept, during the computation of the matrix multiplication operator, the activation X and a portion of the weight W associated with the matrix multiplication operator are first read into the L1 / L2 cache. Subsequently, the computation core performs computation on the activation X and a portion of the weight W in the L1 / L2 cache, and during this computation, any unread portions of the weight W are read into the L1 / L2 cache. After the matrix multiplication operator computation is complete, the activation X is unloaded to a remote memory pool when performing computations for other operators. This technical concept optimizes the computation of the matrix multiplication operator, ensuring that the computational performance of the computation core is not hampered by the bandwidth of the remote memory pool.

[0098] Next, we will introduce this technical concept in detail from three aspects: the data preparation process, the operator execution and data reading and writing process, and the process of transferring activated data to the remote memory pool.

[0099] 1) Data preparation process

[0100] The data preparation process involves preparing for matrix multiplication operations. This process primarily involves reading the activation X and weights W into the L1 / L2 caches. The activation X is typically the output of the previous operator, so it is usually stored in the L3 cache; while the weights W are the weight data obtained after the previous step update, and are therefore usually stored in a remote memory pool. Therefore, the data preparation process mainly involves reading the activation X from the L3 cache and the weights W from the remote memory pool into the L1 / L2 caches.

[0101] For example, such as Figure 7 As shown, during data preparation, activation X can be read from the L3 cache into the L1 / L2 cache, and a portion (or "sub-weights") of the weight W can be read from the remote memory pool into the L1 / L2 cache. In other words, during data preparation, the weight W can be read from the remote memory pool into the L1 / L2 cache in multiple steps instead of all at once. For example, sub-weights can be submatrices with the same number of rows and columns as the weight W, or they can be divided using methods such as row partitioning or simultaneous row and column partitioning. The specific method depends on the actual situation and is not limited here. In addition, when reading sub-weights from the remote memory pool into the L1 / L2 cache, the sub-weights to be read can be directly read from the remote memory pool into the L1 / L2 cache; or the sub-weights to be read can be first read from the remote memory pool into the L3 cache, and then the sub-weights to be read can be read from the L3 cache into the L1 / L2 cache. The specific choice depends on the actual situation and is not limited here.

[0102] For example, such as Figure 7 As shown. Assuming the weight W is an M*N matrix, it can be viewed as consisting of three non-overlapping sub-weights (W1, W2, and W3), and the combination of these sub-weights is equivalent to the weight W. Each sub-weight is assumed to have the same number of rows as the weight W and one-third the number of columns. Therefore, during data preparation, one or more sub-weights can be selected for reading. Figure 7 During the data preparation process, you can choose to read the activation X and the sub-weight W1, that is, read the activation X from the L3 cache into the L1 / L2 cache, and read the sub-weight W1 directly from the remote memory pool into the L1 / L2 cache.

[0103] After the data preparation process is completed, the matrix multiplication operator is calculated.

[0104] 2) Operator execution and data read / write process

[0105] During the calculation process, such as Figure 7As shown, matrix multiplication is performed on the activations and sub-weights stored in the L1 / L2 cache during the data preparation process to obtain sub-results (or "partial results") related to each sub-weight. While calculating the activations and sub-weights, at least one of the sub-weights that has not yet been read is read from the remote memory pool into the L1 / L2 cache. Then, calculations are performed based on the activation X in the L1 / L2 cache and at least one of the unread sub-weights to obtain a sub-result related to at least one of the unread sub-weights. This process is repeated until all sub-results corresponding to all sub-weights are obtained. Finally, the calculated result Y is obtained based on all the sub-results corresponding to all sub-weights.

[0106] To better understand this part, combine Figure 8 The calculation of activation X and weight W is described in detail.

[0107] like Figure 8 As shown in (A), the mathematical relationship between the calculated result Y, the activation X, and the weight W can be expressed as: Y = X * W. From Figure 8 As shown in (B), the weight W can be viewed as a combination of multiple different sub-weights. When the activation X is multiplied by different sub-weights, sub-results related to different sub-weights can be obtained. Figure 8 As shown in (B), these sub-results are all parts of the calculated result Y. The sub-results related to each sub-weight can be accumulated to obtain the calculated result Y.

[0108] As can be seen from the above, as long as a portion of the weight W is read into the L1 / L2 cache during data preparation, the computation core can perform a matrix multiplication of that portion of the weight with the activation X. Furthermore, when performing the matrix multiplication on this portion of the weight, the remaining portion of the weight W can be read. For example, such as... Figure 7 As shown, when the computational core performs matrix multiplication, it first reads the activation X and sub-weight W1 from the L1 / L2 cache, then performs a matrix multiplication on the activation X and sub-weight W1 to obtain the sub-result Y1. This sub-result Y1 is related to the sub-weights. Furthermore, when calculating the sub-weight W1, at least one of the weights that have not yet been read is read. Figure 7In this process, neither sub-weights W2 nor W3 are read into the L1 / L2 cache, so only one needs to be read. Subsequently, the computation core performs a matrix multiplication between the sub-weight W2 in the L1 / L2 cache and the activation X to obtain the sub-result Y2 related to sub-weight W2. During this process, sub-weight W3 is read into the L1 / L2 cache. Of course, to save L1 / L2 cache capacity, the previously obtained sub-result Y1 can also be unloaded from the L1 / L2 cache to the L3 cache. After obtaining the sub-results related to each sub-weight, the sub-results related to each sub-weight are accumulated to obtain the computation result Y. Thus, the time for calculating the activation X and sub-weight W1 masks the time for reading sub-weight W2 from the remote memory pool. This masking refers to the effective hiding of the latency in loading data from the slower remote memory pool into the L1 / L2 cache during the computation process through the scheduling of data reading and computation operations; that is, the observable total computation time does not significantly increase due to the read latency. Similarly, the time it takes to calculate activation X and sub-weight W2 also masks the time it takes for sub-weight W3 to be read from the remote memory pool.

[0109] Furthermore, in training large models, the number of tokens in a batch is generally large, resulting in a large activation value (X). Therefore, the time spent performing matrix multiplication on activation X is also relatively long. This can be used to mask the time spent reading sub-weights from a remote memory pool. Specifically, if activation X is an M*K matrix and the sub-weights are M*N matrices, the computational cost of their product is divided into multiplication and addition. The multiplication cost is MNK, and the addition cost is MN(K-1). Therefore, the total computational cost is approximately 2MNK. In large model training scenarios, activation X is generally a high-dimensional matrix, meaning a matrix with a large M value. Therefore, the total computational cost of multiplying activation X and sub-weights is generally large. Consequently, the computational core takes a long time to calculate activation X and sub-weights, which can mask the time spent reading sub-weights from a remote memory pool.

[0110] 3) Activate the process of transferring data to the remote memory pool.

[0111] The calculation result Y obtained by updating the weights using activation X may be used in subsequent operator calculations, so the calculation result Y can be selected to be unloaded into the L3 cache. After obtaining the sub-results of each sub-weight, the activation X used in this matrix multiplication operator is generally only used during backpropagation. Therefore, activation X is temporarily unused data in subsequent calculations, and can be unloaded into the remote memory pool. For example, after performing a matrix multiplication calculation, the calculation result Y can be unloaded from the L1 / L2 cache to the L3 cache, and the activation X can be unloaded from the L1 / L2 cache to the remote memory pool. Of course, this process of unloading activation X can wait until the next operator calculation.

[0112] In this embodiment, when calculating the matrix multiplication operator, the unread sub-weights are read from the remote memory pool when calculating the matrix multiplication of activation X and sub-weights. This reduces the negative impact of the bandwidth of the remote memory pool on the data calculation of the computing core, so that the computing speed of the computing core is not dragged down by the bandwidth of the remote memory pool. This allows the computing core and the remote memory pool to cooperate efficiently and maximize the overall computing efficiency.

[0113] The above-mentioned technical concept for performing calculations on matrix multiplication operators can be implemented in both single-operator mode and graph mode.

[0114] In other embodiments, this application also proposes a technical concept for calculating matrix multiplication operators in a graph model with operator granularity, such as... Figure 9 As shown, this technical concept is also calculated based on the aforementioned neural network training system. In cases where the graph pattern does not use the aforementioned matrix multiplication operator for computation, this technical concept can be used to compute the matrix multiplication operator. For example, in... Figure 9 In this paper, the technical concept for calculating matrix multiplication operators in a graph model with operator granularity includes three parts: data preparation, operator execution and data read / write, and the transfer of activation data to a remote memory pool. The technical concept will then be described in detail, following these three parts.

[0115] 1) Data preparation process

[0116] During data preparation in graph mode, the weights related to the matrix multiplication operator to be computed can be read from the remote memory pool into the L3 cache while the previous operators of the matrix multiplication operator to be computed are being computed. The previous operators of the matrix multiplication operator to be computed refer to the operators that are located before the matrix multiplication operator to be computed in the computation order. That is, the computation kernel performs the computation of the previous operators before executing the matrix multiplication operator to be computed.

[0117] 2) Operator execution and data read / write process

[0118] In response to a computation instruction for a matrix multiplication operator to be computed, the weights and activations associated with the operator are read from the L3 cache into the L1 / L2 cache. Subsequently, the computation core performs the matrix multiplication operation based on the weights and activations in the L1 / L2 cache. While computing the matrix multiplication operator, the weights associated with the next matrix multiplication operator can be read from the remote memory pool into the L3 cache.

[0119] 3) Activate the process of transferring data to the remote memory pool.

[0120] This section can be found in the aforementioned technical concept for performing calculations on matrix multiplication operators, and will not be repeated here.

[0121] Alternatively, in graph mode, the granularity can be set to network layers. In this case, during data preparation, weights related to matrix multiplication operators in the network layer to be computed can be read from the remote memory pool into the L3 cache. The network layer to be computed can be the next network layer immediately adjacent to the currently computed network layer. During data preparation, weights related to the network layer to be computed can be read from the remote memory pool into the L3 cache during the computation of the previous network layer. Subsequently, activations and weights related to the network layer to be computed can be read from the L3 cache into the L1 / L2 cache. When computing the network layer to be computed, weights related to matrix multiplication operators in the next network layer can be read from the remote memory pool into the L3 cache. Activations related to matrix multiplication operators in the network layer to be computed can be unloaded to the remote memory pool during the computation of the next network layer. In graph mode with network layer granularity, most operators in a single network layer, except for matrix multiplication operators, do not involve reading large amounts of data from the remote memory pool. In other words, the data required for most operators in a network layer is already stored in the L1 / L2 cache, so the overhead of data retrieval is minimal and can be ignored. Therefore, during the computation of these operators, only data transfer occurs between the L1 / L2 and L3 caches, allowing the L3 cache to continuously retrieve data from the remote memory pool. Thus, in graph mode, by retrieving weights related to matrix multiplication operators in the next network layer while performing computations in the current network layer, the negative impact of remote memory pool bandwidth on data computation in the computation core can be reduced.

[0122] To better understand the technical concept of calculating matrix multiplication operators in a graph model with a granularity of network layers, the following section combines... Figure 10 and Figure 11 Let's illustrate with examples.

[0123] Figure 10 This is a schematic diagram of a computational graph. Figure 10 In this process, activation XA first undergoes calculation in layer 1, and then layer 2 is calculated. Figure 10In this context, layer1 contains multiple operators, and calculations within layer1 are performed on these operators. For example, layer1 may include matrix multiplication, softmax, and the Attn operator. Figure 10 In the process, activation XA sequentially passes through the matrix multiplication operator, the Attn operator, and the softmax operator, resulting in the output XB of this network layer. The output XB is then used for layer 2 computation. For example, when the computation core is performing activation XA and layer 1 computation, it can read the weights required for operator computation in layer 2 from remote storage to the near-end / L3 cache.

[0124] like Figure 11 As shown, the data preparation process refers to the matrix multiplication operator to be computed. In Figure 11 In the process, the data preparation involves reading the activations (XA) and weights (WA) related to the matrix multiplication operator calculations in layer 1 from other storage locations into the L1 / L2 cache. Figure 11 In layer 1, there is only one matrix multiplication operator, and the data associated with this operator includes the activation XA and the weight WA. The data preparation process involves reading the activation XA from the L3 cache into the L1 / L2 cache, and reading the weight WA from the remote memory pool into the L1 / L2 cache. The process of reading the weight WA from the remote memory pool into the L1 / L2 cache has been described previously and will not be repeated here. Thus, the data preparation process for the matrix multiplication operator in layer 1 is complete, and the computation of layer 1 is then performed.

[0125] exist Figure 11 In the process, when calculating layer 1, the matrix multiplication of activation XA and weight WA is performed first, followed by the calculation of other operators. Figure 11 During the computation of layer 1, the weights WB related to the matrix multiplication operator in layer 2 can be read into the L3 cache. In other words, during the computation of operators in layer 1, the weights WB can be read from the remote memory pool into the L3 cache. Similarly, during the computation of operators in layer 2, the weights related to the matrix multiplication operator in layer 3 can also be read from the remote memory pool into the L3 cache.

[0126] Additionally, after the matrix multiplication operation in layer 1, the temporarily unused active XA can be unloaded from the L1 / L2 cache to a remote memory pool. The process for calculating layer 2 is similar to that of layer 1, and will not be elaborated upon here.

[0127] The following describes the specific implementation process of the above technical concept.

[0128] For example, Figure 12 This diagram illustrates a flowchart of a neural network model training method provided in an embodiment of this application. It is understood that this method can be executed by any device, apparatus, platform, or cluster of devices with computing and processing capabilities. For example, this method can be executed by a neural network model training apparatus, which can be implemented by software and / or hardware, and can be, but is not limited to, configured in an electronic device or server; typically, it can be configured on a server. Figure 12 As shown, the neural network model training method may include the following steps:

[0129] S1101: During the training process of the current training step, obtain the sample feature vector and the first state data. The sample feature vector is obtained based on the training samples related to the current training step, and the first state data is the state data obtained from the remote memory pool and the previous training step. The current training step is the training step that follows the previous training step in the training process of the neural network model.

[0130] In this embodiment, as Figure 13 As shown, the training process of a neural network can be divided into multiple consecutive training steps. Within the current training step, there are forward propagation and backward propagation. During forward propagation, the AI ​​accelerator reads the first state data obtained from the previous training step from a remote memory pool, such as the weights of the neural network model updated in the previous training step. Additionally, the AI ​​accelerator also acquires sample feature vectors. In a training step, a certain number of training samples are used to train the neural network model. The sample feature vectors here are feature vectors obtained based on the training samples, used to obtain activation data between layers during forward propagation. For example, various methods can be chosen to vectorize the training samples to obtain sample feature vectors. For instance, features can be extracted from the training samples using a pre-trained model (such as BERT, ResNet, etc.). The process of vectorizing training samples can be completed on any device. After obtaining the sample feature vectors, these vectors are sent to the AI ​​accelerator for training. The sample feature vectors here can be complete vectors obtained from the training samples or a portion of a complete vector, depending on the specific situation; no limitation is made here.

[0131] S1102: Train the current training step based on the sample feature vector and the first state data.

[0132] In this embodiment, the sample feature vector is mainly used for forward propagation training in the current training step. During forward propagation, the sample feature vector is processed by the network layers to obtain inter-layer activation data. This activation data is used to calculate the loss and gradient during backpropagation and to update the weights of the neural network model. During the training of the neural network model, the sample feature vector and the first state data are used for training in the current training step.

[0133] S1103: Unload the second state data into the remote memory pool, wherein the second state data is state data obtained based on the current training step, and at least a portion of the second state data is used for training the next training step, which is a training step that follows the current training step in the training process of the neural network model.

[0134] In this embodiment, the state data includes one or more of weights, gradients, activations, or optimizers. Different types of second-state data, such as activations and weights, are obtained during the training process of the current training step. Throughout the training process, this second-state data is not stored permanently in the AI ​​accelerator, but rather in a remote memory pool.

[0135] In this way, by storing state data in a remote memory pool, only operator-level rollback can be performed when the AI ​​accelerator fails. Figure 14 As shown, when the AI ​​accelerator malfunctions, the data related to the current task can be reread from the remote memory pool. This allows more time to be devoted to training, improving the availability of the training cluster.

[0136] For example, when a faulty AI accelerator (or "first AI accelerator") recovers, first and second state data related to the first task can be retrieved from a remote memory pool. The first task is the task whose execution was interrupted due to the fault, determined based on the second state data related to the first AI accelerator in the remote memory pool. The first task is then executed based on the first and second state data related to the first task. It should be noted that recovery of the faulty AI accelerator refers to the disappearance of the fault from the perspective of the training cluster. For example, the faulty AI accelerator can be recovered through repair, restart, or replacement. Furthermore, during the recovery process, it can be determined whether the retrieved second state data related to the first task is missing. If the second state data for executing the first task is missing, the first and second state data corresponding to the second task are retrieved from the remote memory pool. The second task is determined based on the missing second state data. The second task is executed based on the first and second state data corresponding to the second task to obtain the missing second state data. The first task is then executed based on the missing second state data and the first and second state data related to the first task. The techniques used in the fault recovery process here have already been explained in the aforementioned technical concept for troubleshooting when the AI ​​accelerator fails, and will not be repeated here.

[0137] Building upon the aforementioned neural network model training method, to minimize the impact of the second-state data's read speed from the remote memory pool on the AI ​​accelerator's computational efficiency, this method can further optimize matrix multiplication calculations. In this case, S1102 further includes: obtaining the first activation based on the sample feature vector. When calculating the first activation, partial weight data from the first-state data is read from the remote memory pool. Here, the first activation refers to the activation data obtained during the forward propagation process of the current training step based on the sample feature vector. Thus, by simultaneously reading the data to be calculated from the remote memory pool while performing related calculations on the first activation, the calculation and read operations are masked, thereby reducing the adverse impact of the remote memory pool's read speed on the AI ​​accelerator's computational speed. Figure 13 As shown, the first activation is generally not used again after the matrix multiplication operator operation is performed, but before backpropagation. Therefore, the first activation can be unloaded to the remote memory pool after the matrix multiplication operator calculation is completed.

[0138] The following sections will explain the computation and reading processes by masking the following scenarios: 1) when the computation of the first activation is performed using the matrix multiplication operator; and 2) when the computation of the first activation is performed in graph mode.

[0139] 1) The computation of the first activation occurs when the matrix multiplication operator is executed.

[0140] When the computation of the first activation involves performing a matrix multiplication operator, the first activation is read from the L3 cache into the L1 / L2 cache, and at least one sub-weight is read from the remote memory pool into the L1 / L2 cache. Subsequently, based on the first activation and the first weight associated with the current matrix multiplication operator, multiple rounds of matrix multiplication are performed to obtain at least one partial result corresponding to a sub-weight in each round of matrix multiplication. The first weight contains multiple sub-weights, and the combination of these sub-weights is equivalent to the first weight. Based on the partial results corresponding to each sub-weight, the computational result is obtained, which refers to the result of the matrix multiplication of the first activation and the first weight. During the current round of matrix multiplication, based on the first activation and the sub-weights, a matrix multiplication is performed to obtain the partial result corresponding to the sub-weights, and any one or more unread sub-weights are read from the remote memory pool.

[0141] In this embodiment, the first activation is data obtained during the forward propagation process, so it can be pre-stored in the L3 cache. When calculating the matrix multiplication operator, the required data is read from the L3 cache into the L1 / L2 cache. Since the execution of a single operator is a real-time computation, at least one sub-weight can be read from the remote memory pool into the L1 / L2 cache. The current matrix multiplication operator is used to calculate the matrix product of the first weight and the first activation. The calculation of the first weight and the first activation can be split into multiple matrix multiplication calculations. Then, when performing a matrix multiplication calculation, the sub-weights related to the next matrix multiplication calculation are read from the remote memory pool. Thus, a partial result is related to a sub-weight, and the obtained partial results are accumulated to obtain the final calculation result. Unread sub-weights refer to sub-weights located in the remote memory pool that have not been read into the AI ​​accelerator cache. If the calculation of the first activation is performed by executing a matrix multiplication operator, the technical concept for implementing matrix multiplication operator calculations described above can be referenced, and will not be repeated here.

[0142] 2) The calculation of the first activation is performed in graph mode.

[0143] When the computation of the first activation is a computation task with matrix multiplication operators, the computation result related to the first weight is obtained based on the first activation and the first weight associated with the current computation task. The current computation task contains at least one matrix multiplication operator, and the computation task is used to represent the smallest execution unit obtained by partitioning the computation graph based on a pre-set granularity. When performing the operation of the current computation task, the weights associated with the matrix multiplication operators in the next computation task are read from the remote memory pool. The next computation task is used to represent a computation task that follows the current computation task and contains matrix multiplication operators, and the weights associated with the matrix multiplication operators in the next computation task are part of the weight data in the first state data.

[0144] In graph mode, a computational task is an execution unit for training a neural network based on a computation graph, and can correspond to one or more operator nodes in the computation graph. In graph mode, the entire computation graph can be divided according to granularity to obtain various computational tasks. A computational task is the smallest execution unit, and the number of operators covered by a computational task is related to the predefined granularity. For example, when the granularity is network layer, the AI ​​accelerator treats the next network layer to be computed as a computational task; when the granularity is operator, the AI ​​accelerator treats the next single operator to be computed as a computational task. This section will then elaborate on the granularity of network layer and operator respectively.

[0145] When the computation task is an operator, the first weight and first activation are read from the L3 cache into the L1 / L2 cache. Then, matrix multiplication is performed based on the first activation and the first weight to obtain the calculation result related to the first weight. During matrix multiplication, the weights related to the next matrix multiplication operator are read from the remote memory pool. For the case where the computation task is an operator, the technical concept of calculating matrix multiplication operators in the graph model with operator granularity described above can be referenced, and will not be elaborated further here.

[0146] When the computation task is a network layer, the first weight is read from the remote memory pool into the L3 cache. The first weight and the first activation in the L3 cache are then read into the L1 / L2 cache. Subsequently, based on the first activation and the first weight associated with the matrix multiplication operator in the current network layer, matrix multiplication is performed to obtain the computation result associated with the first weight. When performing computation on the current network layer, the weights associated with the matrix multiplication operator of the next network layer are read from the remote memory pool, where the next network layer represents the network layer following the current network layer. When the computation task is a graph, the technical concept of computing matrix multiplication operators in the graph mode with network layer granularity can be referred to above, and will not be elaborated here.

[0147] In this way, the computational efficiency of the AI ​​accelerator is improved by masking the computation and reading processes.

[0148] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments described above can be combined according to actual conditions, and the combined solutions are still within the protection scope of this application.

[0149] Next, based on the methods in the above embodiments, a neural network model training device provided in this application will be introduced.

[0150] For example, Figure 15A schematic diagram of the structure of a neural network model training device provided in an embodiment of this application is shown. Figure 15 As shown, the neural network model training device 1600 includes: an acquisition module 1601 and a processing module 1602.

[0151] The acquisition module 1601 is used to acquire sample feature vectors and first state data during the training process of the current training step. The sample feature vectors are obtained based on training samples related to the current training step, and the first state data are state data obtained from the remote memory pool and the previous training step. The current training step is the training step that follows the previous training step in the training process of the neural network model.

[0152] The processing module 1602 is used to train the current training step based on the sample feature vector and the first state data; and to unload the second state data to the remote memory pool, wherein the second state data is state data obtained based on the current training step, and at least a portion of the second state data is used for training the next training step, which is a training step that follows the current training step in the training process of the neural network model.

[0153] In one possible implementation, the processing module 1602 is further configured to: obtain a first activation based on the sample feature vector and the first state data during the training of the current training step; and read a portion of the weight data from the first state data from the remote memory pool when calculating the first activation.

[0154] In one possible implementation, the first state data includes a first weight. The processing module 1602 is further configured to read partial weight data from the first state data from a remote memory pool when calculating the first activation. During this process: when the calculation of the first activation is performed by executing a matrix multiplication operator, multiple rounds of matrix multiplication are performed based on the first activation and the first weight associated with the current matrix multiplication operator, so that a partial result corresponding to the sub-weight is obtained in any round of matrix multiplication. The first weight includes multiple sub-weights, and the combination of each sub-weight is equivalent to the first weight. Based on the partial results corresponding to each sub-weight, a calculation result is obtained, which refers to the result of matrix multiplication of the first activation and the first weight. Specifically, during the current round of matrix multiplication, matrix multiplication is performed based on the first activation and the sub-weights to obtain a partial result corresponding to the sub-weights, and at least one of the unread sub-weights is read from the remote memory pool. At least one of the unread sub-weights is partial weight data in the first state data.

[0155] In one possible implementation, the first state data includes a first weight. The processing module 1602 is further configured to read a portion of the weight data from the first state data from a remote memory pool when calculating the first activation. During this process: when the calculation of the first activation is a computation task with a matrix multiplication operator, the calculation result related to the first weight is obtained based on the first activation and the first weight associated with the current computation task. The current computation task contains at least one matrix multiplication operator, and the computation task is used to represent the smallest execution unit obtained by partitioning the computation graph based on a pre-set granularity. When performing the operation of the current computation task, the weights associated with the matrix multiplication operator in the next computation task are read from the remote memory pool. The next computation task is used to represent a computation task located after the current computation task and containing a matrix multiplication operator. The weights associated with the matrix multiplication operator in the next computation task are a portion of the weight data in the first state data.

[0156] In one possible implementation, the AI ​​accelerator includes a cache structure, and the processing module 1602 is further configured to store the first activation in the cache structure of the AI ​​accelerator after the first activation is obtained, until the calculation of the first activation is completed, and then unload the first activation from the cache structure to a remote memory pool.

[0157] In one possible implementation, the neural network model includes a large model.

[0158] In one possible implementation, the state data includes one or more of the following: weights, gradients, activations, or optimizers.

[0159] In some embodiments, Figure 15 Both the acquisition module 1601 and the processing module 1602 shown can be implemented in software or in hardware. For example, the implementation of the acquisition module 1601 will be described below. Similarly, the implementation of the processing module 1602 can refer to the implementation of the acquisition module 1601.

[0160] As an example of a software functional unit, module 1601 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 1601 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0161] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0162] As an example of a hardware functional unit, the acquisition module 1601 may include at least one computing device, such as a server. Alternatively, the acquisition module 1601 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0163] The multiple computing devices included in the acquisition module 1601 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 1601 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 1601 can be distributed in the same VPC or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0164] It should be noted that, in other embodiments, the acquisition module 1601 can be used to execute any step in the neural network model training method described in the above embodiments, and the processing module 1602 can also be used to execute any step in the neural network model training method described in the above embodiments. Furthermore, the steps implemented by the acquisition module 1601 and the processing module 1602 can also be specified as needed, and different steps in the neural network model training method described in the above embodiments can be implemented by the acquisition module 1601 and the processing module 1602 respectively. Figure 14 The neural network model training device 700 shown has all the functions of this device.

[0165] This application also provides a computing device 1700. For example... Figure 16 As shown, the computing device 1700 includes a bus 1702, a processor 1704, a memory 1706, and a communication interface 1708. The processor 1704, the memory 1706, and the communication interface 1708 communicate with each other via the bus 1702. The computing device 1700 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1700.

[0166] The 1702 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 16 The bus 1704 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1704 may include a path for transmitting information between various components of the computing device 1700 (e.g., memory 1706, processor 1704, communication interface 1708).

[0167] Processor 1704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0168] The memory 1706 may include volatile memory, such as random access memory (RAM). The processor 1704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0169] The memory 1706 stores executable program code, and the processor 1704 executes the executable program code to implement the aforementioned functions. Figure 14 The functions of the acquisition module 1601 and processing module 1602 shown are used to implement the neural network model training method described in the above embodiments. That is, the memory 1706 stores instructions for executing the neural network model training method described in the above embodiments.

[0170] Alternatively, the memory 1706 stores executable code, and the processor 1704 executes the executable code to implement the aforementioned functions respectively. Figure 14 The neural network model training apparatus 700 shown in the diagram performs the functions of the neural network model training method described in the above embodiments. That is, the memory 1706 stores instructions for executing the neural network model training method described in the above embodiments.

[0171] The communication interface 1708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 1700 and other devices or communication networks.

[0172] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0173] like Figure 17As shown, the computing device cluster includes at least one computing device 1700. The memory 1706 in one or more computing devices 1700 in the computing device cluster may store the same instructions for executing the neural network model training method described in the above embodiments.

[0174] In some possible implementations, the memory 1706 of one or more computing devices 1700 in the computing device cluster may also store partial instructions for executing the neural network model training method described in the above embodiments. In other words, a combination of one or more computing devices 1700 can jointly execute instructions for executing the neural network model training method described in the above embodiments.

[0175] It should be noted that the memory 1706 in different computing devices 1700 within the computing device cluster can store different instructions, each used to execute the aforementioned instructions. Figure 14 This illustrates some of the functions of the neural network model training device 700. Specifically, the instructions stored in the memory 1706 of different computing devices 1700 can implement the functions of one or more modules in the acquisition module 1601 and processing module 1602.

[0176] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 18 One possible implementation is shown. For example... Figure 18 As shown, the two computing devices 1700A and 1700B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this possible implementation, the memory 1706 in computing device 1700A stores instructions for executing the functions of the acquisition module 1601. Simultaneously, the memory 1706 in computing device 1700B stores instructions for executing the functions of the processing module 1602.

[0177] It should be understood that Figure 18 The functions of computing device 1700A shown can also be performed by multiple computing devices 1700. Similarly, the functions of computing device 1700B can also be performed by multiple computing devices 1700.

[0178] This application also provides another computing device cluster. The connection relationships between the computing devices in this computing device cluster can be similarly referred to... Figure 17 and Figure 18 The connection method of the computing device cluster is different in that the memory 1706 of one or more computing devices 1700 in the computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0179] In some possible implementations, the memory 1706 of one or more computing devices 1700 in the computing device cluster may also store partial instructions for executing the aforementioned neural network model training method. In other words, a combination of one or more computing devices 1700 can jointly execute instructions for executing the aforementioned neural network model training method.

[0180] It should be understood that each step of the above method embodiments can be accomplished by hardware logic circuits or software instructions in a processor.

[0181] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0182] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices containing at least one computing device, cause the cluster of computing devices to perform the methods in the above embodiments.

[0183] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0184] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0185] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0186] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0187] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training a neural network model, characterized in that, Applied to AI accelerators, the method includes: During the training process of the current training step, sample feature vectors and first state data are acquired. The sample feature vectors are obtained based on training samples related to the current training step, and the first state data are state data obtained from the remote memory pool and the previous training step. The current training step is a training step that follows the previous training step in the training process of the neural network model. The current training step is trained based on the sample feature vector and the first state data; The second state data is unloaded into the remote memory pool, wherein the second state data is state data obtained based on the current training step, and at least a portion of the second state data is used for training the next training step, which is a training step that follows the current training step in the training process of the neural network model.

2. The method according to claim 1, characterized in that, The training of the current training step based on the sample feature vector and the first state data includes: Based on the sample feature vector, a first activation is obtained, wherein the first activation refers to the activation data obtained during the forward propagation of the sample feature vector in the current training step. When calculating the first activation, a portion of the weight data in the first state data is read from the remote memory pool.

3. The method according to claim 2, characterized in that, The first state data includes a first weight, and the step of reading a portion of the weight data from the remote memory pool when calculating the first activation includes: When the calculation of the first activation is performed during the execution of the matrix multiplication operator, multiple rounds of matrix multiplication are performed based on the first activation and the first weight associated with the current matrix multiplication operator, so that in any round of matrix multiplication, a partial result corresponding to the sub-weight is obtained. The first weight contains multiple sub-weights, and the combination of each sub-weight is equivalent to the first weight. Based on the partial results corresponding to each of the sub-weights, the calculation result is obtained, which refers to the result of matrix multiplication between the first activation and the first weight; Specifically, during the current round of matrix multiplication calculation, matrix multiplication is performed based on the first activation and sub-weights to obtain a partial result corresponding to the sub-weights. At least one of the unread sub-weights is read from the remote memory pool, where the unread sub-weights are partial weight data in the first state data.

4. The method according to claim 2, characterized in that, The first state data includes a first weight, and the step of reading a portion of the weight data from the remote memory pool when calculating the first activation includes: When the computation of the first activation is to perform a computation task with a matrix multiplication operator, the computation result related to the first weight is obtained based on the first activation and the first weight associated with the current computation task. The current computation task contains at least one matrix multiplication operator. The computation task is used to characterize the smallest execution unit obtained by partitioning the computation graph based on a pre-set granularity. When performing the current computation task, weights related to the matrix multiplication operator in the next computation task are read from the remote memory pool, wherein the next computation task is used to characterize a computation task that is located after the current computation task and contains a matrix multiplication operator, and the weights related to the matrix multiplication operator in the next computation task are part of the weight data in the first state data.

5. The method according to any one of claims 2-4, characterized in that, The AI ​​accelerator includes a cache structure, and the method further includes: After the first activation is obtained, the first activation is stored in the cache structure of the AI ​​accelerator until the calculation of the first activation is completed, at which point the first activation is unloaded from the cache structure to the remote memory pool.

6. The method according to any one of claims 1-4, characterized in that, The method further includes: When the AI ​​accelerator recovers from the fault, it retrieves first state data and second state data related to the first task from the remote memory pool, wherein the first task is a task whose execution was interrupted due to the fault, as determined by the second state data related to the AI ​​accelerator in the remote memory pool. The first task is executed based on the first state data and the second state data related to the first task.

7. The method according to claim 6, characterized in that, The method further includes: If the second state data for executing the first task is missing, the first state data and second state data corresponding to the second task are retrieved from the remote memory pool, wherein the second task is determined based on the missing second state data; Based on the first state data and the second state data corresponding to the second task, execute the second task to obtain the missing second state data; The first task is executed based on the missing second state data and the first and second state data related to the first task.

8. The method according to any one of claims 1-7, characterized in that, The neural network model includes large models.

9. The method according to any one of claims 1-4, characterized in that, The state data includes one or more of the following: weights, gradients, activations, or optimizers.

10. A neural network model training device, characterized in that, include: Acquisition module and processing module The acquisition module is used to acquire sample feature vectors and first state data during the training process of the current training step. The sample feature vectors are obtained based on training samples related to the current training step, and the first state data is state data from the remote memory pool obtained from the previous training step. The current training step is a training step that follows the previous training step in the training process of the neural network model. The processing module is used to train the current training step based on the sample feature vector and the first state data; and to unload the second state data into the remote memory pool, wherein the second state data is state data obtained based on the current training step, and at least a portion of the second state data is used for training the next training step, the next training step being a training step that follows the current training step in the training process of the neural network model.

11. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-9, wherein the cluster of computing devices includes at least one computing device.

13. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-9, wherein the computing device cluster includes at least one computing device.

Images