Evaluating and mitigating soft errors in parallel and distributed training and reasoning of converters

Abstract

An apparatus, method, and storage medium for evaluating and mitigating soft errors in parallel and distributed training and reasoning of a transformer are provided. The apparatus comprises two or more processing units (220) capable of communicating with each other and operating together as a transformer for deep learning. Each processing unit (220) is configured to: perform a matrix multiplication on a first matrix and a first parametric matrix to obtain a second matrix, a first column summation vector being added after a last row of the first matrix, a first row summation vector being added after a last column of the first parametric matrix; performing a full reduction operation on the second matrix obtained by the two or more processing units (220) to obtain a third matrix; and determining whether a soft error has occurred by performing a checksum verification on the third matrix.

Description

Technical Field

[0001] Embodiments described herein relate generally to deep learning techniques, and in particular to apparatus, methods, and storage media for evaluating and mitigating soft errors in parallel and distributed training and inference of transformers. Background Art

[0002] With recent advances in deep learning, large models with billions of parameters have been proposed and demonstrated incredible accuracy. For example, the popular Generative Pre-trained Transformer (GPT)-3 language model proposed by OpenAI contains 175 billion parameters; the powerful Megatron-LM from NVIDIA and Microsoft uses 1 trillion parameters. Training such large models is a daunting task because the training time is extremely long even with thousands of state-of-the-art processing units (e.g., graphics processing units (GPUs), central processing units (CPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc.). In order to smoothly perform training within a reasonable time, the efficiency and effectiveness of training should be improved.

[0003] Model parallelism is one of the classic approaches to address such challenges. Model parallelism means that two or more processing units perform training tasks in parallel, and the layer parameters used for training are split between these processing units. With the model parallelism approach, each processing unit can multiply the input tensor by only a portion of the layer parameters and aggregate the outputs of all processing units to obtain the output tensor.

[0004] However, model parallelism cannot tolerate soft errors. Soft errors usually originate from environmental perturbations (e.g., radiation), voltage changes, material decay, or impurities, etc. Soft errors usually appear as bit flips and are often ignored within integrated circuits (ICs) because they disappear once the power is cycled. Although not as destructive as hard errors, soft errors can still cause serious consequences invisibly. For example, if the most significant bit of a floating point number flips, it will greatly change the value of the number. This will cause problems in the neural network, such as incorrect calculation results or predictions during inference and no decrease in model loss during training.

[0005] While the probability of a soft error for a single component or operation is very low (1e-8 level), the probability of soft errors increases as systems become larger and more distributed. For distributed training models with about 1 trillion parameters, the probability of soft errors cannot be ignored due to the large cluster size, frequent network communications, and memory operations. Previous research by Hoang, LH et al., "FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their FaultTolerance using Clipped Activation", showed that in a single machine scenario, the classification accuracy in AlexNet decreases as the error rate increases. In large-scale scenarios, training and inference of transformers will become worse Summary of the invention

[0006] According to one aspect of the present disclosure, a device is provided. The device includes: two or more processing units, which can communicate with each other and operate together as a transformer for deep learning, wherein each of the two or more processing units is configured to: perform matrix multiplication on a first matrix and a first parameter matrix to obtain a second matrix, a first column sum vector is added after the last row of the first matrix, and a first row sum vector is added after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of the elements in the corresponding column of the first matrix, and each element of the first row sum vector is the sum of the elements in the corresponding row of the first parameter matrix; perform an all-reduce operation on the second matrix obtained by the two or more processing units to obtain a third matrix; and determine whether a soft error occurs by performing a checksum verification on the third matrix.

[0007] According to another aspect of the present disclosure, a method is provided. The method includes: performing matrix multiplication on a first matrix and a first parameter matrix by each of two or more processing units to obtain a second matrix, the two or more processing units operating together as a transformer for deep learning, adding a first column sum vector after the last row of the first matrix, adding a first row sum vector after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of the elements in the corresponding column of the first matrix, and each element of the first row sum vector is the sum of the elements in the corresponding row of the first parameter matrix; performing a full reduction operation on the second matrix obtained by the two or more processing units by each of the two or more processing units to obtain a third matrix; and determining whether a soft error occurs by performing a checksum verification on the third matrix by each of the two or more processing units.

[0008] Another aspect of the present disclosure provides a machine-readable storage medium having instructions stored thereon, which, when executed by a machine, causes the machine to perform the above method.

[0009] Another aspect of the present disclosure provides a computing device, comprising means for implementing the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In the accompanying drawings, which are not necessarily drawn to scale, the same numerals may describe similar components in different views. The same numerals with different letter suffixes may represent different instances of similar components. Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which:

[0011] Figure 1 A schematic diagram showing the principle of a fault-tolerant matrix multiplication based algorithm;

[0012] Figure 2 An overview of a system for model parallelism of transformers according to some embodiments of the present disclosure is shown;

[0013] Figure 3 An example of model parallelism in Megatron-LM is shown;

[0014] Figure 4 Another example of model parallelism in Megatron-LM is shown;

[0015] Figure 5 A flowchart showing a process for evaluating and mitigating soft errors in parallel and distributed training and inference of a transformer according to some embodiments of the present disclosure is shown;

[0016] Figure 6 Shows Figure 5 A flowchart of the process of checking and verifying the third matrix mentioned in;

[0017] Figure 7 is a block diagram illustrating components capable of reading instructions from a machine-readable or computer-readable medium and performing any one or more of the methods discussed herein, according to some example embodiments; and

[0018] Figure 8 is a block diagram of an example processor platform according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] The various aspects of the illustrative embodiments will be described using the terms commonly used by those skilled in the art to convey the essence of the present disclosure to other persons skilled in the art. However, it will be apparent to those skilled in the art that many alternative embodiments may be practiced using the parts of the aspects. For purposes of explanation, specific numbers, materials, and configurations are set forth to provide a comprehensive understanding of the illustrative embodiments. However, it will be apparent to those skilled in the art that alternative embodiments may be practiced without the specific details. In other cases, known features may be omitted or simplified to avoid blurring the illustrative embodiments.

[0020] In addition, various operations will be described as multiple discrete operations in sequence in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be interpreted as implying that these operations are necessarily order-dependent. Specifically, these operations do not need to be performed in the order presented.

[0021] The phrases "in an embodiment," "in one embodiment," and "in some embodiments" are used repeatedly herein. The phrase generally does not refer to the same embodiment; however, it may refer to the same embodiment. The terms "including," "having," and "comprising" are synonymous unless the context indicates otherwise. The phrases "A or B" and "A / B" mean "(A), (B), or (A and B)." As used herein, ordinal numbers such as "first," "second," and "third" are only used to distinguish the items that follow and do not indicate the actual order of these items.

[0022] Several methods have been proposed to detect and correct soft errors.

[0023] For example, error correction codes (ECC) can be used to detect and seamlessly correct errors in random access memory (RAM), but at the expense of reduced speed and higher on-chip errors. Studies have shown that ECC can reduce the probability of having a bit error in 4 gigabytes (GB) of RAM to about one in six billion. However, this may not be enough for large-scale distributed scenarios with terabytes (TB) of memory, about 1 trillion parameters, and weeks of training time. In addition, ECC helps less when the error occurs outside the memory.

[0024] Another approach to address soft errors is algorithm-based fault-tolerant computing. The original algorithm-based fault-tolerant matrix multiplication introduces partial sums for checking. It exploits the properties of linear algebra. Figure 1 A schematic diagram showing the principle of an algorithm-based fault-tolerant matrix multiplication is shown (see, for example, Fernando Fernandes dos Santos et al., "Evaluation and Mitigation of Soft-Errors in Neural Network-Based Object Detection in Three GPU Architectures," 47th IEEE / IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), pp. 169-176 (2017), the entire contents of which are incorporated herein for all purposes). As shown, matrix multiplication is performed on matrix A and matrix B to obtain matrix M. In order to detect and correct soft errors, a row checksum vector A is added after the last row of matrix A. c , add a column checksum vector B after the last column of matrix B r . Row checksum vector A c Each element of is the sum of the elements in the corresponding column of matrix A, so the row checksum vector A c It can also be called the column sum vector. Similarly, the column checksum vector B r Each element of is the sum of the elements in the corresponding row of matrix B, so the column checksum vector B r It can also be called the row sum vector. Added row checksum vector A c The matrix A with the added column checksum vector B r The matrix multiplication of the matrix B generates the added row checksum vector M c and column checksum vector M r The matrix M.

[0025] To check whether a soft error occurs, generate a row vector M c ', each element of which is the sum of all elements of the corresponding column in the matrix M, and generates a column vector M r ', each element of which is the sum of all elements of the corresponding row in the matrix M. Check the row vector M element by element c 'Is it equal to the row checksum vector M c (i.e., whether the difference between them is zero) and the column vector M r 'Is it equal to the column checksum vector M r (i.e., whether the difference between them is zero). If the row vector M c 'Equal to the row checksum vector Mc And the column vector M r 'Equal to the column checksum vector M r , it can be determined that no soft error has occurred, otherwise it can be determined that at least one soft error has occurred.

[0026] Specifically, when at least one soft error occurs, the location where the soft error occurs can be determined. r 'The i-th element (denoted as M r '[i]) is not equal to the column checksum vector M r The i-th element (denoted as M r [i]), it can be determined that at least one soft error occurs in the i-th row of the matrix M; or similarly, when the row vector M c 'The jth element (denoted as M c '[j]) is not equal to the row checksum vector M c The jth element (denoted as M c [j]), it can be determined that at least one soft error occurs in the j-th column of the matrix M, where i and j are positive integers. Thus, the error element M[i,j] can be determined. In this case, the error element M[i,j] can be quickly corrected using a row checksum vector or a column checksum vector by the following equation (1):

[0027] M correct [i, j] = M[i, j] - (M r '[i]- M r [i])= M[i, j]-( M c '[j]- M c [j])(1)

[0028] However, the original algorithm can only protect matrix multiplication operations on a single machine. When errors occur in communication, memory storage, or transmission (which is common in distributed training and inference scenarios), the original algorithm loses its protection ability.

[0029] In a model parallelism scenario, communication, memory storage, or transmission between different processing units may occur frequently, and soft errors may occur in these processes. It is crucial to detect and resolve potential soft errors as early as possible. Embodiments of the present application provide a device, method, and storage medium for evaluating and mitigating soft errors in parallel and distributed training and reasoning of a converter, and achieve optimal fault tolerance and performance of parallel and distributed training and reasoning of a converter. The device, method, and storage medium provided herein for evaluating and mitigating soft errors in parallel and distributed training and reasoning of a converter can detect and resolve potential soft errors during communication, memory storage, or transmission between different nodes. No hardware ECC or parity support is required. In addition, it provides users with the flexibility to selectively enable fault tolerance for specific layers, thereby achieving the best balance between fault tolerance and performance.

[0030] Figure 2 An overview of a system 200 for model parallelism of transformers is shown according to some embodiments of the present disclosure.

[0031] The number, capabilities, and / or capacity of the elements of system 200 may vary, depending on whether system 200 is used as a fixed computing device (e.g., a server computer in a data center, a workstation, a desktop computer, etc.) or a mobile computing device (e.g., a smart phone, a tablet computing device, a laptop computer, a gaming console, an Internet of Things (IoT) device, etc.). In various embodiments, system 200 may include one or more components of a data center, a desktop computer, a workstation, a laptop computer, a smart phone, a tablet computer, a digital camera, a smart appliance, a smart home hub, a network device, and / or any other device / system that processes data.

[0032] As a simplified example, the system 200 includes (one or more) input / output (I / O) interfaces 210 and two or more processing units 220 .

[0033] The I / O interface(s) 210 may be configured to receive input data for deep learning operations and / or configuration data for the transformer from a memory / storage device or an input device, and to output results of the deep learning operations to a memory / storage device or an output device.

[0034] In some embodiments, one or more memory / storage devices may be included in the system 200 or may be coupled to the system 200. The memory / storage device may include main memory, disk storage, or any suitable combination thereof. The memory / storage device may include, but is not limited to, any type of volatile or non-volatile memory, such as, for example, dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage, etc.

[0035] In some embodiments, various I / O devices may be present within the system 200 or connected to the system 200 via (one or more) I / O interfaces 210. Input devices may include any physical or virtual means for accepting input, including but not limited to one or more physical or virtual buttons (e.g., a reset button), a physical keyboard, a keypad, a mouse, a touchpad, a touch screen, a microphone, a scanner, a headset, etc. Output devices may be included to display information or otherwise communicate information, such as sensor readings, (one or more) actuator positions, or other similar information. Data and / or graphics may be displayed on output devices. Output devices may include any number and / or combination of audio or visual displays, including but not limited to one or more simple visual outputs / indicators (e.g., binary status indicators (e.g., light emitting diodes (LEDs)) and multi-character visual outputs, or more complex outputs such as a display device or touch screen (e.g., a liquid crystal display (LCD), LED display, quantum dot display, projector, etc.), where output of characters, graphics, multimedia objects, etc. is generated or produced from the operation of system 200. Output devices may also include speakers and / or other audio emitting devices, (one or more) printers, etc. Additionally or alternatively, (one or more) sensors may be used as input devices (e.g., image capture devices, motion capture devices, etc.) and one or more actuators may be used as output devices (e.g., actuators that provide tactile feedback, etc.).

[0036] The configuration data of the converter can be used to configure two or more processing units 220 to operate together as a converter. The two or more processing units 220 may include any type of component with processing or computing capabilities, such as a GPU and a CPU (collectively referred to as an "XPU"), an FPGA, an ASIC, etc.

[0037] Generally speaking, one of the two or more processing units 220 can act as a master processing unit / node to implement the configuration of the two or more processing units 220 to implement the functions of the transformer. For example, the master processing unit / node can split the layer parameters of the transformer between the two or more processing units 120.

[0038] After configuration, two or more processing units 220 may perform operations on input operators in parallel according to the configuration.

[0039] For ease of description only, Megatron-LM is used as an example to introduce the operation of the converter, but it should not be interpreted as limiting the principles of the present disclosure. Figure 3 and Figure 4 Reference is made to “Efficient large-scale language model training on GPU clusters using megatron-LM” by Deepak Narayanan et al., SC’21: International Conference on High Performance Computing, Networking, Storage and Analytics (St. Louis, MO, USA), pp. 58:1–58:15 (November 14-19, 2021), for an illustration of model parallelism in Megatron-LM, the entire contents of which are incorporated herein for all purposes, e.g., as a basis for the inventive concepts of the present application.

[0040] Figure 3 An example of model parallelism in Megatron-LM is shown. In this example, model parallelism is applied to a multilayer perceptron (MLP) layer. The MLP layer is a simple layer consisting of two consecutive matrix multiplications Y=GeLU(XA) and Z=Dropout(YA). GeLU(·) is the activation function applied to each matrix element. The activation function can be approximated as GeLU(x)≈xσ(1.702x), where σ(·) is the normal distribution function. It is used in GPT-3, BERT, and most other transformers. Dropout(·) is a normalization technique during training that discards some elements of the matrix at a given probability.

[0041] For example, these two multiplications can be performed on two processing units as follows:

[0042] 3.1) The layer parameters are received as configuration data, which can be represented as parameter matrices A and B. The parameter matrices A and B are split, for example, by the main processing unit into two segments A=[A1, A2],

[0043] The first processing unit has layer parameters A1, B1 and the second processing unit has layer parameters A2, B2.

[0044] 3.2) The first processing unit and the second processing unit receive the input tensor X and perform partial matrix multiplication in parallel. The first processing unit calculates XA1 and then calculates Y1B1; the second processing unit calculates XA2 and then calculates Y2B2. In this process, the two processing units complete their work independently, and this step does not involve communication.

[0045] 3.3) Apply the all-reduce operation g so that each processing unit has the same output tensor Z = Z1 + Z2. The tensor Z can be fed as the input tensor X to the next layer.

[0046] The dropout operation shown in the figure is for the purpose of alleviating overfitting, which is not discussed in this disclosure.

[0047] The above three steps are repeated to calculate the continuous MLP. This scheme can be used for all types of training and inference of the transformer.

[0048] Figure 4 Another example of model parallelism in Megatron-LM is shown. In this example, the inherent parallelism in the multi-head attention operation is used to divide the self-attention block. The key (K), query (Q) and value (V) matrices can be divided in a column-parallel manner. Then, the output linear layer can directly operate on the divided output (weight matrix divided across rows) of the attention operation. The method splits the matrix multiplication into MLP and self-attention blocks across processing units (e.g., GPUs), while only requiring two full reduction operations (g operators) in the forward pass and two full reductions (f operators) in the backward pass. f and g are conjugate. f is the identity operator in the forward pass and is the full reduction in the backward pass, while g is the opposite.

[0049] and Figure 3 Similar to, for example, Figure 4 The multiplication can be performed on two processing units as follows:

[0050] 4.1) Layer parameters are received as configuration data, which can be represented as parameter matrices (K, Q, V) and B. The parameter matrices (K, Q, V) and B are split into two segments (K = [K1, K2], Q = [Q1, Q2], V = [V1, V2]) by the main processing unit, for example. The first processing unit has layer parameters (K1, Q1, V1), B1, and the second processing unit has layer parameters (K2, Q2, V2), B2.

[0051] 4.2) The first processing unit and the second processing unit receive the input tensor X and perform partial matrix multiplication in parallel. The first processing unit calculates (XK1, XQ1, XV1) and the second processing unit calculates (XK2, XQ2, XV2). During this process, the two processing units complete their work independently and this step does not involve communication.

[0052] 4.3) In parallel, the first processing unit performs the matrix multiplication softmax[(XQ1)(XK1) T ] and further multiplies the result with XV1 to obtain Y1. The second processing unit performs matrix multiplication

[0053] softmax[(XQ2)(XK2) T ] and further multiply the result with XV2 to obtain Y2. In this process, the two processing units complete their work independently, and this step does not involve communication.

[0054] 4.4) The first processing unit and the second processing unit calculate Y1B1 and Y2B2 in parallel. Apply the full reduction operation g so that each processing unit has the same output tensor Z=Z1+Z2. The tensor Z can be fed as the input tensor X to the next layer.

[0055] The dropout operation shown in the figure is for the purpose of alleviating overfitting, which is not discussed in this disclosure.

[0056] For illustrative purposes only, the methods presented in this paper for evaluating and mitigating soft errors in parallel and distributed training and inference of transformers will be combined with Figure 4 It should be noted that the principles of the present application can be applied to any model parallelism scenario where communication, memory storage or transmission between different processing units may occur. Figure 4 The details of the operation of the Megatron-LM shown should not be used to limit the scope of protection of the present application.

[0057] Steps 4.2) and 4.3) both involve matrix multiplication. As mentioned above, in steps 4.2) and 4.3), the two processing units work independently and do not involve communication. Therefore, the reference Figure 1 The original algorithm-based matrix multiplication fault tolerance described is applied to these steps to detect and correct soft errors.

[0058] For example, in step 4.2), a row checksum vector can be added after the last row of the input tensor X, and a row checksum vector can be added after each parameter matrix K. i , Q i 、V i Add a column checksum vector after the last column of . Use reference Figure 1 The described algorithm performs checksum verification on each output of the matrix multiplications XK1, XQ1, XV1, XK2, XQ2, XV2, which will not be repeated here.

[0059] Step 4.4) involves matrix multiplications Y1B1 and Y2B2 performed on the two processing units, respectively, and a full reduction operation g performed on each processing unit to generate the same output tensor Z = Z1 + Z2. In order for each processing unit to perform the full reduction operation g, the two processing units must communicate with each other. Soft errors may occur during the communication process. As mentioned above, reference Figure 1 The original algorithm-based matrix multiplication fault tolerance described cannot detect soft errors that occur during the communication process.

[0060] The following method is proposed to check and correct such soft errors. The first processing unit can add a first column sum vector (i.e., a first row checksum column sum vector) after the last row of the matrix Y1 and add a first row sum vector (i.e., a first column checksum vector) after the last column of the parameter matrix B1, and perform matrix multiplication on the matrix Y1 to which the first column sum vector is added and the parameter matrix B1 to which the first row sum vector is added to obtain Z1 (which can be referred to as Z1') to which two checksum vectors are added. Each element of the first column sum vector is the sum of the elements in the corresponding column of the matrix Y1, and each element of the first row sum vector is the sum of the elements in the corresponding row of the parameter matrix B1. In parallel with the operation of the first processing unit, the second processing unit may add a second column sum vector (i.e., a second row checksum column sum vector) after the last row of the matrix Y2 and a second row sum vector (i.e., a second column checksum vector) after the last column of the parameter matrix B2, and perform matrix multiplication on the matrix Y2 to which the second column sum vector is added and the parameter matrix B2 to which the second row sum vector is added, so as to obtain Z2 (which may be referred to as Z2') to which two checksum vectors are added. Each element of the second column sum vector is the sum of the elements in the corresponding column of the matrix Y2, and each element of the second row sum vector is the sum of the elements in the corresponding row of the parameter matrix B2.

[0061] After that, the first processing unit and the second processing unit communicate with each other so that each processing unit knows the matrices Z1' and Z2'. Each processing unit performs a full reduction operation on the matrices Z1' and Z2' to obtain an output tensor Z'=Z1'+Z2', and determines whether a soft error occurs by performing a checksum verification on the tensor Z'.

[0062] Since Z'=Z1'+Z2', the sum of the row or column checksum vectors of Z1' and Z2' directly forms the row or column checksum vector of Z'.

[0063] With reference Figure 1Similar to the checksum verification described, the checksum verification of the tensor Z' may include: a first verification, i.e., whether a first difference between the elements in the last row of the tensor Z' and the sum of the elements in the corresponding column of the tensor Z is zero; and a second verification, i.e., whether a second difference between the elements in the last column of the tensor Z' and the sum of the elements in the corresponding row of the tensor Z is zero.

[0064] Each processing unit performs two verifications on tensor Z', and if both the first verification and the second verification pass, that is, the first difference between any element in the last row of tensor Z' and the sum of the elements in the corresponding column of tensor Z is zero, and the second difference between any element in the last column of tensor Z' and the sum of the elements in the corresponding row of tensor Z is also zero, it is determined that no soft error has occurred.

[0065] If either or both of the first verification and the second verification fail, the processing unit determines that at least one soft error has occurred. Specifically, if the first difference between the elements in the last column of the i-th row of the tensor Z' and the sum of the elements in the i-th row of the tensor Z is not zero, it can be determined that at least one soft error has occurred in the i-th row of the tensor Z' (or Z), or similarly, if the second difference between the elements in the last row of the j-th column of the tensor Z' and the sum of the elements in the j-th column of the tensor Z is not zero, it can be determined that at least one soft error has occurred in the j-th column of the tensor Z' (or Z), where i and j are positive integers. Thus, the erroneous element Z[i,j] can be determined. In this case, the erroneous element M[i,j] can be quickly corrected by adding the first difference or the second difference to the erroneous element Z[i,j].

[0066] For Figure 4 In the specific example described, the matrix Y1 and the matrix Y2 are obtained from the previous step, in which the checksum verification has been performed, so the matrix Y1 and the matrix Y2 themselves can have row and column checksum vectors included therein. In this case, the first processing unit will not add the first column sum vector after the last row of the matrix Y1, but will omit the last column of the matrix obtained from the previous step to obtain the matrix Y1 to which the first column sum vector is added; the second processing unit will not add the second column sum vector after the last row of the matrix Y2, but will omit the last column of the matrix obtained from the previous step to obtain the matrix Y2 to which the second column sum vector is added.

[0067] Checksum verification of the tensor Z' can provide additional protection for communication, memory storage, and transmission for parallel and distributed training and inference of transformers. The method provided in this paper can not only protect the computation performed on a single machine, but also protect the network transmission and memory copy due to the all-reduce operation, thereby protecting the entire process of a single layer of processing of the transformer.

[0068] Figure 5FIG. 5 is a flowchart of a process 500 for evaluating and mitigating soft errors in parallel and distributed training and inference of a transformer according to some embodiments of the present disclosure. The process 500 may be performed, for example, by Figure 2 The system 200 may be implemented by one or more processors of any computing device. Figure 8 An example of a processor is shown.

[0069] like Figure 5 As shown, the process 500 includes, at block 510, performing matrix multiplication on the first matrix and the first parameter matrix by each of the two or more processing units to obtain a second matrix, wherein a first column sum vector is added after the last row of the first matrix, and a first row sum vector is added after the last column of the first parameter matrix. Each element of the first column sum vector is the sum of the elements in the corresponding column of the first matrix, and each element of the first row sum vector is the sum of the elements in the corresponding row of the first parameter matrix.

[0070] Process 500 includes, at block 520 , performing, by each of the two or more processing units, a full-reduce operation on the second matrix obtained by the two or more processing units to obtain a third matrix.

[0071] The process 500 includes, at block 530, determining by each of the two or more processing units whether a soft error occurs by performing a checksum verification on the third matrix. The checksum verification on the third matrix may include: a first verification, i.e., whether a first difference between an element in a last row of the third matrix and a sum of elements in a corresponding column of the third matrix excluding elements in the last row is zero; a second verification, i.e., whether a second difference between an element in a last column of the third matrix and a sum of elements in a corresponding row of the third matrix excluding elements in the last column is zero.

[0072] Figure 6 Shows Figure 5 Flow chart of the process 600 for checking and verifying the third matrix mentioned in . The process 600 can be, for example, Figure 2 The system 200 may be implemented by one or more processors of any computing device. Figure 8 An example of a processor is shown.

[0073] Process 600 may include, at block 610, determining whether a first difference between the i-th element in the last column of the third matrix and the sum of the elements in the corresponding row of the third matrix except the i-th element in the last column is zero. i is a positive integer and is not greater than the number of rows of the third matrix. If yes, process 600 may loop block 610 to check the next element in the last column of the third matrix until all elements in the last column of the third matrix have been checked. If no, process 600 may continue to block 630 to determine whether at least one soft error has occurred in the i-th row of the third matrix.

[0074] Process 600 may include determining at block 620 whether a second difference between the j-th element in the last row of the third matrix and the sum of the elements in the corresponding column of the third matrix except the j-th element in the last row is zero. j is a positive integer and is not greater than the number of columns of the third matrix. If yes, process 600 may loop block 620 to check the next element in the last row of the third matrix until all elements in the last row of the third matrix have been checked. If no, process 600 may continue to block 640 to determine whether at least one soft error has occurred in the j-th column of the third matrix.

[0075] After determining at least one soft error occurs in the i-th row of the third matrix at block 630 and determining at least one soft error occurs in the j-th column of the third matrix at block 640, process 600 may continue to block 650 to find one or more erroneous elements in the third matrix.

[0076] After all elements in the last row and last column of the third matrix have been checked and the first difference and the second difference are all zero, process 600 may determine that no error has occurred.

[0077] It should be noted that blocks 610 and 630 and blocks 620 and 640 may be executed in parallel or sequentially, which is not limited herein.

[0078] More specifically, Figure 5 The process of 500 and Figure 6 The process 600 may be implemented in one or more modules as a set of logic instructions stored in a machine-readable or computer-readable storage medium (e.g., random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc.), in configurable logic (e.g., programmable logic array (PLA), field programmable gate array (FPGA), complex programmable logic device (CPLD)), in fixed-function logic hardware using circuit technology (e.g., application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS), or transistor-transistor logic (TTL) technology), or any combination thereof.

[0079] For example, executing Figure 5 The process of 500 and Figure 6 The computer program code for the operations shown in process 600 may be written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., JAVA, SMALLTALK, C++, etc.) and traditional procedural programming languages ​​(e.g., "C" programming language or similar programming languages). In addition, the logic instructions may include assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data of integrated circuits, state information of personalized electronic circuits and / or other structural components of hardware (e.g., host processor, central processing unit / CPU, microcontroller, etc.).

[0080] Figure 7 is a block diagram illustrating components capable of reading instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and executing any one or more of the methods discussed herein according to some example embodiments. Specifically, Figure 7 A diagram of hardware resources 700 is shown, including one or more processors (or processor cores) 710, one or more memory / storage devices 720, and one or more communication resources 730, each of which may be communicatively coupled via a bus 740. For embodiments utilizing node virtualization (e.g., NFV), a hypervisor 702 may be executed to provide an execution environment for one or more network slices / sub-slices to utilize the hardware resources 700.

[0081] Processor 710 may include, for example, processor 712 and processor 714, which may be, for example, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP) (e.g., a baseband processor), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof.

[0082] The memory / storage device 720 may include main memory, disk storage, or any suitable combination thereof. The memory / storage device 720 may include, but is not limited to, any type of volatile or non-volatile memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state storage, etc.

[0083] The communication resources 730 may include an interconnect or network interface component or other suitable device to communicate with one or more peripheral devices 704 or one or more databases 706 via the network 708. For example, the communication resources 730 may include a wired communication component (e.g., for coupling via a universal serial bus (USB)), a cellular communication component, an NFC component, Components (e.g. Low energy consumption), components and other communication components.

[0084] The instructions 750 may include software, programs, applications, applet, application programs, or other executable code for causing at least any one of the processors 710 to perform one or more of the methods discussed herein. The instructions 750 may reside in whole or in part within at least one of the processors 710 (e.g., within a cache of the processor), within the memory / storage device 720, or any suitable combination thereof. In addition, any portion of the instructions 750 may be transferred to the hardware resources 700 from any combination of the peripheral device 704 or the database 706. Therefore, the memory of the processor 710, the memory / storage device 720, the peripheral device 704, and the database 706 are examples of computer-readable and machine-readable media.

[0085] Figure 8 800 is a block diagram of an example processor platform according to some embodiments of the present disclosure. The processor platform 800 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cellular phone, a smart phone, a tablet computer such as an iPad, etc.). TM ), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, headphones or other wearable device, or any other type of computing device.

[0086] The processor platform 800 of the illustrated example includes a processor 812. The processor 812 of the illustrated example is hardware. For example, the processor 812 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired series or manufacturer. The hardware processor can be a semiconductor-based (e.g., silicon-based) device. In some embodiments, the processor implements one or more of the above methods or processes.

[0087] The processor 812 of the illustrated example includes a local memory 813 (e.g., a cache). The processor 812 of the illustrated example communicates with a main memory including a volatile memory 814 and a non-volatile memory 816 via a bus 818. The volatile memory 814 may be comprised of a synchronous dynamic random access memory (SDRAM), a dynamic random access memory (DRAM), Dynamic Random Access Memory The main memory 814 and 816 may be implemented by a memory controller.

[0088] The processor platform 800 of the illustrated example also includes an interface circuit 820. The interface circuit 820 may be implemented by any type of interface standard, for example, an Ethernet interface, a universal serial bus (USB), interface, a near field communication (NFC) interface, and / or a PCI Express interface.

[0089] In the example shown, one or more input devices 822 are connected to the interface circuit 820. The input device(s) 822 allow a user to enter data and / or commands into the processor 812. The input device(s) may be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touch screen, a trackpad, a trackball, and / or a voice recognition system.

[0090] One or more output devices 824 are also connected to the interface circuit 820 of the illustrated example. The output device 824 can be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-situ switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. Therefore, the interface circuit 820 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0091] The interface circuitry 820 of the illustrated example also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate the exchange of data with external machines (e.g., any type of computing device) via the network 826. Communications may occur via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, and the like.

[0092] For example, interface circuitry 820 may include a training data set input via input device(s) 822 or retrieved from network 826 .

[0093] The processor platform 800 of the illustrated example also includes one or more mass storage devices 828 for storing software and / or data. Examples of such mass storage devices 828 include floppy disk drives, hard disk drives, optical disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

[0094] The machine-executable instructions 832 may be stored in the mass storage device 828, the volatile memory 814, the non-volatile memory 816, and / or a removable, non-transitory computer-readable storage medium (eg, a CD or DVD).

[0095] The following paragraphs describe examples of various embodiments.

[0096] Example 1 includes a device, comprising: two or more processing units, which are capable of communicating with each other and operating together as a transformer for deep learning, wherein each of the two or more processing units is configured to: perform matrix multiplication on a first matrix and a first parameter matrix to obtain a second matrix, add a first column sum vector after the last row of the first matrix, and add a first row sum vector after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of elements in a corresponding column of the first matrix, and each element of the first row sum vector is the sum of elements in a corresponding row of the first parameter matrix; perform a full reduction operation on the second matrix obtained by the two or more processing units to obtain a third matrix; and determine whether a soft error has occurred by performing checksum verification on the third matrix.

[0097] Example 2 includes the device of Example 1, wherein the checksum verification of the third matrix includes: a first verification, i.e., verifying whether a first difference between the elements in the last row of the third matrix and the sum of the elements in the corresponding columns of the third matrix except the elements in the last row is zero; and a second verification, i.e., verifying whether a second difference between the elements in the last column of the third matrix and the sum of the elements in the corresponding rows of the third matrix except the elements in the last column is zero.

[0098] Example 3 includes the apparatus of Example 2, wherein each of the two or more processing units is configured to determine that a soft error does not occur if both the first verification and the second verification pass.

[0099] Example 4 includes the apparatus of Example 2, wherein each of the two or more processing units is configured to determine that at least one soft error has occurred if at least one of the first verification and the second verification fails.

[0100] Example 5 includes the apparatus of Example 4, wherein each of the two or more processing units is configured to: determine that at least one soft error has occurred in the i-th row of the third matrix when a first difference between an element in the last column of the i-th row of the third matrix and the sum of the elements in the i-th row of the third matrix excluding the elements in the last column is not zero, wherein i is a positive integer; and generate a new value of the erroneous element by adding the first difference to the erroneous element in the i-th column of the third matrix.

[0101] Example 6 includes the apparatus of Example 4, wherein each of the two or more processing units is configured to: determine that at least one soft error occurs in the jth column of the third matrix when a second difference between the elements in the last row of the jth column of the third matrix and the sum of the elements in the jth column of the third matrix excluding the elements in the last row is not zero, wherein j is a positive integer; and generate a new value of the erroneous element by adding the second difference to the erroneous element in the jth row of the third matrix.

[0102] Example 7 includes an apparatus of any one of Examples 1-6, wherein each of the two or more processing units is configured to: receive an input tensor; add a second column sum vector after the last row of the input tensor and add a second row sum vector after the last column of the second parameter matrix, wherein each element of the second column sum vector is the sum of elements in a corresponding column of the input tensor, and each element of the second row sum vector is the sum of elements in a corresponding row of the second parameter matrix; perform matrix multiplication on the input tensor to which the second column sum vector is added and the second parameter matrix to which the second row sum vector is added to obtain a fourth matrix; and check whether a soft error has occurred by performing checksum verification on the fourth matrix.

[0103] Example 8 includes the apparatus of Example 7, wherein the first matrix to which the first column of the sum vector is added is obtained by omitting a last column of the fourth matrix from the fourth matrix.

[0104] Example 9 includes the apparatus of Example 7, wherein one of the two or more processing units is a main processing unit and is configured to split layer parameters of a transformer for deep learning between the two or more processing units to generate corresponding two or more first parameter matrices and corresponding two or more second parameter matrices.

[0105] Example 10 includes the apparatus of Example 9, wherein the corresponding two or more second parameter matrices include parameters of a self-attention layer in a transformer for deep learning, and the corresponding two or more first parameter matrices include parameters of a discard layer in a transformer for deep learning.

[0106] Example 11 includes the apparatus of any of Examples 1-10, wherein the two or more processing units include a graphics processing unit (GPU), a central processing unit (CPU), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0107] Example 12 includes a method, comprising: performing matrix multiplication on a first matrix and a first parameter matrix by each of two or more processing units to obtain a second matrix, the two or more processing units operating together as a transformer for deep learning, adding a first column sum vector after the last row of the first matrix, and adding a first row sum vector after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of elements in a corresponding column of the first matrix, and each element of the first row sum vector is the sum of elements in a corresponding row of the first parameter matrix; performing a full reduction operation on the second matrix obtained by the two or more processing units to obtain a third matrix by each of the two or more processing units; and determining whether a soft error occurs by performing a checksum verification on the third matrix by each of the two or more processing units.

[0108] Example 13 includes the method of Example 12, wherein the checking and verifying the third matrix comprises:

[0109] a first verification of verifying whether a first difference between an element in the last row of the third matrix and a sum of elements in a corresponding column of the third matrix excluding the elements in the last row is zero; and

[0110] The second verification is to verify whether a second difference between the elements in the last column of the third matrix and the sum of the elements in the corresponding row of the third matrix except the elements in the last column is zero.

[0111] Example 14 includes the method of Example 13, further comprising determining, by each of the two or more processing units, that no soft error has occurred if both the first verification and the second verification pass.

[0112] Example 15 includes the method of Example 13, further comprising determining, by each of the two or more processing units, that at least one soft error occurs if at least one of the first verification and the second verification fails.

[0113] Example 16 includes the method of Example 15, further including: determining, by each of the two or more processing units, that at least one soft error has occurred in the i-th row of the third matrix when a first difference between an element in the last column of the i-th row of the third matrix and the sum of elements in the i-th row of the third matrix excluding the elements in the last column is not zero, where i is a positive integer; and generating, by each of the two or more processing units, a new value of the erroneous element by adding the first difference to the erroneous element in the i-th column of the third matrix.

[0114] Example 17 includes the method of Example 15, further including: determining, by each of the two or more processing units, that at least one soft error has occurred in the jth column of the third matrix when a second difference between the elements in the last row of the jth column of the third matrix and the sum of the elements in the jth column of the third matrix excluding the elements in the last row is not zero, wherein j is a positive integer; and generating, by each of the two or more processing units, a new value of the erroneous element by adding the second difference to the erroneous element in the jth row of the third matrix.

[0115] Example 18 includes the method of any one of Examples 12-17, further including: receiving an input tensor by each of the two or more processing units; adding a second column summation vector after the last row of the input tensor and a second row summation vector after the last column of the second parameter matrix by each of the two or more processing units, wherein each element of the second column summation vector is the sum of the elements in the corresponding column of the input tensor, and each element of the second row summation vector is the sum of the elements in the corresponding row of the second parameter matrix; performing matrix multiplication on the input tensor to which the second column summation vector is added and the second parameter matrix to which the second row summation vector is added to obtain a fourth matrix by each of the two or more processing units; and checking whether a soft error has occurred by performing checksum verification on the fourth matrix by each of the two or more processing units.

[0116] Example 19 includes the method of Example 18, wherein the first matrix to which the first column of the sum vector is added is obtained by omitting a last column of the fourth matrix from the fourth matrix.

[0117] Example 20 includes the method of Example 18, wherein one of the two or more processing units is a main processing unit, and the method further includes splitting, by the main processing unit, layer parameters of the transformer for deep learning between the two or more processing units to generate corresponding two or more first parameter matrices and corresponding two or more second parameter matrices.

[0118] Example 21 includes the method of Example 20, wherein the corresponding two or more second parameter matrices include parameters of a self-attention layer in a transformer for deep learning, and the corresponding two or more first parameter matrices include parameters of a discard layer in a transformer for deep learning.

[0119] Example 22 includes the method of any of Examples 12-21, wherein the two or more processing units include a graphics processing unit (GPU), a central processing unit (CPU), or a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0120] Example 23 includes a machine-readable storage medium having instructions stored thereon, which, when executed by a machine, cause the machine to perform the method of any one of Examples 11 to 22.

[0121] Example 24 includes a computing device comprising means for performing the method of any of Examples 11-22.

[0122] Example 25 includes an apparatus comprising one or more processors to implement one or more processes as shown and described in the specification.

[0123] Example 26 includes a method comprising one or more processes as shown and described in the specification.

[0124] Example 27 includes a system comprising one or more memories to store computer readable instructions for implementing one or more processes as shown and described in the specification.

[0125] The above description is intended to be illustrative, not restrictive. For example, the above examples (or one or more aspects thereof) may be used in combination with each other. For example, other embodiments may be used by a person of ordinary skill in the art after reading the above description. The abstract is intended to enable the reader to quickly understand the nature of the technical disclosure, and is submitted with the understanding that the abstract will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above specific embodiments, various features may be combined together to simplify the present disclosure. This should not be interpreted as unclaimed public features being necessary for any claim. On the contrary, the subject matter of the invention may be present in less than all the features of a particular disclosed embodiment. Therefore, the following claims are hereby incorporated into the specific embodiments, wherein each claim is independent of a separate embodiment. The scope of the embodiment should be determined with reference to the appended claims and the full scope of equivalents to which these claims are assigned.

[0126] Although certain embodiments have been shown and described herein for illustrative purposes, the embodiments shown and described may be replaced with a variety of alternative and / or equivalent embodiments or implementations intended to achieve the same purpose without departing from the scope of the present disclosure. The present disclosure is intended to cover any modifications or variations of the embodiments discussed herein. Therefore, it is apparent that the embodiments described herein are limited only by the appended claims and their equivalents.

Claims

1. A device comprising: Two or more processing units capable of communicating with each other and operating together as a transformer for deep learning, wherein each of the two or more processing units is configured to: Performing matrix multiplication on a first matrix and a first parameter matrix to obtain a second matrix, wherein a first column sum vector is added after the last row of the first matrix, and a first row sum vector is added after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of elements in a corresponding column of the first matrix, and each element of the first row sum vector is the sum of elements in a corresponding row of the first parameter matrix; performing a full reduction operation on the second matrix obtained by the two or more processing units to obtain a third matrix; and Whether a soft error occurs is determined by performing a checksum verification on the third matrix.

2. The device according to claim 1, wherein: The checking and verifying of the third matrix includes: a first verification, namely, verifying whether a first difference between an element in a last row of the third matrix and a sum of elements in a corresponding column of the third matrix excluding elements in the last row is zero; and The second verification is to verify whether a second difference between the elements in the last column of the third matrix and the sum of the elements in the corresponding row of the third matrix except the elements in the last column is zero.

3. The device according to claim 2, wherein: Each of the two or more processing units is configured to determine that no soft error has occurred if both the first verification and the second verification are passed.

4. The device according to claim 2, wherein: Each of the two or more processing units is configured to determine that at least one soft error has occurred if at least one of the first verification and the second verification fails.

5. The device according to claim 4, wherein: Each of the two or more processing units is configured to: If the first difference between the elements in the last column of the i-th row of the third matrix and the sum of the elements in the i-th row of the third matrix excluding the elements in the last column is not zero, determining that at least one soft error has occurred in the i-th row of the third matrix, where i is a positive integer; and A new value of the erroneous element is generated by adding the first difference to the erroneous element in the i-th column of the third matrix.

6. The device according to claim 4, wherein: Each of the two or more processing units is configured to: If the second difference between the elements in the last row of the j-th column of the third matrix and the sum of the elements in the j-th column of the third matrix excluding the elements in the last row is not zero, determining that at least one soft error has occurred in the j-th column of the third matrix, where j is a positive integer; and A new value of the erroneous element is generated by adding the second difference to the erroneous element in the j-th row of the third matrix.

7. The device according to claim 1, wherein: Each of the two or more processing units is configured to: Receive input tensor; Adding a second column of sum vectors after the last row of the input tensor and adding a second row of sum vectors after the last column of the second parameter matrix, wherein each element of the second column of sum vectors is the sum of elements in a corresponding column of the input tensor, and each element of the second row of sum vectors is the sum of elements in a corresponding row of the second parameter matrix; performing matrix multiplication on the input tensor to which the second column sum vector is added and the second parameter matrix to which the second row sum vector is added to obtain a fourth matrix; and Whether a soft error occurs is checked by performing checksum verification on the fourth matrix.

8. The device according to claim 7, wherein: The first matrix to which the first column sum vector is added is obtained by omitting the last column of the fourth matrix from the fourth matrix.

9. The device according to claim 7, wherein: One of the two or more processing units is a main processing unit and is configured to split the layer parameters of the transformer for deep learning between the two or more processing units to generate corresponding two or more first parameter matrices and corresponding two or more second parameter matrices.

10. The device according to claim 9, wherein: The corresponding two or more second parameter matrices include parameters of a self-attention layer in the transformer for deep learning, and the corresponding two or more first parameter matrices include parameters of a discard layer in the transformer for deep learning.

11. The device according to any one of claims 1 to 10, wherein: The two or more processing units include a graphics processing unit (GPU), a central processing unit (CPU), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

12. A method comprising: Performing matrix multiplication on a first matrix and a first parameter matrix by each of two or more processing units to obtain a second matrix, the two or more processing units operating together as a transformer for deep learning, a first column sum vector added after the last row of the first matrix, a first row sum vector added after the last column of the first parameter matrix, wherein each element of the first column sum vector is the sum of elements in a corresponding column of the first matrix, and each element of the first row sum vector is the sum of elements in a corresponding row of the first parameter matrix; performing, by each of the two or more processing units, a full reduction operation on the second matrix obtained by the two or more processing units to obtain a third matrix; and Whether a soft error occurs is determined by each of the two or more processing units by performing a checksum verification on the third matrix.

13. The method according to claim 12, wherein: The checking and verifying of the third matrix includes: a first verification, namely, verifying whether a first difference between an element in a last row of the third matrix and a sum of elements in a corresponding column of the third matrix excluding elements in the last row is zero; and The second verification is to verify whether a second difference between the elements in the last column of the third matrix and the sum of the elements in the corresponding row of the third matrix except the elements in the last column is zero.

14. The method according to claim 13, further comprising: Each of the two or more processing units determines, when both the first verification and the second verification are passed, that no soft error has occurred.

15. The method according to claim 13, further comprising: At least one soft error occurs, determined by each of the two or more processing units if at least one of the first verification and the second verification fails.

16. The method according to claim 15, further comprising: determining, by each of the two or more processing units, that at least one soft error has occurred in the i-th row of the third matrix when the first difference between the elements in the last column of the i-th row of the third matrix and the sum of the elements in the i-th row of the third matrix excluding the elements in the last column is not zero, wherein i is a positive integer; and A new value of the erroneous element is generated by each of the two or more processing units by adding the first difference to the erroneous element in the i-th column of the third matrix.

17. The method according to claim 15, further comprising: determining, by each of the two or more processing units, that at least one soft error has occurred in the j-th column of the third matrix when the second difference between the elements in the last row of the j-th column of the third matrix and the sum of the elements in the j-th column of the third matrix excluding the elements in the last row is not zero, wherein j is a positive integer; and A new value of the erroneous element is generated by each of the two or more processing units by adding the second difference to the erroneous element in the j-th row of the third matrix.

18. The method according to claim 12, further comprising: receiving, by each of the two or more processing units, an input tensor; Adding, by each of the two or more processing units, a second column sum vector after the last row of the input tensor and a second row sum vector after the last column of the second parameter matrix, wherein each element of the second column sum vector is the sum of elements in a corresponding column of the input tensor, and each element of the second row sum vector is the sum of elements in a corresponding row of the second parameter matrix; performing, by each of the two or more processing units, matrix multiplication on the input tensor to which the second column sum vector is added and the second parameter matrix to which the second row sum vector is added to obtain a fourth matrix; and Whether a soft error occurs is checked by each of the two or more processing units by performing checksum verification on the fourth matrix.

19. The method according to claim 18, wherein: The first matrix to which the first column sum vector is added is obtained by omitting the last column of the fourth matrix from the fourth matrix.

20. The method according to claim 18, wherein: One of the two or more processing units is a main processing unit, and the method further includes: the main processing unit splitting the layer parameters of the transformer for deep learning between the two or more processing units to generate corresponding two or more first parameter matrices and corresponding two or more second parameter matrices.

21. The method according to claim 20, wherein: The corresponding two or more second parameter matrices include parameters of a self-attention layer in the transformer for deep learning, and the corresponding two or more first parameter matrices include parameters of a discard layer in the transformer for deep learning.

22. The method according to any one of claims 12 to 21, wherein: The two or more units include a graphics processing unit (GPU), a central processing unit (CPU), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

23. A machine-readable storage medium having instructions stored thereon, which when executed by a machine cause the machine to perform the method of any one of claims 11 to 22.

24. A computing device comprising means for performing the method of any one of claims 11 to 22.

Images