High-performance parallel implementation method and system based on Transform transplantation of SW processor

Abstract

The invention provides a high-performance parallel implementation method based on Transform transplantation of a SW processor, and relates to the technical field of processor calculation parallel implementation, slave core calculation is performed by using a SW Athread programming method, each independent function of forward propagation and backward propagation of a ViT algorithm is rewritten into a unified function, a calculation intermediate variable in the forward function is simplified, and the calculation efficiency is improved. Transplanting operation of the ViT algorithm is realized; according to the method, an intermediate variable in a forward function calculation process is simplified, a ViT algorithm based on a Transform architecture is successfully transplanted and operated on a domestic SW processor, unnecessary redundant calculation is removed through an optimization algorithm, and the operation efficiency of a ViT model on a Shenwei super computer is remarkably improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of processor computing parallel implementation, and in particular to a high-performance parallel implementation method and system based on the Shenwei processor Transformer transplantation. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] The Sunway processor is a high-performance processor. However, many commonly used application software are often not suitable for the efficient operation of the Sunway processor, so a lot of transplantation and optimization work is required. Successful transplantation is the first step in the optimization work. Only when the application can be successfully compiled and run can the next step of optimization work begin.

[0004] Transformer is a deep learning model for natural language processing (NLP) tasks. Transformer is based on the self-attention mechanism, which enables the model to learn the relationship between different positions in the input sequence. The architecture of Transformer consists of multiple encoders and / or decoders, each of which contains multiple identical layers. At present, the functional framework of the deep learning model problem of the Shenwei cluster still has problems such as immaturity, instability, and relatively single functional modules. Therefore, it is of great significance to transplant and optimize Transformer.

[0005] In recent years, computer vision researchers have also paid attention to the superiority of the Transformer structure and applied Transformer to computer vision tasks. Transformer-based network structures such as ViT, DETR, and MoCoV3 have shined in visual tasks. The ViT model is implemented based on PyTorch. PyTorch is an open source machine learning library. With its powerful functions and ease of use, it has become one of the most popular deep learning frameworks. PyTorch supports a variety of backend hardware. On the domestic Shenwei multi-core CPU platform, researchers have completed the porting and optimization of PyTorch (SWPyTorch). SWPyTorch retains many excellent features of PyTorch, not only providing users with a simple and easy-to-use Python interface, but also providing a relatively complete C++API. Deep learning users and framework developers can customize and extend PyTorch at all levels according to their own needs.

[0006] Among them, the ViT (Vision Transformer) algorithm is a computer vision model based on the Transformer architecture. Its forward propagation and backward propagation processes are similar to those of the traditional Transformer model. Through continuous iteration and optimization, the ViT algorithm has achieved excellent results in computer vision tasks. However, the Shenwei processor and the X86 compiler and architecture are different. Since there is no porting and development of a version that can run on the Shenwei processor, there are certain limitations in the operation of the initial code. In addition, the Shenwei processor is a unique architecture. Based on the existing model, the running speed is relatively slow and there are certain limitations, which cannot improve its performance and scalability on different hardware platforms. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes a high-performance parallel implementation method and system based on the Transformer transplantation of the Shenwei processor. By simplifying the intermediate variable yT, that is, removing unnecessary redundant calculations through the optimization algorithm, the operating efficiency of the visual Transformer (ViT) model on the Shenwei supercomputer is significantly improved.

[0008] According to some embodiments, the present disclosure adopts the following technical solutions:

[0009] A high-performance parallel implementation method based on the Transformer transplantation of the Shenwei processor includes: transplanting the ViT algorithm to the Shenwei processor, including: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm;

[0010] Among them, the unified forward function swptex_all_forward() calls sw_slave_all_forward() in the slave core;

[0011] Unify the backward function swptex_all_backward() and call sw_all_backward() in the slave core.

[0012] According to some embodiments, the present disclosure adopts the following technical solutions:

[0013] A high-performance parallel implementation system based on the Transformer transplantation of the Shenwei processor, including:

[0014] A parallel implementation module is used to transplant the ViT algorithm to the Shenwei processor, including: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm;

[0015] Among them, the unified forward function swptex_all_forward() calls sw_slave_all_forward() in the slave core;

[0016] Unify the backward function swptex_all_backward() and call sw_all_backward() in the slave core.

[0017] According to some embodiments, the present disclosure adopts the following technical solutions:

[0018] A non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, a high-performance parallel implementation method based on the Shenwei processor Transformer transplantation is implemented.

[0019] According to some embodiments, the present disclosure adopts the following technical solutions:

[0020] An electronic device comprises: a processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the high-performance parallel implementation method based on the Shenwei processor Transformer transplantation.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] The present invention discloses a high-performance parallel implementation method based on the transplantation of the Shenwei processor Transformer. The VIT computer vision algorithm is transplanted and optimized using SWPyTorch, and the Shenwei Athread programming method is used to implement slave-core computing to obtain a simple multi-core code. In order to more clearly understand the program call process and whether the data can be reused before and after, the forward and backward separate functions are rewritten into a unified function to improve its performance and scalability on different hardware platforms. By deeply integrating SWPyTorch's optimization tools and rich API support, it can adapt to a wider range of visual task requirements and faster running speed.

[0023] The present invention discloses a high-performance parallel implementation method based on the Transformer transplantation of the Shenwei processor, which simplifies the intermediate variable yT, that is, through the forward propagation function in vit, it removes unnecessary redundant calculations, and significantly improves the operating efficiency of the visual Transformer (ViT) model on the Shenwei supercomputer. This improvement increases the running speed of the ViT model by 29%, thereby greatly speeding up the speed of data processing and analysis, and improving the efficiency of scientific research and engineering calculations. Improve the utilization of computing resources: reduce unnecessary calculations, so that computing resources are used more effectively and energy consumption is reduced. Improve real-time performance: For tasks that require real-time processing, such as video analysis, autonomous driving, etc., the increase in running speed helps to improve the real-time performance of the system and reduce latency. Reduce costs: Increasing the running speed helps to reduce the consumption of computing resources, thereby reducing operating costs.

[0024] The high-performance parallel implementation method based on the Transformer transplantation of the Shenwei processor disclosed in the present invention has successfully transplanted and run the ViT algorithm based on the Transformer architecture on the domestic Shenwei processor, enriching the autonomous and controllable computing vision ecology and strengthening the application capabilities of domestic processors in the field of artificial intelligence. It provides technical support for the promotion of domestic processors in commercial application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure.

[0026] Figure 1 A schematic diagram of the code for running the forward propagation process of an embodiment of the present disclosure;

[0027] Figure 2 A schematic diagram of running code for the back propagation process of an embodiment of the present disclosure;

[0028] Figure 3 Detailed information of the variable yT and a simplified flowchart of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0031] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0032] Example 1

[0033] In one embodiment of the present disclosure, a high-performance parallel implementation method based on the Shenwei processor Transformer transplantation is provided, including:

[0034] Transplanting the ViT algorithm to the Shenwei processor includes: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm;

[0035] Among them, the basic functions include: __local_gemm_rcr(), __local_trans_head(), __local_norm(), __local_gemm_rrr(), and __local_trans_head_back().

[0036] The above basic function will be called in an intermediate function a, and the calling statement will be replaced with the actual implementation. In this way, the user only needs to call the intermediate function a.

[0037] Among them, the unified forward function swptex_all_forward() calls sw_slave_all_forward() in the slave core;

[0038] Unify the backward function swptex_all_backward() and call sw_all_backward() in the slave core.

[0039] swptex_all_forward() is the original forward propagation function, which is a core concept in neural networks. It describes the process of how to get from the input layer through the network layers to the output layer. The main calculation of Shenwei is to be calculated in the slave core, so the original forward function calls the forward function sw_slave_all_forward() implemented under the slave core. This not only ensures the uniformity of the function name called by the user, but also realizes the transplantation of Shenwei. The unified backward function swptex_all_backward() is also called the backpropagation function. It is a key step in the neural network training process. It is used to calculate the gradients of network parameters (such as weights and biases). These gradients are then used to update the parameters of the network to reduce the value of the loss function.

[0040] Furthermore, in the forward function, the s matrices qk_data and v_data are calculated through matrix multiplication of the function swptex_bmm to obtain yT(B, N, S, D), and then yT is transposed to y through the function swptex_merge, and the N and S dimensions are transposed to S and N. Now the calculation results of qk_data and v_data are omitted from the transposition process and directly put to the address corresponding to y in the sw_mm_AB function, which also reduces the number of transmissions of the y=yT operation. After simplifying the variable yT, the time is increased by 29% compared with not simplifying the variable.

[0041] Specifically, matrix multiplication (swptex_bmm): qk_data and v_data are matrix multiplied to obtain the result yT, whose shape is (B, N, S, D). Here B is the batch size, N is the sequence length, S is the number of heads, and D is the feature dimension.

[0042] Transpose operation (swptex_merge): yT undergoes a transpose operation to transpose the N and S dimensions to obtain y, whose shape is (B, S, N, D).

[0043] During the optimization, the transposition operation is omitted, and the result is placed directly at the final required address in the matrix multiplication. Here are the optimization steps: Matrix multiplication directly calculates the final shape (sw_mm_AB): When performing matrix multiplication, the result of the shape (B, S, N, D) is directly calculated instead of calculating yT of the shape (B, N, S, D) first. This means that during the multiplication process, the results of qk_data and v_data are directly placed in the correct position of y, avoiding the additional transposition step.

[0044] Furthermore, the processor is a master-slave core structure. The master core is mainly responsible for logic and storage. The slave core has multiple computing cores, which are mainly responsible for calculations. From the program level, the program retains the original interface unchanged, calls various original interface functions, and the interface function calls the transplanted slave core function to perform calculations.

[0045] From the machine level, the startup function is on the master core. When calculation is needed, data is sent to the slave core for calculation. After the calculation is completed, the slave core sends the data to the master core. The master core continues to call the logic until the next time the slave core needs to calculate.

[0046] As an example, the ViT (Vision Transformer) algorithm is a computer vision model based on the Transformer architecture, and its forward propagation and backward propagation processes are similar to those of the traditional Transformer model. The following is a detailed introduction to the forward propagation and backward propagation of the ViT algorithm:

[0047] Forward Propagation:

[0048] Image Segmentation:

[0049] 1) Split the input image into patches of fixed size, each containing one pixel.

[0050] 2) Convert each block into a token and perform linear projection to have a continuous embedding space.

[0051] Positional encoding:

[0052] 1) Add position encoding to each token to preserve the spatial information of the image.

[0053] 2) The design of positional encoding is similar to word embedding in natural language processing, but it is designed for image patches.

[0054] Transformer Architecture:

[0055] ViT uses a standard Transformer architecture, including a multi-head self-attention mechanism and a feedforward neural network. ViT's Transformer architecture contains multiple encoder layers, each of which contains a self-attention mechanism and a feedforward neural network.

[0056] Forward propagation process:

[0057] 1) The processed image patch is taken as input and fed into the Transformer architecture.

[0058] 2) After calculations of multi-layer self-attention mechanism and feedforward neural network, the final feature representation is obtained.

[0059] 3) Input the feature representation into the classifier for classification or prediction.

[0060] Back Propagation

[0061] 1) Calculate the loss:

[0062] According to the task requirements, calculate the loss function, such as cross entropy loss.

[0063] The loss function reflects the difference between the model's predictions and the actual results.

[0064] Back propagation process:

[0065] 1) Calculate the gradient of the loss function with respect to the model parameters.

[0066] 2) Update the model parameters to reduce the loss.

[0067] 3) Repeat the forward propagation and back propagation process until the model converges or reaches the preset number of iterations.

[0068] 4)Optimizer and Tuning:

[0069] Use various optimizers, such as Adam, to update model parameters.

[0070] 5) Adjust hyperparameters such as learning rate and batch size to improve the performance of the model.

[0071] Example 2

[0072] In one embodiment of the present disclosure, a high-performance parallel implementation system based on the Transformer transplantation of the Shenwei processor is provided, including:

[0073] A parallel implementation module is used to transplant the ViT algorithm to the Shenwei processor, including: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm;

[0074] Among them, the unified forward function swptex_all_forward() calls sw_slave_all_forward() in the slave core;

[0075] Unify the backward function swptex_all_backward() and call sw_all_backward() in the slave core.

[0076] Example 3

[0077] In one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the high-performance parallel implementation method based on the Transformer transplantation of the Shenwei processor is implemented.

[0078] Example 4

[0079] In one embodiment of the present disclosure, an electronic device is provided, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the high-performance parallel implementation method based on the Transformer transplantation of the Shenwei processor.

[0080] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0082] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A high-performance parallel implementation method based on the Shenwei processor Transformer transplantation, characterized in that: Transplanting the ViT algorithm to the Shenwei processor includes: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm; Among them, the forward function swptex_all_forward() is unified to call sw_slave_all_forward() in the slave core; the backward function swptex_all_backward() is unified to call sw_all_backward() in the slave core.

2. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 1 is characterized in that: The forward propagation includes image segmentation and position encoding, where image segmentation includes: dividing the input image into blocks of fixed size, each block contains one pixel, converting each block into a token, and performing linear projection to make it have a continuous embedding space.

3. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 2 is characterized in that: The position encoding includes: adding position encoding to each token to retain the spatial information of the image; ViT uses the Transformer architecture, and the Transformer architecture of ViT includes multiple encoder layers, each layer includes a self-attention mechanism and a feedforward neural network.

4. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 1 is characterized in that: The forward propagation process includes: taking the processed image block as input and inputting it into the Transformer architecture; obtaining the final feature representation through calculations of a multi-layer self-attention mechanism and a feedforward neural network; and inputting the feature representation into the classifier for subsequent output processing.

5. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 1 is characterized in that: The backpropagation process includes: calculating the gradient of the loss function with respect to the model parameters, updating the model parameters, and reducing the loss; repeating the forward propagation and backpropagation processes until the model converges or reaches the preset number of iterations; using the optimizer to update the model parameters and adjust the learning rate and batch size hyperparameters.

6. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 1 is characterized in that: Simplify the calculation of intermediate variables in the forward function, including: In the forward function, the s matrices qk_data and v_data are calculated by matrix multiplication of the function swptex_bmm to obtain yT(B,N,S,D). The calculation results of qk_data and v_data are omitted from the transposition process and directly put to the address corresponding to y in the sw_mm_AB function.

7. The high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as claimed in claim 1 is characterized in that: The omitted transposition process is to transpose yT to y and transpose N and S to S and N through the function swptex_merge.

8. A high-performance parallel implementation system based on the Shenwei processor Transformer transplantation, characterized by: include: A parallel implementation module is used to transplant the ViT algorithm to the Shenwei processor, including: using the Shenwei Athread programming method to perform slave core calculations, rewriting the individual functions of the forward propagation and backward propagation of the ViT algorithm into a unified function, and simplifying the calculation intermediate variables in the forward function to realize the transplantation and operation of the ViT algorithm; Among them, the forward function swptex_all_forward() is unified to call sw_slave_all_forward() in the slave core; the backward function swptex_all_backward() is unified to call sw_all_backward() in the slave core.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as described in any one of claims 1-7 is implemented.

10. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes the high-performance parallel implementation method based on the Shenwei processor Transformer transplantation as described in any one of claims 1-7.

Images