A performance optimization method and device for deep learning algorithm based on GPU

Abstract

The present application provides a performance optimization method and device for deep learning applications based on a GPU. In the process of GPU loading the assembly instructions corresponding to the deep learning application, the first target assembly instruction in the assembly instruction is replaced with the second target assembly instruction. When executing the instruction, only the second target assembly instruction needs to be executed, and the first target assembly instruction is no longer executed. Moreover, after the second target assembly instruction is executed, other assembly instructions after the first target assembly instruction are executed in sequence, so as to change the execution logic of the high-performance function library in the GPU to obtain a new operator. As can be seen from the above process, the scheme is to combine the new operator with the high-performance function library in the GPU, so that the new operator can also achieve higher computing performance. The scheme can not only meet the GPU's support for the new operator, that is, versatility, but also improve the overall performance of the GPU when executing the new operator, that is, high performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular to a performance optimization method and device for deep learning applications based on a graphics processing unit (GPU). Background Art

[0002] In deep learning applications, execution efficiency is becoming an increasingly important focus. However, a common problem with deep learning is that algorithm developers do not understand the actual deployment hardware, which may lead to low efficiency in the deep learning applications they write. Therefore, deep learning compilers have emerged to specifically solve performance issues encountered in the implementation of deep learning applications.

[0003] However, the current deep learning compilers still have room for performance improvement, which is reflected in the following two ways: one is the chip manufacturer school (or hardware compiler, such as can be integrated into GPU chips), chip manufacturers will deeply optimize the calculation of some specific functions (such as convolution), such as assembly-level code rearrangement, etc. to maximize the performance advantages of hardware. The other is the framework school (can be called software compiler, implemented through software framework), which regards deep learning as a specific application of overall optimization, and obtains the optimal execution result by pre-defining a series of rules and continuously trial and error in the specified search space.

[0004] It can be seen that the vendor faction can maximize the performance of deep learning applications, but does not support new operators that appear in deep learning applications. The advantage of the framework faction is that it can support new operators, but its performance is lower than that of the vendor faction. In a deep learning framework, a deep learning model is usually abstracted into a directed acyclic data flow graph composed of some basic operators. An operator usually includes multiple kernel functions, which are composed of operations provided by the computing device architecture or some GPU function libraries.

[0005] It can be seen that how to make deep learning applications have high performance while improving the ability to support new operators is a problem that needs to be solved urgently. Summary of the invention

[0006] In view of this, the purpose of the present invention is to provide a performance optimization method and device for a deep learning algorithm based on a GPU to solve the above technical problems. The technical solution is as follows:

[0007] In a first aspect, the present application provides a method for optimizing the performance of a deep learning application based on a graphics processor, the method comprising:

[0008] When loading the assembly instructions corresponding to the operator to be optimized in the deep learning application into the register of the GPU, replacing the first target assembly instructions in the assembly instructions with the second target assembly instructions, where the first target assembly instructions are the assembly instructions to be optimized included in the operator to be optimized, and the second target assembly instructions are optimized assembly instructions and do not belong to the operator to be optimized;

[0009] After executing the second target assembly instruction, other assembly instructions after the first target assembly instruction are executed.

[0010] In a possible implementation manner of the first aspect, replacing the first target assembly instruction in the assembly instruction with the second target assembly instruction includes:

[0011] Storing the second target assembly instruction in the second storage space, wherein the second target assembly instruction;

[0012] A first jump instruction is stored in a first storage space used for storing the first target assembly instruction, and the first jump instruction is used to jump to the second storage space.

[0013] In another possible implementation manner of the first aspect, after executing the second target assembly instruction, executing the next assembly instruction adjacent to the first target assembly instruction includes:

[0014] storing a second jump instruction in a next available storage space of the second storage space, wherein the second jump instruction is used to jump to a third storage space, wherein the third storage space stores a next assembly instruction of the first target assembly instruction;

[0015] Execute the second jump instruction to jump to the third storage space, and execute the assembly instructions stored in the third storage space.

[0016] In another possible implementation of the first aspect, the first target assembly instruction is a first type of data read instruction, and the second target assembly instruction is a second type of data read instruction, wherein the reading logic of the first type of data read instruction is different from that of the second type of data read instruction.

[0017] In yet another possible implementation of the first aspect, the first target assembly instruction is a first type of data output instruction, the second target assembly instruction is a second type of data output instruction, and the output logic of the first type of data output instruction is different from that of the second type of data output instruction.

[0018] In another possible implementation manner of the first aspect, a process of determining a first storage space corresponding to the first target assembly instruction includes:

[0019] Determine a storage address segment of the assembly instruction corresponding to the operator to be optimized, and find out a storage address offset corresponding to the first target assembly instruction from the storage address segment;

[0020] An address of a first storage space storing the first target assembly instruction is obtained based on the storage address offset.

[0021] In a second aspect, the present application further provides a performance optimization device for deep learning applications based on a graphics processor, the device comprising:

[0022] An instruction replacement module is used to replace a first target assembly instruction in the assembly instruction with a second target assembly instruction when loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU, wherein the first target assembly instruction is the assembly instruction to be optimized included in the operator to be optimized, and the second target assembly instruction is an optimized assembly instruction and does not belong to the operator to be optimized;

[0023] The instruction execution module is used to execute other assembly instructions after the first target assembly instruction after executing the second target assembly instruction.

[0024] In a possible implementation manner of the second aspect, the instruction replacement module includes:

[0025] A first storage submodule is used to store the second target assembly instruction in a second storage space, wherein the second target assembly instruction;

[0026] The second storage submodule is used to store the first jump instruction in a first storage space used to store the first target assembly instruction, and the first jump instruction is used to jump to the second storage space.

[0027] In a third aspect, the present application also provides an electronic device, comprising a processor and a memory, wherein the memory stores instructions, and when the processor runs the instructions stored in the memory, it implements any of the above-mentioned performance optimization methods for deep learning applications based on a graphics processor.

[0028] In a fourth aspect, the present application also provides a computer-readable storage medium having instructions stored thereon, which, when executed by an electronic device, implement the above-mentioned performance optimization method for deep learning applications based on a graphics processor.

[0029] The performance optimization method of the GPU-based deep learning algorithm provided in the present application, in the process of loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU, the first target assembly instruction in the assembly instruction is replaced with the second target assembly instruction. The first target assembly instruction is the assembly instruction to be optimized in the operator to be optimized, and the second target assembly instruction is the optimized assembly instruction, and does not belong to the optimization operator. When the GPU executes the instruction, it no longer executes the first target assembly instruction, but executes the second target assembly instruction, and after executing the second target assembly instruction, it continues to execute other assembly instructions after the first target assembly instruction in sequence. It can be seen from this process that the scheme is that when the assembly instruction of the original operator compiled by the compiler is loaded into the register, the assembly instruction of the new operator can be used to replace some instructions in the original operator, so that the assembly instruction of the deep learning application loaded into the register contains the instruction of the new operator, and the assembly instruction of the original operator that has not been replaced, and the assembly instruction that has not been replaced can still give full play to the hardware performance advantages of the GPU. In other words, the scheme realizes the combination of the new operator with the high-performance function library in the GPU, so that the new operator can also achieve higher computing performance. Therefore, using this solution, the GPU can not only support new operators for deep learning, that is, versatility, but also improve the overall running performance of the GPU when executing new operators, that is, improve the running performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a flowchart of a performance optimization method for a GPU-based deep learning application provided in an embodiment of the present application;

[0032] Figure 2 It is a flowchart of another performance optimization method for deep learning applications based on GPU provided in an embodiment of the present application;

[0033] Figure 3 It is a schematic diagram of storage space before and after assembly instruction replacement provided by an embodiment of the present application;

[0034] Figure 4 This is a block diagram of a performance optimization device for GPU-based deep learning applications provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following is an introduction to the related technologies involved in this application:

[0036] Current deep learning frameworks (such as CNTK, TensorFlow, and Caffe2) usually abstract a deep learning model into a directed acyclic data flow graph (DAG) composed of some basic operators. Then, the lower-level computing engine schedules and executes the kernel functions corresponding to these nodes in a certain topological order to complete the execution of a model.

[0037] Compute Unified Device Architecture (CUDA) is a general-purpose parallel computing architecture launched by graphics card manufacturer NVDIA. This architecture enables GPUs to solve complex computing problems. It includes the CUDA instruction set architecture and the parallel computing engine inside the GPU. Kernel functions are CUDA parallel computing functions, which are composed of operations provided by CUDA or some GPU function libraries (such as cuDNN, cuBLAS, etc.). An Operator usually includes multiple kernel functions.

[0038] When a kernel function is called, the GPU will start many threads to execute the kernel function at the same time, thus achieving parallelization. While these smaller-granular kernel functions provide flexibility, their frequent calls also become an important factor affecting the performance of deep learning frameworks. The performance overhead brought about is mainly reflected in: data flow graph scheduling overhead, kernel function startup overhead, and data transmission overhead between kernel functions.

[0039] One way to solve these performance problems is kernel fusion, which is to fuse the kernel functions corresponding to multiple nodes in a computational graph into one function, so that the entire data flow graph can be completed with only one function call, thereby reducing the overhead caused by platform scheduling and kernel function startup. It greatly improves data transmission efficiency and thus improves overall computing performance.

[0040] Another way is that the GPU chip design manufacturer will deeply optimize the calculation of specific functions (such as convolution), such as re-arranging the code at the assembly level, so as to maximize the performance of the hardware. That is, the code of some specific functions of deep learning has been deeply optimized when the GPU chip leaves the factory, and the code of this part of the function cannot be modified after the GPU chip leaves the factory, so it is called the manufacturer optimization method. The disadvantage of this optimization method is poor generality. For example, for operators such as convolution that have been deeply optimized in hardware chips, their execution logic is fixed. For example, the convolution kernel scans data in a horizontal scanning manner, performs convolution calculations at the same time, and stores the calculation results in a specific location. This method is only high in performance for traditional convolution methods. The new convolution method (such as deformable convolution) uses a specific method to collect data and then fills it into the matrix, and then calculates it according to the traditional convolution calculation method. Since the new convolution method has a different data loading method from the traditional convolution method, it cannot use the high-performance traditional convolution kernel of the manufacturer optimization method, resulting in the low running performance of this new convolution operator.

[0041] The kernel fusion method has high versatility but its operating performance may be lower than that of the manufacturer optimization method, while the manufacturer optimization method has high operating performance but poor versatility. Therefore, how to provide a solution that has both high performance and high versatility has become an urgent problem to be solved.

[0042] In order to solve the above problems, the inventor of the present application provides a performance optimization method based on GPU deep learning algorithm, which expands the execution logic of the existing high-performance function library by means of assembly-level code injection to support new operators in deep learning applications. Specifically, in the process of loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU, the first target assembly instruction in the assembly instruction is replaced by the second target assembly instruction; wherein the second target assembly instruction is the assembly instruction of the new operator, and the first target assembly instruction is the assembly instruction to be optimized in the operator to be optimized (i.e., the original operator). During execution, only the second target assembly instruction needs to be executed, and the first target assembly instruction is no longer executed. Moreover, after the second target assembly instruction is executed, other assembly instructions after the first target assembly instruction are executed in sequence, thereby realizing the combination of the new operator with the high-performance function library in the GPU, so that the new operator can also achieve higher computing performance. Therefore, using this solution, the GPU can not only support new operators for deep learning, that is, versatility, but also improve the overall running performance of the GPU when executing the new operator, that is, improve the running performance. 。

[0043] See also Figure 1 , shows a flow chart of a performance optimization method for a GPU-based deep learning application provided in an embodiment of the present application, the method is applied to a GPU chip, such as Figure 1 As shown, the method may include the following steps:

[0044] S110, in the process of loading the assembly instructions corresponding to the operator to be optimized in the deep learning application into the register of the GPU, replacing the first target assembly instructions in the assembly instructions with the second target assembly instructions.

[0045] GPU, or graphics processing unit, is the processor of a graphics card, a microprocessor that specializes in performing graphics processing tasks. Since GPU provides a multi-core parallel computing infrastructure, it can support parallel computing of large amounts of data and has a higher read and write speed. Deep learning requires a large amount of parallel repetitive computing, so deep learning applications are usually processed using GPUs.

[0046] Deep learning application developers can use high-level programming languages ​​(such as C) to write programs for the CUDA architecture. When the GPU runs a deep learning application, the high-level programming language of the deep learning application needs to be compiled into a machine language, called assembly instructions, and loaded into the GPU's registers for execution.

[0047] The embodiment of the present application modifies the assembly instructions loaded into the register during the process of loading the assembly instructions corresponding to the deep learning application into the register of the GPU, thereby replacing the first target assembly instructions with the second target assembly instructions.

[0048] The first target assembly instruction is the assembly instruction to be optimized in the operator to be optimized in the deep learning application, that is, the first target assembly instruction is the assembly instruction corresponding to the operation in CUDA.

[0049] The second target assembly instruction is an optimized assembly instruction, not an instruction in the operator to be optimized.

[0050] In an exemplary embodiment, by dynamically hijacking the dynamic link library loaded during the CUDA runtime, new assembly instructions can be injected during the process of loading CUDA assembly instructions, thereby replacing the original assembly instructions in CUDA with new assembly instructions, that is, replacing the first target assembly instructions with the second target assembly instructions.

[0051] After the first target assembly instruction is replaced by the second target assembly instruction, the second target assembly instruction and other assembly instructions in CUDA form a new operator.

[0052] In a possible implementation, if the second target assembly instruction occupies a different storage space from the first target assembly instruction, for example, the storage space occupied by the second target assembly instruction is larger than the storage space occupied by the first target assembly instruction, such as the first target assembly instruction is only one instruction, and the second target assembly instruction includes at least two instructions. In this case, it is impossible to load all the second target assembly instructions into the storage space (herein referred to as the target storage space) used to load the first target assembly instruction. Therefore, a jump instruction can be loaded into the target storage space, and the jump instruction is used to jump to the storage space corresponding to the second target assembly instruction. The end of the second target assembly instruction is another jump instruction, and the jump instruction is used to jump to the next storage address of the target storage space.

[0053] In an exemplary embodiment, the first target assembly instruction is a data read instruction in a certain operator, and the second target assembly instruction is a new data read instruction, that is, the reading logic of the first target assembly instruction is different from that of the second target assembly instruction. When the first target assembly instruction is loaded into the register of the GPU, it is dynamically replaced by loading the second target assembly instruction. Finally, data is read in a new data reading method.

[0054] In another exemplary embodiment, the first target assembly instruction is a data output instruction in a certain operator, and the second target assembly instruction is a new data output instruction, that is, the output logic of the first target assembly instruction is different from that of the second target assembly instruction.

[0055] When the first target assembly instruction is loaded into the register of the GPU, it is dynamically replaced by loading the second target assembly instruction, and finally the data is output in a new data output mode.

[0056] S120, after executing the second target assembly instruction, executing other assembly instructions after the first target assembly instruction in sequence.

[0057] When executing instructions, the loaded assembly instructions are executed in sequence, and after the second target assembly instruction is executed, other assembly instructions after the first target assembly instruction are executed in sequence.

[0058] For example, the first target assembly instruction is a data read instruction, and the first target assembly instruction is replaced with a second target assembly instruction. After the second target assembly instruction is executed, that is, after the data to be operated is loaded, the subsequent operation instructions are continued to be executed.

[0059] The performance optimization method of deep learning applications based on GPU provided in this embodiment, in the process of loading the assembly instructions corresponding to the operator to be optimized in the deep learning application into the register of the GPU, the first target assembly instruction in the assembly instruction is replaced with the second target assembly instruction. The first target assembly instruction is the assembly instruction to be optimized in the operator to be optimized, and the second target assembly instruction is the optimized assembly instruction, and does not belong to the optimization operator. When the GPU executes the instruction, it no longer executes the first target assembly instruction, but executes the second target assembly instruction, and after executing the second target assembly instruction, it continues to execute other assembly instructions after the first target assembly instruction in sequence. It can be seen that the scheme is that when the assembly instructions of the original operator compiled by the compiler are loaded into the register, the assembly instructions of the new operator can be used to replace some instructions in the original operator, so that the assembly instructions of the deep learning application loaded into the register contain the instructions of the new operator, and the assembly instructions of the original operator that have not been replaced, and the assembly instructions that have not been replaced can still give full play to the hardware performance advantages of the GPU. In other words, the scheme realizes the combination of the new operator with the high-performance function library in the GPU, so that the new operator can also achieve higher computing performance. Therefore, using this solution, the GPU can not only support new operators for deep learning, that is, versatility, but also improve the overall running performance of the GPU when executing new operators, that is, improve the running performance.

[0060] See also Figure 2 , shows a flowchart of another performance optimization method for a GPU-based deep learning algorithm provided in an embodiment of the present application, the method is applied to a GPU, such as Figure 2 As shown, the method comprises the following steps:

[0061] S210, obtaining a first storage space of a first target assembly instruction contained in an operator to be optimized in a deep learning application. Deep learning application developers can use a high-level programming language (such as C language) to write programs for the CUDA architecture, and the resulting deep learning application can run at ultra-high performance on a GPU that supports CUDA.

[0062] According to the kernel function to which the first target assembly instruction belongs, a storage space for storing the kernel function code segment is determined, and further according to the encoding rule of the first target assembly instruction, the storage space of the first target assembly instruction is determined, such as called the first storage space.

[0063] In one embodiment of the present application, the compiled instructions of the CUDA-based deep learning application are stored in a fatbinary, which is an image of an ELF (Executable and Linkable Format) structure. The position of the kernel function in the image can be directly located according to the function name of the kernel function. Then, according to the encoding rules of the assembly instruction, the position of the assembly instruction to be optimized in the code segment of the kernel function is determined, such as the offset in the image.

[0064] For example, if the data read instruction of kernel function X needs to be changed, the offset x0 of kernel function X in the image is first obtained by parsing the ELF structure. Then, the code segment of the entire kernel function X is searched according to the offset x0, and the positions of all read instructions in the code segment (i.e., the offset of the read instruction in the image) are obtained according to the encoding rule of the data read instruction, such as y0, y1...yn. S220, in the process of loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU, the assembly instruction corresponding to the first storage space is replaced with the first jump instruction and loaded into the register.

[0065] The first jump instruction is used to jump to a second storage space storing a second target assembly instruction.

[0066] In one embodiment of the present application, the fatBinaryCtl_SetBinary API is dynamically hijacked through the Linux system environment variable LD_PRELOAD to change the assembly instructions loaded into the register.

[0067] For example, the first data reading instruction that was originally supposed to load the first storage space is changed to loading a jump instruction, which is used to jump to the storage space storing the second target assembly instruction, that is, after all instructions are loaded and the jump instruction is executed, it jumps to execute the second target assembly instruction.

[0068] Among them, LD_PRELOAD can affect the runtime link of the program (Runtime linker), allowing the definition of dynamic link libraries that are loaded first before the program runs. In this application, new instructions are injected into the loaded CUDA assembly instructions through the environment variable LD_PRELOAD, so as to achieve the purpose of changing the operator in CUDA, for example, injecting new data reading instructions and / or data output instructions into the CUDA assembly instructions, thereby changing the input and output logic of the kernel function.

[0069] For example, the code segment of kernel function X corresponds to a storage address of y0~yn (y0~yn is an address offset), and a new storage space is opened after the space, such as an address offset range of z0~zn, and the new instruction to be changed, that is, the second target assembly instruction, is stored in the space.

[0070] The compiled instructions of the deep learning application are stored in the fatbinary in the order in which the instructions are executed. When the instructions are loaded from the fatbinary to the GPU register, they are loaded into the register in the order of the instructions in the fatbinary. Therefore, the order of the assembly instructions in the register is the same as that in the fatbinary.

[0071] like Figure 3 As shown, Figure (a) is a schematic diagram of the storage space corresponding to the assembly instructions of the kernel function X before replacement, wherein the y0 position is the data loading instruction (i.e., the first type of data reading instruction) "LD offset0", which is used to load data at an offset of offset0.

[0072] The offset is the distance between the actual address of the storage unit and the segment address of the segment where the unit is located (ie, the head address of the segment), that is, the offset is the position of the actual address relative to the head address of the segment.

[0073] Figure (b) is a schematic diagram of the storage space corresponding to the assembly instructions of the kernel function X after replacement. The data loading instruction at position y0 is replaced with the first jump instruction, i.e., "JMP z0". The storage space corresponding to z0 stores a new data loading instruction (i.e., the second type of data reading instruction) "LD offset1", which is used to load the data stored at the offset position offset1. It can be seen that the second type of data reading instruction and the first type of data reading instruction read data in different storage spaces.

[0074] The first type of data read instruction and the second type of data read instruction have different data read logics. For example, the second type of data read instruction may include a data read instruction and a calculation instruction, wherein the calculation instruction performs operations on the data read by the data read instruction. In this scenario, the data read in the second type of data read instruction may be the same as or different from the data read instruction in the first type, and this application does not impose any special restrictions on this.

[0075] In other embodiments, the first target assembly instruction may also be a data output instruction, that is, the original first-class data output instruction is replaced with a second-class data output instruction, wherein the output logic of the second-class data output instruction is different from that of the first-class data output instruction. For example, the second-class data output instruction includes a calculation instruction and an output instruction, that is, a certain operation is performed on the data before outputting it. This application does not impose any special restrictions on this.

[0076] Among them, y0 and z0 are register addresses, which can be hard-coded into the GPU machine code.

[0077] In one embodiment of the present application, the storage space corresponding to z0 is a newly opened storage space after the storage space for storing the code segment of the kernel function X. Of course, in other embodiments, the storage space corresponding to z0 may also be other unused storage space in the register, which is not limited in the present application.

[0078] S230: Load a second jump instruction into a next storage space adjacent to the second storage space.

[0079] The second jump instruction is used to jump to a next storage space adjacent to the first storage space, and the next assembly instruction adjacent to the first target assembly instruction is stored in the storage space.

[0080] For example, still Figure 3 Taking the example shown as an example, the first storage space is the y0 position, the second storage space is the z0 position, and the next storage space adjacent to z0 is z0+1. The second jump instruction is loaded in the storage space corresponding to z0+1, that is, "JMP y0+1", wherein the y0+1 position (that is, y1) is the next storage space of y0. Therefore, the second jump instruction is an instruction to jump to the storage at the y1 position.

[0081] In another exemplary embodiment, the second target assembly instruction may include multiple instructions, and therefore, the second storage space may be a storage space corresponding to an address segment including multiple storage addresses. In this case, the next storage space of the second storage space refers to the storage space corresponding to the next storage address of the segment tail address of the second storage space.

[0082] For convenience, Figure 3 Only the case where the second storage space is a storage space corresponding to a storage address is shown, that is, the second storage space is the storage space corresponding to the storage address z0. Figure 3 This is merely an example and is not intended to limit the present application.

[0083] S240, executing each assembly instruction in sequence.

[0084] Still Figure 3 The example shown in the figure shows that after changing the loaded instructions during the assembly instruction loading process, Figure 3As shown in (b), the jump instruction stored at the y0 position is executed, jumping to the z0 position, executing the data loading instruction stored at the position, and then continuing to execute the instruction at the z0+1 (i.e., z1) position, i.e., jumping to the y0+1 position, and executing the instruction stored at the position. Through the above process, the data loading instruction at the y0 position is changed to a jump instruction to jump to the z0 position, and a new data loading instruction is loaded at the z0 position, and the original first data loading instruction is changed to the processing logic of jump → second data loading instruction → jump. Thereby, the purpose of replacing the original assembly instruction with the new assembly instruction is achieved, and the instructions that can be replaced are not limited to the data loading instructions in the above embodiment, but can also be data output instructions, or data calculation instructions, and the present application does not limit this.

[0085] The performance optimization method for deep learning applications based on GPU provided in this embodiment, in the process of loading the assembly instructions corresponding to the deep learning application into the register of the GPU, replaces the first target assembly instruction that needs to be replaced with the first jump instruction, the first jump instruction is used to jump to the storage space corresponding to the second target assembly instruction, and execute the second target assembly instruction, after executing the second target assembly instruction, the second jump instruction is used to jump to the next assembly instruction corresponding to the first target assembly instruction, so that the processing logic that should have executed the first target assembly instruction is changed to the processing logic of jump-execute the second target assembly instruction-jump. This scheme injects new assembly-level code, and the code is combined with the high-performance function library in CUDA to obtain a new operator, so that the new operator can also achieve higher computing performance. The GPU has higher flexibility for the new operator, and at the same time, the overall performance of the new operator is improved.

[0086] On the other hand, the present application also provides an embodiment of a performance optimization device for deep learning applications based on a GPU, such as Figure 4 As shown, the device may include an instruction replacement module 110 and an instruction execution module 120. The instruction replacement module 110 is used to replace the first target assembly instruction in the assembly instruction with the second target assembly instruction when loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU.

[0087] The first target assembly instruction is the assembly instruction to be optimized contained in the operator to be optimized, and the second target assembly instruction is the optimized assembly instruction and does not belong to the operator to be optimized.

[0088] In an exemplary embodiment, the instruction replacement module 110 includes: a first storage submodule and a second storage submodule.

[0089] The first storage submodule is used to store the second target assembly instruction in the second storage space, wherein the second target assembly instruction.

[0090] The second storage submodule is used to store the first jump instruction in a first storage space used to store the first target assembly instruction, and the first jump instruction is used to jump to the second storage space.

[0091] In a possible implementation, the first target assembly instruction is a first type of data read instruction, the second target assembly instruction is a second type of data read instruction, and the first type of data read instruction and the second type of data read instruction have different read logics.

[0092] In another possible implementation, the first target assembly instruction is a first type of data output instruction, the second target assembly instruction is a second type of data output instruction, and the output logic of the first type of data output instruction is different from that of the second type of data output instruction.

[0093] The instruction execution module 120 is configured to execute other assembly instructions after the first target assembly instruction after executing the second target assembly instruction.

[0094] In an exemplary embodiment, a second jump instruction is stored in the next available storage space of the second storage space, and the second jump instruction is used to jump to a third storage space, wherein the third storage space stores the next assembly instruction of the first target assembly instruction. The second jump instruction is executed to jump to the third storage space, and the assembly instruction stored in the third storage space is executed.

[0095] The performance optimization device for deep learning applications based on GPU provided in this embodiment, when loading the assembly instructions of the original operator compiled by the compiler into the register of the GPU, uses the assembly instructions of the new operator to replace some instructions in the original operator, so that the assembly instructions loaded into the register include the instructions of the new operator and the assembly instructions of the original operator that have not been replaced, and the assembly instructions that have not been replaced can still give full play to the hardware performance advantages of the GPU. In other words, this scheme realizes the combination of the new operator and the high-performance function library in the GPU, so that the new operator can also achieve higher computing performance. Therefore, using this scheme, the GPU can not only support new operators for deep learning, that is, versatility, but also improve the overall operating performance of the GPU when executing the new operator, that is, improve the operating performance.

[0096] On the other hand, the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores a program that can be run on the processor. When the processor runs the program stored in the memory, any of the above-mentioned performance optimization methods for GPU-based deep learning applications is implemented. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0097] On the other hand, the present application also provides a computer-executable storage medium, which stores a program, and when the program is executed by a computing device, it implements the performance optimization method of the GPU-based deep learning application described in any of the above method embodiments.

[0098] On the other hand, the present application also provides a computer program product, which, when executed on an electronic device, is suitable for executing the performance optimization method for initializing a GPU-based deep learning application as described in any of the above method embodiments.

[0099] On the other hand, the present application also provides a chip system, comprising: at least one processor and an interface, wherein the interface is used to receive code instructions and transmit them to at least one processor; the at least one processor runs the code instructions to implement the performance optimization method for GPU-based deep learning applications described in any of the above method embodiments.

[0100] For the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0101] It should be noted that the technical features recorded in the various embodiments in this specification can be replaced or combined with each other, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0102] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0103] The modules and sub-modules in the devices and terminals in the embodiments of the present application can be combined, divided and deleted according to actual needs.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0105] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0107] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0108] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0109] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A performance optimization method for deep learning applications based on a graphics processor, characterized in that: The method comprises: When loading the assembly instructions corresponding to the operator to be optimized in the deep learning application into the register of the GPU, replacing the first target assembly instructions in the assembly instructions with the second target assembly instructions, the operator to be optimized is the original operator in the unified computing device architecture CUDA, the first target assembly instructions are the assembly instructions corresponding to the operation in the CUDA, and the second target assembly instructions are the optimized assembly instructions and do not belong to the assembly instructions of the original operator in the CUDA; After executing the second target assembly instruction, other assembly instructions after the first target assembly instruction are executed.

2. The method according to claim 1, characterized in that The replacing the first target assembly instruction in the assembly instruction with the second target assembly instruction comprises: Storing the second target assembly instruction in the second storage space, wherein the second target assembly instruction; A first jump instruction is stored in a first storage space used for storing the first target assembly instruction, and the first jump instruction is used to jump to the second storage space.

3. The method according to claim 2, characterized in that After executing the second target assembly instruction, executing the next assembly instruction adjacent to the first target assembly instruction includes: storing a second jump instruction in a next available storage space of the second storage space, wherein the second jump instruction is used to jump to a third storage space, wherein the third storage space stores a next assembly instruction of the first target assembly instruction; Execute the second jump instruction to jump to the third storage space, and execute the assembly instructions stored in the third storage space.

4. The method according to any one of claims 1 to 3, characterized in that: The first target assembly instruction is a first type of data read instruction, and the second target assembly instruction is a second type of data read instruction, wherein the reading logic of the first type of data read instruction is different from that of the second type of data read instruction.

5. The method according to any one of claims 1 to 3, characterized in that: The first target assembly instruction is a first type of data output instruction, and the second target assembly instruction is a second type of data output instruction. The output logic of the first type of data output instruction is different from that of the second type of data output instruction.

6. The method according to claim 2, characterized in that The process of determining the first storage space corresponding to the first target assembly instruction includes: Determine a storage address segment of the assembly instruction corresponding to the operator to be optimized, and find out a storage address offset corresponding to the first target assembly instruction from the storage address segment; An address of a first storage space storing the first target assembly instruction is obtained based on the storage address offset.

7. A performance optimization device for deep learning applications based on a graphics processor, characterized in that: The device comprises: An instruction replacement module, configured to replace a first target assembly instruction in the assembly instruction with a second target assembly instruction when loading the assembly instruction corresponding to the operator to be optimized in the deep learning application into the register of the GPU, wherein the operator to be optimized is an original operator in CUDA, the first target assembly instruction is an assembly instruction corresponding to the operation in the CUDA, and the second target assembly instruction is an optimized assembly instruction and does not belong to the assembly instruction of the original operator in the CUDA; The instruction execution module is used to execute other assembly instructions after the first target assembly instruction after executing the second target assembly instruction.

8. The device according to claim 7, characterized in that The instruction replacement module comprises: A first storage submodule is used to store the second target assembly instruction in a second storage space, wherein the second target assembly instruction; The second storage submodule is used to store the first jump instruction in a first storage space used to store the first target assembly instruction, and the first jump instruction is used to jump to the second storage space.

9. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores instructions, and when the processor runs the instructions stored in the memory, it implements the performance optimization method for deep learning applications based on a graphics processor as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The storage medium stores instructions, which, when executed by an electronic device, implement the performance optimization method for deep learning applications based on a graphics processor as described in any one of claims 1 to 6.

Images