Kernel code loop expansion method, device, equipment and storage medium

Abstract

The present invention discloses a loop expansion method, device, equipment and storage medium of kernel code, which relates to the field of deep learning technology, including: in the process of neural network model inference, respectively capturing the size of the output tensor of each layer of the neural network model; determining the current expansion factor corresponding to each layer according to the size of the output tensor of each layer; generating the current unified computing device architecture code according to each current expansion factor, and compiling the current unified computing device architecture code to obtain the compiled unified computing device architecture code; using the compiled unified computing device architecture code to loop expand the kernel code, and obtain the current loop expansion time; adjusting each current expansion factor according to the current loop expansion time to obtain the target expansion factor corresponding to each layer; loop expanding the kernel code according to each target expansion factor. The present invention reduces the development difficulty and cost, and improves the compatibility of the expansion factor with the hardware architecture.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a loop unrolling method, device, equipment and computer-readable storage medium for kernel code. Background Art

[0002] In the inference process of deep learning models, the deep learning framework relies on the computing power of the hardware architecture to achieve efficient parallel computing. Therefore, it is necessary to optimize the execution efficiency of the kernel code. By writing the Compute Unified Device Architecture (CUDA) kernel code, the hardware architecture can be used for parallel processing, greatly improving the training and inference speed of deep learning models.

[0003] Currently, there are two main methods for loop unrolling in kernel code: manual unrolling and automatic unrolling. The basic idea of ​​manual loop unrolling is to manually unroll the iteration statements in the loop when writing kernel code. Automatic loop unrolling is when the compiler or the UCA programming framework automatically unrolls the loop during the compilation process. Manual unrolling is complex and requires developers to have rich UCA programming experience. The unrolling strategy of automatic unrolling is uncontrollable and may have compatibility issues.

[0004] In summary, how to effectively solve the problems of high complexity of loop unrolling of kernel code, high requirements on developer programming experience, poor compatibility, etc., is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The object of the present invention is to provide a loop expansion method for kernel code, which reduces the development difficulty and cost, improves the compatibility of the expansion factor with the hardware architecture, and improves the efficiency of neural network model reasoning; another object of the present invention is to provide a loop expansion device for kernel code, a computer-readable storage medium and a computer program product.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] A loop unrolling method for kernel code, comprising:

[0008] During the neural network model inference process, the size of the output tensor of each layer of the neural network model is captured respectively;

[0009] Determine the current expansion factor corresponding to each layer according to the size of the output tensor of each layer;

[0010] Generate a current unified computing device architecture code according to each current expansion factor, and compile the current unified computing device architecture code to obtain a compiled unified computing device architecture code;

[0011] Perform loop unrolling on the kernel code using the compiled unified computing device architecture code, and obtain a current loop unrolling time;

[0012] Adjusting each current expansion factor according to the current loop expansion time to obtain a target expansion factor corresponding to each layer;

[0013] The kernel code is loop-unrolled according to each target unrolling factor.

[0014] In a specific implementation of the present invention, adjusting each current expansion factor according to the current loop expansion time includes:

[0015] According to the current loop unrolling time, each current unrolling factor is adjusted using a preset optimization strategy.

[0016] In a specific implementation of the present invention, each current expansion factor is adjusted using a preset optimization strategy, including:

[0017] Get the current expansion factor to be adjusted;

[0018] Obtaining a preset difference between adjacent expansion factors;

[0019] Generating an adjacent expansion factor of the current expansion factor to be adjusted according to the preset difference;

[0020] Each current expansion factor is adjusted according to the adjacent expansion factors.

[0021] In a specific implementation of the present invention, each current expansion factor is adjusted according to the adjacent expansion factor to obtain the target expansion factor corresponding to each layer, including:

[0022] Determine the adjacent expansion factor as a new current expansion factor, and repeat the step of generating a current unified computing device architecture code according to each current expansion factor;

[0023] When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion;

[0024] If yes, replacing the current expansion factor to be adjusted with the adjacent expansion factor;

[0025] If not, retaining the current expansion factor to be adjusted;

[0026] Determine whether each current expansion factor is a globally optimal expansion factor or reaches a preset convergence condition;

[0027] If yes, then each current expansion factor is determined as the target expansion factor corresponding to each layer;

[0028] If not, the step of generating the current unified computing device architecture code according to each current expansion factor is repeated.

[0029] In a specific implementation of the present invention, each current expansion factor is adjusted using a preset optimization strategy, including:

[0030] Randomly generate expansion factors;

[0031] Each current expansion factor is adjusted using a randomly generated expansion factor.

[0032] In a specific implementation of the present invention, each current expansion factor is adjusted using a randomly generated expansion factor to obtain a target expansion factor corresponding to each layer, including:

[0033] Determine the randomly generated expansion factor as a new current expansion factor, and repeat the step of generating the current unified computing device architecture code according to each current expansion factor;

[0034] When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion;

[0035] If yes, the current expansion factor to be adjusted is replaced by the new current expansion factor;

[0036] If not, obtaining the current iteration number, and selecting the current expansion factor from the new current expansion factor and the current expansion factor to be adjusted according to the current iteration number;

[0037] Determining whether the current expansion factor meets a preset optimal solution determination condition;

[0038] If yes, then each current expansion factor is determined as the target expansion factor corresponding to each layer;

[0039] If not, repeat the step of randomly generating the expansion factor.

[0040] In a specific embodiment of the present invention, the size of the output tensor of each layer of the neural network model is captured respectively, including:

[0041] The size of the output tensor of each layer of the neural network model is captured by the hook functions pre-registered in each layer of the neural network model.

[0042] In a specific implementation of the present invention, after obtaining the compiled unified computing device architecture code, the method further includes:

[0043] generating a target forward propagation function according to the compiled unified computing device architecture code;

[0044] Replacing an original forward propagation function of the neural network model with the target forward propagation function;

[0045] Accordingly, the kernel code is loop-unrolled using the compiled unified computing device architecture code, including:

[0046] The compiled unified computing device architecture code is called using the target forward propagation function to perform loop unrolling on the kernel code.

[0047] In a specific implementation of the present invention, after obtaining the target expansion factors corresponding to each layer, the method further includes:

[0048] The target dynamic link library is generated by using each target expansion factor, so that the target dynamic link library is called to perform reasoning during the neural network model reasoning process.

[0049] In a specific implementation of the present invention, after obtaining the compiled unified computing device architecture code, the method further includes:

[0050] The compiled unified computing device architecture code is called to generate a current dynamic link library, so that the current dynamic link library is called to perform reasoning during the neural network model reasoning process.

[0051] In a specific embodiment of the present invention, the size of the output tensor of each layer of the neural network model is captured respectively, including:

[0052] The application programming interface of the automatic computing device architecture loop unrolling optimization module is called to capture the size of the output tensor of each layer of the neural network model respectively; wherein the automatic computing device architecture loop unrolling optimization module is a module integrated into the neural network model through modularization.

[0053] In a specific implementation of the present invention, the current expansion factors corresponding to each layer are determined according to the size of the output tensor of each layer, including:

[0054] Get the hardware architecture characteristics of the graphics processor;

[0055] The current expansion factor corresponding to each layer is determined according to the hardware architecture characteristics and the size of the output tensor of each layer.

[0056] A loop unrolling device for kernel code, comprising:

[0057] Memory for storing computer programs;

[0058] A processor is used to implement the steps of the loop unrolling method of the kernel code as described above when executing the computer program.

[0059] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the loop unrolling method of the kernel code as described above are implemented.

[0060] A computer program product comprises a computer program, which implements the steps of the loop unrolling method of the kernel code as described above when the computer program is executed by a processor.

[0061] The kernel code loop expansion method provided by the present invention captures the size of the output tensor of each layer of the neural network model during the neural network model inference process; determines the current expansion factor corresponding to each layer according to the size of the output tensor of each layer; generates the current unified computing device architecture code according to each current expansion factor, and compiles the current unified computing device architecture code to obtain the compiled unified computing device architecture code; uses the compiled unified computing device architecture code to loop expand the kernel code and obtain the current loop expansion time; adjusts each current expansion factor according to the current loop expansion time to obtain the target expansion factor corresponding to each layer; and loop expands the kernel code according to each target expansion factor.

[0062] It can be seen from the above technical solution that by capturing the size of the output tensor of each layer of the neural network model, the current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer, and the unified computing device architecture code is generated and compiled according to the current expansion factor to obtain the compiled unified computing device architecture code. The current loop expansion time of the kernel code is obtained by using the compiled unified computing device architecture code, the expansion factor is dynamically adjusted according to the current loop expansion time, and the kernel code is loop expanded according to the adjusted target expansion factor, so that the optimal unified computing device architecture code is automatically generated and compiled. Users can achieve performance improvement without in-depth understanding of the unified computing device architecture programming details, reducing development difficulty and cost. The compatibility of the expansion factor and the hardware architecture is improved, so that the unified computing device architecture kernel can maximize the computing power of the hardware architecture, improve the operation speed, and thus significantly improve the efficiency of neural network model reasoning.

[0063] Correspondingly, the present invention also provides a kernel code loop expansion device, equipment and computer-readable storage medium corresponding to the above-mentioned kernel code loop expansion method, which has the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] FIG1 is a flow chart of an implementation of a loop expansion method of a kernel code according to an embodiment of the present invention;

[0066] FIG2 is a structural block diagram of a loop expansion device for kernel code according to an embodiment of the present invention;

[0067] FIG3 is a structural block diagram of a loop expansion device for kernel code according to an embodiment of the present invention;

[0068] FIG. 4 is a schematic diagram of the specific structure of a loop unrolling device for kernel code provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] Deep learning frameworks such as TensorFlow and MXNet have become mainstream tools for modern artificial intelligence research and application. They provide efficient application programming interfaces (APIs) and rich functions to support the development and training of various deep learning models. Optimizing the execution efficiency of graphics processing unit (GPU) code is crucial to improving the overall performance of the model. The unified computing device architecture is a parallel computing platform and programming model that is widely used in accelerated computing for deep learning. However, related deep learning frameworks usually provide some basic unified computing device architecture kernel implementations, but these implementations are often general and are not optimized for the characteristics of specific models and data. Writing efficient unified computing device architecture code requires manually adjusting parameters such as thread configuration and loop unrolling factor to unroll kernel code loops to achieve higher performance. It requires a deep understanding of the graphics processor architecture and parallel programming skills, and different situations require different processing methods, which is difficult to be universal. This is a complex and time-consuming task for most deep learning developers, and manual optimization is costly.

[0070] At present, many deep learning frameworks use fixed optimization strategies, which may perform well when processing tensors of a certain size, but often fail to meet expectations when facing variable data and models. Due to the lack of dynamic adjustment capabilities, fixed optimization strategies cannot fully utilize the computing resources of the graphics processor, resulting in performance bottlenecks. Most existing deep learning frameworks rely on static optimization strategies and empirical rules, lacking a systematic automated optimization mechanism. The selection of expansion factors is usually determined by experiments and experience, which makes it difficult to find the optimal parameter settings in various application scenarios, resulting in limited performance optimization effects. In related schemes, the lack of effective performance measurement and feedback mechanisms makes the optimization process blind and difficult to accurately evaluate the effects of different optimization strategies. This not only increases the difficulty of optimization, but may also lead to suboptimal optimization results and fail to achieve optimal performance.

[0071] To this end, the loop expansion method of the kernel code provided in this application reduces the development difficulty and cost, improves the compatibility of the expansion factor with the hardware architecture, and improves the efficiency of neural network model reasoning.

[0072] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0073] Referring to FIG. 1 , FIG. 1 is a flowchart of an implementation of a loop unrolling method of a kernel code according to an embodiment of the present invention. The method may include the following steps:

[0074] S101: During the neural network model inference process, the size of the output tensor of each layer of the neural network model is captured respectively.

[0075] The neural network model contains multiple network transmission layers. During the reasoning process of the neural network model, each network transmission layer has tensor output, which captures the size of the output tensor of each layer of the neural network model.

[0076] S102: Determine the current expansion factor corresponding to each layer according to the size of the output tensor of each layer.

[0077] After capturing the size of the output tensor of each layer of the neural network model, the current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer. Each layer may contain one or more calculation steps, and each calculation step has a corresponding expansion factor. Therefore, the number of current expansion factors corresponding to each network transmission layer determined according to the size of the output tensor of each layer may be one or more.

[0078] The determined current expansion factor is generally a decimal integer value corresponding to each power of 2.

[0079] S103: Generate a current unified computing device architecture code according to each current expansion factor, and compile the current unified computing device architecture code to obtain a compiled unified computing device architecture code.

[0080] After determining the current expansion factors corresponding to each layer, the current unified computing device architecture code is generated according to each current expansion factor, and the generated current unified computing device architecture code can also be saved as a .cu file, and the current unified computing device architecture code is compiled to obtain the compiled unified computing device architecture code. The captured tensor size information will be used to generate a unified computing device architecture code that matches the specific computing requirements, ensuring that the dimension information of each tensor can be accurately passed to the automatic computing device architecture loop unrolling optimization module.

[0081] S104: Perform loop unrolling on the kernel code using the compiled unified computing device architecture code, and obtain the current loop unrolling time.

[0082] After obtaining the compiled unified computing device architecture code, the kernel code is loop-unrolled using the compiled unified computing device architecture code, and during the loop-unrolling process, the system records the current loop-unrolling time to obtain the current loop-unrolling time.

[0083] S105: adjusting each current expansion factor according to the current loop expansion time to obtain a target expansion factor corresponding to each layer.

[0084] After the current loop expansion time is obtained, each current expansion factor is adjusted according to the current loop expansion time to obtain the target expansion factor corresponding to each layer. For example, the loop expansion time corresponding to the previous loop expansion can be obtained, and the loop expansion time corresponding to the previous loop expansion is compared with the current loop expansion time corresponding to this loop expansion, and each current expansion factor is adjusted according to the comparison result to obtain the target expansion factor corresponding to each layer.

[0085] S106: Perform loop expansion on the kernel code according to each target expansion factor.

[0086] After obtaining the target expansion factors corresponding to each layer, the kernel code is cyclically expanded according to each target expansion factor. By automatically generating and optimizing the unified computing device architecture code and dynamically adjusting the expansion factor, the workload of manual writing and debugging is reduced. The most suitable optimized code can be generated according to the specific tensor size, and the development difficulty and cost are reduced by providing real-time performance feedback. The efficiency and performance of deep learning reasoning are significantly improved.

[0087] By measuring the execution time of different expansion factors in real time, providing quick feedback and adjustments, we can ensure that the best optimization parameters are always used to achieve continuous performance improvement. We can measure the execution time of different expansion factors in real time and dynamically adjust the expansion factors based on the measurement results. The performance feedback mechanism can not only accurately evaluate the effects of different optimization strategies, but also continuously optimize during operation to adapt to changing workloads and hardware environments.

[0088] It can be seen from the above technical solution that by capturing the size of the output tensor of each layer of the neural network model, the current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer, and the unified computing device architecture code is generated and compiled according to the current expansion factor to obtain the compiled unified computing device architecture code. The current loop expansion time of the kernel code is obtained by using the compiled unified computing device architecture code, the expansion factor is dynamically adjusted according to the current loop expansion time, and the kernel code is loop expanded according to the adjusted target expansion factor, so that the optimal unified computing device architecture code is automatically generated and compiled. Users can achieve performance improvement without in-depth understanding of the unified computing device architecture programming details, reducing development difficulty and cost. The compatibility of the expansion factor and the hardware architecture is improved, so that the unified computing device architecture kernel can maximize the computing power of the hardware architecture, improve the operation speed, and thus significantly improve the efficiency of neural network model reasoning.

[0089] It should be noted that, based on the above embodiment, the embodiment of the present invention also provides corresponding improved solutions. In the subsequent embodiments, the same steps or corresponding steps as those in the above embodiment can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be repeated one by one in the following improved embodiments.

[0090] In a specific implementation of the present invention, adjusting each current unrolling factor according to the current loop unrolling time may include the following steps:

[0091] According to the current loop unrolling time, each current unrolling factor is adjusted using a preset optimization strategy.

[0092] An optimization strategy for adjusting the expansion factor is pre-set. After obtaining the current loop expansion time, the preset optimization strategy is used to adjust each current expansion factor according to the current loop expansion time. By pre-setting the optimization strategy to adjust the expansion factor, the expansion factor is gradually tuned, thereby better utilizing the hardware architecture resources.

[0093] In a specific implementation of the present invention, adjusting each current expansion factor using a preset optimization strategy may include the following steps:

[0094] Get the current expansion factor to be adjusted;

[0095] Obtaining a preset difference between adjacent expansion factors;

[0096] Generating an adjacent expansion factor of the current expansion factor to be adjusted according to the preset difference;

[0097] Each current expansion factor is adjusted according to the adjacent expansion factors.

[0098] When adjusting the current expansion factor, the current expansion factor to be adjusted can be obtained, and the preset difference between adjacent expansion factors can be obtained, such as the preset difference can be set to ±1, ±2, etc. The adjacent expansion factors of the current expansion factor to be adjusted are generated according to the preset difference, and each current expansion factor is adjusted according to the adjacent expansion factors. By optimizing the expansion factor near the expansion factor determined according to the size of the tensor, it is avoided to perform cyclic expansion according to the expansion factor that deviates greatly from the optimal expansion factor, and the expansion factor can be adjusted quickly and accurately.

[0099] In a specific implementation of the present invention, adjusting each current expansion factor according to the adjacent expansion factor to obtain the target expansion factor corresponding to each layer may include the following steps:

[0100] Step 1: Determine the adjacent expansion factor as the new current expansion factor, and return to execute step S103;

[0101] Step 2: When the current loop expansion time corresponding to the current loop expansion is obtained, determine whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion. If so, execute step 3; if not, execute step 4.

[0102] Step 3: Replace the current expansion factor to be adjusted with the adjacent expansion factor;

[0103] Step 4: retain the current expansion factor to be adjusted;

[0104] Step 5: Determine whether each current expansion factor is the global optimal expansion factor or reaches the preset convergence condition. If so, execute step 8; if not, return to execute step S103;

[0105] Step 6: Determine each current expansion factor as the target expansion factor corresponding to each layer.

[0106] For the convenience of description, the above eight steps can be combined for explanation.

[0107] When adjusting each current expansion factor according to the adjacent expansion factor, the adjacent expansion factor is determined as the new current expansion factor, and the unified computing device architecture code is regenerated and compiled according to the new current expansion factor, and the kernel code is loop-unrolled according to the new compiled unified computing device architecture code. When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion. If so, the current expansion factor to be adjusted is replaced by the adjacent expansion factor. If not, the current expansion factor to be adjusted is retained. It is determined whether each current expansion factor is the global optimal expansion factor or reaches the preset convergence condition. If so, each current expansion factor is determined as the target expansion factor corresponding to each layer. If not, the unified computing device architecture code is regenerated and compiled according to the current expansion factor, and the kernel code is loop-unrolled according to the compiled unified computing device architecture code. According to the loop expansion time, the current expansion factor is further determined and whether it is the global optimal expansion factor or whether the preset convergence condition is reached. By adjusting each current expansion factor according to the adjacent expansion factor, it is ensured that the expansion factor with the best performance is used for reasoning during the reasoning process.

[0108] In a specific implementation of the present invention, adjusting each current expansion factor using a preset optimization strategy may include the following steps:

[0109] Step 1: Randomly generate expansion factors;

[0110] Step 2: Use the randomly generated expansion factors to adjust each current expansion factor.

[0111] When adjusting the current expansion factor, the expansion factor can be randomly generated, and the randomly generated expansion factor can be used to adjust each current expansion factor, thereby jumping out of the local optimal solution, avoiding falling into the local optimal solution, and improving the accuracy and effectiveness of the expansion factor adjustment.

[0112] In a specific implementation of the present invention, adjusting each current expansion factor by using a randomly generated expansion factor to obtain a target expansion factor corresponding to each layer may include the following steps:

[0113] Step 1: Determine the randomly generated expansion factor as the new current expansion factor, and return to execute step S103;

[0114] Step 2: When the current loop expansion time corresponding to the current loop expansion is obtained, determine whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion. If so, execute step 3; if not, execute step 4.

[0115] Step 3: Replace the current expansion factor to be adjusted with the new current expansion factor;

[0116] Step 4: Obtain the current iteration number, and select the current expansion factor from the new current expansion factor and the current expansion factor to be adjusted according to the current iteration number;

[0117] Step 5: Determine whether the current expansion factor meets the preset optimal solution judgment condition. If yes, execute step 6; if not, return to execute step 1;

[0118] Step 6: Determine each current expansion factor as the target expansion factor corresponding to each layer.

[0119] For the convenience of description, the above six steps can be combined for explanation.

[0120] When the current expansion factors are adjusted by using the randomly generated expansion factors, the randomly generated expansion factors are determined as the new current expansion factors. The randomly generated expansion factors can be values ​​near the current expansion factors, or they can be larger or smaller factors away from the current expansion factors, allowing the local optimal solution to be jumped out, thereby realizing random perturbation to select the new current expansion factors. The unified computing device architecture code is regenerated and compiled according to the new current expansion factors, and the kernel code is loop expanded according to the new compiled unified computing device architecture code. When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion. If so, the current expansion factor to be adjusted is replaced by the new current expansion factor. If not, the current number of iterations is obtained, and the current expansion factor is selected from the new current expansion factor and the current expansion factor to be adjusted according to the current number of iterations. Thus, the system decides whether to accept the new current expansion factor according to a certain probability, and the probability gradually decreases with the number of iterations, thereby avoiding falling into the local optimal solution. Determine whether the current expansion factor meets the preset optimal solution judgment condition. If so, determine each current expansion factor as the target expansion factor corresponding to each layer. If not, re-generate the expansion factor randomly, and then conduct a new round of expansion factor screening. By adjusting each current expansion factor using the randomly generated expansion factor, it is allowed to jump out of the local optimal solution and avoid falling into the local optimal solution.

[0121] In a specific embodiment of the present invention, capturing the size of the output tensor of each layer of the neural network model respectively may include the following steps:

[0122] The size of the output tensor of each layer of the neural network model is captured by pre-registering the hook functions of each layer of the neural network model.

[0123] The hook functions are pre-registered in each layer of the neural network model, and the size of the output tensor of each layer of the neural network model is captured by the hook functions pre-registered in each layer of the neural network model, and the size of the output tensor of each layer captured can be saved in a list inside the automatic computing device architecture loop unrolling optimization module. By pre-registering the hook functions in each layer of the neural network model to capture the size of the output tensor of each layer of the neural network model, the size of the output tensor of each layer in the neural network model can be quickly and accurately captured.

[0124] In a specific implementation of the present invention, after obtaining the compiled unified computing device architecture code, the method may further include the following steps:

[0125] Step 1: Generate a target forward propagation function based on the compiled unified computing device architecture code;

[0126] Step 2: Replace the original forward propagation function of the neural network model with the target forward propagation function;

[0127] Accordingly, loop unrolling of the kernel code using the compiled unified computing device architecture code may include the following steps:

[0128] Loop unrolling of the kernel code is performed using the target forward propagation function calls to the compiled Unified Compute Device Architecture code.

[0129] For the convenience of description, the above steps can be combined for explanation.

[0130] After obtaining the compiled unified computing device architecture code, a target forward propagation (Forward) function is generated according to the compiled unified computing device architecture code, and the target forward propagation function is used to replace the original forward propagation function of the neural network model. The target forward propagation function is used to call the compiled unified computing device architecture code to perform loop expansion on the kernel code, thereby better utilizing the graphics processor hardware resources, improving the computing speed and efficiency, and improving the reasoning performance of the neural network model.

[0131] The forward propagation function refers to the function used to calculate the output result after the input data passes through each layer of the neural network model when the neural network model performs forward propagation.

[0132] In a specific implementation of the present invention, after obtaining the target expansion factors corresponding to each layer, the method may further include the following steps:

[0133] The target dynamic link library is generated by using each target expansion factor, so that the target dynamic link library can be called for reasoning during the neural network model reasoning process.

[0134] After obtaining the target expansion factors corresponding to each layer, the target dynamic link library is generated using each target expansion factor, that is, a .so file is generated, so that the target dynamic link library is called for reasoning during the neural network model reasoning process. By using each target expansion factor to generate the target dynamic link library and directly calling the target dynamic link library for reasoning, the model reasoning speed is further improved.

[0135] In a specific implementation of the present invention, after obtaining the compiled unified computing device architecture code, the method may further include the following steps:

[0136] The compiled unified computing device architecture code is called to generate the current dynamic link library, so that the current dynamic link library is called for reasoning during the neural network model reasoning process.

[0137] After obtaining the compiled unified computing device architecture code, the compiled unified computing device architecture code is called to generate the current dynamic link library, so that the current dynamic link library is called for reasoning during the neural network model reasoning process. By calling the compiled unified computing device architecture code to generate the current dynamic link library, the current dynamic link library is directly called for reasoning, which further improves the model reasoning speed.

[0138] In a specific embodiment of the present invention, capturing the size of the output tensor of each layer of the neural network model respectively may include the following steps:

[0139] The application programming interface of the automatic computing device architecture loop unrolling optimization module is called to capture the size of the output tensor of each layer of the neural network model respectively; wherein the automatic computing device architecture loop unrolling optimization module is a module integrated into the neural network model through modularization.

[0140] The automatic computing device architecture loop unrolling optimization module is pre-integrated into the neural network model through modular design, and the automatic computing device architecture loop unrolling optimization module is initialized, such as by configuring necessary environmental parameters, preparing the unified computing device architecture compiler, loading the initial unrolling factor set, etc. The automatic computing device architecture loop unrolling optimization module is initialized. After the initialization is completed, the application programming interface of the automatic computing device architecture loop unrolling optimization module is called to capture the size of the output tensor of each layer of the neural network model respectively. The modular design makes it possible for the automatic computing device architecture loop unrolling optimization module to be easily integrated into the deep learning framework. Users do not need to have an in-depth understanding of the unified computing device architecture programming details, and can achieve performance improvement with simple application programming interface calls. This design not only lowers the development threshold, but also improves the applicability and versatility of the automatic computing device architecture loop unrolling optimization module.

[0141] In a specific implementation of the present invention, step S102 may include the following steps:

[0142] Step 1: Obtain the hardware architecture characteristics of the graphics processor;

[0143] Step 2: Determine the current expansion factor corresponding to each layer based on the hardware architecture characteristics and the size of the output tensor of each layer.

[0144] For the convenience of description, the above two steps can be combined for explanation.

[0145] The hardware architecture characteristics of the graphics processor are obtained, and the current expansion factors corresponding to each layer are determined according to the hardware architecture characteristics and the size of the output tensor of each layer. By determining the current expansion factor according to the hardware architecture characteristics, the compatibility of the expansion factor and the hardware architecture characteristics is guaranteed.

[0146] You can also pre-set a fault recovery and fallback mechanism. When the optimization effect of a certain expansion factor is not as expected, the automatic calculation device architecture loop expansion optimization module can automatically fall back to the previous optimal expansion factor and make further attempts. This mechanism ensures that even if performance problems are encountered during the automatic adjustment process, it can quickly recover to the optimal state to avoid affecting the overall performance.

[0147] Corresponding to the above method embodiment, the present invention further provides a loop expansion device for kernel code. The loop expansion device for kernel code described below and the loop expansion method for kernel code described above can refer to each other.

[0148] Referring to FIG. 2 , FIG. 2 is a structural block diagram of a loop expansion device for kernel code according to an embodiment of the present invention. The device may include:

[0149] A tensor size obtaining unit 21 is used to capture the size of the output tensor of each layer of the neural network model during the neural network model inference process;

[0150] An expansion factor determination unit 22, used to determine the current expansion factor corresponding to each layer according to the size of the output tensor of each layer;

[0151] A compiled code obtaining unit 23 is used to generate a current unified computing device architecture code according to each current expansion factor, and compile the current unified computing device architecture code to obtain a compiled unified computing device architecture code;

[0152] A loop unrolling time acquisition unit 24 is used to perform loop unrolling on the kernel code using the compiled unified computing device architecture code and obtain the current loop unrolling time;

[0153] An expansion factor adjustment unit 25 is used to adjust each current expansion factor according to the current loop expansion time to obtain a target expansion factor corresponding to each layer;

[0154] The loop unrolling unit 26 is used to perform loop unrolling on the kernel code according to each target unrolling factor.

[0155] It can be seen from the above technical solution that by capturing the size of the output tensor of each layer of the neural network model, the current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer, and the unified computing device architecture code is generated and compiled according to the current expansion factor to obtain the compiled unified computing device architecture code. The current loop expansion time of the kernel code is obtained by using the compiled unified computing device architecture code, the expansion factor is dynamically adjusted according to the current loop expansion time, and the kernel code is loop expanded according to the adjusted target expansion factor, so that the optimal unified computing device architecture code is automatically generated and compiled. Users can achieve performance improvement without in-depth understanding of the unified computing device architecture programming details, reducing development difficulty and cost. The compatibility of the expansion factor and the hardware architecture is improved, so that the unified computing device architecture kernel can maximize the computing power of the hardware architecture, improve the operation speed, and thus significantly improve the efficiency of neural network model reasoning.

[0156] In a specific implementation of the present invention, the expansion factor adjustment unit 25 is specifically a unit that adjusts each current expansion factor according to the current loop expansion time by using a preset optimization strategy.

[0157] In a specific embodiment of the present invention, the expansion factor adjustment unit 25 is specifically a unit for obtaining a current expansion factor to be adjusted; obtaining a preset difference between adjacent expansion factors; generating adjacent expansion factors of the current expansion factor to be adjusted according to the preset difference; and adjusting each current expansion factor according to the adjacent expansion factors.

[0158] In a specific implementation of the present invention, the expansion factor adjustment unit 25 includes:

[0159] A first repetitive execution subunit is used to determine the adjacent expansion factors as new current expansion factors, and repeatedly execute the step of generating a current unified computing device architecture code according to each current expansion factor;

[0160] A first judgment subunit is used to judge whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion when the current loop expansion time corresponding to the current loop expansion is obtained;

[0161] An expansion factor replacement subunit, used to replace the current expansion factor to be adjusted with an adjacent expansion factor when it is determined that the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion;

[0162] An expansion factor retaining subunit, used for retaining the current expansion factor to be adjusted when it is determined that the current loop expansion time is not less than the current loop expansion time corresponding to the previous loop expansion;

[0163] A second judgment subunit is used to judge whether each current expansion factor is a global optimal expansion factor or reaches a preset convergence condition;

[0164] A first target expansion factor determination subunit is used to determine each current expansion factor as a target expansion factor corresponding to each layer when it is determined that each current expansion factor is a global optimal expansion factor or reaches a preset convergence condition;

[0165] The second repetitive execution subunit is used to repeatedly execute the step of generating the current unified computing device architecture code according to each current expansion factor when it is determined that each current expansion factor is not a globally optimal expansion factor and the preset convergence condition is not met.

[0166] In a specific implementation of the present invention, the expansion factor adjustment unit 25 is specifically a unit that randomly generates expansion factors and adjusts each current expansion factor using the randomly generated expansion factors.

[0167] In a specific implementation of the present invention, the expansion factor adjustment unit 25 includes:

[0168] A third repetitive execution subunit is used to determine the randomly generated expansion factor as a new current expansion factor, and repeatedly execute the step of generating a current unified computing device architecture code according to each current expansion factor;

[0169] A third judgment subunit is used to judge whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion when the current loop expansion time corresponding to the current loop expansion is obtained;

[0170] An expansion factor replacement subunit, used to replace the current expansion factor to be adjusted with a new current expansion factor when it is determined that the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion;

[0171] The expansion factor screening subunit is used to obtain the current number of iterations when it is determined that the current loop expansion time is not less than the current loop expansion time corresponding to the previous loop expansion, and screen the current expansion factor from the new current expansion factor and the current expansion factor to be adjusted according to the current number of iterations;

[0172] The fourth judgment subunit is used to judge whether the current expansion factor meets the preset optimal solution judgment condition;

[0173] A second target expansion factor determination subunit is used to determine each current expansion factor as a target expansion factor corresponding to each layer when it is determined that the current expansion factor meets a preset optimal solution determination condition;

[0174] The fourth repetitive execution subunit is used to repeatedly execute the step of randomly generating the expansion factor when it is determined that the current expansion factor does not meet the preset optimal solution determination condition.

[0175] In a specific embodiment of the present invention, the tensor size obtaining unit 21 is specifically a unit that captures the size of the output tensor of each layer of the neural network model through the hook functions pre-registered in each layer of the neural network model.

[0176] In a specific embodiment of the present invention, the device may further include:

[0177] A forward propagation function generation unit, configured to generate a target forward propagation function according to the compiled unified computing device architecture code after obtaining the compiled unified computing device architecture code;

[0178] A forward propagation function replacement unit, used to replace an original forward propagation function of a neural network model with a target forward propagation function;

[0179] The loop unrolling time acquisition unit 24 is specifically a unit for loop unrolling the kernel code by using the target forward propagation function to call the compiled unified computing device architecture code.

[0180] In a specific embodiment of the present invention, the device may further include:

[0181] The target dynamic link library generation unit is used to generate a target dynamic link library using each target expansion factor after obtaining the target expansion factors corresponding to each layer, so that the target dynamic link library can be called for reasoning during the neural network model reasoning process.

[0182] In a specific embodiment of the present invention, the device may further include:

[0183] The current dynamic link library generation unit is used to call the compiled unified computing device architecture code to generate the current dynamic link library after obtaining the compiled unified computing device architecture code, so that the current dynamic link library can be called for reasoning during the neural network model reasoning process.

[0184] In a specific embodiment of the present invention, the tensor size acquisition unit 21 is specifically a unit that calls the application programming interface of the automatic computing device architecture loop unrolling optimization module to capture the size of the output tensor of each layer of the neural network model respectively; wherein the automatic computing device architecture loop unrolling optimization module is a module integrated into the neural network model through modularization.

[0185] In a specific implementation of the present invention, the expansion factor determination unit 22 may include:

[0186] A hardware architecture feature acquisition subunit, used to acquire hardware architecture features of a graphics processor;

[0187] The expansion factor determination subunit is used to determine the current expansion factor corresponding to each layer according to the hardware architecture characteristics and the size of the output tensor of each layer.

[0188] Corresponding to the above method embodiment, referring to FIG. 3 , FIG. 3 is a schematic diagram of a loop unrolling device for kernel code provided by the present invention, and the device may include:

[0189] A memory 332, for storing computer programs;

[0190] The processor 322 is used to implement the steps of the loop unrolling method of the kernel code of the above method embodiment when executing a computer program.

[0191] Specifically, please refer to FIG. 4 , which is a schematic diagram of the specific structure of a loop unrolling device for a kernel code provided in this embodiment. The loop unrolling device for the kernel code may have relatively large differences due to different configurations or performances, and may include a processor (central processing units, CPU) 322 (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 may be configured to communicate with the memory 332, and execute a series of instruction operations in the memory 332 on the loop unrolling device 301 for the kernel code.

[0192] The loop unrolling device 301 of the kernel code may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .

[0193] The steps in the kernel code loop unrolling method described above can be implemented by the structure of the kernel code loop unrolling device.

[0194] Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:

[0195] During the neural network model inference process, the size of the output tensor of each layer of the neural network model is captured respectively; the current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer; the current unified computing device architecture code is generated according to each current expansion factor, and the current unified computing device architecture code is compiled to obtain the compiled unified computing device architecture code; the compiled unified computing device architecture code is used to loop expand the kernel code and obtain the current loop expansion time; the current expansion factors are adjusted according to the current loop expansion time to obtain the target expansion factors corresponding to each layer; the kernel code is loop expanded according to each target expansion factor.

[0196] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0197] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be elaborated here.

[0198] Corresponding to the above method embodiment, the present invention further provides a computer program product, including a computer program, which implements the steps of the loop unrolling method of the above kernel code when executed by a processor.

[0199] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device, computer-readable storage medium, and computer program product disclosed in the embodiment, since they correspond to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0200] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A loop unrolling method for kernel code, characterized in that: include: During the neural network model inference process, the size of the output tensor of each layer of the neural network model is captured respectively; Determine the current expansion factor corresponding to each layer according to the size of the output tensor of each layer; Generate a current unified computing device architecture code according to each current expansion factor, and compile the current unified computing device architecture code to obtain a compiled unified computing device architecture code; Perform loop unrolling on the kernel code using the compiled unified computing device architecture code, and obtain a current loop unrolling time; Adjusting each current expansion factor according to the current loop expansion time to obtain a target expansion factor corresponding to each layer; Performing loop expansion on the kernel code according to each target expansion factor; Adjusting each current unrolling factor according to the current loop unrolling time includes: According to the current loop unrolling time, each current unrolling factor is adjusted using a preset optimization strategy; Use preset optimization strategies to adjust the current expansion factors, including: Get the current expansion factor to be adjusted; Obtaining a preset difference between adjacent expansion factors; Generating an adjacent expansion factor of the current expansion factor to be adjusted according to the preset difference; Adjusting each current expansion factor according to the adjacent expansion factors; Adjusting each current expansion factor according to the adjacent expansion factor includes: Replacing the current expansion factor to be adjusted with the adjacent expansion factor; Use preset optimization strategies to adjust the current expansion factors, including: Randomly generate expansion factors; Using randomly generated expansion factors to adjust each current expansion factor; Use the randomly generated expansion factors to adjust the current expansion factors, including: Determine the randomly generated expansion factor as the new current expansion factor; If the current expansion factors meet the preset optimal solution determination condition, each current expansion factor is determined as the target expansion factor corresponding to each layer; The current expansion factor corresponding to each layer is determined according to the size of the output tensor of each layer, including: Get the hardware architecture characteristics of the graphics processor; The current expansion factor corresponding to each layer is determined according to the hardware architecture characteristics and the size of the output tensor of each layer.

2. The kernel code loop unrolling method according to claim 1, characterized in that: Each current expansion factor is adjusted according to the adjacent expansion factor to obtain a target expansion factor corresponding to each layer, including: Determine the adjacent expansion factor as a new current expansion factor, and repeat the step of generating a current unified computing device architecture code according to each current expansion factor; When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion; If yes, replacing the current expansion factor to be adjusted with the adjacent expansion factor; If not, retaining the current expansion factor to be adjusted; Determine whether each current expansion factor is a globally optimal expansion factor or reaches a preset convergence condition; If yes, then each current expansion factor is determined as the target expansion factor corresponding to each layer; If not, the step of generating the current unified computing device architecture code according to each current expansion factor is repeated.

3. The kernel code loop unrolling method according to claim 1, characterized in that: The current expansion factors are adjusted using the randomly generated expansion factors to obtain the target expansion factors corresponding to each layer, including: Determine the randomly generated expansion factor as a new current expansion factor, and repeat the step of generating the current unified computing device architecture code according to each current expansion factor; When the current loop expansion time corresponding to the current loop expansion is obtained, it is determined whether the current loop expansion time is less than the current loop expansion time corresponding to the previous loop expansion; If yes, the current expansion factor to be adjusted is replaced by the new current expansion factor; If not, obtaining the current iteration number, and selecting the current expansion factor from the new current expansion factor and the current expansion factor to be adjusted according to the current iteration number; Determining whether the current expansion factor meets a preset optimal solution determination condition; If yes, then each current expansion factor is determined as the target expansion factor corresponding to each layer; If not, repeat the step of randomly generating the expansion factor.

4. The kernel code loop unrolling method according to claim 1, characterized in that: The size of the output tensor of each layer of the neural network model is captured separately, including: The size of the output tensor of each layer of the neural network model is captured by the hook functions pre-registered in each layer of the neural network model.

5. The kernel code loop unrolling method according to claim 1, characterized in that: After obtaining the compiled unified computing device architecture code, it also includes: generating a target forward propagation function according to the compiled unified computing device architecture code; Replacing an original forward propagation function of the neural network model with the target forward propagation function; Accordingly, the kernel code is loop-unrolled using the compiled unified computing device architecture code, including: The compiled unified computing device architecture code is called using the target forward propagation function to perform loop unrolling on the kernel code.

6. The kernel code loop unrolling method according to claim 1, characterized in that: After obtaining the target expansion factors corresponding to each layer, it also includes: The target dynamic link library is generated by using each target expansion factor, so that the target dynamic link library is called to perform reasoning during the neural network model reasoning process.

7. The kernel code loop unrolling method according to claim 1, characterized in that: After obtaining the compiled unified computing device architecture code, it also includes: The compiled unified computing device architecture code is called to generate a current dynamic link library, so that the current dynamic link library is called to perform reasoning during the neural network model reasoning process.

8. The kernel code loop unrolling method according to any one of claims 1 to 7, characterized in that: The size of the output tensor of each layer of the neural network model is captured separately, including: The application programming interface of the automatic computing device architecture loop unrolling optimization module is called to capture the size of the output tensor of each layer of the neural network model respectively; wherein the automatic computing device architecture loop unrolling optimization module is a module integrated into the neural network model through modularization.

9. A loop unrolling device for kernel code, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the loop unrolling method of the kernel code as claimed in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the loop unrolling method of the kernel code according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the loop unrolling method of the kernel code as claimed in any one of claims 1 to 8 are implemented.

Images