Winograd automatic performance optimization method and device, computer equipment and storage medium

Abstract

The invention relates to a Winograd automatic performance optimization method and device, computer equipment and a storage medium. The method comprises the steps that an initial Winograd algorithm template based on an NHWC data arrangement format is selected, process optimization is conducted on the initial Winograd algorithm template, and a Winograd algorithm template based on a TensorGEMM + BatchGEMM + TensorGEMM calculation mode is obtained; analyzing floating point calculation amount and memory access times under each parameter template, and performing dynamic parameter selection on the Winograd algorithm template according to the floating point calculation amount and the memory access times to obtain a Winograd algorithm template with optimal parameters; and carrying out automatic performance optimization on the Winograd algorithm template of the optimal parameter by utilizing an Autotuning module Ansor of a TVM (Transport Vector Module). According to the embodiment of the invention, convolution calculation of various uncommon sizes can be efficiently completed, and good performance can be maintained on different hardware architectures and convolution sizes.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and specifically relates to a Winograd automatic performance optimization method, apparatus, computer device, and storage medium. Background Technology

[0002] In recent years, with the rapid development of deep learning, CNNs (Convolutional Neural Networks) have become a core technology for handling complex visual tasks, widely used in image recognition, object detection, autonomous driving, and many other fields. CNN models typically rely on large amounts of data and complex mathematical operations, especially during the model training phase, which requires enormous computing resources. However, with the widespread application of deep learning technology, such as in autonomous vehicles, intelligent monitoring, and personalized medicine, the efficiency and energy efficiency of model inference have become a research focus. Model inference mainly occurs in data centers or on edge devices, scenarios that place stringent requirements on computational efficiency, energy consumption, and real-time performance.

[0003] While GPUs (graphics processing units) and dedicated AI (artificial intelligence) accelerators have demonstrated superior performance in handling deep learning tasks, they still face limitations in terms of cost, energy consumption, and availability. In contrast, CPUs (central processing units), as a more general-purpose computing platform, are widely deployed in various environments, making them a crucial platform for inference tasks. The ubiquitous availability of CPUs means that deep learning models can be deployed on existing infrastructure without requiring additional hardware. Furthermore, the energy efficiency and performance of CPUs have improved significantly in recent years, particularly on x86 and ARM architectures, making them strong candidates for running inference tasks.

[0004] However, in CNN computation, over 90% of the computation is concentrated in the implementation of convolutional layers. Therefore, the computational performance of convolutional layers almost determines the overall performance of the convolutional neural network. Furthermore, convolution is not limited to convolutional neural networks; it is involved in many other network models as well. Thus, the performance of convolution has a significant impact on the entire field of deep learning.

[0005] Currently, there are four main methods for implementing convolution computation: direct convolution, Generalized Matrix Multiplication (GEMM), Fast Fourier Transform (FFT), and Winograd. Direct convolution, according to the definition of convolution, achieves convolution by sliding a convolution kernel across the input tensor and calculating the dot product between the kernel and the input block at each position; however, its computational performance is relatively poor. GEMM-based algorithms, also known as Im2col, map the input tensor to a row- or column-major matrix through format conversion, transforming the convolution operation into a GEMM operation. Using this matrix format, the convolution operation can be performed as a single matrix multiplication, leveraging highly optimized GEMM operations accelerated using the highly optimized Basic Linear Algebra Assembly (BLAS) library. However, this method increases memory consumption, and the tensor-to-matrix conversion can lead to irregular shapes, often resulting in suboptimal performance. The core idea of ​​FFT is to perform convolution operations in the frequency domain, reducing floating-point computation. It can achieve good performance with large convolution kernels, but currently, the convolution kernels used in convolution operations are relatively small, leading to poor performance. The core idea of ​​the Winograd algorithm is to introduce more additions to replace multiplication calculations. It performs well in most scenarios, which has made it widely noticed and studied in both academia and industry.

[0006] Currently, research on the Winograd algorithm mainly focuses on its generalization, extension, and implementation on various architectures. In terms of algorithm optimization, employing mathematical methods to overcome the limitations of the Winograd algorithm is crucial, but this process requires in-depth proof and precise derivation by specialized mathematicians. Therefore, exploring hardware-friendly optimization methods has become an important direction for future research. However, efficiently implementing the Winograd algorithm on devices with different hardware characteristics remains a challenge, especially considering that developing a Winograd algorithm specifically adapted for various hardware is neither economical nor practical. Furthermore, the diversity of deep learning models introduces additional complexity. The size of the input image and the dimensions of the input and output channels differ in each model, resulting in significant differences in the shape and size of the parameters of different convolutional layers. Since matrices of different shapes and sizes require completely different optimal computational implementation methods, developing a high-performance Winograd algorithm also requires solving the performance portability problem across different input shapes of convolutional layers. In this context, developing a Winograd algorithm that can automatically adapt to different input shapes and provide high performance on various hardware becomes extremely important. Summary of the Invention

[0007] This application provides a Winograd automatic performance optimization method, apparatus, computer device, and storage medium, which aims to at least partially solve one of the aforementioned technical problems in the prior art.

[0008] To address the above problems, this application provides the following technical solution:

[0009] A Winograd automatic performance optimization method includes:

[0010] Select an initial Winograd algorithm template based on the NHWC data arrangement format, and optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method.

[0011] Analyze the floating-point computation and memory access count under each parameter template, and dynamically select the parameters of the Winograd algorithm template based on the floating-point computation and memory access count to obtain the Winograd algorithm template with optimal parameters;

[0012] The Winograd algorithm template with the optimal parameters is automatically optimized using the TVM's Auto-tuning module Ansor.

[0013] The technical solution adopted in this application embodiment further includes: optimizing the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method, specifically as follows:

[0014] The Hadamard product operation in the initial Winograd algorithm template is converted to BatchGEM, and the input transformation and output transformation in the initial Winograd algorithm template are both changed to TensorGEMM. This allows the optimized Winograd algorithm template to use TensorGEMM, BatchGEM, and TensorGEMM computation methods alternately in the three stages of input transformation, Hadamard product, and output transformation, resulting in a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM computation method.

[0015] The technical solution adopted in this application embodiment also includes: the calculation process of the Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method includes:

[0016] B T d[ts][ts][TN][C]=B T [ts][ts_r]×d[N][C][H][W]

[0017] V=BT dB[ts][ts][TN][C]=B T d[ts][ts_r][TN]

[0018] The above formula corresponds to the two TensorGEMMs of the input transformation;

[0019] M[ts][ts][TN][K]=V[ts][ts][TN][C_r]×U[ts][ts][C_r][K]

[0020] The above formula corresponds to the BatchGEMM transformed from the Hadamard product operation;

[0021] A T M[m][ts][TN][K]=A T [m][ts_r]×M[ts_r][ts][TN][K]

[0022] O = A T MA[N][E][F][K]=B T [ts][ts_r]×d[N][C][H][W]

[0023] The above formula corresponds to the two TensorGEMMs of the output transformation;

[0024] In the above formulas, d is the input tensor with data format NHWC, N is the input batch size, C is the number of input channels, HW is the input length and width, O is the output tensor with data format NEFK, K is the output channel size, EF is the output length and width, U is the tensor after the pre-calculated convolution kernel transformation with data format tstsCK, ts is the length and width of the matrix sliced ​​out by the Winograd algorithm in each calculation process, in F(m×m,r×r), ts=m+r-1, TN is the number of matrices sliced ​​out by the Winograd algorithm in each calculation process, in F(m×m,r×r), TN=(E / m)×(F / m), and all parameters with _r refer to the cumulative summation of GEMM in this dimension.

[0025] The technical solution adopted in this application embodiment further includes: after selecting an initial Winograd algorithm template based on the NHWC data arrangement format and optimizing the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method, it further includes:

[0026] The Winograd algorithm template, which has already undergone process optimization, is further optimized by employing offline computation of convolution kernel transformation, rearrangement and computation fusion, as well as pre-rearrangement of convolution kernels.

[0027] The technical solution adopted in this application embodiment also includes: the analysis of floating-point computation and memory access times under each parameter template, specifically:

[0028] The floating-point computation and memory access counts of the Winograd algorithm template in the entire computation process are as follows:

[0029] FLOPs = (TN × C × ts) 3 ×2)+(TN×C×K×ts 2 )+(TN×K×ts×m×(ts+m))

[0030] MACs=(1×C×H×W+ts×ts+2×ts×ts×TN×C)+

[0031] (ts×ts×TN×C+ts×ts+2×ts×ts×TN×C)+

[0032] (ts×ts×C×K+ts×ts×C×TN+2×ts×ts×K×TN)+

[0033] (ts×ts×K×TN+ts×m+2×m×ts×K×TN)+

[0034] (ts×m×TN×K+ts×m+2×1×K×E×F)

[0035] The technical solution adopted in this application embodiment further includes: dynamically selecting parameters for the Winograd algorithm template based on the floating-point computational complexity and memory access frequency to obtain the Winograd algorithm template with optimal parameters, specifically as follows:

[0036] The parameters of the Winograd algorithm template are represented as F(m×m, r×r), and the formula for selecting the parameters of the Winograd algorithm template is:

[0037]

[0038] In the formula F f63 F represents the floating-point computation cost corresponding to F(6×6, 3×3). f43 M represents the floating-point computation cost corresponding to F(4×4, 3×3). f63 M represents the number of memory accesses corresponding to F(6×6, 3×3). f43 This represents the number of memory accesses corresponding to F(4×4, 3×3).

[0039] Another technical solution adopted in this application embodiment is: a Winograd automatic performance optimization device, comprising:

[0040] Process optimization module: used to select an initial Winograd algorithm template based on the NHWC data layout format, and to optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method.

[0041] Parameter selection module: used to analyze the floating-point computation and memory access count under each parameter template, and dynamically select the parameters of the Winograd algorithm template based on the floating-point computation and memory access count to obtain the Winograd algorithm template with the optimal parameters;

[0042] Performance optimization module: Used to automatically optimize the performance of the Winograd algorithm template with the optimal parameters using the TVM Auto-tuning module Ansor.

[0043] The technical solution adopted in this application embodiment also includes:

[0044] Template optimization module: This module is used to further optimize the Winograd algorithm template after the process optimization by employing offline calculation of convolution kernel transformation, rearrangement and calculation fusion, as well as convolution kernel pre-rearrangement algorithms.

[0045] Another technical solution adopted in this application embodiment is: a computer device, the computer device including a processor and a memory coupled to the processor, wherein,

[0046] The memory stores program instructions for implementing the Winograd automatic performance optimization method;

[0047] The processor is used to execute the program instructions stored in the memory to control the Winograd automatic performance optimization method.

[0048] Another technical solution adopted in this application embodiment is: a storage medium storing processor-executable program instructions, the program instructions being used to execute the Winograd automatic performance optimization method.

[0049] Compared to existing technologies, the beneficial effects of the embodiments of this application are as follows: The Winograd automatic performance optimization method, apparatus, computer equipment, and storage medium of the embodiments of this application, by reasonably selecting data arrangement and interleaving BatchGEMM and TensorGEMM, design a reconfigurable high-performance initial Winograd algorithm template of TensorGEMM+BatchGEMM+TensorGEMM. Through this initial Winograd algorithm template, with the help of the TVM auto-tuning mechanism, convolution calculations of various uncommon sizes can be completed efficiently. The embodiments of this application also provide a reconfigurable Winograd algorithm library, which can extract several key performance-related runtime parameters from the Winograd algorithm, and generate code with better performance for different hardware architectures and convolution sizes, thereby completing convolution calculations efficiently. Based on the reconfigurable Winograd algorithm library, the embodiments of this application, with the help of the TVM auto-tuning mechanism, search for the optimal parameter combination under the current hardware architecture and convolution size from all parameter combinations, so that the algorithm maintains a good performance on different hardware architectures and convolution sizes. Attached Figure Description

[0050] Figure 1 This is a flowchart of the Winograd automatic performance optimization method according to an embodiment of this application;

[0051] Figure 2 This is a schematic diagram of the logical and physical storage methods of NCHW and NHWC;

[0052] Figure 3 A schematic diagram showing the theoretical speedup ratio, input tensor dilation, and convolutional kernel dilation trends at different m values;

[0053] Figure 4 This is a schematic diagram illustrating the experimental results comparing the performance of embodiments of this application with the Winograd algorithm in FastConv and ARM NN;

[0054] Figure 5 This is a schematic diagram illustrating the experimental results comparing the performance of the embodiments of this application with the Winograd algorithm in MNN;

[0055] Figure 6 This is a schematic diagram of the Winograd automatic performance optimization device according to an embodiment of this application;

[0056] Figure 7 This is a schematic diagram of the computer device structure according to an embodiment of this application;

[0057] Figure 8 This is a schematic diagram of the structure of the storage medium according to an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0059] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or computer device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or computer devices.

[0060] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0061] Specifically, please refer to Figure 1 This is a flowchart of the Winograd automatic performance optimization method according to an embodiment of this application. The Winograd automatic performance optimization method according to an embodiment of this application includes the following steps:

[0062] S10: Select the initial Winograd algorithm template based on the NHWC data arrangement format, and optimize the process of the initial Winograd algorithm template to obtain the Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method.

[0063] In this step, since data in current deep learning frameworks is typically 4-dimensional, there are various different data arrangement formats, such as the NCHW data arrangement in Caffe and PyTorch, and the NHWC data arrangement in TensorFlow. The logical and physical storage methods of NCHW and NHWC are as follows... Figure 2 As shown, although the stored data is the same, different storage orders lead to inconsistent data access characteristics. Therefore, even when performing the same operations on the data, the corresponding computational performance will differ. Comparative analysis reveals that in the NCHW data layout format, the number of channels (C) immediately follows the batch size (N). During convolution operations, memory needs to be accessed across the channel dimension, and memory addresses may jump between different channels, resulting in discontinuous memory access. This discontinuous memory access may lead to cache misses and memory access latency, thereby reducing algorithm performance. In contrast, in the NHWC data layout format, the number of channels (C) is placed last. During convolution operations, memory can be accessed continuously because memory addresses change continuously according to the height and width order, improving cache hit rate and memory access efficiency, thus improving algorithm performance. Therefore, this application embodiment selects the initial Winograd algorithm template based on the NHWC data layout format.

[0064] Furthermore, the optimization method of the initial Winograd algorithm template process in this embodiment is as follows: The initial Winograd algorithm template includes four stages: input transformation, convolution kernel transformation, Hadamard product, and output transformation. During the inference process of the convolutional neural network, the convolution kernel is known and constant; therefore, the tensor after the convolution kernel transformation can be calculated in advance. In the Hadamard product operation stage, since this application adopts the NHWC data layout format, accumulation needs to be performed on the input channels when performing the Hadamard product operation of the Winograd algorithm. To ensure minimal data movement overhead and better match the NHWC data layout format, this embodiment converts the Hadamard product operation in the initial Winograd algorithm template to BatchGEMM to ensure that the number of input channels remains in the innermost dimension throughout the input and output transformation processes. Furthermore, since the accumulation dimension of the Hadamard product operation is in the input channel, while the accumulation dimension of the input transformation and output transformation is the width of a constant matrix, corresponding to the dimension of HW in the input (NHWC), if the input transformation and output transformation use the same GEMM implementation as the Hadamard product operation, it will inevitably introduce the overhead of a new data arrangement transformation. Therefore, in order to better match BatchGEMM, the embodiments of this application change the input transformation and output transformation in the initial Winograd algorithm template to TensorGEMM, so that the Winograd algorithm uses different GEMM implementations in the three stages of input transformation, Hadamard product and output transformation, i.e., TensorGEMM+BatchGEMM+TensorGEMM implementation, thereby greatly reducing the memory access overhead of the Winograd algorithm.

[0065] Based on the above, the calculation process of the optimized Winograd algorithm template is as follows:

[0066] B T d[ts][ts][TN][C]=B T [ts][ts_r]×d[N][C][H][W] (1)

[0067] V=B T dB[ts][ts][TN][C]=B T d[ts][ts_r][TN] (2)

[0068] M[ts][ts][TN][K]=V[ts][ts][TN][C_r]×U[ts][ts][C_r][K] (3)

[0069] A T M[m][ts][TN][K]=A T[m][ts_r]×M[ts_r][ts][TN][K] (4)

[0070] O = A T MA[N][E][F][K]=B T [ts][ts_r]×d[N][C][H][W] (5)

[0071] In equations (1) to (5), (1) and (2) correspond to the two TensorGEMMs of the input transformation, (4) and (5) correspond to the two TensorGEMMs of the output transformation, and equation (3) corresponds to the BatchGEMM transformed from the Hadamard product. In the equations, d is the input tensor with NHWC data format, N is the input batch size, C is the number of input channels, and HW is the input width and height. O is the output tensor with NEFK data format, K is the output channel size, and EF is the output width and height. U is a tensor transformed by a pre-calculated convolution kernel, with the data format tstsCK. ts represents the length and width of the matrix sliced ​​out by the Winograd algorithm at each step of the computation, where ts = m + r - 1 in F(m×m, r×r). TN is the number of matrices sliced ​​out by the Winograd algorithm at each step, where TN = (E / m) × (F / m) in F(m×m, r×r). All parameters with _r indicate the summation of GEMMs along this dimension. In this embodiment, N is assumed to be 1, Pad to be 1, and Stride to be 1. The optimized Winograd algorithm template has only 5 steps, which corresponds to the minimum number of execution steps of the Winograd algorithm. Furthermore, the data arrangement format remains largely unchanged throughout the computation, eliminating unnecessary data arrangement transformation overhead.

[0072] S20: The Winograd algorithm template after process optimization is further optimized by using optimization algorithms such as offline calculation of convolution kernel transformation, rearrangement and calculation fusion, and early rearrangement of convolution kernel;

[0073] In this step, for the Winograd algorithm template after process optimization, this application embodiment further optimizes the Winograd algorithm template by adding various optimization methods such as offline calculation of convolution kernel transformation, rearrangement and calculation fusion, and early rearrangement of convolution kernel, so as to further improve the performance of the Winograd algorithm template.

[0074] Specifically, further optimization algorithms for the Winograd algorithm template after process optimization include:

[0075] S21: Analyze and determine the necessity of offline computation of convolution kernel transformation, and perform offline computation;

[0076] S22: Rearrange and merge the convolution kernels to reduce redundant operations in computation;

[0077] S23: Rearrange the convolution kernel in advance to optimize the computational performance of the convolution kernel.

[0078] Further optimization of the Winograd algorithm template using the above optimization algorithms can improve the overall performance of the Winograd algorithm template.

[0079] S30: Dynamically select parameters for the optimized Winograd algorithm template based on the floating-point computation and memory access times under different parameter templates to obtain the Winograd algorithm template with the optimal parameters;

[0080] In this step, since the floating-point computation and memory access count under different parameter templates have a significant impact on the performance of the Winograd algorithm template, this embodiment of the application analyzes the floating-point computation and memory access count under different parameter templates and sets a selection function to dynamically select parameters for the optimized Winograd algorithm template based on the floating-point computation and memory access count under different parameter templates, thereby obtaining the Winograd algorithm template with the optimal parameters.

[0081] Specifically, according to formulas (1) to (5), the floating-point computations (FLOPs) and memory accesses (MACs) of the Winograd algorithm template in the entire computation process of this application embodiment are listed as follows:

[0082] FLOPs = (TN × C × ts) 3 ×2)+(TN×C×K×ts 2 )+(TN×K×ts×m×(ts+m)) (6)

[0083] MACs=(1×C×H×W+ts×ts+2×ts×ts×TN×C)+(ts×ts×TN×C+ts×ts+2×ts×ts×TN×C)+(ts×ts×C×K+ts ×ts×C×TN+2×ts×ts×K×TN)+(ts×ts×K×TN+ts×m+2×m×ts×K×TN)+(ts×m×TN×K+ts×m+2×1×K×E×F) (7)

[0084] The parameters of the Winograd algorithm template are usually represented as F(m×m, r×r), and there are many different parameter choices. For r×r, the most common kernel sizes in current CNN networks include 3×3, 5×5, and 1×1. The proportion of different convolutional kernel sizes in current common CNN networks is shown in Table 1 below:

[0085] Table 1. Proportion of different convolutional kernels in common CNN networks

[0086]

[0087] As shown in Table 1, Winograd has no acceleration effect when the kernel size is 1×1. Therefore, this embodiment mainly considers 3×3. Additionally, it can be seen that VGG-16 consists entirely of 3×3 convolutional kernels. After using the Winograd algorithm, the VGG-16 network has a better acceleration effect. Therefore, this embodiment mainly selects the VGG-16 network.

[0088] Based on the Winograd fast convolution algorithm, the theoretical speedup formula for different m and r parameters in F(m×m,r×r) can be obtained as follows:

[0089]

[0090] Different values ​​of m and r will result in expansion of both the input tensor and the convolution kernel tensor, requiring targeted selection. The specific expansion formula is as follows:

[0091]

[0092] Table 2 below shows the theoretical speedup and expansion ratios of the input tensor and the convolutional kernel tensor for common m×m block sizes (r×r=3×3), i.e., convolutional kernel sizes of 3×3:

[0093] Table 2. Effects of different chunk sizes on Wiongrad

[0094]

[0095]

[0096] According to formula (8), when r×r=3×3, and m>>r, the theoretical speedup can only approach 9 and cannot be larger than that. Even if the input tensor remains basically unchanged, the expansion of the convolution kernel is very large. Figure 3 The diagram shows the theoretical speedup ratio, input tensor dilation, and convolutional kernel dilation trends under different values ​​of m. From... Figure 3As can be seen, the rate at which the theoretical speedup increases drops sharply with increasing m, while the expansion factor of the convolution kernel increases dramatically. Therefore, m=2, 4, and 6 were initially chosen to implement the Wiongrad template in this embodiment. However, since the 2.25x speedup at m=2 is masked by the input-output transformation, it doesn't offer a significant advantage over direct convolution in practice. Therefore, m=4 and 6 were chosen to implement the Wiongrad template in this embodiment. Odd-numbered cases were not considered here because of the special nature of the constant matrix in the Winograd algorithm's calculation process; there are certain patterns between adjacent rows or columns. Table 3 below lists the FLOPs and MACs of the Winograd algorithm templates for different m values ​​in VGG-16, serving as the basis for selecting templates with different parameters:

[0097] Table 3 shows the effect of different block sizes m on FLOPs and MACs in VGG-16.

[0098]

[0099] Ideally, the optimal choice is to minimize both FLOPs and MACs. If FLOPs are small but MACs are large, then FLOPs should be prioritized. The specific selection formula is as follows:

[0100]

[0101] In the formula F f63 Let F(6×6, 3×3) represent the number of FLOPs. f43 Let M represent the FLOPs corresponding to F(4×4, 3×3). f63 Let M represent the MACs corresponding to F(6×6, 3×3). f43 This represents the MACs corresponding to F(4×4,3×3).

[0102] Based on Table 3 and Formula (11), in this embodiment of the application, the first and last layers of the VGG-16 network are selected with the Winograd algorithm template of parameter F(4×4,3×3), and the other layers are selected with the Winograd algorithm template of parameter F(6×6,3×3).

[0103] S40: Utilize TVM's Auto-tuning module Ansor to automatically optimize the performance of the Winograd algorithm template;

[0104] In this step, the TVM Auto-tuning module Ansor is configured and used to perform fine-grained automatic performance optimization of the Winograd algorithm template. Performance data under different hardware environments is collected, and after analyzing the performance data, the parameters and implementation of the Winograd algorithm template are adjusted to achieve efficient inference and optimal performance under different hardware environments and workloads.

[0105] Based on the above, the Winograd automatic performance optimization method of this application, by reasonably selecting data arrangement and interleaving BatchGEMM and TensorGEMM, designs a reconfigurable high-performance Winograd algorithm template of TensorGEMM+BatchGEMM+TensorGEMM. Through this Winograd algorithm template, leveraging the TVM's auto-tuning mechanism, convolution calculations of various uncommon sizes can be efficiently completed. This application also provides a reconfigurable Winograd algorithm library, which can extract several key performance-related runtime parameters from the Winograd algorithm. This library can generate higher-performance code for different hardware architectures and convolution sizes, thereby efficiently completing convolution calculations. Based on the reconfigurable Winograd algorithm library, this application, with the help of the TVM's auto-tuning mechanism, searches for the optimal parameter combination from all parameter combinations for the current hardware architecture and convolution size, thus ensuring that the algorithm maintains good performance across different hardware architectures and convolution sizes.

[0106] To demonstrate the feasibility and effectiveness of the embodiments of this application, a performance comparison was performed between the embodiments of this application and the Winograd algorithm in FastConv and ARMNN, specifically as follows: Figure 4 The diagram shown illustrates the experimental results comparing the performance of this application's embodiments with that of FastConv and the Winograd algorithm in ARMNN. Figure 4 As can be seen, the embodiments of this application have a speedup of 1.27 times to 6.11 times compared to Arm NN on ARM CPU. Compared to FastConv, which has been manually optimized by experts, it has a speedup of 1.04 times to 4.28 times when the first layer of VGG-16 (C=3, K=64, H / W=224) is not considered. The reason why the first layer of VGG-16 is not as good as FastConv is that C and K are too small, so the overhead of the input transformation and output transformation process cannot be evenly distributed by C and K. However, FastConv has a special optimization for reducing the amount of computation in the input transformation and output transformation process by merging like terms when multiplying the constant matrix due to the special properties of the constant matrix.

[0107] Furthermore, a performance comparison was conducted between the embodiments of this application and the Winograd algorithm in MNN (a lightweight deep neural network engine developed by Alibaba), which is at the state-of-the-art (SOTA) level on x86 CPUs. Specifically, as follows... Figure 5 The diagram shown illustrates the experimental results comparing the performance of this application's embodiment with the Winograd algorithm in MNN. From... Figure 5 It can be seen that even compared to the current state-of-the-art MNNs, the embodiments of this application still have a speedup of 1.10 to 2.72 times under the selected input.

[0108] Please see Figure 6 This is a schematic diagram of the Winograd automatic performance optimization device according to an embodiment of this application. The Winograd automatic performance optimization device 40 according to an embodiment of this application includes:

[0109] Process optimization module 41: used to select an initial Winograd algorithm template based on the NHWC data layout format, and to optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method.

[0110] Template optimization module 42: is used to further optimize the Winograd algorithm template after process optimization by employing offline calculation, rearrangement and calculation fusion of convolution kernel transformation, and convolution kernel pre-rearrangement algorithm;

[0111] Parameter selection module 43: used to analyze the floating-point computation and memory access times under each parameter template, and dynamically select parameters for the Winograd algorithm template based on the floating-point computation and memory access times to obtain the Winograd algorithm template with optimal parameters;

[0112] Performance optimization module 44: Used to automatically optimize the performance of the Winograd algorithm template with the optimal parameters using the TVM's Auto-tuning module Ansor.

[0113] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0114] The apparatus provided in this application can be applied to the foregoing method embodiments. For details, please refer to the description of the above method embodiments, which will not be repeated here.

[0115] Please see Figure 7This is a schematic diagram of a computer device structure according to an embodiment of this application. The computer device 50 includes:

[0116] Memory 51 storing executable program instructions;

[0117] Processor 52 connected to memory 51;

[0118] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: select an initial Winograd algorithm template based on the NHWC data layout format, and optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method; analyze the floating-point computation volume and memory access count under each parameter template, and dynamically select the parameters of the Winograd algorithm template according to the floating-point computation volume and memory access count to obtain the optimal parameter Winograd algorithm template; and use the TVM's Auto-tuning module Ansor to automatically optimize the performance of the optimal parameter Winograd algorithm template.

[0119] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0120] Please see Figure 8This is a schematic diagram of the structure of the storage medium in an embodiment of this application. The storage medium in this embodiment stores program instructions 61 capable of implementing the following steps: selecting an initial Winograd algorithm template based on the NHWC data layout format, and optimizing the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method; analyzing the floating-point computation volume and memory access count under each parameter template, and dynamically selecting parameters for the Winograd algorithm template based on the floating-point computation volume and memory access count to obtain the optimal parameter Winograd algorithm template; and automatically optimizing the performance of the optimal parameter Winograd algorithm template using the TVM's Auto-tuning module Ansor. The program instructions 61 can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network computer device, etc.) or processor to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program instructions, or terminal computer devices such as computers, servers, mobile phones, and tablets. Servers can be standalone servers or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0121] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A Winograd automatic performance optimization method, characterized in that, include: Select an initial Winograd algorithm template based on the NHWC data arrangement format, and optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method. Analyze the floating-point computation and memory access count under each parameter template, and dynamically select the parameters of the Winograd algorithm template based on the floating-point computation and memory access count to obtain the Winograd algorithm template with optimal parameters; The Winograd algorithm template with the optimal parameters is automatically optimized using the TVM's Auto-tuning module Ansor.

2. The Winograd automatic performance optimization method according to claim 1, characterized in that, The process optimization of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM computation method is as follows: The Hadamard product operation in the initial Winograd algorithm template is converted to BatchGEM, and the input transformation and output transformation in the initial Winograd algorithm template are both changed to TensorGEMM. This allows the optimized Winograd algorithm template to use TensorGEMM, BatchGEM, and TensorGEMM computation methods alternately in the three stages of input transformation, Hadamard product, and output transformation, resulting in a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM computation method.

3. The Winograd automatic performance optimization method according to claim 2, characterized in that, The computation process of the Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM computation method includes: B T d[ts][ts][TN][C]=B T [ts][ts_r]×d[N][C][H][W] V=B T dB[ts][ts][TN][C]=B T d[ts][ts_r][TN] The above formula corresponds to the two TensorGEMMs of the input transformation; M[ts][ts][TN][K]=V[ts][ts][TN][C_r]×U[ts][ts][C_r][K] The above formula corresponds to the BatchGEMM transformed from the Hadamard product operation; A T M[m][ts][TN][K]=A T [m][ts_r]×M[ts_r][ts][TN][K] O=A T MA[N][E][F][K]=B T [ts][ts_r]×d[N][C][H][W] The above formula corresponds to the two TensorGEMMs of the output transformation; In the above formulas, d is the input tensor with data format NHWC, N is the input batch size, C is the number of input channels, HW is the input length and width, O is the output tensor with data format NEFK, K is the output channel size, EF is the output length and width, U is the tensor after the pre-calculated convolution kernel transformation with data format tstsCK, ts is the length and width of the matrix sliced ​​out by the Winograd algorithm in each calculation process, in F(m×m,r×r), ts=m+r-1, TN is the number of matrices sliced ​​out by the Winograd algorithm in each calculation process, in F(m×m,r×r), TN=(E / m)×(F / m), and all parameters with _r refer to the cumulative summation of GEMM in this dimension.

4. The Winograd automatic performance optimization method according to any one of claims 1 to 3, characterized in that, After selecting an initial Winograd algorithm template based on the NHWC data layout format and optimizing the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method, the process further includes: The Winograd algorithm template, which has already undergone process optimization, is further optimized by employing offline computation of convolution kernel transformation, rearrangement and computation fusion, as well as pre-rearrangement of convolution kernels.

5. The Winograd automatic performance optimization method according to claim 4, characterized in that, The analysis of floating-point computation and memory access frequency under each parameter template is as follows: The floating-point computation and memory access counts of the Winograd algorithm template in the entire computation process are as follows: FLOPs=(TN×C×ts 3 ×2)+(TN×C×K×ts 2 )+(TN×K×ts×m×(ts+m)) MACs=(1×C×H×W+ts×ts+2×ts×ts×TN×C)+ (ts×ts×TN×C+ts×ts+2×ts×ts×TN×C)+ (ts×ts×C×K+ts×ts×C×TN+2×ts×ts×K×TN)+ (ts×ts×K×TN+ts×m+2×m×ts×K×TN)+ (ts×m×TN×K+ts×m+2×1×K×E×F).

6. The Winograd automatic performance optimization method according to claim 5, characterized in that, The step of dynamically selecting parameters for the Winograd algorithm template based on the floating-point computational complexity and memory access frequency to obtain the optimal Winograd algorithm template is as follows: The parameters of the Winograd algorithm template are represented as F(m×m, r×r), and the formula for selecting the parameters of the Winograd algorithm template is: In the formula F f63 F represents the floating-point computation cost corresponding to F(6×6, 3×3). f43 M represents the floating-point computation cost corresponding to F(4×4, 3×3). f63 M represents the number of memory accesses corresponding to F(6×6, 3×3). f43 This represents the number of memory accesses corresponding to F(4×4, 3×3).

7. A Winograd automatic performance optimization device, characterized in that, include: Process optimization module: used to select an initial Winograd algorithm template based on the NHWC data layout format, and to optimize the process of the initial Winograd algorithm template to obtain a Winograd algorithm template based on the TensorGEMM+BatchGEMM+TensorGEMM calculation method. Parameter selection module: used to analyze the floating-point computation and memory access count under each parameter template, and dynamically select the parameters of the Winograd algorithm template based on the floating-point computation and memory access count to obtain the Winograd algorithm template with the optimal parameters; Performance optimization module: Used to automatically optimize the performance of the Winograd algorithm template with the optimal parameters using the TVM Auto-tuning module Ansor.

8. The Winograd automatic performance optimization device according to claim 7, characterized in that, Also includes: Template optimization module: This module is used to further optimize the Winograd algorithm template after the process optimization by employing offline calculation of convolution kernel transformation, rearrangement and calculation fusion, as well as convolution kernel pre-rearrangement algorithms.

9. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, wherein, The memory stores program instructions for implementing the Winograd automatic performance optimization method according to any one of claims 1-6; The processor is used to execute the program instructions stored in the memory to control the Winograd automatic performance optimization method.

10. A storage medium, characterized in that, The device stores processor-executable program instructions for performing the Winograd automatic performance optimization method according to any one of claims 1 to 6.

Images