ARMv8 architecture-oriented two-dimensional and three-dimensional Winograd convolution optimization method and system

Abstract

The invention discloses a two-dimensional and three-dimensional Winograd convolution optimization method and system for an ARMv8 architecture, and belongs to the technical field of artificial intelligence neural network calculation acceleration. In order to solve the technical problems of insufficient support of Winograd convolution in three-dimensional convolution, mixing precision and various filter sizes, low domain conversion and parallel efficiency and the like, assembly-level microkernel design, data multiplexing optimization, a double-buffer GEMM kernel and a multi-dimensional parallel strategy are mainly adopted. According to the method, high-performance calculation under different convolution scenes can be realized, and the operation efficiency and adaptability of Winograd convolution in an ARMv8 architecture are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence neural network computing acceleration technology, and specifically relates to a two-dimensional and three-dimensional Winograd convolution optimization method and system for ARMv8 architecture. Background Art

[0002] Winograd convolution is a fast convolution algorithm that maps the input data into multiple image sub-blocks (tiles) of predetermined sizes from the spatial domain to the Winograd domain, reducing the number of arithmetic operations compared to direct convolution, thereby improving the computational efficiency of the convolution operation. In theory, Winograd convolution can be extended to any dimension to support the widely used two-dimensional and three-dimensional convolutions. Taking three-dimensional convolution as an example, assuming that t is the row-major index value of the tile into which the input data is divided, and b is the batch index value, the output of the t-th tile of the b-th batch and the k-th output channel is It can be expressed as:

[0003]

[0004] Where ⊙ represents element-by-element multiplication; B, G, and A are the transformation matrices of input, filter, and output, respectively, which can be generated using the Chinese remainder theorem; g = F k,c , that is, the k-th output channel and c-th input channel data of the filter tensor F; d is the element of the t-th tile of the input data in the b-th batch and the c-th input channel; the symbol R represents an operation similar to rotating the tile of the two-dimensional Winograd domain by 90° around the y-axis to perform a third-dimensional domain transformation. U k,c and V b,c are the filter tile and input tile mapped to the Winograd domain (corresponding to the bth batch, k output channels and c input channels), respectively.

[0005] In order to improve the computational intensity, the middle U k,c and The element-by-element multiplication (level BLAS operation) of is regarded as a batch matrix multiplication, and the specific rewriting for batch b is as follows:

[0006]

[0007] Among them, x represents the row-major index of the element in the tile, and its value range is (m and r indicate the use of an r 3 Calculate the filter size m 3 output elements), which is also the batch value where the element-by-element multiplication is treated as a batch matrix multiplication.

[0008] The implementation of Winograd convolution can be divided into three stages: domain conversion of the input and convolution kernel, batch matrix multiplication, and inverse conversion of the output result. Existing technologies mostly apply Winograd algorithms to optimize two-dimensional convolutions, but lack support for three-dimensional Winograd convolutions. The existing implementation disclosed by Zhen Jia et al. (see Jia Z, Zlateski A, Durand F, et al. Optimizing N-dimensional, winograd-based convolution for many-core CPUs [C] / / Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. 2018: 109-123.) recursively uses the one-dimensional Winograd algorithm to implement N-dimensional domain conversion, without making targeted optimizations for two-dimensional and three-dimensional domain conversions. In the existing Winograd convolution implementation for the ARMv8 architecture, only the matrix multiplication part is optimized using the underlying assembly instructions, while the domain conversion is usually implemented using compiler built-in functions (intrinsic), which makes it difficult to improve the conversion efficiency through more sophisticated register and instruction control (see Meng J, Zhuang C, Chen P, et al. Automatic generation of high-performance convolution kernels on ARM CPUs for deep learning [J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 33 (11): 2885-2899.); for the domain conversion of the input data, the inherent feature of element overlap between adjacent tiles of the algorithm is ignored.

[0009] Existing technologies for applying Winograd convolutions to the ARMv8 architecture still face several shortcomings. First, most implementations are limited to accelerating two-dimensional convolutions and lack effective support for three-dimensional convolution scenarios. In particular, they lack targeted optimization methods for high-dimensional convolutions. Second, despite employing batch matrix multiplication to improve computational efficiency, the domain conversion phase still incurs the overhead of cross-row memory accesses and requires additional data packing operations, which hinders overall performance. Furthermore, existing methods fail to fully exploit the overlapping nature of elements between adjacent tiles in the input data, resulting in insufficient data reuse during the conversion phase and affecting conversion efficiency. Optimization at the microkernel level is also limited. Current solutions typically offer only a single configuration, making it difficult to adapt to varying convolution scales. Furthermore, computation and data handling are not effectively overlapped, resulting in low resource utilization. Regarding parallel processing, the use of a fixed-dimensional multi-core parallel strategy lacks flexibility, resulting in poor parallel efficiency for certain convolutional layer sizes. Finally, existing fast Winograd algorithms lack support for diverse convolution requirements, such as mixed-precision computation and varying filter sizes, limiting their widespread application in real-world scenarios. Summary of the Invention

[0010] The purpose of this invention is to address the technical problems of the existing Winograd convolution under the ARMv8 architecture, such as insufficient support for high-dimensional convolution, multiple filter sizes and mixed precision, high domain conversion overhead, and insufficiently refined matrix multiplication microkernel optimization and parallelization strategies, based on the data characteristics of common convolutional neural network models. A Winograd convolution optimization method and system for the ARMv8 architecture are proposed to significantly improve the computational efficiency of Winograd convolution and expand its applicability in various application scenarios.

[0011] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0012] A two-dimensional and three-dimensional Winograd convolution optimization method for the ARMv8 architecture includes the following steps:

[0013] 1) Perform Winograd domain conversion on input data and convolution filter data, and use an assembly-level microkernel to implement the conversion operations of input, filter, and output. During the input conversion process, the overlapping areas between tiles are used to reuse data;

[0014] 2) Perform batch matrix multiplication on the Winograd domain-converted input data and the filter data, using a GEMM microkernel with double buffering to complete the matrix multiplication operation for the corresponding channel dimension, and write the result to the output data block;

[0015] 3) Perform an inverse Winograd domain transformation on the matrix multiplication result, implement the transformation using assembly instructions, and combine it with an optimized data layout to achieve continuous memory access to generate the final convolution output data;

[0016] 4) In steps 1) to 3), a parallel thread grid is configured according to the input, output, and convolution kernel parameters, and a fusion or non-fusion algorithm framework is used to execute multi-dimensional parallel computing tasks.

[0017] Furthermore, the Winograd domain conversion of the input data in step 1) includes: performing a transformation operation on the spatial dimension of the input data using a tile-by-tile or row-by-row / column strategy, wherein the strategy is selected based on Winograd parameters m, r and the number of available vector registers; and the Winograd domain inverse conversion of the output data in step 3) includes: performing an inverse transformation operation on the spatial dimension of the output data using a tile-by-tile or row-by-column strategy, wherein the strategy is selected based on Winograd parameters m, r and the number of available vector registers.

[0018] Furthermore, in step 1), the Winograd domain conversion of the input data is divided into two microkernels for execution when processing three-dimensional convolution. The first microkernel performs domain conversion on some dimensions and stores the results in an intermediate array of a pre-designed data layout to support the second microkernel to perform efficient conversion of the remaining dimensions for continuous memory access; when processing two-dimensional convolution, the domain conversion is completed by a single microkernel in the same stage; in step 3), the Winograd domain inverse conversion of the output data is divided into two microkernels for execution when processing three-dimensional convolution. The first microkernel performs inverse transformation on some dimensions and stores the results in an intermediate array of a pre-designed data layout to support the second microkernel to perform efficient inverse conversion of the remaining dimensions for continuous memory access; when processing two-dimensional convolution, the inverse conversion is completed by a single microkernel in the same stage.

[0019] Furthermore, the Winograd domain conversion of the input data in step 1) further includes: reusing elements of adjacent tiles in the row direction in the conversion calculation through a vector register reuse mechanism to reduce the number of memory accesses.

[0020] Furthermore, during the GEMM microkernel processing in step 2), the input matrix reads continuous vector data along the input channel dimension under the same tile number, and the filter matrix reads continuous vector data along the output channel dimension under the same input channel dimension, thereby achieving efficient vectorized calculation.

[0021] Furthermore, the parameter configuration of the GEMM microkernel in step 2) includes: selecting adaptive kernel shape parameters α and β according to different precisions and vector register capacities, and combining the computational memory access ratio and register usage restrictions to construct a set of adaptive configurations including the main kernel and edge kernels to adapt to various convolution scales and boundary conditions.

[0022] Furthermore, the double buffering mechanism in step 2) includes: dividing the input data register group and the filter data register group into two groups respectively, alternately performing data loading and calculation to achieve overlap of calculation and data transfer.

[0023] Furthermore, the input data and calculation results used in the batch matrix multiplication operation in step 2) adopt a pre-designed intermediate data layout, which supports continuous memory access when reading input data and writing matrix multiplication results, avoiding data rearrangement or packing operations, and enables the Winograd domain inverse transformation microkernel in step 3) to directly read the calculation results and complete the continuous memory access conversion operation.

[0024] Furthermore, the fusion algorithm framework in step 4) includes: completing input domain conversion, batch matrix multiplication and output domain inverse conversion in sequence within the tile block, and performing fusion processing in K dimensions, and performing the corresponding output conversion operation immediately after completing the matrix multiplication of each K block.

[0025] Furthermore, step 4) specifically includes: constructing a two-dimensional or three-dimensional thread grid, dividing the spatial dimension, input channel dimension, and output channel dimension of the convolutional layer into multiple subtasks, and using OpenMP to implement static scheduling and thread group synchronization mechanism.

[0026] Furthermore, the method also includes: based on a preset recursive performance model, searching for the optimal Winograd parameter configuration and algorithm framework under a given hardware platform and convolution task scale, and selecting an execution scheme with the goal of minimizing the estimated total execution time.

[0027] A two-dimensional and three-dimensional Winograd convolution optimization system for ARMv8 architecture includes a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the above method.

[0028] The beneficial effects achieved by the present invention are as follows:

[0029] 1. This paper designs Winograd domain conversion optimization strategies for two-dimensional and three-dimensional convolution respectively, and supports multiple convolution scenarios, including single precision, half precision, and filter size 3 or 5, realizing high-performance application of Winograd algorithm in two-dimensional and three-dimensional convolution.

[0030] 2. The present invention uses assembly instructions to implement an efficient domain conversion microkernel for input, filter, and output. In the input conversion stage, data of overlapping elements of adjacent tiles is reused to reduce data loading overhead and improve conversion efficiency.

[0031] 3. The present invention customizes an optimized data layout according to the conversion process and computing requirements, improves vectorized execution efficiency, and eliminates data packing overhead before matrix multiplication through continuous memory access.

[0032] 4. The present invention designs a GEMM microkernel that supports a double-buffering mechanism and provides multiple sets of microkernel configurations adapted to different convolution scales, achieving overlap between computation and data handling, significantly improving computational resource utilization and floating-point performance.

[0033] 5. This paper constructs two algorithm frameworks, fusion and non-fusion, and combines multi-dimensional parallel strategy and thread grid scheduling mechanism to achieve efficient load balancing and parallel performance under different convolution scales and thread configurations.

[0034] 6. This invention proposes a recursive performance analysis model to guide the optimal parameter configuration and framework selection, and evaluates the calculation, handling and synchronization time under different parameter combinations through theoretical modeling, thereby achieving adaptive optimization.

[0035] 7. This invention improves the computational efficiency of Winograd convolution under the ARMv8 architecture as a whole, and enhances its support for complex convolution application scenarios such as high-dimensional convolution, mixed-precision calculations, and multiple filter configurations. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the convolution process in non-fusion and fusion mode frameworks.

[0037] Figure 2 It is a schematic diagram of the input domain conversion data reuse strategy.

[0038] Figure 3 This is a schematic diagram of the register block and ping-pong pipeline process of the 4×16 microkernel. DETAILED DESCRIPTION

[0039] In order to make the various technical features and advantages or technical effects of the above technical solutions of the present invention more obvious and easy to understand, they are described in detail below through embodiments.

[0040] This embodiment of the present invention discloses a two-dimensional and three-dimensional Winograd convolution optimization method for the ARMv8 architecture. This method uses manually optimized assembly instructions to implement microkernels for each computational stage. It supports two-dimensional and three-dimensional convolutions, applies filter sizes of 3 and 5, and is suitable for operations on 32-bit and 16-bit floating-point numbers. The data layouts for the input data and filter are NC(D)HW and KC(RD)RHRW, respectively. The output data layout after the convolution calculation is NK(D)HW.

[0041] The present invention constructs two computing frameworks: non-fusion mode and fusion mode. The corresponding computing processes are as follows: Figure 1 As shown in the figure. In non-fused mode, the input domain conversion, batch matrix multiplication, and inverse output domain conversion operations are performed independently after the tile is partitioned. In fused mode, the input domain conversion is first performed on the data within the tile block, and then the batch matrix multiplication and inverse output domain conversion are completed within the tile block. A K-dimensional fusion mechanism is introduced, that is, after completing the matrix multiplication within each K tile, the corresponding calculation result is immediately converted to the output domain. The domain conversion of the filter data is performed independently, regardless of the adopted computational framework mode.

[0042] The specific implementation steps, parallel processing strategy and parameter selection method of this method are described in detail below.

[0043] 1. Domain Conversion

[0044] The present invention designs two strategies for domain conversion of spatial dimensions (W and H). The specific strategy to be adopted depends on the Winograd parameters m, r and the available number of processor vector registers. When the vector register capacity is sufficient to simultaneously accommodate all input elements and their converted results in a tile, the tile-by-tile conversion method is preferred; if the register capacity is insufficient, the row-by-row or column-by-column conversion method is adopted. For the tile-by-tile strategy, the domain transformation is performed directly on the tile; for the row-by-row / column strategy, the tile is first multiplied by the right row (for domain conversion of input and filter) or by the left column (for inverse conversion of output) by the corresponding transformation matrix, and the result is then stored in the memory in column-major or row-major order to optimize the efficiency of subsequent data loading. Afterwards, the multiplication operation of the tile with the transformation matrix in the other dimension is completed in the same way.

[0045] To address the r-1 overlapping elements in each tile dimension during input domain conversion, this paper implements overlapping data reuse along the W dimension using assembly instructions through precise control of vector register usage, thereby reducing the amount of data loaded repeatedly and lowering memory access overhead. In a tile-by-tile strategy, reuse is achieved by sharing overlapping elements within the tile row; in a row-by-row strategy, reuse is improved by sharing overlapping elements between adjacent tiles in the same row. Figure 2 The specific implementation methods of the above two data multiplexing strategies are shown by taking m=2, r=3 and m=4, r=3 as examples.

[0046] For the three-dimensional conversion operation involved in three-dimensional convolution, the present invention uses two micro-kernels to perform the conversion process separately. The input data and filter data first complete the domain conversion (W→H) on the first two dimensions in the first micro-kernel. The conversion form can be GgG T or B T dB. After completing the inverse domain conversion of the output data of the matrix multiplication calculation, the first microkernel completes the conversion of the third dimension (i.e., D dimension). The intermediate results after the conversion are stored in a specially designed intermediate array. The array layout has been optimized according to the access pattern to ensure continuous memory access when the second microkernel performs the conversion of the remaining dimensions, thereby improving processing efficiency. The intermediate array layout corresponding to various domain conversion operations will be described in detail later. Different layout designs are suitable for the two domain conversion strategies mentioned above.

[0047] enter:

[0048] Output:

[0049] filter:

[0050] Among them C blk , K blk and T blk where θ is the tile size for input channels C, output channels K, and the total number of tiles T, respectively, to exploit cache locality; θ is the number of data elements accommodated by an ARM NEON vector register; l1, l2, and l3 are the Winograd domain tile sizes in dimensions W, H, and D, respectively. m and r have the same subscript as l. For 2D convolution, data is directly converted to the target domain within the same microkernel.

[0051] In order to ensure that the subsequent matrix multiplication stage can achieve completely continuous memory access and avoid introducing additional data packing overhead, the present invention directly writes the intermediate results of the Winograd domain into the memory in a predefined z-shaped storage format during the domain conversion of the input data and the filter data. This z-shaped format is specially designed by the present invention and can take into account both vectorized loading efficiency and data layout compactness, thereby improving the overall computing performance. After the matrix multiplication is completed, the generated Winograd domain output result is also organized according to the storage format tailored for the output inverse domain conversion microkernel to ensure efficient execution of the subsequent conversion stage. The following lists the two types of z-shaped storage formats used for input data and output data in the fusion mode and non-fusion mode of the present invention.

[0052] Winograd domain input:

[0053] Winograd domain output:

[0054] Winograd domain filter:

[0055] Where L is the number of data elements in a tile of the Winograd domain, that is, the batch value of the Winograd domain calculation completed by batch matrix multiplication; α and β represent the parameters of the matrix multiplication microkernel.

[0056] 2. Batch Matrix Multiplication

[0057] As mentioned above, in order to fully utilize cache locality, the present invention divides the input matrix and the filter matrix into T-sized blocks during the batch matrix multiplication (GEMM) process. blk ×C blk and C blk ×K blk The GEMM kernel multiplies each input sub-block (partitioned in C dimension) with the corresponding filter sub-block and accumulates the calculation results to a matrix of size T blk ×K blk in the output sub-block.

[0058] Specifically, the GEMM microkernel pair size is α×C blk The input matrix is ​​of size C blk The matrix multiplication operation is performed on the filter matrix of size α × β, and the output is a result matrix of size α × β. To improve the pipeline efficiency between calculation and memory access operations, the present invention adopts a double buffer mechanism (also known as ping-pong buffering). By alternating the two buffers, the parallelization of calculation operations and data transfer is achieved, thereby reducing pipeline idle time and improving the utilization of processor resources.

[0059] In the ARMv8 architecture, each vector register is 128 bits wide. For single-precision floating-point numbers (FP32), one register can store θ = 4 elements; for half-precision floating-point numbers (FP16), θ = 8. In the GEMM operation, the input matrix is ​​loaded with continuous θ data in the same T dimension along the C dimension, and the filter matrix is ​​loaded with continuous θ data in the same C dimension along the K dimension to implement vectorized multiplication and addition operations. The microkernel requires a total of θ for each data load. registers for input and filter data, and another Registers are used to store calculation results. Considering the double buffering requirement, the number of registers required for input and filter loading is doubled. The total number of registers used must not exceed the 32 NEON vector registers supported by the ARMv8 architecture.

[0060] To reduce the edge processing burden caused by insufficient K-dimensional endpoints, this paper sets the parameter β to an integer multiple of θ. Under the aforementioned constraints, this paper optimizes the maximization of the computation-to-memory ratio, taking into account factors such as the thin and tall matrix structure actually involved in matrix multiplication, the T-dimensional edge processing requirements, and the parallel granularity, to adaptively select the appropriate GEMM microkernel configuration. The specific configuration is as follows:

[0061] Single-precision floating point (FP32):

[0062] ① Main microkernel α=4, β=16; edge microkernel α=1~3, β=16

[0063] ② Main microkernel α=7, β=8; edge microkernel α=1~6, β=8

[0064] ③ Main microkernel α=10, β=4; edge microkernel α=1~9, β=4

[0065] Half-precision floating point (FP16):

[0066] ① Main microkernel α=7, β=16; edge microkernel α=1~6, β=16

[0067] ② Main microkernel α=10, β=8; edge microkernel α=1~9, β=8

[0068] In addition, to cooperate with the double buffering mechanism, the present invention designs a specific pipeline execution strategy in the GEMM microkernel: a total of 2α vector registers need to be configured for input data, Registers are used for filter data. These registers are divided into two groups: one group participates in the current stage's calculations, and the other group prefetches data for the next stage, achieving overlapping loading and calculation. In a complete iteration, it is divided into 2θ stages. In the first stage, an input data block of size α×θ and a filter data block of size 1×β are loaded into the registers. After using the scalar-vector fused multiply-add instruction, the elements in the input data registers participate in the calculations in the next θ stages, while the next group of data is loaded. The filter register group performs loading and calculation operations in turn in each stage.

[0069] Figure 3 The figure shows the pipeline scheduling and register block strategy when the microkernel is configured with α = 4 and β = 16 in a single-precision floating-point scenario. Stages numbered #1 to #8 constitute a complete double-buffer iteration cycle, where registers V0 to V7 are used to load input data, V8 to V15 are used to load filter data, and V16 to V31 are used to accumulate output results. The above microkernel loops through 8 stages until it completes the traversal of a sub-block C in the C dimension. blk .

[0070] 3. Parallel Strategy

[0071] The present invention proposes a multi-dimensional parallel processing strategy, which uses OpenMP to perform multi-core parallel acceleration on the three main computational stages of Winograd convolution: input domain conversion, batch matrix multiplication, and output domain conversion. The parallel strategy supports any 2 dim Multiple thread number configuration, where dim=2 or dim=3 corresponds to a two-dimensional or three-dimensional thread grid structure respectively. Assuming the total number of parallel threads (cores) is P, the parallel task partitioning scheme of the present invention in a single batch scenario will be described in detail below.

[0072] In the non-fusion mode, the three-layer nested loop structure involved in the input domain conversion is uniformly expanded into a flat structure for thread task division. The total number of parallel subtasks in this stage is Where T represents the total number of tiles, and θ is the number of data elements that each vector register can accommodate. In the batch matrix multiplication stage, parallelism is adopted on the output channel dimension (K) and the Winograd domain tile dimension (L). The total number of subtasks is The output domain conversion combines the T dimension with the K dimension to generate the number Parallel subtasks.

[0073] In the fusion mode, a two-dimensional thread grid structure is used for multi-level parallelism, where the dimensions involved in the parallelism are T, C, and K. To achieve uniformity in thread scheduling, the same two-dimensional thread grid structure is applied to all three computation steps. The first dimension of the thread grid corresponds to the T dimension, which contains A set of tasks, assigned to P T sub-thread groups, each containing threads.

[0074] Specifically, in the input domain conversion stage, each task set is further subdivided in the C-dimensional direction into subtasks for parallel scheduling within the thread group; in the batch matrix multiplication and output domain conversion phase, the task set is divided into subtasks, each of which performs the corresponding matrix multiplication and inverse transformation operations in sequence. If the number of tasks in the outer loop (T dimension) can provide sufficient parallelism, parallel threads are preferentially configured in the T dimension to ensure that the subtasks within the task set are executed by the same thread. If the number of tasks in the T dimension is insufficient to support all thread allocations, fine-grained parallelism is implemented in the C and K dimensions. The above two strategies can flexibly degenerate into a single-level parallel processing mode at different data scales to ensure load balancing and thread utilization efficiency.

[0075] When the batch number N>1, the present invention further extends the parallel strategy:

[0076] For the non-fusion mode, a two-dimensional thread grid parallel structure is adopted to divide the tasks in the N dimension and other dimensions (such as T or K) at the same time;

[0077] For the fusion mode, it is expanded to a three-dimensional thread grid and the N dimension is included in the parallel scheduling range to achieve multi-core acceleration across batches.

[0078] Due to the data dependency between input domain transformation and batch matrix multiplication, the sub-thread group needs to perform intra-group synchronization after completing the former. To this end, the present invention designs an efficient intra-group synchronization mechanism that achieves coordinated control based on perception variables and OpenMP atomic operations. The specific method is as follows: each thread in the group flips a private perception variable and increments a shared atomic counter. When the counter value reaches the total number of threads in the thread group, the last thread to arrive resets the counter and updates the perception variable to notify the remaining threads to continue execution. The above mechanism ensures the correctness and efficiency of parallel synchronization.

[0079] It is worth noting that the data layout of the filter in the Winograd domain is consistent in both fusion and non-fusion modes, and its domain conversion process is an independent operation with natural parallelism. Therefore, the domain conversion operation of the filter is divided into The subtasks can be computed in parallel, thus further improving the overall system throughput.

[0080] 4. Parameter configuration

[0081] In the overall implementation process of the present invention, multiple key adjustable parameters are involved, including tile dimension T, number of input and output channels C, and partition parameter T of K. blk 、C blk , K blk , thread parallel partition parameter P T , and the GEMM microkernel configuration parameters α and β. To achieve optimal computational efficiency, the present invention designs a recursive optimization model based on the thread grid structure to derive the optimal combination of parameters in the parallel framework composed of the d-dimensional thread grid.

[0082] In this model, the thread grid dimension corresponding to the innermost loop is considered the first dimension. The total execution time for each dimension consists of three components: computation time, data transfer time, and synchronization time. Starting from the first dimension, the computation time for each subsequent dimension is recursively calculated by multiplying the total time of the previous dimension by the number of subtasks in that dimension. The overhead of loading the original input data and Winograd domain filter data from main memory remains constant under different parameter configurations and does not affect parameter selection. Therefore, the data transfer time from main memory is excluded from the model's estimated execution time. The final parameter configuration is determined with the goal of minimizing the estimated total execution time.

[0083] The present invention estimates the computational time of the three stages of input domain conversion, batch matrix multiplication, and output domain conversion by the ratio of the number of floating-point operations required for each subtask to the achievable floating-point operations per second (GFLOPS). The computational access memory ratio of each step is calculated by combining the scale parameters (T, C, K, etc.) of the convolutional layer. The theoretical peak GFLOPS of each stage is derived using the Roofline performance model. The data transfer time corresponding to each stage is further estimated by combining the ratio of the amount of memory access data to the available bandwidth. The additional time overhead caused by load imbalance is modeled as synchronization time.

[0084] Taking the fusion mode framework as an example, the thread grid of the first dimension contains threads, participating in the three steps of input domain conversion, batch matrix multiplication and output domain conversion. Let the total floating-point operations of the three corresponding subtasks be T blk ×K blk ×C×L and At the same time, the floating-point performance (in GFLOPS) that can be achieved at each stage is defined as follows: and and the LLC cache bandwidth is The computation time for each step can be estimated by the ratio of the subtask size to the corresponding GFLOPS value. The data transfer time for each step is estimated by the ratio of the amount of memory accessed to the bandwidth, where the required memory accessed data is stored in the LLC cache. Since the threads in the first dimension are load-balanced, synchronization time is negligible. The total estimated time for the first dimension is:

[0085]

[0086] Among them, each item corresponds to the calculation time of the three calculation steps and the memory access time caused by them. In the implementation of the present invention, the following constraints are imposed on the above parameters to optimize resource utilization: T blk is constrained to be a multiple of α, C blk is constrained to be a multiple of 2θ, K blk is constrained to be a multiple of β.

[0087] Under these constraints, the system searches the parameter space for combinations that satisfy these constraints and minimize total estimation time, thereby obtaining the optimal parameter configuration for a specific hardware platform. Furthermore, the present invention can autonomously select an algorithm framework based on the difference in estimation time between the fusion and non-fusion modes, adapting to the performance requirements of different convolutional layer sizes and hardware resource configurations.

[0088] Experimental test:

[0089] To verify the versatility and performance advantages of the Winograd convolution acceleration method proposed in this paper on different hardware platforms, a comparative experiment was designed and conducted on three representative ARMv8 servers. The specific test platforms include:

[0090] 1. Kunpeng 920 processor: Equipped with 48 cores, each core has 64KB of L1 instruction / data cache and 512KB of L2 cache, all cores share 48MB of L3 cache, and the main frequency is 2.6GHz;

[0091] AWS Graviton2 M6g instance: 64 cores, 64KB of L1 cache and 1MB of L2 cache per core, 32MB of L3 cache shared across all cores, and a 2.5GHz clock speed.

[0092] 3. Phytium 2000+ processor: Equipped with 64 cores, each core has 64KB of L1 instruction / data cache, each quad-core cluster shares 2MB of L2 cache, the system has no L3 cache, and the main frequency is 2.2GHz.

[0093] On the above platform, two-dimensional (2D) and three-dimensional (3D) convolution tasks were tested separately, covering different batch size (N=1 and N=number of cores NC) configurations, where N=1 uses 8 threads to complete the convolution process, and when N=NC, the number of threads is set to NC. The convolution layers in multiple typical network structures were used as test workloads, including VggNet-16, FusionNet, C3D, and 3D U-Net. The experiment selected multiple mainstream deep learning inference libraries, including NCNN, ACL, NNPACK, FastConv, nDIRECT, and oneDNN as comparison objects, and measured the running time of each library and the method of the present invention under the same computing tasks.

[0094] To fully reflect the performance advantages, the minimum and maximum speedup ratios of the present invention relative to the comparison libraries under different test configurations and hardware platforms are statistically analyzed. The speedup ratio is defined as "comparison library running time / presentation running time". The results are summarized in Table 1.

[0095] Table 1 The speedup ratio range of the method of the present invention relative to the mainstream library in multi-platform and multi-tasking scenarios

[0096]

[0097]

[0098] Although the present invention has been disclosed as above by way of embodiments, they are not intended to limit the present invention. Any appropriate modification or equivalent substitution of the technical solution of the present invention by a person skilled in the art should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on that defined in the claims.

Claims

1. A two-dimensional and three-dimensional Winograd convolution optimization method for ARMv8 architecture, characterized by: The following steps are involved: 1) Perform Winograd domain conversion on input data and convolution filter data, and use an assembly-level microkernel to implement the conversion operations of input, filter, and output. During the input conversion process, the overlapping areas between tiles are used to reuse data; 2) Perform batch matrix multiplication on the Winograd domain-converted input data and the filter data, using a GEMM microkernel with double buffering to complete the matrix multiplication operation for the corresponding channel dimension, and write the result to the output data block; 3) Perform an inverse Winograd domain transformation on the matrix multiplication result, implement the transformation using assembly instructions, and combine it with an optimized data layout to achieve continuous memory access to generate the final convolution output data; 4) In steps 1) to 3), a parallel thread grid is configured according to the input, output, and convolution kernel parameters, and a fusion or non-fusion algorithm framework is used to execute multi-dimensional parallel computing tasks.

2. The method according to claim 1, wherein The Winograd domain conversion of the input data in step 1) includes: performing a transformation operation on the spatial dimension of the input data using a tile-by-tile or row-by-row / column strategy, wherein the strategy is selected based on the Winograd parameters m, r and the number of available vector registers; the Winograd domain inverse conversion of the output data in step 3) includes: performing an inverse transformation operation on the spatial dimension of the output data using a tile-by-tile or row-by-column strategy, wherein the strategy is selected based on the Winograd parameters m, r and the number of available vector registers.

3. The method according to claim 2, wherein In step 1), the Winograd domain conversion of the input data is divided into two microkernels for execution when processing three-dimensional convolution. The first microkernel performs domain conversion on some dimensions and stores the results in an intermediate array of the pre-designed data layout to support the second microkernel to perform continuous memory access conversion on the remaining dimensions; when processing two-dimensional convolution, the domain conversion is completed by a single microkernel in the same stage; in step 3), the Winograd domain inverse conversion of the output data is divided into two microkernels for execution when processing three-dimensional convolution. The first microkernel performs inverse transformation on some dimensions and stores the results in an intermediate array of the pre-designed data layout to support the second microkernel to perform continuous memory access inverse conversion on the remaining dimensions; when processing two-dimensional convolution, the inverse conversion is completed by a single microkernel in the same stage.

4. The method according to claim 2, wherein The Winograd domain conversion of the input data in step 1) further includes: reusing elements of adjacent tiles in the row direction in the conversion calculation through a vector register reuse mechanism to reduce the number of memory accesses.

5. The method according to claim 1, wherein During the GEMM microkernel processing in step 2), the input matrix reads continuous vector data along the input channel dimension under the same tile number, and the filter matrix reads continuous vector data along the output channel dimension under the same input channel dimension to achieve vectorized calculation.

6. The method according to claim 1, wherein The double buffering mechanism in step 2) includes: dividing the input data register group and the filter data register group into two groups respectively, alternately performing data loading and calculation to achieve overlap of calculation and data transfer.

7. The method according to claim 1, wherein The input data and calculation results used in the batch matrix multiplication operation in step 2) adopt a pre-designed intermediate data layout. This layout supports continuous memory access when reading input data and writing matrix multiplication results, avoiding data rearrangement or packing operations, and enables the Winograd domain inverse transformation microkernel in step 3) to directly read the calculation results and complete the continuous memory access conversion operation.

8. The method according to claim 1, wherein The fusion algorithm framework in step 4) includes: completing the input domain conversion, batch matrix multiplication and output domain inverse conversion in sequence within the tile block, and performing fusion processing in K dimensions. After completing the matrix multiplication of each K block, the corresponding output conversion operation is immediately performed.

9. The method according to claim 1, wherein Step 4) specifically includes: building a two-dimensional or three-dimensional thread grid, dividing the spatial dimension, input channel dimension, and output channel dimension of the convolutional layer into multiple subtasks, and using OpenMP to implement static scheduling and thread group synchronization mechanism.

10. A two-dimensional and three-dimensional Winograd convolution optimization system for ARMv8 architecture, characterized by: The invention comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images