Method, device and equipment for direct convolution implementation of a multi-core processor

Abstract

The application relates to a direct convolution implementation method, device and equipment of a multi-core processor. The method comprises the following steps: constructing a direct convolution cycle rearrangement model of the multi-core processor. The model resets the cycle order of the direct convolution according to the architecture parameters of the multi-core processor, and a microkernel convolution cycle mechanism is obtained. The mechanism comprises seven convolution cycles, and each convolution cycle performs direct convolution iteration operation on picture sample data in different convolution dimensions. According to the mechanism, the convolution data iteration of each layer in the second convolution cycle to the fourth convolution cycle and the sixth convolution cycle is determined, and the data block cache size corresponding to the convolution data is determined. According to the data block cache size, the data access amount of the sixth convolution cycle and the seventh convolution cycle is determined, and the convolution microkernel is constructed in combination with the cache capacity. The seventh convolution cycle calculates a high-dimensional picture through the convolution microkernel. By using the method, the data packaging can be realized, and the overhead caused by memory access can be hidden.

Description

Technical Field

[0001] This application relates to the field of multi-core processor architecture technology, and in particular to a method, apparatus and device for implementing direct convolution in a multi-core processor. Background Technology

[0002] Convolutional Neural Networks (CNNs) have achieved tremendous success in deep learning and are frequently used in popular fields such as computer vision and natural language processing. Convolution is the most important operator in CNNs, accounting for 90% of the total execution time. Im2col+GEMM is a commonly used convolution algorithm in some mainstream deep learning frameworks. It accelerates the process by first transforming the input tensor using the im2col operation and then calling a highly optimized general matrix multiplication (GEMM) function. However, this algorithm has inherent drawbacks. On the one hand, the im2col operation transforms a 4D tensor into a 3D matrix, which occupies more memory, introducing additional time and space overhead. On the other hand, because the transformed matrix has an irregular shape, calling the GEMM method cannot achieve optimal performance. In summary, these drawbacks of the im2col+GEMM method result in unsatisfactory performance on multi-core CPUs.

[0003] Recent research indicates that direct optimization of convolution algorithms can yield performance superior to the im2col+GEMM algorithm. Compared to im2col+GEMM, direct convolution algorithms offer two computational advantages. First, direct convolution does not require pre-transformation of the input tensor, avoiding additional time and space overhead. Second, direct convolution can fully exploit the sparsity of the convolution kernel, thus avoiding unnecessary redundant computation. LIBXSMM is an existing deep learning library optimized for direct convolution on CPUs. This library uses a specially designed data storage format and Batch-Reduce GEMM (BRGEMM) as the convolution microkernel, achieving convolution performance exceeding that of the im2col+GEMM algorithm on multi-core CPUs of both x86 and ARM architectures. However, the LIBXSMM direct convolution method still has two problems. First, the data storage format used by LIBXSMM is incompatible with the data formats supported by mainstream deep learning frameworks (such as NCHW or NHWC). To adapt LIBXSMM to mainstream deep learning frameworks, either secondary development of the mainstream deep learning frameworks is required, or format conversion operations must be added before and after each convolution operation. These modifications all introduce additional overhead. On the other hand, LIBXSMM still uses a GEMM-based convolutional microkernel, thus losing the opportunity for register data reuse under convolutional loads and exhibiting suboptimal performance. In summary, existing optimization methods for direct convolution either sacrifice compatibility with deep learning frameworks by accelerating convolution based on new data formats, or maintain framework compatibility at the expense of performance; therefore, they fail to comprehensively balance framework compatibility and high-performance convolution. Summary of the Invention

[0004] Therefore, it is necessary to provide a general, high-performance direct convolution implementation method, apparatus, and device for multi-core processors to address the aforementioned technical problems.

[0005] A method for implementing direct convolution on a multi-core processor, the method comprising:

[0006] Construct a direct convolutional cyclic rearrangement model for multi-core processors.

[0007] The direct convolution loop rearrangement model resets the loop order of direct convolutions according to the architecture parameters of a multi-core processor, resulting in a microkernel convolution loop mechanism. This microkernel convolution loop mechanism consists of seven convolution loops, where each loop performs direct convolution iterations on the image sample data at different convolution dimensions.

[0008] Based on the microkernel convolution loop mechanism, the data block cache size corresponding to the convolution data is determined by iterating through the convolution data of each layer from the second to the fourth convolution loop and the sixth convolution loop.

[0009] The data access volume of the sixth and seventh convolutional loops is determined based on the data block cache size. The data access volume is combined with the cache capacity to construct the convolutional microkernel.

[0010] The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

[0011] In one embodiment, the method further includes: a first convolutional loop iterating over the sample number dimension of the image sample data; a second convolutional loop iterating over the height dimension of the input tensor of the image sample data to obtain image input tensor height convolutional data; a third convolutional loop iterating over the channel dimension of the input tensor of the image sample data to obtain image input tensor channel convolutional data; a fourth convolutional loop iterating over the channel dimension of the output tensor of the image sample data to obtain image output tensor channel convolutional data; a sixth convolutional loop iterating over the width dimension of the input tensor of the image sample data to obtain image input tensor width convolutional data; a fifth convolutional loop iterating over the height dimension of the image input tensor convolutional data to obtain image input tensor height convolutional data; and a seventh convolutional loop iterating over the channel dimension of the output tensor convolutional data of the image sample data to obtain image output tensor channel convolutional data.

[0012] In one embodiment, the method further includes: determining the size of a first input buffer block based on the image input tensor height convolution data obtained in each iteration of the second convolutional loop according to the microkernel convolutional loop mechanism; determining the size of a second input buffer block based on the image input tensor channel convolution data obtained in each iteration of the third convolutional loop according to the microkernel convolutional loop mechanism; determining the size of a third input buffer block based on the image output tensor channel convolution data obtained in each iteration of the fourth convolutional loop according to the microkernel convolutional loop mechanism; and determining the size of a register buffer based on the image input tensor width convolution data obtained in each iteration of the sixth convolutional loop according to the microkernel convolutional loop mechanism.

[0013] In one embodiment, the method further includes: a fourth convolutional loop calling a register format conversion function in each iteration to convert the data format of the register tensor to obtain standard register tensor data; a fifth convolutional loop marking the height convolutional data of the image input tensor corresponding to the size of the first input buffer block according to the microkernel convolutional loop mechanism; and a seventh convolutional loop marking the channel convolutional data of the image output tensor corresponding to the size of the third input buffer block according to the microkernel convolutional loop mechanism.

[0014] In one embodiment, the method further includes: determining the microkernel output buffer block size based on the data access volume of the convolutional data accessed in each iteration of the sixth convolutional loop (the width of the image input tensor) and the L2 cache capacity; determining the microkernel input buffer block size based on the data access volume of the standard register tensor data accessed in each iteration of the seventh convolutional loop and the L1 cache capacity; and constructing a convolutional microkernel based on the microkernel output buffer block size, the microkernel input buffer block size, the register tensor height, and the register tensor width.

[0015] In one embodiment, the convolutional microkernel includes a packing microkernel and a computation microkernel. The computation microkernel includes an eighth convolutional loop and a ninth convolutional loop. The eighth convolutional loop iterates through the convolutional data of the image input tensor channels according to the size of the second input buffer block, performing convolutional operations. The ninth convolutional loop performs an outer product operation on the input tensor data and the standard register tensor data.

[0016] In one embodiment, the method further includes: a seventh convolutional loop calls the outer product calculation result of the sample data through the convolutional microkernel, and performs a convolution operation on the outer product calculation result to obtain a high-dimensional image.

[0017] 8. A direct convolution implementation apparatus for a multi-core processor, characterized in that the apparatus comprises:

[0018] The Direct Convolutional Cyclic Rearrangement Model Building Module is used to build direct convolutional cyclic rearrangement models for multi-core processors.

[0019] The microkernel convolution loop mechanism building module is used to rearrange the direct convolution loop model according to the architecture parameters of the multi-core processor, thus obtaining the microkernel convolution loop mechanism. The microkernel convolution loop mechanism includes seven convolution loops, where each convolution loop performs direct convolution iteration operations on the image sample data in different convolution dimensions.

[0020] The data block cache size determination module is used to iteratively determine the data block cache size corresponding to the convolutional data for each layer of the convolutional loop from the second to the fourth layer and the sixth layer, based on the microkernel convolutional loop mechanism.

[0021] The convolutional microkernel building module is used to determine the data access volume of the sixth and seventh convolutional loops based on the data block cache size. The data access volume is combined with the cache capacity to build the convolutional microkernel.

[0022] The high-dimensional image acquisition module uses a seventh-layer convolutional loop to calculate the outer product of the image sample data through a convolutional microkernel to obtain the high-dimensional image.

[0023] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0024] Construct a direct convolutional cyclic rearrangement model for multi-core processors.

[0025] The direct convolution loop rearrangement model resets the loop order of direct convolution according to the architecture parameters of the multi-core processor, resulting in a microkernel convolution loop mechanism. The microkernel convolution loop mechanism includes seven convolution loops, where each convolution loop performs direct convolution iterations on the image sample data in different convolution dimensions.

[0026] Based on the microkernel convolution loop mechanism, the data block cache size corresponding to the convolution data is determined by iterating through the convolution data of each layer from the second to the fourth convolution loop and the sixth convolution loop.

[0027] The data access volume of the sixth and seventh convolutional loops is determined based on the data block cache size. The data access volume is combined with the cache capacity to construct the convolutional microkernel.

[0028] The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

[0029] The aforementioned method, apparatus, and device for implementing direct convolution on a multi-core processor utilize a traditional direct convolution algorithm consisting of a single multiply-accumulate statement and seven nested loops. Since there are no data dependencies between the loops in this algorithm, the loops can be appropriately rearranged and divided into blocks to improve locality of reference, thereby enhancing performance. Furthermore, to effectively utilize the processor's floating-point multiply-accumulate (FMA) units and increase the floating-point computation density (FAI) at the register level, a highly efficient convolution computation microkernel should be designed in conjunction with Single Instruction Multiple Data (SIMD) instruction set extensions. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a direct convolution implementation method for a multi-core processor in one embodiment;

[0031] Figure 2 This is a schematic diagram of the original direct convolution algorithm in one embodiment;

[0032] Figure 3 This is a flowchart illustrating the direct convolution optimization steps based on an ARM multi-core CPU in one embodiment.

[0033] Figure 4 This is a flowchart illustrating the iterative transformation steps of direct convolution in one embodiment;

[0034] Figure 5This is a flowchart illustrating the process of determining the size of a direct convolution loop block in one embodiment;

[0035] Figure 6 This is a schematic diagram of the direct convolution computation microkernel execution process in one embodiment;

[0036] Figure 7 This is a structural block diagram of a direct convolution implementation device for a multi-core processor in one embodiment;

[0037] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0039] In one embodiment, such as Figure 1 As shown, a direct convolution implementation method for multi-core processors is provided, including the following steps:

[0040] Step 102: Construct a direct convolutional cyclic rearrangement model for multi-core processors.

[0041] Specifically, a direct convolutional cyclic rearrangement model for multi-core processors is constructed using mainstream data formats.

[0042] Step 104: The direct convolution loop rearrangement model resets the loop order of direct convolution according to the architecture parameters of the multi-core processor, resulting in a microkernel convolution loop mechanism. The microkernel convolution loop mechanism includes seven convolution loops, where each convolution loop performs direct convolution iterations on the image sample data in different convolution dimensions.

[0043] Each convolutional loop is denoted by L, where L1 represents the first convolutional loop, and so on. The meanings of related symbols are defined as shown in Table 1.

[0044] Table 1 Meaning of Parameter Symbols

[0045] N Number of image samples K Number of output tensor channels C Input tensor channel number R Filter tensor height H Input tensor height S Filter tensor width W Input tensor width P Output Tensor Height Q Output Tensor Width str Calculate step size I Input tensor O Output Tensor F Filter tensor

[0046] Step 106: Based on the microkernel convolution loop mechanism, iterate through the convolution data of each layer from the second to the fourth convolution loop and the sixth convolution loop to determine the data block cache size corresponding to the convolution data.

[0047] Step 108: Determine the data access volume of the sixth and seventh convolutional loops based on the data block cache size, and construct the convolutional microkernel by combining the data access volume with the cache capacity.

[0048] Step 110: The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

[0049] The aforementioned method, apparatus, and device for implementing direct convolution on a multi-core processor utilize a traditional direct convolution algorithm consisting of a single multiply-accumulate statement and seven nested loops. Since there are no data dependencies between the loops in this algorithm, the loops can be appropriately rearranged and divided into blocks to improve locality of reference, thereby enhancing performance. Furthermore, to effectively utilize the processor's floating-point multiply-accumulate (FMA) units and increase the floating-point computation density (FAI) at the register level, a highly efficient convolution computation microkernel should be designed in conjunction with Single Instruction Multiple Data (SIMD) instruction set extensions.

[0050] In one embodiment, loops L2 and L3 block the data access of the I tensor so that the data access volume of each iteration can be accommodated in the last level cache (LLC); loops L3 and L4 block the data access of the F tensor so that the data access volume of each iteration can be accommodated in the second level cache; loops L5 and L6 block the data access of the I tensor so that the data access volume of each iteration can be accommodated in the first level cache.

[0051] In one embodiment, the `transform_filter()` function is used at runtime to change the storage format T of the convolution kernel. k T c RS is converted into a storage format that facilitates vector operations. This facilitates continuous data access to tensor F during convolutional microkernel calls.

[0052] In one embodiment, during the first iteration of the L7 loop, i.e., the iteration with loop index kv = kt, the algorithm calls the convolutional microkernel Pack_Micro-kernel. This process involves accessing the non-contiguously stored Input tensor data in memory, performing convolution calculations, and packing the data into a linear buffer B.

[0053] In one embodiment, in each iteration following the first iteration of the L7 loop, the algorithm calls the convolutional microkernel Main_Micro-kernel. This process involves accessing the contiguously stored Input tensor data in the linear buffer B to perform convolution computation.

[0054] In one embodiment, the optimal register allocation scheme is calculated using the Lagrange multiplier method, and the number of vector registers storing tensor I, F, and O data in the microkernel is set to 4, 4, and 24, respectively.

[0055] In one embodiment, in a convolutional microkernel written using SIMD assembly instructions, memory access instructions are inserted after computation instructions that are not related to them.

[0056] In one embodiment, the first convolutional loop iterates over the sample number dimension of the image sample data. The second convolutional loop iterates over the height dimension of the input tensor of the image sample data to obtain the image input tensor height convolutional data. The third convolutional loop iterates over the channel dimension of the input tensor of the image sample data to obtain the image input tensor channel convolutional data. The fourth convolutional loop iterates over the channel dimension of the output tensor of the image sample data to obtain the image output tensor channel convolutional data. The sixth convolutional loop iterates over the width dimension of the input tensor of the image sample data to obtain the image input tensor width convolutional data. The fifth convolutional loop iterates over the height dimension of the image input tensor convolutional data to obtain the image input tensor height convolutional data. The seventh convolutional loop iterates over the channel dimension of the output tensor convolutional data to obtain the image output tensor channel convolutional data.

[0057] In one embodiment, the size of the first input buffer block is determined based on the image input tensor height convolution data obtained in each iteration of the second convolutional loop according to the microkernel convolutional loop mechanism. The size of the second input buffer block is determined based on the image input tensor channel convolution data obtained in each iteration of the third convolutional loop according to the microkernel convolutional loop mechanism. The size of the third input buffer block is determined based on the image output tensor channel convolution data obtained in each iteration of the fourth convolutional loop according to the microkernel convolutional loop mechanism. The size of the register buffer is determined based on the image input tensor width convolution data obtained in each iteration of the sixth convolutional loop according to the microkernel convolutional loop mechanism.

[0058] In one embodiment, the fourth convolutional loop calls a register format conversion function in each iteration to convert the data format of the register tensor, obtaining standard register tensor data. The fifth convolutional loop marks the height convolutional data of the image input tensor corresponding to the size of the first input buffer block according to the microkernel convolutional loop mechanism. The seventh convolutional loop marks the channel convolutional data of the image output tensor corresponding to the size of the third input buffer block according to the same microkernel convolutional loop mechanism.

[0059] In one embodiment, the microkernel output buffer block size is determined based on the data access volume of the image input tensor width convolutional data in each iteration of the sixth convolutional loop and the L2 cache capacity. The microkernel input buffer block size is determined based on the data access volume of the standard register tensor data in each iteration of the seventh convolutional loop and the L1 cache capacity. The convolutional microkernel is constructed based on the microkernel output buffer block size, the microkernel input buffer block size, the register tensor height, and the register tensor width.

[0060] It is worth noting that the microkernel output buffer block size is equal to the third input buffer block size, and the microkernel input buffer block size is equal to the second input buffer block size.

[0061] In one embodiment, a packaged microkernel and a computation microkernel are included. The computation microkernel includes an eighth convolutional loop and a ninth convolutional loop. The eighth convolutional loop iterates through the convolutional data of the image input tensor channels according to the size of the second input buffer block, performing convolutional operations. The ninth convolutional loop performs an outer product operation on the input tensor data and the standard register tensor data.

[0062] In one embodiment, the seventh convolutional loop calls the outer product calculation result of the sample data through the convolutional microkernel, and performs convolution operation on the outer product calculation result to obtain a high-dimensional image.

[0063] In one embodiment, a traditional data storage format of mainstream deep learning frameworks is used to ensure framework compatibility of the algorithm; convolution operations involve Input(I), Filter(F), and Output(O) tensors, which use NCHW, KCRS, and NKPQ data storage formats, respectively. In one embodiment, as... Figure 3 As shown, a method for optimizing direct convolution based on ARM multi-core CPUs is provided, including the following steps:

[0064] Step 202: Based on the ARM multi-core CPU architecture parameters, transform the loop of the direct convolution algorithm; including re-determining the loop order and dividing the loop into blocks to adapt to the size of the processor's cache at each level.

[0065] In this step, the ARM multi-core CPU used is compatible with the ARMv8 instruction set, supports NEON technology, and has a multi-level cache structure. NEON technology provides 32 128-bit general-purpose vector registers.

[0066] This step introduces a pseudocode specification that defines the outermost loop of a direct convolution as a first-level loop, denoted by L1. The representation of the loop depends on the order of loops from the outside in; for example, the third-level loop is represented as L3.

[0067] Step 204: Determine the size of the loop block for direct convolution.

[0068] Step 206: Customize the convolution calculation microkernel of the innermost layer of the direct convolution loop.

[0069] The data type used in this step is single-precision floating-point number, with a bit width of 32 bits. A single NEON vector register can store a maximum of 4 single-precision floating-point numbers.

[0070] The custom convolutional microkernel consists of a computation microkernel and a packaging microkernel. The computation of the microkernel is performed using an outer product operation similar to matrix multiplication.

[0071] In one embodiment, the first convolutional loop L1 iterates over dimension N, and the second and third convolutional loops L2 and L3 iterate over dimensions H and C, respectively; wherein the indices of the second and third convolutional loops L2 and L3 are ht and ct, respectively, and the block sizes of loops L2 and L3 are denoted by T. h and T c The fourth convolutional loop, L4, iterates over dimension K, with its index and block size represented by kt and T, respectively. k express.

[0072] In one embodiment, at the beginning of each iteration of the fourth convolutional loop L4, the transform_filter() function is called to perform format conversion on the block data of tensor F. (The last part, "will be pressed T," appears to be an error and doesn't translate directly.) k T c Transforming tensor F data blocks stored in RS format into T c RSV k Format, V k The vectorized length of dimension K.

[0073] In one embodiment, the fifth convolutional loop L5 is executed after the `transform_filter()` function is called. Loop L5 divides the dimension H into sub-dimensions T. h The process is iterated over, with the index and initial value of the index being hv and ht, respectively. Loop L6 iterates over dimension W, with the index and block size being wv and V, respectively. w Indicates. V w This also represents the vectorized length of dimension W. Loop L7 divides dimension K into blocks and then assigns sub-dimension T. k Perform a traversal, using kv and V as the indices and block sizes, respectively. k express.

[0074] In one embodiment, such as Figure 4 As shown, the specific steps for performing a cyclic transformation on direct convolution are as follows:

[0075] 1) Determine the iteration dimension of the first loop L1 as N.

[0076] 2) Determine the iteration dimension of the second loop L2 as H, and its block size as T. h .

[0077] 3) Determine the iteration dimension of the third loop L3 as C, and its block size as T. c .

[0078] 4) Determine the iteration dimension of the fourth loop L4 as K, and its block size as T. k .

[0079] 5) Determine the iteration dimension of the fifth loop L5 as T. h .

[0080] 6) Determine the iteration dimension of the sixth loop L6 as W, and the block size as V. w .

[0081] 7) Determine the iteration dimension of the seventh loop L7 as T. k .

[0082] 8) Call the convolutional microkernel within the seventh loop (L7).

[0083] In one embodiment, for each iteration of loop L7, the level 1 cache needs to load tensor I (R×T) c ×(V w +S-1)) data elements and tensor F of (2×V) k ×T c To maximize the utilization of the first-level cache, use (×R×S) data elements.

[0084] In one embodiment, for each iteration of loop L6, the secondary cache needs to load (2×R×T) times tensor I. c ×(V w +S-1)) data elements and tensor F of (T) k ×T c To maximize the utilization of the second-level cache, use (×R×S) data elements.

[0085] In one embodiment, such as Figure 5 As shown, the design steps of a convolutional microkernel include:

[0086] 1) Determine the block size T of dimension C by combining the L1 cache capacity and the data access volume in each iteration of the L7 loop. c .

[0087] 2) Determine the block size T of dimension K by combining the L2 cache capacity and the data access volume in each iteration of the L6 loop. k .

[0088] In one embodiment, such as Figure 6 As shown, the computational microkernel includes an eighth convolutional loop L8 and a ninth convolutional loop L9. The specific execution steps include:

[0089] 1) Loop L8 sequentially processes the T values ​​of the image. c The channels are convolved with an iteration factor of cv.

[0090] 2) Within loop L9, load the tensor I data from buffer B into the vector register; load the tensor F data into the vector register; perform the outer product calculation and save the intermediate result into the vector register.

[0091] 3) When loop L8 ends, save the calculation results.

[0092] In one embodiment, Pack_Micro-kernel() stores non-contiguous I tensor data blocks into linear buffer B: placing the memory access instruction LD\ST after the computation instruction FMA, which has no dependency on it, can hide the packing overhead of the memory access instruction.

[0093] In one embodiment, the computation of the microkernel Main_Micro-kernel() includes loops L8 and L9; loop L8 divides dimension C into sub-dimensions T. c The loop iterates through dimension R, with indices denoted by cv. Loop L9 iterates through dimension R, with indices denoted by r. Within loop L9, convolution operations are performed using SIMD instructions; firstly, using... V vector registers store (V w +S-1) I tensor data, V vector registers store (V k ) F tensor data, Each vector register stores (Vw+Vk) O tensor data; (V w +S-1) tensor I data will be combined with (S×V k The outer product calculation is performed on tensor F data, and the number of floating-point calculations is (2 × 3 × V). w ×V k ).

[0094] In one embodiment, V is set w =12, V k =8 makes the register level maximize.

[0095] It should be understood that, although Figure 1 , Figures 3-6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 , Figures 3-6At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0096] In one embodiment, such as Figure 7 As shown, a direct convolution implementation device for a multi-core processor is provided, including: a direct convolution cyclic rearrangement model construction module 702, a microkernel convolution cyclic mechanism construction module 704, a data block cache size determination module 706, a convolution microkernel construction module 708, and a high-dimensional image acquisition module 710, wherein:

[0097] The Direct Convolutional Cyclic Rearrangement Model Building Module 702 is used to build direct convolutional cyclic rearrangement models for multi-core processors.

[0098] The microkernel convolution loop mechanism construction module 704 is used to rearrange the direct convolution loop model according to the architecture parameters of the multi-core processor, resetting the loop order of direct convolution to obtain the microkernel convolution loop mechanism. The microkernel convolution loop mechanism includes seven convolution loops, in which each convolution loop performs direct convolution iteration operations on the image sample data in different convolution dimensions;

[0099] The data block cache size determination module 706 is used to iteratively determine the data block cache size corresponding to the convolutional data for each layer of the convolutional data in the second to fourth convolutional cycles and the sixth convolutional cycle according to the microkernel convolutional loop mechanism.

[0100] The convolutional microkernel building module 708 is used to determine the data access volume of the sixth and seventh convolutional loops based on the data block cache size, and the data access volume is combined with the cache capacity to build the convolutional microkernel.

[0101] The high-dimensional image acquisition module 710 uses a seventh-layer convolutional loop to calculate the outer product of the image sample data through a convolutional microkernel to obtain the high-dimensional image.

[0102] Specific limitations regarding the direct convolution implementation device for multi-core processors can be found in the above description of the limitations of a direct convolution implementation method for multi-core processors, and will not be repeated here. Each module in the aforementioned direct convolution implementation device for multi-core processors can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the computer device, or stored in software in the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0103] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a direct convolution method for a multi-core processor. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0104] Those skilled in the art will understand that Figures 7-8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0105] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0106] Construct a direct convolutional cyclic rearrangement model for multi-core processors.

[0107] The direct convolution loop rearrangement model resets the loop order of direct convolutions according to the architecture parameters of a multi-core processor, resulting in a microkernel convolution loop mechanism. This microkernel convolution loop mechanism consists of seven convolution loops, where each loop performs direct convolution iterations on the image sample data at different convolution dimensions.

[0108] Based on the microkernel convolution loop mechanism, the data block cache size corresponding to the convolution data is determined by iterating through the convolution data of each layer from the second to the fourth convolution loop and the sixth convolution loop.

[0109] The data access volume of the sixth and seventh convolutional loops is determined based on the data block cache size. The data access volume is combined with the cache capacity to construct the convolutional microkernel.

[0110] The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0113] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for implementing direct convolution on a multi-core processor, characterized in that, The method includes: Construct a direct convolutional cyclic rearrangement model for multi-core processors; The direct convolution loop rearrangement model resets the loop order of direct convolution according to the architecture parameters of the multi-core processor to obtain a microkernel convolution loop mechanism; the microkernel convolution loop mechanism includes seven layers of convolution loops, wherein each layer of convolution loop performs direct convolution iteration operations on image sample data in different convolution dimensions; The data block cache size corresponding to the convolutional data is determined iteratively for each layer of the convolutional data in the second to fourth and sixth convolutional loops according to the microkernel convolutional loop mechanism. The data access volume of the sixth and seventh convolutional loops is determined based on the data block cache size, and the data access volume is combined with the cache capacity to construct the convolutional microkernel. The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

2. The method according to claim 1, characterized in that, Each convolutional loop performs direct convolution iterations on the image sample data at different convolutional dimensions, including: The first convolutional loop iterates over the sample number dimension of the image sample data; The second convolutional loop iterates over the height dimension of the input tensor of the image sample data to obtain the image input tensor height convolutional data; The third convolutional loop iterates through the input tensor channel dimensions of the image sample data to obtain the image input tensor channel convolutional data. The fourth convolutional loop iterates through the output tensor channel dimension of the image sample data to obtain the image output tensor channel convolutional data. The sixth convolutional loop iterates over the width dimension of the input tensor of the image sample data to obtain the convolutional data of the image input tensor width. The fifth convolutional loop iterates over the dimensions of the image input tensor height convolutional data to obtain the image input tensor height convolutional data. The seventh convolutional loop iterates over the dimensions of the image output tensor channel convolutional data to obtain the image output tensor channel convolutional data.

3. The method according to claim 2, characterized in that, Based on the microkernel convolution loop mechanism, the data block cache size corresponding to the convolution data is determined iteratively for each layer of the convolution loop from the second to the fourth layer, and the sixth layer, including: The size of the first input buffer block is determined by the image input tensor height convolution data obtained in each iteration of the second layer convolution loop according to the microkernel convolution loop mechanism. The size of the second input buffer block is determined based on the image input tensor channel convolution data obtained in each iteration of the third convolution loop according to the microkernel convolution loop mechanism. The size of the third input buffer block is determined based on the image output tensor channel convolution data obtained in each iteration of the fourth convolution loop according to the microkernel convolution loop mechanism. The register buffer size is determined based on the image input tensor width convolution data obtained in each iteration of the sixth convolution loop according to the microkernel convolution loop mechanism.

4. The method according to claim 3, characterized in that, Before the step of determining the data access volume of the sixth and seventh convolutional loops based on the data block cache size, and constructing the convolutional microkernel by combining the data access volume with the cache capacity, the method further includes: The fourth convolutional loop calls the register format conversion function in each iteration to convert the data format of the register tensor and obtain standard register tensor data. The fifth convolutional loop marks the image input tensor height convolutional data corresponding to the size of the first input buffer block according to the microkernel convolutional loop mechanism; The seventh convolutional loop marks the image output tensor channel convolutional data corresponding to the size of the third input buffer block according to the microkernel convolutional loop mechanism.

5. The method according to claim 4, characterized in that, The data access volume of the sixth and seventh convolutional loops is determined based on the data block cache size. This data access volume, combined with the cache capacity, is used to construct a convolutional microkernel, including: The microkernel output cache block size is determined based on the number of data accesses to the image input tensor width convolution data in each iteration of the sixth convolutional loop and the capacity of the second-level cache. The microkernel input cache block size is determined based on the amount of data accessed to the standard register tensor data in each iteration of the seventh convolutional loop and the L1 cache capacity; Convolutional microkernels are constructed based on the microkernel output buffer block size, the microkernel input buffer block size, the register tensor height, and the register tensor width.

6. The method according to claim 5, characterized in that, The convolutional microkernel includes: a packaging microkernel and a computation microkernel; The computational microkernel includes an eighth convolutional loop and a ninth convolutional loop; The eighth convolutional loop iterates through the image input tensor channel convolutional data according to the size of the second input buffer block to perform convolutional iteration operations. The ninth convolutional loop performs an outer product operation between the input tensor data and the standard register tensor data.

7. The method according to claim 5, characterized in that, The seventh convolutional loop calculates the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image, including: The seventh convolutional loop calls the outer product calculation result of the sample data through the convolutional microkernel, and performs convolution operation on the outer product calculation result to obtain a high-dimensional image.

8. A direct convolution implementation apparatus for a multi-core processor, characterized in that, The device includes: The Direct Convolutional Recursive Rearrangement Model Building Module is used to build direct convolutional recursive rearrangement models for multi-core processors. The microkernel convolution loop mechanism construction module is used to reset the loop order of direct convolution according to the architecture parameters of the multi-core processor to obtain the microkernel convolution loop mechanism; the microkernel convolution loop mechanism includes seven layers of convolution loops, wherein each layer of convolution loop performs direct convolution iteration operations on image sample data in different convolution dimensions; The data block cache size determination module is used to iteratively determine the data block cache size corresponding to the convolutional data for each layer of the convolutional data in the second to fourth and sixth convolutional loops according to the microkernel convolutional loop mechanism. A convolutional microkernel construction module is used to determine the data access volume of the sixth and seventh convolutional loops based on the data block cache size, and the data access volume is combined with the cache capacity to construct the convolutional microkernel. The high-dimensional image acquisition module uses a seventh-layer convolutional loop to calculate the outer product of the image sample data through the convolutional microkernel to obtain a high-dimensional image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images