Computing devices, methods, and related products

Abstract

The present disclosure relates to a computing device, method and related products, wherein the computing device of the present disclosure is included in an integrated circuit device, the integrated circuit integrates one or more combined processing devices, the combined processing device is an artificial intelligence operation unit to support various deep learning and machine learning algorithms, and meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc.

Description

Technical Field

[0001] This disclosure generally relates to the field of computers. Specifically, this disclosure relates to a computing device, method, and related products. More specifically, this disclosure relates to a computing device, chip, circuit board, method for performing Depthwise convolution operations using a computing device, and computer-readable storage medium. Background Technology

[0002] Neural networks are among the most crucial technologies in artificial intelligence and deep learning, with Convolutional Neural Networks (CNNs) being the most important type. The most critical computation in a CNN is the convolution operation within the convolutional layers. The function of convolutional layers is to extract features from the input data. Through multiple convolutions, complex features can be extracted to ensure the network has sufficient expressive power and generalization ability. Neural network models contain a large number of various types of convolution operations, and the computational performance of these operations significantly impacts the overall computational performance of the neural network model.

[0003] Typically, core deep learning operators such as convolution are computationally intensive and require frequent memory accesses on AI chips. To meet the demands of high throughput and low latency, the underlying computing units often need to possess high data parallelism capabilities. Single Instruction Multiple Data (SIMD), as a parallel computing architecture, uses multiple processing units to execute the same instruction simultaneously. A single instruction operates on multiple data sets, thereby achieving parallel processing of multiple data sets and improving program execution speed. The SIMD architecture provides high computational efficiency at the instruction-level parallelism level by performing the same operation on multiple data sets simultaneously with a single instruction.

[0004] However, unlike the Single Instruction Multiple Threads (SIMT) architecture, which uses threads as the parallel granularity, the parallelism of SIMD depends entirely on the vector register width and the data size covered by the instruction. Its computational units are typically lighter and lack thread-level scheduling and masking mechanisms. Therefore, in actual execution, if the amount of data involved in the computation of a single SIMD instruction is insufficient, or if the load on each computation channel is uneven, it will lead to a significant decrease in the utilization of computing resources, making it difficult to achieve the peak performance of the hardware.

[0005] Based on the above characteristics, in high-throughput computing scenarios, SIMD acceleration requires systematic design of data layout and computation process at the algorithm level, so that each SIMD instruction can participate in effective data computation as much as possible, and achieve load balancing among parallel channels, thereby fully releasing the performance advantages brought by instruction-level parallelism.

[0006] Depthwise convolution is a commonly used convolution method in lightweight neural networks. Compared to traditional convolution, it has advantages such as fewer parameters and lower computational cost. SIMD / SIMT acceleration platforms provide hardware support for general convolution computations and have corresponding convolution primitives. However, since depthwise convolution does not perform channel fusion, the convolution primitives can only utilize 1 / 64 of the effective computational power. To avoid wasting computational bandwidth, we attempted to implement convolution computation using SIMD instructions for vector multiplication and accumulation.

[0007] However, as mentioned earlier, the parallelism of SIMD depends entirely on the vector register width and the data size covered by the instruction. If the amount of data involved in the computation of a single SIMD instruction is insufficient, or if the load on each computation channel is uneven, it will lead to a significant decrease in the utilization of computing resources, making it difficult to achieve the peak performance of the hardware. This phenomenon is particularly common in depthwise convolution operators and is a structural bottleneck introduced by the characteristics of the network structure.

[0008] In view of this, there is an urgent need to provide an implementation scheme for Depthwise convolution operations for SIMD instruction architecture, so as to make full use of computing resources, avoid wasting computing power, and significantly improve the operational performance of Depthwise convolution. Summary of the Invention

[0009] In order to at least address one or more of the technical problems mentioned above, this disclosure provides a computing device, a chip, a board, a method for performing Depthwise convolution operations using a computing device, and a computer-readable storage medium in several aspects.

[0010] In a first aspect, this disclosure provides a computing apparatus configured to perform a Depthwise convolution operation, including a storage circuit, a control circuit, and a computation circuit, wherein: the storage circuit is configured to store an input tensor X, a weight tensor K, and an output tensor Y; the control circuit is configured to control the Depthwise convolution operation as follows: determining the vectorization length of the output tensor Y, wherein the vectorization length supports vectorization in any number of dimensions N, H, W, C, and M in an order from the inside to the outside according to the memory loading order, where N... The input tensor is defined by the following dimensions: batch dimension, height dimension, width dimension, channel dimension, and channel repetition factor dimension. It also involves traversing each weight point (kh, kw) on the spatial dimensions KH and KW of the weight tensor K, where KH is the weight height dimension, KW is the weight width dimension, and kh and kw are the indices of the weight points. SIMD instructions are invoked to perform vector operations between the weight point vectors and the corresponding input tensors, and the results of these vector operations are accumulated to obtain the output tensor Y. The computation circuit is configured to execute the SIMD instructions under the control of the control circuit.

[0011] In a second aspect, this disclosure provides a chip that includes the computing device described in the first aspect.

[0012] In a third aspect, this disclosure provides a board including the chip described in the second aspect.

[0013] In a fourth aspect, this disclosure provides a method for performing Depthwise convolution operations using the computing device described in the first aspect.

[0014] In a fifth aspect, this disclosure provides a method for performing Depthwise convolution operations using a computing device, comprising: traversing each weight point (kh, kw) on the spatial dimensions KH and KW of the weight tensor K; determining the vectorization length of the output tensor Y, wherein the vectorization length supports vectorization in any of the N, H, W, C, and M dimensions in the order of memory loading from the inside out, where N is the batch dimension, H is the height dimension, W is the width dimension, C is the channel dimension, and M is the channel repetition factor dimension; determining the range of the corresponding input tensor; invoking SIMD instructions to perform vector operations between the weight point vectors and the corresponding input tensors to obtain vector operation results, where KH is the weight height dimension, KW is the weight width dimension, and kh and kw are the indices of the weight points; and controlling the positional summation of the vector operation results obtained from the operations of each weight point to obtain the output tensor Y.

[0015] In a sixth aspect, this disclosure provides a computing device comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, which, when loaded and executed by the processor, cause the computing device to perform the method according to the fifth aspect described above.

[0016] In a seventh aspect, this disclosure provides a computer-readable storage medium storing program instructions adapted for loading by a processor and executing the method of the fifth aspect of this disclosure.

[0017] Based on the above-described scheme for performing depthwise convolution operations, the convolution operation is expanded primarily using the spatial dimensions (KH and KW dimensions) of the weight tensor (convolution kernel). With fixed weight points (kernel points), the output tensor is expanded across multiple dimensions (N, H, W, C, M), theoretically allowing the vectorization length to be extended to N×H×W×C×M. Typically, the spatial size of the feature map (neuron) is much larger than the convolution kernel size; therefore, the scheme of this disclosed embodiment can significantly increase the usable vector length and greatly improve the utilization rate of SIMD instructions. The above and other advantages of this disclosed embodiment will become readily apparent from the following detailed description. Attached Figure Description

[0018] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0019] Figure 1 This diagram shows the structure of the board card according to an embodiment of this disclosure; Figure 2 This diagram illustrates the structure of the combined processing apparatus according to an embodiment of the present disclosure. Figure 3 This diagram illustrates the internal structure of a processor core when the computing device is a single-core or multi-core device. Figure 4 It shows Figure 2 A simplified diagram of the internal structure of a computing device with multiple cores; Figure 5 An exemplary flowchart of a method for performing a Depthwise convolution operation controlled by a control circuit according to an embodiment of this disclosure is shown; Figure 6 A schematic diagram of the processing flow for each weight point on the convolution kernel in a first case according to some embodiments of this disclosure is shown; Figures 7A-7CThe mapping relationship between inputs and outputs is shown in a first case according to an embodiment of this disclosure; Figure 8 A schematic diagram of the processing flow for each point on the convolution kernel in a second case according to some embodiments of this disclosure is shown; Figures 9A-9B The mapping relationship between inputs and outputs is shown in a second case according to an embodiment of this disclosure; Figure 10 An exemplary flowchart of a control method for a third case, based on some embodiments of the present disclosure, is shown; Figures 11A-11D The mapping relationship between inputs and outputs is shown in a third case according to an embodiment of this disclosure; Figure 12 The mapping relationship between input and output is shown in a fourth case according to an embodiment of this disclosure. Detailed Implementation

[0020] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0021] It should be understood that the terms “comprising” and “including” used in this disclosure and claims indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0022] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0023] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0024] The specific embodiments disclosed herein will now be described in detail with reference to the accompanying drawings.

[0025] Exemplary hardware environment Figure 1 A schematic diagram of the structure of a board 10 according to an embodiment of this disclosure is shown. Figure 1 As shown, board 10 includes chip 101, which is a system-on-a-chip (SoC) integrating one or more combined processing devices. These combined processing devices are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in cloud intelligence. A significant characteristic of cloud intelligence applications is the large volume of input data, placing high demands on the platform's storage and computing capabilities. Board 10 in this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0026] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as PCIe (Peripheral Component Interconnect express) interface.

[0027] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller, also known as a microcontroller unit (MCU).

[0028] Figure 2 This is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. (As shown...) Figure 2 As shown, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and a storage device (DRAM) 204.

[0029] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0030] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from on-chip storage device of computing device 201 and transmit it to processing device 203.

[0031] Processing device 203, as a general-purpose processing device, performs basic controls including but not limited to data transfer and starting / stopping computing device 201. Depending on the implementation, processing device 203 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, computing device 201 disclosed herein can be considered as having a single-core structure or a homogeneous multi-core structure. However, when computing device 201 and processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0032] Storage device 204 is used to store data to be processed. It may be DRAM (Dynamic Random Access Memory), DDR (Double Data Rate) memory, typically 16G or larger, used to store data of computing device 201 and / or processing device 203.

[0033] Figure 3 It shows Figure 2 The computing device 201 is a schematic diagram of the internal structure of the processor core when it is a single-core or multi-core device. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three main modules: control module 31 (also known as controller or control circuit), arithmetic module 32 (also known as arithmetic unit or arithmetic circuit), and storage module 33 (also known as memory or storage circuit).

[0034] The control module 31 coordinates and controls the operation of the computation module 32 and the storage module 33 to complete the deep learning task. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 fetches instructions from the processing device 203, and the instruction decode unit 312 decodes the fetched instructions and sends the decoding result as control information to the computation module 32 and the storage module 33.

[0035] The computation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 322 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution. Each processor core may include multiple matrix operation units.

[0036] Storage module 33 is used to store or move relevant data, including Neuron RAM (NRAM) 331, Weight RAM (WRAM) 332, and Direct Memory Access (DMA) 333. NRAM 331 stores input neurons, output neurons, and intermediate results after computation; WRAM 332 stores the convolution kernels, i.e., weights, of the deep learning network; DMA 333 is connected to DRAM 204 via bus 34 and is responsible for data transfer between computing device 301 and DRAM 204. It should be noted that NRAM and WRAM here can be two storage regions formed by dividing the same memory in logical storage space, or they can be two independent memories; no specific limitation is made here.

[0037] Figure 4 It shows Figure 2 The computing device 201 shown is a simplified internal structure diagram for a multi-core system. Multi-core computing devices can be abstracted using a hierarchical hardware model. As shown, the multi-core computing device 400, as a system-on-a-chip, includes at least one computing cluster, and each computing cluster includes multiple processor cores. In other words, the multi-core computing device 400 is a system-on-a-chip... Computational clusters The processor cores are structured in layers.

[0038] From the perspective of the system-on-a-chip hierarchy, as shown in the figure, the multi-core computing device 400 includes an external memory controller 41, a peripheral communication module 42, an on-chip interconnect module 43, a global synchronization module 44, and multiple computing clusters 45.

[0039] There can be multiple external storage controllers 41; two are shown as an example in the figure. These controllers are used to respond to access requests issued by the processor core to access external storage devices (e.g., Figure 2 The DRAM 204 in the chip allows data to be read from or written to external devices. The peripheral communication module 42 is used to communicate via an interface device (…). Figure 2 202) Receives from processing device ( Figure 2 The control signal of 203) activates the computing device ( Figure 2 (201) Performs tasks. The on-chip interconnect module 43 connects the external storage controller 41, the peripheral communication module 42, and multiple computing clusters 45 to transmit data and control signals between the modules. The global synchronization module 44 is, for example, a global barrier controller (GBC) to coordinate the working progress of each computing cluster and ensure information synchronization. The multiple computing clusters 45 are the computing cores of the multi-core computing device 400. Four are exemplarily shown on each die in the figure. With the development of hardware, the multi-core computing device 400 disclosed herein may also include 8, 16, 64, or even more computing clusters 45. The computing clusters 45 are used to efficiently execute deep learning algorithms.

[0040] From the perspective of the computing cluster hierarchy, as shown in the figure, each computing cluster 45 includes multiple processor cores 406 as control and computing units, and also a shared memory core 407 as a storage unit. Furthermore, each computing cluster may also include a local synchronization module 412 to coordinate the working progress of each processor core within the computing cluster, ensuring information synchronization. Four processor cores 406 are exemplarily shown in the figure; this disclosure does not limit the number of processor cores 406.

[0041] The storage core 407 is primarily used for storage and communication, namely storing shared data or intermediate results between processor cores 406, and performing communication between computing clusters 45 and DRAM 204, communication between computing clusters 45, and communication between processor cores 406. In other embodiments, the storage core 407 has scalar operation capabilities and is used to perform scalar operations.

[0042] Storage core 407 includes a shared memory unit (SMEM) 408, a broadcast bus 409, a cluster direct memory access (CDMA) module 410, and a global direct memory access (GDMA) module 411. SMEM 408 acts as a high-performance data relay station. Data multiplexed between different processor cores 406 within the same compute cluster 45 does not need to be obtained from the DRAM 204 by each processor core 406 individually. Instead, it is relayed between processor cores 406 via SMEM 408. Storage core 407 only needs to quickly distribute the multiplexed data from SMEM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access. Broadcast bus 409, CDMA 410, and GDMA 411 are used to perform communication between processor cores 406, communication between compute clusters 45, and data transfer between compute cluster 45 and DRAM 204, respectively.

[0043] From the perspective of processor core hierarchy, the structure of a single processor core can be similar to... Figure 3 The structural diagram shown will not be described in detail here.

[0044] Exemplary Depthwise Convolution Operation Principle Convolutional layers in a neural network model perform convolution operations, extracting features by applying convolutional kernels (also called filters, weights, etc.) to the input feature map (also known as input data, neurons, or input neurons). A neural network model may contain various convolutional layers, such as those performing forward regular 3D convolution operations or those performing forward depthwise convolution operations. In reverse training, it may be necessary to calculate weight gradients and neuron gradients, involving reverse depthwise convolution operations. The embodiments disclosed herein can be optimized for these different types of depthwise convolution operations. These different types of convolution operators are described below.

[0045] First, we define the conventions for the various parameters in the convolution operator, where variables with uppercase letters represent the total number of dimensions, and variables with all lowercase letters represent the index of the corresponding dimension.

[0046] The input tensor X has four dimensions: N, the batch size; H, the height of the input feature map; W, the width of the input feature map; and Ci, the number of input channels. The input tensor X is expressed by the formula: , The H and W dimensions can be viewed as the spatial dimensions of a two-dimensional image; therefore, in this disclosure, the H and W dimensions are also referred to as spatial dimensions.

[0047] The weight tensor K has four dimensions: KH, the height of the convolution kernel; KW, the width of the convolution kernel; Ci, the number of input channels, which is the same as the Ci of the input tensor; and M, the channel repetition factor. The weight tensor K is expressed by the formula: , Similarly, the KH and KW dimensions can also be the spatial dimensions of the weights / convolution kernels.

[0048] The convolution-related parameters are defined as follows: Padding is denoted as: , representing top fill, bottom fill, left fill, and right fill respectively; the stride is denoted as: Let H and W represent the convolution strides in dimensions H and W, respectively; dilation is denoted as: , where represent the expansion coefficients of the H and W dimensions, respectively.

[0049] The output tensor Y has four dimensions: N, the batch size; Ho, the height of the output feature map; Wo, the width of the output feature map; and Co, the number of output channels. The output tensor Y is expressed by the formula: , .

[0050] Once the input tensor X, weight tensor K, and convolution-related parameters are determined, the output tensor Y can be obtained. The parameters are related as follows: Effective kernel size, taking dilation into account. , They are respectively: (1) (2) The spatial dimension of the output tensor is: (3) (4) Channel number relationship: Since the last dimension of the weight tensor K is the channel repetition factor M, that is, the channels are expanded by a factor of M, the number of output channels is: (5) For the sake of a concise representation of the subsequent convolution calculation formula, the output tensor Y is equivalently transformed into 5 dimensions: .

[0051] Next, the convolution calculation is characterized using a formula.

[0052] For forward depthwise convolution calculations, first define the output position ( , The corresponding input sampling coordinates () , ) (including padding, stride, dilation): (6) (7) The formula for calculating forward depthwise convolution can be expressed as: (8) As can be seen from the above formula (8), the forward Depthwise convolution only slides and accumulates KH along the KH and KW dimensions. The sum of KW products is not accumulated in the C-channel dimension.

[0053] The computation of inverse depthwise convolution typically involves calculating the gradients of neurons, and is therefore also called inverse diffX convolution computation. Assuming... (9) in It is the loss function.

[0054] The inverse of the input tensor (diffX) is: (10) The formula for calculating the inverse depthwise convolution can be expressed as: (11) The coordinate indices still satisfy the previous formulas (6) and (7).

[0055] As can be seen from the above formula (11), the inverse Depthwise convolution, in addition to sliding and accumulating KH in the KH and KW dimensions, In addition to the sum of KW products, it is also accumulated in the M dimension, which means that the M-fold expansion of channels is restored to the original number of channels.

[0056] In this paper, input feature maps, input data, neurons, or input neurons are used interchangeably; convolution kernels, filters, or weights are used interchangeably; and output feature maps, output data, and output neurons are used interchangeably. Furthermore, regardless of the type of convolution operation involved, the data can be divided into input feature maps, convolution kernels, and output feature maps. For example, in the inverse operation, dY corresponds to the input feature map, the weights K correspond to the convolution kernel, and dX corresponds to the output feature map.

[0057] It is important to note that the neural networks provided in this disclosure can be applied to various fields and can process various types of data. For example, in image recognition and processing, such as facial recognition, object detection, image classification, and medical image analysis, data types can include pixel data, image features, etc.; in natural language processing (NLP), such as language translation, sentiment analysis, text summarization, and speech recognition, data types can include text data, word embeddings, sentence structures, etc.; in speech recognition and processing, such as intelligent assistants, automatic caption generation, and speech-to-text conversion, data types can include audio signals, spectrograms, etc.; in recommendation systems, such as personalized content recommendation, product recommendation, and advertising, data types can include user behavior data, item features, rating data, etc.; in healthcare, such as disease diagnosis, drug discovery, and gene sequence analysis, data types can include medical records, biomarker data, genomic data, etc.; in the financial field, such as risk assessment, fraud detection, and stock market prediction, data types can include transaction data, user credit scores, market data, etc.; and in autonomous driving, such as vehicle perception, decision-making, and path planning, data types can include sensor data, environmental features, traffic signals, etc. In gaming and entertainment, AI applications such as game AI, virtual reality, and animation generation can include data types like game state data and user interaction data. In scientific research, applications such as physics simulation, chemical compound prediction, and astrophysical analysis can include data types like experimental data, simulation results, and observational data. Generative AI applications are another example. Generative AI refers to artificial intelligence technologies that use complex algorithms, models, and rules to learn from large-scale datasets to create new, original content, including, but not limited to, text, images, audio, video, and code. Correspondingly, the input neuron data can be image data, audio data, video data, speech data, text data, document data, etc., and the corresponding output neuron data can also be image data, audio data, video data, speech data, text data, document data, etc. The output of the neural network can include a probability score of an image belonging to a specific object category, a probability score of a document being about a specific topic, a probability score of the correct translation of a text fragment in the target language from a text fragment in the source language, or a probability score of the correct transcription of a text fragment from spoken language.

[0058] Exemplary SIMD instructions implement a depthwise convolution scheme On various SIMD / SIMT acceleration platforms, the implementation of convolution operators usually follows a unified idea: to abstract convolution as sliding a spatial window on the input tensor and accumulating the elements and weights within the window through multiplication and addition.

[0059] Some platforms provide hardware support for general convolution calculations and have corresponding convolution primitives. However, depthwise convolution does not perform fusion in the C-channel dimension, so the convolution primitives can only utilize 1 / 64 of the effective computational power. To avoid wasting computational bandwidth, we attempted to implement convolution calculations using SIMD instructions for vector multiplication and accumulation.

[0060] For forward depthwise convolution computation, the core idea of ​​using SIMD instructions is: the weight tensor K slides on the input tensor X according to the output position. Each slide completes a local multiplication-addition reduction, resulting in the output position.

[0061] For inverse depthwise convolution computation, When all values ​​are 1, rotating the weight tensor K by 180 degrees converts it into a forward depthwise convolution calculation. When not all values ​​are 1, there are two approaches: One approach is to perform "zero-padding" on dY, which involves inserting (stride - 1) zeros between two adjacent valid dY values. The core idea is to preemptively eliminate the influence of the stride. This method is mathematically very intuitive and can directly reuse the forward convolution calculation process. However, it introduces many unnecessary memory accesses during copying and multiplies many invalid zeros during computation, which is detrimental to high-performance implementation.

[0062] Another approach is to group the weight tensors K by stride, with each group of convolutional kernels handling only one type of spatial alignment. For each group of kernels, perform a backward convolution with stride=1 using the corresponding dY (this can be converted to a forward convolution calculation using the same stride-1 approach as before). When writing back to diffX, offset by stride and add the result to the correct position. This method avoids introducing unnecessary computations and is commonly used in engineering implementations.

[0063] For example, the pseudocode implementation of SIMD instructions for forward and backward depthwise convolution can be as follows: for n in [0, N): / / 1. Process batch by batch for ho in [0, Ho): / / 2. Output height for wo in [0, Wo): / / 3. Output width T = 0 for kh in [0, KH): / / 4. Kernel height / / SIMD vectorization computation has dimensions of KW, C, M / / Forward convolution calculation if (conv) { / / 1. Extract the value at the corresponding position in the input feature map X_L / / 2. If multiply > 1, broadcast X_L to obtain X_L1 X_L1 / / 3. Extract the values ​​at the corresponding positions of the convolution kernel and multiply them. Buffer = X_L1 W_L / / 4. Calculate the convolution result T += reduce_sum(Buffer, KW) } else { / / Inverse convolution calculation / / 1. Extract the value at the corresponding position in the input feature map dY_L / / 2. Extract the values ​​at the corresponding positions of the convolution kernel and multiply them. Buffer = dY_L W_L / / 3. If multiply > 1, sum each multiply instance into one. Buffer = reduce_sum(Buffer, multiply) / / 4. Calculate the convolution result T += reduce_sum(Buffer, KW) } / / 5. Write the result back to video memory write_back(T) As can be seen from the pseudocode above, the core idea of ​​the calculation is still to calculate the results at each output position in the order of N->Ho->Wo. For each output position, the value at the corresponding convolution window position on the input feature map is extracted, multiplied and accumulated with the weights to obtain the value at the output position.

[0064] The above solution has the following problems: First, there's the issue of short vectorization length. In the implementation described above, the vectorization computation of SIMD instructions mainly unfolds along three dimensions: KW, C, and M. Theoretically, the resulting vector length can be KW×C×M. However, in actual network structures, these dimensions are usually quite small: KW is typically 1, 3, or 5, the channel repetition factor M is often small, and in depthwise convolutions or lightweight networks, the number of input channels C may also be low.

[0065] In the above scenario, the actual data size involved in vectorization computation is much smaller than the ideal vector width of hardware SIMD, which can easily lead to idle vector channels and underutilization of computational resources, resulting in wasted computing power. This phenomenon is particularly common in depthwise convolution operators and is a structural bottleneck introduced by the characteristics of the network structure.

[0066] Second, the reverse diffX calculation requires additional weight rotation processing. In order to reuse the calculation process of the forward convolution, the weight tensor K is typically rotated 180 degrees in space before the reverse diffX calculation. This rotation operation requires preprocessing the entire weight tensor, rearranging its original spatial layout.

[0067] This process is not well-suited for efficient implementation using SIMD instructions and often requires additional temporary storage space to store the rotated weights. Therefore, in reverse computation, not only is an extra data processing step added, but additional memory access and storage overhead are also introduced, which has a certain impact on overall performance.

[0068] Third, the storage reuse rate is low under the sliding window computation method. Convolution operators typically use a sliding window for computation, that is, at each output position, a multiplication-addition-reduce operation is performed on a local window corresponding to the input tensor. Due to the limitations of computing resources and available storage space, the input data and weight data within the window often cannot be loaded into the contiguous memory space required for computation at once, and can only be loaded in batches and participate in the computation.

[0069] In this scenario, the calculation process for one output position may require loading data within the same window multiple times. When the sliding window moves to the next output position, even though the new window highly overlaps with the previous window in space, the previously loaded data cannot be fully preserved or reused, so these overlapping areas still need to be loaded multiple times again.

[0070] The result is that a large amount of input data that could have been reused is repeatedly moved between adjacent output positions, significantly increasing the number of data loading operations, increasing storage bandwidth pressure, and resulting in low overall data reuse efficiency. This repeated loading phenomenon is particularly pronounced when the feature map size or window size is large, thus limiting the overall performance of the operator.

[0071] Exemplary SIMD instructions implement a Depthwise convolution optimization scheme To address one or more of the problems existing in the Depthwise convolution computation mentioned above, this disclosure proposes a high-performance implementation scheme for the Depthwise convolution operator oriented towards the SIMD instruction architecture. This scheme redesigns the unfolding method of convolution computation from the perspective of the computational structure itself, thereby improving vectorization efficiency and reducing unnecessary data movement.

[0072] The core idea of ​​this scheme is that in the case of depthwise convolution, there is no fusion relationship between channels. Therefore, when traversing the spatial dimensions (KH and KW dimensions) of the weight tensor K and fixing the position of a certain weight point (kernel space point), a clear linear mapping relationship is presented between the input tensor X and the output tensor Y. This mapping relationship has good continuity in the channel dimension and the channel repetition dimension, making it naturally suitable for vectorized computation using SIMD instructions.

[0073] Based on this characteristic, this scheme shifts the computational focus from the traditional sliding window mode to a computational unfolding method dominated by the dimension of the weight tensor space, thereby making fuller use of the parallel computing capabilities of the SIMD instruction architecture while maintaining computational correctness.

[0074] The following description will continue to use the notation and parameter conventions described above to introduce the design concept and implementation process of this high-performance Depthwise convolution algorithm.

[0075] The Depthwise convolution operation described in this disclosure is implemented based on the hardware environment described above. Specifically, embodiments of this disclosure provide a computing device configured to perform Depthwise convolution operations, including a storage circuit, a control circuit, and a computation circuit, wherein: the storage circuit is configured to store an input tensor X, a weight tensor K, and an output tensor Y; the control circuit is configured to control the Depthwise convolution operation as follows: determine the vectorized length of the output tensor, traverse each weight point in the spatial dimension of the weight tensor, call SIMD instructions to perform vector operations between the weight point vector and the corresponding input tensor, and accumulate the vector operation results to obtain a vectorized output tensor; the computation circuit is configured to execute SIMD instructions under the control of the control circuit to perform the Depthwise convolution operation.

[0076] Figure 5 An exemplary flowchart of a method for performing a Depthwise convolution operation controlled by a control circuit according to an embodiment of this disclosure is shown.

[0077] As shown in the figure, in step 510, the vectorization length of the output tensor Y is determined. This vectorization length supports vectorization of any number of dimensions in the N, H, W, C, and M dimensions, in the order of memory loading from the inside out. Here, N is the batch dimension, H is the height dimension, W is the width dimension, C is the channel dimension, and M is the channel repetition factor dimension. Specifically, the vectorization length of the output tensor can be determined by taking several dimensions in the N, H, W, C, and M dimensions, in the order of memory loading from the inside out, based on the size of the output tensor Y in these dimensions and the hardware configuration information of the computing device. The hardware configuration information of the computing device may include, but is not limited to, the available memory capacity limit within the processor core and the maximum amount of data that the computing device can process.

[0078] Next, in step 520, it is checked whether there are any unprocessed weight points in order to traverse each weight point (kh, kw) on the spatial dimensions KH and KW of the weight tensor K, where KH is the weight height dimension, KW is the weight width dimension, and kh and kw are the indices of the weight points. If so, that is, there are still unprocessed weight points, then proceed to step 530.

[0079] Next, in step 530, the range of the corresponding input tensor is determined. As can be seen from the previous analysis of the convolution operation principle, when the weight points are fixed, there is a linear mapping relationship between the coordinate indices of the output tensor Y and the input tensor X in the spatial dimensions (H and W dimensions) (see formulas (6) and (7)). Therefore, during the calculation process, each time a weight point is loaded, the input region of the required input tensor can be determined in reverse based on the output region of the current output tensor.

[0080] It's understandable that, since some data from the input neurons used by different weight points overlap during traversal, the maximum possible input tensor region can be pre-loaded. Then, during subsequent calculations for each weight point, the corresponding input data can be selected by controlling the start and end indices. This approach effectively reduces memory access and avoids repeatedly loading data.

[0081] Specifically, in some embodiments, the range of the corresponding input tensor can be determined as follows: the dimension of the corresponding input tensor is determined based on the dimension included in the vectorized length of the output tensor Y; and the range of the corresponding input tensor is determined based on the linear mapping relationship between the coordinate indices of the output tensor Y and the input tensor X in the H and / or W dimensions at the current weight point. According to the principle of convolution, besides the linear mapping relationship in spatial dimensions, there is a fixed correspondence between the input tensor and the output tensor in other dimensions. Therefore, the region of the input tensor can be inferred from the region of the output tensor.

[0082] Furthermore, in some embodiments, the control circuit can be further configured to determine the range of the corresponding input tensor as follows: based on the padding parameter in the convolution parameters of the Depthwise convolution operation, determine the valid start and end positions of the input tensor corresponding to the current weight point in the W and / or H dimensions. Due to the presence of the convolution padding parameter, invalid padding values ​​may exist in the input tensor. The range of the input tensor corresponding to each weight point (kh, kw) may be different; therefore, during calculation, it is necessary to individually confirm whether the input index of the input tensor corresponding to that weight point (kh, kw) is out of bounds to ensure that boundary conditions are handled correctly. Input data with indices exceeding the boundary can be discarded to avoid invalid calculations.

[0083] Next, in step 540, SIMD instructions are invoked to perform vector operations between the weight point vectors and the corresponding input tensors, yielding the vector operation result. Depending on the type of Depthwise convolution operation, different SIMD instructions can be invoked to implement the Depthwise convolution operation.

[0084] Given enough space to fill the space, the shape of the vector of weight points is [C,M], and the shape of the input tensor is [N,H,W,C,1]. The shape of the input tensor can be considered as [N...M]. H [W,C,1], logically performs a broadcast multiplication of these two vectors. Therefore, in some embodiments, the control circuit can be further configured to invoke SIMD instructions as follows: in response to a forward Depthwise convolution operation, invoke a first vector operation instruction to broadcast or copy the lowest dimension C of the input tensor into M copies and then perform a multiplication operation with the weight vector. In the forward Depthwise convolution operation, the hardware supports [1,W]. The [H,W] pattern represents a broadcast multiplication instruction. If the channel repetition factor M equals 1, the broadcast multiplication instruction is invoked, and the broadcast multiplication calculation can be viewed as [1, C]. [N H [W, C]. If the channel repetition factor M is greater than 1, then the C channel dimension needs to be broadcast or copied into M copies before performing the calculation.

[0085] Depending on the types of operations supported by the instruction architecture and the cost of copy expansion, the above vector operations can be performed by calling broadcast multiplication instructions or using loop processing (e.g., for loops). In some implementations, the control circuit can be further configured to call SIMD instructions as follows: when M is extremely small, the input tensor is expanded to [N, H, W, C, M] for multiplication, and the hardware-supported broadcast multiplication instructions are called. In this case, the broadcast multiplication calculation can be regarded as: [1, C M] [N H W,C When M is of medium size, the vector of weight points can be transposed to [M, C], and processed M times in a for loop. Each calculation can be considered as [1, C]. [N H If M is particularly large, the shape of the weight points can be equivalently considered as [1,C,1], and the shape of the input tensor is [N,H,W,C,M] (equivalent to [N...H...W...C...M]). H W,C,M]), for loop N H W C times, each time performing multiplication in the lowest dimension, is equivalent to [1]. [M].

[0086] In other embodiments, the control circuit may be further configured to invoke SIMD instructions as follows: in response to a depthwise convolution operation being a reverse depthwise convolution operation, invoke a second vector operation instruction to perform vector multiplication of the weight point vector with the corresponding input tensor and to accumulate the channel repetition factor dimension M.

[0087] continue Figure 5 In step 550, the vector operation results obtained from the control weight point operation are accumulated positionally.

[0088] To avoid repeatedly writing back to main memory, a buffer can be constructed for the output tensor Y, storing the cached tensor T. This buffer is cleared before each calculation. For different weight points, the result of the vector operation is only needed to be accumulated into the corresponding position in the buffer based on the offset of that weight point.

[0089] In these embodiments, the aforementioned storage circuit is further configured with a buffer area for storing a buffer tensor T. A linear mapping exists between the buffer tensor T and the coordinate indices of the input tensor X in the H and / or W dimensions, and a translation relationship exists between the buffer tensor T and the coordinate indices of the output tensor Y in the H and / or W dimensions. At this time, the control circuit is further configured to perform positional accumulation of the control vector operation results as follows: based on the indices of the currently used weight point vector in the KW and KH dimensions, a translation value corresponding to the current vector operation result is determined; based on this translation value, the current vector operation result is accumulated to the corresponding position of the buffer tensor T.

[0090] This completes the processing of a single weight point. After traversing all weight points, i.e., when the judgment result in step 520 is negative, the process can proceed to step 560, where the output tensor Y is obtained based on the positional accumulation result of all weight points. Specifically, the control circuit can be further configured to obtain the output tensor Y by controlling the process as follows: after traversing all weight points, the data in the buffer tensor T is written back to the output tensor Y according to the aforementioned translation relationship between T and Y.

[0091] As can be seen from the above scheme, the starting point of this disclosed embodiment stems from a re-examination of the structural characteristics of convolutional computation. A key observation is that when the spatial location (kh, kw) of the convolutional kernel is fixed, the input tensor X and the output tensor Y no longer exhibit a complex mapping relationship that changes with window movement, but degenerates into a deterministic and regular linear mapping relationship. Especially in the case of depthwise convolution, this mapping is completely independent and continuous in the channel dimension (C) and its repetition dimension (M), while it only manifests as regular index offsets in the spatial dimension and batch dimension. This structural feature means that once the kernel point is determined, the essence of convolutional computation can be transformed from "multiplication and addition reduction within a local window" to "batch linear accumulation across multiple locations," thus naturally adapting to the parallel execution model of SIMD instructions in terms of computational form.

[0092] Therefore, in the above scheme, by unfolding the convolution operation primarily based on the spatial dimensions (KH and KW dimensions) of the weight tensor (convolution kernel), and with fixed weight points (kernel points), the output tensor is expanded and calculated across multiple dimensions (N, H, W, C, and M). Theoretically, the vectorization length can be extended to N×H×W×C×M. Typically, the spatial size of the feature map (neuron) is much larger than the convolution kernel size; therefore, the scheme in this disclosed embodiment can significantly increase the usable vector length and greatly improve the utilization rate of SIMD instructions.

[0093] The aforementioned computing device may have one or more processor cores. In this hardware environment, regardless of whether inter-core partitioning (multi-core parallel processing) or intra-core partitioning (single-core multi-round loop processing) is used, the order of partitioning computational dimensions follows the hierarchical relationship of N, H, W, C, M. That is, N-dimensional partitioning is performed first, followed by H-dimensional partitioning, and so on.

[0094] In the case of inter-core partitioning, the output tensor is divided according to the processor cores (computation cores). Although different computation cores may have some overlapping access to the input tensor during the computation process, since each output element is always written by only one computation core, there is no conflict between output writes, thus avoiding the atomic addition operations required for concurrent writes across multiple cores.

[0095] When splitting within a core, the limitations of available memory capacity within a single computing core must be considered. For memory loading, the strategy follows an inside-out approach, prioritizing expansion in the innermost M dimension, followed by the C dimension, and then gradually extending to the W, H, and N dimensions.

[0096] In the above scheme, since the Depthwise convolution calculation process is no longer constrained by the requirement that "a single window must be completed at once", the splitting of the calculation segment is more flexible and supports vectorized lengths of any number of dimensions. Therefore, the dimensions such as N, H, and W can be adaptively split according to the on-chip storage capacity, thereby efficiently organizing the calculation within the limited on-chip space.

[0097] If the M and C dimensions are large and the processor core's memory utilization is high enough, the remaining dimensions of the output tensor can be processed in a single-point manner, making the implementation logic relatively simple. In these embodiments, when traversing the weight tensor K, the control circuit is further configured to control the calculation process as follows: in response to M×C being greater than or equal to a first threshold, the calculation is performed in a single-point manner along the hierarchical order of the N, H, and W dimensions, where the vectorization length is ≤ M×C. The specific vectorization length can be determined based on the maximum amount of data that the processor core can process. If the maximum number of C dimensions that the processor core can process is max_seg_c, then the vectorization length is M×max_seg_c. The first threshold can be set based on the available memory capacity of the processor core.

[0098] When the M and C dimensions are small, the processor core's memory utilization is too low. However, when the W dimension is large, and the processor core's memory utilization is high enough after adding the W dimension, the H and N dimensions can be processed one by one. In this case, the calculation process can be simplified to a one-dimensional (1D) scene. That is, from the perspective of coordinate mapping, the H dimension has a size of 1, and there is only the W dimension, so the spatial dimensions (H, W) are simplified to 1D. In these embodiments, when traversing the weight tensor K, the control circuit is further configured to control the calculation process as follows: in response to M×C being less than a first threshold but M×C×W being greater than or equal to the first threshold, the calculation is split and calculated in a single-point manner according to the hierarchical order of the N and H dimensions. At this time, the vectorization length is ≤ M×C×W. If the maximum number of W dimensions that the processor core can process is max_seg_w, then the vectorization length is M×C×max_seg_w.

[0099] When the dimensions M, C, and W are all small, and the memory utilization of the processor core is too low, the computation can naturally be extended to two-dimensional (2D) scene processing. That is, from the perspective of coordinate mapping, multiple points in the H dimension need to be introduced. The H dimension is greater than 1, and the W dimension is also greater than 1, so the spatial dimension is 2D. In these embodiments, when traversing the weight tensor K, the control circuit is further configured to control the computation process as follows: in response to M×C×W being less than a first threshold, the computation is split into single points in the N dimension, at which point the vectorization length is ≤ M×C×W×H; or, in response to M×C×W×H being less than the first threshold, no splitting computation is required, at which point the vectorization length is ≤ M×C×W×H×N. If, in the case of splitting computation into single points in the N dimension, the maximum number of H dimensions that the processor core can process is max_seg_h, then the vectorization length is M×C×W×max_seg_h. If the maximum number of N dimensions that a processor core can process is max_seg_n, then the vectorization length is M×C×W×max_seg_n.

[0100] Regardless of whether the scene is 1D or 2D, and regardless of whether the convolution stride is 1 or greater, the convolution operation scheme of this embodiment can be adapted using a unified framework. Therefore, based on the aforementioned calculation core of weight points, an output buffer reconstruction scheme is added. As mentioned earlier, a buffer tensor T is constructed in the buffer area. There is a linear mapping relationship between the buffer tensor T and the coordinate indices of the input tensor X in the H and / or W dimensions, and a translation relationship between the buffer tensor T and the coordinate indices of the output tensor Y in the H and / or W dimensions. Furthermore, to support 2D scenes, a precise masking mechanism can be used to process boundaries and invalid regions, enabling unified vectorized computation across multiple dimensions such as N, H, W, C, and M in two-dimensional and even higher-dimensional scenes.

[0101] The following sections will proceed from easiest to most difficult, first introducing the processing of 1D scenes, then 2D scenes. In 1D scenes, only the W dimension remains; at this point, we can further process the data using the stride dimension. w Whether it is 1 point or not, two cases will be handled.

[0102] First scenario: 1D scene, The value is 1.

[0103] In this case, the control circuit can be further configured to: set the translation amount in the translation relationship between the cache tensor T and the output tensor Y in the W-dimensional dimension to 0; and set the linear mapping relationship between the cache tensor T and the input tensor X in the W-dimensional dimension to correspond to the linear mapping transformation between the output tensor Y and the input tensor X in the W-dimensional dimension. In other words, in the first case, the cache tensor T and the output tensor Y have the same layout.

[0104] At this point, the formula for calculating forward depthwise convolution can be simplified to: (12) As can be seen from formula (12), when traversing the KW dimensions of the convolution kernel, the KW dimension is determined. The indices of X and Y in the W dimension only undergo translation transformation.

[0105] Figure 6 A schematic diagram of the processing flow for each weight point on the convolution kernel is shown in a first case according to some embodiments of this disclosure.

[0106] As shown in the figure, boundary processing is first performed in step 610. Because each The range of the corresponding input neurons may differ, therefore it is necessary to first determine this range during computation. Checking if the index of the corresponding input neuron is out of bounds ensures that boundary conditions are handled correctly. Out-of-bounds indices can be removed directly.

[0107] Next, in step 620, the weight tensor K and the input tensor X can be directly vectorized and loaded along contiguous memory. Specifically, for the weight tensor K, the currently processed... This data That is, its shape is [C, M], and it is stored contiguously in memory.

[0108] For the input tensor X, since the lowest dimension (C dimension) of X needs to be broadcast, we can handle it in different cases: When the channel dimension C and the channel repetition factor M are large, for example, C×M exceeds a specified threshold, the input tensor X can be traversed one by one along the W dimension, and the data of the corresponding point on the W dimension can be loaded. Its shape is [C, 1]. Then press [C, M]. Multiplication is performed in the broadcast form of [C, 1], where [C, M] are the weights and [C, 1] are the input neurons.

[0109] When the channel dimension C and the channel repetition factor M are small, for example, C×M is less than a specified threshold, the lower dimension (i.e., dimension C) of the input tensor X can be copied M times before performing the [C, M] operation. [N Multiplication in the case of W, C, M].

[0110] Finally, in step 630, the multiplication results are accumulated in the output buffer to obtain the final convolution output.

[0111] The calculations above are performed primarily based on the spatial dimension of the convolution kernel. In the first 1D scenario, the calculations are traversed along the KW dimension of the convolution kernel. For different Only need to be based on The corresponding offset is used to accumulate the result of this calculation into the corresponding position in the output buffer. Finally, the result in the buffer can be directly used as the output tensor Y, and then copied back to main memory after the calculation is completed.

[0112] For inverse depthwise convolution, the calculation formula can be simplified to: (13) In the computation of reverse Depthwise, the main difference from forward Depthwise is that broadcasting is no longer performed. Instead, the channel repeating dimension M is reduced and accumulated directly after multiplication. The rest of the processing flow, including boundary processing and buffer accumulation strategy, is consistent with forward Depthwise convolution.

[0113] To illustrate the convolution process more clearly, graphical examples are provided below. In these examples, it is assumed that the width W of the input tensor X is 5, and the width KW of the weight tensor K is 3. , All are 1.

[0114] Figure 7A This illustrates the case of traversing the first point of the KW-dimensional convolution kernel. As shown in the figure, selecting points within the KW-dimensional convolution kernel... The first point (the shaded box) shows the value of 0. The figure illustrates the linear mapping between the output tensor Y and the input tensor X (in the first case, this linear mapping is also a translation relationship), where the output tensor Y is marked in dark and X in light. Each box shows the HW space coordinates corresponding to the input tensor X and the output tensor Y.

[0115] As shown in the figure, when performing a forward depthwise convolution, the effective region of X in dimension W is [0, 3]. The correspondence between the starting positions is: the starting position of X, x(0,0), corresponds to y(0,1), and the offset of Y relative to X in dimension W is +1. When performing a backward diffX convolution, the effective region of Y in dimension W is [1,4], and the offset of X relative to Y in dimension W is 0 (illegal values ​​have been clipped).

[0116] Figure 7B This illustrates the case of traversing the second point along the KW dimension of the convolution kernel. As shown in the figure, selecting points along the KW dimension of the convolution kernel... The second point (the slashed box) equals 1. The figure shows the mapping relationship between the output tensor Y and the input tensor X. Similarly, the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0117] As shown in the figure, when performing a forward depthwise convolution, the effective region of X in dimension W is [0, 4], and the offset of Y relative to X in dimension W is 0. When performing a backward diffX convolution, the effective region of Y in dimension W is [0, 4]. The correspondence between the starting positions is: the starting position of X, x(0,0), corresponds to y(0,0), and the offset of X relative to Y in dimension W is 0.

[0118] Figure 7C This illustrates the case of traversing the third point of the KW dimension of the convolution kernel. As shown in the figure, selecting points within the KW dimension of the convolution kernel... =2, the third point (the slashed box). The figure shows the mapping relationship between the output tensor Y and the input tensor X. Similarly, the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0119] As shown in the figure, when performing a forward depthwise convolution, the effective region of X in dimension W is [1, 4]. The correspondence between the starting positions is: the starting position of X, x(0,1), corresponds to y(0,0), and the offset of Y relative to X in dimension W is 0 (illegal values ​​have been clipped). When performing a backward diffX convolution, the effective region of Y in dimension W is [0,3], and the offset of X relative to Y in dimension W is +1.

[0120] The second scenario: 1D scene, Greater than 1.

[0121] Recalling the previous text, in Depthwise convolution, the output position... With input position The mapping relationship between them is as follows: (7) when When the value is greater than 1, output the index of tensor Y. index of input tensor X The relationship between them is no longer a simple translation. As mentioned earlier, there are two approaches to this: zero-insertion expansion or multiple loading in groups by stride. These approaches either introduce a large amount of unnecessary I / O and multiplication operations, or require each group to load input data independently, resulting in poor reusability.

[0122] This disclosure provides a unified framework that effectively avoids the two problems mentioned above through variable substitution and cache design.

[0123] Specifically, firstly, by introducing and The parameter incorporates the effect of the stride into the index transformation.

[0124] In this case, the control circuit can be further configured to: introduce bias parameters and This ensures a linear mapping between the cache tensor T and the input tensor X in the W-dimensional coordinates, and a translational relationship between the cache tensor T and the output tensor Y in the W-dimensional coordinates. This is achieved by introducing a bias parameter. and It can absorb the effect of the convolution stride into the index, allowing for the unified processing of forward depthwise convolution and reverse diffx convolution while maintaining the efficiency of SIMD, without introducing invalid zero padding or repeatedly loading data.

[0125] Secondly, the grouping cardinality is introduced. The spatial points of the convolution kernel (referred to as kernel points) are arranged according to... Grouping, eliminating strides within each group, and reusing the same cache T.

[0126] Specifically, the control circuit can be further configured to: introduce a grouping base. All value points are grouped by base number Grouping is performed where weight points within the same group have the same bias parameter. The corresponding input tensors have overlapping regions, and the same cached tensor T is used for result accumulation; different groups have different bias parameters. There is no overlap between the input tensors corresponding to different groups.

[0127] In this way, forward and reverse computations can share the same set of index mapping logic. This scheme avoids the invalid computation and I / O caused by zero-insertion expansion in traditional methods, and also avoids the overhead of repeatedly loading input data after grouping. It can be applied to general scenarios with arbitrary stride, dilation, and padding. The following is a detailed description of this scheme.

[0128] The simplified formula for calculating forward depthwise convolution in a 1D scene is as follows: (14) The simplified formula for calculating inverse depthwise convolution in a 1D scene is as follows: (15) From the output location above With input position Starting from the mapping relationship formula (7), the goal is to find a continuous integer index. , making It can be expressed in the form of formula (16), and and The only difference between them is a constant: (16) Therefore, a selectable smallest natural number is introduced. and an integer , so that: (17) And define: (18) From formulas (17) and (18), it can be seen that for a given... In other words, and They are all constants.

[0129] At this point, the mapping formula (7) can be rewritten as:

[0130]

[0131] (19) remember: (20) It can correspond to an intermediate cache tensor T, which is cached in the cache area, and its W-dimensional index relationship satisfies formula (20).

[0132] As can be seen from the above description (Formulas 17 and 18), for a fixed kernel point (i.e., a fixed...), ), and All are constants. From formula (20), it can be seen that there is only a translation mapping relationship between the cache tensor T and the output tensor Y, that is, the output index... With cached tensor index It is a translation relationship. From formula (16), it can be seen that there is a linear mapping between the buffer tensor T and the input tensor X, and >=1, satisfying the injective condition. When When traversing all valid output positions, It also traverses a continuous interval of integers and maps... → w It is injective (because stride ≥ 1).

[0133] This means that if we construct an intermediate cache tensor T, its space index is... Therefore, the relationship between T and the output tensor Y is simply a translation (index offset). ), and the relationship between T and the input tensor X is Sampling relationship. In this way, the effect of the stride is absorbed into the definition of T, and does not need to appear in the main calculation loop. Therefore, whether it is a forward or backward depthwise convolution, by properly handling the mapping relationship of the cached tensor T during the traversal of the weight tensor K, an effective output result can be obtained.

[0134] That is, the coordinate index of the cache tensor T in the W dimension. Coordinate index of input tensor X in dimension W They have the following linear mapping relationship: Bias parameters It makes The smallest natural number that holds true, where It is the KW dimension index of the current weight point. It is the dilation parameter in the KW dimension. It is the left padding parameter. This is the convolution stride in dimension W. The coordinate index of the cache tensor T in dimension W. Coordinate index of the output tensor Y in the W dimension The following translation relationship exists between them: Bias parameters .

[0135] Furthermore, it can be seen from formula (17) that, It is the smallest natural number that satisfies the congruence condition. Different It may lead to different If each If each kernel point has its own independent cache tensor T, then the input data needs to be loaded separately, resulting in poor reusability.

[0136] In some embodiments disclosed herein, the kernel points of the convolution kernel can be grouped to avoid the aforementioned problems.

[0137] Specifically, a selectable minimum positive integer is introduced. , so that: (twenty one) this It has the following properties: when Increase When, the value of the congruence increases It is exactly multiples of integers, therefore It remains unchanged.

[0138] Therefore, the kernel points of the weight tensor K in the KW dimension are modulated. Divide into several groups, and all within the same group Having the same Values ​​can share a single cache tensor. Input tensors X corresponding to cache tensors T within the same group have overlapping regions. Values ​​from different groups... Different, its effect on Since the moduli are different, the input tensors X corresponding to the cached tensors in different groups have no overlapping regions. Because the input tensors corresponding to different groups do not overlap, they can be processed independently and in parallel.

[0139] Figure 8 A schematic diagram of the processing flow for each point on the convolution kernel is shown in a second case according to some embodiments of this disclosure.

[0140] As shown in the figure, firstly in step 810, the weight tensor K is adjusted according to... Divide into different sub-weight groups, and share a common buffer to store the buffer tensor T. Initialize the buffer by flushing it with 0.

[0141] Next, in step 820, within each sub-weight group, operations are performed on each weight point to determine the effective range of the corresponding input tensor. Then, in step 830, the vectorization loading of the input tensor and the SIMD instruction operation with the weight point vector are performed. During forward convolution calculations, due to stride... w Since the input tensor is greater than 1, it needs to be vectorized and loaded accordingly with a stride. The SIMD instruction operation is the same as in the first case. Finally, in step 840, buffer accumulation and output tensor copying are performed. Similarly, during the inverse diffX convolution calculation, the output tensor is copied back accordingly with a stride. Unless otherwise specified, the processing steps are similar to those in the first case and will not be elaborated here.

[0142] To illustrate the convolution process more clearly, graphical examples are provided below. In these examples, it is assumed that the width W of the input tensor X is 7, and the width KW of the weight tensor K is 2. Both are 1. It is 2. The value is 3. In these examples, Since KW is 2, the weight K is divided into 2 sub-weight groups, each containing one weight point.

[0143] Figure 9A This illustrates the case where the first point is traversed within the first sub-weight group of the KW-dimensional convolution kernel. As shown in the figure, selecting points within the KW-dimensional convolution kernel... Points equal to 0 (shaded boxes). The figure shows the mapping relationship between the output tensor Y and the input tensor X, where the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0144] As shown in the figure, when performing a forward depthwise convolution, the effective region of X in dimension W is (2,5), and the offset of Y relative to X in dimension W is +1. When performing a backward diffX convolution, the effective region of Y in dimension W is [1,2], and the offset of X relative to Y in dimension W is +2.

[0145] Figure 9B This illustrates the case of traversing the first point in the second sub-kernel group along the KW dimension of the convolution kernel. As shown in the figure, selecting points along the KW dimension of the convolution kernel... Points equal to 1 (cut-out boxes). The figure shows the mapping relationship between the output tensor Y and the input tensor X. Similarly, the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0146] As shown in the figure, when performing a forward depthwise convolution, the effective region of X in dimension W is (1,4), and the offset of Y relative to X in dimension W is 0. When performing a backward diffX convolution, the effective region of Y in dimension W is [0,1], and the offset of X relative to Y in dimension W is +1.

[0147] The third scenario: 2D scene. and It equals 1.

[0148] Compared to 1D scenes, 2D scenes have an additional H-dimensional coordinate relationship that needs to be handled.

[0149] In some embodiments, in a 2D scene, when and When the value equals 1, the calculation can be decomposed into traversing each weight point (kh, kw) in the spatial dimensions KH and KW of the convolution kernel. After determining kh and kw, there is only a translation transformation between the input tensor X and the output tensor Y. If each weight point is processed cyclically in the H (Ho) dimension, the processing of the remaining W, C, and M dimensions can be referenced from the 1D scene.

[0150] However, this method cannot vectorize the H (Ho) dimension. When the total number of W, C, and M dimensions is small, the degree of vectorization is still low.

[0151] To address this, this disclosure further proposes a method that can also vectorize the N and H (Ho) dimensions. The core idea of ​​this method is to supplement additional columns to vectorize the N and Ho dimensions as well. This involves processing multiple rows of data at once, then eliminating the influence of invalid columns using an integer mask, and finally pruning invalid columns when copying back the output tensor. This approach allows the SIMD vectorization length to be extended to N×Ho×Wo×C×M, thereby significantly improving parallelism.

[0152] Figure 10 An exemplary flowchart of a control method for a third case, based on some embodiments of the present disclosure, is shown.

[0153] As shown in the figure, in step 1010, a two-dimensional cache block T is first constructed, that is, a cache tensor T is constructed. The row height H of the cache tensor T is... T Let Ho be the row height of the output tensor Y, and W be the column width of the buffer tensor T. T Let max(W, Wo) be the larger of the input tensor width dimension W and the output tensor width dimension Wo. Typically, W and Wo are similar, but may differ slightly due to padding. Here, the column width W... T Taking the maximum value is to ensure that the data in all rows is continuously aligned in memory, so that SIMD operations can be performed across rows.

[0154] Next, in step 1020, the corresponding input tensor X is copied as a continuous vector containing multiple data rows.

[0155] In some embodiments, the control circuit is further configured to copy the corresponding input tensor X as a continuous vector containing multiple data rows as follows: if the width dimension W of the input tensor X is less than W... T If the width is less than W, then copy the input tensor by the step size. T To fill in the blanks with zeros, add an extra column of zeros on the right to make the column widths consistent.

[0156] It is understandable that if the width dimension W of the input tensor X is equal to W... TThen, multiple rows of data can be copied continuously, such as rows Ho, and flattened into a vector. This multi-row flattening method can also vectorize the H (Ho) dimension.

[0157] Next, in step 1030, the weight point traversal process is entered to check if there are any unprocessed weight points. If so, step 1040 is entered, and for the current weight point (kh, kw), the SIMD instruction is called to perform vector operations between the weight point vector and multiple valid data rows selected from the previously copied continuous vectors to obtain the first operation result.

[0158] The process of filtering valid data rows is similar to the process of determining the range of the corresponding input tensor, that is, determining the valid indices of the W and H dimensions. Then, the corresponding input data is read, and vector multiplication is performed with the weight point vector to obtain the first operation result. This allows multiple rows and columns to be processed at once.

[0159] Next, in step 1050, a multiplication operation is performed between the integer mask and the first operation result to obtain the second operation result. The integer mask can eliminate the influence of invalid data introduced by padding or extra columns. The integer mask can set 0 at the position corresponding to invalid data and set 1 at the rest.

[0160] Finally, in step 1060, based on the translation relationship between the input tensor X and the output tensor Y in the H and W dimensions, the second operation result is accumulated to the corresponding position of the cache tensor T. Since the convolution stride in both the H and W dimensions is 1, the coordinate indices of the cache tensor T and the output tensor Y are consistent, and the result is directly accumulated to the cache tensor T according to the offset caused by the translation relationship between the input tensor X and the output tensor Y.

[0161] After iterating through all the weight points of the weight tensor K in this way, that is, when the result of the judgment in step 1030 is negative, jump to step 1070. When writing the cache tensor T back to the output tensor Y, the invalid columns in the cache tensor T are pruned by copying by step size.

[0162] To illustrate the convolution process more clearly, graphical examples are provided below. In these examples, it is assumed that the height H and width W of the input tensor X are both 3, and the height KH and width KW of the weight tensor K are both 2. , , All are 1.

[0163] The calculation process of forward depthwise convolution is as follows: Construct a cache tensor T for the cache area, with row height H. T The column width is 4, and the column width is W. TThe value is 4. The input tensor X is copied with a stride equal to the column width (4) to obtain [x(0,0),x(0,1),x(0,2),0,x(1,0),x(1,1),x(1,2),0,x(2,0),x(2,1),x(2,2),0]. Then, the SIMD operation is called to perform the operation by traversing the weight points one by one.

[0164] Figure 11A This illustrates the case of traversing the first point in the KW and KH dimensions of the convolution kernel. As shown in the figure, selecting points in the KH and KW dimensions of the convolution kernel... The first point (the shaded box) shows the linear mapping between the output tensor Y and the input tensor X (in the third case, this linear mapping is also a translation relationship), where the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0165] The input tensor X is multiplied by the weight points. The result of the multiplication is then multiplied by an integer mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to the cached tensor T with a 1-row offset and a 1-column offset. As shown in the figure, the extra columns are added precisely to "x_pad, y(2,0)" and "x_pad, y(3,0)", without interfering with the valid data.

[0166] Figure 11B This illustrates the case of traversing the second point along the dimensions KH and KW of the convolution kernel. As shown in the figure, selecting points along the dimensions KH and KW of the convolution kernel... The second point (the slashed box) equals 1. The figure shows the mapping relationship between the output tensor Y and the input tensor X. Similarly, the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0167] The input tensor X is multiplied by the weight points. The result of the multiplication is then multiplied by an integer mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to the cached tensor T with a row offset of 1 and a column offset of 0. As shown in the figure, the extra columns are added precisely to "x_pad, y(1,3)" and "x_pad, y(2,3)", without interfering with the valid data.

[0168] Figure 11C This illustrates the case of traversing the third point in the KH and KW dimensions of the convolution kernel. As shown in the figure, selecting points in the KH and KW dimensions of the convolution kernel... The third point (the shaded box) represents the value of 0. The figure illustrates the mapping between the output tensor Y and the input tensor X; similarly, the output tensor Y is marked in dark, and X in light. Each box shows the HW space coordinates corresponding to the input tensor X and the output tensor Y.

[0169] The input tensor X is multiplied by the weight points. The result of the multiplication is then multiplied by an integer mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to the cached tensor T with a row offset of 0 and a column offset of 1. As shown in the figure, the extra columns are added precisely to "x_pad, y(1,0)" and "x_pad, y(2,0)", without interfering with the valid data.

[0170] Figure 11D This illustrates the case of traversing the fourth point in the KH and KW dimensions of the convolution kernel. As shown in the figure, selecting points in the KH and KW dimensions of the convolution kernel... =1, the fourth point (diagonal box). The figure shows the mapping relationship between the output tensor Y and the input tensor X. Similarly, the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0171] The input tensor X is multiplied by the weight points. The result of the multiplication is then multiplied by an integer mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to the cached tensor T with a row offset of 0 and a column offset of 0. As shown in the figure, the extra columns are added precisely to "x_pad, y(1,3)" and "x_pad, y(2,3)", without interfering with the valid data.

[0172] The calculation process of the reverse depthwise convolution operation is as follows: Construct a cache tensor T for the cache area, with row height H. T The column width is 3, and the column width is W. T The value is 4. Copy dY to obtain [y(0,0),y(0,1),y(0,2),y(0,3),y(1,0),y(1,1),y(1,2),y(1,3), y(2,0),y(2,1),y(2,2),y(2,3)]. Then iterate through the weight points one by one and call SIMD to perform the operation. A graphical representation can still be found by referring to... Figures 11A-11D .

[0173] Traversal of convolution kernel selection The weight points are defined, and the effective behavior of dY is [1,3]. dY is multiplied by the weight points and accumulated along with dimension M. The result is then multiplied by an int mask of [0,1,1,1] to eliminate the influence of extra columns. The result is then added to T with a row offset of 0 and a column offset of -1. The invalid columns "x_pad, y(2,0)" and "x_pad, y(3,0)" are added to the fourth column (extra columns), which can be directly clipped during copying.

[0174] Next, select The weight points are defined, and the effective behavior of dY is [1,3]. dY is multiplied by the weight points and accumulated along with dimension M. The result is then multiplied by an integer mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to T with a row offset of 0 and a column offset of 0. The invalid columns "x_pad, y(1,3)" and "x_pad, y(2,3)" are added to the fourth column (extra columns), which can be directly clipped during copying.

[0175] Next, select The weight points are defined, and the effective behavior of dY is [0,2]. dY is multiplied by the weight points and accumulated along with dimension M. The result is then multiplied by an integer mask of [0,1,1,1] to eliminate the influence of extra columns. The result is then added to T with a row offset of 0 and a column offset of -1. The invalid columns "x_pad, y(1,0)" and "x_pad, y(2,0)" are added to the fourth column (extra columns), which can be directly clipped during copying.

[0176] Final choice The weight points of dY are defined, and the effective behavior of dY is [0,2]. dY is multiplied by the kernel points and accumulated along with dimension M. The result is then multiplied by an int mask of [1,1,1,0] to eliminate the influence of extra columns. The result is then added to T with a row offset of 0 and a column offset of 0. The invalid columns "x_pad, y(0,3)" and "x_pad, y(1,3)" are added to the fourth column (extra columns), which can be directly clipped during copying.

[0177] Finally, use `memcpy` with a stride to copy T back to the output tensor Y. The stride allows you to truncate the fourth column during the copy process.

[0178] The fourth scenario: 2D scenes. or Greater than 1.

[0179] In the fourth case, it is necessary to combine the solutions for step sizes greater than 1 and the solutions for 2D.

[0180] In some embodiments, the control circuit is further configured to: introduce bias parameters in the W dimension respectively. and Bias parameters in the H dimension and This ensures a linear mapping between the cache tensor T and the input tensor X in the W and H dimensions, and a translation relationship between the cache tensor T and the output tensor Y in the W and H dimensions. In other words, the step sizes in both the H and W dimensions need to be incorporated into the index transformation. It can be understood that this scheme is also compatible with scenarios where any one or two step sizes are 1; the difference lies only in the specific values ​​of the introduced parameters. When the step size of a certain dimension (H or W) is 1, the bias parameter corresponding to that dimension... , .

[0181] More specifically, the coordinate index of the cache tensor T in the W dimension. Coordinate index of input tensor X in dimension W They have the following linear mapping relationship: ;where the bias parameter It makes The smallest natural number that holds true, where It is the KW dimension index of the current weight point. It is the dilation parameter in the KW dimension. It is the left padding parameter. It is the convolution stride in the W dimension.

[0182] The coordinate index of the cache tensor T in the H dimension Coordinate index of input tensor X in dimension H They have the following linear mapping relationship: ;where the bias parameter It makes The smallest natural number that holds true, where It is the KH dimension index of the current weight point. It is the dilation parameter in the KH dimension. It is the upper fill parameter. It is the convolution stride in the H dimension.

[0183] Furthermore, the coordinate index of the cache tensor T in the W dimension. Coordinate index of the output tensor Y in the W dimension The following translation relationship exists between them: ;where the bias parameter .

[0184] The coordinate index of the cache tensor T in the H dimension Coordinate index of the output tensor Y in the H dimension The following translation relationship exists between them: ;where the bias parameter .

[0185] In some embodiments, the control circuitry is further configured to: introduce a grouping base. and All value points are grouped by base number and Grouping is performed where weight points within the same group have the same bias parameter. and The corresponding input tensors have overlapping regions, and the same cached tensor T is used for result accumulation; different groups have different bias parameters. and There is no overlap between the input tensors corresponding to different groups.

[0186] Specifically, group cardinality It makes The smallest integer that is true; the cardinality of grouping It makes The smallest integer that is true.

[0187] The application of the above bias parameters and grouping cardinality is similar to that described in the previous case combined with the second case, except that it is extended to the H dimension, so it will not be described in detail here.

[0188] Regarding the setting of the cache tensor T, when weight grouping is introduced due to any convolution stride greater than 1, if only the cache tensor T of one group is considered, then the row height should be the number of effective rows affected within the group, and the column width should be the larger of the width of the input tensor copied by stride and the number of effective columns affected within the group. However, the cache tensors of different groups may have different sizes, especially different column widths. When the column widths of the cache are different, the input data needs to be reloaded, increasing the memory access load. Therefore, in some embodiments, the cache size of each group can be uniformly set to the largest row height and the largest column width among all groups, thereby avoiding repeated data loading.

[0189] For the fourth case, the boundary handling, masking, and coordinate mapping can be found in the previous descriptions.

[0190] To illustrate the convolution process more clearly, graphical examples are provided below. In these examples, it is assumed that the height H and width W of the input tensor X are both 5, the height KH and width KW of the weight tensor K are both 3, and the height and width of the output tensor Y are both 3. , , Both are 1. , Both are 2. At this time, , Both are 2, so they need to be divided into 4 groups.

[0191] The first group has 2 valid rows, 2 columns affected in the positive direction, and 3 columns affected in the negative direction.

[0192] The second group has 2 valid rows, 3 columns affected in the positive direction, and 3 columns affected in the negative direction.

[0193] The third group has 3 valid rows, 2 columns affected in the positive direction, and 3 columns affected in the negative direction.

[0194] The fourth group has 3 valid rows, 3 columns affected in the positive direction, and 3 columns affected in the negative direction.

[0195] Taking all factors into consideration, the largest row height and the largest column width are selected, and the HW dimension of the cache tensor T is 3×3.

[0196] Figure 12 This illustrates the case of traversing the first point in the KW and KH dimensions of the convolution kernel. As shown in the figure, selecting points in the KH and KW dimensions of the convolution kernel... The first point where = 0 (the shaded box). The figure shows the linear mapping relationship between the output tensor Y and the input tensor X, where the output tensor Y is marked in dark and X is marked in light. Each box shows the HW space coordinates corresponding to the input tensor X and the HW space coordinates corresponding to the output tensor Y.

[0197] The calculation process of forward depthwise convolution is as follows: The input tensor X is copied with a stride equal to the column width (3) to obtain [x(1,1),x(1,3),0,x(3,1),x(3,3),0]. Then, the SIMD instruction is called to perform vector multiplication between the input tensor X and the weight points. The result of the multiplication is multiplied using an int mask [1,1,0] to eliminate the influence of extra columns. The result is then added to the cache tensor T in the buffer area with a row offset of 1 and a column offset of 1.

[0198] The calculation process of the reverse depthwise convolution operation is as follows: Copy dY to obtain [y(0,0),y(0,1),y(0,2),y(1,0),y(1,1),y(1,2),y(2,0),y(2,1),y(2,2)]. Select the valid row [1,2], perform vector multiplication between dY and the weight points, and accumulate the M dimensions. Multiply the result using an int mask [0,1,1] to eliminate the influence of extra columns. Then add the result to the cache tensor T of the cache area with a row offset of 0 and a column offset of -1.

[0199] The remaining weight points can be processed in the same way, and will not be described in detail here.

[0200] The following is pseudocode for implementing Depthwise convolution computation.

[0201] If the total number of dimensions M, C, and W is sufficient, corresponding to the 1D scenario mentioned earlier, the pseudocode for depthwise convolution calculation is as follows: / / Calculate base_h, base_w for kh_base in [0, base_h): for kw_base in [0, base_w): T = 0 for kh = kh_base; kh <KH; kh += base_h: for kw = kw_base; kw <KW; kw += base_w: for n in [0, N): / / Iterate through the H dimensions of the input, and denote the valid h array as valid_h. for h in valid_h: / / This loop contains SIMD instructions, with a vectorization length of W. C M / / Forward convolution calculation if (conv) { / / 1. Calculate the valid start and end points of dimension W, and denote this vector segment as X_L X_L / / 2. If multiply > 1, broadcast X_L to obtain X_L1 X_L1 = broadcast(X_L) / / 3. Extract the value at the corresponding position of the convolution kernel and perform multiplication. Buffer = W_L X_L1 / / 4. Calculate the column offset and add the Buffer to T T[w_offset] += Buffer } else { / / Inverse diffX convolution calculation / / 1. Calculate the valid start and end points of dimension W, and denote this vector segment as dY_L dY_L / / 2. Extract the value at the corresponding position of the convolution kernel and perform multiplication. Buffer = W_L dY_L / / 3. If multiply > 1, sum each multiply instance into one. Buffer = reduce_sum(Buffer, multiply) / / 4. Calculate the column offset and add the Buffer to T T[w_offset] += Buffer } / / Copy T back to video memory write_back(T) If the total number of dimensions M, C, and W is relatively small, corresponding to the 2D scenario mentioned earlier, the pseudocode for depthwise convolution calculation is as follows: / / Calculate base_h, base_w for kh_base in [0, base_h): for kw_base in [0, base_w): T = 0 for kh = kh_base; kh <KH; kh += base_h: for kw = kw_base; kw <KW; kw += base_w: / / This loop contains SIMD instructions, and the vectorization length can be N. H W C M / / Note: If it can be placed all at once, it can be vectorized in 5 dimensions. / / If there's not enough space, use a for loop to slice. / / Forward convolution calculation if (conv) { / / 1. Calculate the valid start and end rows for dimension W, with column widths consistent with T. X_L / / 2. If multiply > 1, broadcast X_L to obtain X_L1 X_L1 = broadcast(X_L) / / 3. Extract the value at the corresponding position of the convolution kernel and perform multiplication. Buffer = W_L X_L1 / / 4. Construct an int mask and perform multiplication to eliminate the effect of invalid columns. Buffer = Buffer mask / / 5. Calculate the row and column offsets, and add the Buffer to T. T[h_offset][w_offset]+= Buffer } else { / / Inverse diffX convolution calculation / / 1. Calculate the valid start and end rows for dimension W, with column widths consistent with T. dY_L / / 2. Extract the value at the corresponding position of the convolution kernel and perform multiplication. Buffer = W_L dY_L / / 3. If multiply > 1, sum each multiply instance into one. Buffer = reduce_sum(Buffer, multiply) / / 4. Construct an int mask and perform multiplication to eliminate the effect of invalid columns. Buffer = Buffer mask / / 5. Calculate the row and column offsets, and add the Buffer to T. T[h_offset][w_offset]+= Buffer } / / Copy T back to video memory with stride. write_back(T) The foregoing describes various embodiments of the Depthwise convolution operation optimization scheme for SIMD instruction architectures provided in this disclosure. As can be seen from the above description, the embodiments disclosed herein have the following advantages: First, the degree of vectorization is significantly improved. In the original implementation, the vectorization length of SIMD instructions was mainly limited by three dimensions: KW, C, and M, with a theoretical vectorization length of KW×C×M. Since the convolution kernel size and channel repetition factor are usually small, this vectorization length is often insufficient to fully utilize the hardware SIMD computing resources in practical scenarios.

[0202] In the scheme of this disclosed embodiment, the computation unfolding method is dominated by the spatial dimension of the convolution kernel. With fixed weight points, the output tensor is unfolded in multiple dimensions of N, H, W, C, and M, so that the theoretical vectorization length can be extended to N×H×W×C×M. Typically, the spatial size of the feature map is much larger than the size of the convolution kernel; therefore, the scheme of this disclosed embodiment can significantly increase the usable vector length and greatly improve the utilization rate of SIMD instructions. To cooperate with the vectorization scheme, the specific implementation only needs to handle the beginning and end boundaries and offset relationships of the input and output tensor coordinate indices, but this part of the logical overhead is relatively small and can be ignored. In the actual computation process, vector multiplication and addition instructions are mainly executed. These instructions have extremely high execution efficiency in hardware, while avoiding expensive vector reduction operations such as reduce_sum(buffer, kw).

[0203] Second, the reverse diffX computation can omit the weight rotation operation. In the traditional implementation of reverse diffX, it is usually necessary to perform spatial rotation on the weight tensor K to reuse the computational form of the forward convolution. This preprocessing not only introduces additional computational steps and requires additional storage space, but also hinders SIMD acceleration.

[0204] The solution disclosed in this embodiment avoids dependence on weight rotation in its computational expansion method, allowing the calculation of inverse diffX to directly use the original weight tensor, thereby omitting the additional step of weight rotation. This improvement reduces implementation complexity while minimizing memory access and temporary storage overhead.

[0205] Third, it reduces on-chip space requirements and increases memory utilization. Traditional sliding window implementations center computation on the output position, requiring complete coverage of the corresponding input window in a single computation. Limited by this computational organization, data within the input window often needs to be loaded multiple times, and the convolution kernel typically needs to participate in the computation as a whole. To ensure window continuity, under conditions of segmented loading or limited on-chip caching, data in these overlapping areas inevitably needs to be repeatedly loaded, resulting in significant redundant memory access overhead. This not only increases the instantaneous occupancy of on-chip storage space but also makes it difficult to effectively reuse highly overlapping input data between adjacent output positions, thus placing higher demands on memory bandwidth.

[0206] The solution disclosed in this embodiment no longer uses a sliding window as the computational unit, but instead uses fixed weight points as the computational core. During the computation process, only one weight point is loaded at a time, and the required input region is determined in reverse based on the current output region. The corresponding input data only needs to be loaded once, and can then be reused repeatedly in the computation of multiple output positions, avoiding the overhead of repeated loading caused by window sliding. At the same time, since the computation is no longer constrained by the requirement that "a single window must be completed at once", the splitting of computational segments is more flexible. The dimensions such as N, H, and W can be adaptively split according to the on-chip storage capacity, thereby efficiently organizing computation within the limited on-chip space.

[0207] In this way, the solution of this disclosed embodiment significantly reduces the instantaneous demand for on-chip storage space, reduces unnecessary data movement, improves the reuse efficiency of input data and weight data, and improves the overall memory space utilization and operator execution efficiency.

[0208] The inventors conducted a comparative analysis of the performance of the new and old solutions.

[0209] The following analysis was conducted for the 1D scene. Specific scene parameters are as follows: In the inverse diffX calculation, N (batch size) is 16, group_count is 144, and M (channel repetition factor) is 1. The width W of the input feature map dY is 397, and the number of channels C is 144. That is, when group_count = X_c, it is the depthwise case, and the width KW of the weights K is 32. Padding parameters... The values ​​are 15 and 16 respectively; stride and dilation are both 1. The width of the output feature map dX is 397, and the number of channels is 144. The data type is half.

[0210] When implementing the old algorithm, the calculation process includes: rotating the weight K by 180 degrees; traversing N dimensions; loading dY and K in segments according to the memory space limit; traversing the W dimensions of dY and simulating convolution operations using a sliding window; and performing vector multiplication between dY and K. The vector segment length is 32. 144. Further reduce the KW dimension of the result from the previous step. The vector segment length is 32. Finally, copy it back to dX.

[0211] In the above calculation process, the number of iterations is 16. 397 iterations. The effective computational efficiency is 397 / (397+31). The effective I / O efficiency is affected by the overlapping loading caused by the sliding window. Since the convolution window width is 32, 31 elements need to be repeatedly loaded between adjacent segments to maintain window continuity. When loading 128 elements at a time, there will be 3 such overlapping loads in the process of covering the total width of 397, so the effective loading efficiency is 397 / (397 + 31 × 3).

[0212] The hardware implementation time of the old algorithm was tested on a certain type of chip, as shown in Table 1 below:

[0213] Table 1 Under the same hardware environment, when implementing the algorithm of this disclosure embodiment, the calculation process includes: traversing the KW dimensions; loading dY in segments; calculating the index information of the effective segments; and performing vector multiplication between dY and the weight points. The maximum length of the vector segment can be 397. 144. The result of the previous step is added to the buffer by index offset. The vector segment length can be up to 397. 144. Finally, copy the data from the buffer back to dX.

[0214] In the above calculation process, the loop count is 32. Effective computational efficiency is maximized. Effective I / O efficiency is maximized.

[0215] The hardware implementation time of the algorithm in this embodiment, tested on the same model of chip, is shown in Table 2 below:

[0216] Table 2 The above tests show that, on the same chip model, the old algorithm's hardware implementation time is 343µs, while the algorithm in this disclosed embodiment has a hardware implementation time of 31µs, representing a reduction of approximately 90%. It can be seen that the algorithm in this disclosed embodiment has more efficient memory utilization, significantly improving I / O performance, and its increased vectorization also significantly improves computational performance.

[0217] In 2D scenarios, the solution disclosed in this embodiment can perform vectorization processing on more dimensions, resulting in a greater performance improvement over the old solution compared to 1D scenarios.

[0218] The technical solutions of the embodiments disclosed herein have been explained in detail and thoroughly. Convolutional computation is one of the most fundamental and critical computational units in the deep learning system, and its implementation efficiency largely determines the overall performance of model inference and training. Among various convolutional operators, depthwise convolution, due to its complete decoupling between channels, low arithmetic density, and computation process highly dependent on memory access behavior, has become one of the most challenging typical scenarios under modern general-purpose processors and dedicated accelerator architectures. Traditional implementation methods are intuitive in mathematical expression and clear in computational logic, but their computational organization naturally revolves around a sliding window, resulting in inherent deficiencies in SIMD vectorization efficiency and data reuse capabilities, making it difficult to fully unleash the parallel computing potential of hardware.

[0219] The starting point for this disclosure is a re-examination of the structural characteristics of convolutional computation. A key observation is that when the spatial location (kh, kw) of the convolutional kernel is fixed, the relationship between the input tensor X and the output tensor Y no longer exhibits a complex mapping relationship that changes with window movement, but degenerates into a deterministic and regular linear mapping relationship. Especially in the context of depthwise convolution, this mapping is completely independent and continuous in the channel dimension (C) and its repetition dimension (M), while it only manifests as regular index offsets in the spatial and batch dimensions. This structural feature means that once the weight points are determined, the essence of convolutional computation can be transformed from "multiplication and addition reduction within a local window" to "batch linear accumulation across multiple locations," thus naturally adapting to the parallel execution model of SIMD instructions in terms of computational form.

[0220] Based on this understanding, this disclosure proposes a convolution computation scheme that uses weight points as the core of computation and is supplemented by output cache reconstruction. This scheme splits the computation core along the spatial dimension of the weights, constructs a cache tensor T with the same layout as the output tensor within each fixed weight point, and combines this with a precise masking mechanism to handle boundaries and invalid regions. This allows for unified vectorized computation across multiple dimensions such as N, H, W, C, and M in two-dimensional and even higher-dimensional scenarios. Under this computational organization, although there is some overlap in access between input tensors and different computation cores, the output write region is strictly divided, and there are no write conflicts between computational units, thus avoiding costly synchronization operations such as atomic addition.

[0221] Compared to the traditional sliding window implementation centered on the output position, the solution disclosed in this embodiment achieves a fundamental shift in computational organization: input data can be loaded once and reused in multiple output positions; the vectorization length is no longer limited by the spatial size of the convolution kernel; and no additional spatial rotation processing of the weight tensor is required during the inverse diffX calculation. This not only significantly improves the utilization rate of the SIMD computing unit but also effectively reduces memory access pressure and improves the efficiency of on-chip memory resource utilization.

[0222] In summary, this disclosed embodiment, starting from the structural characteristics of convolution computation, systematically reveals the key property of the implicit linear mapping between input and output under fixed weight points, and constructs an efficient and highly scalable computational paradigm based on this. This scheme effectively alleviates the long-standing performance bottleneck in depthwise convolution, and also provides a new design perspective and reference path for the high-performance implementation of other low-arithmetic-density operators with sliding windows as their core in modern SIMD architectures.

[0223] This disclosure also provides a chip that may include the computing device of any of the embodiments described above. Furthermore, this disclosure also provides a board that may include the aforementioned chip.

[0224] The above optimization method can be implemented through an instruction program. Therefore, this disclosure also provides a computing device, including: a processor configured to execute program instructions; and a memory configured to store the program instructions, which, when loaded and executed by the processor, cause the computing device to perform any embodiment of the aforementioned method for performing Depthwise convolution operations using SIMD instructions.

[0225] This disclosure also provides a computer-readable storage medium storing program instructions that, when loaded and executed by a processor, cause the processor to perform any of the aforementioned methods for performing Depthwise convolution operations using SIMD instructions.

[0226] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various circuits in the device embodiments described above, this document has decomposed them based on their logical functions, but in actual implementation, there may be other decomposition methods. The physical implementation of the hardware structure of the circuit may include, but is not limited to, physical devices, which may include, but are not limited to, devices such as transistors or memristors. In view of this, the various devices described herein (e.g., computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any suitable storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0227] In this disclosed embodiment, the term "instruction" can include software instructions, hardware instructions, firmware instructions, or any combination thereof. Software instructions typically refer to statements or commands in a programming language; they are high-level abstractions, such as function calls, machine code, and bytecode. Hardware instructions, also known as machine instructions or instruction set architecture (ISA) instructions, are low-level commands that computer hardware can directly recognize and execute, such as processor instructions and CPU instructions. Firmware instructions include, for example, opcodes and microcode. Furthermore, different CPU architectures and different instruction sets can have different hardware instruction sets. Based on the complexity of the computer instruction set, hardware design, execution speed, compiler complexity, and instruction format, they can be further classified as: CISC (Complex Instruction Set Computer), RISC (Reduced Instruction Set Computer), and VLIW (Very Long Instruction Word).

[0228] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0229] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A computing device configured to perform a depthwise convolution operation, comprising a storage circuit, a control circuit, and an arithmetic circuit, wherein: The storage circuit is configured to store the input tensor X, the weight tensor K, and the output tensor Y. The control circuit is configured to control the Depthwise convolution operation as follows: Determine the vectorization length of the output tensor Y, wherein the vectorization length supports vectorization in any of the N, H, W, C, and M dimensions in the order of memory loading from the inside out, where N is the batch dimension, H is the height dimension, W is the width dimension, C is the channel dimension, and M is the channel repetition factor dimension; and Traverse each weight point (kh, kw) on the spatial dimensions KH and KW of the weight tensor K, where KH is the weight height dimension, KW is the weight width dimension, and kh and kw are the indices of the weight points. Call the SIMD instruction to perform vector operations between the weight point vector and the corresponding input tensor and accumulate the results of the vector operations to obtain the output tensor. The computing circuit is configured to execute the SIMD instructions under the control of the control circuit.

2. The computing device according to claim 1, wherein, The control circuit is further configured to control the Depthwise convolution operation as follows: When traversing each weight point: determine the range of the corresponding input tensor; call the SIMD instruction to perform vector operations between the weight point vector and the corresponding input tensor to obtain the vector operation result; And the positional summation of the vector operation results obtained from the control weight point operation; as well as After traversing all the value points, the output tensor Y is obtained based on the accumulated result.

3. The computing device according to claim 2, wherein, The control circuit is further configured to determine the vectorized length of the output tensor Y as follows: Based on the size of the output tensor Y in the N, H, W, C, and M dimensions and the hardware configuration information of the computing device, the vectorization length is determined by taking several dimensions in the order of memory loading from the inside to the outside.

4. The computing device according to any one of claims 2-3, wherein, The control circuit is further configured to determine the range of the corresponding input tensor as follows: The dimensions of the corresponding input tensor are determined based on the dimensions contained in the vectorized length of the output tensor Y; and Based on the linear mapping relationship between the coordinate indices of the output tensor Y and the input tensor X in the H and / or W dimensions at the current weight point, the range of the corresponding input tensor is determined.

5. The computing device according to claim 4, wherein, The control circuit is further configured to determine the range of the corresponding input tensor as follows: Based on the padding parameters in the convolution parameters of the Depthwise convolution operation, determine the valid start and end positions of the input tensor corresponding to the current weight point in the W and / or H dimensions.

6. The computing device according to any one of claims 2-5, wherein, The control circuit is further configured to invoke SIMD instructions as follows: In response to the fact that the Depthwise convolution operation is a forward Depthwise convolution operation, the first vector operation instruction is invoked to broadcast or copy the lowest dimension C of the input tensor into M copies and then perform a multiplication operation with the weight point vector.

7. The computing device according to any one of claims 2-6, wherein, The control circuit is further configured to invoke SIMD instructions as follows: In response to the depthwise convolution operation being a reverse depthwise convolution operation, a second vector operation instruction is invoked to perform vector multiplication of the weight point vector with the corresponding input tensor and to accumulate the channel repetition factor dimension M.

8. The computing device according to any one of claims 2-7, wherein, The storage circuit is further configured with a buffer area for storing a buffer tensor T. The buffer tensor T has a linear mapping relationship with the coordinate indices of the input tensor X in the H and / or W dimensions. The buffer tensor T has a translation relationship with the coordinate indices of the output tensor Y in the H and / or W dimensions. The control circuit is further configured to perform positional accumulation of the results of operations according to the following control vector: Based on the index of the currently used weight point vector in the KW and KH dimensions, determine the translation value corresponding to the current vector operation result; The current vector operation result is accumulated at the corresponding position of the cache tensor T based on the translation value.

9. The computing device according to claim 8, wherein, The control circuit is further configured to obtain the output tensor Y by controlling it as follows: After traversing all value points, the data in the cache tensor T is written back to the output tensor Y according to the translation relationship.

10. The computing device according to any one of claims 8-9, wherein, When traversing the weight tensor K, the control circuit is further configured to control the calculation process as follows: In response to M×C being greater than or equal to the first threshold, the vectorization length is split and calculated in a single-point manner in each dimension according to the hierarchical order of N, H, and W dimensions, and the vectorization length is ≤ M×C.

11. The computing device according to any one of claims 8-9, wherein, When traversing the weight tensor K, the control circuit is further configured to control the calculation process as follows: In response to M×C being less than the first threshold and M×C×W being greater than or equal to the first threshold, the vectorization length is calculated in a single-point manner in each dimension according to the hierarchical order of N and H dimensions, and the vectorization length is ≤ M×C×W.

12. The computing device according to claim 11, wherein, When the stride of the W dimension of the Depthwise convolution is... When the value is 1, the control circuit is further configured to: Set the translation amount in the translation relationship between the cache tensor T and the output tensor Y in the W dimension to 0; and Let the linear mapping relationship between the cache tensor T and the coordinate index of the input tensor X in the W dimension correspond to the linear mapping transformation between the output tensor Y and the coordinate index of the input tensor X in the W dimension.

13. The computing device according to claim 11, wherein, When the stride of the W dimension of the Depthwise convolution is... When the value is greater than 1, the control circuit is further configured to: Introducing bias parameters and This ensures a linear mapping between the cache tensor T and the coordinate indices of the input tensor X in the W dimension, and a translational relationship between the cache tensor T and the coordinate indices of the output tensor Y in the W dimension.

14. The computing device according to claim 13, wherein, The coordinate index of the cache tensor T in the W dimension Coordinate index of the input tensor X in the W dimension They have the following linear mapping relationship: ; The bias parameter It makes The smallest natural number that holds true, where It is the KW dimension index of the current weight point. It is the dilation parameter in the KW dimension. It is the left padding parameter. It is the convolution stride in dimension W; The coordinate index of the cache tensor T in the W dimension The coordinate index of the output tensor Y in the W dimension The following translation relationship exists between them: ; The bias parameter .

15. The computing device according to claim 14, wherein, The control circuit is further configured to: Introducing group cardinality All value points are grouped by base number Grouping is performed where weight points within the same group have the same bias parameter. The corresponding input tensors have overlapping regions, so the same cached tensor T is used to accumulate the results. Different groups have different bias parameters. There is no overlap between the input tensors corresponding to different groups.

16. The computing device according to claim 15, wherein, The grouping cardinality It makes The smallest integer that is true.

17. The computing device according to any one of claims 8-9, wherein, When traversing the weight tensor K, the control circuit is further configured to control the calculation process as follows: In response to M×C×W being less than the first threshold, the calculation is performed by splitting the vectorization in N dimensions in a single-point manner, wherein the vectorization length is ≤ M×C×W×H; or In response to M×C×W×H being less than the first threshold, no splitting calculation is required, and the vectorization length is ≤M×C×W×H×N.

18. The computing device according to claim 17, wherein, When the stride of the W dimension and H dimension of the Depthwise convolution is... and When both are 1, the control circuit is further configured to: Construct the row height H of the cache tensor T T Ho is the row height of the output tensor Y, and W is the column width of the buffer tensor T. T The larger of the input tensor width dimension W and the output tensor width dimension Wo; The corresponding input tensor X is copied as a continuous vector containing multiple data rows; When traversing the weight tensor K Invoking SIMD instructions to perform vector operations between the weight point vector and multiple valid data rows selected from the continuous vectors, yielding a first operation result; The second result is obtained by multiplying the integer mask with the first operation result. as well as Based on the translation relationship between the input tensor X and the output tensor Y in the H and W dimensions, the second operation result is accumulated at the corresponding position of the cache tensor T.

19. The computing device according to claim 18, wherein, The control circuit is further configured to copy the corresponding input tensor X as a continuous vector containing multiple data rows as follows: If the width dimension W of the input tensor X is less than W T Then copy the input tensor by step size; if the width is less than W... T The position is filled with zeros.

20. The computing device according to any one of claims 18-19, wherein, The integer mask has a value of 0 at the position corresponding to invalid data, and a value of 1 at the other positions.

21. The computing device according to any one of claims 18-20, wherein, The control circuit is further configured to: When writing the cached tensor T back to the output tensor Y, invalid columns in the cached tensor T are pruned by copying with a step size.

22. The computing device according to claim 17, wherein, When the stride of the W dimension and H dimension of the Depthwise convolution is... and When any value is greater than 1, the control circuit is further configured to: Introduce bias parameters in the W dimension respectively and and the bias parameter of the H dimension and This ensures a linear mapping between the cache tensor T and the coordinate indices of the input tensor X in the W and H dimensions, and a translational relationship between the cache tensor T and the coordinate indices of the output tensor Y in the W and H dimensions.

23. The computing device according to claim 22, wherein, The coordinate index of the cache tensor T in the W dimension Coordinate index of the input tensor X in the W dimension They have the following linear mapping relationship: ; The bias parameter It makes The smallest natural number that holds true, where It is the KW dimension index of the current weight point. It is the dilation parameter in the KW dimension. It is the left padding parameter. It is the convolution stride in dimension W; The coordinate index of the cache tensor T in the H dimension The coordinate index of the input tensor X in the H dimension They have the following linear mapping relationship: ; The bias parameter It makes The smallest natural number that holds true, where It is the KH dimension index of the current weight point. It is the dilation parameter in the KH dimension. It is the upper fill parameter. It is the convolution stride in dimension H; The coordinate index of the cache tensor T in the W dimension The coordinate index of the output tensor Y in the W dimension The following translation relationship exists between them: ; The bias parameter ; The coordinate index of the cache tensor T in the H dimension The coordinate index of the output tensor Y in the H dimension The following translation relationship exists between them: ; The bias parameter .

24. The computing device according to claim 23, wherein, The control circuit is further configured to: Introducing group cardinality and All value points are grouped by base number and Grouping is performed where weight points within the same group have the same bias parameter. and The corresponding input tensors have overlapping regions, so the same cached tensor T is used to accumulate the results. Different groups have different bias parameters. and There is no overlap between the input tensors corresponding to different groups.

25. The computing device according to claim 24, wherein, Grouping cardinality It makes The smallest integer that holds true; the cardinality of grouping It makes The smallest integer that is true.

26. The computing device according to any one of claims 24-25, wherein, The size of the cache tensor T for each group is set to the largest row height and the largest column width among all groups.

27. A chip comprising a computing device according to any one of claims 1-26.

28. A circuit board comprising the chip according to claim 27.

29. A method for performing a depthwise convolution operation using a computing device according to any one of claims 1-26.

Images