3D Convolution Operation Device and 3D Convolution Operation Method

Abstract

Embodiments of the present application disclose a three-dimensional convolution operation device and a three-dimensional convolution operation method. The three-dimensional convolution operation method includes: performing a dimension exchange on an input data to continuously arrange a plurality of elements of the input data in the depth dimension and the channel dimension together to generate a first data; performing a convolution operation on the first data and a second data corresponding to a first weight data in a block manner to generate an operation data; and reordering the operation data according to the original dimension form of the input data to generate an output data.

Description

Technical Field

[0001] This application relates to a convolution operation device, and more particularly to a three-dimensional convolution operation device and a three-dimensional convolution operation method that perform three-dimensional convolution operations by reordering the data dimension form. Background Art

[0002] Convolution operations are commonly used in artificial neural network models to determine whether there are similar features among multiple data. In the prior art, when calculating three-dimensional convolution operations, multiple data values in the depth dimension and the channel dimension are accumulated. In the existing data format, multiple data values in the depth dimension and multiple data values in the channel dimension are stored dispersedly in memory. Thus, this will result in a higher complexity of three-dimensional convolution operations. In addition, when performing three-dimensional convolution operations, the convolution operation device needs to spend more time reading out the dispersed multiple data values, resulting in a lower data access efficiency of the convolution operation device, and thus the processing efficiency of three-dimensional convolution operations is not good. Summary of the Invention

[0003] In some embodiments, this application provides a three-dimensional convolution operation device and a three-dimensional convolution operation method that can improve the processing efficiency of convolution operations to improve the deficiencies of the prior art.

[0004] In some embodiments, the three-dimensional convolution operation method includes: performing a dimension exchange on an input data to continuously arrange multiple elements of the input data in the depth dimension and the channel dimension together, thereby generating a first data; performing a convolution operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data; and reordering the operation data according to the original dimension form of the input data to generate an output data.

[0005] In some embodiments, the three-dimensional convolution operation device includes a buffer, a direct memory access circuit, a dimension exchange circuit, and a convolution operation circuit. The direct memory access circuit reads an input data from an external memory and stores the input data into the buffer. The dimension exchange circuit reads the input data from the buffer and performs a dimension conversion on the input data to continuously arrange multiple elements of the input data in the depth dimension and the channel dimension together to generate a first data. The convolution operation circuit performs a convolution operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data. The dimension exchange circuit also reorders the operation data according to an original dimension form of the input data to generate an output data.

[0006] The features, implementations, and effects of this application will be described in detail in the following preferred embodiments in conjunction with the drawings. Brief Description of the Drawings

[0007] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those skilled in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0008] Figure 1 Schematic diagram of the three-dimensional convolution operation device provided by the embodiment of the present application;

[0009] Figure 2 Flowchart of the three-dimensional convolution operation method provided by the embodiment of the present application;

[0010] Figure 3 For the embodiment of the present application to Figure 1 Schematic diagram of dimension exchange of the input data to generate the first data;

[0011] Figure 4A Schematic diagram of the first data after being segmented provided by the embodiment of the present application;

[0012] Figure 4B Schematic diagram of the second data after being segmented provided by the embodiment of the present application;

[0013] Figure 5A Data flow diagram of performing the operation of a single convolution operation layer provided by the embodiment of the present application; and

[0014] Figure 5B Data flow diagram of performing the operations of multiple convolution operation layers provided by the embodiment of the present application.

[0015]

Symbol Explanation

[0016] 100: Three-dimensional convolution operation device

[0017] 100A: External memory

[0018] 110: Direct memory access circuit

[0019] 120: Buffer

[0020] 130: Dimension exchange circuit

[0021] 140: Convolution operation circuit

[0022] 200: Three-dimensional convolution operation method

[0023] BL1~BL4: Dividing line

[0024] C, Ci, Ck, Co: Channel dimension

[0025] D000, D001, D010, D011, D100, D101: Data

[0026] D110, D111, D200, D201, D210, D211: Data

[0027] D1~D3: Data

[0028] DC: Operational Data

[0029] DC’: Stored Data

[0030] DIN: Input Data

[0031] DO: Output Data

[0032] DW1, DW2: Weight Data

[0033] D, Di, Dk: Depth Dimension

[0034] H, Hi, Hk: Height Dimension

[0035] S205, S210, S215, S220, S225, S230, S235, S240, S245, S250: Steps

[0036] S501~S504, S511~S516: Steps W, Wi, Wk: Width Dimension Detailed Implementation Manner

[0037] All the terms used in this article have their ordinary meanings. The definitions of the above terms in commonly used dictionaries, including the usage examples of any of the terms discussed herein, are only for illustration and should not limit the scope and meaning of this application. Similarly, this application is not limited only to the various embodiments shown in this specification.

[0038] Regarding the use of "coupled" or "connected" in this article, it can refer to two or more components making direct physical or electrical contact with each other, or making indirect physical or electrical contact with each other. It can also refer to the steps or actions of two or more components. As used herein, the term "circuit" can be a device composed of at least one transistor and / or at least one active or passive component connected in a certain manner to process signals.

[0039] In some embodiments, this application reorders the dimensional form of data to enable the direct access memory circuit to read the input data and weight data more efficiently, thereby improving the overall efficiency of the convolution operation.

[0040] Figure 1Schematic diagram of the three-dimensional convolution operation device 100 provided by the embodiments of the present application. In some embodiments, the three-dimensional convolution operation device 100 may be controlled by an operation platform (which runs on at least one computer host). In some embodiments, the three-dimensional convolution operation device 100 may include a processor (not shown), and other circuits in the three-dimensional convolution operation device 100 may be controlled by the processor.

[0041] The three-dimensional convolution operation device 100 includes a direct memory access (DMA) circuit 110, a buffer 120, a dimension exchange circuit 130, and a convolution operation circuit 140. The direct memory access circuit 110 can read the input data DIN and the weight data DW1 from the external memory 100A, and store the above two data in the buffer 120. In some embodiments, the external memory 100A may be (but not limited to) a dynamic random access memory. In some embodiments, the buffer 120 may be implemented by (but not limited to) a static random access memory.

[0042] In some embodiments, the convolution operation performed by the three-dimensional convolution operation device 100 is a three-dimensional convolution operation. Accordingly, the input data DIN may be a five-dimensional tensor, and its original dimension form is a five-dimensional form. For example, the arrangement order of the original dimension form may be expressed as (N, Di, Hi, Wi, Ci), where N is the batch and may be the dimension value of the highest dimension in the input data DIN, Di is the depth dimension, Hi is the height dimension, Wi is the width dimension, and Ci is the channel dimension. For example, in the following Figure 3 example, the original dimension form (N, Di, Hi, Wi, Ci) of the input data DIN is (1, 3, 2, 2, 5), which means that the number of elements (or data values) of the input data DIN in the depth dimension is 3), the number of elements in the height dimension is 2, the number of elements in the width dimension is 2, and the number of elements in the channel dimension is 5. Similarly, the original dimension form of the weight data DW1 can be expressed as (Dk, Hk, Wk, Ck, Co), where Dk is the depth dimension, Hk is the height dimension, Wk is the width dimension, C is the channel dimension, and Co is the dimension value of the highest dimension of the weight data DW1 (which is the same as the numerical value of the channel dimension of the output data DO).

[0043] The dimension exchange circuit 130 reads the input data DIN and the weight data DW1 from the buffer 120, and performs dimension exchange on the input data DIN according to a preset dimension form (which can be specified by the operation platform), so as to continuously arrange multiple elements of the input data DIN in the depth dimension and the channel dimension together, thereby generating the data D1, and storing the data D1 into the buffer 120. In some embodiments, the dimension exchange circuit 130 also performs dimension exchange on the weight data DW1 according to the default dimension form to continuously arrange multiple elements of the weight data DW1 in the depth dimension and the channel dimension together, thereby generating the data D2, and storing the data D2 into the buffer 120. The direct memory access circuit 110 can read the data D1 and the data D2 from the buffer 120, and transfer these data to the external memory 100A. Through the above steps, the input data DIN with the original dimension form and the weight data DW1 with the original dimension form will be respectively reordered into the data D1 with the default dimension form and the data D2 with the default dimension form. In this way, the efficiency of the convolution operation can be improved. The steps of dimension form conversion will be described in detail later with reference to Figure 3 In some embodiments, the dimension exchange circuit 130 can be implemented by a data processing circuit that executes a specific process or software. In some embodiments, if the weight data DW1 is constant data, the operation platform can store the data D2 corresponding to the weight data DW1 in the external memory 100A in advance to further accelerate the processing efficiency of the convolution operation.

[0044] The direct memory access circuit 110 can read the data D1 and the data D2 from the external memory 100A in blocks to the buffer 120. The convolution operation circuit 140 can read the data D1 and the data D2 from the buffer 120, and perform convolution operations on the data D1 and the data D2 in blocks to generate the operation data DC. In some embodiments, the operation platform (or the processor of the three-dimensional convolution operation device 100) divides the data D1 and the data D2 into multiple data blocks according to the system access bandwidth, the capacity of the buffer 120, the dimension size of the data D1, and the dimension size of the data D2. In this way, the operation platform (or the processor of the three-dimensional convolution operation device 100) can control the direct access memory circuit 110 to sequentially read these data blocks of the data D1 and these data blocks of the data D2 into the buffer 120, and control the convolution operation circuit 140 to sequentially read these data blocks of the data D1 and these data blocks of the data D2 from the buffer 120, and perform convolution operations block by block. After the convolution operation circuit 140 completes the convolution operations on all data blocks, the convolution operation circuit 140 can generate the operation data DC, and store it in the external memory 100A via the buffer 120 and the direct memory access circuit 110. In some embodiments, the convolution operation circuit 140 can be implemented by a digital signal processing circuit.

[0045] The dimension exchange circuit 130 can read the operation data DC via the direct memory access circuit 110 and the buffer 120, and reorder the operation data DC according to the original dimension form of the input data DIN to generate the output data DO. The dimension exchange circuit 130 can transfer the output data DO to the external memory 100A via the buffer 120 and the direct memory access circuit 110. In this way, other devices in the operation platform can correctly access the output data DO for subsequent applications.

[0046] Figure 2 The flowchart of the three-dimensional convolution operation method 200 provided by the embodiment of the present application. In some embodiments, the three-dimensional convolution operation method 200 can be executed by (but not limited to) Figure 1 the three-dimensional convolution operation device 100. To easily illustrate the related steps of the three-dimensional convolution operation device 100, please refer to Figure 1 and Figure 2 .

[0047] In step S205, the input data is dimensionally exchanged to continuously arrange multiple elements of the input data in the depth dimension and the channel dimension, thereby generating the first data. In step S210, the weight data is dimensionally exchanged to continuously arrange multiple elements of the weight data in the depth dimension and the channel dimension, thereby generating the second data. As described above, the direct memory access circuit 110 can read the input data DIN and the weight data DW1 from the external memory 100A and store these data in the buffer 120. The dimension exchange circuit 130 can read the data DIN and the weight data DW1 from the buffer 120, continuously arrange multiple elements of the input data DIN in the depth dimension and the channel dimension to generate the data D1, and continuously arrange multiple elements of the weight data DW1 in the depth dimension and the channel dimension to generate the data D2. Then, the dimension exchange circuit 130 transfers the data D1 and the data D2 to the external memory 100A via the buffer 120 and the direct memory access circuit 110.

[0048] Figure 3 For the Figure 1 input data DIN, the schematic diagram of dimensionally exchanging to generate the data D1 provided by the embodiment of the present application. As described above, the original dimension form of the input data DIN can be expressed as (N, Di, Hi, Wi, Ci). In Figure 3In the example, the original dimension form (N, Di, Hi, Wi, Ci) is (1, 2, 3, 2, 5). In other words, the input data DIN can be divided into three data groups in the height dimension Hi (i.e., multiple data corresponding to Hi = 0, 1, 2). In each data group, it can be further divided into two sub-data groups in the width dimension Wi (i.e., multiple data corresponding to Wi = 0, 1), and in each sub-data group, it can be further divided into two pieces of data in the depth dimension Di (i.e., multiple data corresponding to Di = 0, 1), and each piece of data includes 5 elements (or called data values; i.e., corresponding to Ci = 5). Specifically, the input data DIN includes multiple pieces of data D000, D001, D010, D011,..., D210, and D211, where the data D000 represents that its corresponding height dimension Hi, width dimension Wi, and depth dimension Di are all 0, and the data D001 represents that its corresponding height dimension Hi, width dimension Wi, and depth dimension Di are 0, 0, and 1 in sequence. And so on, the corresponding relationship between the above multiple pieces of data and their original dimension form should be understandable.

[0049] As Figure 3 shown, data D1 can be generated by continuously arranging multiple elements of the input data DIN in the depth dimension Di and the channel dimension Ci. The default dimension form of data D1 sequentially includes batch, height dimension, width dimension, depth dimension, and channel dimension, which can be sequentially noted as (N, H, W, D, C). Among them, different from the original dimension form of the input data DIN, in the preset dimension form, the depth dimension D and the channel dimension C are adjacent. Through the aforementioned dimension exchange, it can be seen that the multiple pieces of data in data D1 (i.e., data D000, D001, D010, D011,..., D210, and D211) are continuously arranged. In other words, these data can be continuously stored in the external memory 100A (and / or buffer 120). In this way, during the process of performing convolution operations, the direct memory access circuit 110 can continuously read multiple pieces of data in data D1 from the external memory 100A to perform convolution operations.

[0050] To explain it in another way, in a two-dimensional convolution operation, the convolution kernel (equivalent to the weight data DW1) slides in the width dimension and height dimension of the input data and performs multiplication and addition operations with corresponding multiple elements in the channel dimension simultaneously to generate the convolution operation result. In contrast, in a three-dimensional convolution operation, the convolution kernel further performs multiplication and addition operations with corresponding multiple elements in the depth dimension of the input data to generate the convolution operation result. Since the three-dimensional convolution operation accumulates in both the depth dimension and the channel dimension, multiple elements in the depth dimension and channel dimension of the input data DIN can be continuously arranged together to generate data D1. In this way, the operation of the three-dimensional convolution operation can be simplified to an operation similar to the two-dimensional convolution operation, thereby reducing the complexity of the three-dimensional convolution operation and increasing the processing efficiency.

[0051] Specifically, during the convolution operation process, the convolution operation circuit 140 can continuously read two pieces of data in data D1 via the direct memory access circuit 110 and the buffer 120 to perform one convolution operation. For example, these two pieces of data can be data D000 and D001, which include multiple (e.g., 10) elements corresponding to different depths (Di is 0 or 1), and these elements correspond to the same width (Wi is 0) and the same height (Hi is 0). Through the way of dimension exchange and continuous reading, not only can multiple pieces of data be continuously arranged together, but also the dimensionality of the depth dimension can be equivalently reduced. For example, the dimensional form (N, H, W, D, C) presented by data D1 during continuous reading is equivalent to (1, 3, 2, 1, 10), where the dimensionality of the depth dimension D is equivalently reduced to 1, and the dimensionality of the channel dimension becomes 10. In this way, the number of elements read by the direct memory access circuit 110 each time can be increased to improve the working efficiency of the direct memory access circuit 110, thereby enhancing the computing efficiency of the convolution operation. Figure 3 Only taking the input data DIN and data D1 as examples for illustration, it should be understood that Figure 3 the same steps in

[0052] also apply to the weight data DW1 and data D2 (or the weight data DW2 and data D3 mentioned later), so they will not be repeated here. Figure 1 and Figure 2 In step S215, according to the capacity of the buffer, the first data and the second data are each divided into multiple data blocks. In step S220, one data block among the multiple data blocks corresponding to the first data is read into the buffer. In step S225, one data block among the multiple data blocks corresponding to the second data is read into the buffer. In step S230, a convolution operation is performed according to the multiple data blocks stored in the buffer to generate partial data in the operation data. In step S235, the partial data is stored in the external memory.

[0053] As described above, the computing platform (or the processor of the three-dimensional convolution computing device 100) can divide data D1 and data D2 into multiple data blocks respectively according to the access bandwidth, the capacity of the buffer 120, the size of data D1, and the size of data D2. In some embodiments, the divided data blocks meet the following conditions: the value of the channel dimension of data D1 is equal to the value of the channel dimension of data D2; and the sliding (or offset) value of data D2 in the channel dimension is equal to the value of the channel dimension of the output data DO, but the present application is not limited thereto. After dividing data D1 and data D2 into multiple data blocks respectively, the direct memory access circuit 110 can read data D1 and data D2 into the buffer 120 in blocks (that is, read one data block of data D1 and one data block of data D2 into the buffer 120 each time) to provide these data blocks to the convolution computing circuit 140 to perform a convolution operation and generate partial data of the computing data DC (equivalent to the result of this convolution operation). The direct memory access circuit 110 can read out this partial data from the buffer 120 and transfer it to the external memory 100A. In some embodiments, data D1 and data D2 can be divided into multiple data blocks via an existing scheduling algorithm or a block convolution algorithm.

[0054] Figure 4A FIG. is a schematic diagram of the divided data D1 provided by the embodiments of the present application. In Figure 4A , a square in the channel dimension Ci represents a tensor data in data D1. Since the channel dimension Ci and the depth dimension Di are combined into the same dimension (in this example, the value of the channel dimension Ci is 8), data D1 is divided into blocks based on the dividing line BL1 (represented by a dotted line) in this dimension, and is divided into blocks based on the dividing line BL2 and the dividing line BL3 (represented by dotted lines) in the height dimension Hi and the width dimension Wi respectively. In this way, data D1 can be divided into 16 data blocks. For easy understanding, in Figure 4A , the corresponding setting methods of 4 data blocks are shown by dots and slashes respectively, and the positions of the remaining data blocks can be inferred by analogy. In practical applications, the size of data D1 is usually larger than the capacity of the buffer 120. Therefore, the direct memory access circuit 110 can read one data block of data D1 into the buffer 120 in blocks for the convolution computing circuit 140 to perform a convolution operation.

[0055] Figure 4BSchematic diagram of the data D2 after being segmented provided by the embodiment of the present application. In this example, the data D2 is divided into multiple data blocks based on the demarcation line BL4 (drawn as a dotted line) in the channel dimension Ck (or the depth dimension Ck; the two are combined into the same dimension). Since the size of the data D2 is usually small, it may not be further divided in the height dimension Hk and the width dimension Wk in this example, but the present application is not limited thereto. For ease of understanding, in Figure 4B the corresponding setting methods of multiple data blocks are shown by dots and slashes respectively. The direct memory access circuit 110 can read a data block of the data D2 into the buffer 120 in a block-by-block manner for the convolution operation circuit 140 to perform a convolution operation.

[0056] Continue to refer to Figure 2 , in step S240, it is confirmed whether the convolution operation is completed. If the convolution operation is completed (that is, all data blocks are calculated), step S245 is executed. Alternatively, if the convolution operation is not completed, step S215 is executed again to read the next data block of the data D1 and the data D2 to continue the convolution operation. By repeating the above steps, the complete operation data DC can be obtained. In step S245, it is confirmed whether the next layer of the network is still a convolution operation. If the next layer of the network is still a convolution operation, step S210 is executed again, and the convolution operation of the next layer is executed again using the foregoing multiple steps. The description of step S245 will be referred to later with reference to Figure 5A and Figure 5B description. Alternatively, if the next layer of the network is not a convolution operation, step S250 is executed. In step S250, the operation data is reordered in the original dimension form to generate the output data.

[0057] For example, if the next layer of the network is not a convolution operation, the direct access memory circuit 110 can read the operation data DC from the external memory 100A and transfer the operation data DC to the buffer 120. The dimension exchange circuit 130 can read the operation data DC from the buffer 120, reorder the operation data DC in the original dimension form of the input data DIN to generate the output data DO, and store the output data DO in the buffer 120. The direct access memory circuit 110 can read the output data DO from the buffer 120 and transfer it to the external memory 100A. In this way, other devices in the operation platform or system can use the output data DO to perform subsequent data processing. In other words, through step S250, the dimension form of the output data DO can be restored to the original dimension form applicable to the operation platform, so that other networks in the neural network model can correctly use the output data DO.

[0058] The multiple steps of the above three-dimensional convolution operation method 200 are only examples, and it is not limited that they must be executed in the order of this example. Without violating the step manner and scope of the embodiments of the present application, various steps in the three-dimensional convolution operation method 200 can be appropriately added, replaced, omitted, or executed in a different order (for example, they can be executed simultaneously or partially simultaneously).

[0059] Figure 5A This is a data flow diagram for performing operations of a single convolution operation layer provided by an embodiment of the present application. In this example, the neural network model operated by the three-dimensional convolution operation device 100 includes a single convolution layer (that is, the aforementioned convolution operation includes a single convolution operation layer). In step S501, dimension exchange is performed on the input data DIN (that is, multiple elements of the input data DIN in the depth dimension and the channel dimension are continuously arranged together) to generate data D1. In step S502, dimension exchange is performed on the weight data DW1 (that is, multiple elements of the weight data DW1 in the depth dimension and the channel dimension are continuously arranged together) to generate data D2. In step S503, the operations of a single convolution operation layer are performed on data D1 and data D2 in blocks to generate operation data DC (equivalent to Figure 2 steps S215 to S240). In step S504, the operation data DC is reordered according to the original dimension form to generate output data DO.

[0060] Figure 5A For the multiple steps of Figure 2 reference can be made to the descriptions of the multiple steps of the foregoing

[0061] Figure 5B This is a data flow diagram for performing operations of multiple convolution operation layers provided by an embodiment of the present application. Compared with Figure 5A In the example of Figure 5B the neural network model operated by the three-dimensional convolution operation device 100 includes a multi-layer convolution network. For example, the aforementioned convolution operation includes a first convolution operation layer and a second convolution operation layer.

[0062] In step S511, dimension exchange is performed on the input data DIN to generate data D1. In step S512, dimension exchange is performed on the weight data DW1 to generate data D2. In step S513, the operations of the first convolution operation layer are performed on data D1 and data D2 in blocks to generate temporary data DC' (which can be stored in Figure 1buffer 120). In step S514, dimension swapping is performed on the weight data DW2 (i.e., multiple elements of the weight data DW2 in the depth dimension and the channel dimension are continuously arranged together, where the weight data DW2 is equivalent to the convolution kernel of the second convolutional layer) to generate data D3 (which can be stored in Figure 1 the external memory 100A and can be transferred to the buffer 120 via the direct memory access circuit 110). In step S515, the operation of the second convolutional operation layer is performed on the temporary data DC' and the data D3 in blocks to generate the operation data DC. In step S516, the operation data DC is reordered according to the original dimension form to generate the output data DO. Figure 5B The multiple steps of Figure 2 can refer to the descriptions of the multiple steps in the foregoing Figure 2 , so they will not be repeated here. For example, steps S513 and S515 can refer to

[0063] the descriptions of steps S215 to S240 in

[0064] . In some other embodiments, if the weight data DW2 is constant data, the operation platform can store the data D3 corresponding to the weight data DW2 in the external memory 100A in advance.

[0065] It should be noted that in the embodiments of the present application, "multiple" includes "two" or "more than two".

[0066] In summary, the three-dimensional convolution operation device and the three-dimensional operation method provided by the embodiments of the present application can utilize the reordering of the dimension form of the data to improve the access efficiency of the direct memory access circuit. Further, through the above reordering, the complexity of the three-dimensional convolution operation can be reduced, so that the three-dimensional convolution device can implement the three-dimensional convolution operation by performing steps similar to or the same as those of the two-dimensional convolution operation. In this way, the processing efficiency of the three-dimensional convolution operation can be improved.

[0067] The three-dimensional convolution operation device and the three-dimensional convolution operation method provided by the embodiments of the present application have been introduced in detail above. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A three-dimensional convolution operation method, characterized in that, Comprising: Performing dimension swapping on an input data to continuously arrange a plurality of elements of the input data in the depth dimension and the channel dimension together, thereby generating a first data; Performing a convolution operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data; Performing dimension swapping on the first weight data to continuously arrange a plurality of elements of the first weight data in the depth dimension and the channel dimension together, thereby generating the second data; And Reordering the operation data according to the original dimension form of the input data to generate an output data; wherein, performing the convolution operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data includes: Continuously reading a plurality of elements corresponding to different depths in the first data to perform the convolution operation, wherein the plurality of elements correspond to the same width and the same height; The convolution operation includes a first convolution operation layer and a second convolution operation layer, and performing the convolution operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data includes: Performing the operation of the first convolution operation layer on the first data and the second data in blocks to generate a temporary data; Performing dimension swapping on a second weight data to continuously arrange a plurality of elements of the second weight data in the depth dimension and the channel dimension together, thereby generating a third data; and Performing the operation of the second convolution operation layer on the temporary data and the third data in blocks to generate the operation data.

2. A three-dimensional convolution operation device, characterized in that, Comprising: A buffer; A direct memory access circuit that reads an input data from an external memory and stores the input data into the buffer; wherein, the external memory also stores a second data, and a plurality of elements of the second data in the depth dimension and the width dimension are continuously arranged together; A dimension swapping circuit that reads the input data from the buffer and performs dimension swapping on the input data to continuously arrange a plurality of elements of the input data in the depth dimension and the channel dimension together to generate a first data; and A convolutional operation circuit performs a convolutional operation on the first data and a second data corresponding to a first weight data in blocks to generate an operation data; the direct memory access circuit also reads the first weight data from the external memory to the buffer, and the dimension exchange circuit also reads the first weight data from the buffer and performs dimension exchange on the first weight data to continuously arrange a plurality of elements of the first weight data in the depth dimension and the channel dimension together to generate the second data; the convolutional operation circuit continuously reads a plurality of elements corresponding to different depths in the first data to perform the convolutional operation, where the plurality of elements correspond to the same width and the same height; the convolutional operation includes a first convolutional operation layer and a second convolutional operation layer, and the convolutional operation circuit performs the operation of the first convolutional operation layer on the first data and the second data in blocks to generate a temporary data, and performs the operation of the second convolutional operation layer on the temporary data and a third data corresponding to a second weight data in blocks to generate the operation data; the direct memory access circuit also reads the second weight data from the external memory to the buffer, and the dimension exchange circuit also reads the second weight data from the buffer and performs dimension exchange on the second weight data to continuously arrange a plurality of elements of the second weight data in the depth dimension and the channel dimension together to generate the third data; Wherein, the dimension exchange circuit also reorders the operation data according to an original dimension form of the input data to generate an output data.

3. The three-dimensional convolution operation device according to claim 2, characterized in that, The first data generated by the dimension exchange circuit is stored in the external memory via the buffer and the direct memory access circuit, and the convolutional operation circuit reads out the first data in blocks via the direct memory access circuit and the buffer.

Images