Acceleration methods, devices and terminal equipment for convolution computation

Abstract

The present application is applicable to the field of artificial intelligence technology, and provides a method, apparatus, terminal device, storage medium and circuit system for accelerating convolution calculation, wherein a method for accelerating convolution calculation includes: after detecting a preset instruction, obtaining a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle; wherein, in each operation cycle: storing the convolution elements of the current cycle of at least two convolution kernels into a first storage unit array, and storing the characteristic elements of the current cycle of at least two characteristic matrices into a second storage unit array; according to the operation rules of the target operation units corresponding to the convolution elements of the current cycle and the characteristic elements of the current cycle in the operation unit array, controlling each target operation unit to perform parallel operation to obtain the operation results of each target operation unit; obtaining the output result of the operation cycle according to the operation results of each target operation unit.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and in particular, relates to a method, apparatus, terminal device, computer-readable storage medium, and circuit system for accelerating convolution calculations. Background Art

[0002] Convolutional neural network is an efficient recognition method that has been developed in the field of artificial intelligence in recent years and has attracted widespread attention. It has outstanding performance in fields such as image processing.

[0003] Convolutional neural networks can generally include convolution layers, pooling layers, and fully connected layers (FC layers). In the convolution layer, the input feature map can be convolved with the convolution kernel corresponding to the convolution layer, so as to extract features from the input feature map and obtain the output feature map. When performing convolution operations, since the dimensions and number of parameters of the input feature map and the convolution kernel are often large, the amount of data is large, and the hardware equipment takes a long time to perform convolution calculations. Therefore, there is an urgent need for a method that can perform convolution operations efficiently. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a method, apparatus, terminal device, computer-readable storage medium and circuit system for accelerating convolution calculations, which can perform convolution operations more efficiently.

[0005] In a first aspect, an embodiment of the present application provides a method for accelerating convolution calculation, including:

[0006] After detecting the preset instruction, obtaining a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle, wherein the preset instruction is used to instruct at least two convolution kernels to perform a convolution operation with at least two feature matrices;

[0007] Among them, in each operation cycle:

[0008] The convolution elements of the current period of at least two convolution kernels are stored in a first storage unit array, and the characteristic elements of the current period of at least two characteristic matrices are stored in a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the convolution element of the current period of each convolution kernel in the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located;

[0009] According to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one;

[0010] The output result of the operation cycle is obtained according to the operation results of each target operation unit.

[0011] In a second aspect, an embodiment of the present application provides an acceleration device for convolution calculation, including:

[0012] A processing module, configured to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices;

[0013] Wherein, in each operation cycle, the device further includes:

[0014] A storage module, used for storing the convolution elements of the current period of at least two convolution kernels into a first storage unit array, and storing the characteristic elements of the current period of at least two characteristic matrices into a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the convolution element of the current period of each convolution kernel in the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located;

[0015] A control module, used to control each target operation unit to perform parallel operation according to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, so as to obtain the operation results of each target operation unit, wherein the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one;

[0016] The output module is used to obtain the output result of the operation cycle according to the operation results of each target operation unit.

[0017] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the acceleration method of the convolution calculation as in the first aspect when executing the computer program.

[0018] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the acceleration method of the convolution calculation as in the first aspect is implemented.

[0019] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the acceleration method of the convolution calculation mentioned above in the first aspect.

[0020] In a sixth aspect, an embodiment of the present application provides a circuit system, comprising a first storage cell array, a second storage cell array, an operation cell array, and a control unit, wherein the rows in the operation cell array correspond one-to-one to the rows in the first storage cell array, and the columns in the operation cell array correspond one-to-one to the columns in the second storage cell array;

[0021] A control unit, configured to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices;

[0022] Among them, in each operation cycle:

[0023] The first storage unit array is used to store the convolution elements of the current period of at least two convolution kernels. If there are any two convolution kernels, the feature matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array:

[0024] A second storage unit array is used to store the characteristic elements of the current period of at least two characteristic matrices. If there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the characteristic elements of the current period corresponding to the characteristic matrices corresponding to the two convolution kernels are located in different rows in the second storage unit array;

[0025] Each target operation unit in the operation unit array is used to perform parallel operation according to the corresponding convolution elements of the current period, the characteristic elements of the current period and the operation rules, so as to obtain the operation results of each target operation unit;

[0026] The control unit is used to obtain the output result of the operation cycle according to the operation results of each target operation unit.

[0027] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0028] In an embodiment of the present application, after detecting a preset instruction, a target operation result corresponding to the preset instruction can be obtained based on an output result corresponding to at least one operation cycle, wherein in each operation cycle, the convolution elements of at least two convolution kernels in this cycle are stored in a first storage unit array, and the feature elements of at least two feature matrices in this cycle are stored in a second storage unit array, so that the data of different convolution kernels and different feature matrices in this operation cycle can be conveniently stored through the first storage unit array and the second storage unit array, and can be transmitted to the operation unit array in parallel.

[0029] Then, according to the operation rules of the target operation units corresponding to the convolution elements of this cycle and the characteristic elements of this cycle in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, and then the output result of the operation cycle is obtained according to the operation results of each target operation unit. Among them, if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operation of the two convolution kernels are different, then the convolution elements of this cycle corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array, and the columns where the convolution elements of this cycle of each convolution kernel in the two convolution kernels are located correspond to the rows where the corresponding characteristic elements of this cycle are located, the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one. It can be seen that through the embodiments of the present application, hardware resources can be reasonably allocated, and different target operation units can be used to realize the parallel operation of multiple dimensions such as the convolution kernels corresponding to different characteristic elements and the different convolution elements of the convolution kernel in this cycle, which greatly improves the operation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 It is a schematic diagram of the implementation process of a convolution calculation acceleration method provided in an embodiment of the present application;

[0032] Figure 2 It is a schematic diagram of performing convolution operation based on hardware structure provided in an embodiment of the present application;

[0033] Figure 3 It is another operation schematic diagram of performing convolution operation based on hardware structure provided in an embodiment of the present application;

[0034] Figure 4 It is a schematic diagram of rearranging the economic elements of this period provided by an embodiment of the present application;

[0035] Figure 5 It is a schematic diagram of performing parallel operations on convolution kernels corresponding to the same feature matrix based on the hardware structure provided in an embodiment of the present application;

[0036] Figure 6 It is a schematic diagram of a grouped convolution operation provided in an embodiment of the present application;

[0037] Figure 7 It is a schematic diagram of screening out valid data from matrix output results provided by an embodiment of the present application;

[0038] Figure 8 This is a schematic diagram of splicing valid data provided by an embodiment of the present application;

[0039] Fig. 9 is a schematic diagram of an acceleration device for convolution calculation provided in an embodiment of the present application;

[0040] Fig.10 is a schematic diagram of a terminal device provided in an embodiment of the present application;

[0041] Fig.11 It is a schematic diagram of a circuit system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0043] Before explaining the embodiments of the present application, some terms in the embodiments of the present application are briefly introduced.

[0044] 1. Convolution operation

[0045] The convolution operation in the embodiment of the present application is a convolution operation between two matrices. In a convolutional neural network, a convolution operation generally involves a convolution operation between a convolution kernel and a feature matrix, wherein the convolution kernel and the feature matrix can be two-dimensional or multi-dimensional matrices, and the sizes of the convolution kernel and the feature matrix can be various. Specifically, in the convolution operation, the convolution kernel can be rotated 180 degrees, and then the feature matrix can be traversed with a preset step size, and at each position in the traversal process, the corresponding elements are multiplied and then summed for the feature submatrix with the same size as the convolution kernel. At this time, a corresponding output value can be obtained at different positions, and according to each output value, an output feature layer obtained by convolution operation of the convolution kernel and the feature matrix can be obtained. In a convolutional neural network, if there are multiple convolution kernels, multiple output feature layers can be obtained, and multiple output feature layers can be combined to obtain an output feature map.

[0046] Among them, if the size of the feature matrix is ​​C*H*W, there are N convolution kernels, the number of output feature maps is the same as the number of convolution kernels, which is N. The size of each convolution kernel is C*K*K, and the total number of parameters of N convolution kernels is N*C*K*K.

[0047] In some application scenarios, the elements in the convolution kernel can be rearranged to obtain a new matrix based on the rules of the convolution operation, thereby converting the convolution operation into other operations between the new matrix and the input feature matrix, such as converting it into a multiplication and then summing the elements in the multiplied matrix.

[0048] 2. Convolution kernel

[0049] Through the convolution operation, the convolution kernel can be used to extract features from the feature matrix. The number of channels of the convolution kernel refers to the number of feature layers contained in the convolution kernel, and the number of channels of the convolution kernel can be determined according to the actual scenario.

[0050] In actual application, each convolution kernel in the convolutional neural network slides on the feature matrix of the convolution layer and performs operations with the feature submatrices at various positions of the feature matrix. After traversing the feature matrix, a feature layer corresponding to the output of the convolution kernel can be obtained. The number of output eigenvalues ​​on the feature layer is the same as the number of feature submatrices at different positions of the feature matrix. It can be understood that the number of output channels corresponding to the output feature map is the same as the number of convolution kernels.

[0051] 3. Feature Matrix

[0052] In the embodiment of the present application, the feature matrix is ​​a matrix that performs convolution operations with the convolution kernel. The number of input channels corresponding to the feature matrix refers to the number of feature layers contained in the feature matrix.

[0053] The convolution kernel can slide on the feature matrix with a preset step size to perform element-by-element multiplication and summation with the corresponding feature submatrix at each position until the feature matrix is ​​traversed. The size of the feature submatrix is ​​the same as the size of the convolution kernel, and the number of feature submatrices can be the number of elements contained in an output feature layer, which can also be called the number of output eigenvalues.

[0054] The embodiments of the present application are described in detail below.

[0055] The acceleration method for convolution calculation provided in the embodiment of the present application can be applied to terminal devices.

[0056] Exemplarily, the terminal device may be a desktop computer, a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a server, a personal digital assistant (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific type of the terminal device.

[0057] In some examples, the terminal device may include a control unit, and the terminal device executes the embodiments of the present application through the control unit.

[0058] Specifically, the control unit can be connected to the first storage unit array, the second storage unit array and the operation unit array respectively, so as to control them respectively. For example, the control unit can send the corresponding data to be stored to the first storage unit array and the second storage unit array respectively, and can also send instructions to clear data to the first storage unit array and the second storage unit array respectively. In addition, the control unit can also send instructions to the operation unit array to instruct the operation unit to perform operations, and obtain the operation results of each operation unit, and then process them to obtain output results.

[0059] Specifically, the acceleration method of the convolution calculation may include:

[0060] After detecting the preset instruction, a target operation result corresponding to the preset instruction is obtained according to an output result corresponding to at least one operation cycle, and the preset instruction is used to instruct at least two convolution kernels to perform convolution operations with at least two feature matrices.

[0061] Among them, the convolution kernel is the convolution kernel of a convolutional neural network. The convolutional neural network is used for image processing or video processing. For example, the convolutional neural network is used in application scenarios such as image classification, target detection, image restoration or video classification. The convolution kernel can be used to extract features of an image or video. The at least two feature matrices can be partial pixel matrices or all pixel matrices of an image or video input to the convolutional neural network, or can be part or all of the matrices output by a certain intermediate layer of the convolutional neural network after the image or video is input to the convolutional neural network.

[0062] In the embodiment of the present application, the duration of each operation cycle can be the same or different. Specifically, a specific instruction can be generated at a fixed time interval to trigger each operation cycle, or the next operation cycle can be triggered after the output result of the previous operation cycle is obtained.

[0063] The correspondence between the at least two convolution kernels and the at least two feature matrices for the convolution operation can be determined according to the actual application scenario. For example, the at least two convolution kernels can be convolved with the at least two feature matrices in a one-to-one correspondence, or at least two convolution kernels can be convolved with the same feature matrix, or one convolution kernel can be convolved with different feature matrices, etc.

[0064] The number of operation cycles can also be determined according to the actual application scenario. In one example, multiple feature sub-matrices can be obtained by sliding the convolution kernel on the feature matrix. In each operation cycle, the convolution elements at the same position on each channel of the convolution kernel and the feature elements at the same position on each channel of the corresponding feature sub-matrix can be operated. At this time, the number of operation cycles can be the same as the number of elements in a feature layer in the convolution kernel.

[0065] In the embodiment of the present application, there may be multiple ways to obtain the target operation result corresponding to the preset instruction according to the output result corresponding to at least one operation cycle.

[0066] Exemplarily, each output result may be in the form of an output matrix. Based on the rules of the convolution operation (e.g., the elements involved in the accumulation step in the convolution operation and their arrangement rules, etc.), the elements in each output matrix may be concatenated and accumulated for specific rows or columns, etc., so as to obtain an output feature map corresponding to the convolution operation.

[0067] Among them, Figure 1 As shown, in each operation cycle, the method includes:

[0068] Step S101, storing the current period convolution elements of at least two convolution kernels into a first storage unit array, and storing the current period characteristic elements of at least two characteristic matrices into a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the current period convolution elements corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the current period convolution element of each of the two convolution kernels is located corresponds to the row where the corresponding current period characteristic element is located.

[0069] The first memory cell array may include a plurality of regularly arranged first memory cells, and the second memory cell array may also include a plurality of regularly arranged second memory cells.

[0070] In the embodiment of the present application, the storage element array corresponding to the first storage cell array and the storage cell array corresponding to the second storage cell array can be subjected to matrix multiplication operations through the operation unit array. Therefore, the distribution mode of the convolution elements of this period in the first storage cell array and the distribution mode of the characteristic elements of this period in the second storage cell array can be determined based on the connection mode between the operation unit array and the first storage cell array and the second storage cell array.

[0071] Specifically, the current-period convolution elements of different convolution kernels are respectively stored in different rows of the first storage unit array. For a convolution kernel, the feature layer of each channel in the convolution kernel can obtain the current-period convolution elements from the same position. At this time, the number of current-period convolution elements of the convolution kernel is the same as the number of channels of the convolution kernel, and each current-period convolution element of the convolution kernel can be located in different columns of the same row.

[0072] The characteristic elements of the current period of the characteristic matrix can be stored in the second storage cell array. Different rows in the second storage cell array can be used to store the characteristic elements of the current period of different channels of the characteristic matrix. In some examples, if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the characteristic elements of the current period of the two characteristic matrices corresponding to the two convolution kernels are also stored in different rows in the second storage cell array. Different columns in the second storage cell array can be used to store the characteristic elements of the current period of different characteristic sub-matrices corresponding to the characteristic matrix.

[0073] It should be noted that the specific sizes of the first storage cell array and the second storage cell array are not limited here. In some examples, the first storage cell array and the second storage cell array can be predetermined based on existing computing resources and information such as the parameter amount of the correlation matrix in the convolution calculation involved. When executing the embodiment of the present application, there may be idle storage cells in the first storage cell array and the second storage cell array.

[0074] In some embodiments, a ratio of the number of rows in the first storage cell array to the number of channels of a convolution kernel is greater than 2.

[0075] In an embodiment of the present application, the size of the integer part of the ratio of the number of rows in the first storage cell array to the number of channels of each convolution kernel can be used to determine the size of the efficiency improvement in a specified dimension; wherein the specified dimension refers to the dimension of the convolution kernel for performing convolution operations on different feature matrices. If the ratio of the number of rows in the first storage cell array to the number of channels of each convolution kernel is greater than 2, then at least two convolution kernels can have convolution elements of this period corresponding to them located in different columns in the first storage cell array, and, generally speaking, the first storage cell array can have at least two rows, so the related operations of the convolution kernels for performing convolution operations on different feature matrices can be processed in parallel, thereby improving the operation speed.

[0076] Step S102, according to the operation rules of the target operation units corresponding to the convolution elements of this period and the characteristic elements of this period in the operation unit array, control each target operation unit to perform parallel operation to obtain the operation results of each target operation unit, the rows in the operation unit array correspond one-to-one to the rows in the first storage unit array, and the columns in the operation unit array correspond one-to-one to the columns in the second storage unit array.

[0077] The operation unit array includes at least two operation units, and the target operation unit is the operation unit that obtains the convolution element of the current period and the characteristic element of the current period. Among them, the first storage unit in the first storage unit array may send the stored convolution element of the current period to the corresponding target operation unit, and the second storage unit in the second storage unit array may send the stored characteristic element of the current period to the corresponding target operation unit.

[0078] In the embodiment of the present application, the rows in the operation unit array correspond one-to-one to the rows in the first storage unit array, and the columns in the operation unit array correspond one-to-one to the columns in the second storage unit array. Therefore, the operation unit array can be used to perform matrix multiplication operations on the storage element array corresponding to the first storage unit array and the storage cell array corresponding to the second storage unit array.

[0079] In a specific application, the operation unit in the i-th row and j-th column of the operation unit array can obtain a row of convolution elements of the current period from the first storage unit in the i-th row of the first storage unit array, and obtain a column of characteristic elements of the current period from the second storage unit in the j-th column of the second storage unit array, and then multiply the convolution elements of the current period in the row and the characteristic elements of the current period in the column one by one according to the order, and then accumulate them to obtain the operation result of the operation unit in the i-th row and j-th column.

[0080] It can be understood that in some examples, based on actual computing needs, in order to avoid wasting hardware resources, the number of columns of the first storage cell array can be the same as the number of rows of the second storage cell array, the number of rows of the operation unit array can be the same as the number of rows of the first storage array, and the number of columns of the operation unit array can be the same as the number of columns of the second storage cell array.

[0081] The specific implementation of the embodiment of the present application is described below with an example.

[0082] In some scenarios, in order to improve computing efficiency, hardware is designed for different channels of the convolution kernel in each convolution operation to perform parallel operations. However, in some cases, the number of channels of the convolution kernel in a single convolution operation is not large, resulting in a failure to reasonably apply the hardware resources designed based on different channels of the convolution kernel, resulting in reduced hardware utilization, resulting in lower computing speed and longer computing time.

[0083] like Figure 2 As shown, during hardware design, Figure 2 Different rows in the first storage cell array are designed to store data of different convolution kernels.

[0084] The second storage unit array is used to store data from different channels and different characteristic sub-matrices of the same characteristic matrix.

[0085] Each row in the operation unit array is numbered based on hexadecimal, and 16 numbers with tail numbers ranging from 00 to 0F are obtained; the tail numbers of each column in the operation unit array are numbered from 00 to 07. Each operation unit includes 4 multiply accumulate (MAC) units, so that the operation unit can realize the multiply accumulate operation between the corresponding 4 convolution elements of this period and the corresponding 4 characteristic elements of this period.

[0086] However, in cases such as grouped convolution operations and the presence of multiple parallel convolutional layers, the number of channels corresponding to a single feature matrix (i.e., input channels) and the number of channels corresponding to the output feature map (i.e., output channels) may be small. For example, in some examples, such as Figure 2 As shown, both the input channel and the output channel are 1. Figure 2 The utilization rate of the hardware resources in a single operation cycle is low, and a large number of hardware resources are in an idle state (ie, computing units displayed as idle states).

[0087] In a specific example of the embodiment of the present application, Figure 3 As shown, based on the original hardware design, when the convolution elements of the current period of different convolution kernels are stored in the first storage unit array, it is presented as shown in the array diagram on the far left. At this time, the data of each row in the first storage unit array can actually only be operated with the characteristic elements of the current period in the first row and first column of the second storage unit array. When each convolution kernel needs to operate with different characteristic matrices, the element storage method of the leftmost array diagram cannot meet the requirements.

[0088] Therefore, in Figure 3 In the example, the elements in the leftmost feature map can be rearranged so that the four convolution elements Co0_ci0 to Co3_ci0 are used as the convolution elements of this cycle, and then subsequent parallel operations are performed. This allows the related operations of the convolution kernels that perform convolution operations on different feature matrices to be processed in parallel, greatly improving the operation speed.

[0089] like Figure 4 As shown, it can be an exemplary schematic diagram for rearranging the elements in the leftmost feature map.

[0090] The rearrangement circuit may include a selector and a shift unit. Since the number of convolution kernels is 8 and the number of columns of the first storage unit array is only 4, in one operation cycle, the four convolution elements Co0_ci0 to Co3_ci0 may be selected for parallel operation, and then the four convolution elements Co4_ci0 to Co7_ci0 may be selected for parallel operation in the next operation cycle.

[0091] It should be noted that, in some other examples, the storage method of the convolution elements of the current period in the first storage unit array may be predetermined, and then the convolution elements of the current period are directly stored as Figure 3 The first memory cell array is in the form of a shift unit that performs a shift operation without rearranging the shift unit in the circuit.

[0092] In one embodiment, if there are any two convolution kernels, and the feature matrices corresponding to the convolution operations of the two convolution kernels are the same, then the convolution elements of this period corresponding to the two convolution kernels are located in the same column and in different rows in the first storage unit array.

[0093] It should be noted that the convolution elements of the current period corresponding to the two convolution kernels are located in the same column in the first storage unit array, which can be the same multiple columns in the first storage unit array, rather than being limited to being located in the same column. For example, in some examples, the convolution elements of the current period corresponding to the two convolution kernels are located in columns 1 to 4 in the first storage unit array.

[0094] In an embodiment of the present application, if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operations of the two convolution kernels are the same, then it can be considered that the characteristic elements corresponding to the convolution operations of the two convolution kernels are also the same. At this time, the convolution elements of the current period corresponding to the two convolution kernels are located in the same column and different rows in the first storage unit array. The convolution elements of the current period corresponding to the two convolution kernels can be subsequently calculated with the characteristic elements of the current period in the same row in the second storage unit array through the operation unit array, without repeating the same characteristic elements through another row in the second storage unit array, thereby saving hardware resources.

[0095] like Figure 5 As shown, a parallel computing method is used based on the hardware resources in the embodiment of the present application for performing convolution operations with convolution kernels having the same feature matrix corresponding to the convolution operations.

[0096] In which, in a certain operating cycle, the first storage unit array can store the convolution elements of the current cycle from 8 convolution kernels, and the 8 convolution kernels all perform convolution operations with the same feature matrix. At this time, the 4 convolution elements of the current cycle corresponding to each of the 8 convolution kernels are distributed in the 1st to 4th columns of the first storage unit array.

[0097] In the operation unit array, the specific operation rules of the operation unit PE00 are:

[0098] Co0_ci0*Ci0_d0+Co0_ci1*Ci1_d0+Co0_ci2*Ci2_d0+Co0_ci3*Ci3_d0

[0099] The specific operation rules of the operation unit PE01 are:

[0100] Co0_ci0*Ci0_d1+Co0_ci1*Ci1_d1+Co0_ci2*Ci2_d1+Co0_ci3*Ci3_d1

[0101] The specific operation rules of the operation unit PE10 are:

[0102] Co1_ci0*Ci0_d0+Co1_ci1*Ci1_d0+Co1_ci2*Ci2_d0+Co1_ci3*Ci3_d0

[0103] Similarly, the operation rules of other operation units can be obtained.

[0104] In some embodiments, the preset instructions are used to instruct group convolution operations on a convolution kernel set and a preset feature matrix, at least two feature matrices are obtained by segmenting the preset feature matrix according to the channels of the preset feature matrix, the convolution kernel set includes at least two convolution kernels, and each group corresponding to the group convolution operation includes a feature matrix and at least one convolution kernel.

[0105] Grouped convolution operation is a type of operation in the convolution layer that is different from ordinary convolution operation.

[0106] In the grouped convolution operation, assuming that the size of the preset feature matrix is ​​C*H*W, and the number of output feature maps is N, if the preset feature matrix is ​​divided into G feature matrices, the number of feature matrices in each group is C / G, the number of output feature maps in each group is N / G, the size of each convolution kernel is C / G*K*K, the total number of convolution kernels is still N, the number of convolution kernels in each group is N / G, the convolution kernel only performs convolution operations with the feature matrices in the same group, and the total number of convolution kernel parameters is N*C / G*K*K. It can be understood that the total number of parameters of the convolution kernel in the grouped convolution operation is reduced to 1 / G of the original compared to the ordinary convolution operation with the same input matrix.

[0107] like Figure 6 The figure shows an exemplary schematic diagram of grouped convolution operation.

[0108] In this example, the input channels of the preset feature matrix are 16, the output channels of the output feature map are 8, and the number of groups is 4. Then, each group includes a feature matrix with 4 input channels and 2 convolution kernels. At this time, in the grouped convolution operation, there are at least two convolution kernels whose corresponding feature matrices for convolution operation are different. Through the embodiments of the present application, the operation efficiency of the grouped convolution operation can be improved.

[0109] Step S103, obtaining the output result of the operation cycle according to the operation results of each target operation unit.

[0110] In the embodiment of the present application, the operation results of each target operation unit can be in the form of a matrix. In some examples, each operation result can be spliced ​​and processed based on a preset data arrangement method to obtain the output result of the operation cycle. In other examples, if the form of each operation result already meets the preset data arrangement requirements, the output result can be directly obtained.

[0111] In some embodiments, according to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, including:

[0112] Sending an instruction signal to the operation unit array so that each operation unit in the operation unit array performs parallel operation according to the instruction signal to obtain a matrix output result of the operation unit array;

[0113] Filter out valid data from the matrix output results. The valid data is the calculation result obtained by the target operation unit according to the corresponding convolution elements of this period and the characteristic elements of this period.

[0114] According to the operation results of each target operation unit, the output results of the operation cycle are obtained, including:

[0115] Based on the valid data, the output result of the operation cycle is obtained.

[0116] Among them, the distribution of the operation units where the valid data is located can be determined based on the information of the rows storing the convolution elements of the current period in the first storage unit array and the information of the columns storing the characteristic elements of the current period in the second storage unit array, so as to determine the valid data from the matrix output result based on the distribution. In addition, the valid data can also be screened out according to the input data of each operation unit.

[0117] After obtaining valid data, the valid data can be spliced ​​and processed based on the preset data arrangement mode to obtain the output result of the operation cycle. Of course, in some examples, if the form of the screened valid data meets the preset data arrangement requirements, the output result can be directly obtained.

[0118] In the embodiment of the present application, each operation unit can be driven to perform parallel operation by an indication signal, and then valid data can be screened out from the matrix output results of the operation unit array, without distinguishing the target operation unit from each operation unit before parallel calculation, and generating corresponding control signals for each target operation unit. It can be seen that the embodiment of the present application can effectively improve the processing efficiency by uniformly controlling the operation unit array and uniformly screening according to the matrix output results, avoiding the consumption of processing resources and long time consumption caused by the separate control of each target operation unit.

[0119] like Figure 7 The figure shows a schematic diagram of an example of filtering out valid data from the matrix output results.

[0120] Among them, the valid data is distributed in the 1st to 4th rows and the 1st to 16th columns in the matrix output result, and the invalid data is distributed in the 5th to 8th rows and the 1st to 16th columns in the matrix output result. Therefore, the valid data can be obtained by screening the data of the 1st to 4th rows and the 1st to 16th columns from the matrix output result.

[0121] like Figure 8 As shown, in some examples, after obtaining valid data, based on the data in rows 1 to 4 and columns 1 to 16 in the matrix output result and the data arrangement method of the preset output result, the valid data can be spliced ​​to obtain the output result of the operation cycle.

[0122] In some embodiments, during each operation cycle:

[0123] After obtaining the output result of the operation cycle according to the operation results of each target operation unit, the method further includes:

[0124] The data in the first storage unit array, the second storage unit array and the operation unit array are cleared.

[0125] In an embodiment of the present application, in an operation cycle, after obtaining the output result of the operation cycle, the corresponding data in the first storage cell array, the second storage cell array and the operation unit array can be cleared to provide space for data storage and corresponding operations in the next operation cycle.

[0126] In an embodiment of the present application, after detecting a preset instruction, a target operation result corresponding to the preset instruction can be obtained based on an output result corresponding to at least one operation cycle, wherein in each operation cycle, the convolution elements of at least two convolution kernels in this cycle are stored in a first storage unit array, and the feature elements of at least two feature matrices in this cycle are stored in a second storage unit array, so that the data of different convolution kernels and different feature matrices in this operation cycle can be conveniently stored through the first storage unit array and the second storage unit array, and can be transmitted to the operation unit array in parallel.

[0127] Then, according to the operation rules of the target operation units corresponding to the convolution elements of this cycle and the characteristic elements of this cycle in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, and then the output result of the operation cycle is obtained according to the operation results of each target operation unit. Among them, if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operation of the two convolution kernels are different, then the convolution elements of this cycle corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array, and the columns where the convolution elements of this cycle of each convolution kernel in the two convolution kernels are located correspond to the rows where the corresponding characteristic elements of this cycle are located, the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one. It can be seen that through the embodiments of the present application, hardware resources can be reasonably allocated, and different target operation units can be used to realize the parallel operation of multiple dimensions such as the convolution kernels corresponding to different characteristic elements and the different convolution elements of the convolution kernel in this cycle, which greatly improves the operation speed. Thereby, the operation speed of the convolutional neural network is improved, and when the convolutional neural network is used for image processing or video processing, the processing result can be outputted faster. For example, when the convolutional neural network is used for image classification, the classification result can be outputted faster; when the convolutional neural network is used for target detection, the detection result can be outputted faster; when the convolutional neural network is used for image restoration, the result can be outputted faster; when the convolutional neural network is used for video classification, the classification result can be outputted faster.

[0128] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0129] Corresponding to the acceleration method of the convolution calculation described in the above embodiment, Fig. 9 A structural block diagram of an acceleration device for convolution calculation provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0130] Reference Fig. 9 , the convolution calculation acceleration device 9 includes:

[0131] The processing module 901 is used to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, where the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices;

[0132] Wherein, in each operation cycle, the device 9 further comprises:

[0133] A storage module 902 is used to store the convolution elements of the current period of at least two convolution kernels into a first storage unit array, and store the characteristic elements of the current period of at least two characteristic matrices into a second storage unit array, wherein if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array, and the column where the convolution element of the current period of each convolution kernel in the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located;

[0134] A control module 903 is used to control each target operation unit to perform parallel operation according to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, so as to obtain the operation results of each target operation unit, wherein the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one;

[0135] The output module 904 is used to obtain the output result of the operation cycle according to the operation results of each target operation unit.

[0136] Optionally, the preset instruction is used to instruct to perform grouped convolution operations on a convolution kernel set and a preset feature matrix, at least two feature matrices are obtained by segmenting the preset feature matrix according to the channels of the preset feature matrix, the convolution kernel set includes at least two convolution kernels, and each group corresponding to the grouped convolution operation includes a feature matrix and at least one convolution kernel.

[0137] Optionally, a ratio of the number of rows in the first storage unit array to the number of channels of a convolution kernel is greater than 2.

[0138] Optionally, in each computation cycle:

[0139] The device 9 also includes:

[0140] The clearing module is used to clear the data in the first storage unit array, the second storage unit array and the computing unit array.

[0141] Optionally, the control module 903 includes:

[0142] A sending unit, used to send an instruction signal to the operation unit array, so that each operation unit in the operation unit array performs parallel operation according to the instruction signal to obtain a matrix output result of the operation unit array;

[0143] A screening unit is used to screen out valid data from the matrix output result, where the valid data is the operation result obtained by the target operation unit according to the corresponding convolution element of the current period and the characteristic element of the current period;

[0144] The output module 904 is used to:

[0145] Based on the valid data, the output result of the operation cycle is obtained.

[0146] Optionally, if there are any two convolution kernels, and the feature matrices corresponding to the convolution operations of the two convolution kernels are the same, then the convolution elements of this period corresponding to the two convolution kernels are located in the same column and in different rows in the first storage unit array.

[0147] Optionally, the weight value in the operation rule of the target operation unit is obtained according to the weight value of the corresponding convolution kernel and / or the weight value of the corresponding feature matrix.

[0148] In an embodiment of the present application, after detecting a preset instruction, a target operation result corresponding to the preset instruction can be obtained based on an output result corresponding to at least one operation cycle, wherein in each operation cycle, the convolution elements of at least two convolution kernels in this cycle are stored in a first storage unit array, and the feature elements of at least two feature matrices in this cycle are stored in a second storage unit array, so that the data of different convolution kernels and different feature matrices in this operation cycle can be conveniently stored through the first storage unit array and the second storage unit array, and can be transmitted to the operation unit array in parallel.

[0149] Then, according to the operation rules of the target operation units corresponding to the convolution elements of this cycle and the characteristic elements of this cycle in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, and then the output result of the operation cycle is obtained according to the operation results of each target operation unit. Among them, if there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operation of the two convolution kernels are different, then the convolution elements of this cycle corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array, and the columns where the convolution elements of this cycle of each convolution kernel in the two convolution kernels are located correspond to the rows where the corresponding characteristic elements of this cycle are located, the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one. It can be seen that through the embodiments of the present application, hardware resources can be reasonably allocated, and different target operation units can be used to realize the parallel operation of multiple dimensions such as the convolution kernels corresponding to different characteristic elements and the different convolution elements of the convolution kernel in this cycle, which greatly improves the operation speed.

[0150] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0151] Fig.10 Schematic diagram of a terminal device provided by an embodiment of the present application. Fig.10 As shown, the terminal device 10 of this embodiment includes: a processor 100, a memory 101, and a computer program 102 stored in the memory 101 and executable on the processor 100. When the processor 100 executes the computer program 102, the steps in the above-mentioned acceleration method embodiments of each convolution calculation are implemented, for example Figure 1 Alternatively, when the processor 100 executes the computer program 102, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Fig. 9 The functions of units 901 to 903 are shown.

[0152] Exemplarily, the computer program 102 may be divided into one or more modules / units, one or more modules / units are stored in the memory 101, and are executed by the processor 100 to complete the embodiment of the present application. One or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 102 in the terminal device 10. For example, the computer program 102 may be divided into a processing module, a storage module, a control module, and an output module, and the specific functions of each module are as follows:

[0153] A processing module, configured to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices;

[0154] Among them, in each operation cycle:

[0155] A storage module, used for storing the convolution elements of the current period of at least two convolution kernels into a first storage unit array, and storing the characteristic elements of the current period of at least two characteristic matrices into a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the convolution element of the current period of each convolution kernel in the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located;

[0156] A control module, used to control each target operation unit to perform parallel operation according to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, so as to obtain the operation results of each target operation unit, wherein the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one;

[0157] The output module is used to obtain the output result of the operation cycle according to the operation results of each target operation unit.

[0158] The terminal device 10 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will appreciate that Fig.10 It is only an example of the terminal device 10 and does not constitute a limitation of the terminal device 10. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0159] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0160] The memory 101 may be an internal storage unit of the terminal device 10, such as a hard disk or memory of the terminal device 10. The memory 101 may also be an external storage device of the terminal device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 10. Further, the memory 101 may also include both an internal storage unit of the terminal device 10 and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal device. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0161] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0162] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0163] The embodiment of the present application provides a circuit system 11, including a first storage cell array 110, a second storage cell array 111, a computing unit array 112, and a control unit 113, wherein the rows in the computing unit array 112 correspond to the rows in the first storage cell array 110, and the columns in the computing unit array 112 correspond to the columns in the second storage cell array 111.

[0164] A control unit 113 is used to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices;

[0165] Among them, in each operation cycle:

[0166] The first storage unit array 110 is used to store the convolution elements of the current period of at least two convolution kernels. If there are any two convolution kernels, the feature matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array 110:

[0167] The second storage unit array 111 is used to store the characteristic elements of the current period of at least two characteristic matrices. If there are any two convolution kernels, and the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the characteristic elements of the current period corresponding to the characteristic matrices corresponding to the two convolution kernels are located in different rows in the second storage unit array 111;

[0168] Each target operation unit in the operation unit array 112 is used to perform parallel operations according to the corresponding convolution elements of the current period, the characteristic elements of the current period and the operation rules, so as to obtain the operation results of each target operation unit;

[0169] The control unit 113 is used to obtain the output result of the operation cycle according to the operation results of each target operation unit.

[0170] The related descriptions of the first storage cell array 110 , the second storage cell array 111 , the operation unit array 112 and the control unit 113 in the circuit system 11 may refer to the above method embodiments.

[0171] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0173] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0174] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for accelerating convolution calculation, characterized in that: include: After detecting a preset instruction, obtaining a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle, wherein the preset instruction is used to instruct at least two convolution kernels to perform a convolution operation with at least two feature matrices; Among them, in each operation cycle: The convolution elements of the current period of at least two of the convolution kernels are stored in a first storage unit array, and the characteristic elements of the current period of at least two of the characteristic matrices are stored in a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operation of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the convolution element of the current period of each convolution kernel in the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located; wherein, the first storage unit array includes a plurality of first storage units arranged regularly, and the second storage unit array includes a plurality of second storage units arranged regularly; According to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, wherein the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one; The output result of the operation cycle is obtained according to the operation results of each of the target operation units.

2. The acceleration method according to claim 1, characterized in that: The preset instruction is used to instruct to perform a grouped convolution operation on a convolution kernel set and a preset feature matrix, the at least two feature matrices are obtained by segmenting the preset feature matrix according to the channels of the preset feature matrix, the convolution kernel set includes the at least two convolution kernels, and each group corresponding to the grouped convolution operation includes one of the feature matrices and at least one of the convolution kernels.

3. The acceleration method according to claim 1, characterized in that: The ratio of the number of rows in the first storage unit array to the number of channels of one of the convolution kernels is greater than 2.

4. The acceleration method according to claim 1, characterized in that: According to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, each target operation unit is controlled to perform parallel operation to obtain the operation results of each target operation unit, including: Sending an instruction signal to the operation unit array, so that each operation unit in the operation unit array performs parallel operation according to the instruction signal to obtain a matrix output result of the operation unit array; Filter out valid data from the matrix output result, where the valid data is a calculation result obtained by the target operation unit according to the corresponding convolution element of the current period and the characteristic element of the current period; The step of obtaining the output result of the operation cycle according to the operation results of each of the target operation units includes: An output result of the operation cycle is obtained according to the valid data.

5. The acceleration method according to claim 1, characterized in that: If there are any two of the convolution kernels, and the feature matrices corresponding to the convolution operations of the two convolution kernels are the same, then the convolution elements of this period corresponding to the two convolution kernels are located in the same column and in different rows in the first storage unit array.

6. The acceleration method according to any one of claims 1 to 5, characterized in that: The weight value in the operation rule of the target operation unit is obtained according to the weight value of the corresponding convolution kernel and / or the weight value of the corresponding feature matrix.

7. A convolution calculation acceleration device, characterized in that: include: A processing module, configured to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform a convolution operation with at least two feature matrices; Wherein, in each operation cycle, the device includes: A storage module, used for storing the convolution elements of the current period of at least two of the convolution kernels into a first storage unit array, and storing the characteristic elements of the current period of at least two of the characteristic matrices into a second storage unit array, wherein, if there are any two convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and different rows in the first storage unit array, and the column where the convolution element of the current period of each of the two convolution kernels is located corresponds to the row where the corresponding characteristic element of the current period is located; wherein, the first storage unit array includes a plurality of first storage units arranged regularly, and the second storage unit array includes a plurality of second storage units arranged regularly; A control module, used for controlling each target operation unit to perform parallel operation according to the operation rules of the target operation units corresponding to the convolution elements of the current period and the characteristic elements of the current period in the operation unit array, so as to obtain the operation results of each target operation unit, wherein the rows in the operation unit array correspond to the rows in the first storage unit array one-to-one, and the columns in the operation unit array correspond to the columns in the second storage unit array one-to-one; The output module is used to obtain the output result of the operation cycle according to the operation results of each of the target operation units.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the acceleration method for convolution calculation as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the acceleration method for convolution calculation as described in any one of claims 1 to 6 is implemented.

10. A circuit system, characterized in that: The method comprises a first storage cell array, a second storage cell array, a computing cell array and a control unit, wherein the rows in the computing cell array correspond to the rows in the first storage cell array one-to-one, and the columns in the computing cell array correspond to the columns in the second storage cell array one-to-one; The control unit is used to obtain a target operation result corresponding to the preset instruction according to an output result corresponding to at least one operation cycle after detecting the preset instruction, wherein the preset instruction is used to instruct at least two convolution kernels to perform convolution operation with at least two feature matrices; Among them, in each operation cycle: The first storage unit array is used to store the convolution elements of the current period of at least two of the convolution kernels. If there are any two of the convolution kernels, the feature matrices corresponding to the convolution operations of the two convolution kernels are different, then the convolution elements of the current period corresponding to the two convolution kernels are respectively located in different columns and rows in the first storage unit array: wherein the first storage unit array includes a plurality of first storage units arranged regularly; The second storage unit array is used to store the characteristic elements of the current period of at least two of the characteristic matrices. If there are any two of the convolution kernels, the characteristic matrices corresponding to the convolution operations of the two convolution kernels are different, then the characteristic elements of the current period corresponding to the characteristic matrices corresponding to the two convolution kernels are located in different rows in the second storage unit array; wherein the second storage unit array includes a plurality of regularly arranged second storage units; Each target operation unit in the operation unit array is used to perform parallel operation according to the corresponding convolution element of the current period, the characteristic element of the current period and the operation rule, so as to obtain the operation result of each target operation unit; The control unit is used to obtain the output result of the operation cycle according to the operation results of each of the target operation units.

Images