Computing device, chip, deconvolution operation method, electronic device, and storage medium

Abstract

The present disclosure relates to an operation device, a chip, a deconvolution operation method, an electronic device and a storage medium. The device comprises: a control unit configured to obtain original parameters of a deconvolution operation, determine target parameters of an equivalent convolution operation corresponding to the deconvolution operation according to the original parameters of the deconvolution operation, send the target parameters to a convolution operation unit, and determine a result of the equivalent convolution operation as a result of the deconvolution operation; and the convolution operation unit configured to perform the equivalent convolution operation according to the target parameters, and send the result of the equivalent convolution operation to the control unit.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a computing device, a chip, a deconvolution operation method, an electronic device, and a storage medium. Background Technology

[0002] In recent years, artificial intelligence (AI) technology has made rapid progress, and its applications in data recognition and processing have become increasingly widespread. Fundamentally, AI technology is built upon basic operations such as convolution, deconvolution, pooling, and sampling; therefore, the processing power of these basic operations determines the operational capabilities of AI technology.

[0003] Taking deconvolution as an example, related technologies can only implement deconvolution operations on general-purpose chips that support matrix operations and random access memory, such as GPUs (graphics processing units) and CPUs (central processing units). It cannot be executed on other chips, especially not on acceleration chips dedicated to artificial intelligence technology. This not only greatly reduces the efficiency of deconvolution operations, but also limits the range of hardware options for deconvolution operations. Summary of the Invention

[0004] This disclosure provides a computing device, a chip, a deconvolution operation method, an electronic device, and a storage medium to address the deficiencies in related technologies.

[0005] According to a first aspect of the present disclosure, a computing device is provided, comprising:

[0006] The control unit is used to acquire the original parameters of the deconvolution operation, determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, send the target parameters to the convolution operation unit, and determine the result of the equivalent convolution operation as the result of the deconvolution operation.

[0007] The convolution operation unit is used to perform the equivalent convolution operation according to the target parameters and send the result of the equivalent convolution operation to the control unit.

[0008] In one embodiment, the original parameters include at least one of the following:

[0009] The original input data, the original convolution kernel, the original convolution stride, the original padding values, and the size of the output data.

[0010] In one embodiment, the target parameter includes target input data;

[0011] When the control unit determines the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: determine the size of the target input data of the equivalent convolution operation and the position of each original element in the original input data in the target input data based on the size of the original input data, the size of the original convolution kernel, the original convolution stride, the original padding value, and the size of the output data; and add each original element in the original input data to the corresponding position of the target input data based on the position of each original element in the original input data in the target input data, and add preset values ​​to other positions in the target input data to obtain the target input data.

[0012] In one embodiment, the chip further includes a vector processing unit and a storage unit;

[0013] The control unit is configured to add each original element in the original input data to the corresponding position in the target input data according to the position of each original element in the original input data in the target input data, and add preset values ​​to other positions in the target input data to obtain the target input data. Specifically, it is configured to: obtain a target data space of a corresponding size in the storage unit according to the size of the target input data, and initialize the target element at each position in the target data space to the preset value; determine the position of each original element in the target data space according to the position of each original element in the original input data in the target input data; and send the address of the original input data and the position of each original element in the target data space to the vector operation unit.

[0014] The storage unit is used to: store the target input data;

[0015] The vector operation unit is used to: obtain each original element in the original input data according to the address of the original input data, and use each original element to update the target element at the corresponding position in the target data space.

[0016] In one embodiment, when the control unit sends the address of the original input data and the position of each original element in the target data space to the vector processing unit, it is specifically configured to: determine the configuration information of the vector processing unit based on the address of the original input data, the address of the target data space, and the position of each original element in the target data space, and send the configuration information to the vector processing unit, wherein the configuration information of the vector processing unit includes at least one of the following: the starting address of the input data, the operation step size, and the size value of each data dimension, and the starting address of the output data, the operation step size, the size value of each data dimension, and the loop value of each data dimension;

[0017] The vector operation unit is used to obtain each original element in the original input data according to the address of the original input data, and to update the target element at the corresponding position in the target data space using each original element. Specifically, it is used to update the target element at the position of each original element in the target data space according to the configuration information to the sum of the corresponding original element and the preset value.

[0018] In one embodiment, the target parameters include a target convolutional kernel;

[0019] When the control unit is used to determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: rotate the data in the original convolution kernel in a centrally symmetric manner to obtain the target convolution kernel of the equivalent convolution operation.

[0020] In one embodiment, the original convolutional kernel includes multiple channels;

[0021] When the control unit determines the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: perform a center-symmetric rotation on the data of each channel in the original convolution kernel to obtain the data of each channel of the target convolution kernel.

[0022] In one embodiment, the target parameters include the target convolution stride and / or the target padding value;

[0023] When the control unit is used to determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: determine the target convolution stride of the equivalent convolution operation as a preset stride; and / or, determine the target padding value of the equivalent convolution operation as a preset padding value.

[0024] In one embodiment, when the control unit sends the target parameters to the convolution operation unit, it is specifically configured to: configure the starting address of the target input data and the size value of each data dimension, the starting address of the target convolution kernel and the size value of each data dimension, the target convolution stride, and the target padding value into the register corresponding to the convolution operation unit;

[0025] When the vector operation unit performs the equivalent convolution operation based on the target parameters, it is specifically used to: perform the equivalent convolution operation using the configuration information in the register to obtain the result of the equivalent convolution operation.

[0026] According to a second aspect of the present disclosure, a chip is provided, including the computing device described in the first aspect.

[0027] According to a third aspect of the present disclosure, a deconvolution operation method is provided, comprising:

[0028] Obtain the original parameters for the deconvolution operation;

[0029] Based on the original parameters of the deconvolution operation, determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation;

[0030] Using a convolution operation unit, the equivalent convolution operation is performed based on the target parameters, and the result of the equivalent convolution operation is determined as the result of the deconvolution operation.

[0031] In one embodiment, the original parameters include at least one of the following:

[0032] The original input data, the original convolution kernel, the original convolution stride, the original padding values, and the size of the output data.

[0033] In one embodiment, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0034] Based on the size of the original input data, the size of the original convolution kernel, the original convolution stride, the original padding value, and the size of the output data, determine the size of the target input data for the equivalent convolution operation, and the position of each original element in the original input data in the target input data;

[0035] Based on the position of each original element in the original input data in the target input data, each original element in the original input data is added to the corresponding position in the target input data, and preset values ​​are added to other positions in the target input data to obtain the target input data.

[0036] In one embodiment, the step of adding each original element in the original input data to the corresponding position in the target input data based on the position of the original elements in the original input data in the target input data, and adding preset values ​​to other positions in the target input data to obtain the target input data, includes:

[0037] Based on the size of the target input data, obtain a target data space of the corresponding size on the shared memory, and initialize the target element at each position in the target data space to the preset value;

[0038] Based on the position of each original element in the original input data in the target input data, determine the position of each original element in the target data space;

[0039] Using a vector operation unit, each original element in the original input data is obtained according to the address of the original input data, and each original element is used to update the target element at the corresponding position in the target data space.

[0040] In one embodiment, the step of using a vector operation unit to obtain each original element in the original input data according to the address of the original input data, and using each original element to update the target element at the corresponding position in the target data space, includes:

[0041] Based on the address of the original input data, the address of the target data space, and the position of each original element in the target data space, the configuration information of the vector operation unit is determined, wherein the configuration information of the vector operation unit includes at least one of the following: the starting address of the input data, the operation step size, and the size value of each data dimension, and the starting address of the output data, the operation step size, the size value of each data dimension, and the loop value of each data dimension;

[0042] According to the configuration information, the vector operation unit updates the target element at the position of each original element in the target data space to the sum of the corresponding original element and the preset value.

[0043] In one embodiment, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0044] The data in the original convolution kernel is rotated symmetrically to obtain the target convolution kernel for the equivalent convolution operation.

[0045] In one embodiment, the original convolutional kernel includes multiple channels;

[0046] The step of performing a centrosymmetric rotation on the data in the original convolution kernel to obtain the target convolution kernel for the equivalent convolution operation includes:

[0047] The data of each channel in the original convolution kernel is rotated symmetrically to obtain the data of each channel in the target convolution kernel.

[0048] In one embodiment, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0049] The target convolution stride of the equivalent convolution operation is determined to be a preset stride; and / or,

[0050] The target padding value of the equivalent convolution operation is determined as a preset padding value.

[0051] In one embodiment, the step of using a convolution operation unit to perform the equivalent convolution operation based on the target parameters, and determining the result of the equivalent convolution operation as the result of the deconvolution operation, includes:

[0052] The starting address of the target input data and the size value of each data dimension, the starting address of the target convolution kernel and the size value of each data dimension, the target convolution stride, and the target padding value are configured into the register corresponding to the convolution operation unit. The convolution operation unit uses the configuration information in the register to perform an equivalent convolution operation to obtain the result of the equivalent convolution operation.

[0053] According to a fourth aspect of the present disclosure, an electronic device is provided, the device including a memory and a processor, the memory being configured to store computer instructions executable on the processor, the processor being configured to implement the method of the third aspect when executing the computer instructions; or...

[0054] The device includes the chip described in the second aspect.

[0055] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0056] As can be seen from the above embodiments, by setting a control unit, the original parameters of the deconvolution operation can be obtained. Based on the original parameters of the deconvolution operation, the target parameters of the equivalent convolution operation corresponding to the deconvolution operation are determined, and the target parameters are sent to the convolution operation unit. The result of the equivalent convolution operation is also determined as the result of the deconvolution operation. By setting a convolution operation unit, the equivalent convolution operation can be performed according to the target parameters, and the result of the equivalent convolution operation is sent to the control unit. This computing device uses the control unit to convert the deconvolution operation into the corresponding equivalent convolution operation, and then uses the convolution operation unit to execute the equivalent convolution operation. This allows convolution operation units that do not support deconvolution operations to calculate the result of the deconvolution operation. This improves the efficiency of the deconvolution operation and expands the hardware options for deconvolution operations. For example, an AI-specific acceleration chip can be used to perform the deconvolution operation, fully utilizing the parallel computing capabilities of the acceleration chip and expanding the types of operators supported by the acceleration chip. This allows deconvolution operations to be performed even on acceleration chips that do not support random memory access.

[0057] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0059] Figure 1 This is a schematic diagram of the structure of a computing unit shown in an exemplary embodiment of this disclosure;

[0060] Figure 2 This is a schematic diagram illustrating a centrally symmetric rotation of a deconvolution kernel according to an exemplary embodiment of this disclosure;

[0061] Figure 3 This is a schematic diagram illustrating the convolution process of target input data according to an exemplary embodiment of this disclosure;

[0062] Figure 4 This is a flowchart illustrating a deconvolution operation method according to an exemplary embodiment of this disclosure;

[0063] Figure 5 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this disclosure. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0065] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0066] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0067] In related technologies, deep learning frameworks (such as Caffe) used to perform deconvolution operations generally call the backpropagation of convolution to achieve deconvolution, which means that the deconvolution operation is split into two steps: matrix multiplication and processing by the col2im function. Therefore, hardware support for matrix multiplication and random access memory capability are required. However, acceleration chips dedicated to artificial intelligence technologies such as AI accelerators lack matrix operation units and do not have random access memory capability, which makes it impossible for acceleration chips dedicated to artificial intelligence technologies to perform deconvolution operations.

[0068] Firstly, at least one embodiment of this disclosure provides a computing device, please refer to the appendix. Figure 1 The system includes: a control unit 101, configured to acquire the original parameters of the deconvolution operation, determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, send the target parameters to the convolution operation unit, and determine the result of the equivalent convolution operation as the result of the deconvolution operation; and a convolution operation unit (MPU) 102, configured to perform the equivalent convolution operation based on the target parameters and send the result of the equivalent convolution operation to the control unit.

[0069] The computing device can be an acceleration chip specifically designed for artificial intelligence, such as an AI accelerator, or it can be a partial structure of the aforementioned chip.

[0070] The original parameters may include at least one of the following: original input data, original convolution kernel, original convolution stride, original padding values, and the size of the output data. The original input data can be the input feature map (input tensor) for the deconvolution operation. The original input data may include the starting address of the original input data, the size value of each data dimension, etc. The address can be the storage address of the original input data in the storage unit. The data dimensions may include the number (n), channels (c), width (w), and height (h), etc. The original input data can be read based on its starting address and the size value of each data dimension. For example, if the starting address of the original input data is address A, and the dimensions of the number, channels, width, and height are 2, 2, 3, and 3 respectively, then the first three data points starting from address A are the first channel of the first feature map. The first row of data, starting from address A, consists of the 4th-6th data points representing the first channel of the first feature map; the 7th-9th data points represent the third row of the first channel of the first feature map; the 10th-12th data points represent the first row of the second channel of the first feature map; the 13th-15th data points represent the second row of the second channel of the first feature map; the 16th-18th data points represent the third row of the second channel of the first feature map; the 19th-21st data points represent the first row of the first channel of the second feature map; and so on. The original convolutional kernel can include its starting address, the size of each data dimension, etc. The address can be the storage address of the original convolutional kernel in the memory unit, and the dimensions can include channels (c), width (w), and height (h). The size of the output data can include the size of each data dimension, such as the number of output data (n), channels (c), width (w), and height (h).

[0071] The raw parameters for deconvolution can be input by the user or generated by the upstream computation process.

[0072] The equivalent convolution operation, corresponding to deconvolution, refers to a convolution operation that yields the same result as deconvolution. The target parameters of the equivalent convolution operation can include the target input data, the target convolution kernel, the target convolution stride, and the target padding value.

[0073] The target input data changes with the size of the original input and output data, therefore generating the target input data requires online execution. However, the weights in the original convolutional kernel of the deconvolution operation are fixed parameters of the deconvolution layer in the neural network, so they can be processed offline. This means that a dedicated function is developed in the chip's compiler (toolchain) to convert the original convolutional kernel into the target convolutional kernel. Then, during neural network compilation, the chip's compiler (toolchain) converts the original convolutional kernel (i.e., the old weight tensor) to generate and save the target convolutional kernel (i.e., the new weight tensor), making the target convolutional kernel the parameters of the neural network (i.e., the weights used in hardware convolution operations). Therefore, during deconvolution operations, there is no need for online conversion of the original convolutional kernel; the saved target convolutional kernel can be used directly.

[0074] In one embodiment, the control unit can determine the target input data based on the original input data, original convolution stride, original padding value, and output data size from the original parameters. Furthermore, the computing device may include a storage unit, in which the control unit can generate the target input data and determine its starting address and size values ​​in each dimension.

[0075] In another embodiment, the control unit can generate a target convolutional kernel based on the original convolutional kernel. For example, based on the relationship between convolution operations and their corresponding equivalent convolution operations, the control unit can perform a centrosymmetric rotation on the data in the original convolutional kernel to obtain the target convolutional kernel of the equivalent convolution operation. It is understood that when the original convolutional kernel includes multiple channels, the data in each channel of the original convolutional kernel can be centrosymmetrically rotated separately to obtain the data for each channel of the target convolutional kernel. Please refer to the appendix. Figure 2 The data can be rotated 180° to achieve a centrally symmetrical rotation.

[0076] In yet another embodiment, the control unit may determine the target convolution stride of the equivalent convolution operation as a preset stride. For example, the preset stride may be 1.

[0077] In yet another embodiment, the control unit may determine the target padding value of the equivalent convolution operation as a preset padding value. For example, the preset padding value may be 0.

[0078] The convolution operation unit has corresponding registers. By configuring the registers with data and parameters, the convolution operation unit can read data from the storage unit and perform convolution operations according to the configuration information in the registers. The registers can be configured through their interface. Based on this, when the controller sends the target parameters to the convolution operation unit, it can configure the starting address and size values ​​of each data dimension of the target input data, the starting address and size values ​​of each data dimension of the target convolution kernel, the target convolution stride, and the target padding value into the registers corresponding to the convolution operation unit. When the convolution operation unit performs the equivalent convolution operation according to the target parameters, it can use the configuration information in the registers to perform the equivalent convolution operation and obtain the result of the equivalent convolution operation.

[0079] The convolution operation unit can read the target input data from memory (e.g., storage unit) according to the starting address of the target input data and the size value of each data dimension, and can read the target convolution kernel from the parameters of the neural network according to the starting address of the target convolution kernel and the size value of each data dimension. At the same time, it can use the target convolution kernel to convolve the target input data according to the target convolution stride and the target padding value, thereby obtaining the result of the convolution operation.

[0080] As can be seen from the above embodiments, by setting a control unit, the original parameters of the deconvolution operation can be obtained. Based on the original parameters of the deconvolution operation, the target parameters of the equivalent convolution operation corresponding to the deconvolution operation are determined, and the target parameters are sent to the convolution operation unit. The result of the equivalent convolution operation is also determined as the result of the deconvolution operation. By setting a convolution operation unit, the equivalent convolution operation can be performed according to the target parameters, and the result of the equivalent convolution operation is sent to the control unit. This computing device uses the control unit to convert the deconvolution operation into the corresponding equivalent convolution operation, and then uses the convolution operation unit to execute the equivalent convolution operation. This allows convolution operation units that do not support deconvolution operations to calculate the result of the deconvolution operation. This improves the efficiency of the deconvolution operation and expands the hardware options for deconvolution operations. For example, an AI-specific acceleration chip can be used to perform the deconvolution operation, fully utilizing the parallel computing capabilities of the acceleration chip and expanding the types of operators supported by the acceleration chip. This allows deconvolution operations to be performed even on acceleration chips that do not support random memory access. In some embodiments of this disclosure, when the control unit is used to determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: determine the size of the target input data of the equivalent convolution operation and the position of each original element in the original input data in the target input data based on the size of the original input data, the size of the original convolution kernel, the original convolution stride, the original padding value, and the size of the output data; and add each original element in the original input data to the corresponding position of the target input data based on the position of each original element in the original input data in the target input data, and add preset values ​​to other positions in the target input data to obtain the target input data.

[0081] Next, we will briefly introduce the size relationship between the deconvolution operation and its corresponding equivalent convolution operation.

[0082] From the forward perspective of the convolution operation, assuming the input data size is i, and the kernel size, stride, and padding value are k, s, and p respectively, the formula for calculating the size of the output data o of the convolution operation is as follows:

[0083]

[0084] Assuming the input data size for deconvolution is o, and the kernel size, stride, and padding are k, s, and p respectively, the formula for calculating the size of the output data o' is as follows:

[0085] o′=(o-1)*s+k-2p+adj

[0086] adj = (i + 2p - k) % s

[0087] Based on the above size relationship, the target input data can be obtained in the following way: First, insert s-1 preset values ​​(e.g., 0) between adjacent original elements of the original input data. Then, fill the edges in all directions with p' = kp-1 preset values. Finally, fill the right and bottom sides with adj preset values, where s is the original convolution stride, k is the size of each channel of the original convolution kernel, p is the original padding value, and adj = (o + 2p - k) % s, where o is the size of each channel of the output data. For example, if the size of the original input data for deconvolution is 3*3, the original convolution kernel size k, the original convolution stride s, and the original padding value p are 3, 2, and 1 respectively, and the size of the output data for deconvolution is 6*6, then adj = (6 + 2*1 - 3) % 2 = 1. Figure 4 The convolution process of the target input data obtained under the above parameters is given, where the data below is the target input data (i.e., bottom tensor) and the data above is the output data of the deconvolution operation (i.e., top tensor).

[0088] Furthermore, the size of the target input data can be calculated using the following formula: If the size of the original input data for the deconvolution operation in each dimension is n*c*h*w, then the size of the target input data in each dimension is n*c*h_itm*w_itm. Let m be the number of rows and columns of inserted zeros (m = s-1). The number of preset values ​​for padding the top, bottom, left, and right sides are pad_u, pad_d, pad_l, and pad_r, respectively. Then w_itm = w + pad_l + pad_r + (w-1)*m, h_itm = h + pad_u + pad_d + (h-1)*m.

[0089] Furthermore, the position of each original element in the target input data can be determined based on the above filling process.

[0090] The computing device further includes a storage unit and a vector processing unit (VPU). The storage unit is used to store the target input data. The control unit, based on the position of each original element in the original input data within the target input data, adds each original element from the original input data to the corresponding position in the target input data, and adds preset values ​​to other positions in the target input data. When the target input data is obtained, it can:

[0091] First, based on the size of the target input data, a target data space of the corresponding size is obtained on the storage unit, and the target element at each position in the target data space is initialized to the preset value; for example, a bottom tensor of the corresponding size can be requested on the shared memory of the AI ​​chip, and the FillZero operator developed on the AI ​​chip can be used to fill the bottom tensor with 0.

[0092] Next, based on the position of each original element in the original input data within the target input data, the position of each original element in the target data space is determined. For example, the relative positional relationship between each original element and the first data point of the target input data can be converted into the relative positional relationship between each original element and the first position in the target data space, thus serving as the position of the original element in the target data space.

[0093] Finally, the address of the original input data and the position of each original element in the target data space are sent to the vector processing unit. For example, the configuration information of the vector processing unit can be determined based on the address of the original input data, the address of the target data space, and the position of each original element in the target data space, and the configuration information is sent to the vector processing unit. The configuration information of the vector processing unit includes at least one of the following: the starting address of the input data, the operation step size, and the size value of each data dimension; and the starting address of the output data, the operation step size, the size value of each data dimension, and the loop value of each data dimension.

[0094] Based on this, the vector processing unit can obtain each original element in the original input data according to the address of the original input data, and use each original element to update the target element at the corresponding position in the target data space. For example, according to the configuration information, the vector processing unit updates the target element at the position of each original element in the target data space to the sum of the corresponding original element and the preset value.

[0095] Please refer to the appendix. Figure 3 The target input data shown (i.e., the data below) corresponds to the original input data, which is a series of 3*3 dark boxes. The only difference between the two is the address interval of each dimension, but the loop value size of each dimension (i.e., the loop value size of the width dimension is 3 and the loop value size of the channel dimension is 9) remains unchanged.

[0096] The operations in the example can be accomplished using the Move operator implemented on the AI ​​chip. The Move operator copies data in the chip's SRAM (Static Random-Access Memory), moving data from the location of the source tensor (src tensor) to the storage space corresponding to the target tensor (dst tensor), supporting non-contiguous access and storage of data. The vector operation unit can determine the address of each original element based on the starting address of the input data and the size value of each data dimension. Since each original element needs to be read and added to the target data space, the operation step size of the input data can be set to 1. Then, the vector operation unit can sequentially read each original element based on the starting address of the input data and the size value of each data dimension. The loop value of a certain data dimension of the output data refers to the number of data points output in that dimension. Figure 3 Taking the target input data as an example, its width dimension has a cycle value of 3, meaning that after outputting 3 consecutive data points in a row, it can switch to the next row and start the next output from the first data point of the next row. Its channel dimension has a cycle value of 9, meaning that after outputting 9 consecutive data points in a channel, it can switch to the first data point of the first row of the next channel and start the next output. The output data's step size refers to the distance between adjacent data points in each dimension within the channel. Figure 4 Taking the target input data as an example, with a step size of 2, the distance between two consecutive data points in a row within the width dimension is 2, and the distance between two adjacent rows containing output data within the height dimension is 2. The dimension value of a certain data dimension of the output data refers to the number of data positions within that dimension. The dimension value can be used to switch data dimensions. Figure 4 Taking the target input data as an example, if the width dimension has a value of 8, then when switching rows, the address of the first row can be increased by 8 to obtain the address of the next row; similarly, when switching channels, the address of the first channel can be increased by 64 to obtain the address of the next channel. Therefore, the vector operation unit can sequentially determine each output position in the target input data based on the output data's address, operation step size, loop value of each data dimension, and size value of each data dimension. In summary, the vector operation unit can sequentially read each piece of raw data, and after each reading, add the raw data to a preset value (e.g., 0), and output the sum to the corresponding output position to update the preset value at that position. After all the raw data has been read, processed, and output, the data in the target data space on the storage unit is the target output data.

[0097] In this embodiment, the size of the target output data and the position of each original element in the target input data are first determined. Then, each original element is added to the corresponding position in the target input data as the target element, and the target elements in other positions of the target input data are set to preset values ​​(e.g., 0), thus obtaining the target input data. This method is simple and accurate. In particular, by using a vector operation unit to sequentially read each original element, perform calculations, and output the calculation results to a target data space that has been pre-allocated and initialized with preset target elements, the method is simple and convenient. It also leverages the characteristics and advantages of the vector operation unit, improving computational efficiency and accuracy.

[0098] The deconvolution operation method disclosed herein implements deconvolution operations on AI chips and other AI technology-specific acceleration chips using convolution operations. It does not rely on general matrix multiplication and random memory read / write functions, and is not limited by deconvolution parameters. It can support larger-scale deconvolution operations through toolchain partitioning. Furthermore, the main computations during execution leverage the parallel computing capabilities of the AI ​​chip, resulting in high computational efficiency. Based on this method, deconvolution operators can be built into the compiler corresponding to the AI ​​chip, thereby enabling the AI ​​chip to possess deconvolution functionality.

[0099] Secondly, at least one embodiment of this disclosure provides a chip including the computing device described in the first aspect.

[0100] Thirdly, at least one embodiment of this disclosure provides a deconvolution operation method, please refer to the appendix. Figure 4 It illustrates the process of the method, including steps S401 to S403.

[0101] The method can be applied to the computing device described in the first aspect, the chip described in the second aspect, or other hardware structures, such as acceleration chips (AI accelerators) for artificial intelligence.

[0102] In step S401, the original parameters of the deconvolution operation are obtained.

[0103] In step S402, the target parameters of the equivalent convolution operation corresponding to the deconvolution operation are determined based on the original parameters of the deconvolution operation.

[0104] In step S403, the convolution operation unit is used to perform the equivalent convolution operation according to the target parameters, and the result of the equivalent convolution operation is determined as the result of the deconvolution operation.

[0105] In some embodiments of this disclosure, the original parameters include at least one of the following:

[0106] The original input data, the original convolution kernel, the original convolution stride, the original padding values, and the size of the output data.

[0107] In some embodiments of this disclosure, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0108] Based on the size of the original input data, the size of the original convolution kernel, the original convolution stride, the original padding value, and the size of the output data, determine the size of the target input data for the equivalent convolution operation, and the position of each original element in the original input data in the target input data;

[0109] Based on the position of each original element in the original input data in the target input data, each original element in the original input data is added to the corresponding position in the target input data, and preset values ​​are added to other positions in the target input data to obtain the target input data.

[0110] In some embodiments of this disclosure, the step of adding each original element in the original input data to the corresponding position in the target input data based on the position of the original element in the original input data in the target input data, and adding preset values ​​to other positions in the target input data to obtain the target input data, includes:

[0111] Based on the size of the target input data, obtain a target data space of the corresponding size on the shared memory, and initialize the target element at each position in the target data space to the preset value;

[0112] Based on the position of each original element in the original input data in the target input data, determine the position of each original element in the target data space;

[0113] Using a vector operation unit, each original element in the original input data is obtained according to the address of the original input data, and each original element is used to update the target element at the corresponding position in the target data space.

[0114] In some embodiments of this disclosure, the step of using a vector operation unit to obtain each original element in the original input data according to the address of the original input data, and using each original element to update the target element at the corresponding position in the target data space, includes:

[0115] Based on the address of the original input data, the address of the target data space, and the position of each original element in the target data space, the configuration information of the vector operation unit is determined, wherein the configuration information of the vector operation unit includes at least one of the following: the starting address of the input data, the operation step size, and the size value of each data dimension, and the starting address of the output data, the operation step size, the size value of each data dimension, and the loop value of each data dimension;

[0116] According to the configuration information, the vector operation unit updates the target element at the position of each original element in the target data space to the sum of the corresponding original element and the preset value.

[0117] In some embodiments of this disclosure, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0118] The data in the original convolution kernel is rotated symmetrically to obtain the target convolution kernel for the equivalent convolution operation.

[0119] In some embodiments of this disclosure, the original convolutional kernel includes multiple channels;

[0120] The step of performing a centrosymmetric rotation on the data in the original convolution kernel to obtain the target convolution kernel for the equivalent convolution operation includes:

[0121] The data of each channel in the original convolution kernel is rotated symmetrically to obtain the data of each channel in the target convolution kernel.

[0122] In some embodiments of this disclosure, determining the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation includes:

[0123] The target convolution stride of the equivalent convolution operation is determined to be a preset stride; and / or,

[0124] The target padding value of the equivalent convolution operation is determined as a preset padding value.

[0125] In some embodiments of this disclosure, the step of using a convolution operation unit to perform the equivalent convolution operation based on the target parameters, and determining the result of the equivalent convolution operation as the result of the deconvolution operation, includes:

[0126] The starting address of the target input data and the size value of each data dimension, the starting address of the target convolution kernel and the size value of each data dimension, the target convolution stride, and the target padding value are configured into the register corresponding to the convolution operation unit. The convolution operation unit uses the configuration information in the register to perform an equivalent convolution operation to obtain the result of the equivalent convolution operation.

[0127] The details of the steps involved in the above methods have been described in detail in the first aspect of the relevant parts of the computing device, and will not be repeated here.

[0128] As can be seen from the above embodiments, by obtaining the original parameters of the deconvolution operation, the target parameters of the equivalent convolution operation corresponding to the deconvolution operation can be determined based on the original parameters of the deconvolution operation. Finally, the equivalent convolution operation can be performed using the convolution operation unit according to the target parameters, and the result of the equivalent convolution operation is determined as the result of the deconvolution operation. This method converts the deconvolution operation into the corresponding equivalent convolution operation, and then uses the convolution operation unit to execute the equivalent convolution operation. This allows convolution operation units that do not support deconvolution operations to calculate the result of the deconvolution operation. This not only improves the efficiency of the deconvolution operation, but also expands the hardware selection range for the deconvolution operation. For example, artificial intelligence-specific acceleration chips can be used to perform the deconvolution operation, giving full play to the parallel computing capabilities of the acceleration chip and expanding the types of operators supported by the acceleration chip, so that deconvolution operations can also be completed on acceleration chips that do not support random memory access capabilities.

[0129] The deconvolution operation method disclosed herein implements deconvolution operations on AI chips and other AI technology-specific acceleration chips using convolution operations. It does not rely on general matrix multiplication and random memory read / write functions, and is not limited by deconvolution parameters. It can support larger-scale deconvolution operations through toolchain partitioning. Furthermore, the main computations during execution leverage the parallel computing capabilities of the AI ​​chip, resulting in high computational efficiency. Based on this method, deconvolution operators can be built into the compiler corresponding to the AI ​​chip, thereby enabling the AI ​​chip to possess deconvolution functionality.

[0130] Fourthly, at least one embodiment of this disclosure provides a device, please refer to the appendix. Figure 5 The diagram illustrates the structure of the device, which includes a memory and a processor. The memory stores computer instructions that can run on the processor, and the processor performs operations based on the method described in any of the third aspects when executing the computer instructions; or, the device includes the chip described in the second aspect.

[0131] Fifthly, at least one embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the third aspects.

[0132] In this disclosure, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0133] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0134] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An arithmetic device, characterized by comprising: The computing device is an AI-specific acceleration chip that does not have matrix operation functions or random access memory capabilities. The computing device includes: The compiler is configured with dedicated functions for compiling neural networks, and when compiling the neural network, it converts the convolution kernels of the deconvolution layers of the neural network into the convolution kernels of the corresponding equivalent convolution layers based on the dedicated functions. The control unit is used to acquire the original parameters of the deconvolution operation, determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, send the target parameters to the convolution operation unit, and determine the result of the equivalent convolution operation as the result of the deconvolution operation. A convolution operation unit is used to perform the equivalent convolution operation according to the target parameters and send the result of the equivalent convolution operation to the control unit; The original parameters include the original convolution kernel, which is the convolution kernel of the deconvolution layer of the neural network. The target parameters include the convolution kernel of the equivalent convolution layer corresponding to the deconvolution layer. When the control unit determines the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: obtain the convolution kernel of the equivalent convolution layer corresponding to the deconvolution layer in the neural network from the compiler. The target parameters include the target convolutional kernel; When the control unit is used to determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: rotate the data in the original convolution kernel in a centrally symmetric manner to obtain the target convolution kernel of the equivalent convolution operation.

2. The operation device according to claim 1, characterized by The original parameters include at least one of the following: The original input data, the original convolution stride, the original padding values, and the size of the output data.

3. The operation device according to claim 2, wherein The target parameters include the target input data; When the control unit determines the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: determine the size of the target input data of the equivalent convolution operation and the position of each original element in the original input data in the target input data based on the size of the original input data, the size of the original convolution kernel, the original convolution stride, the original padding value, and the size of the output data; and add each original element in the original input data to the corresponding position of the target input data based on the position of each original element in the original input data in the target input data, and add preset values ​​to other positions in the target input data to obtain the target input data.

4. The operation device according to claim 3, wherein The chip also includes a vector processing unit and a storage unit; The control unit is configured to add each original element in the original input data to the corresponding position in the target input data according to the position of each original element in the original input data in the target input data, and add preset values ​​to other positions in the target input data to obtain the target input data. Specifically, it is configured to: obtain a target data space of a corresponding size in the storage unit according to the size of the target input data, and initialize the target element at each position in the target data space to the preset value; determine the position of each original element in the target data space according to the position of each original element in the original input data in the target input data; and send the address of the original input data and the position of each original element in the target data space to the vector operation unit. The storage unit is used to: store the target input data; The vector operation unit is used to: obtain each original element in the original input data according to the address of the original input data, and use each original element to update the target element at the corresponding position in the target data space.

5. The operation device according to claim 4, wherein When the control unit sends the address of the original input data and the position of each original element in the target data space to the vector operation unit, it is specifically configured to: determine the configuration information of the vector operation unit based on the address of the original input data, the address of the target data space, and the position of each original element in the target data space, and send the configuration information to the vector operation unit. The configuration information of the vector operation unit includes at least one of the following: the starting address of the input data, the operation step size, and the size value of each data dimension; and the starting address of the output data, the operation step size, the size value of each data dimension, and the loop value of each data dimension. The vector operation unit is used to obtain each original element in the original input data according to the address of the original input data, and to update the target element at the corresponding position in the target data space using each original element. Specifically, it is used to update the target element at the position of each original element in the target data space according to the configuration information to the sum of the corresponding original element and the preset value.

6. The operation device according to claim 2, wherein The original convolutional kernel includes multiple channels; When the control unit determines the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: perform a center-symmetric rotation on the data of each channel in the original convolution kernel to obtain the data of each channel of the target convolution kernel.

7. The operation device according to claim 4, wherein The target parameters include the target convolution stride and / or the target padding value; When the control unit is used to determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation based on the original parameters of the deconvolution operation, it is specifically used to: determine the target convolution stride of the equivalent convolution operation as a preset stride; and / or, determine the target padding value of the equivalent convolution operation as a preset padding value.

8. The operation device according to claim 7, wherein When the control unit sends the target parameters to the convolution operation unit, it is specifically configured to: configure the starting address of the target input data and the size value of each data dimension, the starting address of the target convolution kernel and the size value of each data dimension, the target convolution stride, and the target padding value into the register corresponding to the convolution operation unit; When the vector operation unit performs the equivalent convolution operation based on the target parameters, it is specifically used to: perform the equivalent convolution operation using the configuration information in the register to obtain the result of the equivalent convolution operation.

9. A chip, characterized in that, Includes the computing device as described in any one of claims 1 to 8.

10. A deconvolution operation method, characterized in that, Applied to the computing device according to any one of claims 1 to 8, or the chip according to claim 9, the method comprises: Obtain the original parameters for the deconvolution operation; Based on the original parameters of the deconvolution operation, determine the target parameters of the equivalent convolution operation corresponding to the deconvolution operation; Using a convolution operation unit, the equivalent convolution operation is performed based on the target parameters, and the result of the equivalent convolution operation is determined as the result of the deconvolution operation.

11. An electronic device, characterized in that, The device includes a memory and a processor, the memory storing computer instructions executable on the processor, and the processor implementing the method of claim 10 when executing the computer instructions; or, The device includes the chip as described in claim 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of claim 10.

Images