Computing system for operating convolutional neural network, input data acquisition method of convolutional neural network and medium

Abstract

The invention provides a computing system for operating a convolutional neural network, an input data acquisition method of the convolutional neural network and a medium. The computing system comprises a storage control circuit; the storage control circuit splits the convolutional neural network to obtain a to-be-calculated target convolutional layer, obtains a target parameter corresponding to the target convolutional layer, and generates n data initial addresses and a data calling length corresponding to each data initial address when convolution calculation is performed on the target convolutional layer each time according to the target parameter, according to the n data initial addresses and the data calling length corresponding to each data initial address, reading calculation data during each convolution calculation; on the basis, the storage control circuit is introduced, the data storage address corresponding to multiple convolution calculations is quickly, simply and efficiently obtained through the target parameter, the calculation data reading time of each convolution calculation is shortened by utilizing the storage control circuit, and the calculation data reading efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of circuits, and in particular to a computing system for running a convolutional neural network, a method for acquiring input data of the convolutional neural network, and a medium. Background Art

[0002] Convolutional neural network is an artificial intelligence (AI) algorithm that is very effective in recognizing various input signals, such as images and speech. The main principle of convolution calculation is to use a specific convolution weight parameter matrix to calculate the inner product of the input and convolution weight of a small area of ​​the input signal. After completing one calculation, it moves to the next small area to continue the calculation.

[0003] In each calculation, the data to be convolved needs to be selected from the input signal. Figure 1 As shown in the figure, if the convolution kernel size of the convolutional neural network is 3×3, then 3 adjacent data in the same row of the input signal and 3 adjacent rows of data, a total of 9 data, are selected as calculation data. Since the input signal is stored in the memory, and the input signal is stored in the memory using one or more continuous address memories, the data storage addresses of different rows of data are different, and the data storage addresses of different rows of data are not adjacent. Therefore, when performing a convolution calculation, the corresponding calculation data needs to be read from these non-adjacent data storage addresses in order to successfully perform the convolution calculation.

[0004] However, each time a convolution calculation is performed, in order to read the corresponding calculation data from non-adjacent data storage addresses, it is necessary to repeatedly traverse all data storage addresses to determine the data storage addresses corresponding to these calculation data, resulting in a long calculation data reading time for each convolution calculation, low efficiency, and more consumption of computing resources of the computing chip used for the convolution calculation. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a computing system for running a convolutional neural network, a method and medium for acquiring input data of a convolutional neural network, which can shorten the computing data reading time for each convolution calculation, improve efficiency, and avoid the consumption of computing resources of the computing chip used for convolution calculation.

[0006] The present application provides a computing system for running a convolutional neural network, the computing system comprising a storage control circuit;

[0007] The storage control circuit is used to split the convolutional neural network to obtain a target convolutional layer to be calculated;

[0008] The storage control circuit is used to obtain target parameters corresponding to the target convolution layer, where the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all addresses of the calculation data required for the target convolution layer;

[0009] The storage control circuit is used to generate n data starting addresses and a data call length corresponding to each data starting address when performing each convolution calculation for the target convolution layer according to the target parameter, wherein n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal;

[0010] The storage control circuit is used to read the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each of the data starting addresses.

[0011] Optionally, the size of the convolution kernel of the target convolution layer includes a convolution height and a convolution width;

[0012] The n is the convolution height, and the data call length corresponding to each of the data start addresses is calculated using the convolution width and the number of channels of the input signal.

[0013] Optionally, the storage control circuit includes a convolution configuration register and a convolution input reading circuit;

[0014] The convolution configuration register is used to obtain target parameters corresponding to the target convolution layer;

[0015] The convolution input reading circuit is used to generate n data starting addresses and a data call length corresponding to each data starting address for each convolution calculation performed on the target convolution layer according to the target parameters, and read the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each data starting address.

[0016] Optionally, the target parameters further include a padding mode and current coordinates, where the current coordinates are used to identify a current pixel position of the input signal when each convolution calculation is performed on the target convolution layer.

[0017] Optionally, the computing system includes static memory, dynamic memory or flash memory;

[0018] The storage control circuit is used to obtain all addresses of calculation data required for the target convolution layer from the static memory, the dynamic memory or the flash memory.

[0019] Optionally, the static memory and the storage control circuit are located in the same chip;

[0020] Or, the dynamic memory and the storage control circuit are located in the same chip;

[0021] Or, the flash memory and the storage control circuit are located in the same chip;

[0022] Alternatively, the static memory, the dynamic memory and the flash memory are not located in the same chip as the storage control circuit.

[0023] Optionally, the storage control circuit is also used to detect the maximum and minimum values ​​of the calculated data, and determine a target multiple that satisfies the target data format based on the maximum and minimum values; scale the calculated data according to the target multiple to obtain scaled data, so as to perform convolution calculation using the scaled data.

[0024] Optionally, the computing system comprises a computing circuit;

[0025] The calculation circuit is used to obtain the calculation data transmitted by the storage control circuit, and perform convolution calculation according to the calculation data.

[0026] The present application provides a method for obtaining input data of a convolutional neural network, which is applied to any of the computing systems described above, and the method includes:

[0027] Split the convolutional neural network to obtain the target convolution layer to be processed;

[0028] Obtain target parameters corresponding to the target convolution layer, where the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all addresses of the calculation data required for the target convolution layer;

[0029] Generate n data starting addresses and a data call length corresponding to each data starting address when performing each convolution calculation for the target convolution layer according to the target parameter, wherein n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal;

[0030] The calculation data for each convolution calculation is read according to the n data starting addresses and the data call length corresponding to each of the data starting addresses.

[0031] The present application provides a computer-readable medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0032] The present application provides a computing system for running a convolutional neural network, and the computing system includes: a storage control circuit, that is, the computing chip is not used to determine the data storage address of each convolution calculation from the memory and read the calculation data, so as to avoid the consumption of computing resources of the computing chip used for convolution calculation; the storage control circuit obtains the target convolution layer to be calculated by splitting the convolutional neural network, and the calculation of the target convolution layer includes multiple convolution calculations; the storage control circuit obtains the target parameters corresponding to the target convolution layer to facilitate the subsequent acquisition of the calculation data of multiple convolution calculations, wherein the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all the addresses of the calculation data required for the target convolution layer; the storage control circuit generates n data starting addresses and the data call length corresponding to each data starting address when performing each convolution calculation for the target convolution layer according to the target parameters, wherein n is the size of the convolution kernel of the target convolution layer, and each data starting address is a data call length corresponding to each data starting address. The data call length corresponding to the starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal; the storage control circuit directly reads the calculation data for each convolution calculation based on the n data starting addresses and the data call length corresponding to each data starting address, that is, the storage control circuit can obtain the data storage addresses of multiple convolution calculations based on the target parameters, and then directly read the calculation data based on the obtained data storage addresses, so that the subsequent computing chip can directly use the calculation data for convolution calculation; based on this, the present application introduces a storage control circuit, and uses the storage control circuit to obtain the target parameters corresponding to the convolution calculation of the target convolution layer, and quickly, simply and efficiently obtains the data storage addresses corresponding to multiple convolution calculations through the target parameters, and efficiently reads the calculation data based on the data storage addresses, without repeatedly traversing all the data storage addresses to determine the data storage addresses corresponding to these calculation data and reading the calculation data, thereby using the storage control circuit to shorten the calculation data reading time for each convolution calculation and improve the calculation data reading efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 A schematic diagram of convolution calculation of a single-channel image is shown;

[0035] Figure 2 A schematic diagram of convolution calculation of a multi-channel image is shown;

[0036] Figure 3 A schematic diagram of computing data storage is shown;

[0037] Figure 4 A schematic diagram of the structure of a computing system for running a convolutional neural network provided in an embodiment of the present application is shown;

[0038] Figure 5 A schematic diagram of target parameters of a target convolution layer provided in an embodiment of the present application is shown;

[0039] Figure 6 A schematic diagram showing all addresses of calculation data required for a target convolutional layer provided in an embodiment of the present application is shown;

[0040] Figure 7 A schematic diagram of a calculation data scaling process provided by an embodiment of the present application is shown;

[0041] Figure 8 Another schematic diagram of computing data scaling processing provided by an embodiment of the present application is shown;

[0042] Fig. 9 A schematic flow chart of a method for acquiring input data for a convolutional neural network provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0045] The main principle of convolution calculation is to use a specific convolution weight parameter matrix to calculate the inner product of the input and convolution weight of a small area of ​​the input signal. After completing one calculation, it moves to the next small area to continue the calculation.

[0046] In each calculation, the data for convolution calculation needs to be selected from the input signal. For example, if the input signal is an input image, refer to Figure 1 As shown, Figure 1 This is a schematic diagram of convolution calculation for a single-channel image. If the convolution kernel size of the convolutional neural network is 3×3, 3 adjacent data in the same row of the input image and 3 adjacent rows of data are selected, a total of 9 data as calculation data. Figure 1 In the example, we can select the 9 data determined by the convolution core with the data 4 in the 2nd row and the 3rd column as the calculation data. Figure 1 Use grayscale fill to highlight.

[0047] because Figure 1 The convolution calculation is for a single-channel image. If the input image is a multi-channel image, each position point in the input image corresponds to multiple channels, that is, each position point corresponds to multiple data. Figure 2 As shown, Figure 2 This is a convolution calculation diagram for a multi-channel image. If the convolution kernel size of the convolutional neural network is 3×3 and the number of channels of the multi-channel image is 4, the number of data to be calculated is 3×3×4.

[0048] Since the input signal is stored in the memory, and the input signal is stored in the memory using one or more continuous address memories. The input signal can be stored in the memory according to a pre-set data storage order, such as channel, row, column or row, column, channel, etc., where channel, row, column refers to the continuous storage of data of different channels at the same location, and then the data is continuously stored in the order of adjacent columns in the same row, and finally the data is continuously stored in the order of rows. Figure 3 As shown, Figure 3 A schematic diagram of a computational data storage. Figure 3 Data is stored in the order of channels, rows, and columns.

[0049] From the above description, we can know that the data storage addresses of different rows of data are different, and the data storage addresses of different rows of data are not adjacent. Therefore, when performing a convolution calculation, it is necessary to read the corresponding calculation data from these non-adjacent data storage addresses in order to successfully perform the convolution calculation.

[0050] However, each time a convolution calculation is performed, in order to read the corresponding calculation data from non-adjacent data storage addresses, it is necessary to repeatedly traverse all data storage addresses to determine the data storage addresses corresponding to these calculation data, resulting in a long calculation data reading time for each convolution calculation, low efficiency, and more consumption of computing resources of the computing chip used for the convolution calculation.

[0051] Even if all data storage addresses are not repeatedly traversed to determine the data storage addresses corresponding to these calculation data, the computing chip is also required to analyze the data starting address and data call length of the memory of the currently called calculation data for each convolution calculation. The same convolution calculation also needs to search for the data starting addresses of non-adjacent data storage addresses multiple times. For example, if the convolution kernel size of the convolutional neural network is 3×3, and the three rows of calculation data are not stored adjacent to each other, it is necessary to determine the data starting address of each row. Multiple searches for the data starting address consume a lot of computing time, and thus greatly consume the computing resources of the computing chip.

[0052] Based on this, an embodiment of the present application provides a computing system for running a convolutional neural network, and the computing system includes: a storage control circuit, that is, the computing chip is not used to determine the data storage address of each convolution calculation from the memory and read the calculation data, which can avoid the consumption of computing resources of the computing chip used for convolution calculation; the storage control circuit obtains the target convolution layer to be calculated by splitting the convolutional neural network, and the calculation of the target convolution layer includes multiple convolution calculations; the storage control circuit obtains the target parameters corresponding to the target convolution layer to facilitate the subsequent acquisition of the calculation data of multiple convolution calculations, wherein the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all the addresses of the calculation data required for the target convolution layer; the storage control circuit generates n data starting addresses and the data call length corresponding to each data starting address for each convolution calculation for the target convolution layer according to the target parameters, wherein n is the size of the convolution kernel of the target convolution layer, and each The data call length corresponding to the data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal; the storage control circuit directly reads the calculation data for each convolution calculation based on the n data starting addresses and the data call length corresponding to each data starting address, that is, the storage control circuit can obtain the data storage addresses of multiple convolution calculations based on the target parameters, and then directly read the calculation data based on the obtained data storage addresses, so that the subsequent calculation chip can directly use the calculation data for convolution calculation; based on this, the present application introduces a storage control circuit, and uses the storage control circuit to obtain the target parameters corresponding to the convolution calculation of the target convolution layer, and quickly, simply and efficiently obtains the data storage addresses corresponding to multiple convolution calculations through the target parameters, and efficiently reads the calculation data based on the data storage address, without repeatedly traversing all the data storage addresses to determine the data storage addresses corresponding to these calculation data and reading the calculation data, thereby using the storage control circuit to shorten the calculation data reading time for each convolution calculation and improve the calculation data reading efficiency.

[0053] In order to better understand the technical solution and technical effects of the present application, specific embodiments will be described in detail below with reference to the accompanying drawings.

[0054] See also Figure 4 , which is a schematic diagram of the structure of a computing system for running a convolutional neural network provided in an embodiment of the present application.

[0055] The computing system for running a convolutional neural network provided in this embodiment includes: a storage control circuit 110, referring to Figure 4 shown.

[0056] Taking into account that the convolution calculation of the convolutional neural network includes convolution calculation of multiple layers of convolutional layers, the storage control circuit splits the convolutional neural network to obtain the target convolution layer to be calculated, and finally obtains the calculation result of the entire convolutional neural network through the convolution calculation of multiple target convolution layers.

[0057] In order to realize the convolution calculation of the target convolution layer, it is necessary to obtain the calculation data of the input target convolution layer. First, it is necessary to determine the data storage address of the calculation data in the memory. Therefore, the storage control circuit obtains the target parameters corresponding to the target convolution layer, so as to obtain the data storage address of the calculation data using the target parameters later. Among them, the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and the full address of the calculation data required for the target convolution layer. The full address of the calculation data required by the target convolution layer is the continuous storage address of the calculation data required for multiple convolution calculations of the target convolution layer. Each time a convolution calculation is performed, the data storage address corresponding to the calculation data required for the convolution calculation can be determined from the full address. The data storage address corresponding to the calculation data required for each convolution calculation may not be adjacent.

[0058] If the input signal is an input image, the target parameters include the size of the input image, the number of channels of the input image, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, and the vertical step size of the convolution sliding. The size of the input image includes the input height and the input width, and the size of the convolution kernel of the target convolution layer includes the convolution height and the convolution width.

[0059] As an example, refer to Figure 5 As shown, Figure 5 A schematic diagram of target parameters of a target convolutional layer provided in an embodiment of the present application. Figure 5 The input height and input width are both 5, the convolution height and convolution width are both 3, the horizontal step size of the convolution sliding and the vertical step size of the convolution sliding are both 1, and the number of channels of the input image is 4.

[0060] The storage control circuit generates n data starting addresses and a data call length corresponding to each data starting address for each convolution calculation for the target convolution layer according to the target parameters, wherein n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal.

[0061] Specifically, since the calculated data is stored in the memory according to a preset data storage order, when the data storage order is determined, the size of the convolution kernel of the target convolution layer is known, and multiple data starting addresses of the calculated data required for each convolution calculation when the target convolution layer performs multiple convolution calculations can be determined. Accordingly, when the data storage order is determined, the number of channels of the input signal and the size of the convolution kernel of the target convolution layer can be determined. When the target convolution layer performs multiple convolution calculations, the data call length corresponding to each data starting address in each convolution calculation can be determined, that is, the data end position corresponding to each data starting address in each convolution calculation can be determined.

[0062] When the data storage order of all addresses of the calculation data required by the target convolution layer is determined, if the input signal is an input image, n is the convolution height, that is, the number of data starting addresses for each convolution calculation is determined according to the convolution height; the data call length corresponding to each data starting address is calculated using the convolution width and the number of channels of the input signal.

[0063] exist Figure 5 Based on the reference Figure 6 As shown, Figure 6 A schematic diagram of all addresses of the calculation data required for a target convolution layer provided in an embodiment of the present application. The size of the convolution kernel based on the target convolution layer is 3 for both the convolution height and the convolution width, and the number of channels of the input image is 4. Figure 6 The data storage order in is channel, row, and column. Therefore, the number of data starting addresses for each convolution calculation for the target convolution layer is 3, that is, the data starting address of each row in the 3 rows is determined. According to the data storage order of channels, rows, and columns, combined with the convolution width of 3 and the number of channels of the input image of 4, the data call length corresponding to each data starting address is calculated to be 12, that is, the data call length of each row in the 3 rows is determined to be 12.

[0064] After the storage control circuit generates n data starting addresses and the data call length corresponding to each data starting address for each convolution calculation for the target convolution layer according to the target parameters, the storage control circuit reads the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each data starting address. That is, the storage control circuit can obtain the data storage addresses of multiple convolution calculations according to the target parameters, and then directly reads the calculation data according to the obtained data storage addresses, so that the subsequent calculation chip can directly use the calculation data for convolution calculation.

[0065] The present application introduces a storage control circuit, and uses the storage control circuit to obtain target parameters corresponding to the convolution calculation of the target convolution layer, and quickly, simply and efficiently obtains the data storage addresses corresponding to multiple convolution calculations through the target parameters, and efficiently reads the calculation data according to the data storage address, without repeatedly traversing all the data storage addresses to determine the data storage addresses corresponding to these calculation data and reading the calculation data, thereby using the storage control circuit to shorten the calculation data reading time for each convolution calculation and improve the calculation data reading efficiency.

[0066] In the embodiment of the present application, the target parameters also include the padding mode and the current coordinates. When performing convolution calculations on the target convolution layer, there may be a number of calculation data that does not meet the convolution kernel size requirements when the convolution kernel is edge data. In this case, the padding mode is used to provide data on the number of calculation data that meet the convolution kernel size requirements. Figure 5 As shown in the figure, edge data refers to the 2 rows or columns of data at the outermost side of the input image. The current coordinates are used to identify the current pixel position of the input signal when performing each convolution calculation on the target convolution layer, that is, the current coordinates can be used to identify the convolution core when performing each convolution calculation on the target convolution layer, avoiding the problem of being unable to continue the calculation after the convolution calculation on the target convolution layer is interrupted due to special circumstances.

[0067] In an embodiment of the present application, the storage control circuit 110 includes a convolution configuration register 111 and a convolution input reading circuit 112. Figure 4As shown. The convolution configuration register is used to obtain the target parameters corresponding to the target convolution layer, that is, the convolution configuration register is used to obtain the target parameters including the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, all addresses of the calculation data required for the target convolution layer, the padding mode and the current coordinates; the convolution input reading circuit is used to generate n data starting addresses and the data call length corresponding to each data starting address for each convolution calculation for the target convolution layer according to the target parameters, and read the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each data starting address. In other words, obtaining the target parameters and generating the data starting address, generating the data call length and reading the calculation data are completed using the convolution configuration register and the convolution input reading circuit respectively.

[0068] In an embodiment of the present application, the computing system further includes a computing circuit. After reading and obtaining the computing data, the storage control circuit transmits the computing data to the computing circuit, and the computing circuit obtains the computing data transmitted by the storage control circuit, and performs convolution calculation according to the computing data. In other words, the present application adopts an additional storage control circuit to determine the data storage address of the convolution calculation and read the computing data, that is, the computing circuit is not used to determine the data storage address of each convolution calculation from the memory and read the computing data, which can avoid the consumption of computing resources of the computing circuit used for the convolution calculation.

[0069] For example, the computing circuit may exist in the hardware form of a computing chip.

[0070] In an embodiment of the present application, the computing system further includes a static memory (SRAM), a dynamic memory (DRAM) or a flash memory (FLASH). Static memory, dynamic memory or flash memory are different types of memory. When the type of memory is a static memory, the static memory and the computing circuit can be arranged on the same chip, such as a computing chip. When the type of memory is a dynamic memory or flash memory, the dynamic memory or flash memory and the computing circuit are not arranged on the same chip, for example, the computing circuit is arranged in the computing chip, and the dynamic memory or flash memory is arranged in another independent chip.

[0071] The static memory, dynamic memory or flash memory stores the calculation data required for the target convolution layer to perform convolution calculations, and the calculation data is stored in a data storage order, that is, all addresses of the calculation data are determined according to the data storage order. After the data storage order is determined, the storage control circuit obtains all addresses of the calculation data required for the target convolution layer from the static memory, dynamic memory or flash memory, so as to facilitate the subsequent rapid determination of the n data starting addresses and data call length for each convolution calculation for the target convolution layer according to all addresses of the calculation data.

[0072] In the embodiments of the present application, based on the different types of memories included in the computing system, the hardware form of the storage control circuit in the computing system is also correspondingly different, meeting various configuration requirements for introducing the storage control circuit in the computing system.

[0073] As a possible implementation manner, when the type of the memory is a static memory, the static memory and the storage control circuit are located in the same chip, such as a computing chip.

[0074] As another possible implementation, when the memory type is a dynamic memory, the dynamic memory and the storage control circuit are arranged in the same chip.

[0075] As another possible implementation, when the type of the memory is a flash memory, the flash memory and the storage control circuit are arranged on the same chip.

[0076] As another possible implementation, regardless of whether the type of memory is one or more of static memory, dynamic memory and flash memory, the static memory, dynamic memory and flash memory are not located in the same chip as the storage control circuit, that is, the storage control circuit is separately arranged in an independent chip and is electrically connected to the memory and the computing circuit.

[0077] In the embodiment of the present application, considering that the calculation circuit may have requirements for the data format of the calculation data when performing convolution calculation using the calculation data, the data format of the calculation data is different from that of the calculation data stored in the memory. For example, the input signal in the memory uses a data format with a large number of bits, such as 32-bit floating point numbers, 16-bit fixed point numbers, and 16-bit floating point numbers, while the calculation circuit uses a data format with a small number of bits, such as 8-bit fixed point numbers and 8-bit unsigned fixed point numbers as the input quantity for calculation. After the calculation data is read from the memory, the number of bits must be discarded and simplified. At this time, in order to smoothly perform the convolution calculation, the storage control circuit can detect the maximum and minimum values ​​of the calculation data, and determine the target multiple that meets the target data format according to the maximum and minimum values; the calculation data is scaled according to the target multiple to obtain the scaled data, so as to use the scaled data for convolution calculation. Among them, the target data format is the data format required when the calculation circuit performs convolution calculation, and the target multiple is the multiple of the data format required when the calculation data is scaled from the data format that meets the memory requirements to the data format required when the calculation circuit performs convolution calculation.

[0078] In practical applications, the calculation circuit and the storage control circuit can also save the target multiple so that the data format can be restored according to the target multiple after the convolution calculation result is obtained.

[0079] As an example, refer to Figure 7 As shown, Figure 7A schematic diagram of a calculation data scaling process provided in an embodiment of the present application is shown. When performing any convolution step, the size of the convolution kernel is 3×3. The calculated data of this convolution calculation are 1022, 200, 768, -200, 122, -716, -19, 301 and 121 in row and column order, with a maximum value of 1022 and a minimum value of -716. The target data format is an 8-bit signed fixed-point number. According to 1022 and -716, the target multiple that satisfies the 8-bit signed fixed-point number is determined to be 1 / 8. According to the target multiple of 1 / 8, the calculated data is scaled, that is, 1022, 200, 768, -200, 122, -716, -19, 301 and 121 are divided by 8 respectively, and the scaled data are 127, 25, 96, -28, 15, -90, -3, 37 and 15 respectively, and the target multiple is saved as 1 / 8, so that the data format can be restored according to 1 / 8 after the convolution calculation result is obtained later.

[0080] In the embodiment of the present application, considering that the calculation data for each convolution calculation is different, and the maximum and minimum values ​​of the calculation data are different, the target multiples are also different, that is, each convolution calculation is independently scaled to avoid the loss of data accuracy caused by using the same target multiple for multiple convolution calculations. In other words, each step of sliding convolution is independently scaled and fixed-pointed, thereby avoiding global common fixed-pointing, reducing data accuracy loss, and improving data calculation accuracy.

[0081] As an example, in Figure 7 Based on the reference Figure 8 As shown, Figure 8 Another schematic diagram of calculating data scaling processing provided by an embodiment of the present application is shown. Figure 7 In the other convolution step shown in the figure, the size of the convolution kernel is 3×3. The calculation data of this convolution calculation are 200, 73, 134, -220, 122, -252, -30, 101 and 121 in row and column order. The maximum value is 220 and the minimum value is -252. The target data format is 8-bit signed fixed-point number. If and Figure 7 The same target multiple is 1 / 8, then the precision loss of the calculated data after scaling becomes larger, which is not conducive to convolution calculation. At this time, according to 220 and -252, the target multiple that satisfies the 8-bit signed fixed-point number is determined to be 1 / 2. The calculated data is scaled according to the target multiple of 1 / 2, that is, 200, 73, 134, -220, 122, -252, -30, 101 and 121 are divided by 2 respectively, and the scaled data are 100, 36, 67, -110, 61, -126, -15, 50 and 60 respectively, and the target multiple is saved as 1 / 2, which is convenient for the subsequent data format restoration according to 1 / 2 after the convolution calculation results are obtained.

[0082] As a possible implementation, the storage control circuit includes a maximum and minimum value automatic detection circuit and a data automatic scaling circuit. The maximum and minimum value automatic detection circuit is used to detect the maximum and minimum values ​​of the calculated data, and determine the target multiple that meets the target data format according to the maximum and minimum values. The data automatic scaling circuit is used to scale the calculated data according to the target multiple to obtain scaled data, so as to perform convolution calculation using the scaled data.

[0083] It can be seen that the computing system for running a convolutional neural network provided in the embodiment of the present application is controlled by a hardware-based storage control circuit, which eliminates the need to determine the data storage address of the calculated data from the instruction input starting address multiple times for each convolution, greatly reduces the time consumed in data reading, and does not require a computing chip to read data, thereby simplifying the programming and compilation difficulty of the convolutional neural network when it is deployed on a chip.

[0084] Based on the computing system for running a convolutional neural network provided in the above embodiment, the embodiment of the present application also provides a method for obtaining input data of a convolutional neural network, which is described in detail below in conjunction with the accompanying drawings.

[0085] See also Fig. 9 , which is a flow chart of a method for obtaining input data of a convolutional neural network provided in an embodiment of the present application. The method for obtaining input data of a convolutional neural network provided in this embodiment is applied to the computing system for running the convolutional neural network provided in the above embodiment.

[0086] The method for obtaining input data of a convolutional neural network provided in this embodiment includes the following steps:

[0087] S101, split the convolutional neural network to obtain a target convolutional layer to be processed.

[0088] S102, obtaining target parameters corresponding to the target convolution layer, the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all addresses of the calculation data required for the target convolution layer.

[0089] S103, generating n data starting addresses and a data call length corresponding to each data starting address for each convolution calculation for the target convolution layer according to the target parameters, where n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal.

[0090] S104, reading the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each data starting address.

[0091] In some embodiments, the size of the convolution kernel of the target convolution layer includes a convolution height and a convolution width;

[0092] The n is the convolution height, and the data call length corresponding to each of the data start addresses is calculated using the convolution width and the number of channels of the input signal.

[0093] In some embodiments, the target parameters also include a padding mode and a current coordinate, where the current coordinate is used to identify a current pixel position of the input signal when each convolution calculation is performed on the target convolution layer.

[0094] In some embodiments, the maximum and minimum values ​​of the calculation data are detected, and a target multiple that satisfies the target data format is determined based on the maximum and minimum values; the calculation data is scaled according to the target multiple to obtain scaled data, so as to perform convolution calculation using the scaled data.

[0095] In some embodiments, a convolution calculation is performed based on the calculation data.

[0096] The embodiment of the present application also provides a computer-readable medium for storing program code, where the program code is used to execute any one of the methods of the aforementioned embodiments.

[0097] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0098] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0099] The above is only a preferred implementation of the present application. Although the present application has been disclosed as a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.

Claims

1. A computing system for running a convolutional neural network, characterized in that: The computing system includes a storage control circuit; The storage control circuit is used to split the convolutional neural network to obtain a target convolutional layer to be calculated; The storage control circuit is used to obtain target parameters corresponding to the target convolution layer, where the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all addresses of the calculation data required for the target convolution layer; The storage control circuit is used to generate n data starting addresses and a data call length corresponding to each data starting address when performing each convolution calculation for the target convolution layer according to the target parameter, wherein n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal; The storage control circuit is used to read the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each of the data starting addresses.

2. The computing system according to claim 1, characterized in that The size of the convolution kernel of the target convolution layer includes convolution height and convolution width; The n is the convolution height, and the data call length corresponding to each of the data start addresses is calculated using the convolution width and the number of channels of the input signal.

3. The computing system according to claim 1, characterized in that The storage control circuit includes a convolution configuration register and a convolution input reading circuit; The convolution configuration register is used to obtain target parameters corresponding to the target convolution layer; The convolution input reading circuit is used to generate n data starting addresses and a data call length corresponding to each data starting address for each convolution calculation performed on the target convolution layer according to the target parameters, and read the calculation data for each convolution calculation according to the n data starting addresses and the data call length corresponding to each data starting address.

4. The computing system according to claim 1, characterized in that: The target parameters also include a padding mode and a current coordinate, where the current coordinate is used to identify a current pixel position of the input signal when each convolution calculation is performed on the target convolution layer.

5. The computing system according to claim 1, characterized in that: The computing system includes static memory, dynamic memory or flash memory; The storage control circuit is used to obtain all addresses of calculation data required for the target convolution layer from the static memory, the dynamic memory or the flash memory.

6. The computing system according to claim 5, characterized in that: The static memory and the storage control circuit are located on the same chip; Or, the dynamic memory and the storage control circuit are located in the same chip; Or, the flash memory and the storage control circuit are located in the same chip; Alternatively, the static memory, the dynamic memory and the flash memory are not located in the same chip as the storage control circuit.

7. The computing system according to claim 1, characterized in that: The storage control circuit is also used to detect the maximum value and the minimum value of the calculated data, and determine the target multiple that meets the target data format according to the maximum value and the minimum value; scale the calculated data according to the target multiple to obtain scaled data, so as to perform convolution calculation using the scaled data.

8. The computing system according to any one of claims 1 to 7, characterized in that: The computing system includes a computing circuit; The calculation circuit is used to obtain the calculation data transmitted by the storage control circuit, and perform convolution calculation according to the calculation data.

9. A method for obtaining input data of a convolutional neural network, characterized in that: Applied to the computing system according to any one of claims 1 to 8, the method comprising: Split the convolutional neural network to obtain the target convolution layer to be processed; Obtain target parameters corresponding to the target convolution layer, where the target parameters include the size of the input signal, the number of channels of the input signal, the size of the convolution kernel of the target convolution layer, the horizontal step size of the convolution sliding, the vertical step size of the convolution sliding, and all addresses of the calculation data required for the target convolution layer; Generate n data starting addresses and a data call length corresponding to each data starting address when performing each convolution calculation for the target convolution layer according to the target parameter, wherein n is the size of the convolution kernel of the target convolution layer, and the data call length corresponding to each data starting address is calculated using the size of the convolution kernel of the target convolution layer and the number of channels of the input signal; The calculation data for each convolution calculation is read according to the n data starting addresses and the data call length corresponding to each of the data starting addresses.

10. A computer-readable medium, characterized in that The method comprises instructions which, when executed on a computer, cause the computer to perform the method as claimed in claim 9.

Images