Parameter selection method and parameter selection system for real-time neural network operation architecture

Abstract

The invention discloses a parameter selection method and a parameter selection system for a real-time neural network operation framework. The parameter selection method comprises: obtaining an access policy combination for a target real-time neural network operation architecture, the access policy combination comprising a plurality of access policies; each access policy defines a plurality of access parameters used when the data access circuit accesses the memory. The parameter selection method also includes accessing the memory according to each access policy by configuring a plurality of data access circuits, and simultaneously configuring a plurality of operation channels to perform a convolution operation process for each access policy to obtain an optimized access policy used for performing the convolution operation process.

Description

Technical Field

[0001] This invention relates to a method and system, and more particularly to a parameter selection method and system for real-time neural network computing architectures. Background Technology

[0002] In recent years, artificial intelligence has developed rapidly, and neural network models have been widely applied in daily life and technology. Depending on the application, neural network models are divided into non-real-time and real-time computational architectures. In non-real-time computational architectures, all data needs to be loaded into memory before computation. In real-time neural network architectures, for example, when applied to audio noise reduction, real-time computation is performed simultaneously with data input.

[0003] However, neural network architectures have various model parameters, and different combinations of these parameters will result in a wide variety of variations in the neural network architecture. Furthermore, the settings of the computational circuits used will also affect the results of the neural network's execution. However, it is impossible to determine in advance how to adjust these parameters to achieve the best performance. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a parameter selection method and parameter selection system for real-time neural network computing architecture, which can gradually find the best parameter combination and improve computing efficiency during data processing, in order to address the shortcomings of the prior art.

[0005] To address the aforementioned technical problems, one technical solution adopted by this invention is to provide a parameter selection method for a real-time neural network computing architecture. This method includes the following steps performed by a computing device of a parameter selection system: obtaining a combination of access strategies for a target real-time neural network computing architecture, wherein the combination of access strategies includes multiple access strategies, and the target real-time neural network computing architecture includes a memory, multiple data access circuits, and multiple computing channels. The multiple data access circuits are connected to the memory via a bus. Each access strategy defines multiple access parameters used by the multiple data access circuits when accessing the memory. The multiple computing channels are respectively connected to the multiple data access circuits and each includes multiple processing unit circuits. The parameter selection method further includes configuring the multiple data access circuits to access the memory according to each access strategy, and simultaneously configuring the multiple computing channels to perform a convolution operation for each access strategy, to obtain an optimized access strategy used to perform the convolution operation.

[0006] To address the aforementioned technical problems, another technical solution adopted by the present invention is to provide a parameter selection system for a real-time neural network computing architecture, comprising a computing device and a target real-time neural network computing architecture. The target real-time neural network computing architecture includes a memory, multiple data access circuits, and multiple computing channels. The multiple data access circuits are connected to the memory via a bus. The multiple computing channels are respectively connected to the multiple data access circuits and each includes multiple processing element circuits. The computing device is configured to perform the following steps: obtaining a combination of access strategies for the target real-time neural network computing architecture, wherein the combination of access strategies includes multiple access strategies, and each access strategy defines multiple access parameters used by the multiple data access circuits when accessing the memory; and configuring the multiple data access circuits to access the memory according to each access strategy, while simultaneously configuring the multiple computing channels to perform a convolution operation for each access strategy, to obtain an optimized access strategy used to perform the convolution operation.

[0007] One of the beneficial effects of the present invention is that the parameter selection method and parameter selection system for real-time neural network computing architecture provided by the present invention can gradually find the best parameter combination during data processing, and has a progressive adjustment mechanism for immediate optimization, thus effectively improving computing efficiency.

[0008] To further understand the features and technical content of the present invention, please refer to the following detailed description and accompanying drawings. However, the drawings provided are for reference and illustration only and are not intended to limit the present invention. Attached Figure Description

[0009] Figure 1 This is a functional block diagram of a parameter selection system for a real-time neural network computing architecture according to an embodiment of the present invention.

[0010] Figure 2 This is a functional block diagram of the target real-time neural network computing architecture in an embodiment of the present invention.

[0011] Figure 3 This is a flowchart of the convolution operation process according to an embodiment of the present invention.

[0012] Figure 4 This is a simplified architecture diagram of a convolutional neural network model.

[0013] Figure 5 This is a flowchart of a parameter selection method for a real-time neural network computing architecture according to an embodiment of the present invention.

[0014] Figure 6 This is another flowchart of a parameter selection method for a real-time neural network computing architecture according to an embodiment of the present invention. Detailed Implementation

[0015] The following specific embodiments illustrate the implementation of the "parameter selection method and parameter selection system for real-time neural network computing architecture" disclosed in this invention. Those skilled in the art can understand the advantages and effects of this invention based on the content disclosed in this specification. This invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of this invention. Furthermore, it should be stated in advance that the accompanying drawings of this invention are for simple illustration only and are not drawn to actual dimensions. The following embodiments will further describe the relevant technical content of this invention in detail, but the disclosed content is not intended to limit the scope of protection of this invention. In addition, the term "or" used herein may include, depending on the actual situation, any combination of any one or more of the associated listed items.

[0016] Figure 1 This is a functional block diagram of a parameter selection system for a real-time neural network computing architecture according to an embodiment of the present invention. (Reference) Figure 1 As shown, this embodiment of the invention provides a parameter selection system 1 for a real-time neural network computing architecture, which includes a computing device 10 and a target real-time neural network computing architecture 12. The computing device 10 may be, for example, a general-purpose computer system, while the target real-time neural network computing architecture 12 may be contained within or electrically connected to the computing device 10; this invention does not limit their relative relationship. Specifically, the computing device 10 and the target real-time neural network computing architecture 12 may include architectures implemented in one or more of hardware, software, and firmware; this invention also does not limit the specific implementation of the computing device 10 and the target real-time neural network computing architecture 12.

[0017] Figure 2 This is a functional block diagram of the target real-time neural network computing architecture according to an embodiment of the present invention. Please refer to... Figure 2 The target real-time neural network computing architecture 12 includes a memory 120, multiple data access circuits 122 and multiple processing unit circuits 124.

[0018] Data access circuit 122 can be connected to memory 120 via memory controller MC and bus BS. Memory 120 may include multiple memory blocks 1200, each memory block 1200 being, for example, a memory bank. Each data access circuit 122 can be configured to retrieve data to be processed from memory 120 according to access strategy STG. For example, access strategy STG may define multiple access parameters used by the multiple data access circuits when accessing memory, including the order in which the multiple memory blocks 1200 are read. For example, if there are 16 memory blocks 1200, the access strategy STG may be multiple memory addresses arranged in sequence, such as 0x00, 0x10, 0x20, etc. Furthermore, in some embodiments, access strategy STG also includes the method by which each data access circuit 122 allocates access to memory 120 via bus BS, for example, how many memory blocks 1200 are read by each data access circuit 122 in a single read time interval.

[0019] On the other hand, multiple processing element (PE) circuits 124 are used to perform a convolution operation based on a convolutional neural network model. Each data access circuit 122 can be connected to a computation block, and each computation block can include one or more processing element circuits 124, such as... Figure 2 The diagram may show four examples, but the invention is not limited thereto. The computational data retrieved from memory 120 can be input into the corresponding processing element circuit 124 to perform the convolution operation.

[0020] Figure 3 This is a flowchart of the convolution operation process according to an embodiment of the present invention. Figure 4 This is a simplified architecture diagram of a convolutional neural network model. (Reference) Figure 3 and Figure 4 The convolution operation process may include the following steps:

[0021] Step S10: Retrieve the data to be processed from memory according to the access strategy. In this step, the operation can begin as long as a portion of the data to be processed is input.

[0022] Step S11: Input the data to be processed into multiple channels. In this step, each channel can process a set of convolutional kernels.

[0023] Step S12: Using each convolution kernel, perform convolution operations on the data to be processed according to the first direction stride to generate multiple output data.

[0024] by Figure 4 For example, Figure 4The convolutional neural network model 2 has four channels. The amount of data to be processed can be, for example, a matrix with a width of 11 and a height of 5, and the four channels represent four sets of convolutional kernels. Each convolutional kernel has a kernel size of... Figure 4 For example, the core size has a width of 5, a height of 5, and a channel size of 4.

[0025] Furthermore, the stride in the first direction indicates the amount of data to be moved in the first direction (e.g., the X direction) after convolution. Figure 4 The first direction step size is 2. Therefore, the data to be processed is convolved with the first set of convolution kernels through the first channel, and the four output results after convolution are used as the output data. Similarly, the second to fourth sets of convolution kernels can be convolved with the data to be processed in the same way to obtain the results produced by each set of convolution kernels.

[0026] Furthermore, when new data to be processed is input, and the convolution operation is to be performed in the next layer (second direction), each convolution kernel is also used to perform convolution operation on the newly input data to be processed according to the second direction stride. The operation directions of the first direction stride and the second direction stride are different. For example, if the first direction stride represents the number of steps the convolution kernel moves in the X direction, then the second direction stride represents the number of steps the convolution kernel moves in the Y direction.

[0027] Please refer to this again. Figure 2 When using Figure 1 The target real-time neural network computing architecture 12 is used for Figure 3 During the convolution operation, Figure 2 Each of the four operation channels (tiles) contains four processing element circuits, and the four output results of each channel can be calculated by the four processing element circuits 124 respectively. Similarly, when performing convolution operation on the data to be processed using the second set of convolution kernels of the second channel, it can be performed in the second operation channel, and the four corresponding output results can be calculated by the four processing element circuits 124.

[0028] However, as described above, when the memory 120 is read, the data to be processed and the convolution kernel data can be obtained. The access strategy used when accessing the memory 120, such as the order in which the memory blocks are read and the method of bus allocation for reading memory, will affect computational efficiency. Therefore, it is necessary to find the most suitable combination of parameters so that the target real-time neural network computing architecture 12 can execute at optimal performance. It is worth mentioning that the width and height of the data to be processed, the first direction stride, the second direction stride, the number of input channels, the number of output channels, and the width and height of the convolution kernel are set by registers. In some embodiments, dedicated memory can be accessed separately to obtain the above data for use by the registers; this invention is not limited thereto.

[0029] Please refer to Figure 5 This is a flowchart of a parameter selection method for a real-time neural network computing architecture according to an embodiment of the present invention. To find the most suitable parameter combination, the present invention also provides a parameter selection method for a real-time neural network computing architecture, including a configurable computing device 10 to perform the following steps:

[0030] Step S20: Obtain a combination of access strategies for the target real-time neural network computing architecture. In this step, the combination of access strategies includes an access strategy STG for each data access circuit 122, and each access strategy STG defines multiple access parameters used by multiple data access circuits when accessing memory.

[0031] Step S21: For each access strategy, obtain the data processing time spent performing the convolution operation, and use the access parameter with the shortest data processing time as the optimized access strategy.

[0032] Please refer to Figure 6 This is another flowchart illustrating a parameter selection method for a real-time neural network computing architecture according to an embodiment of the present invention. Figure 6 For example, for the data to be processed in the test, the parameter selection methods also include:

[0033] Step S30: Test the data processing time of the first layer of data to be processed using one of the access strategies' access parameter combinations.

[0034] Step S31: Determine if the calculation time is short. If yes, proceed to step S32 and record the current calculation time. If no, proceed to step S33 and record the previous calculation time.

[0035] After steps S31 and S32, step S34 can be entered: change the access strategy and test the next set of access parameter combinations.

[0036] After testing the first layer of data to be processed, the process proceeds to step S35 via a second-direction step to test the data processing time of the second layer of data to be processed. This process continues until the data processing time of the last layer of data to be processed is tested. Then, the second-direction step is applied again, returning to step S30 to obtain the access parameter combination of the next access strategy from S34 to test the first layer of data to be processed. Steps S31 to S35 are then executed until all layers of parameter combinations have been tested. It should be noted that each layer of data to be processed goes through a loop from steps S31 to S35 (including determining if the processing time is short, recording the processing time, and testing the access parameter combination).

[0037] Beneficial effects of the embodiments

[0038] One of the beneficial effects of the present invention is that the parameter selection method and parameter selection system for real-time neural network computing architecture provided by the present invention can gradually find the best parameter combination during data processing, and has a progressive adjustment mechanism for immediate optimization, thus effectively improving computing efficiency.

[0039] The above-disclosed content is only a preferred and feasible embodiment of the present invention, and is not intended to limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made based on the content of the present invention specification and drawings are included in the scope of the patent application of the present invention.

[0040] Explanation of reference numerals in the attached figures

[0041] 1: Parameter selection system for real-time neural network computing architecture

[0042] 10: Computing device

[0043] 12: Target Real-Time Neural Network Computation Architecture

[0044] 120: Memory

[0045] 122: Data Access Circuit

[0046] 124: Processing Component Circuit

[0047] MC: Memory Controller

[0048] BS: Bus

[0049] 1200: Memory Block

[0050] 2: Convolutional Neural Network Model

[0051] STG: Access Strategy

[0052] S10, S11, S12, S20, S21, S30, S31, S32, S33, S34, S35: Steps

Claims

1. A parameter selection method for a real-time neural network computing architecture, comprising performing the following steps via a computing device of a parameter selection system: Obtain the combination of access strategies for the target real-time neural network computing architecture, where, The access strategy combination includes multiple access strategies, and the target real-time neural network computing architecture includes: Memory; Multiple data access circuits are connected to the memory via a bus, wherein each access strategy defines multiple access parameters used by the multiple data access circuits when accessing the memory; and Multiple processing channels are respectively connected to the multiple data access circuits and each includes multiple processing element circuits; By configuring the plurality of data access circuits to access the memory according to each access strategy, and simultaneously configuring the plurality of computing channels to perform a convolution operation for each access strategy, the optimized access strategy used to perform the convolution operation is obtained.

2. The parameter selection method according to claim 1, wherein, The memory includes multiple memory blocks, and multiple access parameters in each access strategy define the order in which the multiple data access circuits read the multiple memory blocks and the method of reading the multiple memory blocks through the bus.

3. The parameter selection method according to claim 2, wherein, The convolution operation process includes: The data to be processed, retrieved from the memory according to the corresponding access strategy, is input into multiple channels of the convolutional neural network model, each channel including multiple convolutional kernels; and Using each convolution kernel, convolution operations are performed based on the stride in the first direction and the stride in the second direction to generate multiple output data, while recording the corresponding data processing time.

4. The parameter selection method according to claim 3, wherein, Each convolution kernel has a kernel size, and the operation directions of the first direction step and the second direction step are different.

5. The parameter selection method according to claim 4, wherein, The step of obtaining the optimized access strategy used to perform the convolution operation includes, for each access strategy, obtaining the data processing time spent performing the convolution operation, and using the access parameter with the shortest data processing time as the optimized access strategy.

6. A parameter selection system for a real-time neural network computing architecture, comprising: Computing device; as well as The target real-time neural network computing architecture includes: Memory; Multiple data access circuits are connected to the memory via a bus; and Multiple processing channels are respectively connected to the multiple data access circuits and each includes multiple processing element circuits; The computing device is configured to perform the following steps: Obtain a combination of access strategies for a target real-time neural network computing architecture, wherein the combination of access strategies includes multiple access strategies, and each access strategy defines multiple access parameters used by the multiple data access circuits when accessing the memory; and By configuring the plurality of data access circuits to access the memory according to each access strategy, and simultaneously configuring the plurality of computing channels to perform a convolution operation for each access strategy, the optimized access strategy used to perform the convolution operation is obtained.

7. The parameter selection system according to claim 6, wherein, The memory includes multiple memory blocks, and multiple access parameters in each access strategy define the order in which the multiple data access circuits read the multiple memory blocks and the method of reading the multiple memory blocks through the bus.

8. The parameter selection system according to claim 7, wherein, The convolution operation process includes: The data to be processed, retrieved from the memory according to the corresponding access strategy, is input into multiple channels of the convolutional neural network model, each channel including multiple convolutional kernels; and Using each convolution kernel, convolution operations are performed based on the stride in the first direction and the stride in the second direction to generate multiple output data, while recording the corresponding data processing time.

9. The parameter selection system according to claim 8, wherein, Each convolution kernel has a kernel size, and the operation directions of the first direction step and the second direction step are different.

10. The parameter selection system according to claim 9, wherein, The step of obtaining the optimized access strategy used to perform the convolution operation includes, for each access strategy, obtaining the data processing time spent performing the convolution operation, and using the access parameter with the shortest data processing time as the optimized access strategy.

Images