Instruction scheduling system and method for neural network acceleration computation, device

Abstract

The application relates to the field of artificial intelligence chips, and provides an instruction scheduling system and method for neural network acceleration calculation and equipment. The system comprises a scheduler for acquiring target instructions stored in a storage space corresponding to the identification of the target instructions; the target instructions are analyzed, and data moving instructions and data operation instructions are generated based on the analysis results; a moving module moves feature map data of a first data block stored in a data storage to a cache space in N batches based on the data moving instructions; the scheduler issues a trigger command for executing the data operation instructions to a data operation unit whenever it is determined that a batch of feature map data is moved to the cache space; a plurality of data operation units perform configuration operations based on the parameters of the received data operation instructions; when the trigger command is received, the data operation instructions are executed based on the feature map data in the cache space, a calculation result is obtained based on the execution result, and the second data block is stored to the data storage. The hardware resource utilization rate is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence chips, and in particular to a neural network acceleration instruction scheduling system, method and device. BACKGROUND

[0002] In the current data processing mode, the stored data and the computing resources are separated, and the stored data is an adjunct to the instruction execution, which can be guided by instructions, read, parsed, executed and stored. Reading the instruction, then parsing and executing, reading the data is a serial process. Although pipeline design is adopted to compensate for the time loss of serial operation as much as possible, the utilization rate of hardware resources is still not high. SUMMARY

[0003] The present application provides a neural network acceleration instruction scheduling system, method and device to solve the problem of low utilization rate of hardware resources in the prior art and improve the utilization rate of hardware resources.

[0004] The present application provides a neural network acceleration instruction scheduling system, which comprises a scheduler, a moving module and a plurality of data operation units.

[0005] The scheduler is configured to obtain the identifier of the target instruction required by the acceleration computing task of the current layer of the neural network, and obtain the target instruction stored in the storage space corresponding to the identifier of the target instruction. The target instruction is parsed, and data moving instructions and data operation instructions are generated based on the parsing result. The parameters of the data operation instructions are configured and issued to the plurality of data operation units. The data moving instructions are issued to the moving module.

[0006] The moving module is configured to move the feature map data of a first data block stored in the data storage to the cache space in N batches based on the data moving instructions, where N is a positive integer greater than or equal to 1.

[0007] The scheduler is further configured to issue a trigger command to execute the data operation instructions to the data operation unit whenever it is determined that a batch of feature map data has been moved to the cache space.

[0008] The plurality of data operation units are configured to perform configuration operations based on the received parameter configuration of the data operation instructions. When the trigger command is received, the data operation instructions are executed based on the feature map data in the cache space, and the calculation result of the current layer of the neural network is obtained based on the execution result and stored in the data storage as a second data block.

[0009] The application provides a neural network acceleration instruction scheduling system, the cache space comprises a first subspace and a second subspace; the neural network acceleration instruction scheduling system further comprises a first configuration module corresponding to the first subspace and a second configuration module corresponding to the second subspace;

[0010] The moving module is specifically used for moving feature map data to the first subspace and the second subspace alternately.

[0011] The scheduler is specifically used for alternately issuing the parameter configuration of the data operation instruction to the first configuration module and the second configuration module; when the moving module moves feature map data to the first subspace, the parameter configuration of the data operation instruction is issued to the first configuration module; when the moving module moves feature map data to the second subspace, the parameter configuration of the data operation instruction is issued to the second configuration module.

[0012] The plurality of data operation units are specifically used for alternately receiving the parameter configuration of the first configuration module and the second configuration module; in the case that the configuration operation is performed based on the received parameter configuration of the first configuration module, when the trigger command is received, the data operation instruction is executed based on the feature map data in the first subspace; in the case that the configuration operation is performed based on the received parameter configuration of the second configuration module, the data operation instruction is executed based on the feature map data in the second subspace.

[0013] In the case that the size of the storage space required by the first data block is greater than the size of the first subspace and the second subspace, the value of N is greater than 1.

[0014] The application provides a neural network acceleration instruction scheduling system, the target instruction comprises:

[0015] The information of the first data block;

[0016] The information of the second data block;

[0017] In the case that the target instruction is not the last instruction, the target instruction further comprises the identification of the next instruction of the target instruction, and the next instruction is an instruction required by an acceleration calculation task of the next layer of the neural network;

[0018] The scheduler is specifically used for issuing the information of the first data block and the information of the second data block as parameter configuration to the plurality of data operation units; after the target instruction is executed, the next instruction stored in the storage space corresponding to the identification of the next instruction is acquired.

[0019] According to the instruction scheduling system for neural network acceleration calculation provided by the application, the moving module is specifically configured to, in the case that the target instruction is executed and the next instruction is obtained, switch to moving the feature map data of the second data block to one of the first subspace and the second subspace which is not occupied by the feature map data moved by the last target instruction.

[0020] According to the instruction scheduling system for neural network acceleration calculation provided by the application, the instruction scheduling system further comprises M instruction registers, the M instruction registers are used for storing the identifiers of the first instructions of the acceleration calculation tasks of the M neural networks, and M is a positive integer greater than 1.

[0021] The scheduler is further configured to schedule the instructions of the acceleration calculation tasks of the M neural networks in a polling manner based on the identifiers of the first instructions of the acceleration calculation tasks of the M neural networks stored in the M instruction registers.

[0022] According to the instruction scheduling system for neural network acceleration calculation provided by the application, the scheduler is further configured to determine, based on the key value in the hash table, whether the first data block required by the target instruction is ready in the data storage, and activate the target instruction when the first data block is ready.

[0023] The application further provides an instruction scheduling method based on the instruction scheduling system for neural network acceleration calculation.

[0024] The scheduler obtains the identifier of the target instruction required by the acceleration calculation task of the current layer of the neural network, and obtains the target instruction stored in the storage space corresponding to the identifier of the target instruction; the target instruction is analyzed, and data moving instructions and data operation instructions are generated based on the analysis result; the parameters of the data operation instructions are configured and issued to the plurality of data operation units; and the data moving instructions are issued to the moving module.

[0025] The moving module moves the feature map data of the first data block stored in the data storage to the cache space in N batches based on the data moving instructions, and N is a positive integer greater than or equal to 1.

[0026] The scheduler issues a trigger command for executing the data operation instructions to the data operation unit whenever it is determined that a batch of feature map data is moved to the cache space.

[0027] The plurality of data operation units perform configuration operations based on parameters of the received data operation instructions; when the trigger command is received, the data operation instructions are executed based on the feature map data in the cache space, and the calculation result of the current layer of the neural network is obtained based on the execution result and stored in the data storage as a second data block.

[0028] The application further provides a neural network accelerator comprising the instruction scheduling system for neural network acceleration calculation.

[0029] The application further provides an electronic device comprising the neural network accelerator.

[0030] The instruction scheduling system for neural network acceleration calculation can make data storage and calculation integrated, realize efficient calculation under data driving, thereby maximally utilizing hardware resources, avoiding waste of hardware resources, and improving the utilization rate of hardware resources. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0032] Figure 1 is one of the structural schematic diagrams of the instruction scheduling system for neural network acceleration calculation provided by the application;

[0033] Figure 2 is another structural schematic diagram of the instruction scheduling system for neural network acceleration calculation provided by the application;

[0034] Figure 3 is a flow schematic diagram of the instruction scheduling method provided by the application. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0036] The following combination Figures 1 to 2 The present invention describes the instruction scheduling system for accelerating computing of neural networks.

[0037] The present invention provides an instruction scheduling system for accelerating calculation of neural network, such as Figure 1 As shown, it includes a scheduler 110, a moving module 120 and multiple data operation units 130:

[0038] The scheduler 110 is configured to obtain an identifier of a target instruction required for an accelerated computing task of a current layer of a neural network, and obtain the target instruction stored in a storage space corresponding to the identifier of the target instruction; parse the target instruction, and generate a data movement instruction and a data operation instruction based on the parsing result; send parameter configurations of the data operation instruction to the multiple data operation units 130; and send the data movement instruction to the movement module 120;

[0039] The moving module 120 is configured to move the feature map data of the first data block stored in the data memory 140 to the cache space 150 in N batches based on the data moving instruction, where N is a positive integer greater than or equal to 1;

[0040] The scheduler 110 is further configured to send a trigger command for executing the data operation instruction to the data operation unit 130 whenever it is determined that a batch of feature map data has been moved to the cache space 150;

[0041] The multiple data operation units 130 are used to perform configuration operations based on the parameter configuration of the received data operation instructions; when the trigger command is received, the data operation instructions are executed based on the feature map data in the cache space 150, and the calculation results of the current layer of the neural network are obtained based on the execution results and stored in the data memory 140 as a second data block.

[0042] The solution of this embodiment can be applied to artificial intelligence chips in image processing scenarios to achieve neural network acceleration on the inference side.

[0043] In actual application, for a neural network requiring acceleration calculation, a corresponding instruction can be set for each network layer of the neural network to form an instruction sequence that can be scheduled by the scheduler 110. For example, if the number of network layers of a neural network is 10, 10 instructions can be set according to the order of acceleration calculation of the 10 network layers of the neural network, and the 10 instructions can form an instruction sequence.

[0044] Each instruction in the instruction sequence has a corresponding identifier, and the content in the instruction is stored in the storage space corresponding to the identifier of the instruction. In actual application, an instruction register can be set, and the identifier of the first instruction of the instruction sequence can be stored in the instruction register.

[0045] The scheduler 110 can read each instruction requiring execution in sequence according to the instruction sequence and issue execution, thereby completing the acceleration calculation task of the neural network. Specifically, the identifier of the first instruction of the acceleration calculation task of the neural network can be first read from the instruction register, then the first instruction stored in the storage space corresponding to the identifier of the first instruction can be obtained, after the execution of the first instruction is completed, the identifier of the next instruction is read, the next instruction stored in the storage space corresponding to the identifier of the next instruction is obtained and executed, and so on, until the identifier of the last instruction is read, the last instruction stored in the storage space corresponding to the identifier of the last instruction is obtained and executed.

[0046] The above target instruction is the instruction currently requiring execution. The scheduler 110 can obtain the identifier of the target instruction required by the acceleration calculation task of the current layer (i.e., the current network layer) of the neural network, and obtain the target instruction stored in the storage space corresponding to the identifier of the target instruction; the target instruction is parsed, and a data movement instruction and a data operation instruction are generated based on the parsing result. Each data operation unit 130 corresponds to a different data operation instruction (i.e., a sub-instruction), so that the decomposition of the target instruction is realized. In related technologies, this decomposition process is generally completed offline by a compiler, so that the number of system instructions is large, the instruction load is heavy, the transmission of the instructions becomes a system bottleneck, and the calculation module is in a waiting state. In the present embodiment, the decomposition of the target instruction to each data operation unit can be completed by the scheduler, thereby avoiding the problem.

[0047] The scheduler 110 issues the parameter configuration of the data operation instruction to the plurality of data operation units 130, and each data operation unit 130 receives the parameter configuration of the corresponding data operation instruction. At this time, the plurality of data operation units 130 do not immediately start to execute the data operation instruction, but perform configuration operation based on the parameter configuration of the data operation instruction issued by the scheduler 110.

[0048] The scheduler 110 issues the data migration instruction to the migration module 120. After receiving the data migration instruction, the migration module 120 can migrate the feature map data of the first data block stored in the data storage 140 to the cache space 150 in N batches. The data storage 140 can be, but is not limited to, a double data rate synchronous dynamic random access memory (DDR SDRAM), which can be referred to as DDR for short. The migration module 120 can be a direct memory access (DMA) controller.

[0049] When the feature map data of the first data block is less, the size of the storage space required by the feature map data of the first data block is smaller than the size of the cache space 150, and the cache space 150 is sufficient to cache all the feature map data of the first data block. The migration module 120 can migrate all the feature map data of the first data block to the cache space 150 in 1 batch, that is, N = 1. When the migration module 120 migrates all the feature map data of the first data block to the cache space 150, it means that the feature map data in the cache space 150 is accurate. At this time, the scheduler 110 can issue a trigger command to start executing the data operation instruction to the plurality of data operation units 130, and the plurality of data operation units 130 start executing the data operation instruction, thereby completing the acceleration calculation task of the current layer of the neural network. The trigger command is a command for triggering the data operation instruction to start execution.

[0050] The data block refers to a three-dimensional block formed by n-channel r-row c-column feature map data. The feature map data in the data block is stored according to a data arrangement order. The data arrangement order of the feature map data of the data block is obtained in the following manner:

[0051] The data block is divided into m1*m2*m3 sub-three-dimensional blocks in units of a preset sub-three-dimensional block; when part of the data of the data block is insufficient to form a sub-three-dimensional block, data padding is performed on the basis of the part of the data of the data block to obtain a complete sub-three-dimensional block; the size of the sub-three-dimensional block is n'-channel r'-row c'-column, wherein n', r', and c' are positive integers, n' is not greater than n, r' is not greater than r, and c' is not greater than c, the value of m1 is the integer part of the ratio of n to n', the value of m2 is the integer part of the ratio of r to r', and the value of m3 is the integer part of the ratio of c to c';

[0052] According to the scanning mode corresponding to the data block, a sub-stereoscopic block currently needed to be scanned is selected from each sub-stereoscopic block in turn to obtain a first arrangement order, and the following steps are performed on the currently selected sub-stereoscopic block: data in the sub-stereoscopic block is scanned according to the scanning mode corresponding to the sub-stereoscopic block to obtain a second arrangement order. The data arrangement order of the data block is that each sub-stereoscopic block in the data block is stored according to the first arrangement order, and the data in the sub-stereoscopic block is stored according to the second arrangement order.

[0053] The scanning mode corresponding to the data block and the scanning mode corresponding to the sub-stereoscopic block can be the same or different. In actual application, the scanning mode that can be used can be feature map priority scanning, or can be channel priority scanning, or can be hybrid scanning. A certain pixel point in the data block is denoted as n.r.c, which represents the pixel point of the rth row and the cth column of the nth channel. The last row, the last column or the last channel can be marked by end. Based on this, different scanning modes are introduced below.

[0054] Feature map priority scanning: when scanning data, the feature map data of one channel can be scanned first, and then the feature map data of the next channel is scanned, that is, the scanning mode of first column direction, then row direction, and then channel direction, or the scanning mode of first row direction, then column direction, and then channel direction. For example, data is scanned in the following scanning order:

[0055] The first row of the first channel feature map data: 1.1.1, 1.1.2, 1.1.3, …, 1.1.end, the second row of the first channel feature map data: 1.2.1, 1.2.2, …, 1.2.end, …, the last row of the first channel feature map data: 1.end.1, 1.end.2, …, 1.end.end, the first row of the second channel feature map data: 2.1.1, 2.1.2, 2.1.3, …, 2.1.end, the second row of the second channel feature map data: 2.2.1, 2.2.2, …, 2.2.end, …, the last row of the second channel feature map data: 2.end.1, 2.end.2, …, 2.end.end, …, the last pixel point of the last channel feature map data: end.end.end.

[0056] Channel priority scanning: when scanning data, the scanning mode of first channel direction, then column direction, and then row direction, or the scanning mode of first channel direction, then row direction, and then column direction can also be used. Specifically, the first row of pixel point data of the first channel is scanned first, then the first row of pixel point data of other channels is scanned, then the second row of pixel point data of all channels is scanned, and so on until all pixel point data is scanned. For example, data is scanned in the following scanning order:

[0057] The first column of the first row of the feature map data under all channels: 1.1.1, 2.1.1, 3.1.1,..., end.1.1, the second column of the first row of the feature map data under all channels: 1.1.2, 2.1.2, 3.1.2,..., end.1.2,..., the last column of the first row of the feature map data under all channels: 1.1.end, 2.1.end,..., end.1.end, the first column of the second row of the feature map data under all channels: 1.2.1, 2.2.1,..., end.2.1, the second column of the second row of the feature map data under all channels: 1.2.2, 2.2.2,..., end.2.2,..., the last column of the last row of the feature map data under all channels: 1.end.end,..., end.end.end.

[0058] Mixed scanning: when scanning data, the scanning mode can be first in the column direction, then in the channel direction, and then in the row direction, or the scanning mode can be first in the row direction, then in the channel direction, and then in the column direction. Specifically, the first row of pixel point data in the first channel is scanned first, then the first row of pixel point data in other channels is scanned, and then the second row of pixel point data in all channels is scanned, until all pixel points are scanned. For example, the data is scanned in the following scanning order: the first row of feature map data in the first channel: 1.1.1, 1.1.2, 1.1.3,..., 1.1.end, the first row of feature map data in the second channel: 2.1.1, 2.1.2,..., 2.1.end,..., the first row of feature map data in the last channel: end.1.1, end.1.2,..., end.1.end, the second row of feature map data in the first channel: 1.2.1, 1.2.2, 1.2.3,..., 1.2.end, the second row of feature map data in the second channel: 2.2.1, 2.2.2,..., 2.2.end,..., the last row of feature map data in the last channel: end.end.1, end.end.2,..., end.end.end.

[0059] Therefore, the first data block, which is obtained by the above data arrangement order, is the data block required for the accelerated computing task of the current layer of the neural network, and the data block is a rearranged (i.e., data rearrangement) data block for hardware computing. The first data block is a three-dimensional block formed by N1 channels, R1 rows, and C1 columns of feature map data. The feature map data in the first data block is stored according to the first data arrangement order. The first data arrangement order is the data arrangement order adopted by the first data block.

[0060] In the case that the feature map data of the first data block is relatively large, the required storage space of the feature map data of the first data block is greater than or equal to the size of the cache space 150, and the cache space 150 is insufficient to cache all the feature map data of the first data block. The moving module 120 can move the feature map data of the first data block into the cache space 150 in more than two batches, that is, N is greater than 1. In this case, the plurality of data operation units 130 can execute the data operation instruction through the processing of more than two batches of feature map data, and the scheduler 110 can issue a trigger command for executing the data operation instruction for each batch of feature map data. For example, the moving module 120 can move the feature map data of the first data block into the cache space 150 in three batches, and then the scheduler 110 can issue a trigger command three times. The moving module 120 can feed back a signal of moving completion to the scheduler 110 after moving a batch of feature map data. The scheduler 110 can determine that the moving module 120 has moved a batch of feature map data into the cache space 150 based on the feedback signal of moving completion. Accordingly, when the scheduler 110 determines that a batch of feature map data has been accurately moved into the cache space 150, the scheduler 110 can issue a trigger command for executing the data operation instruction to the plurality of data operation units 130, and the plurality of data operation units 130 execute the data operation instruction based on the feature map data in the cache space 150 after receiving the trigger command, thereby finally completing the acceleration calculation task of the current layer of the neural network.

[0061] In addition, the moving module 120 can also move the weight data from the data storage 140 to the corresponding weight data cache space, and move the bias data to the corresponding bias data cache space.

[0062] After obtaining the calculation result of the current layer of the neural network, the second data block is stored in the data storage 140.

[0063] The second data block is a three-dimensional block formed by N2 channels, R2 rows and C2 columns of feature map data. The second data block is obtained in the above data arrangement order. The feature map data in the second data block is stored according to the second data arrangement order. The second data arrangement order is the data arrangement order adopted by the second data block.

[0064] Each data operation unit 130 in the plurality of data operation units 130 is used to perform a set data operation, which can be set according to actual needs.

[0065] In the implementation, the output of the plurality of data operation units 130 can be stored in the data storage 140 in the second data block by the moving module 120.

[0066] In the embodiment, in the process of neural network acceleration calculation, after the data movement instruction and the data operation instruction are generated for the target instruction required by the acceleration calculation task of the current layer of the neural network, the data operation unit 130 does not directly execute the data operation instruction, but after the batch of feature map data is prepared by the movement module 120, the scheduler 110 issues a trigger command to make the data operation unit 130 execute the data operation instruction. In this way, the required calculation operation instruction can be configured in advance, and after the data is ready, the data is calculated in parallel at high speed according to the configuration requirements, so as to maximize the utilization of hardware resources and avoid waste of hardware resources, thereby improving the utilization rate of hardware resources.

[0067] In the example embodiment, as shown in Figure 2 The instruction scheduling system for neural network acceleration calculation further includes a first configuration module 161 corresponding to the first subspace 151 and a second configuration module 162 corresponding to the second subspace 152.

[0068] The movement module 120 is specifically configured to alternately move feature map data to the first subspace 151 and the second subspace 152.

[0069] The scheduler 110 is specifically configured to alternately issue parameter configurations of the data operation instruction to the first configuration module 161 and the second configuration module 162. When the movement module 120 moves feature map data to the first subspace 151, the parameter configurations of the data operation instruction are issued to the first configuration module 161. When the movement module 120 moves feature map data to the second subspace 152, the parameter configurations of the data operation instruction are issued to the second configuration module 162.

[0070] The plurality of data operation units 130 are specifically configured to alternately receive parameter configurations of the first configuration module 161 and the second configuration module 162. In the case of performing configuration operation based on the received parameter configurations of the first configuration module 161, when the trigger command is received, the data operation instruction is executed based on the feature map data in the first subspace 151. In the case of performing configuration operation based on the received parameter configurations of the second configuration module 162, the data operation instruction is executed based on the feature map data in the second subspace 152.

[0071] In this embodiment, in order to further ensure efficient data processing, a ping-pong design structure is provided. Specifically, the cache space 150 is set to include two subspaces, i.e., a first subspace 151 and a second subspace 152. The first subspace 151 and the second subspace 152 are used to alternately store feature map data, so as to realize ping-pong between the first subspace 151 and the second subspace 152. A corresponding first configuration module 161 is set for the first subspace 151, and a corresponding second configuration module 162 is set for the second subspace 152. The first configuration module 161 and the second configuration module 162 can be used to alternately receive parameter configurations of data operation instructions, so as to realize ping-pong between the first configuration module 161 and the second configuration module 162.

[0072] In actual application, the moving module 120 uses whether the first subspace and the second space are released or idle as a marker to alternately load data, and the ping-pong operation completes seamless docking of data. Based on two adjacent clock cycles, one of the first subspace 151 and the second subspace 152 caches feature map data in the previous clock cycle, and the other caches feature map data in the next clock cycle. In this way, in each clock cycle, there is a subspace that has already prepared feature map data. The scheduler 110 can continue to issue the next trigger command in the next clock cycle after issuing the trigger command in the previous clock cycle, thereby greatly improving the data processing efficiency.

[0073] In an example embodiment, in the case where the size of the storage space required by the first data block is greater than the size of the first subspace 151 and the second subspace 152, N is greater than 1. In this way, the moving module 120 can move the feature map data of the first data block into the cache space 150 by more than two batches, and the feature map data in the more than two batches can be alternately moved into the first subspace 151 and the second subspace 152. For example, N=4, the first batch of feature map data is moved to the first subspace 151, the second batch of feature map data is moved to the second subspace 152, the third batch of feature map data is moved to the first subspace 151, and the fourth batch of feature map data is moved to the second subspace 152.

[0074] In an example embodiment, the target instruction can include:

[0075] information of the first data block, which includes a storage starting address of the first data block, a size of the first data block, a data organization form of the first data block, a data operation mode corresponding to the first data block, such as a storage starting address of weight data corresponding to the first data block, a data organization form of the weight data, a storage starting address of bias data, a size parameter of a convolution kernel, a step of convolution, a padding mode of convolution, a nonlinear activation mode, and a pooling mode, etc.

[0076] The information of the second data block includes a storage start address of the second data block, a size of the second data block, and a data organization form of the second data block.

[0077] In a case where the target instruction is not the last instruction, the target instruction further includes an identification of a next instruction of the target instruction, and the next instruction is an instruction required by an acceleration calculation task of a next layer of the neural network.

[0078] The scheduler 110 is specifically configured to configure and issue the information of the first data block and the information of the second data block as parameters to the plurality of data operation units 130; and after the target instruction is executed, the next instruction stored in a storage space corresponding to the identification of the next instruction is obtained.

[0079] The data organization form can be a data arrangement order in storage.

[0080] Exemplarily, the target instruction includes the following fields:

[0081] next id: indicating an identification of a next instruction;

[0082] in-start: indicating a storage start address of a first data block of input;

[0083] in-len: indicating a size of the first data block of input;

[0084] in-fmt: indicating a data organization form of the first data block of input;

[0085] out-start: indicating a storage start address of a second data block of output;

[0086] out-len: indicating a size of the second data block of output;

[0087] out-fmt: indicating a data organization form of the second data block of output;

[0088] w-start: indicating a storage start address of weight data corresponding to the first data block of input;

[0089] w-fmt: indicating a data organization form of the weight data;

[0090] b-start: indicating a storage start address of bias data;

[0091] conv: indicating a size parameter of a convolution kernel;

[0092] stride: indicating a stride of convolution;

[0093] pad: represents the padding mode of convolution;

[0094] fun: represents the nonlinear activation mode;

[0095] pool: represents the pooling mode.

[0096] In this embodiment, the target instruction can carry information of the first data block and information of the second data block, so that the data moving instruction and the data operation instruction can be accurately generated to instruct the acceleration calculation.

[0097] In an example embodiment, the moving module is specifically configured to, in a case where the target instruction is executed and the next instruction is obtained, switch to moving the feature map data of the second data block to one of the first subspace and the second subspace which is not occupied by the last feature map data moved under the target instruction.

[0098] The last feature map data moved under the target instruction is the last batch of feature map data of the first data block moved under the target instruction.

[0099] For the entire neural network, in order to further improve the calculation efficiency, for the adjacent target instruction and the next instruction in the instruction sequence, in a case where the target instruction is executed and the next instruction is obtained, the first batch of feature map data of the second data block under the next instruction is moved to a subspace different from the subspace occupied by the last batch of feature map data under the target instruction, so that after the target instruction is executed, the data of the next instruction is already prepared, and thus the next instruction can be executed with zero time delay, thereby improving the data processing efficiency.

[0100] For example, in a case where the sizes of the storage spaces required by the second data block and the first data block are both less than or equal to the sizes of the first subspace 151 and the second subspace 152, N is equal to 1; the moving module 120 is specifically configured to, in a case where the target instruction is obtained, move the feature map data of the first data block to one of the first subspace 151 and the second subspace 152, and in a case where the next instruction is obtained, switch to moving the feature map data of the second data block to the other one of the first subspace 151 and the second subspace 152. In this way, for the adjacent target instruction and the next instruction in the instruction sequence, the corresponding feature map data is moved to different subspaces in a ping-pong manner, so that after the target instruction is executed, the data of the next instruction is already prepared, and thus the next instruction can be executed with zero time delay, thereby improving the data processing efficiency.

[0101] In an example embodiment, the instruction scheduling system for accelerating neural network computation can further comprise M instruction registers, where M is a positive integer greater than 1, for storing the first instruction identifiers of the M neural network acceleration tasks.

[0102] The scheduler 110 is further configured to schedule the instructions of the M neural network acceleration tasks in a round-robin manner based on the first instruction identifiers of the M neural network acceleration tasks stored in the M instruction registers.

[0103] In the round-robin manner, one neural network acceleration task is served in one time period, and different neural network acceleration tasks are served in different time periods. Alternatively, in the round-robin manner, after one neural network acceleration task is executed, the next neural network acceleration task is checked until all the neural network acceleration tasks are executed.

[0104] In practical applications, the M neural network acceleration tasks can be scheduled in parallel, so as to further make full use of hardware resources. Each neural network acceleration task has a corresponding instruction sequence, and the first instruction identifier of the instruction sequence is stored in the corresponding instruction register. The scheduler 110 can schedule the instruction sequence of each neural network acceleration task in a round-robin manner, so as to improve the acceleration efficiency.

[0105] In an example embodiment, the scheduler 110 is further configured to determine whether the first data block required by the target instruction is ready in the data storage 140 based on the key value in the hash table, and activate the target instruction when it is ready.

[0106] The hash table, also known as a hash table, includes keys and corresponding key values. The key value in the hash table can represent whether the feature map data of the first data block required by the target instruction is ready.

[0107] In practical applications, before the target instruction is executed, the scheduler 110 can check whether the feature map data to be processed by the target instruction is ready. If it is not ready, the target instruction is suspended and enters a dormant state. When the feature map data enters the data storage 140 (such as on-chip DDR) from the outside, the target instruction is activated. Whether the data is ready can be embodied by a hash table, so as to maintain the validity of the data. In this way, when the feature map data to be processed is not ready, the execution of the target instruction is suspended, which can prevent misoperation.

[0108] The instruction scheduling method provided by the present application is described below. The instruction scheduling method described below can be mutually referred to the instruction scheduling system for accelerating neural network computation described above.

[0109] The embodiment provides an instruction scheduling method based on the instruction scheduling system of the neural network acceleration calculation provided in any of the above embodiments, which comprises the following steps: Figure 3

[0110] In step 301, the scheduler 110 acquires the identification of the target instruction required by the acceleration calculation task of the current layer of the neural network, and acquires the target instruction stored in the storage space corresponding to the identification of the target instruction; the target instruction is analyzed, and data moving instructions and data operation instructions are generated based on the analysis result; the data operation instructions are issued to the plurality of data operation units 130; and the parameter configuration of the data moving instructions is issued to the moving module 120.

[0111] In step 302, the moving module 120 moves the feature map data of the first data block stored in the data storage 140 to the cache space 150 in N batches based on the data moving instructions, where N is a positive integer greater than or equal to 1.

[0112] In step 303, the scheduler 110 issues a trigger command for executing the data operation instructions to the data operation unit 130 whenever it is determined that a batch of feature map data is moved to the cache space 150.

[0113] In step 304, the plurality of data operation units 130 perform configuration operations based on the parameter configuration of the received data operation instructions; when the trigger command is received, the data operation instructions are executed based on the feature map data in the cache space 150, the calculation result of the current layer of the neural network is obtained based on the execution result, and the second data block is stored to the data storage 140.

[0114] In the exemplary embodiment, the cache space 150 comprises a first subspace 151 and a second subspace 152; the instruction scheduling system of the neural network acceleration calculation further comprises a first configuration module 161 corresponding to the first subspace 151 and a second configuration module 162 corresponding to the second subspace 152.

[0115] The moving module 120 alternately moves feature map data to the first subspace 151 and the second subspace 152.

[0116] ​The scheduler 110 alternately issues parameter configurations of the data operation instruction to the first configuration module 161 and the second configuration module 162; wherein, when the moving module 120 moves feature map data to the first subspace 151, the parameter configurations of the data operation instruction are issued to the first configuration module 161; when the moving module 120 moves feature map data to the second subspace 152, the parameter configurations of the data operation instruction are issued to the second configuration module 162.

[0117] The plurality of data operation units 130 alternately receive parameter configurations of the first configuration module 161 and the second configuration module 162; in the case of performing configuration operation based on the received parameter configurations of the first configuration module 161, when the trigger command is received, the data operation instruction is executed based on feature map data in the first subspace 151; in the case of performing configuration operation based on the received parameter configurations of the second configuration module 162, the data operation instruction is executed based on feature map data of the second subspace 152.

[0118] In an exemplary embodiment, in the case where the size of the storage space required by the first data block is greater than the size of the first subspace 151 and the second subspace 152, N is greater than 1.

[0119] In an exemplary embodiment, the target instruction includes:

[0120] Information of the first data block;

[0121] Information of the second data block;

[0122] In the case where the target instruction is not the last instruction, the target instruction further includes an identification of the next instruction of the target instruction, and the next instruction is an instruction required by an acceleration calculation task of the next layer of the neural network;

[0123] The scheduler 110 issues the information of the first data block and the information of the second data block as parameter configurations to the plurality of data operation units 130; after the target instruction is executed, the next instruction stored in the storage space corresponding to the identification of the next instruction is obtained.

[0124] In an exemplary embodiment, in the case where the moving module executes the target instruction and obtains the next instruction, the moving module switches to moving feature map data of the second data block to one of the first subspace and the second subspace which is not occupied by feature map data last moved by the target instruction.

[0125] In the example embodiment, the instruction scheduling system for neural network acceleration computing further comprises M instruction registers for storing the first instruction identifiers of the M neural network acceleration computing tasks, M being a positive integer greater than 1.

[0126] The scheduler 110 schedules the instructions of the M neural network acceleration computing tasks in a round-robin manner based on the first instruction identifiers of the M neural network acceleration computing tasks stored in the M instruction registers.

[0127] In the example embodiment, the scheduler 110 determines whether the first data block required by the target instruction in the data storage 140 is ready based on the key value in the hash table, and activates the target instruction when ready.

[0128] The present application also provides a neural network accelerator comprising the instruction scheduling system for neural network acceleration computing as provided in any of the above embodiments.

[0129] The present application also provides an electronic device comprising the neural network accelerator as provided in the above embodiments.

[0130] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0131] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in terms of the contribution to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in the various embodiments or some parts of the embodiments.

[0132] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An instruction scheduling system for accelerating computation of neural networks, characterized in that, The scheduling device, the moving module and the plurality of data operation units are included. The scheduling device is configured to obtain an identifier of a target instruction required by an acceleration computing task of a current layer of a neural network, and obtain the target instruction stored in a storage space corresponding to the identifier of the target instruction; analyze the target instruction, and generate a data moving instruction and a data operation instruction based on an analysis result; The scheduling device is configured to configure parameters of the data operation instruction to the plurality of data operation units; The scheduling device is configured to configure the data moving instruction to the moving module; The moving module is configured to move feature map data of a first data block stored in a data storage to a cache space in N batches based on the data moving instruction, where N is a positive integer greater than or equal to 1; The scheduling device is further configured to issue a trigger command for executing the data operation instruction to the data operation units whenever it is determined that a batch of feature map data is moved to the cache space; The plurality of data operation units are configured to perform configuration operations based on the received parameter configuration of the data operation instruction; When the trigger command is received, the data operation instruction is executed based on the feature map data in the cache space, and a computing result of the current layer of the neural network is obtained based on an execution result and stored in the data storage as a second data block; The cache space includes a first subspace and a second subspace; the instruction scheduling system for neural network acceleration computing further includes a first configuration module corresponding to the first subspace and a second configuration module corresponding to the second subspace; The moving module is specifically configured to alternately move feature map data to the first subspace and the second subspace; The scheduling device is specifically configured to alternately configure the parameter configuration of the data operation instruction to the first configuration module and the second configuration module; when the moving module moves feature map data to the first subspace, the parameter configuration of the data operation instruction is configured to the first configuration module; when the moving module moves feature map data to the second subspace, the parameter configuration of the data operation instruction is configured to the second configuration module; The plurality of data operation units are specifically configured to alternately receive the parameter configuration of the first configuration module and the second configuration module; when the parameter configuration of the first configuration module is received, the data operation instruction is executed based on the feature map data in the first subspace if configuration operations are performed based on the received parameter configuration of the first configuration module; when the parameter configuration of the second configuration module is received, the data operation instruction is executed based on the feature map data in the second subspace if configuration operations are performed based on the received parameter configuration of the second configuration module.

2. The instruction scheduling system for neural network acceleration computing of claim 1, wherein, In a case where the size of the storage space required by the first data block is greater than the size of the first subspace and the second subspace, N is greater than 1.

3. The instruction scheduling system for neural network acceleration computing of claim 1, wherein, The target instruction includes: Information of the first data block; Information of the second data block; In a case where the target instruction is not the last instruction, the target instruction further includes an identifier of a next instruction of the target instruction, and the next instruction is an instruction required by an acceleration computing task of a next layer of the neural network; The scheduler is specifically configured to configure and issue the information of the first data block and the information of the second data block as parameters to the plurality of data operation units; and obtain the next instruction stored in a storage space corresponding to an identifier of the next instruction after the target instruction is executed.

4. The instruction scheduling system for neural network acceleration computing of claim 3, wherein, The moving module is specifically configured to, in a case where the target instruction is executed and the next instruction is obtained, switch to moving the feature map data of the second data block to one of the first subspace and the second subspace which is not occupied by the feature map data moved last by the target instruction.

5. The instruction scheduling system for neural network acceleration computing of claim 1, wherein, The neural network accelerator further comprises M instruction registers, the M instruction registers being configured to store identifiers of first instructions of M neural network acceleration tasks, M being a positive integer greater than 1. The scheduler is further configured to schedule the instructions of the M neural network acceleration tasks in a polling manner based on the identifiers of the first instructions of the M neural network acceleration tasks stored in the M instruction registers.

6. The instruction scheduling system for neural network accelerated computation of claim 1, wherein, The scheduler is further configured to determine, based on a key value in a hash table, whether the first data block required by the target instruction is ready in the data storage, and activate the target instruction when the first data block is ready.

7. An instruction scheduling method for an instruction scheduling system based on neural network accelerated computing according to any one of claims 1 to 6, characterized in that, The neural network accelerator comprises: The scheduler obtains an identifier of a target instruction required by an acceleration task of a current layer of a neural network, and obtains the target instruction stored in a storage space corresponding to the identifier of the target instruction; analyzes the target instruction, and generates a data moving instruction and a data operation instruction based on an analysis result; The scheduler configures and issues parameters of the data operation instruction to the plurality of data operation units; The scheduler issues the data moving instruction to the moving module; The moving module moves feature map data of a first data block stored in a data storage to a cache space in N batches based on the data moving instruction, N being a positive integer greater than or equal to 1. The scheduler issues a trigger command for executing the data operation instruction to the data operation units whenever it is determined that a batch of feature map data is moved to the cache space. The plurality of data operation units perform configuration operations based on the parameters of the received data operation instruction. When the trigger command is received, the data operation instruction is executed based on the feature map data in the cache space, and a calculation result of the current layer of the neural network is obtained based on an execution result and stored in the data storage as a second data block.

8. A neural network accelerator, comprising: The neural network acceleration instruction scheduling system comprises any one of the neural network accelerators according to claims 1 to 6.

9. An electronic device, comprising: The neural network accelerator comprises the neural network accelerator according to claim 8.

Images