A neural network acceleration coprocessor, processing system and processing method

Abstract

A neural network acceleration coprocessor, processing system and processing method, the system includes a coprocessor, a main processor and a memory; the main processor is used to send extended instructions; the memory is used to store data; the coprocessor is used to receive the extended instructions sent by the main processor, read input data from the memory according to the received extended instructions, perform neural network calculations on the input data to obtain output data, and store the output data in the memory. By setting the coprocessor, the coprocessor processes the time-consuming operations in the convolutional neural network, and the main processor controls the coprocessor to perform neural network calculations on the input data through extended instructions, thereby reducing the utilization rate of the CPU. Compared with pure software, the convolution operation efficiency is improved by more than 20 times.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and chip design, and in particular to a neural network acceleration coprocessor, a processing system and a processing method. Background Art

[0002] The convolution layer is the core computing module in the convolutional neural network. Usually, the convolution layer calculation accounts for more than 90% of the entire convolutional network calculation. Figure 1 The figure shows the process of calculating an output feature map by convolution. Each input feature map corresponds to a convolution kernel. The dashed boxes of different colors in the input map correspond to different outputs. Each output is obtained by multiplying and summing the convolution kernels at the same position of different input maps. Each output is the result of local information processing of the input, reflecting the local feature information. The same input feature map uses the same convolution kernel for feature processing, which is the weight sharing mechanism in the convolution network. The convolution layer can complete the local feature extraction of the input feature map, and complete the processing of the entire input feature map by sliding the convolution kernel on the input feature map.

[0003] The calculation formula of the convolutional layer is as shown in Formula 1.

[0004]

[0005] Among them, out(f0,x,y) represents the value corresponding to the x,y position in the f0th output feature map, W is the convolution kernel weight matrix, and in represents the input feature map, b represents the bias of the convolution layer, K is the convolution kernel size, and S is the sliding step of the convolution kernel. It can be seen that the output of a convolution layer is obtained by multiplying and accumulating the corresponding area and the convolution kernel in the N-channel input feature map.

[0006] As can be seen from Formula 1, convolution involves a large number of multiplication and addition operations, and the amount of calculation is very large. If it is implemented in a pure software way, the computing efficiency is very low. It is necessary to develop a neural network acceleration algorithm and an external hardware processor to cooperate with AI instructions to achieve algorithm acceleration. Summary of the invention

[0007] The embodiments of the present invention provide a neural network acceleration coprocessor, a processing system and a processing method to solve the problem that the computational efficiency of convolutional layer calculations currently processed by pure software is very low.

[0008] An embodiment of one aspect of the present invention provides a neural network acceleration coprocessor, including a control module, an address generation module, a multiplication and accumulation module, and an output saturation module;

[0009] The address generation module is used to match storage addresses for input data and corresponding output data;

[0010] The multiplication-accumulation module is used to perform neural network convolution operations;

[0011] The output saturation module is used to limit the range of output data and output the operation result;

[0012] The control module is used to receive the extended instruction sent by the main processor, control the address generation module to match the input and corresponding output data addresses according to the extended instruction, read data from the memory according to the matching address, control the multiplication and addition module to perform convolution calculation on the read data, control the output saturation module to output the calculation result, and store the output result in the memory according to the matching output data address.

[0013] Preferably, the extended instruction includes a configuration instruction for initializing convolution parameters and an operation instruction for performing a convolution operation;

[0014] The configuration instruction is a single-cycle instruction, which is used to configure the address and parameters of input and output data and the parameters of the convolution kernel;

[0015] The operation instruction is a variable multi-cycle instruction, and the number of execution cycles of the instruction is determined by the parameters of the convolution operation set in the first-level instruction.

[0016] In any one of the above embodiments, preferably, the configuration instructions include the following first to sixth instructions;

[0017] The first instruction is used to set the number of input and output tensor channels;

[0018] The second instruction is used to set the input and output tensor sizes;

[0019] The third instruction is used to set the size and step size of the convolution kernel;

[0020] The fourth instruction is used to set the fill size and the filter weight data starting address;

[0021] The fifth instruction is used to set the starting address of input and output data;

[0022] The sixth instruction is used to set the offset and deviation data starting address.

[0023] Preferably, any one of the above embodiments also includes a control module decoding the extended instruction, reading the operand according to the decoded extended instruction, and writing the read operand into a register; the operand is used to transmit the storage address of the input data in the memory; and reading the input data from the memory according to the storage address pointed to by the operand.

[0024] The present invention also provides a neural network acceleration processing system, comprising the above-mentioned coprocessor, a main processor and a memory;

[0025] A memory for storing data;

[0026] The main processor is used to send expansion instructions;

[0027] The coprocessor is used to receive the extended instructions sent by the main processor, read the input data from the memory according to the received extended instructions, perform neural network calculations on the input data, obtain output data, and store the output data in the memory.

[0028] The present invention also provides a neural network acceleration processing method, which is applied to the above processing system and includes the following steps:

[0029] The main processor sends an extended instruction to the coprocessor;

[0030] The coprocessor receives the extended instruction sent by the main processor, reads input data from the memory according to the received extended instruction, performs a neural network operation on the read input data to obtain output data, and writes the output data into the memory;

[0031] The main processor reads the output data stored by the coprocessor from the memory to complete the neural network algorithm processing.

[0032] Preferably, in any one of the above embodiments, the main processor sends an extension instruction to the coprocessor to control the coprocessor to initialize the configuration of convolution parameters and perform the convolution operation;

[0033] The initialization convolution parameter configuration includes single-cycle configuration of input and output data addresses, parameters and convolution kernel parameters; the execution cycle of the convolution operation is determined by the convolution operation parameters set in the first-level instruction.

[0034] Preferably, in any of the above embodiments, when configuring the initialization convolution parameters, the following operations are included:

[0035] Set the number of input and output tensor channels; set the input and output tensor sizes; set the size and step size of the convolution kernel; set the padding size and the starting address of the filter weight data; set the starting address of the input and output data; set the starting address of the offset and deviation data.

[0036] Preferably, in any of the above embodiments, each extended instruction is written in the following encoding format:

[0037] a. The first bit interval of the instruction is the Opcode encoding segment; b. Set three bits to control whether to read the source register and write the target register.

[0038] Preferably, in any one of the above embodiments, when reading input data from the memory according to the received extended instruction, the coprocessor also decodes the extended instruction, reads the operand according to the decoded extended instruction, and writes the read operand into the register; the operand is used to transmit the storage address of the input data in the memory; and reads the input data from the memory according to the storage address pointed to by the operand.

[0039] Beneficial Effects

[0040] 1. The neural network acceleration processing method, coprocessor and processing system provided in this application, when processing a neural network, set a coprocessor, and use the coprocessor to handle time-consuming operations in the convolutional neural network. The main processor controls the coprocessor to perform neural network calculations on the input data through extended instructions, thereby reducing the CPU utilization rate. Compared with pure software, the convolution operation efficiency is increased by more than 20 times;

[0041] 2. When configuring the coprocessor through extended instructions, this application simplifies the way of extending the coprocessor instruction set, sets 7 extended instructions, performs parameter initialization and convolution operations, and has a simple algorithm, which improves the robustness and stability of the system and meets the flexible and changeable computing requirements of the algorithm.

[0042] 3. When decoding the extended instruction, the coprocessor of the present application will directly read the operand and send it to the register, and use the operand for address transfer. Compared with directly reading and writing the data address, the reading and writing speed is faster and the buffer structure utilization rate is higher.

[0043] 4. The extended instruction encoding format adopted in this application uses the first bit interval (low seven bits) of the instruction as the Opcode encoding segment. By using each instruction group and additional encoding space of this encoding segment, more coprocessor instructions can be encoded, greatly improving the scalability of the coprocessor. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0045] Figure 1 A schematic diagram of the input and output features of the convolutional neural network model proposed in the background technology of this application;

[0046] Figure 2 A structural block diagram of a neural network acceleration coprocessor provided in an embodiment of the present application;

[0047] Figure 3 A structural block diagram of a neural network processing system provided in an embodiment of the present application;

[0048] Figure 4 A flowchart of a neural network acceleration processing method provided in an embodiment of the present application;

[0049] Figure 5 This is a schematic diagram of the execution flow of the convolution calculation coprocessor in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of data acceleration calculation for another example of this application; DETAILED DESCRIPTION

[0051] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0052] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention.

[0053] like Figure 2 As shown, an embodiment of one aspect of the present invention provides a neural network acceleration coprocessor, including a control module 301, an address generation module 302, a multiplication and accumulation module 303 and an output saturation module 304;

[0054] The address generation module 302 is used to match storage addresses for input data and corresponding output data;

[0055] The multiplication-accumulation module 303 is used to perform neural network convolution operations;

[0056] The output saturation module 304 is used to limit the range of output data and output the operation result;

[0057] The control module 301 is used to receive the extended instructions sent by the main processor, control the address generation module to match the input and corresponding output data addresses according to the extended instructions, read data from the memory according to the matching addresses, control the multiplication and accumulation module to perform convolution calculation on the read data, control the output saturation module to output the calculation results, and store the output results in the memory according to the matching output data address.

[0058] The extended instructions include configuration instructions for initializing convolution parameters and operation instructions for performing convolution operations; further, the configuration instructions are single-cycle instructions for configuring the addresses and parameters of input and output data and the parameters of the convolution kernel;

[0059] The operation instruction is a variable multi-cycle instruction, and the number of execution cycles of the instruction is determined by the parameters of the convolution operation set in the first-level instruction.

[0060] The configuration instructions include the following first to sixth instructions;

[0061] The first instruction is used to set the number of input and output tensor channels;

[0062] The second instruction is used to set the input and output tensor sizes;

[0063] The third instruction is used to set the size and step size of the convolution kernel;

[0064] The fourth instruction is used to set the fill size and the filter weight data starting address;

[0065] The fifth instruction is used to set the starting address of input and output data;

[0066] The sixth instruction is used to set the offset and deviation data starting address.

[0067] like Figure 5 As shown, in this embodiment, 7 extended instructions are defined for the implementation of the convolution operation coprocessor, of which 6 configuration instructions are INIT_CH for setting the number of input and output tensor channels, INIT_IM for setting the size of the input and output tensor, INIT_FS for setting the filter kernel size and step size, INIT_PW for setting the padding size and the starting address of the filter weight data, INIT_IMADDR for setting the starting address of the input and output data, and INIT_BIAS for setting the starting address of the offset and deviation data; instructions for initializing the convolution parameters; the instructions for initializing the parameters are all single-cycle instructions, and when the corresponding instructions are received, the operands will be directly read out and sent to the register for subsequent calculations.

[0068] Specifically, the control module decodes the extended instruction, reads the operand according to the decoded extended instruction, and writes the read operand into the register; the operand is used to transmit the storage address of the input data in the memory; and reads the input data from the memory according to the storage address pointed to by the operand.

[0069] The LOOP instruction is used to perform convolution operation. The LOOP instruction for performing convolution operation is a variable multi-cycle instruction, and the number of execution cycles of the instruction is determined by the parameters related to the convolution operation.

[0070] The specific definitions of each instruction are shown in Table 1.

[0071]

[0072]

[0073] Table 1 Convolution coprocessor instruction list

[0074] In the above table, Opcode refers to the encoding segment, which is used to encode custom-0, custom-1, custom-2, and custom-3 instruction groups. The xs1, xs2, and xd bits are used to control whether to read the source register and write the destination register. The funct7 interval can be used as additional encoding space to encode more instructions, so a custom instruction group can encode 128 instructions using the funct7 interval.

[0075] In this embodiment, a coprocessor is set up, and after receiving the extended instruction, the coprocessor processes the time-consuming operations in the convolutional neural network, performs neural network convolution calculations on the input data, and flexibly combines the convolution, pooling and activation operations of the convolutional neural network, which can be adapted to a variety of lightweight convolutional neural networks.

[0076] like Figure 3 As shown, the present invention also provides a neural network acceleration processing system, including the above-mentioned coprocessor 3, and also including a main processor 1 and a memory 2; the memory 2 is used to store data; the main processor 1 is used to send extended instructions; the coprocessor 3 is used to receive the extended instructions sent by the main processor, read the input data from the memory according to the received extended instructions, perform neural network calculations on the input data, obtain output data, and store the output data in the memory.

[0077] like Figure 4 As shown, the present invention also provides a neural network acceleration processing method, which is applied to the above processing system and includes the following steps:

[0078] S1, the main processor sends an extended instruction to the coprocessor;

[0079] S2, the coprocessor receives the extended instruction sent by the main processor, reads input data from the memory according to the received extended instruction, performs a neural network operation on the read input data to obtain output data, and writes the output data into the memory;

[0080] S3. The main processor reads the output data stored by the coprocessor from the memory to complete the neural network algorithm processing.

[0081] Among them, in S1 or S2, the extended instructions include configuration instructions for initializing convolution parameters and operation instructions for performing convolution operations; the configuration instructions are single-cycle instructions, which are used to configure the addresses, parameters of input and output data and the parameters of the convolution kernel; the operation instructions are variable multi-cycle instructions, and the number of execution cycles of the instruction is determined by the parameters of the convolution operation set in the first-level instructions.

[0082] During configuration, the following operations are included: setting the number of input and output tensor channels; setting the input and output tensor sizes; setting the size and step size of the convolution kernel; setting the padding size and the starting address of the filter weight data; setting the starting address of the input and output data; setting the starting address of the offset and deviation data.

[0083] As shown in Table 1 above, in this embodiment, 7 extended instructions are defined for the implementation of the convolution operation coprocessor, of which 6 configuration instructions are INIT_CH for setting the number of input and output tensor channels, INIT_IM for setting the input and output tensor size, INIT_FS for setting the filter kernel size and step size, INIT_PW for setting the padding size and filter weight data starting address, INIT_IMADDR for setting the starting address of input and output data, and INIT_BIAS for setting the offset and deviation data starting address; instructions for initializing convolution parameters; several instructions for parameter initialization are single-cycle instructions. When the corresponding instructions are received, the operands will be directly read out and sent to the register for subsequent calculations. The operation instruction uses the LOOP instruction to perform the convolution operation. The LOOP instruction for performing the convolution operation is a variable multi-cycle instruction, and the number of execution cycles of the instruction is determined by the parameters related to the convolution operation.

[0084] As shown in the above table, the encoding format of the extended instruction includes 1. The first bit interval of the instruction is the Opcode encoding segment; 2. Three bits are set to control whether the source register needs to be read and the target register needs to be written. Specifically, in the above table, Opcode refers to the encoding segment, and the custom-0, custom-1, custom-2 and custom-3 instruction groups are used to encode. Generally, the first bit to the sixth bit are used as the first bit interval as the Opcode encoding segment, and the xs1, xs2 and xd bits are used to control whether the source register needs to be read and the target register needs to be written. The funct7 interval can be used as an additional encoding space to encode more instructions, so a Custom instruction group can use the funct7 interval to encode 128 instructions.

[0085] When this method is applied to the above system for testing, the following results are obtained: Figure 6In the embodiment shown, the Cifar-10 dataset is a dataset collected by two of Hinton's apprentices, Alex Krizhevsky and Ilya Sutskever, for the purpose of facilitating the recognition of common objects. The dataset contains 10 categories of image sets, namely airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks, with a total of 60,000 images, where each category of image set contains 6,000 images. The pixel size of each image in the dataset is 32*32, and the number of channels is RGB3 channels.

[0086] The processing results are shown in the following table.

[0087] Processing stage Coprocessor-based Pure software Speedup convolution 4675254 94595789 20.23

[0088] Table 2 is a list of processing results

[0089] In this embodiment, through comparison in the convolution calculation stage, the current pure software processing method requires 94595789 cycles, while the coprocessor-based processing method provided by the embodiment of the present invention requires 4675254 cycles, and the acceleration ratio is 20.23.

[0090] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0091] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. A neural network acceleration coprocessor, characterized in that: It includes a control module, an address generation module, a multiplication and accumulation module and an output saturation module; The control module is used to receive the extended instruction sent by the main processor, control the address generation module to match the input and corresponding output data addresses according to the extended instruction, read data from the memory according to the matching address, control the multiplication and accumulation module to perform convolution calculation on the read data, control the output saturation module to output the calculation result, and store the output result in the memory according to the matched output data address; The address generation module is used to match storage addresses for input data and corresponding output data; The multiplication-accumulation module is used to perform neural network convolution operations; The output saturation module is used to limit the range of output data and output the operation result; The extended instructions include configuration instructions for initializing convolution parameters and operation instructions for performing convolution operations; The configuration instruction is a single-cycle instruction, which is used to configure the address and parameters of input and output data and the parameters of the convolution kernel; The operation instruction is a variable multi-cycle instruction, and the number of execution cycles of the instruction is determined by the parameters of the convolution operation set in the first-level instruction; The configuration instructions include the following first to sixth instructions; The first instruction is used to set the number of input and output tensor channels; The second instruction is used to set the input and output tensor sizes; The third instruction is used to set the size and step size of the convolution kernel; The fourth instruction is used to set the fill size and the filter weight data starting address; The fifth instruction is used to set the starting address of input and output data; The sixth instruction is used to set the offset and deviation data starting address; It also includes a control module that decodes the extended instruction, reads an operand according to the decoded extended instruction, and writes the read operand into a register; the operand is used to transmit the storage address of the input data in the memory; and reads the input data from the memory according to the storage address pointed to by the operand.

2. A neural network acceleration processing system, characterized in that: The coprocessor according to claim 1 further comprises a main processor and a memory; The memory is used to store data; The main processor is used to send an extended instruction; The coprocessor is used to receive the extended instructions sent by the main processor, read input data from the memory according to the received extended instructions, perform neural network calculations on the input data, obtain output data, and store the output data in the memory.

3. A neural network acceleration processing method, characterized in that: The processing system as claimed in claim 2 comprises the following steps: The main processor sends an extended instruction to the coprocessor; The coprocessor receives the extended instruction sent by the main processor, reads input data from the memory according to the received extended instruction, performs a neural network operation on the read input data to obtain output data, and writes the output data into the memory; The main processor reads the output data stored by the coprocessor from the memory and completes the neural network algorithm processing; The main processor sends an extended instruction to the coprocessor to control the coprocessor to initialize the configuration of convolution parameters and perform convolution operations; The initialization convolution parameter configuration includes single-cycle configuration of input and output data addresses, parameters and convolution kernel parameters; the execution cycle of the convolution operation is determined by the convolution operation parameters set in the first-level instruction; When configuring the initialization convolution parameters, the following operations are included: Set the number of input and output tensor channels; Set the input and output tensor sizes; Set the size and step size of the convolution kernel; Set the fill size and filter weight data starting address; Set the starting address of input and output data; Set the offset and deviation data starting address; It also includes writing each extended instruction in the following encoding format: a. The first bit interval of the instruction is the Opcode encoding segment; b. Set three bits to control whether to read the source register and write the target register; When reading input data from the memory according to the received extended instruction, it also includes The coprocessor decodes the extended instruction, reads the operand according to the decoded extended instruction, and writes the read operand into the register; the operand is used to transmit the storage address of the input data in the memory; and reads the input data from the memory according to the storage address pointed to by the operand.

Images