Processing device, processing method, chip and electronic device

Abstract

The present disclosure relates to a processing device, comprising: a storage part for storing data, the data comprising to-be-operated data of a neural network; an operation part for operating the to-be-operated data, comprising accelerating the to-be-operated data by using an adder and / or a multi-type multiplier; and a control part for controlling the storage part and the operation part, comprising accelerating operation according to the to-be-operated data. The device of the present disclosure has the advantages of high flexibility, high configurability, fast operation speed, low power consumption, etc.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and further to the field of artificial intelligence. Background Technology

[0002] With the advent of the big data era, neural network algorithms have become a research hotspot in the field of artificial intelligence in recent years, finding widespread application in pattern recognition, image analysis, and intelligent robots. However, because these algorithms are often used to process large amounts of data, involving multi-layered structures and a large number of computations, they place higher demands on computational speed and specifications. On the one hand, this places demands on the computational scale and speed of the arithmetic unit (ALU), as it must be able to quickly complete the required computational tasks given the large amount of data and computational demands of neural network algorithms. On the other hand, it places demands on the dynamic configurability of the ALU, because during computation, the data specifications of a large amount of data may not be the same, and the data bit width and data quantity may also differ between layers. Furthermore, while tolerating a certain range of precision loss, reducing the data bit width can effectively improve the computational efficiency of the ALU. This trade-off between precision and efficiency increases the diversity of data bit width and the dynamic variability of data bit width during computation, therefore the ALU needs to be dynamically configurable to meet the requirements of different data specifications and data bit widths.

[0003] In existing technologies, one way to implement neural networks is to perform operations directly on a general-purpose processor (CPU) or graphics processing unit (GPU). This method, due to its architecture's need for versatility, cannot effectively accelerate neural network operations, or its on-chip cache is too small to meet the demands of large-scale neural network operations. Another approach is to configure and perform operations on a Field-Programmable Gate Array (FPGA) or Application-Specific Integrated Circuit (ASIC). However, FPGAs have low clock speeds and high power consumption, and configurable programming of neural network operations is limited by the FPGA's resources, thus hindering acceleration and failing to meet the requirements of different data specifications and bit widths. Furthermore, even when using ASICs to design and accelerate neural networks, these devices are largely limited by their storage and processing capabilities, supporting only fixed-length data storage and operations, and lacking the flexibility to dynamically configure operations between data with different bit widths.

[0004] Public content

[0005] (a) Technical problems to be solved

[0006] The purpose of this disclosure is to provide a processing apparatus and method with dynamically configurable bit width for calculating data, in order to solve at least one of the technical problems described above.

[0007] (II) Technical Solution

[0008] According to one aspect of this disclosure, a processing apparatus with dynamically configurable bit width is provided, comprising:

[0009] The storage section is used to store data, including data to be processed by the neural network;

[0010] The computation section is used to perform operations on the data to be computed in the neural network, including using adders and multipliers to perform calculations on the data to be computed in neural networks with different computational bit widths; and

[0011] The control section is used to control the storage section and the arithmetic section, including determining the types of multipliers and adders in the arithmetic section according to the data to be processed in order to perform the operation.

[0012] According to another aspect of this disclosure, a chip is provided that includes the aforementioned means.

[0013] According to another aspect of this disclosure, a chip packaging structure is provided, including the chip described above.

[0014] According to another aspect of this disclosure, a board is provided, including the above-described chip packaging structure.

[0015] According to another aspect of this disclosure, an electronic device is provided, including the aforementioned circuit board.

[0016] According to another aspect of this disclosure, a method for computing a dynamically configurable bit-width neural network processor is provided, comprising the steps of:

[0017] The control section generates control commands and transmits them to the storage and processing sections.

[0018] The storage section inputs the neural network's computational data into the computation section based on the received control instructions;

[0019] The arithmetic section selects the corresponding type of multiplier and adder in the first arithmetic module according to the received control instructions;

[0020] The computation section performs calculations on the data to be computed in neural networks with different computational bit widths, based on the input data to be computed, the neural network parameters, and the control instructions.

[0021] (III) Beneficial Effects

[0022] The dynamically configurable computational bit-width processing apparatus and method disclosed herein utilizes block storage management tailored to the characteristics of neural networks. In the first computation module, it effectively accelerates computations by leveraging the characteristics of data with different bit widths. Furthermore, through rational scheduling by the control module, it can employ a pipelined parallel execution approach, further reducing runtime. This allows for effective acceleration of neural network algorithms based on varying needs, dynamically configuring the computational bit width to meet diverse computational requirements. It offers advantages such as high flexibility, high configurability, high computational speed, and low power consumption. The apparatus disclosed herein accelerates the computation process of convolutional neural networks, reduces on-chip and off-chip data exchange, saves storage space, and does not include an online learning phase. It also accelerates the computation process of large-scale convolutional neural networks. By employing basic multipliers, sparse multipliers, and / or fused vector multipliers to change the computation order, it significantly reduces the counting operations on shift values, thereby effectively improving the performance-to-power ratio. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the processing device of a dynamically configurable neural network processor with a computational bit width provided in an embodiment of the present disclosure.

[0024] Figure 2 This is a schematic diagram of the processing device structure of a neural network processor with dynamically configurable computational bit width provided in another embodiment of this disclosure.

[0025] Figure 3 This is a schematic diagram of the structure of a neural network processor with dynamically configurable computational bit width provided in another embodiment of the present disclosure.

[0026] Figure 4 This is a schematic diagram of another embodiment of the computationally configurable neural network processor provided in yet another embodiment of this disclosure.

[0027] Figure 5 This is a schematic diagram of the structure of a basic multiplier device according to an embodiment of the present disclosure.

[0028] Figure 6 This is a schematic diagram of the structure of a sparse multiplier device according to an embodiment of the present disclosure.

[0029] Figure 7 This is a schematic diagram of the structure of a device for performing vector multiplication using a basic multiplier or a sparse multiplier, as provided in one embodiment of this disclosure.

[0030] Figure 8 This is a schematic diagram of the structure of a device for performing vector multiplication using a fusion vector multiplier according to an embodiment of this disclosure.

[0031] Figure 9This is a structural diagram illustrating the specific implementation process of the fusion vector multiplier device and other multiplier devices provided in this disclosure.

[0032] Figure 10 This is a flowchart of a method for computing a dynamically configurable neural network processor according to an embodiment of this disclosure. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0034] Figure 1 This is a schematic diagram of the processing device of the dynamically configurable neural network processor with computational bit width provided in an embodiment of this disclosure. Figure 1 As shown, this device is mainly divided into three parts: a control section, a computation section, and a storage section. The control section sends control signals to the computation and storage sections to control their operation and coordinate data transmission between them. The storage section stores relevant data, including computation results, input data (including data to be computed in the neural network and control instructions), neural network parameters, and lookup tables. It may also include intermediate results, final results, neurons, synapses, cached data, etc. The specific content, organization, and access methods of the stored data can be planned differently according to different needs. The computation section contains various arithmetic units for data computation, including adders, basic multipliers, sparse multipliers, and / or fused vector multipliers. These arithmetic units can perform operations on data with different computational bit widths, allowing for operations between data of different bit widths according to different requirements.

[0035] Figure 2 This is a schematic diagram of the structure of a neural network processor device according to an embodiment of the present disclosure. Figure 2As shown, the structure of this device is as follows: the control section connects each module or sub-module of the storage section and the computation section, consisting of a first-in-first-out (FIFO) queue and a control processor. The FIFO queue stores control signals, and the control processor retrieves the control signals to be executed, analyzes the control logic, and then controls and coordinates the storage and computation sections. The storage section is divided into three modules: an input storage module, an output storage module, and a synaptic storage module. The output storage module can be used to store intermediate and final results. The computation section is mainly used to accelerate the operations of convolutional layers, fully connected layers, and / or pooling layers. Therefore, the computation section is also mainly divided into three modules: a multiplier module, an adder tree module, and a module for performing nonlinear operations (e.g., this module can perform sigmoid function operations). These three modules can be executed in parallel using a pipelined approach. This device can accelerate the computation process of convolutional neural networks, reduce on-chip and off-chip data exchange, and save storage space.

[0036] Figure 3 This is a schematic diagram of the processing apparatus of a neural network processor according to another embodiment of the present disclosure. Figure 3 As shown, the structure of this device is such that the control section connects each module of the storage section and the computation section, consisting of a first-in-first-out (FIFO) queue and a control processor. The FIFO queue stores control signals, and the control processor retrieves the control signals to be executed, analyzes the control logic, and then controls and coordinates the storage and computation sections. The storage section mainly includes an input storage module, an output storage module, and a synaptic storage module. In this device, the synaptic storage module is divided into multiple modules, and multiple computation units are configured within it. During each computation, the input storage module transmits input data to all computation units, the synaptic storage module transmits synaptic data to the corresponding computation units, and the computation modules write the results to the output storage module after computation. This significantly improves computational efficiency in large-scale computations with many parameters. This device can effectively accelerate the computation process of convolutional neural networks, and is especially suitable for networks with large scale and many parameters.

[0037] Figure 4 This is a schematic diagram of the structure of a neural network processor device according to another embodiment of the present disclosure. Figure 4As shown, the structure of this device is as follows: the control section connects each module of the storage section and the arithmetic section, consisting of an instruction queue and a decoder. Each time a new instruction is executed, a new instruction is retrieved from the instruction queue and sent to the decoder. The decoder decodes the instruction and sends the control information to each storage module and arithmetic module. The storage section includes four modules: an input storage module, an output storage module, a synaptic storage module, and a buffer module. The output storage module can be used to store intermediate and final results. Each time the input storage module and synaptic storage module transmit data to the arithmetic section, the data is first passed to the buffer module and then sent to the corresponding arithmetic module. Similarly, when the arithmetic module finishes its operation, the result is first sent to the buffer module and then written from the buffer module to the output storage module. The arithmetic section mainly consists of multiple arithmetic modules, including a first arithmetic module and a second arithmetic module. The arithmetic modules can perform related operations in parallel and can also exchange data, thereby reducing the reuse distance of data with locality and further improving the operation speed. The first computation module is primarily used to accelerate linear operations with the same or different computational bit widths in neural network algorithms, including: matrix multiplication, addition, and mixed multiplication and addition; matrix and vector; matrix and constant; vector; vector and constant; constant and constant. It can also be used for comparison operations, selecting maximum / minimum values, etc. Preferred operations include dot product, matrix multiplication, and / or matrix addition. The second computation module completes operations not performed in the first module, including nonlinear operations, division, and either individual addition or multiplication. This allows for dynamic adjustment of the bit width during computation based on control instructions, further improving the hardware utilization of the computation and storage sections.

[0038] Figure 5 This is a schematic diagram of a basic multiplier device for this apparatus, representing another embodiment of this disclosure, which can meet the requirement of dynamically configurable computation bit width. For example... Figure 5As shown, the multiplicand has M bits and the multiplier has N bits, where M and N are both positive integers. This means the number of bits in the multiplicand and multiplier can be equal or unequal. The lower n bits of the multiplier (n is a positive integer, and 1 < n ≤ N) are input into the input selection. When the lower n bits of the multiplier are ANDed with the multiplicand, i.e., the multiplier bit is 1, the multiplicand itself is used; otherwise, 0 is used. Simultaneously, the multiplier is shifted into the first shift register, shifting the lower n bits out. The next input into the input selection will be the new lower n bits. The result after input selection is shifted upwards into the second shift register for corresponding shifting, and then sent to the adder tree for accumulation. Here, the data after input selection and shifting is accumulated along with the previous accumulated result. The resulting result is stored as an intermediate result in the result register. When the multiplicand is next input selected and shifted, the result register retrieves the intermediate result and sends it to the adder tree for accumulation. The multiplication operation ends when all multipliers are 0.

[0039] To more clearly illustrate the operation flow of this basic multiplier, we provide a specific example, assuming the multiplicand is 10111011 (M = 8) and the multiplier is 1011 (N = 4).

[0040] When n=2, that is, when shifting 2 bits each time, the operation process is as follows: First, take out the lowest 2 bits of the multiplier (11) and send them together with the multiplicand into the input selection unit. The selection is always the multiplicand itself. The multiplicand is sent to the first shift register. The multiplicand selected by the lowest bit does not need to be shifted, that is, 10111011. The multiplicand selected by the second lowest bit is shifted left by 1 bit, that is, 101110110, and sent into the addition tree. Since there are no numbers added before, the result register is the sum of 10111011 and 101110110, that is, 1000110001. Then, the multiplier is shifted right by 2 bits, and its lowest two bits, 10, are taken and sent to the input selection unit along with the multiplicand, resulting in 0 and 10111011. Next, through the shift register, 0 is shifted left by 2 bits and remains 0, while 10111011 is shifted left by 3 bits, becoming 10111011000. This, along with 1000110001 from the result register, is sent to the addition tree for operation, resulting in 100000001001, which is then sent to the result register. At this point, the multiplier is shifted right by 2 bits, all bits being 0, indicating the end of the operation. The result register contains the final result, 100000001001.

[0041] Figure 6 This is a schematic diagram of a sparse multiplier device provided in this disclosure, which can meet the requirement of dynamically configurable computational bit width. As the name suggests, a sparse multiplier is designed for sparse operations, that is, when the multiplier or multiplicand is represented by sparse representations of 1s, it can further improve the efficiency of the operation and speed up the operation. Figure 6 As shown, the multiplicand has M bits and the multiplier has N bits, where M and N are both positive integers. This means the number of bits in the multiplicand and multiplier can be equal or unequal. Here, the multiplier uses a sparse representation, indicating the position of 1s in the multiplier using absolute or relative positions. Our arithmetic unit is configurable; therefore, the internal devices of the arithmetic unit can be configured as needed when different representation methods are used. For example, if no shifting is required when accumulating in the result register, the shift register connected to the result register can be specified to be inactive, and the shift information of the multiplier can also be omitted from this shift register. All relevant details can be adjusted as needed to handle the shifting of the multiplicand and the accumulation of the result.

[0042] To more clearly illustrate the operation flow of this sparse multiplier, we provide a specific embodiment. Assume the multiplicand is 10111011 (M = 8), and the multiplier is 00100010 (N = 8). When using absolute representation for the multiplier, the position of the 1 in the multiplier is indicated by absolute positions. Let's assume the rightmost bit is called the 0th bit, the bit to the left of the 0th bit is called the 1st bit, and so on. Therefore, the multiplier is represented as (1, 5). Simultaneously, we require that the shift register connected to the result register in this embodiment is not active; the multiplier data does not need to be passed to this shift register. First, the first digit of the multiplier, 1, is retrieved, indicating a 1 at the 1st bit. The multiplicand is then sent to the shift register, shifted by one bit to become 101110110, and sent to the adder. Since the numbers were previously added, the result sent to the result register is 101110110. Then, the position of the next 1 in the multiplier, 5, is retrieved and sent to the shift register along with the multiplicand. In the shift register, the multiplicand is shifted right by 5 bits to obtain 1011101100000, which is then fed into the adder. Simultaneously, the result 101110110 is retrieved from the result register. Since this absolute representation method does not require shifting, this result can be directly fed into the adder for accumulation, resulting in 1100011010110. The accumulated result is then fed back into the result register. At this point, all the 1s in the multiplier have been calculated, so the operation ends. If a relative representation of the multiplier is used, and its representation is defined as the number of bits between any two non-zero numbers starting from the first non-zero digit in the highest (leftmost) bit to the lowest bit, for 00100010, the distance between the first and next non-zero digits is 4 bits, and the distance from the second non-zero digit to the lowest bit is 1 bit, hence represented as (4, 1). Here, we require that both the shift register connected to the result register and the shift register connected to the multiplicand in this embodiment be operational. First, the first digit of the multiplier, 4, is retrieved and stored in two shift registers. The multiplicand is then shifted right by 4 bits, and the data in the result register is also shifted right by 4 bits before being added to the adder for accumulation. At this point, the data in the result register is 0, resulting in the accumulated value 101110110000, which is then stored in the result register. Next, the second digit of the multiplier, 1, is retrieved and stored in a shift register, resulting in 101110110 and 1011101100000, which are then added to the adder for accumulation, yielding the result 1100011010110. This result is then stored in the result register. At this point, all the 1s in the multiplier have been calculated, and the operation ends. This effectively utilizes the sparsity of data, performing only valid operations—operations between non-zero data. This reduces invalid operations, speeds up the operation, and improves the performance-to-power ratio.

[0043] Figure 8This is a schematic diagram of a device for performing vector multiplication using a fusion vector multiplier according to an embodiment of this disclosure. Here, we assume that the vector is being computed... and The inner product value is used to feed the data of the corresponding dimension into the multiplier for calculation, such as... Figure 8 As shown. Here, it is required that... and The dimensions are all the same, (N+1), but the bit width of each dimension is not necessarily the same. It is assumed that n bits are used for each operation, where n is greater than 1 and not greater than... A positive integer with a bit width of one dimension. First, the lower n bits of B0 and A0 are simultaneously fed into the first input selection. The lower n bits of B0 are then ANDed with A0, and the result is fed into a subsequent shift register for shifting. After shifting, the result is fed into an addition tree. During this process, each dimension undergoes the same operation as the first dimension. Then, the data from these dimensions is accumulated using the addition tree, and the value in the result register is fed back into the addition tree for further accumulation. The accumulated result is then fed into the result register. Simultaneously, the bit width of each dimension's B... i The value (i = 0, 1, ..., N) is loaded into the shift register, shifted right by n bits, and then the above operation is repeated to retrieve the shifted B. i The lowest n bits of the value of (i = 0, 1, ..., N) and the corresponding A i The values ​​(i = 0, 1, ..., N) are fed together into the input selection, then shifted into a shift register, and finally accumulated in an addition tree. This process is repeated until the value of B in each dimension is reached. i When all values ​​of (i = 0, 1, ..., N) are 0, the operation ends, and the data in the result register is the final result. This multiplier allows for flexible configuration of the bit width of the data to be operated on, eliminating the need to recount the shift bits of the multiplicand after each multiplication. Furthermore, when the data bit width is low or the vector bit width is high, it can fully utilize the characteristics of low data bit width and high vector dimension, enabling parallel execution of the process in a pipelined manner, reducing runtime, further accelerating computation, and improving performance-to-power ratio.

[0044] We can use various methods to perform vector dot product operations, combining... Figure 7 , Figure 8 and Figure 9 Let me explain. First, we assume... and The dimension is 8, that is, N=7. The bit width is 8 bits, that is Each dimension is 8 bits, i.e., Ai = {ai7…ai1ai0}, where i = 0, 1, …, 7; The bit width is 4 bits, that is Each dimension of the vector is 4 bits, i.e., Bi = {bi3bi2bi1bi0}, where i = 0, 1, ..., 7. Then the vector dot product...

[0045] The computational flow when using a basic multiplier or the aforementioned basic or sparse multiplier (assuming n is 2, i.e., the multiplier shifts by 2 bits each time) consists of two stages: first, the product of each component is calculated separately, and then the sum is performed, such as... Figure 7 As shown. Specifically, for a certain dimension A i and B i Perform the calculation, then clear the shift counter. Take B in the first clock cycle. i The lowest two 'b's i0 ,b i1 Input selection, shifting, and feeding into the adder yields A. i *b i0 b i1 The value of B is incremented by 2 in the second clock cycle; i Right-shift by 2 bits and take the two least significant bits to get the least significant bit b. i2 ,b i3 Input selection and shifting to obtain A i *b i2 b i3 Add the result to the previous sum to get the final result A. i *b i0 b i1 b i2 b i3 That is, to obtain the final result A of this dimension. i *B i To proceed with the next dimension of the calculation, input A. i+1 and B i+1 The shift counter is reset to zero... until each dimension has been calculated, resulting in (A0*B0, A1*B1, ..., A7*B7), completing stage 1. Then, in stage 2, the product is fed into an addition tree for addition, yielding the final vector dot product. In Phase 1, you can choose one multiplier to calculate each dimension sequentially; alternatively, you can provide multiple multipliers to perform parallel operations, with one multiplier completing the calculation for one dimension, such as... Figure 7 As shown. When multiple multipliers are used, the multiplier B for each dimension... i All shifted values ​​need to be recounted. The multiplier used in this stage can be either the basic multiplier described above or a sparse multiplier.

[0046] Using a fused vector multiplier, the horizontal accumulation operation is performed directly as a whole, and its structure is as follows: Figure 8 As shown, after the product of one component in each dimension is completed, it is fed into the addition tree for accumulation until the calculation is complete, yielding the final result. The calculation process is as follows: Figure 9 As shown in the elliptical frame, in the first clock cycle, A is calculated for each dimension. i *b i0 The product of (i = 0, 1, ..., 7) is stored in the addition tree and accumulated. The result is then stored in the result register, and the shift counter is incremented by 1. In the second clock cycle, 2*A is calculated for each dimension based on the shift counter. i *b i1 The product of (i = 0, 1, ..., 7) and the data in the result register are fed into the addition tree for accumulation, and the shift counter is incremented by 1; in the third clock cycle, 4*A is calculated for each dimension based on the shift counter. i *b i2 The product of (i = 0, 1, ..., 7) and the data in the result register are fed into the addition tree for accumulation, and the shift counter is incremented by 1; finally, in the fourth clock cycle, 8*A is calculated. i *b i3 The product of (i = 0, 1, ..., 7) and the data in the result register are fed into the addition tree for accumulation to obtain the final result. Therefore, we obtain the desired result after 4 operation cycles, involving a total of 3 shifts. In contrast, a typical multiplier requires a shift operation for each data operation; that is, with 4 operands, a total of 4 * 3 = 12 shift operations are needed. Therefore, our design, by changing the order of operations, significantly reduces the counting operations for shift values, thereby effectively improving the performance-to-power ratio.

[0047] In summary, this device and method can significantly improve the computation speed of neural networks, while also possessing dynamic configurability, meeting the requirements of diverse data bit widths and dynamic variability of data bit widths during computation. It has advantages such as high flexibility, high configurability, fast computation speed, and low power consumption.

[0048] In one embodiment, this disclosure discloses a chip that includes the aforementioned neural network processor.

[0049] In one embodiment, this disclosure discloses a chip package structure that includes the aforementioned chip.

[0050] In one embodiment, this disclosure discloses a circuit board that includes the above-described chip packaging structure.

[0051] In one embodiment, this disclosure discloses an electronic device that includes the aforementioned circuit board.

[0052] Electronic devices include, but are not limited to, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, mobile phones, dashcams, navigators, sensors, cameras, cloud servers, cameras, camcorders, projectors, watches, headphones, mobile storage, wearable devices, vehicles, home appliances, and / or medical devices.

[0053] The means of transportation include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical equipment includes MRI scanners, ultrasound scanners and / or electrocardiographs.

[0054] According to another aspect of the embodiments of this disclosure, a method for computing a dynamically configurable bit-width neural network processor is also provided, see [link to relevant documentation]. Figure 10 As shown, the steps include:

[0055] S0: The control section generates control commands and transmits them to the storage section and the arithmetic section;

[0056] S1: The storage section inputs the data to be processed by the neural network into the processing section according to the received control instructions;

[0057] S2: The arithmetic section selects the corresponding type of multiplier and adder in the first arithmetic module according to the received control instructions;

[0058] S3: The computation section performs calculations on the data to be computed in neural networks with different computational bit widths based on the input data to be computed, the neural network parameters, and the control instructions.

[0059] Furthermore, in step S3, the first operation module includes using an adder, as well as a basic multiplier, a sparse multiplier, and / or a fused vector multiplier to perform operations on the data to be operated on in the neural network.

[0060] It should be understood that the disclosed apparatus and methods can be implemented in other ways, as provided in the embodiments of this disclosure. For example, the apparatus embodiments described above are merely illustrative, and the division of parts or modules is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple parts or modules may be combined or integrated into a system, or some features may be ignored or not executed.

[0061] Those skilled in the art will understand that all the above functional units / modules can be hardware, such as circuits, including digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, physical devices, including, but not limited to, transistors, memristors, etc. The computing module in the computing device can be any suitable hardware processor, such as CPU, GPU, FPGA, DSP, and ASIC, etc. The storage unit can be any suitable magnetic storage medium or magneto-optical storage medium, such as RRAM, DRAM, SRAM, EDRAM, HBM, HMC, etc. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this disclosure. It should be understood that the above descriptions are merely specific embodiments of this disclosure and are not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A processing apparatus, characterized in that... include: The storage section is used to store data, including data to be processed by the neural network; The computation section is used to perform operations on the data to be computed in the neural network, including using adders and multipliers to perform calculations on the data to be computed in neural networks with different computational bit widths; and The control section controls the storage and arithmetic sections, including determining the types of multipliers and adders in the arithmetic section based on the data to be processed in order to perform the arithmetic operations; wherein, The storage portion includes: The input storage module is used to store the data to be processed in the neural network; The output storage module is used to store the calculation results; The caching module is used for buffered storage of data; The input storage module includes: Neuron storage module, used to store neuron parameters; Synaptic storage module, used to store synaptic parameters; The output storage module includes: The intermediate result storage submodule is used to store intermediate calculation results; and The final result storage submodule is used to store the final calculation results.

2. The apparatus according to claim 1, characterized in that, The synaptic storage module comprises multiple synaptic storage modules, used to store multiple synaptic parameters.

3. The apparatus according to claim 2, characterized in that, The computation section includes multiple computation units, each corresponding to one or more synaptic modules. During computation, the input storage module transmits input data to all computation units, the synaptic storage module transmits synaptic data to the corresponding computation unit, and after the computation unit performs the computation, it writes the result to the output storage module. The computation section includes: a first computation module, used to perform computations on data of different bit widths.

4. The apparatus according to claim 3, characterized in that, The first computing module includes: Adders, as well as basic multipliers, sparse multipliers, and / or fused vector multipliers, are used to accelerate computation of data with varying bit widths in neural networks; among them, The basic multiplier is used to perform the operation of dividing the multiplier into multiple low-width data, multiplying each data by the multiplicand, and then summing them up. The sparse multiplier is used to perform multiplication operations when the multiplier and / or multiplicand are represented in a sparse manner; The fusion vector multiplier is used for vector multiplication operations; The control section connects each module or sub-module of the storage section and the arithmetic section, and includes a first-in-first-out queue and a control processor. The first-in-first-out queue is used to store control signals, and the control processor is used to retrieve control signals to be executed.

5. The apparatus according to claim 3, characterized in that, The operations on data with different computational bit widths include: dot product, matrix multiplication, addition, and mixed multiplication and addition; matrix and vector multiplication, addition, and mixed multiplication and addition; matrix and constant multiplication, addition, and mixed multiplication and addition; vector multiplication, addition, and mixed multiplication and addition; vector and constant multiplication, addition, and mixed multiplication and addition; constant and constant multiplication, addition, and mixed multiplication and addition; comparison and selection of maximum / minimum values; and other operations that can be broken down into multiplication, addition, or mixed multiplication and addition.

6. The apparatus according to claim 5, characterized in that, The computation section further includes a second computation module for performing operations including nonlinear operations, division operations, separate addition operations, or separate multiplication operations; wherein... The first operation module includes a multiplier submodule and an adder tree submodule; The second operation module, the multiplier submodule, and the addition tree submodule are executed in parallel using a pipelined approach.

7. A method of using the apparatus according to any one of claims 1-6, characterized in that... Including the following steps: The control section generates control commands and transmits them to the storage and processing sections. The storage section inputs the neural network's computational data into the computation section based on the received control instructions; The arithmetic section selects the corresponding type of multiplier and adder in the first arithmetic module according to the received control instructions; The computation section performs calculations on the data to be processed by the neural network with different computational bit widths based on the input data, neural network parameters, and control instructions, and sends the calculation results back to the storage unit; among which, The computation section performs operations on the neural network input data, neural network parameters, and control instructions, using adders, basic multipliers, sparse multipliers, and / or fused vector multipliers to perform operations on the neural network input data.

8. The method according to claim 7, characterized in that, This also includes performing nonlinear operations, division operations, separate addition operations, or separate multiplication operations in the arithmetic section; among which, The operation using the basic multiplier includes: dividing the multiplier into multiple low-width data, multiplying each data by the multiplicand, and then summing the results; The operation using the sparse multiplier includes: performing multiplication when the multiplier and / or multiplicand are represented in a sparse manner; and The operation using the fused vector multiplier includes performing vector multiplication or inner product operations.

9. The method according to claim 7, characterized in that, The storage section includes a synaptic storage module that stores synaptic parameters from neural network parameters.

10. The method according to claim 9, characterized in that, The synaptic module comprises multiple synaptic modules and stores multiple synaptic parameters; the computation section includes multiple computation units, each corresponding to one or more synaptic modules. During computation, the input storage module transmits input data to all computation units, the synaptic storage module transmits synaptic data to the corresponding computation unit, and after the computation unit performs the computation, it writes the result to the output storage module.

11. The method according to claim 7, characterized in that, The neural network further performs a second operation on the data to be processed, which includes nonlinear operations, division operations, single addition operations, or single multiplication operations; wherein, The first operation module also performs multiplier operations and addition tree operations; The second operation, the multiplier operation, and the addition tree operation are executed in parallel using a pipelined approach.

12. The method according to claim 7, characterized in that, Operations on the data to be computed in neural networks with different computational bit widths include: dot product, matrix multiplication, addition, and mixed multiplication and addition; matrix and vector multiplication, addition, and mixed multiplication and addition; matrix and constant multiplication, addition, and mixed multiplication and addition; vector multiplication, addition, and mixed multiplication and addition; vector and constant multiplication, addition, and mixed multiplication and addition; constant and constant multiplication, addition, and mixed multiplication and addition; comparison and selection of maximum / minimum values; and other operations that can be decomposed into multiplication, addition, or mixed multiplication and addition.

13. The method according to claim 7, characterized in that, The computational part performs convolutional layer operations, fully connected layer operations, and / or pooling layer operations.

Images