Data processing apparatus, neural network processor, chip and electronic device
By setting up a channel region in the NPU and optimizing the signal line layout, the problem of excessive channel area was solved, resulting in a reduction in NPU size and power consumption.
Patent Information
- Application Number
- CN202011352929.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-27
- Publication Date
- 2026-02-24
- Estimated Expiration
- 2040-11-27
AI Technical Summary
The channel region in existing neural network processors (NPUs) occupies too large an area, resulting in an increase in NPU size, and the excessively long signal lines increase power consumption.
A channel region is set between the control unit and the M-th level logic unit, and the first signal line is arranged vertically on one side of the first M-1 level logic unit to directly connect the control unit and the M-th level logic unit, thereby reducing the area occupied by the channel region.
By optimizing the layout of the channel region, the size and power consumption of the NPU were reduced, and data processing efficiency was improved.
Smart Images
Figure CN114548386B_ABST
Abstract
Description
Technical Field
[0001] This application relates to, but is not limited to, the field of electronic information technology, and in particular to a data processing device, a neural network processor, a chip, and an electronic device. Background Technology
[0002] In recent years, with the increasing application of artificial intelligence (AI) technology, neural network processing units (NPUs), along with CPUs and graphics processing units (GPUs), have become the three core components in the smartphone field. The core of the NPU model is convolution operation, and the essence of convolution is matrix multiplication and addition. Therefore, by accelerating matrix multiplication and addition operations through hardware, the goal of accelerating deep learning training and inference can be achieved.
[0003] Currently, NPUs utilize multi-level processing units to perform multiplication and addition operations on the data transmitted from the control unit, and then transmit the operation results back to the control unit through the logic unit in the last-level processing unit. Therefore, a channel area needs to be laid out between the last-level logic unit and the control unit to realize the return of the operation results. Currently, channel areas are set at the bottom of the last-level logic unit and on one side of the control unit and the multi-level logic unit, which makes the channel area too large, resulting in a large size of NPU. Summary of the Invention
[0004] This application provides a data processing device, neural network processor, chip, and electronic device that can reduce the area occupied by the channel region, thereby reducing the size of the NPU.
[0005] The technical solution of this application is implemented as follows:
[0006] This application provides a data processing apparatus, the apparatus comprising:
[0007] M-level logic units and control units, where M is an integer greater than 1, the M-level logic units are stacked, and the first-level logic unit in the M-level logic units is connected to the control unit;
[0008] A channel region is provided between the control unit and the M-th level logic unit; the channel region is located on one side of the preceding M-1 level logic unit; and a first signal line is arranged longitudinally within the channel region.
[0009] The first signal line is used to directly connect the control unit and the Mth level logic unit to realize data transmission between the Mth level logic unit and the control unit.
[0010] In the above device, a first port is provided at a first position at the bottom of the control unit, and a second port is provided at a second position at the top of the M-level logic unit. The first position is the position at the bottom of the control unit that connects with the channel region, and the second position is the position at the top of the M-level logic unit that connects with the channel region. The bottom of the control unit is opposite to the top of the M-level logic unit.
[0011] The first port and the second port are directly connected via the first signal line.
[0012] In the above device, the length of the Mth level logic unit is greater than the length of the preceding M-1 level logic units.
[0013] In the above device, the length of the Mth level logic unit is the sum of the length of the preceding M-1 level logic units and the width of the channel region.
[0014] In the above-described device, the logic implementation of the M-th level logic unit is the same as that of the preceding M-1 level logic units.
[0015] In the above-mentioned device, the device further includes an M-level computing unit and an M-level storage unit, wherein each level of the M-level logic unit is connected to a computing unit and a storage unit respectively.
[0016] In the above-described device, the control unit is used to acquire the data to be processed and transmit the data to be processed to the M-level logic unit;
[0017] The i-th level storage unit is used to store the i-th level data to be processed;
[0018] The i-th level logic unit is used to retrieve the i-th level data to be processed from the i-th level storage unit and send it to the i-th level computing unit.
[0019] The i-th level computing unit is used to perform calculations on the i-th level data to be processed, obtain the i-th level data processing result, and send it to the i-th level logic unit.
[0020] When i is less than M, the i-th level logic unit is also used to transmit the data processing result of the i-th level to the (i+1)-th level logic unit; and
[0021] When i equals M, the M-th level logic unit is also used to transmit the M-th level data processing result to the control unit.
[0022] In the above device, a plurality of second signal lines are arranged laterally in the channel region, and the number of the plurality of second signal lines corresponds to the number of the first M-1 level logic units.
[0023] In the above-described device, when the channel region is located on one side between the logic unit and the computing unit, each logic unit is connected to a computing unit at the same level through a second signal line in the channel region.
[0024] In the above-described device, when the channel region is located on one side between the logic unit and the memory unit, each logic unit is connected to a memory unit at the same level through a second signal line in the channel region.
[0025] This application provides a neural network processor, which includes the data processing device described in any of the above claims.
[0026] This application provides a chip including the neural network processor described above.
[0027] This application provides an electronic device, which includes the chip described above.
[0028] This application provides a data processing device, a neural network processor, a chip, and an electronic device. The data processing device includes an M-level logic unit and a control unit, where M is an integer greater than 1. The M-level logic units are stacked, and the first-level logic unit in the M-level logic unit is connected to the control unit. A channel region is provided between the control unit and the M-level logic unit. The channel region is located on one side of the preceding M-1 level logic units. A first signal line is arranged vertically within the channel region. The first signal line is used to directly connect the control unit and the M-level logic unit to realize data transmission between the M-level logic unit and the control unit. Using the above processor implementation scheme, the channel region is located between the M-level logic unit and the control unit, and on one side of the preceding M-1 level logic units. This eliminates the need to occupy the space below the M-level logic unit to arrange the channel region, greatly reducing the area occupied by the channel region and thus reducing the size of the NPU. Attached Figure Description
[0029] Figure 1 This application provides a schematic diagram illustrating the matrix multiplication and addition process of a convolutional neural network in an embodiment of the present application.
[0030] Figure 2 A schematic diagram of the physical implementation model of a data processing device provided in an embodiment of this application;
[0031] Figure 3 A schematic diagram of the physical implementation structure of a data processing device provided in related technologies;
[0032] Figure 4 This is a schematic diagram of the structure of a processor 1 according to an embodiment of this application;
[0033] Figure 5 This is a schematic diagram of an exemplary NPU layout proposed in an embodiment of this application;
[0034] Figure 6 This is a schematic diagram of the structure of a neural network processor 2 provided in an embodiment of this application;
[0035] Figure 7 This is a schematic diagram of the structure of a chip 3 provided in an embodiment of this application;
[0036] Figure 8 This is a schematic diagram of the structure of an electronic device 4 provided in an embodiment of this application. Detailed Implementation
[0037] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.
[0038] The technical solutions of this application will be described in detail below through embodiments and in conjunction with the accompanying drawings. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0039] It should be noted that in this application example, terms such as "first" and "second" are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0040] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.
[0041] With the increasing application of Neural Network Processing Units (NPUs) in smart terminals, the importance of NPUs is also growing. The core operation of NPUs is convolution, which is essentially matrix multiplication and addition. By accelerating matrix multiplication and addition operations through hardware, the goal of accelerating deep learning training and inference can be achieved.
[0042] Figure 1 This application provides a schematic diagram illustrating the matrix multiplication and addition process of a convolutional neural network, as shown in the embodiments of this application. Figure 1 As shown, since matrix multiplication and addition have the characteristics of 3D computation, a 3D hardware computing unit 101 can be designed to perform multiplication and addition on the elements of the matrix in parallel. Figure 1 The three-dimensional hardware computing unit 101 is a 4×4×4 three-dimensional computing unit. The 4×4×4 three-dimensional computing unit is used to perform multiplication and addition operations on two 4×4 matrices (matrix A and matrix B). The process of multiplication and addition of matrices by the 4×4×4 three-dimensional computing unit is as follows:
[0043] First, the 4×4×4 three-dimensional computing unit can compute the product of each element in parallel. Each unit in the 4×4×4 three-dimensional computing unit obtains the elements in matrix A and matrix B respectively, and performs multiplication calculations on the obtained elements.
[0044] Where, matrix A = Matrix B = .
[0045] For example, in a 4×4×4 three-dimensional computing cell, the four cells in the 0th column of the front surface calculate A respectively. 11 ×B 11 A 21 ×B 11 A 31 ×B 11 A 41 ×B 11 The last four cells of the last column of the front surface in a 4×4×4 three-dimensional computational cell calculate A respectively. 11 ×B 14 A 21 ×B 14 A 31 ×B 14 A 41 ×B 14 The four cells in the 0th column of the upper surface of the 4×4×4 three-dimensional computational cell respectively calculate A. 14 ×B 41 A 13 ×B 31 A 12 ×B 21 A 11 ×B 11 The last four cells of the top surface in a 4×4×4 three-dimensional computational cell calculate A respectively. 14 ×B 44 A 13 ×B 34 A 12 ×B 24 A 11 ×B 14 .
[0046] Secondly, the 64 products calculated are summed along a certain dimension, for example, from the front surface to the back surface, to obtain matrix A×B, which contains 16 elements.
[0047] Finally, the resulting matrix A×B is summed with matrix C to obtain matrix A×B+C. The matrix summation can be performed in parallel.
[0048] The above describes the multiplication operation of two 4×4 matrices, but in matrix multiplication... sum matrix In the case of both being 8×8 matrices, but through Figure 1 The three-dimensional hardware computing unit 101 performs two 8×8 matrix operations as follows:
[0049] Among them, matrix ,matrix .in, , , , , , , , Both are 4×4 matrices. Therefore, we can obtain... The result is: This transforms the multiplication of two 8×8 matrices into the addition and multiplication of two 4×4 matrices.
[0050] In this embodiment, the three-dimensional hardware computing unit 101 is a 4×4×4 three-dimensional computing unit, which can perform parallel multiplication operations on matrices that are multiples of 4. In other embodiments, the three-dimensional hardware computing unit 101 can be a 3×3×3 three-dimensional computing unit, which can perform parallel multiplication operations on matrices that are multiples of 3. Alternatively, the three-dimensional hardware computing unit 101 can be a 2×2×2 three-dimensional computing unit or a 5×5×5 three-dimensional computing unit, etc. This embodiment does not limit the scope of the application.
[0051] In the case where the three-dimensional hardware computing unit 101 is a 4×4×4 three-dimensional computing unit, if the dimension of the matrix to be calculated is not a multiple of 4, several rows of 0 and several rows and columns of 0 can be added to the matrix to obtain a matrix with a dimension that is a multiple of 4.
[0052] In some embodiments, the computing power of the NPU can be increased by increasing the number of units in the three-dimensional hardware computing unit 101. For example, an 8×4×4 three-dimensional computing unit has twice the computing power of a 4×4×4 three-dimensional computing unit, and a 16×4×4 three-dimensional computing unit has twice the computing power of an 8×4×4 three-dimensional computing unit.
[0053] Figure 2 This is a schematic diagram of the physical implementation model of a data processing device provided in an embodiment of this application, such as... Figure 2 As shown, the data processing device may include a control module and a multi-level processing module, for example, Figure 2In a multi-level processing module, the number of processing levels can be 4, 8, or 16, etc. Each level of the multi-level processing module includes a computing unit, a logic unit, and a memory unit. The computing units and logic units of each level of the processing module can communicate with each other, and the logic units and memory units of each level of the processing module can communicate with each other.
[0054] Each level of logic unit is used to acquire the calculation result data of each level and pass the calculation result data to the next level of logic unit. In this way, the last level of logic unit can acquire multiple calculation result data calculated by the multi-level calculation units respectively. The last level of logic unit can combine the multiple calculation result data to obtain feature map data and transmit the feature map data to the control module so that the control module can control the feature map data accordingly.
[0055] In the embodiments of this application, the operation of any logic unit to acquire data, process data, receive data, or forward data can be accomplished by a component within that logic unit.
[0056] In some embodiments, the number of levels of the logic unit can be related to the computing power of the NPU. For example, when the number of units in the three-dimensional hardware computing unit is 4, the number of levels of the logic unit is 4; when the number of units in the three-dimensional hardware computing unit is 8, the number of levels of the logic unit is 8; when the number of units in the three-dimensional hardware computing unit is 16, the number of levels of the logic unit is 16. Of course, the number of levels of the logic unit can be other than these, and is not limited here.
[0057] To enable the last-stage logic unit to transmit the obtained feature map data back to the control module, the following is currently provided: Figure 3 The chip layout shown has a large bit width of data interaction between the control module and the last-level logic unit module, and multiplexing is used between the multiple logic units. Therefore, channel regions are set at the bottom of the last-level logic unit, the left side of the multiple logic units, and the left side of the control module. Ports are set at the bottom of the last-level logic unit and the left side of the control module. The signal lines in the channel regions connect the ports at the bottom of the last-level logic unit and the ports on the left side of the control module. The above-mentioned channel regions increase the physical area of the NPU chip. The longer the signal lines in the channels, the more buffers (relay units) are needed to drive them, which greatly increases the number of buffers and thus increases the power consumption of the NPU chip.
[0058] To address the aforementioned problems, this application proposes a data processing device, a neural network processor, a chip, and an electronic device, which are specifically described through the following embodiments.
[0059] This application provides a data processing device 1, such as... Figure 4 As shown, the device 1 includes:
[0060] M-level logic unit 10 and control unit 11, where M is an integer greater than 1, the M-level logic unit 10 is stacked, and the first-level logic unit in the M-level logic unit 10 is connected to the control unit;
[0061] A channel region is provided between the control unit 11 and the M-th level logic unit 100; the channel region is located on one side of the preceding M-1 level logic unit 102; and a first signal line is arranged longitudinally within the channel region.
[0062] The first signal line is used to directly connect the control unit 11 and the M-level logic unit 100 to realize data transmission between the M-level logic unit 100 and the control unit 11.
[0063] The data processing device in this embodiment includes a control unit and an M-level logic unit. The M-level logic unit consists of a first M-1 level logic unit and an M-th level logic unit. The first M-1 level logic unit is the logic unit before the M-th level logic unit. The number of the first M-1 level logic units is one or more.
[0064] M-level logic units can have at least one of the following characteristics: M-level logic units can be the same size; M-level logic units can be cascaded; any two adjacent logic units in an M-level logic unit can be in contact; and the sides of M-level logic units can be flush when stacked.
[0065] It should be noted that M is a multiple of 4. The larger M is, the more data the data processing device can process in one cycle or one clock cycle. M can be equal to 4, 8, 16, 32, etc.
[0066] It should be noted that the data processing device is used to implement convolution operations, which are essentially matrix multiplication and addition operations. These operations are implemented using multiple levels of logic units. For example, a 4x4 matrix requires four levels of logic units, an 8x8 matrix requires eight levels, and a 16x16 matrix requires sixteen levels. Therefore, a four-level logic unit can implement the multiplication and addition operations for a 4x4 matrix; an eight-level logic unit can implement the multiplication and addition operations for 4x4 and / or 8x8 matrices; and a sixteen-level logic unit can implement the multiplication and addition operations for 4x4, 8x8, and / or 16x16 matrices.
[0067] For example, when using an eight-level logic unit to implement the multiplication and addition operation of a 4*4 matrix, the first four levels of the eight-level logic unit can be used to perform the multiplication and addition operation of the 4*4 matrix, and the last four levels of the logic unit can be used to pass the operation result.
[0068] In this embodiment, a channel region is provided between the control unit and the M-level logic unit. A first signal line is arranged vertically in the channel region to connect the M-level logic unit and the control unit. The M-level logic unit can transmit the calculation result back to the control unit through the first signal line.
[0069] In this embodiment, the channel region is located on one side of the first M-1 level logic unit; wherein, the channel region can be located on the left side of the first M-1 level logic unit, or the channel region can be located on the right side of the first M-1 level logic unit, depending on the actual situation, and this embodiment does not impose a specific limitation. It should be noted that the M-level logic units are arranged vertically in sequence, and the left and right sides of the first M-1 level logic unit are the lateral orientations of the first M-1 level logic unit.
[0070] Optionally, a first port is provided at a first position at the bottom of the control unit 11, and a second port is provided at a second position at the top of the M-level logic unit 100. The first position is the position at the bottom of the control unit 11 that connects with the channel region, and the second position is the position at the top of the M-level logic unit 100 that connects with the channel region. The bottom of the control unit 11 is opposite to the top of the M-level logic unit 100.
[0071] The first port and the second port are directly connected via the first signal line.
[0072] In this embodiment, a first port is provided at the bottom of the control unit and a second port is provided at the top of the M-level logic unit. The bottom of the control unit and the top of the M-level logic unit are opposite each other. The first port is located at the bottom of the control unit and in a first position connected to the channel region. The second port is located at the top of the M-level logic unit and in a second position connected to the channel region. The first port and the second port are directly connected through a first signal line arranged vertically in the channel region. The M-level logic unit transmits the calculation result to the control unit through the second port at the top, via the first signal line in the channel region and the first port at the bottom of the control unit.
[0073] Optionally, the length of the Mth level logic unit 100 is greater than the length of the preceding M-1 level logic units 102.
[0074] Optionally, the length of the Mth level logic unit 100 is the sum of the length of the preceding M-1 level logic units 102 and the width of the channel region.
[0075] It should be noted that in order to reserve a channel region on one side of the first M-1 level logic unit and to make the top of the M-th level logic unit connect with the channel region, the M-th level logic unit needs to be instantiated separately, so that the length of the M-th level logic unit is greater than the length of the first M-1 level logic unit. At this time, the difference in length between the M-th level logic unit and the first M-1 level logic unit can be used as the width of the channel region. Thus, the position of the channel region is reserved on one side of the first M-1 level logic unit.
[0076] Optionally, the logic implementation of the M-th level logic unit 100 is the same as the logic implementation of the first M-1 level logic units 102.
[0077] It should be noted that the logic implementation of the M-th level logic unit is the same as that of the previous M-1 level logic units, that is, the initial code of the two is the same. This application only achieves the effect of reducing the area of the channel region in the NPU by adjusting the size of the M-th level logic unit to be larger than that of the previous M-1 level logic units.
[0078] Furthermore, the length of the M-th level logic unit can be greater than the sum of the lengths of the preceding M-1 level logic units and the width of the channel region. The second position at the top of the M-th level logic unit can be connected to the channel region. This application does not impose specific limitations on this aspect.
[0079] Optionally, the device further includes an M-level computing unit and an M-level storage unit, wherein each level of the M-level logic unit is connected to a computing unit and a storage unit respectively.
[0080] In some embodiments, the computing unit and the storage unit may be connected to opposite sides of each level of logic unit. M-level computing units and M-level storage units may be stacked. In other embodiments, the computing unit and the storage unit may be connected to adjacent sides of each level of logic unit.
[0081] The logic unit can be electrically connected to the computing unit and the storage unit respectively, thereby enabling data interaction between the data logic unit and the computing unit, and between the logic unit and the storage unit.
[0082] In some embodiments, each logic unit may abut against a computing unit, or each logic unit may abut against a storage unit. In this embodiment, the abutment between two units may be: two units are in contact connection, and there are no other traces between the two units besides the trace used to connect them.
[0083] M-level compute units are side-flush when stacked, and / or M-level storage units are side-flush when stacked.
[0084] In some embodiments, any two adjacent computing units in the M-level computing unit can be electrically connected, and / or any two adjacent storage units in the M-level storage unit can be electrically connected, thereby enabling data interaction between adjacent computing units, or enabling data interaction between adjacent storage units.
[0085] The size and / or architecture of M-level computing units can be the same, and the M-level computing units can be configured using the same configuration method, and / or the size and / or architecture of M-level storage units can be the same, and the M-level storage units can be configured using the same configuration method.
[0086] In the embodiments of this application, any one of the M-level computing units includes: an addition computing unit and / or a multiplication computing unit.
[0087] The computing unit can perform addition and / or multiplication on the data to be computed. Each level of computing unit can acquire the data to be computed, perform calculations on the data, obtain the data processing results, and send the obtained data processing results to the logic unit at the same level and the logic units below it, and then to the last level logic unit (the Mth level logic unit). Thus, the last level logic unit can obtain M data processing results to send to the control unit based on the data processing results calculated by each level of computing unit, and transmit the M data processing results to the control unit through the through channel.
[0088] The M-level storage unit in this application embodiment includes one or a combination of the following: registers, register groups consisting of at least two registers, random access memory (RAM), read-only memory (ROM), cache, flash memory, and double data rate synchronous dynamic random access memory (DDR).
[0089] The following describes the data processing process of each level (i-th level) logic unit, each level (i-th level) computing unit, and each level storage unit in the data processing device:
[0090] The control unit is used to acquire data to be processed and transmit the data to be processed to the M-level logic unit; the i-th level storage unit is used to store the i-th level data to be processed; the i-th level logic unit is used to acquire the i-th level data to be processed from the i-th level storage unit and send it to the i-th level computing unit; the i-th level computing unit is used to perform calculations on the i-th level data to be processed, obtain the i-th level data processing result, and send it to the i-th level logic unit; when i is less than M, the i-th level logic unit is also used to transmit the i-th level data processing result to the (i+1)-th level logic unit; and when i is equal to M, the M-th level logic unit is also used to transmit the M-th level data processing result to the control unit.
[0091] The data to be processed can be matrix A in the above embodiments (or ), B (or ) and C.
[0092] The data to be processed may include two parts: the first part may include image feature data, and the second part may include convolutional data and / or bias terms used to perform convolutional processing on the image feature data.
[0093] In some embodiments, the first portion of data may include the matrix A described above or The second part of the data may include the aforementioned matrix B or In other embodiments, the first portion of data may include the matrix B described above or The second part of the data may include the aforementioned matrix A or In some embodiments, the first or second portion of data may include matrix C.
[0094] In some embodiments, the data to be processed can first be sent to the first-level logic unit. The first-level logic unit obtains the data that the first-level computing unit needs to process and stores the first-level data to be processed in the first-level storage unit. The first-level logic unit sends the remaining data to the second-level logic unit. The second-level logic unit obtains the data that the second-level computing unit needs to process and stores the second-level data to be processed in the second-level storage unit. The remaining data is then sent to the third-level logic unit until the Mth-level logic unit obtains the Mth-level data to be processed.
[0095] The data to be processed at level i can be all or part of the data that the level i computing unit needs to process. For example, a part of the data at level i can be the second part of the data at level i.
[0096] In this way, each level of the M-level computing unit will obtain the first-level data processing result, and transmit it to the M-level logic unit through the logic unit of each level and the lower-level logic unit of each level, so that the M-level logic unit can obtain the M-level data processing result.
[0097] The control unit in this embodiment can be used to control target data. In some embodiments, the control unit can perform subsequent processing on the obtained target data. For example, the control unit can input the obtained target data into the activation layer, and the data obtained through the activation layer can be sent to the pooling layer to obtain pooled data. The pooled data can then be input into the M-level logic unit through the control unit, or the pooled data can be input into the fully connected layer.
[0098] It should be noted that when an M-level logic unit performs convolution calculations on the data to be processed, the data processing result can be the feature map data corresponding to the convolution calculation result.
[0099] Optionally, each logic unit in the M-level logic unit is also horizontally connected to a computing unit and a storage unit.
[0100] It should be noted that for each level of logic unit, each level of logic unit is horizontally connected to a computing unit and a storage unit. The first level of logic unit, the first level of computing unit, and the first level of storage unit constitute the first level of processing unit.
[0101] It should be noted that each memory unit may include one register or a register group consisting of at least two registers.
[0102] It should be noted that the data parameters of multiple storage units are the same.
[0103] Optionally, a plurality of second signal lines are arranged laterally in the channel region, and the number of the plurality of second signal lines corresponds to the number of the first M-1 level logic units.
[0104] In this embodiment, multiple second signal lines are arranged horizontally in the channel region. The number of the multiple second signal lines corresponds to the number of the first M-1 level logic units. The multiple second signal lines are used to connect the first M-1 level logic units to the corresponding computing units, or to connect the first M-1 level logic units to the corresponding storage units. Specifically, the main bodies connected by the multiple second signal lines are determined according to the setting position of the channel region, and this embodiment does not limit this.
[0105] In an alternative embodiment, when the channel region is located on one side between the logic unit and the computing unit, each logic unit is connected to a computing unit at the same level through a second signal line in the channel region.
[0106] In another alternative embodiment, when the channel region is located on one side between the logic unit and the memory unit, each logic unit is connected to a memory unit at the same level through a second signal line in the channel region.
[0107] For example, Figure 5 The NPU layout proposed in this application includes a control unit and four levels of processing units. Each level of processing unit corresponds to a logic unit, a memory unit, and an addition / multiplication unit. The logic implementation of the four levels of logic units is the same. The dimensions of the first, second, and third level logic units are the same, and the length of the fourth level logic unit is greater than the length of the first three level logic units. The length difference between the fourth level logic unit and the first three level logic units is the width of the channel region. The channel region is connected to the bottom of the control unit and the top of the fourth level logic unit, respectively. An input port (not shown) is provided at the junction of the channel region and the top of the fourth level logic unit, and an output port (not shown) is provided at the junction of the channel region and the bottom of the control unit. The input port and the output port are connected by signal lines (not shown) arranged vertically within the channel region. The channel region is located to the left of the first three level logic units, and the first three level logic units are connected to the corresponding multiplication / addition units on the left side by signal lines (not shown) arranged horizontally within the channel region.
[0108] It is understood that in the embodiments of this application, without changing the original code, the area occupied by the channel region in the NPU is reduced by simply instantiating the size of the M-th level logic unit, thereby reducing the area of the NPU chip; at the same time, due to the reduction in the length of the channel region, the number of buffers in the timing path is reduced, which is beneficial to timing convergence and thus reduces the power consumption of the NPU chip.
[0109] Figure 6 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of this application, such as... Figure 6 As shown, the neural network processor 2 may include the data processing device 1 in any of the above embodiments.
[0110] The neural network processor 2 can be a programmable logic device, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).
[0111] This application embodiment may provide a chip that includes the aforementioned neural network processor.
[0112] Figure 7 This is a schematic diagram of the structure of a chip provided in an embodiment of this application, such as... Figure 7 As shown, the chip 3 includes not only the aforementioned neural network processor 2, but also a central processing unit (CPU) 301 and a graphics processing unit (GPU) 302. The GPU 302, CPU 301, and neural network processor 2 can be packaged together using a packaging process, which includes one of the following: chip on board (COB) packaging, system in a package (SIP), system on a chip (SOC) packaging, or chip stacking.
[0113] In some embodiments, chip 3 may include neural network processor 2 and central processing unit 301, but not graphics processor 302. In other embodiments, chip 3 may include neural network processor 2 and graphics processor 302, but not central processing unit 301.
[0114] In some implementations, the chip 3 may also include an input interface (not shown). The graphics processor 302, the central processing unit 301, or the neural network processor 2 can control the input interface to communicate with other devices or the chip 3, specifically, to acquire information or data sent by other devices or chips.
[0115] In some implementations, the chip 3 may also include an output interface (not shown). The graphics processor 302, the central processing unit 301, or the neural network processor 2 can control the output interface to communicate with other devices or the chip 3, specifically, to output information or data to other devices or the chip 3.
[0116] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0117] This application provides an electronic device 4. Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, in an embodiment of this application, the electronic device 4 includes the chip 3 described in the above embodiments.
[0118] It should be noted that, in the embodiments of this application, electronic devices can refer to terminal devices, which may include: servers, mobile phones, tablets, laptops, PDAs, personal digital assistants, portable media players, smart speakers, navigation devices, display devices, wearable devices such as smartwatches, virtual reality (VR) devices, augmented reality (AR) devices, pedometers, digital TVs, desktop computers, devices in intelligent driving, wireless-fidelity (Wi-Fi) access points, evolved base stations, next-generation communication base stations such as 5G base stations, small cells, micro cells, or transmission reception points (TRPs), and may also be relay stations, access points, or vehicle-mounted devices, etc., any device capable of performing convolutional processing on data. A processor is disposed therein. The specific electronic devices described in the embodiments of this application are not limited.
[0119] It should be noted that, in the embodiments of this application, the electronic device 4 may include not only the chip 3, but also devices such as a display screen and a battery to provide corresponding functions. The embodiments of this application do not limit this.
[0120] It is understood that in the embodiments of this application, the electronic device 4 includes the chip 3 described above. The chip 3 has its channel region located between the control unit and the M-th level logic unit, on one side of the first M-1 level logic unit, thereby reducing the size of the chip. Therefore, the electronic device 4 can implement convolution operations in neural networks using a smaller chip.
[0121] It should be noted that the descriptions of the neural network processor, chip, and electronic device embodiments are similar to the descriptions of the data processing device embodiments described above, and the same embodiments have the same or similar beneficial effects. For technical details not disclosed in the neural network processor, chip, and electronic device embodiments of this application, please refer to the descriptions of the data processing device embodiments of this application for understanding.
[0122] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.
[0124] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.
Claims
1. A data processing apparatus, characterized in that, The device includes: M-level logic units and control units, where M is an integer greater than 1, the M-level logic units are arranged vertically in sequence, and the first-level logic unit in the M-level logic units is connected to the control unit; Wherein, a channel region is provided between the control unit and the M-th level logic unit; the channel region is located on one side of the preceding M-1 level logic unit; a first signal line is arranged longitudinally within the channel region; the width of the channel region is the length difference between the M-th level logic unit and the preceding M-1 level logic unit; The first signal line is used to directly connect the control unit and the Mth level logic unit to realize data transmission between the Mth level logic unit and the control unit.
2. The apparatus according to claim 1, characterized in that, A first port is provided at a first position at the bottom of the control unit, and a second port is provided at a second position at the top of the M-level logic unit. The first position is the position at the bottom of the control unit that connects with the channel region, and the second position is the position at the top of the M-level logic unit that connects with the channel region. The bottom of the control unit is opposite to the top of the M-level logic unit. The first port and the second port are directly connected via the first signal line.
3. The apparatus according to claim 1, characterized in that, The logic implementation of the M-th level logic unit is the same as that of the preceding M-1 level logic units.
4. The apparatus according to claim 1, characterized in that, The device further includes an M-level computing unit and an M-level storage unit, wherein each level of the M-level logic unit is connected to a computing unit and a storage unit respectively.
5. The apparatus according to claim 4, characterized in that, The control unit is used to acquire the data to be processed and transmit the data to be processed to the M-level logic unit; The i-th level storage unit is used to store the i-th level data to be processed; The i-th level logic unit is used to retrieve the i-th level data to be processed from the i-th level storage unit and send it to the i-th level computing unit. The i-th level computing unit is used to perform calculations on the i-th level data to be processed, obtain the i-th level data processing result, and send it to the i-th level logic unit. When i is less than M, the i-th level logic unit is also used to transmit the data processing result of the i-th level to the (i+1)-th level logic unit. as well as When i equals M, the M-th level logic unit is also used to transmit the M-th level data processing result to the control unit.
6. The apparatus according to claim 5, characterized in that, Multiple second signal lines are arranged horizontally in the channel region, and the number of the multiple second signal lines corresponds to the number of the first M-1 level logic units.
7. The apparatus according to claim 6, characterized in that, When the channel region is located on one side between the logic unit and the computing unit, each logic unit is connected to a computing unit at the same level through a second signal line in the channel region.
8. The apparatus according to claim 6, characterized in that, When the channel region is located on one side between the logic unit and the memory unit, each logic unit is connected to a memory unit at the same level through a second signal line in the channel region.
9. A neural network processor, characterized in that, The neural network processor includes the data processing apparatus according to any one of claims 1 to 8.
10. A chip, characterized in that, Includes the neural network processor of claim 9.
11. An electronic device, characterized in that, The electronic device includes the chip as described in claim 10.