Buffer operation circuit for neural network, chip and computing device
By setting up one-to-one corresponding computing units and buffer units in the neural network circuit, efficient computing and storage of data in the sliding window is achieved, and the problems of hardware resource occupation and high memory access are solved, and the efficiency and applicability of the circuit are improved.
Patent Information
- Application Number
- CN202310552614.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-22
AI Technical Summary
Under the acceleration performance requirements, the existing neural network pooling computing circuit has excessive hardware resource usage and high memory access times, resulting in increased power consumption and area costs, and low buffer unit utilization, which cannot effectively reduce the number of memory access times.
Design a buffering operation circuit for neural networks. By setting up a plurality of operation units and buffer units one by one, each operation unit performs calculations for data in the window area covered by the sliding window, and the data in the window area is input. The unit reads and inputs the corresponding calculation unit from the memory, and stores the calculation results into the corresponding buffer unit. At the end of the calculation, the output unit writes the results to the memory to realize the buffer storage of data one by one, making full use of the buffer unit to reduce the Number of memory accesses.
The area occupied by the circuit is reduced, the storage efficiency and overall operation efficiency of the buffer unit are improved, the number of accesses to the memory is reduced, and tensors of various sizes are applied, which improves the universality of the circuit.
Smart Images

Figure CN119003441A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of integrated circuit design, and in particular to a buffer operation circuit, chip and computing device for a neural network. Background Art
[0002] With the rise of artificial intelligence (AI) technology, the computing power requirements of hardware systems are getting higher and higher. AI acceleration chips are chips specifically used for AI algorithms, providing efficient computing power support for AI systems. Pooling is a common operator in AI algorithms. Because it is closely related to the convolution operator, the acceleration of the Pooling operator is one of the important functions of AI acceleration chips, which has an important impact on the acceleration capability of the entire chip.
[0003] The operation of the pooling operator is mainly to sample a tensor according to certain parameter settings (such as sliding window size, window sliding step), and then perform certain operations on the sampled data (such as maximum value, minimum value, average value, etc.), and output a downsampled and compressed tensor.
[0004] For the design of pooling operator circuits, two key issues usually need to be addressed:
[0005] First, under certain acceleration performance requirements, how to reduce the hardware resources occupied by the acceleration circuit, because more hardware resources mean higher power consumption and area cost.
[0006] Second, under certain acceleration performance requirements, how to reduce the number of memory accesses of the acceleration circuit to the storage unit (such as static random access memory). Because memory access to the storage unit consumes a lot of power. In addition, reducing the number of memory accesses can save the system's memory access bandwidth.
[0007] Existing acceleration circuits (such as pooling operation circuits) for performing operations on tensors in neural networks according to sliding windows mainly include the following two solutions.
[0008] In the first scheme, the buffer unit of the acceleration circuit buffers and stores the data of the input tensor. When the data buffering of the selected data area of each sliding window is completed, the operation unit performs functional operation, outputs the operation result, and writes it to the memory. The number of buffer units in this scheme is determined according to the size of the sliding window it supports and its specific buffering strategy.
[0009] Solution 2: The buffer unit of the acceleration circuit buffers and stores the data of the output tensor. During the period when the input unit reads the data of the input tensor, the function calculation is completed and the calculation result is stored in the buffer unit. The buffer unit stores a row of data of the output tensor. The number of buffer units in this solution is related to the buffer strategy of each row of the output tensor set.
[0010] In the above solution 1, since the pooling operation is a downsampling operation, the size of the input tensor of each operation is always larger than the size of the output tensor, so the amount of input data is always larger than the amount of output data, and buffering the input data will require a large amount of buffer resources, thereby increasing the area cost of the circuit. The size of the sliding window that this solution can support is limited by the buffer resources, that is, in order to support the operation of a larger sliding window, more buffer resources need to be designed.
[0011] In the above-mentioned solution 2, since all the buffer units are used to buffer and store one row of data of the output tensor, when the number of output rows of data is small, part of the buffer area will be idle, thereby reducing the utilization rate of the buffer unit.
[0012] The scheme has limited reuse of input data, increases the number of reads of external SRAM (Static Random-Access Memory), and increases system power consumption. Because only one row of result data is buffered and output, the reuse of input data between multiple rows cannot be utilized. Summary of the invention
[0013] An embodiment of the present disclosure provides a buffer operation circuit for a neural network, the circuit comprising: a first preset number of operation units, a first preset number of buffer units, an input unit, an output unit and a memory, wherein the operation units and the buffer units correspond one to one; an input unit, used to determine, according to the size of the sliding window and the sliding step, to which the window area to which each data included in the input tensor stored in the memory belongs, wherein each window area in the input tensor corresponds to an operation unit; reading the data in the input tensor from the memory, and inputting the read data into the corresponding operation unit; an operation unit in the first preset number of operation units, used to operate the received data according to a preset operation method, obtain an operation result set, and input the operation result set into the corresponding buffer unit; an output unit, used to obtain an operation result set from the corresponding buffer unit when the operation of any operation unit in the first preset number of operation units is completed, and store the obtained operation result set into the memory as the data included in the output tensor.
[0014] In some embodiments, the operation unit in the first preset number of operation units includes a second preset number of operation sub-units, each channel of the input tensor corresponds to an operation sub-unit, and the operation sub-unit in the second preset number of operation sub-units is used to perform operations on the data of the corresponding channels of the input according to a preset operation method.
[0015] In some embodiments, the buffer units in the first preset number of buffer units include a second preset number of data registers, each data register corresponds to an operator unit, and each data register in the second preset number of data registers is used to store the operation result output by the corresponding operator unit.
[0016] In some embodiments, the input unit includes a first counter group, which is used to count the points corresponding to the data read by the input unit from the input tensor; the input unit is further used to: read the data corresponding to each point from the memory in sequence according to the arrangement order of the points in the input tensor; use the counter group to count the data currently read; determine at least one operation unit corresponding to the data currently read according to the current count value of the counter group, and input the data currently read into at least one operation unit.
[0017] In some embodiments, the first counter group includes at least one of the following counters: a first type of counter for counting the points included in each window area in the tensor, a second type of counter for counting the partial points that have been read in each row in the tensor, a third type of counter for counting the number of partial rows that have been read in the tensor, a fourth type of counter for counting all the points that have been read in each row in the tensor, a fifth type of counter for counting the total number of rows that have been read in the tensor, and a sixth type of counter for counting the number of channels that have been read in the tensor.
[0018] In some embodiments, the input unit is further used to: in response to the current count value of the first counter group indicating that the operation on the data included in the target window area has been completed, send an operation end prompt signal corresponding to the target window area to the output unit; the output unit is further used to: in response to receiving the operation end prompt signal sent by the input unit, write the operation result set in the buffer unit corresponding to the target window area into the memory.
[0019] In some embodiments, the output unit includes a second counter group for counting the set of operation results written into the memory after the operation results in the buffer unit corresponding to the target window area are written into the memory.
[0020] In some embodiments, the second counter group includes at least one of the following counters: a seventh type of counter for counting the number of partial points written in each row of the output tensor, an eighth type of counter for counting the number of partial rows written in the output tensor, a ninth type of counter for counting all the points written in each row of the output tensor, a tenth type of counter for counting the total number of rows written in the output tensor, and an eleventh type of counter for counting the number of channels written in the output tensor.
[0021] According to another aspect of an embodiment of the present disclosure, a chip is provided, which includes the above-mentioned buffer operation circuit for a neural network.
[0022] According to another aspect of an embodiment of the present disclosure, a computing device is provided, which includes the above chip.
[0023] The buffer operation circuit, chip and computing device for neural networks provided by the above-mentioned embodiments of the present disclosure, by setting a plurality of operation units and buffer units corresponding to each other, when performing tensor operations on the neural network, each operation unit operates on the data in the window area covered by the corresponding sliding window, the data in the window area is read from the memory by the input unit and input into the corresponding operation unit, the operation unit stores the operation result set into the corresponding buffer unit, and when the operation of any operation unit is completed, the output unit stores the operation result set in the corresponding buffer unit into the memory as the data included in the output tensor. The multiple buffer units adopted in the embodiment of the present disclosure greatly reduce the circuit area occupied by the single buffer unit with a larger capacity commonly used currently; and the data of the output tensor is buffered and stored in the corresponding buffer unit point by point. Therefore, each buffer unit can be fully utilized in the operation process to avoid the situation where part of the space in the buffer unit is idle, thereby improving the storage efficiency of the buffer unit; in addition, compared with the currently commonly used pooling operation circuit, which needs to read a large amount of repeated data from the memory as the sliding window moves, when the data corresponding to each batch of points in the output tensor is calculated by each operation unit in the embodiment of the present disclosure, it is only necessary to read the data corresponding to the current batch contained in the input tensor in the memory once, and the read data can be reused by multiple operation units at the same time, which greatly reduces the number of accesses to the memory and improves the operating efficiency of the entire circuit; the circuit provided by the embodiment of the present disclosure is not limited by the size of the sliding window and the sliding step size, and can also be applicable to tensors of various sizes, thereby improving the universality of circuit application.
[0024] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps;
[0026] Figure 1 is a schematic diagram of a buffer operation circuit for a neural network provided by an exemplary embodiment of the present disclosure;
[0027] Figure 2 is a structural schematic diagram of an operation unit and a buffer unit included in a buffer operation circuit for a neural network provided by an exemplary embodiment of the present disclosure;
[0028] Figure 3 Schematic diagram of various window areas of a buffer operation circuit for a neural network provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0029] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
[0030] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.
[0031] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.
[0032] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
[0033] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.
[0034] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
[0035] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.
[0036] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0037] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.
[0038] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.
[0039] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0040] Application Overview
[0041] The disclosed embodiment aims to solve the above-mentioned problem and proposes a buffer operation circuit for a neural network, wherein the circuit is provided with a plurality of operation units and a plurality of buffer units in one-to-one correspondence, wherein each operation unit corresponds to an area covered by a sliding window, and the corresponding buffer unit stores the calculation result, and the circuit area occupied by the circuit is greatly reduced; and the data of the output tensor is buffered and stored in the corresponding buffer unit point by point, so that each buffer unit can be fully utilized during the operation process to avoid the situation where part of the space in the buffer unit is idle, thereby improving the storage efficiency of the buffer unit; in addition, when each operation unit calculates the data corresponding to each batch of points in the output tensor, it is only necessary to read the data corresponding to the current batch contained in the input tensor in the memory once, and the read data can be reused by multiple operation units at the same time, which greatly reduces the number of accesses to the memory and improves the operation efficiency of the entire circuit; the circuit is not limited by the size of the sliding window and the sliding step size, and can also be applicable to tensors of various sizes, thereby improving the universality of circuit application.
[0042] Exemplary Structure
[0043] Figure 1 The schematic diagram of the structure of a buffer operation circuit for a neural network provided by an exemplary embodiment of the present disclosure is shown in FIG. The various components included in the circuit can be integrated into one chip, or can be set in different chips or circuit boards, and data communication links are established between these chips or circuit boards.
[0044] like Figure 1 As shown, the circuit includes: a first preset number of operation units 101, a first preset number of buffer units 102, an input unit 103, an output unit 104 and a memory 105, wherein the operation units and the buffer units correspond to each other one by one. The above-mentioned input unit 103 and output unit 104 can be devices that include input and output interfaces and can perform certain logic, control and other functions, such as microcontrollers, microprocessors, etc. The operation unit can be composed of a circuit that can perform logical operations, such as a device that can perform operations such as maximum value, minimum value, average value, etc. The buffer unit can be composed of devices such as registers, SRAM, etc. that can temporarily store data. The memory 105 can be various types of storage devices, such as SRAM, DRAM (Dynamic Random Access Memory), etc.
[0045] Typically, the first preset number of operation units 101 and the first preset number of buffer units 102 are arranged in an array, such as Figure 1 As shown, the array includes n rows, each row includes m computing units and m buffer units, so the first preset number is m*n. Figure 1 The operation units included in the 0th row are marked as Func_0_0, Func_0_1, ..., Func_0_m-1, and the buffer units included in the 0th row are marked as Buffer_0_0, Buffer_0_1, ..., Buffer_0_m-1.
[0046] In this embodiment, the input unit 103 is used to determine the window area to which each data included in the input tensor stored in the memory 105 belongs according to the size of the sliding window (kernel) and the sliding step (stride). Among them, each window area in the input tensor corresponds to an operation unit. The input tensor is usually composed of multiple matrices, and the number of matrices is also called the number of channels. The data corresponding to the same position in each channel can be called the data corresponding to a point of the input tensor in this embodiment. The input unit 103 can determine the window area covered by the sliding window after each sliding according to the preset sliding window size and sliding step, and then determine the window area to which each point in the input tensor belongs.
[0047] The input unit 103 further reads data in the input tensor from the memory 105 and inputs the read data into the corresponding operation unit.
[0048] As an example, Figure 3As shown, the width w of the input tensor is 6, the height h is 5, the number of channels c is 128 (c0-c127), and the points marked 0-29 in the figure correspond to the data of the 128 channels of the input tensor. The size of the sliding window is 3*3, the sliding step is 1, and the data corresponding to the 9 points in the window area covered by the sliding window in the input tensor (i.e., as shown in 301-312 in the figure) are calculated by the corresponding operation units. It can be seen from the figure that the same point in the feature tensor is contained in at least one window area. Therefore, the input unit 103 can input the data corresponding to the same point into at least one corresponding operation unit.
[0049] In this embodiment, the operation units in the first preset number of operation units 101 are used to operate on the received data according to a preset operation method to obtain an operation result set, and input the operation result set into a corresponding buffer unit.
[0050] The above-mentioned preset operation mode can be any operation mode. For example, the circuit can be used as a pooling operator in a neural network accelerator, that is, the operation unit performs operations such as averaging, taking the maximum value, taking the minimum value, etc. on the data of each channel in the corresponding window area. The operation unit can output an operation result set after one operation, and the operation result set includes at least one value, and each value corresponds to a channel. For example, if the operation unit Func_0_0 performs operations on Figure 3 The operation is performed on the first 64 channels (c0-c63) of the 9 points included in the window area 301, and the output operation result set includes 64 values.
[0051] It should be noted that, usually, since the operation unit needs to update the result of this operation according to the result of the previous operation on an input point during the operation, the operation result set in the buffer unit can be updated according to Figure 1 As shown by the arrow in , the operation result set is input as an operand into the corresponding operation unit, so that the operation unit continues to operate based on the last operation result set and the data in the input tensor received this time.
[0052] In this embodiment, the output unit 104 is used to obtain a set of operation results from the corresponding buffer unit when any operation unit in the first preset number of operation units 101 completes operation, and store the obtained set of operation results as data included in the output tensor into the memory 105.
[0053] Specifically, when the data corresponding to all points in a window area are input into the corresponding operation unit, the operation result set output by the operation unit is the operation result set at the end of the operation, and the output unit 104 can read the operation result set from the buffer unit corresponding to the operation unit. The output unit 104 can also assign an identifier to the operation result set to mark the position of the operation result set corresponding to the point in the output tensor.
[0054] The circuit provided by the above-mentioned embodiment of the present disclosure sets a plurality of one-to-one corresponding operation units and buffer units. When performing tensor operations on a neural network, each operation unit operates on the data in the window area covered by the corresponding sliding window. The data in the window area is read from the memory by the input unit and input into the corresponding operation unit. The operation unit stores the operation result set into the corresponding buffer unit. When the operation of any operation unit is completed, the output unit stores the operation result set in the corresponding buffer unit into the memory as the data included in the output tensor. The multiple buffer units adopted in the embodiment of the present disclosure greatly reduce the circuit area occupied by the single buffer unit with a larger capacity commonly used currently; and the data of the output tensor is buffered and stored in the corresponding buffer unit point by point. Therefore, each buffer unit can be fully utilized in the operation process to avoid the situation where part of the space in the buffer unit is idle, thereby improving the storage efficiency of the buffer unit; in addition, compared with the currently commonly used pooling operation circuit, which needs to read a large amount of repeated data from the memory as the sliding window moves, when the data corresponding to each batch of points in the output tensor is calculated by each operation unit in the embodiment of the present disclosure, it is only necessary to read the data corresponding to the current batch contained in the input tensor in the memory once, and the read data can be reused by multiple operation units at the same time, which greatly reduces the number of accesses to the memory and improves the operating efficiency of the entire circuit; the circuit provided by the embodiment of the present disclosure is not limited by the size of the sliding window and the sliding step size, and can also be applicable to tensors of various sizes, thereby improving the universality of circuit application.
[0055] In some optional implementations, the operation units in the first preset number of operation units 101 include a second preset number of operation sub-units, each channel of the input tensor corresponds to an operation sub-unit, and the operation sub-units in the second preset number of operation sub-units are used to perform operations on the data of the corresponding channels of the input according to a preset operation method.
[0056] like Figure 2 As shown in the figure, for the operation unit Func_0_0, it includes 64 operation sub-units (marked as Func_0_0_0-Func_0_0_63 in the figure), that is, the second preset number is 64, and each operation sub-unit corresponds to a channel of the input tensor. If the number of channels of the input tensor is 128, then Func_0_0 corresponds to Figure 3The first 64 channels of the 9 points in the window area 301 in the input unit 103 are operated respectively, and each operation subunit outputs an operation result, that is, Func_0_0 outputs an operation result set containing 64 values. After Func_0_0 completes the operation on the first 64 channels, the input unit 103 can continue to input the data of the last 64 channels (c64-c127) according to the preset operation rules, and Func_0_0 operates on the last 64 channels (for example, after all the operation units have completed the operation on the first 64 channels, Func_0_0 continues to operate on the last 64 channels; or, Func_0_0 Figure 3 After the calculation of the first 64 channels of the window area 301 is completed, the calculation of the last 64 channels of the window area is continued and completed).
[0057] This embodiment sets a second preset number of operation sub-units in each operation unit, and each operation sub-unit calculates the corresponding channel, so as to realize parallel calculation of multi-channel input tensors and flexibly set the number of channels for each operation, thereby helping to improve the operation efficiency of the circuit.
[0058] In some optional implementations, the buffer units in the first preset number of buffer units 102 include a second preset number of data registers, each data register corresponds to an operator unit, and each data register in the second preset number of data registers is used to store the operation result output by the corresponding operator unit.
[0059] As an example, Figure 2 As shown, the buffer unit Buffer_0_0 includes 64 data registers, which are marked as 0-63 in the figure, and are used to store 64 operation results output by Func_0_0, that is, each data register is used to store the operation result of a channel of the corresponding sliding area.
[0060] This embodiment sets a second preset number of data registers in each buffer unit to store the operation results of the corresponding channels, thereby realizing parallel storage and updating of the operation results of multiple channels in multiple window areas, fully improving the utilization rate of each buffer unit, thereby helping to reduce the idle rate of the buffer unit and reducing the area of the buffer unit in the circuit.
[0061] In some optional implementations, the input unit 103 includes a first counter group for counting points corresponding to data read by the input unit 103 from the input tensor.
[0062] The input unit 103 is further used for:
[0063] First, according to the arrangement order of the points in the input tensor, the data corresponding to each point is read from the memory 105 in sequence.
[0064] For example, you can follow Figure 3 The serial numbers of the points shown in the figure, i.e. 0-29, are read in sequence to obtain the data corresponding to each point.
[0065] Then, the counter group is used to count the data currently read.
[0066] For example, the counters included in the counter group may count the number of rows and columns of points currently read.
[0067] At least one operation unit corresponding to the currently read data is determined according to the current count value of the counter group, and the currently read data is input into the at least one operation unit.
[0068] For example, based on the count values of the number of rows and columns of the currently read point, the input unit 103 can determine the position of the currently read point in the input tensor and the window area to which the currently read point belongs, and then determine the corresponding operation unit.
[0069] As an example, Figure 3 As shown, it shows an exemplary schematic diagram of each window area when operating on the data of an input tensor. The size of the input tensor w×h×c is 6×5×128. The number of operation units and buffer units used in this example is 9. Since the operation units and buffer units are arranged in the form of an array in the circuit, that is, according to Figure 1 The arrangement shown in the figure shows that the operation units and buffer units in this example are 3 rows and 3 columns, that is, the operation units in the first row include Func_0_0, Func_0_1, and Func_0_2, and the corresponding buffer units include Buffer_0_0, Buffer_0_1, and Buffer_0_2. Figure 3 The window areas 301-303 in the second row include Func_1_0, Func_1_1, and Func_1_2, and the corresponding buffer units include Buffer_1_0, Buffer_1_1, and Buffer_1_2. Figure 3 The window area 304-306 in the second row of operation units includes Func_2_0, Func_2_1, Func_2_2, and the corresponding buffer units include Buffer_2_0, Buffer_2_1, Buffer_2_2, corresponding Figure 3 When reading each point included in the input tensor, the sliding window is read in the order of moving horizontally 3 times and vertically 3 times. That is, in the first round of reading, 5 points are read in each row, and a total of 5 rows are read, so as to adapt to Figure 3 The distribution pattern of the middle window area 301-309.
[0070] After the first round of reading and calculation, Figure 3 The last three columns of data are sequentially read from the middle window area 310-312.
[0071] Specifically, for Figure 3 For the first 64 channels of the input tensor shown, the input unit 103 unit performs the first round of reading as follows:
[0072] Read the data of point 0, determine that point 0 belongs to window area 301 according to the count value, input the data of point 0 into the operation unit Func_0_0, Func_0_0 does not need to be operated at this time, and directly inputs the data of point 0 into Buffer_0_0;
[0073] Read the data of point 1, determine that point 1 belongs to windows 301 and 302, input the data of point 1 into operation units Func_0_0 and Func_0_1, Func_0_0 operates on the data cached in Buffer_0_0 and the data of point 1, and uses the operation result to update Buffer_0_0. Func_0_1 does not need to operate, and directly inputs the data of point 1 into Buffer_0_1;
[0074] Read the data of point 2, determine that point 2 belongs to windows 301, 302 and 303, input the data of point 2 into operation units Func_0_0, Func_0_1 and Func_0_2, Func_0_0 operates on the data cached in Buffer_0_0 and the data of point 2, and updates Buffer_0_0 with the obtained operation result, Func_0_1 operates on the data cached in Buffer_0_1 and the data of point 2, and updates Buffer_0_1 with the obtained operation result, Func_0_2 does not need to operate, and directly inputs the data of point 2 into Buffer_0_2;
[0075] And so on, according to Figure 3In the order shown by the arrows, Func_0_0, Func_0_1, Func_0_2, Func_1_0, Func_1_1, Func_1_2, Func_2_0, Func_2_1, and Func_2_2 respectively operate on window areas 301-309, and the operation results are stored in Buffer_0_0, Buffer_0_1, Buffer_0_2, Buffer_1_0, Buffer_1_1, Buffer_1_2, Buffer_2_0, Buffer_2_1, and Buffer_2_2 respectively. When point 28 is read, Func_2_2 operates on the data cached in Buffer_2_2 and the data at point 28, and updates Buffer_2_2 with the obtained operation results.
[0076] In targeting Figure 3 After the output point calculation corresponding to each window area in the window areas 301-309 shown in the figure is completed, the output unit 104 writes the data in each buffer unit into the memory 105. According to the steps similar to the first round of reading and calculation, Figure 3 The window areas 310-312 shown are respectively operated by Func_0_0, Func_1_0, and Func_2_0, and the operation results are respectively stored in Buffer_0_0, Buffer_1_0, and Buffer_2_0.
[0077] After the input unit 103 finishes reading the data of the first 64 channels of the input tensor, it continues to read the data of the last 64 channels of the input tensor according to the above steps.
[0078] This embodiment reads the data corresponding to each point in the input tensor from the memory in sequence, counts through the first counter group, and can quickly and accurately determine the operation unit corresponding to each read point according to the count value, so as to quickly and accurately perform operations such as pooling operations on the input vector.
[0079] In some optional implementations, the first counter group includes at least one of the following counters:
[0080] A first type of counter for counting the points included in each window area in the tensor, a second type of counter for counting the partial points that have been read in each row in the tensor, a third type of counter for counting the number of partial rows that have been read in the tensor, a fourth type of counter for counting all the points that have been read in each row in the tensor, a fifth type of counter for counting the total number of rows that have been read in the tensor, and a sixth type of counter for counting the number of channels that have been read in the tensor.
[0081] The number of the first-class counters can correspond to the operation units and buffer units. For example, if the operation units and buffer units are 3 rows and 3 columns, the number of the first-class counters is 9, corresponding to one operation unit and one buffer unit respectively. For example, the first-class counters corresponding to Func_0_0 and Buffer_0_0 are marked as atom_wcnt_0_0 and atom_hcnt_0_0. Figure 3 The window area 301 shown in the figure counts the columns of points included in it, and atom_hcnt_0_0 counts Figure 3 The rows of points included in the window area 301 shown are counted. Figure 3 When the first 64 channels of the input tensor shown in the figure are operated, that is, after reading the data of point 0, atom_wcnt_0_0 is incremented by 1; after reading the data of point 1, atom_wcnt_0_0 is incremented by 1; after reading the data of point 2, atom_wcnt_0_0 is reset, and atom_hcnt_0_0 is incremented by 1; after reading the data of point 6, atom_wcnt_0_0 is incremented by 1; and so on, after reading the data of point 14, atom_wcnt_0_0 is reset, and atom_hcnt_0_0 is reset. At this time, it means that the data of the points in the window area 301 have been read, that is, Func_0_0 has completed the operation of the points in the window area 301. At this time, the data in Buffer_0_0 is the operation result of the first 64 channels of the points in the window area 301.
[0082] The read partial points and read partial rows in each row may be the number of columns and rows of read points in each round of operation when multiple rounds of operation are performed using all the operation units and buffer units in the above example.
[0083] The second type of counter is labeled r_mini_wcnt. Figure 3 When the data is read sequentially from the window areas 301-309 shown, r_mini_wcnt is incremented by 1 after reading the data of one point in each row. If the data of the last point in a row is read, r_mini_wcnt is reset.
[0084] The third type of counter is labeled r_mini_hcnt. Figure 3 When reading data sequentially from the window areas 301-309 shown in the figure, after reading the data of the last point in each row, r_mini_hcnt is incremented by 1. If the last point in the last row (i.e. Figure 3 After the data at point 28) is read, r_mini_hcnt is reset.
[0085] The fourth counter and the fifth counter are marked as r_big_wcnt and r_big_hcnt, that is, after each round of reading, all rows and columns that have been read are counted.
[0086] The sixth type of counter is marked as r_ccnt, which records the number of channels that have been calculated after calculating some channels of all points of the input tensor. Figure 3 After the operation of the first 64 channels of the input tensor is completed, r_ccnt is incremented by 64, and after the operation of the last 64 channels of the input tensor is completed, r_ccnt is reset.
[0087] The various types of counters included in the first counter group provided in this embodiment can realize detailed and accurate marking of each point read from the input tensor, so that the position of the read point in the input tensor can be accurately and quickly determined by using the count value, thereby improving the calculation accuracy of the neural network accelerator.
[0088] In some optional implementations, the input unit 103 is further configured to:
[0089] In response to the current count value of the first counter group indicating that the operation on the data included in the target window area has been completed, an operation completion prompt signal corresponding to the target window area is sent to the output unit 104 .
[0090] The target window area is the window area in each window area where the operation has been completed. As an example, if the above counters atom_wcnt_0_0 and atom_hcnt_0_0 are both reset, it means Figure 3 The data included in the target window area 301 shown has been calculated. At this time, the input unit 103 can send a calculation completion prompt signal to the output unit 104.
[0091] The output unit 104 is further configured to:
[0092] In response to receiving the operation completion prompt signal sent by the input unit 103 , the operation result set in the buffer unit corresponding to the target window area is written into the memory 105 .
[0093] In this embodiment, the input unit sends a calculation completion prompt signal to the output unit, so that when the calculation of the target window area is completed, the output unit can write the calculation result set into the memory in time, thereby improving the efficiency of circuit calculation.
[0094] In some optional implementations, the output unit 104 includes a second counter group for counting the set of operation results written into the memory 105 after the operation results in the buffer unit corresponding to the target window area are written into the memory 105 .
[0095] Since the operation result set is the data corresponding to the point in the output tensor, each operation result set outputted can be counted to determine the corresponding point of each operation result set in the output tensor. For example, the second counter group can count the number of rows and columns of the point corresponding to the currently output operation result set in the output tensor.
[0096] By setting a second counter group in the output unit, this embodiment can quickly and accurately determine the corresponding points of the operation result set in the output tensor after the operation unit completes the operation, thereby helping to improve the operation efficiency of the neural network accelerator.
[0097] In some optional implementations, the second counter group includes at least one of the following counters:
[0098] A seventh counter for counting the number of partial points written in each row of the output tensor, an eighth counter for counting the number of partial rows written in the output tensor, a ninth counter for counting all the points written in each row of the output tensor, a tenth counter for counting all the rows written in the output tensor, and an eleventh counter for counting the number of channels written in the output tensor.
[0099] The partial points written in each row and the partial rows written in the above-mentioned example can be the number of columns and rows of corresponding points in the output tensor of the set of operation results output after each round of operation when multiple rounds of operation are performed using all the operation units and buffer units as in the above-mentioned example.
[0100] For example, the seventh type of counter is marked as w_mini_wcnt. Figure 3 During the sequential operation of the window areas 301-309 shown, while the sliding window slides horizontally by one row, i.e., during the period shown in the window areas 301-303, each time a set of operation results is output, w_mini_wcnt is incremented by 1. After the data of the last point of a sliding row is output, w_mini_wcnt is reset.
[0101] The eighth type of counter is marked as w_mini_hcnt. Figure 3 During the sequential operation of the window areas 301-309 shown, after the sliding window slides horizontally for one row and outputs the last set of operation results for one row (i.e., after the operation of the window areas shown in 303 and 306 is completed), w_mini_hcnt is incremented by 1. If the last row of sliding is completed and the last set of operation results for the last row is output (i.e., after the operation of the window area shown in 309 is completed), w_mini_hcnt is reset.
[0102] The ninth and tenth counters are marked as w_big_wcnt and w_big_hcnt, that is, after each round of operation, all rows and columns of the output tensor in which the operation results have been output are counted.
[0103] The eleventh type of counter is marked as w_ccnt, which records the number of channels of the output operation result set after the operation is completed on some channels of all points of the input tensor. Figure 3 After the operation of the first 64 channels of the input tensor is completed, w_ccnt is incremented by 64, and after the operation of the last 64 channels of the input tensor is completed, w_ccnt is reset.
[0104] The various types of counters included in the second counter group provided in this embodiment can realize detailed and accurate marking of each output point, so that the position of the point corresponding to the output operation result set in the input and output tensor can be accurately and quickly determined by using the count value, thereby helping to improve the overall computing efficiency of the neural network accelerator.
[0105] The embodiment of the present disclosure further provides a chip on which a buffer operation circuit for a neural network is integrated. The technical details of the buffer operation circuit for a neural network are as follows: Figure 1-Figure 3 As shown in the relevant description, no further description is given here.
[0106] The embodiments of the present disclosure also provide a computing device, which includes the chip described in the above embodiments. In addition, the computing device may also include an input device, an output device, and necessary memory, etc. Among them, the input device may include such as a mouse, a keyboard, a touch screen, a communication network connector, etc., for inputting input tensors to be calculated, etc. The output device may include such as a display, a printer, and a communication network and its connected remote output device, etc., for outputting the output tensors described in the above embodiments, etc. The memory is used to store the data input by the above input device, as well as the data generated during the operation of the buffer operation circuit for the neural network. The memory may include volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.
[0107] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0108] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other.
[0109] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0110] The circuits of the present disclosure may be implemented in many ways. For example, the circuits of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps of the method used in the circuit is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the functions of the circuits of the present disclosure. Therefore, the present disclosure also covers recording media storing programs for executing the functions of the circuits of the present disclosure.
[0111] It should also be noted that in the circuit of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as an equivalent solution of the present disclosure.
[0112] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0113] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A buffer operation circuit for a neural network, comprising: A first preset number of operation units, a first preset number of buffer units, an input unit, an output unit, and a memory, wherein the operation units and the buffer units correspond one to one; The input unit is used to determine, according to the size of the sliding window and the sliding step size, the window area to which each data included in the input tensor stored in the memory belongs, wherein each window area in the input tensor corresponds to an operation unit; read the data in the input tensor from the memory, and input the read data into the corresponding operation unit; The operation unit in the first preset number of operation units is used to operate the received data according to a preset operation method to obtain an operation result set, and input the operation result set into a corresponding buffer unit; The output unit is used to obtain a set of operation results from the corresponding buffer unit when the operation of any operation unit among the first preset number of operation units is completed, and store the obtained set of operation results as data included in the output tensor into the memory.
2. The circuit according to claim 1, wherein The operation units in the first preset number of operation units include a second preset number of operation sub-units, each channel of the input tensor corresponds to an operation sub-unit, and the operation sub-units in the second preset number of operation sub-units are used to operate on the data of the corresponding channels of the input according to the preset operation method.
3. The circuit according to claim 2, wherein: The buffer units in the first preset number of buffer units include a second preset number of data registers, each data register corresponds to an operator unit, and each data register in the second preset number of data registers is used to store the operation result output by the corresponding operator unit.
4. The circuit according to claim 1, wherein: The input unit includes a first counter group for counting points corresponding to data read by the input unit from the input tensor; The input unit is further used for: Reading data corresponding to each point from the memory in sequence according to the arrangement order of the points in the input tensor; Counting the currently read data using the counter group; At least one operation unit corresponding to the currently read data is determined according to the current count value of the counter group, and the currently read data is input into the at least one operation unit.
5. The circuit according to claim 4, wherein: The first counter group includes at least one of the following counters: A first type of counter for counting the points included in each window area in the tensor, a second type of counter for counting the partial points that have been read in each row in the tensor, a third type of counter for counting the number of partial rows that have been read in the tensor, a fourth type of counter for counting all the points that have been read in each row in the tensor, a fifth type of counter for counting the total number of rows that have been read in the tensor, and a sixth type of counter for counting the number of channels that have been read in the tensor.
6. The circuit according to claim 4, wherein: The input unit is further used for: In response to the current count value of the first counter group indicating that the operation on the data included in the target window area has been completed, sending a prompt signal indicating that the operation is completed corresponding to the target window area to the output unit; The output unit is further used for: In response to receiving the calculation end prompt signal sent by the input unit, the calculation result set in the buffer unit corresponding to the target window area is written into the memory.
7. The circuit according to claim 6, wherein: The output unit includes a second counter group, which is used to count the set of operation results written into the memory after the operation results in the buffer unit corresponding to the target window area are written into the memory.
8. The circuit according to claim 7, wherein: The second counter group includes at least one of the following counters: A seventh type of counter for counting the number of partial points written in each row of the output tensor, an eighth type of counter for counting the number of partial rows written in the output tensor, a ninth type of counter for counting all the points written in each row of the output tensor, a tenth type of counter for counting the total number of rows written in the output tensor, and an eleventh type of counter for counting the number of channels written in the output tensor.
9. A chip, characterized in that: The invention comprises a buffer operation circuit for a neural network according to any one of claims 1 to 8.
10. A computing device, characterized in that: Comprising the chip according to claim 9.
Citation Information
Patent Citations
Device for executing LSTM neural network operation and operation method
CN108268939A
Convolutional neural network accelerator based on FPGA parallelism degree self-adaption
CN113191493A
In-memory computing circuit chip based on magnetic cache and computing device
CN113539318A
Neural network processing system, instruction generation method and device and electronic equipment
CN114724103A
