Distributed processing device, neural network processing unit and chip system

Abstract

The invention discloses a distributed processing device, a neural network processing unit and a chip system, and belongs to the technical field of data processing. The distributed processing device comprises a plurality of processing modules, the processing modules are used for cooperatively processing a first task in a distributed mode, and the processing cycles needed by the processing modules for processing sub-tasks distributed to the first task are different; and the plurality of processing modules are respectively used for processing the distributed sub-tasks of the first task according to the synchronization period of the first task, and the synchronization period of the first task is determined based on the maximum value in the processing periods respectively corresponding to the plurality of processing modules. According to the distributed processing device, the processing cycles of processing the same task by the plurality of processing modules are the same, so that the effect of synchronously controlling the plurality of processing modules is achieved, processing errors caused by different processing cycles are avoided, and the accuracy of distributed processing is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a distributed processing device, a neural network processing unit, and a chip system. Background Art

[0002] In the field of data processing technology, the same task can be distributed to multiple processing modules for processing, and then the processing results of multiple processing modules are aggregated to obtain the final processing result of the task. This distributed processing method of multiple processing modules and the synchronization performance between multiple processing modules will affect the overall performance of the distributed processing device. Summary of the invention

[0003] The present application provides a distributed processing device, a neural network processing unit and a chip system, so that multiple processing modules in the distributed processing device have the same processing cycle for the same task.

[0004] In a first aspect, a distributed processing device is provided, the distributed processing device comprising a plurality of processing modules, the plurality of processing modules being used for distributed collaborative processing of a first task, the processing cycles required for the plurality of processing modules to process subtasks of the first task being distributed to the subtasks being different;

[0005] The multiple processing modules are respectively used to process the subtasks of the first task distributed thereto according to the synchronization period of the first task, and the synchronization period of the first task is determined based on the maximum value of the processing periods respectively corresponding to the multiple processing modules.

[0006] In a possible implementation manner, the multiple processing modules are sequentially connected in a chain manner;

[0007] A first processing module located at the end of the plurality of processing modules is used to send a first processing cycle of the first processing module to a second processing module connected upstream;

[0008] the second processing module is configured to receive the first processing cycle, determine a first intermediate cycle which is larger between the first processing cycle and a second processing cycle of the second processing module, and send the first intermediate cycle to a third processing module connected upstream;

[0009] The third processing module is used to receive the first intermediate cycle and determine a second intermediate cycle which is larger between the first intermediate cycle and a third processing cycle of the third processing module;

[0010] In the case that the third processing module is located at the first position among the multiple processing modules, the third processing module is further configured to use the second intermediate cycle as a synchronization cycle of the first task.

[0011] In a possible implementation manner, when the third processing module is not located at the first position among the multiple processing modules, the third processing module is further configured to send the second intermediate cycle to a fourth processing module connected upstream;

[0012] The fourth processing module is used to receive the second intermediate cycle and determine a third intermediate cycle which is larger between the second intermediate cycle and a fourth processing cycle of the fourth processing module;

[0013] In the case where the fourth processing module is located at the first position among the multiple processing modules, the fourth processing module is further configured to use the third intermediate cycle as a synchronization cycle of the first task.

[0014] In a possible implementation manner, the third processing module is further configured to send a synchronization period of the first task to the second processing module;

[0015] The second processing module is further configured to receive the synchronization period of the first task and send the synchronization period of the first task to the first processing module.

[0016] In a possible implementation, any processing module among the multiple processing modules includes a synchronization cycle queue, and the synchronization cycle queue is used to store the synchronization cycle of each task;

[0017] Any one of the processing modules is further used to obtain the synchronization period of the first task, and write the synchronization period of the first task into the synchronization period queue;

[0018] Any one of the processing modules is used to process the subtasks of the first task distributed thereto according to the synchronization cycle of the first task when the synchronization cycle of the first task is read from the synchronization cycle queue.

[0019] In a possible implementation, the distributed processing device further includes an aggregation module; the aggregation module is used to aggregate the processing results corresponding to the multiple processing modules respectively, when the multiple processing modules have completed the subtasks of the first task to which they are distributed, to obtain the processing results of the first task.

[0020] In a possible implementation, any one of the multiple processing modules is also used to send the processing results of each processing cycle in the synchronization cycle of the first task to the aggregation module; and when the synchronization cycle of the first task is reached, a completion signal is sent to the aggregation module, and the completion signal indicates that any one of the processing modules has completed processing the subtask of the first task distributed to it.

[0021] In a possible implementation manner, the first task is the sum of N multiplication operations, and the subtask to which any processing module is distributed is the sum of M multiplication operations of the N multiplication operations, where M is a positive integer and N is a positive integer greater than M;

[0022] Any of the processing modules is a computing unit cluster PEC, the PEC includes P multiplication units and an accumulator, the P multiplication units are used for distributed processing of multiplication operations, the accumulator is used to obtain the sum of the operation results of the P multiplication units, and P is a positive integer;

[0023] The processing cycle of any one of the processing modules is a value obtained by rounding up the quotient of M' and P, where M' is the number of multiplication operations in which the multiplier is not zero among the M multiplication operations.

[0024] In a possible implementation manner, the synchronization period of the first task is greater than the processing period of any processing module;

[0025] Any one of the processing modules is used to calculate the sum of the M' multiplication operations through the P multiplication units and an accumulator within a processing cycle of the any one of the processing modules;

[0026] Zero-padding calculation is performed outside the processing cycle of any processing module and within the synchronization cycle of the first task.

[0027] In a second aspect, a neural network processing unit is also provided, and the neural network processing unit includes the distributed processing device described in the first aspect.

[0028] In a third aspect, a chip system is also provided, comprising a processor and the neural network processing unit described in the second aspect.

[0029] The technical solution provided by this application can at least bring the following beneficial effects:

[0030] The distributed processing device provided by the present application, for the scenario where the same task is distributed to multiple processing modules for joint processing, obtains the synchronization cycle according to the maximum processing cycle of the multiple processing modules, so that the multiple processing modules can all process the same task according to the synchronization cycle. As a result, the processing cycles of the multiple processing modules for processing the same task are the same, achieving the effect of synchronous control of the multiple processing modules, avoiding processing errors caused by different processing cycles, and improving the accuracy of distributed processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 is a structural diagram of a distributed processing device provided in an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of the structure of another distributed processing device provided in an embodiment of the present application;

[0034] Figure 3 It is a schematic diagram of the organizational structure of a PEC provided in an embodiment of the present application;

[0035] Figure 4 It is a schematic diagram of the organizational structure of a MAC provided in an embodiment of the present application;

[0036] Figure 5 It is a schematic diagram of a convolution operation process provided by an embodiment of the present application;

[0037] Figure 6 This is a schematic diagram of a task allocation process provided by an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of data compression / non-compression provided by an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of a MAC processing process provided by an embodiment of the present application;

[0040] Fig. 9 is a schematic diagram of a synchronization cycle acquisition process provided by an embodiment of the present application;

[0041] Fig.10 It is a schematic diagram of a fifo queue and processing cycle provided in an embodiment of the present application;

[0042] Fig.11 It is a schematic diagram of a synchronization process and a processing process provided by an embodiment of the present application;

[0043] Fig.12 is a structural schematic diagram of a neural network processing unit provided in an embodiment of the present application;

[0044] Fig.13 is a schematic diagram of the structure of a chip system provided in an embodiment of the present application;

[0045] Fig.14It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0047] In the field of data processing technology, distributed processing can effectively improve the data processing speed. Distributed processing is to connect multiple processing modules to jointly process the same task. For example, the same task is divided into multiple subtasks, and the multiple subtasks are respectively issued to multiple processing modules, that is, one processing module is used to process one subtask, and finally the processing results of multiple modules are aggregated to obtain the task result.

[0048] In some scenarios, the processing cycles required by multiple processing modules for the subtasks of the same task may be different. The processing cycle refers to the number of cycles required for the processing module to complete the corresponding subtask. For example, the amount of data that a processing module can process at one time is a, and the amount of data of the subtask to be processed is b, then the processing cycle is the rounded-up value of b / a. Therefore, if the amount of data distributed to different processing modules is different, or the amount of data that different processing modules can process at a single time is different (i.e., the processing capabilities are different), the processing cycles of multiple processing modules for the subtasks of the same task will be different. This will lead to errors when aggregating the processing results of different tasks, such as aggregating the processing results of the subtasks of the first task into the second task.

[0049] The embodiment of the present application provides a distributed processing device, wherein the distributed processing device includes multiple processing modules capable of synchronously processing the same task. For example, Figure 1 A schematic diagram of the structure of a distributed device provided in an embodiment of the present application. Optionally, the processing module may be a computer device, a device cluster, a processor or chip in a device, a computing unit or computing cluster on a processor, etc. For example, the processing module may be a processing element (PE) or a computing unit cluster (PEC) in a neural processing unit (NPU). Among them, the module types between multiple processing modules may be the same or different, and the embodiment of the present application does not limit this.

[0050] Optionally, the distributed processing device provided in the embodiment of the present application further includes an aggregation module, which is used to aggregate the processing results of multiple processing modules. The embodiment of the present application does not limit the connection relationship between the multiple processing modules and the aggregation module. Optionally, taking the number of the multiple processing modules as 3 as an example, Figure 2As shown in (1), multiple processing modules can be connected in a chain, and the last processing module is connected to the aggregation module, that is, the previous processing module sends the processing result to the next processing module until it is transmitted to the aggregation module. Or, Figure 2 As shown in (2), multiple processing modules can be connected to the aggregation module respectively, that is, the multiple processing modules send the processing results to the aggregation module respectively.

[0051] For example, taking the processing module as PEC, Figure 3 A schematic diagram of the organizational structure of a PEC provided in an embodiment of the present application. Figure 3 As shown, the four PECs are organized in the form of accumulation, and are connected by an adder (ADDR) on the right side of each PEC. The four PECs are used to distribute the same task. Each PEC includes four PEs, which are responsible for processing the corresponding subtasks. Taking each PE as responsible for multiplication operations as an example, each PEC can complete four multiplication operations in one processing cycle. Therefore, if the processing cycles of the four PECs for the same task are not synchronized, processing errors will easily occur, for example, the processing results of the second task are incorrectly aggregated into the processing results of the first task. Therefore. How to ensure that multiple PECs have the same processing cycle for the same task is an urgent problem to be solved.

[0052] In an embodiment of the present application, the distributed device includes multiple processing modules for distributed and collaborative processing of a first task, and the processing cycles required for processing the subtasks of the first task distributed to the multiple processing modules are different; the multiple processing modules are respectively used to process the subtasks of the first task distributed to them according to the synchronization cycle of the first task, and the synchronization cycle of the first task is determined based on the maximum value of the processing cycles corresponding to the multiple processing modules.

[0053] Thus, the distributed processing device obtains the synchronization cycle according to the maximum processing cycle of the multiple processing modules for the scenario where the same task is distributed to multiple processing modules for joint processing, so that the multiple processing modules can all process the same task according to the synchronization cycle. Thus, the processing cycles of the multiple processing modules for processing the same task are the same, achieving the effect of synchronous control of the multiple processing modules, avoiding processing errors caused by different processing cycles, and improving the accuracy of distributed processing.

[0054] In the embodiment of the present application, after the first task is distributed to multiple processing modules, the multiple processing modules correspond to a subtask of the first task respectively. The processing cycles corresponding to the multiple processing modules are the number of cycles required for the multiple processing modules to process the corresponding subtasks respectively, and the processing cycle is determined by the single processing capacity of the processing module and the task volume of the distributed subtasks.

[0055] In a possible implementation, a synchronization unit is added to any processing module among the multiple processing modules, and the synchronization unit is used to perform operations related to obtaining the synchronization period of the first task. The synchronization unit can be implemented by hardware, or by executing corresponding software implementation by hardware, which is not limited in the embodiments of the present application. For example, Figure 4 A schematic diagram of the organizational structure of a multiply-add calculation (MAC) provided in an embodiment of the present application, wherein the multiple processing modules may be Figure 4 The multiple PECs shown in FIG. 1 correspond to the distributed processing device of the embodiment of the present application. Figure 3 , the PEC also includes a synchronization unit, and the original four PEs and ADDR can be regarded as processing units. In addition, an accumulator is connected to the right of the last PEC, and the accumulator is responsible for accumulating the processing results of multiple processing cycles to obtain the final processing result of the first task. The accumulator corresponds to the aggregation module of the embodiment of the present application.

[0056] Exemplarily, the first task is the sum of N multiplication operations, and the subtask to which any processing module is distributed is the sum of M multiplication operations among the N multiplication operations, where M is a positive integer and N is a positive integer greater than M. Any processing module among the multiple processing modules is a PEC, and the PEC includes P multiplication units (for example, a PE responsible for multiplication operations) and an accumulator, the P multiplication units are used for distributed processing of multiplication operations, and the accumulator is used to obtain the sum of the operation results of the P multiplication units, where P is a positive integer. Then the processing cycle corresponding to any processing module is the value of the quotient of M and P rounded up. In the zero-jumping operation scenario, the processing cycle of any processing module is the value of the quotient of M' and P rounded up, and M' is the number of multiplication operations in which the multiplier is not zero among the M multiplication operations.

[0057] The zero-skipping operation scenario refers to skipping multiplication operations with zero multipliers, which can be used to improve computing efficiency. Zero-skipping operations often occur in sparse neural networks, which refer to neural networks with many zero activations, such as deep neural networks (DNN) or convolutional neural networks (CNN).

[0058] One reason why sparse neural networks have many zeros is that they use the Rectified Linear Unit (ReLU) as an activation function, which sets negative values ​​to zero, and this sparsity becomes more sparse as the depth of the neural network layer increases. Another reason is that many neural networks exist in the form of autoencoders or generative adversarial networks, which contain decoder layers that upsample the input feature maps using zero insertion, resulting in more than 75% zeros. Therefore, efficiency can be effectively improved by skipping zero operations.

[0059] In the scenario of accelerating by zero skipping operation, it is easy to cause different processing cycles between multiple PECs, that is, to cause PEC to be out of sync. Figure 5 Take the convolution operation process in the CNN shown in the figure as an example. For the input feature map, a convolution kernel performs a convolution operation to obtain an element in the output feature map. Among them, a convolution operation includes R*S*C multiplication operations and R*S*C-1 addition operations. R, S and C are the three side lengths of the convolution kernel respectively. Figure 5 The calculation process shown is: (1*1+2*2+3*3+4*4+5*5+6*6+7*7+8*8+9*9)*4=1140, and the multiplication operations required are 4*3*3=36 times and the addition operations are 35 times.

[0060] In order to speed up the convolution operation, a convolution operation can be used as a task and distributed to multiple processing modules. Each processing module processes a part of the multiplication operation, and then adds the results of the multiplication operation together to obtain the processing result, that is, an element of the output feature map, which can be represented by F. Figure 4 Taking the 4 PECs shown in the figure as an example, when R=2, S=2, and C=16, 2*2*16 multiplication operations are evenly distributed to the 4 PECs. For example, each PEC is assigned 16 Cs with different R and S, such as Figure 6 As shown, PEC3 is allocated R0S0-C16, PEC2 is allocated R0S1-C16, PEC1 is allocated R1S0-C16, and PEC0 is allocated R1S1-C16, that is, each PEC is allocated 16 multiplication operations.

[0061] Since each PEC only includes 4 PEs, one processing cycle can complete 4 multiplication operations. Therefore, 16 multiplication operations require 4 processing cycles, that is, 4 cycles, and the accumulator also needs to accumulate 4 cycles. If the input data is not subjected to zero-skipping operation, the processing cycle required to be calculated by each PEC is consistent, that is, the synchronization cycle is 4, and the synchronization cycle of multiple tasks (i.e., multiple convolution operations) is 4. Then, an element F of the output feature map is obtained after calculating 4 cycles. If the input data is subjected to zero-skipping operation, because the multiplication operation data assigned to each PEC is different, the processing cycle required to be calculated by each PEC may be inconsistent after the data is compressed by zero-skipping.

[0062] For example, Figure 7 As shown in the figure, for the 16 multiplication operations of the input, 4 cycles are required without compression (i.e. without zero-skipping operation); and only 1 cycle is required with compression (i.e. with zero-skipping operation). Therefore, in the scenario of zero-skipping operation, even if the number of multiplication operations assigned to multiple PECs is the same, after zero-skipping compression, the number of cycles for calculating the same element between multiple PECs may be inconsistent, which may lead to calculation errors or failure to exert the overall performance of distributed processing.

[0063] In the embodiment of the present application, the processing cycle of PEC may refer to the cycle of a clock pulse, and 4 processing cycles are 4 pulses. Figure 4 The four PECs shown are all triggered by the rising edge of the same clock pulse, that is, one clock pulse completes one cycle of processing, and four cycles are required, which means that four clock pulses are required for processing. A clock pulse is a periodically repeated signal used to time and synchronize operations in electronic systems. A clock pulse is usually a square wave signal with a fixed frequency and duty cycle, and is mainly used to synchronize the operation of the processor. Figure 8 A schematic diagram of a MAC processing process provided by an embodiment of the present application. The data allocated to multiple PECs is compressed by the skip 0 processor and then input into the corresponding PEC, and pulse-by-pulse calculation is performed according to the control of the clock pulse (clock, clk).

[0064] like Figure 8It can be seen from the calculation process shown on the upper right side that at the first rising edge, PEC3 calculates the input data in3 to obtain the calculation result 1, and sends the calculation result 1 to PEC2; at the second rising edge, PEC2 calculates the input data in2 to obtain the calculation result 2, and sends the calculation results 1 and 2 to PEC1; at the third rising edge, PEC1 calculates the input data in1 to obtain the calculation result 3, and sends the calculation results 1-calculation result 3 to PEC0; at the fourth rising edge, PEC0 calculates the input data in0 to obtain the calculation result 4, and sends the calculation results 1-calculation result 4 to the accumulator; at the fifth rising edge, the accumulator accumulates the calculation results 1-calculation result 4 to obtain the output result out. In this process, since the processing cycles of PEC3-PEC0 are the same, which are all 1, the calculation is synchronized and there is no calculation error.

[0065] For example, if in0 = [1006 0080 0030 0000], the number of valid digits after zero processing is 4, and one cycle is required for calculation; in1 = [0030 0000 0000 0006], the number of valid digits after zero processing is 2, and one cycle is required for calculation; in2 = [0040 0010 0050 6070], the number of valid digits after zero processing is 5, and two cycles are required for calculation; in3 = [0036 0000 0050 0005], the number of valid digits after zero processing is 2, and one cycle is required. Then according to Figure 8 It can be seen from the calculation process shown on the lower right side that due to the different processing cycles of PEC3-PEC0, that is, PEC2 calculated 2 cycles, while PEC3, PEC1 and PEC0 calculated 1 cycle, the accumulator has accumulated and output at the fifth rising edge, but the second round of calculation of PEC2 has not been sent to the accumulator, resulting in the loss of the data of the second round of calculation of PEC2. Alternatively, it may also cause the data of the second round of calculation of PEC2 to be accumulated in the next calculation task.

[0066] Therefore, the embodiment of the present application obtains the synchronization cycle to make the processing cycles of multiple processing modules processing the same task consistent, which can avoid the above-mentioned calculation errors caused by asynchrony. Optionally, the synchronization cycle can be the maximum value of the processing cycles corresponding to the multiple processing modules, that is, the maximum processing cycle; or it can be a value greater than the maximum processing cycle.

[0067] The embodiment of the present application does not limit the method for obtaining the synchronization period of the first task. Optionally, multiple processing modules broadcast processing periods to each other, so that any processing module can receive processing periods sent by other processing modules, and then determine the maximum processing period as the synchronization period; or, the control end collects the processing periods corresponding to each processing module, and then calculates the synchronization period according to the processing periods corresponding to each processing module and sends it to each processing module, so that any processing module can obtain the synchronization period.

[0068] For the scenario in which multiple processing modules are chained in sequence, the method for obtaining the synchronization period of the first task can be: the first processing module that is at the last position among the multiple processing modules is used to send the first processing period of the first processing module to the second processing module connected upstream; the second processing module is used to receive the first processing period, determine the larger first intermediate period between the first processing period and the second processing period of the second processing module, and send the first intermediate period to the third processing module connected upstream; the third processing module is used to receive the first intermediate period, determine the larger second intermediate period between the first intermediate period and the third processing period of the third processing module; when the third processing module is at the first position among the multiple processing modules, the third processing module is also used to use the second intermediate period as the synchronization period of the first task.

[0069] The third processing module is further used to send the synchronization cycle of the first task to the second processing module; the second processing module is further used to receive the synchronization cycle of the first task and send the synchronization cycle of the first task to the first processing module.

[0070] Optionally, when the third processing module is not located at the first position among the multiple processing modules, the third processing module is further used to send the second intermediate cycle to the fourth processing module connected upstream; the fourth processing module is used to receive the second intermediate cycle and determine the larger third intermediate cycle between the second intermediate cycle and the fourth processing cycle of the fourth processing module; when the fourth processing module is located at the first position among the multiple processing modules, the fourth processing module is further used to use the third intermediate cycle as the synchronization cycle of the first task.

[0071] Among them, the fourth processing module is also used to send the synchronization cycle of the first task to the third processing module; the third processing module is also used to receive the synchronization cycle of the first task and send the synchronization cycle of the first task to the second processing module; the second processing module is also used to receive the synchronization cycle of the first task and send the synchronization cycle of the first task to the first processing module.

[0072] In summary, the process by which multiple processing modules obtain the synchronization cycle mainly includes two sub-processes, namely, the process of searching for the maximum processing cycle upstream and transmitting the maximum processing cycle downstream, so that each processing module can obtain the maximum processing cycle, and then can determine a consistent synchronization cycle based on the maximum processing cycle.

[0073] For example, see Fig. 9 The schematic diagram of the acquisition process of the synchronization cycle shown in the figure. Take the processing module as PEC as an example, starting from the last PEC0, the maximum cycle is searched for upstream PEC1, PEC2 and PEC3 in turn, that is, PEC0 sends cycle0 corresponding to PEC0 to PEC1; PEC1 compares the size of cycle0 with the size of cycle1 corresponding to PEC1, and sends the larger cycle` to PEC2; PEC2 compares the size of cycle` with the size of cycle2 corresponding to PEC2, and sends the larger cycle`` to PEC3; PEC3 compares the size of cycle`` with the size of cycle3 corresponding to PEC3, and takes the larger value of cycle`` and cycle3 as the maximum cycle (i.e., the synchronization cycle). Then, PEC3 transmits the maximum cycle to downstream PEC2, PEC1 and PEC0 in turn.

[0074] In a possible implementation, any of the multiple processing modules includes a synchronization cycle queue. For example, the synchronization unit of any processing module includes a synchronization cycle queue, and the synchronization cycle queue is used to store the synchronization cycle of each task. Any processing module is also used to obtain the synchronization cycle of the first task and write the synchronization cycle of the first task into the synchronization cycle queue. Optionally, any processing module also includes a downstream cycle queue and a local cycle queue. The local cycle queue is used to store the processing cycles required for calculating the subtasks of processing each task, and the downstream cycle queue is used to store the processing cycles sent by the downstream connected processing module. The method of storing the cycle data of each stage in a queue makes the synchronization process clearer and less prone to errors.

[0075] The embodiment of the present application does not limit the queue, for example, a first-in-first-out (FIFO) queue can be used. The FIFO queue allows insertion operation at one end (called the tail of the queue) and deletion operation at the other end (called the head of the queue). The data structure characteristic of the FIFO queue is that the element that enters the queue earliest will be removed from the queue first, which ensures the sequentiality of data processing. In this case, since the data will be deleted after reading, another local cycle queue, i.e., two local cycle queues, can be added. One of the local cycle queues is used to compare with the processing cycle in the downstream cycle queue, and the other local cycle queue is used to record the processing cycle of multiple subtasks.

[0076] For example, for the above Figure 4 In the MAC shown in FIG. 1 , each PEC includes a synchronization unit, and the synchronization unit includes four fifo queues, namely pre_comp_fifo (i.e., downstream cycle queue), max_cnt_fifo (i.e., synchronization cycle queue), pec_cnt_fifo (i.e., local cycle queue for recording), and pec_comp_fifo (i.e., local cycle queue for comparison). The four fifo queues can be as follows Fig.10 As shown in (1), the four fifo queues are responsible for storing the number of calculation cycles required for each element of the output feature map. For the last PEC, such as PEC0, since there is no downstream PEC, all pre_comp_fifo can be set to zero.

[0077] Among them, the writing process of the fifo queue includes: each time PEC reads data from the memory of PEC (such as the multipliers of the above-mentioned 16 multiplication operations), it calculates the corresponding cycle number according to the read data, such as the cycle number of F0, and writes it into pec_cnt_fifo and pec_comp_fifo. As long as pec_cnt_fifo is not full, PEC can always fetch data from the memory; when PEC receives the larger cycle number after comparison sent by the downstream, it writes it into pre_comp_fifo; when PEC receives the maximum cycle number sent by the upstream, it writes it into max_cnt_fifo.

[0078] Taking the synchronization period as the maximum number of cycles and the first task as calculating F0 as an example, the process of obtaining the maximum number of cycles includes: each PEC compares the recorded pre_comp_fifo and pec_comp_fifo, and passes the maximum value therein upstream to the pre_comp_fifo of the connected PEC, so that the last PEC can obtain the maximum number of cycles required to calculate F0 after the transfer; after the last PEC obtains the maximum number of cycles, it records it in max_cnt_fifo, and then passes the maximum number of cycles downstream to the max_cnt_fifo of the connected PEC. Thus, each PEC will record the maximum number of cycles in max_cnt_fifo. The synchronization process is a pipeline process, and while obtaining the maximum number of cycles of F0, it is also obtaining the maximum number of cycles of F1, which is the second task, and then the maximum number of cycles of each task is obtained.

[0079] After obtaining the synchronization period of the first task, any processing module can process the distributed subtasks according to the synchronization period of the first task. Optionally, multiple processing modules are also used for distributed collaborative processing of the second task; multiple processing modules are also used to process the distributed subtasks of the second task according to the synchronization period of the second task, and the synchronization period of the second task is determined based on the maximum value of the processing periods corresponding to the multiple processing modules.

[0080] In the case where any processing module includes a synchronous cycle queue, it is used to process the subtasks of the first task distributed thereto according to the synchronous cycle of the first task when the synchronous cycle of the first task is read from the synchronous cycle queue. Thus, through the synchronous cycle queue, the synchronization process and the processing process can be two independent pipelines, the synchronous cycle of each subtask is obtained on the synchronization pipeline and written into the synchronous cycle queue, the synchronous cycle queue is read on the processing pipeline, and each subtask is processed according to the synchronous cycle of each subtask.

[0081] In an embodiment of the present application, if the synchronization period of the first task is greater than the processing period of any processing module; any processing module is used to calculate the sum of M' multiplication operations through P multiplication units and an accumulator within the processing period of any processing module; and perform zero-padding calculation outside the processing period of the processing module and within the synchronization period of the first task.

[0082] If the synchronization period of the first task is greater than the processing period of any processing module, then any processing module can complete the subtask of the first task distributed to it within the processing period of any processing module. Then, in order to achieve synchronization, any processing module needs to process to the synchronization period to complete the subtask of the first task. Therefore, any processing module can perform zero-filling calculation between the processing period of any processing module and the synchronization period. Still taking scenario 5 as an example, if the synchronization period of the first task is greater than the processing period of any processing module; any processing module is used to calculate the sum of M' multiplication operations through P multiplication units and an accumulator within the processing period of any processing module; and perform zero-filling calculation outside the processing period of the processing module and within the synchronization period of the first task.

[0083] In a possible implementation, the aggregation module in the distributed processing device is used to aggregate the processing results corresponding to the multiple processing modules to obtain the processing result of the first task when the multiple processing modules have completed the subtasks of the first task to which they are distributed. In this case, any one of the multiple processing modules is also used to send the processing result of each processing cycle in the synchronization cycle of the first task to the aggregation module; and when the synchronization cycle of the first task is reached, send a completion signal to the aggregation module, and the completion signal indicates that any one of the processing modules has completed the processing of the subtasks of the first task to which it is distributed. For example, the aggregation module may refer to Figure 4 For the accumulator shown, the completion signal can be a high level.

[0084] For example, for the above-mentioned zero-jumping operation scenario, taking PEC3-PEC0 distributed computing of 5 Fs as an example, one F corresponds to one task, and the number of cycles corresponding to the subtasks of the 5 Fs processed by PEC3-PEC0 can be as follows: Fig.10 As shown in (2), taking PEC0 as an example, it takes 1 cycle to process F0, 1 cycle to process F1, 4 cycles to process F2, 1 cycle to process F3, 4 cycles to process F4, and 3 cycles to process F5.

[0085] Next, for Fig.10 The processing situation shown in (2) is combined with Fig.11 The synchronization process and processing process are explained with examples. Fig.11 As shown, the synchronization process: for each PEC, write the calculated cycle number into pec_cnt_fifo and pec_comp_fifo. As long as the fifo is not full, the writing operation can be continued until the fifo is full. After the fifo is full, the data can be deleted and rewritten from the beginning. The update process of each queue can refer to the above description of fifo, which will not be repeated here. Processing process: For each PEC, as long as there is data in max_cnt_fifo, the synchronization cycle is obtained, and the data can be taken from the memory to the PE for calculation.

[0086] from Fig.11It can be seen that in the first four rising edges, PEC0 searches upstream for the maximum number of cycles of F0; at the fifth rising edge, the maximum number of cycles of F0 included in PEC3's max_cnt_fifo is 4; at the sixth rising edge, PEC3 starts to calculate F0 through 4 cycles, resulting in a processing delay of 5 clock pulses, and at the same time, the maximum number of cycles of F0 included in PEC2's max_cnt_fifo is 4; at the seventh rising edge, PEC2 starts to calculate F0 through 4 cycles, and at the same time, the maximum number of cycles of F0 included in PEC1's max_cnt_fifo is 4, and so on. In other words, since both the synchronization and calculation processes are pipelined, except for the first F0, which has a delay of several clock pulses to obtain the synchronization cycle, there is data in max_cnt_fifo at other times. Therefore, the calculation will not be blocked during the entire calculation process, so that the synchronization process will not back pressure the calculation pipeline, and clock-level synchronization is achieved.

[0087] Fig.12 A neural network processing unit is provided in an embodiment of the present application, and the neural network processing unit includes any distributed processing device shown in the above embodiments. For example, the network processing unit is an NPU, and the distributed processing device is as follows: Figure 4 shown.

[0088] Fig.13 is a schematic diagram of a chip system provided in an embodiment of the present application, the chip system includes a processor and Fig.12 The neural network processing unit shown. Among them, the processor includes but is not limited to a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an AI (Artificial Intelligence) processor or coprocessor, etc. The GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen, the AI ​​processor is used to process computing operations related to machine learning, and the NPU is used to accelerate neural network calculations. Exemplarily, the chip system can be an AI SoC (System on Chip) chip.

[0089] Fig.14 A schematic diagram of a computer device provided in an embodiment of the present application, wherein the computer device 800 includes Fig.13 The chip system 801 is shown.

[0090] exist Fig.14In the embodiment, the chip system 801 is coupled to the memory 802, and it should be understood that the computer device 800 also supports other memory configurations known in the art. The memory 802 may include one or more computer-readable storage media, which may be non-transitory, and the computer-readable storage medium stores at least one computer program, which is loaded and executed by the chip system 801.

[0091] The memory 802 may also include a high-speed random access memory and a non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, which is used to be executed by the chip system 801.

[0092] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0093] Fig.14 Also shown is a display 806 coupled to the chip system 801 via a display controller 804. In some cases, the computer device 800 can be used for wireless communication. Fig.14 Also shown are a speaker 809 and a microphone 810 coupled to the chip system 801 via a codec 811 ; and a wireless antenna 808 coupled to the wireless controller 805 .

[0094] The display 806 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display 806 is a touch display screen, the display 806 also has the ability to collect touch signals on the surface or above the surface of the display 806. At this time, the display 806 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display 806 can be one, set on the front panel of the terminal; in other embodiments, the display 806 can be at least two, respectively set on different surfaces of the terminal or in a folding design; in other embodiments, the display 806 can be a flexible display screen, set on a curved surface or a folding surface of the terminal. Even, the display 806 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display 806 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0095] Microphone 810 is used to collect sound waves from the user and the environment, and input the sound waves to the chip system 801 for processing. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones 810, which are respectively arranged at different parts of the terminal. Microphone 810 can also be an array microphone or an omnidirectional acquisition microphone. Speaker 809 is used to convert electrical signals from chip system 801 into sound waves. Speaker 809 can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for purposes such as ranging.

[0096] The chip system 801 and the memory 802 may be included in a system-in-package or system-on-chip device.

[0097] Input device 807 and power supply 803 are coupled to system-on-chip device 812. Optionally, as Fig.14 As shown, when one or more optional blocks are present, the display 806, input device 807, speaker 809, microphone 810, wireless antenna 808, and power supply 803 are external to the system-on-chip device 812. However, each of the display 806, input device 807, speaker 809, microphone 810, wireless antenna 808, and power supply 803 may be coupled to a component of the system-on-chip device 812, such as an interface or controller.

[0098] The power supply 803 is used to power various components in the terminal. The power supply 803 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 803 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0099] In one possible implementation, the chip system 801 and the memory 802 may be integrated into a set-top box, a server, a music player, a video player, an entertainment unit, a navigation device, a personal digital assistant (PDA), a fixed location data unit, a computer, a laptop computer, a tablet computer, a communication device, a mobile phone, or other similar devices.

[0100] Those skilled in the art will understand that Fig.14 The structure shown in the figure does not constitute a limitation on the computer device. The computer device may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0101] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0102] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A distributed processing device, It is characterized in that The distributed processing device comprises a plurality of processing modules, the plurality of processing modules are used for distributed collaborative processing of a first task, and the processing cycles required for processing the subtasks of the first task distributed to the plurality of processing modules are different; The multiple processing modules are respectively used to process the subtasks of the first task distributed thereto according to the synchronization period of the first task, and the synchronization period of the first task is determined based on the maximum value of the processing periods respectively corresponding to the multiple processing modules.

2. The device according to claim 1, It is characterized in that The multiple processing modules are sequentially connected in a chain manner; A first processing module located at the end of the plurality of processing modules is used to send a first processing cycle of the first processing module to a second processing module connected upstream; the second processing module is configured to receive the first processing cycle, determine a first intermediate cycle which is larger between the first processing cycle and a second processing cycle of the second processing module, and send the first intermediate cycle to a third processing module connected upstream; The third processing module is configured to receive the first intermediate cycle and determine a second intermediate cycle which is larger between the first intermediate cycle and a third processing cycle of the third processing module; In the case that the third processing module is located at the first position among the multiple processing modules, the third processing module is further configured to use the second intermediate cycle as a synchronization cycle of the first task.

3. The device according to claim 2, It is characterized in that In the case that the third processing module is not located at the first position among the multiple processing modules, the third processing module is further configured to send the second intermediate cycle to a fourth processing module connected upstream; The fourth processing module is used to receive the second intermediate cycle and determine a third intermediate cycle which is larger between the second intermediate cycle and a fourth processing cycle of the fourth processing module; In the case where the fourth processing module is located at the first position among the multiple processing modules, the fourth processing module is further configured to use the third intermediate cycle as a synchronization cycle of the first task.

4. The device according to claim 2, It is characterized in that The third processing module is further used to send the synchronization period of the first task to the second processing module; The second processing module is further configured to receive the synchronization period of the first task and send the synchronization period of the first task to the first processing module.

5. The device according to claim 1, It is characterized in that Any processing module among the multiple processing modules includes a synchronization cycle queue, and the synchronization cycle queue is used to store the synchronization cycle of each task; Any one of the processing modules is further used to obtain the synchronization period of the first task, and write the synchronization period of the first task into the synchronization period queue; Any one of the processing modules is used to process the subtasks of the first task distributed thereto according to the synchronization cycle of the first task when the synchronization cycle of the first task is read from the synchronization cycle queue.

6. The device according to claim 1, It is characterized in that The distributed processing device also includes an aggregation module; the aggregation module is used to aggregate the processing results corresponding to the multiple processing modules respectively when the multiple processing modules have completed the subtasks of the first task to which they are distributed, so as to obtain the processing result of the first task.

7. The device according to claim 6, It is characterized in that Any one of the multiple processing modules is also used to send the processing results of each processing cycle in the synchronization cycle of the first task to the aggregation module; and when the synchronization cycle of the first task is reached, send a completion signal to the aggregation module, and the completion signal indicates that any one of the processing modules has completed processing the subtask of the first task distributed to it.

8. The device according to any one of claims 1 to 7, It is characterized in that The first task is the sum of N multiplication operations, and the subtask to which any processing module is distributed is the sum of M multiplication operations among the N multiplication operations, where M is a positive integer and N is a positive integer greater than M; Any of the processing modules is a computing unit cluster PEC, the PEC includes P multiplication units and an accumulator, the P multiplication units are used for distributed processing of multiplication operations, the accumulator is used to obtain the sum of the operation results of the P multiplication units, and P is a positive integer; The processing cycle of any one of the processing modules is a value obtained by rounding up the quotient of M' and P, where M' is the number of multiplication operations in which the multiplier is not zero among the M multiplication operations.

9. The device according to claim 8, It is characterized in that The synchronization period of the first task is greater than the processing period of any processing module; Any one of the processing modules is used to calculate the sum of the M' multiplication operations through the P multiplication units and an accumulator within a processing cycle of the any one of the processing modules; Zero-padding calculation is performed outside the processing cycle of any processing module and within the synchronization cycle of the first task.

10. A neural network processing unit, It is characterized in that The neural network processing unit includes a distributed processing device as described in any one of claims 1-9.

11. A chip system, It is characterized in that The chip system includes a processor and a neural network processing unit as described in claim 10.

Images