Neural network chip and data scheduling method

Abstract

The invention provides a neural network chip and a data scheduling method. The neural network chip comprises a configurable module, a plurality of computing nodes and an interconnection topology. The interconnection topology comprises a plurality of data paths, and each data path is connected with two computing nodes. And the configurable module controls on or off of a data path connected with the computing nodes for realizing computing of the neural network layer in the plurality of computing nodes according to configuration information corresponding to the neural network layer in the neural network model, so that data scheduling is realized in the neural network chip, and the overall operation efficiency of the neural network chip is improved.

Description

Technical Field

[0001] The present application relates to the field of integrated circuit technology, and in particular to a neural network chip and a data scheduling method. Background Art

[0002] With the rapid development of deep learning, dedicated hardware accelerators (neural network chips) for neural networks have emerged to meet the increasing demand for computing power. In recent years, the specifications of neural network chips have been continuously improved, and the number of computing cores (or computing nodes) included has increased.

[0003] The computing power of each computing node in the neural network chip is limited. For example, the number of input channels and / or output channels of the neural network layer processed at a single time is limited. If the input channel or output channel of a neural network layer is large, the calculation of the neural network layer cannot be achieved by a single processing on a single computing node.

[0004] In related technologies, in order to speed up the calculation, the input channel calculation and the output channel calculation are usually split to multiple computing nodes, and other control devices (such as central processing units) schedule data input to multiple computing nodes. This solution, which requires the use of additional control devices to implement input channel splitting and output channel splitting, consumes more communication resources between the control device and the neural network chip, and results in lower operating efficiency of the neural network chip. Summary of the invention

[0005] The present application provides a neural network chip and a data scheduling method, which can enable the neural network chip to dynamically schedule data on-chip and improve the operating efficiency of the neural network chip.

[0006] In a first aspect, the present application provides a neural network chip, which includes: a first computing node, a second computing node, a third computing node, a fourth computing node, a first data path, a second data path, a third data path, a fourth data path, and a configurable module.

[0007] Among them, the first computing node is used to output the input data of the first neural network layer in the neural network, the second computing node and the third computing node are used to implement the calculation of the first neural network layer, and the fourth computing node aggregates the calculation results of the second computing node and the third computing node; the first data path is used to connect the first computing node and the second computing node, the second data path is used to connect the first computing node and the third computing node, the third data path is used to connect the fourth computing node and the second computing node, and the fourth data path is used to connect the fourth computing node and the third computing node.

[0008] Among them, the configurable module is used to receive configuration information corresponding to the first neural network layer, and the configuration information is used to instruct the input channel splitting or output channel splitting of the first neural network layer; the configurable module is also used to control the opening and closing of the first data path, the second data path, the third data path and the fourth data path according to the configuration information.

[0009] In the above implementation, multiple data paths and configurable modules are set inside the neural network chip. The configurable modules control the opening and closing of the data paths according to pre-stored configuration information, so that the neural network chip can realize data scheduling within the chip when the input channel is split and the output channel is split, thereby reducing the communication resource consumption between the neural network chip and the outside world, and achieving the purpose of improving the operating efficiency of the neural network chip.

[0010] In a possible implementation manner of the first aspect, the configuration information includes an input channel number X of the second computing node and an input channel number Y of the third computing node, and the configurable module is specifically used to: when the first data path and the second data path are both in an off state, control the first data path and the second data path to be opened in sequence according to X and Y, so that data from the 1st channel to the Xth channel of the input data are input to the second computing node, and data from the X+1th channel to the X+Yth channel of the input data are input to the third computing node; after the second computing node and the third computing node complete calculation, control the third data path and the fourth data path to be opened, and the fourth computing node is used to add the calculation result output by the second computing node and the calculation result output by the third computing node.

[0011] In the above implementation, the configurable module pre-stores the number of input channels of the nodes involved in the input channel splitting, and then can control the data paths of the nodes to be opened in sequence according to the number of input channels of the nodes. By controlling the number of channels of the input nodes, the data of multiple input channels of the input data can be split to multiple nodes for calculation.

[0012] In a possible implementation of the first aspect, the configurable module controls the first data path and the second data path to be opened in sequence according to the X and the Y, including: controlling the first data path to be opened; when the number of channels received by the second computing node is equal to the X, controlling the first data path to be closed and controlling the second data path to be opened; when the number of channels received by the third computing node is equal to the Y, controlling the second data path to be closed.

[0013] In a possible implementation manner of the first aspect, the second computing node is further used to perform channel identification detection when receiving data from the 1st channel to the Xth channel of the input data, and to send a first message to the configurable module, wherein the first message is used to indicate that the second computing node has detected the channel identification, and the configurable module is used to determine the number of channels received by the second computing node according to the first message; the third computing node is further used to perform channel identification detection when receiving data from the X+1th channel to the X+Yth channel of the input data, and to send a second message to the configurable module, wherein the second message is used to indicate that the third computing node has detected the channel identification, and the configurable module is used to determine the number of channels received by the third computing node according to the second message.

[0014] In the above implementation, the node performs channel identification detection when receiving data, and sends a message of identification detection to the configurable module, so that the configurable module can know the number of channels received by the node.

[0015] In a possible implementation of the first aspect, the configuration information includes a first identifier of the fourth computing node, the first identifier is used to indicate output channel splitting of the first neural network layer, and the configurable module is specifically used to: control the opening of the first data path and the second data path so that the second computing node and the third computing node both calculate the input data; after determining that the calculation of the second computing node and the third computing node is completed, control the third data path and the fourth data path to be opened in sequence, and the fourth computing node is used to splice the calculation results of the second computing node and the third computing node.

[0016] In the above implementation, the configurable module can realize output channel splitting according to the pre-stored first identifier, and input the calculation results of different output channels output by each node into the next node in a certain order.

[0017] In a possible implementation of the first aspect, the configurable module controls the third data path and the fourth data path to be opened in sequence, including: when the third data path and the fourth data path are both in an off state, controlling the third data path to be opened; when it is determined that the transmission of the calculation result of the second computing node is completed, controlling the third data path to be closed and the fourth data path to be opened; when it is determined that the transmission of the calculation result of the third computing node is completed, controlling the fourth data path to be closed.

[0018] In a possible implementation of the first aspect, the second computing node is further used to send a third message to the configurable module, wherein the third message is used to indicate that the transmission of the calculation result of the second computing node is complete; the second computing node is further used to send a fourth message to the configurable module, wherein the fourth message is used to complete the transmission of the calculation result of the third computing node.

[0019] In the above implementation, the node sends a message indicating that the calculation result transmission is completed to the configurable module, so that the configurable module can know the number of nodes that have completed the transmission.

[0020] In a second aspect, the present application provides a data scheduling method for a neural network chip. The neural network chip includes: a first computing node, a second computing node, a third computing node, a fourth computing node, a first data path, a second data path, a third data path, and a fourth data path, wherein the first computing node is used to output input data of a neural network layer of a neural network model, the second computing node and the third computing node are used to implement the calculation of the neural network layer, the fourth computing node is used to aggregate the calculation results of the second computing node and the calculation results of the third computing node, the first data path is used to connect the first computing node and the second computing node, the second data path is used to connect the first computing node and the third computing node, the third data path is used to connect the fourth computing node and the second computing node, and the fourth data path is used to connect the fourth computing node and the third computing node.

[0021] The method includes: receiving configuration information corresponding to the first neural network layer, the configuration information is used to indicate input channel splitting or output channel splitting of the first neural network layer; controlling the opening and closing of the first data path, the second data path, the third data path and the fourth data path according to the configuration information.

[0022] In a possible implementation manner of the second aspect, the configuration information includes an input channel number X of the second computing node and an input channel number Y of the third computing node, and controlling the opening and closing of the first data path, the second data path, the third data path, and the fourth data path according to the configuration information includes: when the first data path and the second data path are both in a closed state, controlling the first data path and the second data path to be opened in sequence according to X and Y, so that data from the 1st channel to the Xth channel of the input data are input to the second computing node, and data from the X+1th channel to the X+Yth channel of the input data are input to the third computing node; after the second computing node and the third computing node complete calculation, controlling the third data path and the fourth data path to be opened, and the fourth computing node is used to add a calculation result output by the second computing node and a calculation result output by the third computing node.

[0023] In a possible implementation of the second aspect, controlling the first data path and the second data path to be opened in sequence according to the X and the Y includes: controlling the first data path to be opened; when the number of channels received by the second computing node is equal to the X, controlling the first data path to be closed and controlling the second data path to be opened; when the number of channels received by the third computing node is equal to the Y, controlling the second data path to be closed.

[0024] In a possible implementation manner of the second aspect, the second computing node is further used to perform channel identification detection when receiving data from the 1st channel to the Xth channel of the input data, and send a first message to the configurable module, the first message being used to indicate that the second computing node has detected the channel identification, and the method further includes: before controlling the first data path to be shut down and controlling the second data path to be opened, determining the number of channels received by the second computing node according to the first message; the third computing node is further used to perform channel identification detection when receiving data from the X+1th channel to the X+Yth channel of the input data, and send a second message to the configurable module, the second message being used to indicate that the third computing node has detected the channel identification, and the method further includes: before controlling the second data path to be shut down, determining the number of channels received by the third computing node according to the second message.

[0025] In a possible implementation of the second aspect, the configuration information includes a first identifier of the fourth computing node, the first identifier is used to indicate output channel splitting of the first neural network layer, and the controlling the opening and closing of the first data path, the second data path, the third data path and the fourth data path according to the configuration information includes: controlling the opening of the first data path and the second data path, the second computing node is used to calculate the input data, and the third computing node is used to calculate the input data; after determining that the calculation of the second computing node and the third computing node is completed, controlling the third data path and the fourth data path to be opened in sequence, and the fourth computing node is used to splice the calculation results of the second computing node and the calculation results of the third computing node.

[0026] In a possible implementation of the second aspect, the controlling the third data path and the fourth data path to be opened in sequence includes: when the third data path and the fourth data path are both in an off state, controlling the third data path to be opened; when it is determined that the transmission of the calculation result of the second computing node is completed, controlling the third data path to be closed and the fourth data path to be opened; when it is determined that the transmission of the calculation result of the third computing node is completed, controlling the fourth data path to be closed.

[0027] In a possible implementation of the second aspect, the second computing node is further used to send a third message to the configurable module, wherein the third message is used to indicate that the transmission of the calculation result of the second computing node is complete; the second computing node is further used to send a fourth message to the configurable module, wherein the fourth message is used to complete the transmission of the calculation result of the third computing node.

[0028] In a third aspect, the present application provides a computing device, which includes the neural network chip described in the fourth aspect, or the neural network chip provided by the first aspect or any possible implementation of the first aspect.

[0029] In a fourth aspect, the present application provides a computer-readable storage medium, characterized in that it includes instructions, which, when executed on a computer, enable the computer to execute a data scheduling method provided by implementing the second aspect or any possible implementation of the second aspect.

[0030] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the data scheduling method provided by the second aspect or any possible implementation of the second aspect.

[0031] Any of the data scheduling or computer storage media or computer program products provided above are used in the above-mentioned neural network chip. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding schemes in the neural network chip provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structural schematic diagram of a crossbar array of computing nodes in a neural network chip provided in an embodiment of the present application;

[0033] Figure 2 It is a schematic diagram of the structure of a neural network chip provided in an embodiment of the present application;

[0034] Figure 3 It is a schematic diagram of the structure of the interconnection topology in a neural network chip provided in an embodiment of the present application;

[0035] Figure 4 It is a structural schematic diagram of a switching matrix in an interconnection topology provided in an embodiment of the present application;

[0036] Figure 5 It is a structural schematic diagram of a configurable module in a neural network chip provided in an embodiment of the present application;

[0037] Figure 6a This is a method provided by the embodiment of the present application. Figure 4 The structure shown is a schematic diagram of the structure for realizing input channel splitting;

[0038] Figure 6b This is a method provided by the embodiment of the present application. Figure 4 The structure shown is a schematic diagram of the structure for realizing output channel splitting;

[0039] Figure 7a This is a functional schematic diagram of a configurable module provided in an embodiment of the present application for implementing input channel splitting;

[0040] Figure 7b This is a method provided by the embodiment of the present application. Figure 7a Schematic diagram of the comparison module;

[0041] Figure 8a This is a functional schematic diagram of a configurable module provided in an embodiment of the present application for implementing input channel splitting;

[0042] Figure 8b This is a method provided by the embodiment of the present application. Figure 8a Schematic diagram of the comparison module;

[0043] Fig. 9 This is a method provided by the embodiment of the present application. Figure 2 A flow chart of a data scheduling method for a neural network chip shown;

[0044] Fig.10 This embodiment of the present application provides a method comprising Figure 2 Schematic diagram of the structure of the computing device of the neural network chip shown. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0047] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple screen terminals refers to two or more screen terminals.

[0048] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0049] Based on the background technology, a neural network chip includes a large number of processing elements (PEs). Taking the widely used integrated storage and computing neural network chip as an example, each PE may include one or more processing elements such as Figure 1 The cross array shown. Each PE pre-stores the weights corresponding to each neural network layer in the neural network, thereby realizing the calculation of the corresponding neural network layer. Each cross array includes multiple rows and columns of sub-units and a decoding circuit, a driving circuit and a readout circuit connected to the storage and calculation unit. The decoding circuit is used to convert data into a format recognizable by the sub-unit. The driving circuit is used to input the format-converted data into each sub-unit. The readout circuit is used to output the calculation results of each sub-unit.

[0050] The structure of PE limits the computing power of PE. In other words, the number of input channels and output channels of the neural network layer processed by each PE at a time will be limited by the input channels and output channels of the PE itself. Therefore, in some implementations, when the neural network chip implements the calculation of a neural network layer in the neural network, if the input channels or output channels of the neural network layer are too many and cannot be processed once on a single array, the neural network layer needs to be split into input channels and output channels through the scheduler, that is, the calculation of the neural network layer is split to multiple PEs for implementation.

[0051] For example, when the number of channels of the input data of the neural network layer is greater than the number of input channels of a PE in the neural network chip, the scheduler can split the input data to multiple PEs for calculation. Each PE calculates the data of different input channels respectively, and the calculation results output by each PE are the calculation results of the same output channel. After obtaining the calculation results of each PE, the scheduler adds the calculation results of each PE to obtain the output data corresponding to the input data.

[0052] For example, when the number of channels of the output data of the neural network layer is greater than the number of output channels of a PE in the neural network chip, the input data of the neural network layer can be input to multiple PEs through the scheduler. Each PE calculates different output channels for the input data and then outputs the calculation results of different output channels. After obtaining the calculation results of each PE, the scheduler splices the calculation results of each PE to obtain the output data corresponding to the input data.

[0053] The scheduler may generally include a central processing unit (CPU) or a field programmable gate array (FPGA) chip.

[0054] There are some problems with the above implementation scheme that uses a scheduler to implement input channel splitting and output channel splitting. On the one hand, the scheduler needs to interact and calculate (add or splice) with the neural network chip multiple times, which increases the communication overhead and computing overhead of the scheduler. On the other hand, the input channel splitting and output channel splitting are implemented through the cooperation of the scheduler and the neural network chip, which does not fully utilize the computing power of the neural network chip and has low computing efficiency. On the other hand, the scheduler needs to pre-configure the corresponding program code, and the programming process of the program code is relatively complicated, which limits the promotion and large-scale application of neural network chips.

[0055] To this end, an embodiment of the present application provides a neural network chip that can solve the above problems.

[0056] Figure 2 Schematic diagram of the structure of a neural network chip provided in the embodiment of the present application. Figure 2 As shown, the neural network chip includes multiple PEs (PE0 to PEN), interconnection topology and configurable modules. Each PE may include one or more Figure 1 The crossbar array shown. Each PE is connected through an interconnection topology, and the interconnection topology may include a data path between any two PEs. The configurable module is used to control the opening or closing of the data path between each PE during the neural network chip to implement the neural network calculation process, thereby realizing the input channel splitting and / or output channel splitting of the neural network layer in the neural network.

[0057] The following is a detailed introduction to the interconnection topology and configurable modules.

[0058] Figure 3 Schematic diagram of an interconnection topology provided in an embodiment of the present application. Figure 3 As shown, the interconnection topology may include multiple switch matrices (switch matrix 0 to switch matrix 5). Switch matrix 0 connects switch matrix 1 to switch matrix 5. Each switch matrix in switch matrix 1 to switch matrix 5 connects multiple PEs in PE0 to PEN.

[0059] The switching matrix may include one or more switching nodes and multiple switches, each switch is connected to a PE, and each switch is connected through a switching node. The data path between any two PEs connected by each switching matrix includes a switch connected to the two PEs. The number of switching nodes and switches can be designed according to the number of actual PEs and the number of interfaces of the switching nodes, and this application does not make specific restrictions.

[0060] Take switching matrix 1 as an example. Figure 4 As shown, the switch matrix 1 can connect PE0 to PE5. The switch matrix 1 may include: Figure 4 The switching node 1 and multiple switches (switch 0 to switch 5) connected to the switching node 1 are shown, and switches 0 to switch 5 are connected to PE0 to PE5 respectively. The data path between PE0 and PE1 includes switch 0 and switch 1, and the data path between PE0 and PE3 includes switch 0 and switch 3. The data path between any two PEs is similar, and will not be described here. The data path between any two PEs connected by the switching matrix 1 can be opened or closed under the control of the configurable module.

[0061] It should be noted that Figure 3 and Figure 4 The structure shown is merely an example of the present application and does not represent a limitation of the present application on neural network chips.

[0062] Figure 5Schematic diagram of a configurable module provided in an embodiment of the present application. Figure 5 As shown, the configurable module may include a storage module 501 and a calculation module 502 .

[0063] Among them, the storage module 501 is used to store the routing information and configuration information of the neural network chip. The routing information is used to indicate the connection relationship of the PEs used to implement the calculation of each neural network layer in the neural network among the N PEs. The configuration information includes input channel splitting information or output channel splitting information. The input channel splitting information is used to indicate the input channel splitting of the neural network layer. The output channel splitting information is used to indicate the output channel splitting of the neural network layer.

[0064] The operation module 502 is used to control the opening or closing of the data path between each PE according to the connection relationship indicated by the routing information and the configuration information.

[0065] In the case of input channel splitting and output channel splitting, the input data will be split into multiple data or the output data will be split into multiple data. When participating in the next calculation, each data has a fixed weight. In order to obtain the correct calculation result, it is necessary to ensure that the multiple data after splitting match the weight. Since the weight corresponding to each data is pre-stored in the PE participating in the next calculation, in the case of input channel splitting and output channel splitting, it is necessary to ensure the order of multiple data input into PE to ensure the correct calculation result. Figure 4 For example, PE0 can be used to output the input data of the neural network layer of the neural network, PE1~PE4 can be used to implement the calculation of the neural network layer, and PE5 is used to aggregate (add or splice) the calculation results output by PE1~PE4. When the input channel of the neural network layer is split, PE1~PE4 needs to calculate the data of different input channels respectively, and the order of the data of the channels input to each PE needs to be guaranteed. When the output channel of the neural network layer is split, PE1~PE4 needs to calculate the data of different output channels respectively, and the order of the output of each PE and the input of PE5 needs to be guaranteed.

[0066] The configurable module can control the opening and closing of the data path of the PE connection according to the pre-stored routing information and configuration information, thereby controlling the input sequence of data in the case of input channel splitting and output channel splitting.

[0067] It should be noted that the input channel splitting and output channel splitting described in the embodiments of the present application are not limited to the case where the number of input channels or output channels of a PE cannot meet the number of input channels or output channels of the neural network layer. When the number of input channels or output channels of a PE meets the number of input channels or output channels of the neural network layer, the input channels or output channels of the neural network layer can also be split into multiple PEs for calculation through the above scheme, thereby improving the calculation efficiency.

[0068] For example, since the computing power of a single PE is limited by the parallelism of the readout circuit, in order to speed up the calculation, the calculation of different output channels of the same layer network can be split to multiple PEs for calculation. In this case, each PE uses the weights corresponding to different output channels to calculate the calculation results of different output channels.

[0069] The following describes in detail the process of the operation module 502 controlling the data path of the PE connection in the case of input channel splitting and output channel splitting.

[0070] Input channel split

[0071] When the number of input channels of input data of a neural network layer of a neural network is large, the input channels of the neural network layer can be split into multiple PEs for calculation.

[0072] by Figure 4 For example, PE0 can be used to output the input data of the neural network layer of the neural network, PE1~PE4 can be used to calculate the data of different input channels of the input data, and PE5 is used to add the calculation results output by PE1~PE4. In other words, the data of multiple input channels of the input data of the neural network layer output by PE0 are split and input to PE1~PE4, and the data input to PE1~PE4 is the data of different input channels output by PE0. PE1~PE4 calculates the data of their respective inputs and outputs different calculation results of the same output channel. PE5 adds the different calculation results of the same output channel to obtain the output data of the neural network layer.

[0073] In Figure 4 In the case where the structure shown implements input channel splitting, the routing information may include source node identifiers of PE1-PE5, and the input channel splitting information may specifically include the number of input channels of PE1-PE4.

[0074] Specifically, the source node identifiers of PE1 to PE4 are all PE0, and the source node identifier of PE5 is PE1 to PE4. In this way, the operation module 502 can control the data path between PE0 and PE1 to PE4 to be opened in sequence according to the connection relationship indicated by the routing information and the number of input channels of PE1 to PE4, so that the data of each input channel output by PE0 is respectively input to PE1 to PE4 for calculation. After PE1 to PE4 completes the calculation, the data path between PE1 to PE4 and PE5 is controlled to be opened, so that PE5 adds the calculation results of PE1 to PE4.

[0075] Taking the input data of the neural network layer (the output of PE0) including 10 input channels, and the number of input channels of PE1 to PE4 being 2, 3, 3, and 2 respectively as an example, the control process of the operation module 502 is introduced.

[0076] First, when the data paths between PE0 and PE1 to PE4 are all shut down, Figure 6a As shown, the operation module 502 can control the data paths between PE0 and PE1 to PE4 to be opened in sequence in the order of PE1 to PE4. Among them, PE1 to PE4 use the same weight to calculate the data input by each of them, and the weight is pre-configured in each PE. It should be noted that the embodiment of the present application does not impose specific restrictions on the opening time and opening order of the data path between PE5 and PE1 to PE4, as long as the calculation results of PE1 to PE4 can be input into PE5.

[0077] The operation module 502 controls the data paths between PE0 and PE1 to PE4 to be opened in sequence in the order of PE1 to PE4, which may specifically include the following steps S101 to S105.

[0078] S101. Control the data path between PE0 and PE1 to be opened, so that the data of the first and second input channels in the input data are input to PE1.

[0079] S102. After determining that PE1 receives the data of the first and second input channels in the input data, control the data path between PE0 and PE1 to be closed and control the data path between PE0 and PE2 to be opened, so that the data of the third to fifth input channels in the input data are input to PE2.

[0080] S103. After determining that PE2 receives the data of the 3rd to 5th input channels in the input data, control the data path between PE0 and PE2 to be closed and control the data path between PE0 and PE3 to be opened, so that the data of the 6th to 8th input channels in the input data are input to PE3.

[0081] S104. After determining that PE3 receives data from the 6th to 8th input channels in the input data, control the data path between PE0 and PE3 to be closed and control the data path between PE0 and PE4 to be opened, so that the data from the 9th and 10th input channels in the input data are input to PE4.

[0082] S105. After determining that PE4 receives the data of the 9th and 10th input channels in the input data, control the data path between PE0 and PE4 to be shut down.

[0083] Secondly, the operation module 502 can control the Figure 6b The data path between PE5 and PE1-PE4 is shown to be open, so that the calculation results of PE1-PE4 can be input into PE5. PE5 adds the calculation results of PE1-PE4 to obtain the output data of the neural network layer.

[0084] Output channel split

[0085] When the number of output channels of output data of a neural network layer of a neural network is large, the output channels of the neural network layer can be split to multiple PEs for calculation.

[0086] by Figure 4 For example, PE0 can be used to output the input data of the neural network layer of the neural network, PE1~PE4 can be used to perform calculations on different output channels of the input data, and PE5 is used to splice the calculation results output by PE1~PE4. In other words, the input data of the neural network layer output by PE0 is multicast to PE1~PE4, and the data input to PE1~PE4 is the same data (i.e., the data of multiple input channels of the input data). PE1~PE4 calculates different output channels of the input data and outputs the calculation results of different output channels. PE5 splices the calculation results of different output channels to obtain the output data of the neural network layer.

[0087] In Figure 4 In the case where the structure shown implements output channel splitting, the routing information may include source node identifiers of PE1 to PE5, and the output channel splitting information may specifically include the first identifier of PE5, which is used to indicate that the output channel of the neural network layer is split.

[0088] Specifically, the source node identifiers of PE1 to PE4 are all PE0, and the source node identifier of PE5 is PE1 to PE4. In this way, the operation module 502 can control the opening of the data path between PE0 and PE1 to PE4 according to the connection relationship indicated by the routing information and the first identifier of PE5, so that the output of PE0 is multicast to PE1 to PE4 for calculation of different output channels. After PE1 to PE4 completes the calculation, the data path between PE1 to PE4 and PE5 is controlled to be opened in sequence, and the calculation results of PE1 to PE4 are sequentially input into PE5 in the order of the output channels for splicing.

[0089] Taking the output data of the neural network layer including 8 output channels and the number of output channels of PE1 to PE4 being 2 as an example, the control process of the operation module 502 is introduced.

[0090] First, when the data paths between PE0 and PE1-PE4 are all turned off, the operation module 502 can first control Figure 6b The data paths between PE0 and PE1 to PE4 are opened at the same time, and PE1 to PE4 receive the data of each input channel output by PE0. Among them, PE1 to PE4 use the weights of different output channels to calculate the data output by PE0, and the weights are pre-configured in each PE. For example, PE1 uses the weights corresponding to the 1st and 2nd output channels of the neural network layer to calculate the output of PE0 and obtain the calculation results of the 1st and 2nd output channels; PE2 uses the weights corresponding to the 3rd and 4th output channels of the neural network layer to calculate the output of PE0 and obtain the calculation results of the 3rd and 4th output channels; PE3 uses the weights corresponding to the 5th and 6th output channels of the neural network layer to calculate the output of PE0 and obtain the calculation results of the 5th and 6th output channels; PE4 uses the weights corresponding to the 7th and 8th output channels of the neural network layer to calculate the output of PE0 and obtain the calculation results of the 7th and 8th output channels.

[0091] Secondly, the operation module 502 can determine that PE1-PE4 has completed the calculation of the input data respectively, such as Figure 6b As shown, the data paths between PE5 and PE1 to PE4 are opened in sequence in the order of PE1 to PE4, so that the calculation results of PE1 to PE4 are input into PE5 in sequence. PE5 concatenates the calculation results of PE1 to PE4 in the order of output channels to obtain the output data of the neural network layer.

[0092] The operation module 502 controls the data paths between PE5 and PE1 to PE4 to be opened in sequence in the order of PE1 to PE4, which may specifically include the following steps S201 to S204.

[0093] S201. Control the data path between PE1 and PE5 to be opened, so that the calculation results of the first and second output channels obtained by PE1 are input into PE5.

[0094] S202. After determining that the calculation result of PE1 has been transmitted, control the data path between PE2 and PE5 to be opened, so that the calculation results of the third and fourth output channels obtained by PE2 are input to PE5.

[0095] S203. After determining that the transmission of the calculation result of PE2 is completed, control the data path between PE3 and PE5 to be opened, so that the calculation results of the 5th and 6th output channels obtained by PE3 are input to PE5.

[0096] S204. After determining that the transmission of the calculation result of PE3 is completed, control the data path between PE4 and PE5 to be opened, so that the calculation results of the 7th and 8th output channels obtained by PE4 are input to PE5.

[0097] In this embodiment, the operation module 502 may include: Figure 5 The counting module 5021, the modulus module 5022 and the comparison module 5023 are shown. The operation module 502 may also include a control module ( Figure 5 The control module is used to control the counting module 5021, the modulus module 5022 and the comparison module 5023 to update parameters. The specific functions of each module in the operation module 502 in the case of input channel splitting and output channel splitting are introduced below.

[0098] First, Figure 4 The structure shown implements input channel splitting and introduces the control module, counting module 5021 and comparison module 5023.

[0099] like Figure 7a As shown, the counting module 5021 is used to determine the number of channels received by the PE (including PE1-PE4).

[0100] Among them, the control module can be used to control the counting module 5021 to initialize and update the number of channels received by the PE. Each PE in PE1~PE4 can detect the channel identifier when receiving the data output by PE0, and send a message to the control module indicating that the PE has detected the channel identifier. For example, PE1 can send message 1 to the control module to notify the control module that PE1 has detected the channel identifier; PE2 can send message 2 to the control module to notify the control module that PE2 has detected the channel identifier. The control module can increase the number of channels received by the PE by 1 each time it receives a message sent by the PE indicating that the PE has detected the channel identifier.

[0101] like Figure 7aAs shown, the comparison module 5023 is used to output the control signal corresponding to each PE by comparing the number of channels received by each PE and the comparison value. The control signal corresponding to the PE includes an on signal or a off signal, which is used to control the opening or closing of the data path between the PE and PE0. Specifically, the comparison module 5023 may include one or more comparators compare, for example, it may include compare1~4, corresponding to PE1~PE4 respectively. Figure 7b As shown, when compare1 determines that the number of channels received by PE1 is 0 or 1, the number of channels received by PE1 is less than the number of input channels of PE1 (2), so the comparison module 5023 outputs an on signal; when compare1 determines that the number of channels received by PE1 is 2, the number of channels received by PE1 is equal to the number of input channels of PE1, so compare1 outputs a shut-down signal.

[0102] The control module may also be used to set the comparison value of the comparison module 5023 according to the number of input channels of the PE stored in the storage module 501. For example, when the comparison module 5023 includes one comparator, the control module may set the comparison value of the comparator to be the number of input channels of PE1; when the number of channels received by PE1 is equal to the number of input channels of PE1, the control module may set the comparison value of the comparator to be the number of input channels of PE2. For another example, when the comparison module 5023 includes multiple comparators, the control module may set the comparison values ​​of the multiple comparators to be the number of input channels of PE1-PE4 respectively.

[0103] Secondly, Figure 4 The structure shown implements output channel splitting and introduces the control module, counting module 5021, modulus module 5022 and comparison module 5023.

[0104] like Figure 8a As shown, the counting module 5021 is used to determine the number E of PEs that have completed the transmission. Similarly, the control module can be used to control the counting module 5021 to initialize and update the number E. After each PE in PE1 to PE4 completes the transmission of the calculation result, it can send a message indicating that the transmission of the calculation result is completed to the control module. For example, PE1 can send message 3 to the control module to notify the control module that the transmission of the calculation result of PE1 is completed; PE2 can send message 4 to the control module to notify the control module that the transmission of the calculation result of PE2 is completed. The control module can instruct the counting module 5021 to add 1 to the number E each time it receives a message sent by a PE indicating that the transmission of the calculation result is completed.

[0105] like Figure 8aAs shown, the modulo module 5022 is used to divide the quantity E by the total quantity F of PEs to obtain a remainder G. The control module may set the divisor (total quantity F) of the modulo module 5022 according to the total quantity of PEs that implement the neural network layer calculation. For example, when PE1-PE4 implement the neural network layer calculation, F is 4.

[0106] like Figure 8a As shown, the comparison module 5023 is used to output the control signal of each PE by comparing the remainder G with the comparison value of each PE. For example, the comparison module 5023 includes multiple compares, such as Figure 8b As shown, compare1 is used to compare the remainder G with the comparison value of PE1, and output the control signal of PE1; compare2 is used to compare the remainder G with the comparison value of PE2, and output the control signal of PE2; compare3 is used to compare the remainder G with the comparison value of PE3, and output the control signal of PE3; compare4 is used to compare the remainder G with the comparison value of PE4, and output the control signal of PE4.

[0107] The control module is used to determine the comparison value of each PE according to the PE calculation order and the total number F. For example, the comparison value of PE can be F-PE calculation order-1. Figure 4 Taking PE1-PE4 shown in the figure as an example, F is 4, and the calculation order of PE1-PE4 is 1, 2, 3, 4 respectively, then the comparison value of PE1 is 0, the comparison value of PE2 is 1, the comparison value of PE3 is 2, and the comparison value of PE4 is 3.

[0108] The above Figure 1 In the scheme shown, multiple data paths and configurable modules are set inside the neural network chip. The configurable modules are used to control the opening and closing of the data paths, so that the neural network chip can realize on-chip scheduling of data during input channel splitting and output channel splitting, reduce the communication interaction between the neural network chip and the outside world, and thus improve the operation efficiency of the neural network chip.

[0109] Based on the above-mentioned neural network chip and the functions of the configurable modules therein, an embodiment of the present application also provides a scheduling method for a neural network chip.

[0110] Fig. 9 This is a flow chart of a scheduling method for a neural network chip provided in an embodiment of the present application. The method can be executed by a configurable module in the neural network chip, and the method can include S901 and S902.

[0111] In S901, configuration information corresponding to the neural network layer is received, and the configuration information is used to indicate input channel splitting or output channel splitting of the neural network layer. The configurable module may receive the configuration information from other devices. For example, the configurable module may receive the configuration information from a CPU or FPGA chip of a device where the neural network chip is located.

[0112] In S902, the first data path, the second data path, the third data path and the fourth data path are controlled to be turned on and off according to the configuration information. Figures 6a to 8b The introduction of will not be repeated here.

[0113] Fig.10 It is a schematic diagram of the hardware structure of a computing device 1000 provided in an embodiment of the present application.

[0114] 8 , the computing device 1000 may include a neural network chip 1001, a memory 1002, a communication interface 1003, and a bus 1004. The neural network chip 1001, the memory 1002, and the communication interface 1003 are connected to each other via the bus 1004. The neural network chip 1001, the memory 1002, and the communication interface 1003 may also be connected in other connection modes other than the bus 1004. In addition, the computing device 1000 may also include a processor ( Fig.10 (not shown), the processor may include a central processing unit CPU or a FPGA chip.

[0115] Among them, the memory 1002 can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, optical storage, hard disk, etc.

[0116] Among them, the neural network chip 1001 can Figure 2The structure shown. The processor may be a general-purpose processor, which may be a processor that performs specific steps and / or operations by reading and executing the contents stored in a memory (e.g., memory 1002). For example, the general-purpose processor may be a central processing unit (CPU). The neural network chip 1001 may include at least one circuit to perform the functions of the various components (including PE, data path, and configurable module) in the aforementioned neural network chip.

[0117] The communication interface 1003 includes input / output (I / O) interfaces, physical interfaces, and logical interfaces for interconnecting devices within the computing device 1000, and interfaces for interconnecting the computing device 1000 with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber optic interface, an ATM interface, etc.

[0118] The bus 1004 may be any type of communication bus for interconnecting the neural network chip 1001, the memory 1002 and the communication interface 1003, such as a system bus.

[0119] The above devices may be arranged on independent chips, or at least partially or completely on the same chip. Whether to arrange each device independently on different chips or to integrate them on one or more chips often depends on the needs of product design. The embodiments of the present application do not limit the specific implementation form of the above devices.

[0120] Fig.10 The computing device 1000 shown is merely exemplary. During implementation, the computing device 1000 may also include other components, which are not listed one by one in this article.

[0121] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0122] It is understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. It should be understood that in the embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0123] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present application. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application should be included in the scope of protection of the present application.

Claims

1. A neural network chip, characterized in that: The neural network chip comprises: a first computing node, a second computing node, a third computing node, a fourth computing node, a first data path, a second data path, a third data path, a fourth data path and a configurable module; The first computing node is used to output input data of a first neural network layer in the neural network, the second computing node and the third computing node are used to implement the calculation of the first neural network layer, and the fourth computing node aggregates the calculation results of the second computing node and the calculation results of the third computing node; The first data path is used to connect the first computing node and the second computing node, the second data path is used to connect the first computing node and the third computing node, the third data path is used to connect the fourth computing node and the second computing node, and the fourth data path is used to connect the fourth computing node and the third computing node; The configurable module is used to receive configuration information corresponding to the first neural network layer, where the configuration information is used to instruct to perform input channel splitting or output channel splitting on the first neural network layer; The configurable module is further used to control the opening and closing of the first data path, the second data path, the third data path and the fourth data path according to the configuration information.

2. The neural network chip according to claim 1, characterized in that: The configuration information includes the number X of input channels of the second computing node and the number Y of input channels of the third computing node, and the configurable module is specifically used for: When both the first data path and the second data path are in the off state, the first data path and the second data path are controlled to be opened in sequence according to X and Y, so that data from the 1st channel to the Xth channel of the input data are input to the second computing node, and data from the X+1th channel to the X+Yth channel of the input data are input to the third computing node; After the second computing node and the third computing node complete the calculation, the third data path and the fourth data path are controlled to be opened, and the fourth computing node is used to add the calculation result output by the second computing node and the calculation result output by the third computing node.

3. The neural network chip according to claim 2, characterized in that: The configurable module controls the first data path and the second data path to be opened in sequence according to the X and the Y, comprising: Controlling the opening of the first data path; When the number of channels received by the second computing node is equal to X, controlling the first data path to be closed and controlling the second data path to be opened; When the number of channels received by the third computing node is equal to Y, the second data path is controlled to be shut down.

4. The neural network chip according to claim 3, characterized in that: The second computing node is further used to perform channel identification detection when receiving data from the 1st channel to the Xth channel of the input data, and send a first message to the configurable module, wherein the first message is used to indicate that the second computing node has detected the channel identification, and the configurable module is used to determine the number of channels received by the second computing node according to the first message; The third computing node is further used to perform channel identifier detection in data from the X+1th channel to the X+Yth channel of the received input data, and to send a second message to the configurable module, wherein the second message is used to indicate that the third computing node has detected the channel identifier, and the configurable module is used to determine the number of channels received by the third computing node according to the second message.

5. The neural network chip according to claim 1, characterized in that: The configuration information includes a first identifier of the fourth computing node, the first identifier is used to indicate that the output channel of the first neural network layer is split, and the configurable module is specifically used to: Controlling the first data path and the second data path to be opened, so that the second computing node and the third computing node both perform calculations on the input data; After determining that the second computing node and the third computing node have completed calculations, the third data path and the fourth data path are controlled to be opened in sequence, and the fourth computing node is used to splice the calculation results of the second computing node and the calculation results of the third computing node.

6. The neural network chip according to claim 5, characterized in that: The configurable module controls the third data path and the fourth data path to be opened in sequence, including: When the third data path and the fourth data path are both in the off state, controlling the third data path to be turned on; In the case of determining that the transmission of the calculation result of the second computing node is completed, controlling the third data path to be turned off and the fourth data path to be turned on; In the case where it is determined that the transmission of the calculation result of the third calculation node is completed, the fourth data path is controlled to be shut down.

7. The neural network chip according to claim 6, characterized in that: The second computing node is further used to send a third message to the configurable module, where the third message is used to indicate that the transmission of the computing result of the second computing node is completed; The second computing node is further used to send a fourth message to the configurable module, where the fourth message is used to complete the transmission of the computing result of the third computing node.

8. A data scheduling method for a neural network chip, characterized in that: The neural network chip includes: a first computing node, a second computing node, a third computing node, a fourth computing node, a first data path, a second data path, a third data path, and a fourth data path, wherein the first computing node is used to output input data of a first neural network layer of a neural network model, the second computing node and the third computing node are used to implement the calculation of the first neural network layer, the fourth computing node is used to aggregate the calculation result of the second computing node and the calculation result of the third computing node, the first data path is used to connect the first computing node and the second computing node, the second data path is used to connect the first computing node and the third computing node, the third data path is used to connect the fourth computing node and the second computing node, and the fourth data path is used to connect the fourth computing node and the third computing node, and the method includes: Receive configuration information corresponding to the first neural network layer, where the configuration information is used to instruct to perform input channel splitting or output channel splitting on the first neural network layer; The first data path, the second data path, the third data path and the fourth data path are controlled to be turned on and off according to the configuration information.

9. The method according to claim 8, characterized in that The configuration information includes the number X of input channels of the second computing node and the number Y of input channels of the third computing node, and the controlling the opening and closing of the first data path, the second data path, the third data path, and the fourth data path according to the configuration information includes: When both the first data path and the second data path are in the off state, the first data path and the second data path are controlled to be opened in sequence according to X and Y, so that data from the 1st channel to the Xth channel of the input data are input to the second computing node, and data from the X+ith channel to the X+Yth channel of the input data are input to the third computing node; After the second computing node and the third computing node complete the calculation, the third data path and the fourth data path are controlled to be opened, and the fourth computing node is used to add the calculation result output by the second computing node and the calculation result output by the third computing node.

10. The method according to claim 9, characterized in that The controlling the first data path and the second data path to be opened in sequence according to the X and the Y comprises: Controlling the opening of the first data path; When the number of channels received by the second computing node is equal to X, controlling the first data path to be closed and controlling the second data path to be opened; When the number of channels received by the third computing node is equal to Y, the second data path is controlled to be shut down.

11. The method according to claim 10, characterized in that The second computing node is further used to perform channel identification detection in the case of receiving data from the 1st channel to the Xth channel of the input data, and send a first message to the configurable module, wherein the first message is used to indicate that the second computing node has detected the channel identification. The method further includes: before controlling the first data path to be turned off and controlling the second data path to be turned on, determining the number of channels received by the second computing node according to the first message; The third computing node is further used to perform channel identification detection in the data from the X+1th channel to the X+Yth channel of the received input data, and to send a second message to the configurable module, wherein the second message is used to indicate that the third computing node has detected the channel identification. The method further includes: before controlling the second data path to be shut down, determining the number of channels received by the third computing node according to the second message.

12. The method according to claim 8, characterized in that The configuration information includes a first identifier of the fourth computing node, the first identifier is used to indicate that the output channel of the first neural network layer is split, and the controlling the opening and closing of the first data path, the second data path, the third data path, and the fourth data path according to the configuration information includes: Controlling the first data path and the second data path to be opened, the second computing node is used to calculate the input data, and the third computing node is used to calculate the input data; After determining that the second computing node and the third computing node have completed calculations, the third data path and the fourth data path are controlled to be opened in sequence, and the fourth computing node is used to splice the calculation results of the second computing node and the calculation results of the third computing node.

13. The method according to claim 12, characterized in that The controlling the third data path and the fourth data path to be opened in sequence comprises: When the third data path and the fourth data path are both in the off state, controlling the third data path to be turned on; In the case of determining that the transmission of the calculation result of the second computing node is completed, controlling the third data path to be turned off and the fourth data path to be turned on; In the case where it is determined that the transmission of the calculation result of the third calculation node is completed, the fourth data path is controlled to be shut down.

14. The method according to claim 13, characterized in that The second computing node is further used to send a third message to the configurable module, where the third message is used to indicate that the transmission of the computing result of the second computing node is completed; The second computing node is further used to send a fourth message to the configurable module, where the fourth message is used to complete the transmission of the computing result of the third computing node.

15. A computing device, characterized in that: The computing device includes the neural network chip described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 8 to 14.

Images