Neural network structure division method, device, computer equipment and storage medium

Abstract

The embodiment of the present invention discloses a method for structural division of a neural network, wherein the neural network includes multiple layers, and the method includes: dividing each layer in the neural network into multiple nodes, and obtaining the data output amount and data calculation amount corresponding to each node; according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip, each node is allocated to at least one chip. The technical solution of the embodiment of the present invention provides a new neural network slicing method that takes into account both the data calculation amount of a single chip and the data transmission amount between chips. It can minimize the data transmission amount between chips while ensuring that the data calculation amount of each chip is as balanced as possible, and improve the collaborative computing performance of each chip after slicing while completely avoiding the investment of manpower costs.

Description

Technical Field

[0001] The embodiments of the present invention relate to computer technology, specifically to the fields of neural network and artificial intelligence technology, and in particular to a method, device, computer equipment and storage medium for structural division of a neural network. Background Art

[0002] At present, in order to improve the computing speed of neural networks, neural networks can be loaded into physical chips, and the computing cores on the physical chips can realize the computing functions of each layer of the neural network.

[0003] In fact, each chip has a storage limit and a computing power limit. Therefore, it is often impossible to concentrate the calculations of all layers of a neural network on one chip, which requires neural network slicing. Slicing means dividing the layers of the neural network onto one or more chips, and then realizing the computational processing of the entire neural network through the collaborative work of the above chips.

[0004] In the prior art, neural network slicing is mainly achieved manually. The above slicing method has high labor cost and low efficiency, and cannot guarantee that each chip has good computing performance when working together. Summary of the invention

[0005] Embodiments of the present invention provide a method, apparatus, computer equipment and storage medium for dividing the structure of a neural network, so as to provide a new neural network slicing method that takes into account both the amount of data calculation on a single chip and the amount of data transmission between chips.

[0006] In a first aspect, an embodiment of the present invention provides a method for structural division of a neural network, wherein the neural network includes multiple layers, and the method includes:

[0007] Dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and the data calculation amount corresponding to each of the nodes;

[0008] According to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip, each of the nodes is allocated to at least one chip.

[0009] In a second aspect, an embodiment of the present invention further provides a structure division device for a neural network, wherein the neural network includes multiple layers, and the device includes:

[0010] A node division module, used to divide each layer in the neural network into multiple nodes, and obtain the data output amount and data calculation amount corresponding to each of the nodes;

[0011] The node allocation module is used to allocate each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip.

[0012] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0013] one or more processors;

[0014] a storage device for storing one or more programs,

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the structure division method of the neural network as described in any embodiment of the present invention.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for structural division of a neural network as described in any embodiment of the present invention.

[0017] The technical solution of the embodiment of the present invention, when slicing a neural network, divides each layer of the neural network into nodes, and considers the data output amount corresponding to each node, the data calculation amount of each node and the standard calculation amount of the chip at the same time, and allocates each node to at least one chip. This provides a new neural network slicing method that takes into account both the data calculation amount of a single chip and the data transmission amount between chips. It can minimize the data transmission amount between chips while ensuring that the data calculation amount of each chip is as balanced as possible, reduce the data transmission amount between chips, shorten the transmission time, and evenly distribute the load of multiple chips after slicing, thereby maintaining a balanced computing intensity for each chip, and completely avoiding the investment in manpower costs while improving the collaborative computing performance of each chip after slicing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of implementing a method for dividing the structure of a neural network in the first embodiment of the present invention;

[0019] Figure 2a It is a flow chart of the implementation of a method for dividing the structure of a neural network in the second embodiment of the present invention;

[0020] Figure 2b is a schematic diagram of the structure of a neural network applicable to an embodiment of the present invention;

[0021] Figure 3a It is a flow chart of a method for dividing the structure of a neural network in Embodiment 3 of the present invention;

[0022] Figure 3bIt is a structural schematic diagram of a slicing process of each node applicable to an embodiment of the present invention;

[0023] Figure 4 is a structural diagram of a neural network structure division device in Embodiment 4 of the present invention;

[0024] Figure 5a is a structural diagram of a computer device in Embodiment 5 of the present invention;

[0025] Figure 5b It is an internal structure diagram of a processor applicable to the fifth embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0027] It should also be noted that, for ease of description, only the part relevant to the present invention but not all content is shown in the accompanying drawings. It should be mentioned before discussing exemplary embodiments in more detail that some exemplary embodiments are described as processing or methods depicted as flow charts. Although the flow chart describes each operation (or step) as sequential processing, many operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. When its operation is completed, the processing can be terminated, but it can also have additional steps not included in the accompanying drawings. The processing can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0028] Embodiment 1

[0029] Figure 1 This is a flowchart of a method for dividing the structure of a neural network provided in Embodiment 1 of the present invention. This embodiment is applicable to the case where multiple layers of a neural network are divided into at least one chip for operation. The method of the embodiment of the present invention can be executed by a neural network structure division device, which can be implemented by software and / or hardware, and can generally be integrated in a computer device with data processing function. The method specifically includes the following steps:

[0030] S110, dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and the data calculation amount corresponding to each of the nodes.

[0031] The neural network includes multiple layers, or multi-layer neural networks, and each layer of the neural network can implement the set data processing operation. Accordingly, each layer of the neural network can be assigned to one or more computing cores in the chip for calculation. Generally speaking, when the adjacent subsequent layer of the neural network receives the data calculated by the previous layer of the neural network, it can perform corresponding calculations based on the data.

[0032] In this embodiment, a neural network slicing method based on nodes is proposed. Accordingly, each layer of the neural network can be first divided into multiple nodes, that is, one node includes one layer or multiple adjacent layers in the neural network.

[0033] Optionally, each layer in the neural network can be divided into different nodes. For example, the neural network includes: a first layer neural network, a second layer neural network and a third layer neural network, and then the first layer neural network can be divided into node A, the second layer neural network can be divided into node B, and the third layer neural network can be divided into node C.

[0034] Among them, adjacent multi-layer neural networks may form a feedforward loop or a feedback loop.

[0035] The feedforward loop specifically refers to a certain layer of neural network (for example, the i-th layer of neural network). In addition to transmitting the calculated data to the adjacent subsequent layer of neural network, it also transmits the data to one or more layers after the subsequent layer of neural network (for example, the last layer to which the data is transmitted is the j-th layer of neural network). Therefore, a feedforward loop can be formed from the i-th layer of neural network to the j-th layer of neural network.

[0036] The so-called feedback loop specifically refers to a certain layer of neural network (for example, the m-th layer of neural network). In addition to transmitting the calculated data to the adjacent next layer of neural network, it also transmits the data to one or more layers before the layer of neural network (for example, the front layer to which the data is transmitted is the n-th layer of neural network). Then, a feedback loop can be formed from the n-th layer of neural network to the m-th layer of neural network.

[0037] The inventors have found through studying the performance of the feedback loop that in the feedback loop, the previous layer of the neural network needs to be calculated based on the data fed back by a certain layer of the neural network. Therefore, when slicing, if the above two layers of neural networks are divided into different chips, it will require a long data waiting time and a time alignment operation, which will increase the computational complexity of the entire algorithm and the computational time of the entire neural network. Therefore, each layer of the neural network in the feedback loop of the neural network can be divided into the same node to ensure that when slicing, the feedback loop is used as a unit and each layer of the neural network in the feedback loop is divided into one chip as a whole.

[0038] Similarly, when slicing, if two layers of neural networks in the same feedforward loop are divided into different chips, unnecessary computational complexity and waiting time will also be introduced. Therefore, it is optional to divide each layer of the neural network in the feedforward loop of the neural network into the same node to ensure that when slicing, the feedforward loop is used as a unit to divide each layer of the neural network in the feedforward loop into one chip as a whole.

[0039] After each layer in the neural network is divided into at least one node, the data output amount and data calculation amount corresponding to each of the nodes can be obtained.

[0040] The data output of a node refers to the amount of data outputted by the node, and the data calculation of a node refers to the amount of data calculation performed by the node. It is understandable that the structure of each layer in the neural network is known, and the data calculation method performed by each layer is also known. Therefore, the data output and data calculation of each layer in the neural network can be predetermined, and the data output and data calculation of the nodes composed of one or more layers of the neural network can be predetermined.

[0041] S120. Allocate each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node, and the standard calculation amount of the chip.

[0042] In this embodiment, the inventors mainly considered two indicators that will affect the collaborative computing performance of multiple chips after the neural network is sliced: one indicator is the rationality of the chip computing amount, that is, the computing amount of one or more layers of the neural network divided into each chip on the chip should match the standard computing amount that the chip can provide (the standard computing amount is specifically the maximum computing amount that a chip can provide in one calculation). For example, the standard computing amount that a chip can provide is A (MB). According to actual usage experience, the actual computing amount of the chip is preferably 70%-80% of the standard computing amount. If the actual computing amount is too small, it will cause a waste of chip performance. If the actual computing amount is too large, it will cause excessive computing load on the chip. Furthermore, the standard computing amount of the chip can be referred to, and each node can be reasonably allocated to at least one chip according to the corresponding data computing amount;

[0043] Another indicator is the data transmission efficiency between multiple chips. It is understandable that the data transmission speed within a chip is much greater than the data transmission speed between different chips. Therefore, the larger the amount of data that needs to be transmitted between two chips, the lower the data transmission efficiency between the two chips. Furthermore, in this embodiment, it is further considered that the data output of the sharding node cannot be too large to ensure the data transmission efficiency between chips. A sharding node refers to a node and one or more nodes before the node that will be divided into one chip, and one or more nodes after the node will be divided into another chip, that is, the data output of the sharding node determines the data transmission volume between the two chips.

[0044] Based on the above creative analysis, the inventor proposed a new neural network slicing method that takes into account both the single-chip data computing amount and the data transmission amount between chips, so as to maximize the compromise between the rationality of chip computing amount and the data transmission efficiency between multiple chips.

[0045] Optionally, each layer in the neural network corresponds to at least one hardware environment, wherein different hardware environments correspond to different types of chips, such as a CPU (Central Processing Unit) chip and an APU (Accelerated Processing Unit) chip.

[0046] It is understandable that each layer in a neural network may need to perform corresponding calculations in different hardware environments (or different types of chips). Therefore, it is necessary to further consider slicing each layer of the neural network according to the hardware environment of each layer in the neural network. For example, neural network B includes: a first layer neural network, a second layer neural network, a third layer neural network, and a fourth layer neural network. Among them, the first layer neural network, the second layer neural network, and the fourth layer neural network need to run on an APU chip, and the third layer neural network needs to run on a CPU chip. At this time, before considering the data transmission and data output of each layer, the third layer neural network needs to be sliced ​​separately first.

[0047] It should be noted again that the various embodiments of the present invention are sliced ​​in units of nodes, so it is necessary to ensure that the various layers of neural networks included in a node correspond to the same hardware environment. Considering that there is data transmission inside the feedback loop and the feedforward loop, the various layers of neural networks on the feedback loop and the various layers of neural networks on the feedforward loop generally correspond to the same hardware environment. Therefore, the method of dividing the nodes separately for each layer, or dividing the feedback loop and the feedforward loop to the same node, can meet the requirement that the various layers of neural networks included in the above-mentioned node correspond to the same hardware environment.

[0048] Accordingly, when each layer in the neural network corresponds to multiple hardware environments, or when multiple types of chips are required to perform collaborative operations on the neural network, allocating each node to at least one chip according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip may include:

[0049] According to the data output volume of each node, the data calculation volume of each node, the hardware environment of each node and the standard calculation volume of different types of chips, each node is allocated to at least one chip, and different hardware environments correspond to different types of chips.

[0050] That is, when each layer in a neural network corresponds to multiple hardware environments, when slicing the neural network, it is necessary to further consider the hardware execution environment of each node.

[0051] The technical solution of the embodiment of the present invention, when slicing a neural network, divides each layer of the neural network into nodes, and considers the data output amount corresponding to each node, the data calculation amount of each node and the standard calculation amount of the chip at the same time, and allocates each node to at least one chip. This provides a new neural network slicing method that takes into account both the data calculation amount of a single chip and the data transmission amount between chips. It can minimize the data transmission amount between chips while ensuring that the data calculation amount of each chip is as balanced as possible, reduce the data transmission amount between chips, shorten the transmission time, and evenly distribute the load of multiple chips after slicing, thereby maintaining a balanced computing intensity for each chip, and completely avoiding the investment in manpower costs while improving the collaborative computing performance of each chip after slicing.

[0052] Embodiment 2

[0053] Figure 2a This is a flowchart of a method for dividing the structure of a neural network provided in the second embodiment of the present invention. This embodiment is refined based on the above embodiment. In this embodiment, each layer in the neural network is divided into multiple nodes, and the operation of obtaining the data output amount and the data calculation amount corresponding to each of the nodes is refined. Accordingly, the method of this embodiment specifically includes:

[0054] S210: When it is determined that at least one feedback loop is identified in the neural network, multiple layers corresponding to the same feedback loop are divided into the same node.

[0055] In this embodiment, it can be first determined whether there is one or more feedback loops in the neural network. If it is determined that there are, multiple layers corresponding to the same feedback loop are divided into the same node.

[0056] S220: In determining in the neural network, at least one feedforward loop is identified, and multiple layers corresponding to the same feedforward loop are divided into the same node.

[0057] In this embodiment, it is possible to continue to determine whether there is one or more feedforward loops in the neural network. If it is determined that there are, multiple layers corresponding to the same feedforward loop are divided into the same node.

[0058] S230, dividing the remaining layers in the neural network into different nodes one by one.

[0059] Among them, Figure 2b FIG. 2 shows a schematic diagram of a neural network structure applicable to an embodiment of the present invention. Figure 2bAs shown, the neural network is composed of the Ath layer neural network to the Nth layer neural network, and includes in the neural network: a feedback loop composed of the Cth layer to the Eth layer neural network, and a feedforward loop composed of the Gth layer to the Ith layer neural network. Therefore, in order to reduce the time delay or complex time alignment control caused by dividing the layers in the above feedback loop or feedforward loop into different chips, in this embodiment, the Cth layer to the Eth layer neural network in the feedback loop can be divided into one node, and the Gth layer to the Ith layer neural network in the feedforward loop can be divided into one node.

[0060] At the same time, the remaining layers of the neural network except the feedback loop and the feedforward loop are divided into different nodes respectively. For example, the A-th layer of the neural network is divided into one node, the B-th layer of the neural network is divided into one node, and so on, the N-th layer of the neural network is divided into one node.

[0061] S240, obtaining the data output amount and data calculation amount corresponding to each of the nodes.

[0062] In this embodiment, the data calculation amount of each node specifically refers to the sum of the data calculation amounts of each layer of the neural network in the node. For example, if node 1 only corresponds to the Ath layer of the neural network, the Ath layer of the neural network can be directly used as the data calculation amount of node 1. If node 3 corresponds to the feedback loop composed of the Cth layer and the Eth layer of the neural network, the sum of the data calculation amounts of each layer in the Cth layer, the Dth layer of the neural network and the Eth layer of the neural network can be used as the data calculation amount of node 3.

[0063] In this embodiment, when calculating the data output amount of each node, it is necessary to comprehensively consider the structure of at least one layer of the neural network corresponding to each node.

[0064] Optionally, if it is determined that the currently processed target node corresponds to a feedforward loop, the data output amount of the last layer of the feedforward loop is determined as the data output amount of the target node.

[0065] like Figure 2b As shown, a target node corresponds to a feedforward loop composed of the Gth layer to the Ith layer of the neural network, then the data output 1M (MB) of the last layer of the feedforward loop (the Ith layer of the neural network) can be used as the data output of the target node.

[0066] Optionally, if it is determined that the target node currently being processed is the first layer in a feedforward loop, the sum of branch data output amounts of at least two output branches of the target node is determined as the data output amount of the target node.

[0067] like Figure 2bAs shown, assuming that the G-th to I-th layers of the neural network are not divided into one node, and the G-th layer of the neural network (the first layer in the feedforward loop) is divided into a target node alone. The target node has two output branches, one is the standard output branch that normally transmits data to the H-th layer of the neural network, and the branch data output of this branch is 10M, and the other is the feedforward output branch that feeds forward data to the I-th layer of the neural network, and the branch data output of this branch is 5M. Therefore, 10M+5M=15M can be used as the data output of the target node.

[0068] Optionally, if it is determined that the target node currently being processed is the last layer in a feedforward loop, the data output amount of the target node is determined as the data output amount of the target node.

[0069] like Figure 2b As shown, it is assumed that the G-th to I-th layers of the neural network are not divided into one node, and the I-th layer of the neural network (the last layer in the feedforward loop) is divided into a target node alone, and then the data output 1M of the I-th layer of the neural network can be used as the data output of the target node.

[0070] Optionally, if it is determined that the target node currently being processed is a layer other than the first and last layers in a feedforward loop, the data output amount of the target node and the sum of the branch data output amount of the feedforward output branch of the first layer in the feedforward loop are determined as the data output amount of the target node.

[0071] like Figure 2b As shown, assuming that the G-th to I-th layers of the neural network are not divided into one node, and the H-th layer of the neural network (other layers except the first and last layers in the feedforward loop) is divided into a target node alone, the data output of the H-th layer of the neural network is 10M, and the branch data output of the feedforward output branch of the first layer (G-th layer of the neural network) in the feedforward loop to which the H-th layer of the neural network belongs is 5M. Therefore, 10M+5M=15M can be used as the data output of the target node.

[0072] Optionally, if a node corresponds to a layer in a neural network, and the layer does not belong to a feedforward network in the neural network, the data output of the layer can be directly used as the data output of the node. Figure 2b As shown, node 1 corresponds to the A-th layer of the neural network, and the data output volume 30M of the A-th layer of the neural network can be used as the data output volume of node 1.

[0073] Optionally, if a node corresponds to a feedback loop in a neural network, the data output of the last node in the feedback loop can be directly used as the data output of the node. For example, as shown in 2b, node 3 corresponds to a feedback loop consisting of a C-th layer to an E-th layer neural network, and the data output of the E-th layer neural network, 50M, can be used as the data output of node 3.

[0074] S250. Allocate each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node, and the standard calculation amount of the chip.

[0075] The technical solution of the embodiment of the present invention divides the feedback loop or feedforward loop included in the neural network into the same node, and divides the chip with the node as the smallest unit. While taking into account the data calculation amount of a single chip and the data transmission amount between chips, it reduces the time delay or complex time alignment control caused by dividing each layer in the feedback loop or feedforward loop in the neural network into different chips, so as to further improve the collaborative computing performance of each chip after the neural network is sliced.

[0076] Embodiment 3

[0077] Figure 3a This is a flowchart of a method for dividing the structure of a neural network provided in Embodiment 3 of the present invention. This embodiment is refined based on the above embodiment. In this embodiment, the operation of allocating each node to at least one chip is further refined according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip. Accordingly, the method of this embodiment specifically includes:

[0078] S310, dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and data calculation amount corresponding to each of the nodes.

[0079] S320, obtaining the first node among all nodes as the current processing node.

[0080] In this embodiment, each node is arranged according to the arrangement order of each layer in the neural network, and then each node can be divided into one or more chips in sequence starting from the first node. The current processing node specifically refers to the node that currently needs to be divided into a new chip.

[0081] in, Figure 3b A schematic diagram of a slicing process for each node applicable to an embodiment of the present invention is shown. Figure 3b As shown, node 1 can be used as the slicing starting point, and the slicing process of each node can be implemented in the order from node 1 to node 10.

[0082] S330 , obtaining a target chip whose chip type matches the hardware environment of the current processing node.

[0083] After the current processing node is acquired, it is necessary to first determine, based on the hardware environment of the current processing node, what type of target chip (eg, CPU or APU) the current processing node needs to be allocated to for execution.

[0084] S340, determine whether there is at least one continuous subsequent node that belongs to the same hardware environment as the current processing node: if so, execute S350; otherwise, execute S360.

[0085] As mentioned above, nodes corresponding to different hardware environments inherently need to be sliced ​​into different types of chips. Therefore, after obtaining the target chip that matches the current processing chip, you can take the current processing node as the starting point and continue to traverse each node to obtain one or more consecutive subsequent nodes that belong to the same hardware environment as the current processing node. The above-mentioned current processing node and at least one subsequent node can be divided into one or more target chips of the same type.

[0086] like Figure 3b As shown, among nodes 1 to 10 corresponding to the neural network, only node 8 needs to be divided into the CPU chip for calculation, and other nodes can be calculated on the APU chip. Therefore, after starting the traversal with node 1 as the current processing node, nodes 2-node 7 can be obtained as at least one continuous subsequent node belonging to the same hardware environment as node 1, and then the above nodes 1-node 7 can be divided into one or more APU chips.

[0087] After completing the division of nodes 1-7, node 8 can be obtained as the current processing node, and node 8 can be divided into a CPU chip separately. Finally, node 9 can be used as the current processing node, and node 9 and node 10 can be divided into one or more APU chips.

[0088] S350 , calculating the expected computing amount range of the target chip according to the standard computing amount of the target chip and the preset ratio range, and executing S370 .

[0089] Among them, S350, S370-S3130 specifically describe in detail the process of how to divide the current processing node and at least one subsequent node into one or more target chips of the same type.

[0090] In this embodiment, the above process is described in detail by taking the process of dividing the above nodes 1 to 7 into one or more APU chips as an example.

[0091] First, after determining the target chip that matches the hardware environment of the current processing node and at least one subsequent node, the expected computing amount interval of the target chip can be calculated based on the standard computing amount of the target chip (also referred to as the maximum computing amount of a single calculation) and the preset ratio range. The expected computing amount interval specifically refers to the computing amount expected to be run in the target chip.

[0092] For example, if the standard computing capacity of the target chip is 100MB and the preset ratio range is (70%-80%), then the expected computing capacity interval corresponding to the target chip can be calculated as 100MB*0.7-100MB*0.8, that is, 70MB-80MB. Therefore, the sum of the data computing capacity of one or more nodes allocated on each target chip is preferably within the above-mentioned expected computing capacity interval.

[0093] S360 , assign the current processing node to only one target chip, and execute S3150 .

[0094] S370. From the current processing node and each subsequent node, obtain one node in turn as the current node, and update the node calculation amount according to the data calculation amount of the current node.

[0095] The node calculation amount is used to measure the cumulative sum of the data calculation amount of each node allocated on a target chip, and the node calculation amount has a preset initial value. Optionally, when a corresponding node needs to be allocated to a new target chip, the node calculation amount needs to be cleared to 0.

[0096] S380, determine the relationship between the total amount of node calculation and the expected amount of calculation interval: if it is determined that the total amount of node calculation is less than or equal to the minimum value of the expected amount of calculation interval, execute S390; if it is determined that the total amount of node calculation falls within the expected amount of calculation interval, execute S3100; if the total amount of node calculation is greater than or equal to the maximum value of the expected amount of calculation interval, execute S3110.

[0097] As in the previous example, you can sequentially obtain a node from node 1 to node 7 as the current node, and accumulate the node calculation amount according to the data calculation amount of the current node. If the node calculation amount accumulated by the current node does not exceed the expected calculation amount interval, the current node is not suitable as a sharding node, and you can continue to obtain the next node for accumulation. If the node calculation amount accumulated by the current node is in the expected calculation amount interval, the current node can be used as a candidate for a sharding node (that is, a candidate node), and the node can be added to the candidate node set for subsequent actual sharding nodes (that is, target nodes). If the current accumulated node calculation amount is greater than or equal to the maximum value of the expected calculation amount interval, it means that a sharding is required before the current node. At this time, it is necessary to determine the target node from the candidate node set for sharding. That is, the operation performed by S3120.

[0098] S390, continue to determine whether the current node is the last post-order node: if not, return to execute S370; if yes, execute S3110.

[0099] S3100, adding the current node to the candidate node set, and returning to execute S390.

[0100] S3110. Determine whether the candidate node set includes at least one candidate node: if so, execute S3120; otherwise, execute S3130.

[0101] S3120, determine the target node in each candidate node according to the data output amount of each candidate node in the candidate node set, and divide each node between the current first unallocated node and the target node into a target chip, and execute S3140.

[0102] The unallocated nodes specifically refer to nodes that are not allocated to a certain chip.

[0103] In an optional implementation of this embodiment, determining the target node from among the candidate nodes according to the data output amount of each candidate node in the candidate node set may include:

[0104] Obtaining a minimum data output amount from among the data output amounts of the candidate nodes;

[0105] If it is determined that the minimum data output amount is less than or equal to the transmission upper limit of the target chip, the candidate node corresponding to the minimum data output amount is determined as the target node.

[0106] In this embodiment, each candidate node included in the candidate node set will make the data calculation amount of the single target chip within the expected calculation amount range. Furthermore, a candidate node with the minimum data output amount can be selected from the above candidate nodes, and when the minimum output amount meets the target chip transmission upper limit, the candidate node with the minimum data output amount is used as the final sharding node, that is, the target node. The transmission upper limit refers to the maximum amount of data that can be transmitted when the target chip performs inter-chip data transmission.

[0107] Furthermore, if it is determined that the minimum data output amount is greater than the transmission upper limit of the target chip, it means that each node in the candidate node set is not suitable as the target node of the actual sharding. Therefore, the first candidate node in the candidate node set can be used as the backtracking starting point, and the node with a data output amount less than or equal to the transmission upper limit of the target chip can be traced back forward to be used as the target node.

[0108] In a specific example, if it is determined that when sharding the first target node, node 3 and node 4 are both candidate nodes, and the data output of the above nodes 3 and 4 are both higher than the transmission upper limit of a target chip, then node 3 can be used as the backtracking starting point, and a node whose data output is less than or equal to the transmission upper limit of the target chip can be obtained along the direction of node 2-node 1 as the target node.

[0109] S3130 , all unallocated nodes in the current processing node and each subsequent node are divided into one target chip, and S3150 is executed.

[0110] Among them, S3130 mainly considers such a situation that when the current processing node and all unallocated nodes in each subsequent node are allocated to a target chip, the accumulated total node calculation amount does not reach the expected calculation amount range of the target chip. At this time, all the above-mentioned unallocated nodes can be directly divided into one target chip.

[0111] S3140. Determine whether the target node is the last post-order node: if not, return to execute S370; otherwise, execute S3150.

[0112] S3150, determine whether there is a subsequent node to the current last allocated node: if so, execute S3160; otherwise, end the process.

[0113] The last allocated node refers to the last node among all nodes allocated to a certain chip.

[0114] S3160: After taking the next node as the new current processing node, return to execute S330.

[0115] The technical solution of the embodiment of the present invention, when slicing a neural network, divides each layer of the neural network into nodes, and considers the data output amount corresponding to each node, the data calculation amount of each node and the standard calculation amount of the chip at the same time, and allocates each node to at least one chip. This provides a new neural network slicing method that takes into account both the data calculation amount of a single chip and the data transmission amount between chips. It can minimize the data transmission amount between chips while ensuring that the data calculation amount of each chip is as balanced as possible, reduce the data transmission amount between chips, shorten the transmission time, and evenly distribute the load of multiple chips after slicing, thereby maintaining a balanced computing intensity for each chip, and completely avoiding the investment in manpower costs while improving the collaborative computing performance of each chip after slicing.

[0116] Embodiment 4

[0117] Figure 4 4 is a structural diagram of a neural network structure division device provided in Embodiment 4 of the present invention. The neural network includes multiple layers, and the device includes: a node division module 410 and a node allocation module 420, wherein:

[0118] A node division module 410 is used to divide each layer in the neural network into multiple nodes, and obtain the data output amount and data calculation amount corresponding to each node;

[0119] The node allocation module 420 is used to allocate each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip.

[0120] The technical solution of the embodiment of the present invention, when slicing a neural network, divides each layer of the neural network into nodes, and considers the data output amount corresponding to each node, the data calculation amount of each node and the standard calculation amount of the chip at the same time, and allocates each node to at least one chip. This provides a new neural network slicing method that takes into account both the data calculation amount of a single chip and the data transmission amount between chips. It can minimize the data transmission amount between chips while ensuring that the data calculation amount of each chip is as balanced as possible, reduce the data transmission amount between chips, shorten the transmission time, and evenly distribute the load of multiple chips after slicing, thereby maintaining a balanced computing intensity for each chip, and completely avoiding the investment in manpower costs while improving the collaborative computing performance of each chip after slicing.

[0121] Based on the above embodiments, the node division module 410 may include:

[0122] A feedback loop division unit, configured to divide a plurality of layers corresponding to the same feedback loop into the same node when it is determined that at least one feedback loop is identified in the neural network;

[0123] The remaining layer division unit is used to divide the remaining layers in the neural network into different nodes one by one.

[0124] Based on the above embodiments, the present invention may further include: a feedforward loop division unit, which is used to:

[0125] Before dividing the remaining layers in the neural network into different nodes one by one, when it is determined that at least one feedforward loop is identified in the neural network, multiple layers corresponding to the same feedforward loop are divided into the same node.

[0126] Based on the above embodiments, the node partitioning module 410 may be specifically used for at least one of the following:

[0127] If it is determined that the currently processed target node corresponds to a feedforward loop, the data output amount of the last layer of the feedforward loop is determined as the data output amount of the target node;

[0128] If it is determined that the currently processed target node is the first layer in a feedforward loop, the sum of the branch data output amounts of at least two output branches of the target node is determined as the data output amount of the target node;

[0129] If it is determined that the target node currently being processed is the last layer in a feedforward loop, the data output amount of the target node is determined as the data output amount of the target node;

[0130] If it is determined that the target node currently being processed is a layer other than the first and last layers in a feedforward loop, the data output amount of the target node and the sum of the branch data output amounts of the feedforward output branch of the first layer in the feedforward loop are determined as the data output amount of the target node.

[0131] Based on the above embodiments, each layer in the neural network may correspond to at least one hardware environment;

[0132] The node allocation module 420 may be specifically used for:

[0133] According to the data output volume of each node, the data calculation volume of each node, the hardware environment of each node and the standard calculation volume of different types of chips, each node is allocated to at least one chip, and different hardware environments correspond to different types of chips.

[0134] Based on the above embodiments, the node allocation module 420 may specifically include:

[0135] A target chip acquisition unit is used to acquire the first node among the nodes as the current processing node, and acquire a target chip whose chip type matches the hardware environment of the current processing node;

[0136] The first type of node processing unit is used to allocate the current processing node and each subsequent node to at least one target chip according to the data output amount, data calculation amount and standard calculation amount of the target chip of the current processing node and each subsequent node if there is at least one continuous subsequent node belonging to the same hardware environment as the current processing node;

[0137] The second type of node processing unit is used to allocate only the current processing node to one target chip if the node following the current processing node does not belong to the same hardware environment as the current processing node;

[0138] The repeated execution unit is used to, if there is a subsequent node to the current last allocated node, take the subsequent node as the new current processing node, and then return to execute the operation of obtaining a target chip whose chip type matches the hardware environment of the current processing node until all nodes are processed.

[0139] Based on the above embodiments, the first type of node processing unit may include:

[0140] An expected computing amount interval calculation subunit, used to calculate the expected computing amount interval of the target chip according to the standard computing amount of the target chip and a preset ratio range;

[0141] A node calculation total amount updating unit is used to sequentially obtain a node from the current processing node and each subsequent node as the current node, and update the node calculation total amount according to the data calculation amount of the current node, wherein the node calculation total amount has a preset initial value;

[0142] A first return execution subunit, configured to return to execute the operation of sequentially acquiring a node as the current node if the total node calculation amount is less than or equal to the minimum value of the expected calculation amount interval and the current node has a subsequent node;

[0143] The second return execution subunit is used for adding the current node to the candidate node set if the total amount of node calculation falls within the expected amount of calculation interval, and returning to execute the operation of sequentially obtaining a node as the current node until the total amount of node calculation is greater than or equal to the maximum value of the expected amount of calculation interval or the current node is the last node;

[0144] The node division subunit is used to determine the target node among the candidate nodes according to the data output amount of each candidate node in the candidate node set, and divide each node between the current first unassigned node and the target node into a target chip;

[0145] The third return execution sub-unit is used to return and execute the operation of obtaining a node as the current node in turn among the current processing node and each subsequent node, and updating the node calculation amount according to the data calculation amount of the current node until the sharding processing of all nodes is completed.

[0146] Based on the above embodiments, the node division subunit may include:

[0147] Obtaining a minimum data output amount from among the data output amounts of the candidate nodes;

[0148] If it is determined that the minimum data output amount is less than or equal to the transmission upper limit of the target chip, the candidate node corresponding to the minimum data output amount is determined as the target node.

[0149] On the basis of the above embodiments, the node division subunit can also be used for: after obtaining the minimum data output amount among the data output amounts of each of the candidate nodes, if it is determined that the minimum data output amount is greater than the transmission upper limit of the target chip, then taking the first candidate node in the candidate node set as the backtracking starting point, backtracking forward to obtain a node whose data output amount is less than or equal to the transmission upper limit of the target chip as the target node.

[0150] The neural network structure division device provided in the embodiment of the present invention can execute the neural network structure division method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0151] Embodiment 5

[0152] Figure 5a A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention is shown in FIG. Figure 5a As shown, the computer device includes a processor 50, a memory 51, an input device 52 and an output device 53; the number of processors 50 in the computer device can be one or more. Figure 5a In the example, four processors 50 are used; the processor 50, memory 51, input device 52 and output device 53 in the computer device can be connected by a bus or other means. Figure 5a The example of connecting through bus is taken in the following.

[0153] exist Figure 5a In the embodiment, the four processors 50 can cooperate with each other to implement the method described in any embodiment of the present invention.

[0154] Among them, Figure 5b The internal structure diagram of a processor applicable to the embodiment of the present invention is shown in FIG. Figure 5bAs shown, the processor 50 corresponds to the chip described in any embodiment of the present invention, and each chip includes one or more computing cores 503. Figure 5b The four computing cores shown in the figure are only examples. In fact, a required number of computing cores can be set on the chip according to actual needs, and this embodiment does not limit this. Each computing core 503 includes a computing unit 501 and a storage unit 502. The computing unit 501 is used to implement the core calculation in the computing core 503, and the storage unit 502 is used to store the data calculated in the computing core 503 on the chip.

[0155] The memory 51 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as the modules corresponding to the multi-task parallel processing method in the embodiment of the present invention. The processor 50 executes various functional applications and data processing of the computer device by running the software programs, instructions and modules stored in the memory 51, that is, implements the structure division method of the neural network as described in any embodiment of the present invention, wherein the neural network includes multiple layers, and the method includes:

[0156] Dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and the data calculation amount corresponding to each of the nodes;

[0157] According to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip, each of the nodes is allocated to at least one chip.

[0158] The memory 51 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 51 may further include a memory remotely arranged relative to the processor 50, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The input device 52 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the computer device. The output device 53 may include a display device such as a display screen.

[0160] Embodiment 6

[0161] Embodiment 6 of the present invention further provides a storage medium containing computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute the structure division method of a neural network as described in any embodiment of the present invention, wherein the neural network includes multiple layers, and the method includes:

[0162] Dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and the data calculation amount corresponding to each of the nodes;

[0163] According to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip, each of the nodes is allocated to at least one chip.

[0164] Of course, the computer executable instructions of a storage medium including computer executable instructions provided by an embodiment of the present invention are not limited to the above method operations, and can also execute related operations in the method provided by any embodiment of the present invention.

[0165] Through the above description of the implementation mode, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention.

[0166] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for structural division of a neural network, characterized in that: The neural network includes multiple layers, and the method includes: Dividing each layer in the neural network into a plurality of nodes, and obtaining the data output amount and the data calculation amount corresponding to each of the nodes; Allocating each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip; Wherein, each layer in the neural network corresponds to at least one hardware environment; each node is allocated to at least one chip according to the data output of each node, the data calculation amount of each node and the standard calculation amount of the chip, including: each node is allocated to at least one chip according to the data output of each node, the data calculation amount of each node, the hardware environment of each node and the standard calculation amount of different types of chips, and different hardware environments correspond to different types of chips; wherein, each node is allocated to at least one chip according to the data output of each node, the data calculation amount of each node, the hardware environment of each node and the standard calculation amount of different types of chips, including: obtaining the first node in each node as the current processing node, and obtaining a target chip whose chip type matches the hardware environment of the current processing node; if there is at least one continuous subsequent node belonging to the same hardware environment as the current processing node, the current processing node and each subsequent node are allocated to at least one target chip according to the data output amount, data calculation amount and standard calculation amount of the target chip of the current processing node and each subsequent node; if the subsequent node of the current processing node does not belong to the same hardware environment as the current processing node, the current processing node is only allocated to one target chip; if there is a subsequent node of the current last allocated node, the subsequent node is used as the new current processing node, and then the operation of obtaining the target chip whose chip type matches the hardware environment of the current processing node is returned until all nodes are processed; Among them, according to the data output amount, data calculation amount and standard calculation amount of the target chip of the current processing node and each subsequent node, the current processing node and each subsequent node are allocated to at least one target chip, including: according to the standard calculation amount of the target chip and the preset ratio range, the expected calculation amount interval of the target chip is calculated; among the current processing node and each subsequent node, one node is sequentially obtained as the current node, and according to the data calculation amount of the current node, the node calculation amount is updated, and the node calculation amount has a preset initial value; if the node calculation amount is less than or equal to the minimum value of the expected calculation amount interval and the current node has a subsequent node, then return to execute the operation of sequentially obtaining one node as the current node; if If the total amount of node calculation falls within the expected amount of calculation range, the current node is added to the candidate node set, and the operation of obtaining a node as the current node in turn is returned to be executed until the total amount of node calculation is greater than or equal to the maximum value of the expected amount of calculation range or the current node is the last node; according to the data output amount of each candidate node in the candidate node set, the target node is determined in each candidate node, and each node between the current first unassigned node and the target node is divided into a target chip; the operation of obtaining a node as the current node in turn from the current processing node and each subsequent node according to the data calculation amount of the current node is returned to be executed, and the operation of updating the total amount of node calculation is updated according to the data calculation amount of the current node until the sharding processing of all nodes is completed.

2. The method according to claim 1, characterized in that Divide each layer in the neural network into multiple nodes, including: In the neural network, if at least one feedback loop is identified, multiple layers corresponding to the same feedback loop are grouped into the same node; The remaining layers in the neural network are divided into different nodes one by one.

3. The method according to claim 2, characterized in that Before dividing the remaining layers in the neural network into different nodes one by one, the method further includes: In the neural network, if at least one feedforward loop is identified, multiple layers corresponding to the same feedforward loop are divided into the same node.

4. The method according to claim 1, wherein obtaining the data output amount corresponding to each of the nodes comprises at least one of the following: If it is determined that the currently processed target node corresponds to a feedforward loop, the data output amount of the last layer of the feedforward loop is determined as the data output amount of the target node; If it is determined that the currently processed target node is the first layer in a feedforward loop, the sum of the branch data output amounts of at least two output branches of the target node is determined as the data output amount of the target node; If it is determined that the target node currently being processed is the last layer in a feedforward loop, the data output amount of the target node is determined as the data output amount of the target node; If it is determined that the target node currently being processed is a layer other than the first and last layers in a feedforward loop, the data output amount of the target node and the sum of the branch data output amounts of the feedforward output branch of the first layer in the feedforward loop are determined as the data output amount of the target node.

5. The method according to claim 1, characterized in that According to the data output amount of each candidate node in the candidate node set, a target node is determined among the candidate nodes, including: Obtaining a minimum data output amount from among the data output amounts of the candidate nodes; If it is determined that the minimum data output amount is less than or equal to the transmission upper limit of the target chip, the candidate node corresponding to the minimum data output amount is determined as the target node.

6. The method according to claim 5, characterized in that After obtaining the minimum data output amount among the data output amounts of the candidate nodes, the method further includes: If it is determined that the minimum data output amount is greater than the transmission upper limit of the target chip, the first candidate node in the candidate node set is used as the backtracking starting point, and the node with a data output amount less than or equal to the transmission upper limit of the target chip is backtracked forward as the target node.

7. A structure division device for a neural network, characterized in that: The neural network includes multiple layers, and the device includes: A node division module, used to divide each layer in the neural network into multiple nodes, and obtain the data output amount and data calculation amount corresponding to each of the nodes; A node allocation module, used for allocating each of the nodes to at least one chip according to the data output amount of each node, the data calculation amount of each node and the standard calculation amount of the chip; Wherein, each layer in the neural network corresponds to at least one hardware environment; the node allocation module is specifically used to: allocate each node to at least one chip according to the data output amount of each node, the data calculation amount of each node, the hardware environment of each node and the standard calculation amount of different types of chips, and different hardware environments correspond to different types of chips; wherein the node allocation module specifically includes: A target chip acquisition unit is used to acquire the first node among the nodes as the current processing node, and acquire a target chip whose chip type matches the hardware environment of the current processing node; The first type of node processing unit is used to allocate the current processing node and each subsequent node to at least one target chip according to the data output amount, data calculation amount and standard calculation amount of the target chip of the current processing node and each subsequent node if there is at least one continuous subsequent node belonging to the same hardware environment as the current processing node; The second type of node processing unit is used to allocate only the current processing node to one target chip if the node following the current processing node does not belong to the same hardware environment as the current processing node; A repeating execution unit, for, if there is a subsequent node to the currently last allocated node, taking the subsequent node as a new current processing node, and returning to execute the operation of obtaining a target chip whose chip type matches the hardware environment of the current processing node, until all nodes are processed; Among them, according to the data output amount, data calculation amount and standard calculation amount of the target chip of the current processing node and each subsequent node, the current processing node and each subsequent node are allocated to at least one target chip, including: according to the standard calculation amount of the target chip and the preset ratio range, the expected calculation amount interval of the target chip is calculated; among the current processing node and each subsequent node, one node is sequentially obtained as the current node, and according to the data calculation amount of the current node, the node calculation amount is updated, and the node calculation amount has a preset initial value; if the node calculation amount is less than or equal to the minimum value of the expected calculation amount interval and the current node has a subsequent node, then return to execute the operation of sequentially obtaining one node as the current node; if If the total amount of node calculation falls within the expected amount of calculation range, the current node is added to the candidate node set, and the operation of obtaining a node as the current node in turn is returned to be executed until the total amount of node calculation is greater than or equal to the maximum value of the expected amount of calculation range or the current node is the last node; according to the data output amount of each candidate node in the candidate node set, the target node is determined in each candidate node, and each node between the current first unassigned node and the target node is divided into a target chip; the operation of obtaining a node as the current node in turn from the current processing node and each subsequent node according to the data calculation amount of the current node is returned to be executed, and the operation of updating the total amount of node calculation is updated according to the data calculation amount of the current node until the sharding processing of all nodes is completed.

8. The device according to claim 7, characterized in that The node division module comprises: A feedback loop division unit, configured to divide a plurality of layers corresponding to the same feedback loop into the same node if at least one feedback loop is identified in the neural network; The remaining layer division unit is used to divide the remaining layers in the neural network into different nodes one by one.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for structural division of a neural network as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for structural division of a neural network as described in any one of claims 1 to 6 is implemented.

Images