Neural network splitting method and apparatus, electronic device, computer readable medium

Abstract

The present disclosure provides a neural network splitting method, comprising: according to the positional relationship and connection relationship of a plurality of neurons of a neural network, the plurality of neurons are divided into a plurality of macro groups; wherein each macro group comprises a plurality of neurons, the neurons in the same macro group are used for mapping to one operation unit processing, and the neurons in different macro groups are used for mapping to different operation unit processing. The present disclosure also provides a neural network splitting device, an electronic device and a computer readable medium.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method and apparatus for neural network decomposition, an electronic device, and a computer-readable medium. Background Technology

[0002] With the development of technology (especially artificial intelligence technology), a single computing unit can no longer meet the needs of neural network training and inference.

[0003] Breaking down neural networks into multiple parts, with each part processed on a different computational unit, has become the best choice for neural network training and inference. Summary of the Invention

[0004] This disclosure provides a method and apparatus for neural network decomposition based on a many-core system, a processing core, an electronic device, and a computer-readable medium.

[0005] In a first aspect, this disclosure provides a method for splitting a neural network, which includes: dividing multiple neurons into multiple macrogroups according to the positional and connection relationships of multiple neurons in the neural network; wherein each macrogroup includes multiple neurons, neurons in the same macrogroup are used to be mapped to one computational unit for processing, and neurons in different macrogroups are used to be mapped to different computational units for processing.

[0006] Secondly, this disclosure provides a neural network splitting device, which includes: a splitting module, used to divide multiple neurons into multiple macrogroups according to the positional and connection relationships of multiple neurons in the neural network; wherein each macrogroup includes multiple neurons, neurons in the same macrogroup are used to be mapped to one computing unit for processing, and neurons in different macrogroups are used to be mapped to different computing units for processing.

[0007] Thirdly, this disclosure provides an electronic device comprising: a plurality of processing cores; and an on-chip network configured to interact with data and external data between the plurality of processing cores; wherein one or more of the processing cores store one or more instructions, and the one or more instructions are executed by the one or more processing cores to enable the one or more processing cores to perform the neural network splitting method described above.

[0008] Fourthly, this disclosure provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processing core, implements the aforementioned neural network decomposition method.

[0009] The neural network decomposition method, apparatus, processing core, electronic device, and computer-readable medium disclosed herein decompose the neural network according to the positional and connectivity relationships of multiple neurons. After the decomposition and mapping of the neural network are completed, the information transmission cost and time required between the various parts of the neural network can be significantly reduced.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0012] Figure 1 A flowchart of a neural network decomposition method provided in this disclosure embodiment;

[0013] Figure 2 A flowchart of a neural network decomposition method provided in this disclosure embodiment;

[0014] Figure 3 A flowchart of a neural network decomposition method provided in this disclosure embodiment;

[0015] Figure 4 A flowchart illustrating some steps of a neural network decomposition method provided in this disclosure embodiment;

[0016] Figure 5 A schematic diagram illustrating a neural network decomposition method provided in an embodiment of this disclosure;

[0017] Figure 6 A block diagram of a neural network splitting device provided in an embodiment of this disclosure;

[0018] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0020] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0021] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0023] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0024] In some embodiments, a neural network can be run through a processing system (e.g., a many-core system, a server cluster, etc.) having multiple computing units, each computing unit being, for example, one or a group of processing cores in a many-core system, or one or a group of servers in a server cluster. This disclosure does not limit the specific structure of the processing system and computing units.

[0025] In some related technologies, neural networks are split according to the location information of neurons, such as their layer, that is, neurons that are close in spatial location are assigned to the same processing unit for processing.

[0026] However, because the connection between neurons was not considered when splitting the neural network, that is, the transmission of information between neurons was not considered, it is easy for neurons assigned to different operation units to transmit information very frequently during the operation of the neural network. This further leads to frequent transmission of information between different operation units, which in turn affects the operation of the neural network.

[0027] Figure 1 A flowchart of a neural network decomposition method provided in an embodiment of this disclosure.

[0028] This disclosure provides a neural network splitting method. The method is applied to a neural network splitting device, which can be a terminal or server (e.g., a compiler), or a computing unit or hardware processing component in a many-core system. This disclosure does not limit the specific structure of the neural network splitting device.

[0029] Reference Figure 1 The neural network decomposition method includes:

[0030] S101. Based on the positional and connectivity relationships of multiple neurons in the neural network, the multiple neurons are divided into multiple macrogroups.

[0031] The neural network splitting device acquires the positional and connectivity relationships of neurons in a neural network and, based on these relationships, divides the neurons into multiple macrogroups, each of which contains multiple neurons.

[0032] In this embodiment of the present disclosure, the neural network can be composed of multiple neurons, each neuron being connected to some neurons in the neural network, and sending information or receiving information from other neurons through synapses.

[0033] The positional relationships of multiple neurons include the spatial position information of each neuron and the relative spatial position information of each neuron with other neurons. The connection relationships of multiple neurons include the connection relationships of each neuron with other neurons.

[0034] Of course, a neural network is not a physical structure that exists in reality. Therefore, the above location information and connections represent the spatial positions that each neuron should have, and the connections that should exist between neurons, according to the theoretical structure of the neural network (such as a topology diagram).

[0035] The principle for a neural network splitting device to divide multiple neurons into multiple macrogroups can be as follows: neurons with connections should be placed in the same macrogroup as much as possible; the connections between neurons in different macrogroups should be minimized; and neurons in close proximity should be placed in the same macrogroup or adjacent macrogroups as much as possible.

[0036] For example, neurons connected to a large number of neurons and the neurons connected to them can be grouped into the same macrogroup, while neurons connected to a small number of neurons and the neurons connected to them can be grouped into different macrogroups; non-sparse neurons (i.e., neurons that frequently send information to or receive information from the neurons connected to them) and the neurons connected to them can be grouped into the same macrogroup, while sparse neurons (i.e., neurons that do not frequently send information to or receive information from the neurons connected to them) and the neurons connected to them can be grouped into different macrogroups.

[0037] There is a one-to-one correspondence between macrogroups and computational units. Neurons in the same macrogroup are used to map to one computational unit for processing, while neurons in different macrogroups are used to map to different computational units for processing. Adjacent macrogroups refer to macrogroups whose corresponding computational units are adjacent (i.e., the physical channel between computational units is shorter and the information transmission rate is faster).

[0038] Each computing unit is a relatively independent device or component with computing capabilities, which can map and process a portion of the neurons in a neural network; for example, each computing unit can be a server.

[0039] Compared to related technologies, neural network decomposition devices, when dividing multiple neurons into multiple macrogroups, not only consider the positional relationship between neurons, placing neurons with similar positions in the same or adjacent macrogroups as much as possible, but also consider the connection relationship between neurons, placing neurons with connections in the same macrogroup as much as possible, and minimizing the number of connections between neurons in different macrogroups.

[0040] In other words, during the operation of a neural network, neurons that transmit and receive information should ideally be in the same macrogroup, and neurons in different macrogroups should ideally not transmit information to each other. This ensures that information transmission occurs within the same macrogroup.

[0041] Furthermore, since macrogroups and computational units are in one-to-one correspondence, that is, neurons that transmit and receive information are mapped to one computational unit for processing, and there is as little information transmission and reception between neurons mapped to different computational units. This ensures that the vast majority of information transmission and reception between neurons occurs within the same computational unit, and there is very little information transmission and reception between different computational units.

[0042] The cost and time required for information transmission within the same computing unit are much less than those required for information transmission between different computing units. Obviously, compared with the existing technology that does not consider the connection relationship between neurons and splits the neural network according to information such as the layer of the neural network neurons, the neural network splitting method of this disclosure embodiment can significantly reduce the cost and time required for information transmission between the various parts of the neural network after completing the splitting and mapping of the neural network.

[0043] Figure 2 A flowchart of another neural network splitting method provided in an embodiment of this disclosure.

[0044] Reference Figure 2 This disclosure provides a method for splitting a neural network, the method comprising:

[0045] S201. Based on the positional and connectivity relationships of multiple neurons in the neural network, the multiple neurons are divided into multiple macrogroups.

[0046] The neural network splitting device acquires the positional and connectivity relationships of neurons in a neural network and, based on these relationships, divides the neurons into multiple macrogroups, each of which contains multiple neurons.

[0047] S202. Based on the positional relationship of multiple neurons in each macrogroup and the load information of the corresponding computing unit, the multiple neurons of the macrogroup are divided into multiple subgroups.

[0048] The neural network splitting device divides multiple neurons of a neural network into multiple macrogroups, and then divides multiple neurons of each macrogroup into multiple subgroups, each subgroup including at least one neuron.

[0049] In this process, neurons in the same subgroup are mapped to a sub-operation unit of the operation unit corresponding to their macrogroup for processing, and neurons in different subgroups of the same macrogroup are mapped to different sub-operation units of the operation unit corresponding to that macrogroup for processing.

[0050] Each sub-operation unit is a relatively independent computational device within an operation unit that can map and process a portion of the neurons in a neural network, such as a processing core in a many-core system.

[0051] The neural network decomposition device divides multiple neurons of each macrogroup into multiple subgroups based on the positional relationship of the multiple neurons in the macrogroup and the load information of the corresponding computing unit.

[0052] For example, a neural network splitting device can divide multiple neurons in a macrogroup into multiple subgroups according to the positional relationship of the neurons, but it should ensure that the maximum amount of resources required by the neurons in each subgroup during the operation of the neural network does not exceed the load information of the sub-operation unit corresponding to that subgroup.

[0053] For example, the load information of a sub-operation unit may include at least one of memory, transmission bandwidth, and computing power. For instance, when the load information of a sub-operation unit includes transmission bandwidth, since data transmission may be required between sub-operation units, when dividing multiple neurons of a macrogroup into multiple subgroups, the transmission bandwidth required for data transmission between neurons in one subgroup and neurons in other subgroups is less than or equal to the maximum transmission bandwidth of the sub-operation unit corresponding to that subgroup. This disclosure does not limit the type of load information for sub-operation units or the limitations on each type of load information.

[0054] Figure 3 A flowchart of yet another neural network decomposition method provided in this disclosure embodiment.

[0055] Reference Figure 3 This disclosure provides a method for splitting a neural network, the method comprising:

[0056] S301. Based on the positional relationship of multiple neurons in the neural network, construct a heatmap of the connection density of multiple neurons in the neural network.

[0057] After obtaining the positional and connectivity relationships of neurons in a neural network, the neural network decomposition device first constructs a heatmap of the connection density of multiple neurons in the neural network based on their positional relationships and connectivity relationships.

[0058] In this heatmap, the heat value at any location represents the connection density of the neuron at that location.

[0059] In other words, at locations with high thermal values ​​in the connection density heatmap, the neurons corresponding to these locations are mostly non-sparse neurons. These neurons are mostly connected, and they frequently send information to or receive information from the neurons they are connected to.

[0060] In locations with low thermal values ​​on the connection density heatmap, the neurons at these locations are mostly sparse neurons. These neurons have fewer connections and rarely send or receive information from their connected neurons.

[0061] In other words, the heat values ​​of the connection density heatmap represent the degree to which neurons participate in computation and information transmission during the operation of the neural network. The higher the heat value, the greater the probability that the neurons at the location will participate in computation and information transmission during the operation of the neural network, and the more important the computation they participate in. Conversely, the lower the heat value, the lower the probability that the neurons at the location will participate in computation and information transmission during the operation of the neural network, and the less important the computation they participate in.

[0062] For reference Figure 5 The first image is a heatmap of the connection density of some neurons in a neural network. The heat values ​​of different shaded areas are different. The heat value of the bidirectional shaded area in the middle of the shaded area is the highest. Taking these areas as the center and moving outward, the heat value of the right-side shaded area is lower than that of the bidirectional shaded area but higher than that of the left-side shaded area. The heat value of the left-side shaded area is higher than that of the blank area.

[0063] The neurons corresponding to the bidirectional shaded regions are mostly non-sparse neurons, which need to participate in calculations and transmit information during the operation of the neural network, and the calculations they participate in are relatively important. The connection density of the neurons corresponding to the right oblique shaded regions is less than that of the neurons corresponding to the bidirectional shaded regions, and their computational workload may be less than that of the neurons corresponding to the bidirectional shaded regions. Similarly, the computational workload of the neurons corresponding to the left oblique shaded regions may be less than that of the neurons corresponding to the right oblique shaded regions.

[0064] S302. Based on the heat value distribution of the connection density heatmap, divide the connection density heatmap into multiple regions, and determine that all neurons corresponding to each region are multiple neurons included in a macrogroup.

[0065] The neural network splitting device divides the connection density heatmap into multiple regions based on the heat value distribution of the generated connection density heatmap.

[0066] For example, refer to Figure 5 The neural network splitting device divides the connection density heatmap into multiple regions, centered on the high-value region (i.e., the region corresponding to the bidirectional shaded area in the first image) and with the low-value region (i.e., the blank area in the first image) as the edges (as shown in the reference). Figure 5 The second image is divided into 3 regions, and the multiple neurons corresponding to each region are the multiple neurons included in a macrogroup.

[0067] Since the connection density heatmap is generated according to the positional relationship of neurons in the neural network, that is, the neurons corresponding to adjacent positions in the connection density heatmap are also adjacent neurons in the neural network, the neural network splitting device divides the region with the high heat value region in the connection density heatmap as the center, which ensures that the neurons corresponding to the high heat value region and the neurons adjacent to these neurons (or the connected neurons) are in the same macrogroup. The neural network splitting device divides the region with the low heat value region in the connection density heatmap as the edge, which ensures that the neurons corresponding to the edge of the region have few connections with the neurons adjacent to them, and that they do not frequently send or receive information, which also ensures that the connections and information transmission between macrogroups are relatively few.

[0068] S303. Based on the heat value distribution of the region of the connection density heatmap corresponding to each macrogroup and the load information of each sub-operation unit of the operation unit corresponding to the macrogroup, the region of the connection density heatmap corresponding to the macrogroup is divided into multiple sub-regions, and all neurons corresponding to each sub-region are determined to be at least one neuron included in a subdivision group.

[0069] After dividing the connection density heatmap into multiple regions, the neural network decomposition device further subdivides each region into multiple sub-regions.

[0070] Each sub-region corresponds one-to-one with a sub-group, and the neurons corresponding to each sub-region are the neurons included in the sub-group corresponding to that sub-region.

[0071] The neural network decomposition device divides each region into multiple subgroups based on the thermal value distribution of each region and the load information of each sub-operation unit of the operation unit corresponding to the macrogroup of that region.

[0072] For example, the load information of each sub-operation unit may include the memory and computing power of that sub-operation unit.

[0073] The neural network decomposition device can divide each region into multiple sub-regions according to spatial relationships based on the location information of neurons (as shown in the reference). Figure 5 The third image in the picture is... Figure 5 The area in the lower left corner of the second image is divided into multiple sub-regions. These sub-regions are created by using multiple perpendicular lines to divide the area into sub-regions of varying sizes, with each small polygon representing a sub-region.

[0074] Due to the average thermal value of each sub-region This represents the average connection density of neurons corresponding to that sub-region. The higher the connection density of neurons, the greater the computational load corresponding to the neurons during the operation of the neural network, and the higher the requirements for the computational power of the computing unit.

[0075] Therefore, given that the load information of each sub-operation unit includes its computational power, the neural network partitioning device can divide each region into multiple sub-regions by the area S of each sub-region. i (i is the sub-region number) and the average thermal value of that sub-region The computational power required by the neurons corresponding to this sub-region during neural network operation is calculated. Furthermore, the neural network splitting device can also utilize the total computational power P of the sub-operation units corresponding to this sub-region. max The computational power in the load information of the corresponding sub-operation unit of the sub-region is used as a constraint condition for sub-region division, which is that the computational power required for the neural network to run using the calculated computation is no greater than the total computational power of the sub-operation units.

[0076] For example, the area S of each sub-region can be used. i (i is the sub-region number) and the average thermal value of that sub-region The product of the three factors—the first preset coefficient α (α being a constant)—should not exceed the total computing power P of the sub-operation unit corresponding to that sub-region. max That is, the formula As a constraint on the division of sub-regions.

[0077] Where i is the sub-region number, which is a positive integer, and α is a constant, specifically an adjustment coefficient, which can be flexibly set according to the actual situation of the neural network. S is the average thermal value of subregion i. i Let P be the area of ​​the subregion numbered i. max This represents the total computational power of the sub-operation units corresponding to the sub-region (i.e., the sub-region numbered i).

[0078] Similarly, since the connection density of neurons also represents the number of synapses of the corresponding neurons, and the number of synapses of neurons is directly proportional to the memory required during the operation of the neural network, that is, the higher the connection density of neurons, the greater the memory required by neurons during the operation of the neural network, and the higher the memory requirement of the computing unit.

[0079] Therefore, when the load information of each sub-operation unit includes the memory of that sub-operation unit, the neural network splitting device, in the process of dividing each region into multiple sub-regions, uses the area S of each sub-region... i The average thermal value of this sub-region The device can also calculate the memory required by the neurons corresponding to the sub-region during neural network operation. Furthermore, the neural network splitting device can also use the total memory M of the sub-operation units corresponding to the sub-region. max The memory used in the load information of the corresponding sub-operation unit of the sub-region is determined by the constraint that the memory required for the neural network to run is no greater than the total memory of the sub-operation unit.

[0080] For example, the area S of each sub-region can be used. i The average thermal value of this sub-region The product of the second preset coefficient β (β is a constant) and the preset value is not greater than the total memory M of the sub-operation unit corresponding to the sub-region. max That is, the formula As a constraint on the division of sub-regions.

[0081] Where i is the sub-region number, which is a positive integer; β is a constant, which, like α, is an adjustment coefficient that can be flexibly set according to the actual situation of the neural network. S represents the average thermal value of the subregion numbered i; i M is the area of ​​the subregion numbered i; max This is the total memory of the sub-operation unit corresponding to the sub-region (i.e., the sub-region numbered i).

[0082] In summary, when the load information of each sub-operation unit includes both its computational power and its memory, the neural network splitting device can use the formula... As a constraint on the division of sub-regions.

[0083] Where i is the sub-region number, which is a positive integer, α is a constant, and β is a constant. S is the average thermal value of subregion i. i Let P be the area of ​​the subregion numbered i. max M represents the total computational power of the sub-operation units corresponding to this sub-region (i.e., the sub-region numbered i). max This represents the total memory of the sub-operation unit corresponding to this sub-region.

[0084] In some embodiments, the sum of thermal values ​​at all locations in a subregion can be calculated using a two-dimensional integral of the thermal value over the subregion to define the division of the subregion.

[0085] That is, when the load information of each sub-operation unit includes the computing power of that sub-operation unit, the neural network decomposition device can use the formula γ×∫∫ (x，y)∈S H(x,y)dxdy≤P max This serves as a constraint for subregion partitioning.

[0086] When the load information of each sub-operation unit includes the memory of that sub-operation unit, the neural network decomposition device can use the formula δ×∫∫ (x，y)∈S H(x,y)dxdy≤M max This serves as a constraint for subregion partitioning.

[0087] When the load information of each sub-operation unit includes both its computational power and its memory, the neural network decomposition device can use the formula... As a constraint on the division of sub-regions.

[0088] Among them, ∫∫ (x，y)∈S H(x, y)dxdy represents the two-dimensional integral of the thermal value over subregion S (i.e., subregion i), and the result represents the sum of the thermal values ​​at all locations in subregion S. γ is a preset third coefficient and is a constant, δ is a preset fourth coefficient and is a constant, and P... max M represents the total computational power of the sub-operation units corresponding to this sub-region (i.e., the sub-region numbered i). max This represents the total memory of the sub-operation unit corresponding to this sub-region.

[0089] The two-dimensional integral of the thermal value over a sub-region represents the sum of the thermal values ​​at all locations in that sub-region. The higher the sum of the thermal values ​​at all locations in that sub-region, the greater the connection density of the neurons corresponding to that sub-region, and the greater the computational load for the neurons corresponding to that sub-region.

[0090] In summary, the computational power required for a sub-region increases with the two-dimensional integral of the thermal value over the sub-region. Therefore, the constraint condition for sub-region partitioning can be that the product of the two-dimensional integral of the thermal value over the sub-region and the third preset coefficient γ is not greater than the computational power of the sub-operation unit corresponding to that sub-region. Here, γ is also an adjustment coefficient, which can be flexibly set according to the actual situation of the neural network.

[0091] Similarly, the greater the connection density of neurons corresponding to a sub-region, the more synapses the neurons in that sub-region will have, and the more memory will be required. In other words, the memory required for a sub-region increases with the two-dimensional integral of the heat value over the sub-region. Therefore, the product of the two-dimensional integral of the heat value over the sub-region and the fourth preset coefficient δ can be used as a constraint condition for sub-region division, where the product is no greater than the memory of the sub-operation unit corresponding to the sub-region. Here, δ is also an adjustment coefficient, which can be flexibly set according to the actual situation of the neural network.

[0092] After the neural network partitioning device further divides the network into sub-regions, it can assign the numbers of the neurons corresponding to the sub-regions to their corresponding sub-operation units, so that the corresponding neurons can be mapped to the sub-operation units for processing.

[0093] Figure 4 A flowchart illustrating some steps of a neural network decomposition method provided in this embodiment of the disclosure.

[0094] Reference Figure 4 The present disclosure provides a method for splitting a neural network, which includes a step (step S301) of constructing a connection density heatmap of multiple neurons in the neural network based on the positional relationships of multiple neurons and the connection relationships of multiple neurons in the neural network.

[0095] S401. Based on the positional relationship of multiple neurons in the neural network, construct a plane diagram of the connection relationship of multiple neurons in the neural network.

[0096] The neural network decomposition device constructs a connection relationship plane diagram of the multiple neurons of the neural network according to the positional relationship (specifically, spatial positional relationship) of the multiple neurons of the neural network. In the connection relationship plane diagram, the connecting lines represent the connections between neurons. The more connecting lines corresponding to a position, the more connections there are between the neurons corresponding to that position.

[0097] S402. Determine the connection density of neurons at each location in the connection relationship plane diagram.

[0098] The neural network splitting device determines the connection density of neurons at each position in the connection plane diagram based on the connection plane diagram.

[0099] In the connectivity diagram, the connection density of a neuron at each position can be obtained by counting the number of fan-in connections and fan-out connections of all neurons at that position. The number of fan-in connections for each neuron is the number of its predecessor neurons (i.e., neurons whose output is input to the neuron in a certain proportion), and the number of fan-out connections for each neuron is the number of its successor neurons (i.e., neurons whose output is input to the neuron in a certain proportion).

[0100] S403. Construct a heatmap of the connection density of multiple neurons in a neural network based on the connection density of neurons at each position in the connection relationship plane diagram.

[0101] After obtaining the connection density of neurons at each position in the connection relationship plane, the neural network splitting device determines the heat value of each position based on the connection density of neurons at each position, and constructs a connection density heat map based on the heat value of each position.

[0102] Of course, the neural network splitting device can also construct the connection density heatmap in other ways, which will not be described in detail here.

[0103] Continue to refer to Figure 4 The step of dividing a connection density heatmap into multiple regions based on the heat value distribution of the connection density heatmap according to an embodiment of the present disclosure (step S302) includes:

[0104] S404. Determine multiple thermal value boundaries based on the thermal value distribution of the connection density heatmap, and divide the connection density heatmap into multiple regions based on the thermal value boundaries.

[0105] After acquiring the connection density heatmap, the neural network splitting device determines multiple heat value boundaries based on the heat value distribution of the connection density heatmap, and divides the connection density heatmap into multiple regions based on the heat value boundaries.

[0106] For example, the neural network splitting device determines the heat value division threshold based on the heat value distribution of the connection density heatmap. The position where the heat value is less than the heat value division threshold can be used as the region boundary. Multiple heat value boundaries are determined based on the heat value division threshold. These heat value boundaries are the region boundaries. The connection density heatmap is divided into multiple regions based on these region boundaries.

[0107] Of course, the neural network splitting device can also divide the connection density heatmap into multiple regions in other ways, which will not be described in detail here.

[0108] Figure 6 This is a block diagram of a neural network splitting device provided in an embodiment of the present disclosure.

[0109] Reference Figure 6 This disclosure provides a neural network splitting device 600, which includes:

[0110] The splitting module 601 is used to divide multiple neurons into multiple macrogroups according to the positional and connection relationships of multiple neurons in the neural network; wherein each macrogroup includes multiple neurons, neurons in the same macrogroup are used to map to one operation unit for processing, and neurons in different macrogroups are used to map to different operation units for processing.

[0111] This disclosure also provides a processing core that includes the neural network splitting device described above.

[0112] Figure 7 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0113] Reference Figure 7 This disclosure provides an electronic device that includes multiple processing cores 701 and an on-chip network 702. The multiple processing cores 701 are all connected to the on-chip network 702, which is used to exchange data between the multiple processing cores and external data.

[0114] One or more processing cores 701 store one or more instructions, and the one or more instructions are executed by one or more processing cores 701 to enable one or more processing cores 701 to execute the neural network decomposition method described above.

[0115] Furthermore, this disclosure also provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processing core, implements the aforementioned neural network decomposition method.

[0116] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0117] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for decomposing a neural network, comprising: Based on the positional and connectivity relationships of multiple neurons in a neural network, the neurons are divided into multiple macrogroups. Each macrogroup includes multiple neurons. Neurons in the same macrogroup are mapped to one processing unit for processing, while neurons in different macrogroups are mapped to different processing units for processing. The step of dividing multiple neurons into multiple macrogroups based on their positional and connectivity relationships within the neural network includes: Based on the positional relationship of multiple neurons in the neural network, a connection density heatmap of multiple neurons in the neural network is constructed according to the connection relationship of multiple neurons in the neural network. The heat value of the connection density heatmap at any position represents the connection density of the neuron corresponding to that position. Based on the thermal value distribution of the connection density heatmap, the connection density heatmap is divided into multiple regions, and all neurons corresponding to each region are determined to be multiple neurons included in a macrogroup.

2. The neural network decomposition method according to claim 1, wherein, The step of constructing a connection density heatmap of the multiple neurons of the neural network based on their positional relationships and connection relationships includes: Based on the positional relationships of the multiple neurons in the neural network, a connection relationship diagram of the multiple neurons in the neural network is constructed according to the connection relationships of the multiple neurons in the neural network. The connection density of the neuron corresponding to each position in the connection relationship plane is determined. The connection density of the neuron corresponding to each position is determined based on the number of fan-in connections and the number of fan-out connections of the neuron corresponding to that position. The number of fan-in connections of each neuron is the number of predecessor neurons connected to that neuron, and the number of fan-out connections of each neuron is the number of successor neurons connected to that neuron. Based on the connection density of neurons at each position in the connection relationship plane diagram, a heatmap of the connection density of multiple neurons in the neural network is constructed.

3. The neural network decomposition method according to claim 1, wherein, The step of dividing the connection density heatmap into multiple regions based on the heat value distribution of the connection density heatmap includes: Based on the thermal value distribution of the connection density heatmap, multiple thermal value boundaries are determined, and the connection density heatmap is divided into multiple regions based on the thermal value boundaries.

4. The neural network decomposition method according to claim 1, wherein, After dividing the multiple neurons into multiple macrogroups based on their positional and connectivity relationships within the neural network, the process further includes: Based on the positional relationship of multiple neurons in each macrogroup and the load information of the corresponding computation unit, the multiple neurons of the macrogroup are divided into multiple subgroups; each subgroup includes at least one neuron, and the neurons in the same subgroup are used to map to a subcomputation unit of the computation unit corresponding to the macrogroup for processing, while the neurons in different subgroups are used to map to different subcomputation units of the computation unit corresponding to the macrogroup for processing.

5. The neural network decomposition method according to claim 4, wherein, The load information includes transmission bandwidth. The step of dividing the multiple neurons of a macrogroup into multiple subgroups based on the positional relationship of the multiple neurons in each macrogroup and the load information of the corresponding computational unit includes: When dividing multiple neurons in each macrogroup into multiple subgroups, the transmission bandwidth required for data transmission between neurons in one subgroup of the macrogroup and neurons in other subgroups of the macrogroup is less than or equal to the maximum transmission bandwidth of the suboperation unit corresponding to that subgroup.

6. The neural network decomposition method according to claim 1, wherein, After classifying the multiple neurons into multiple macrogroups based on their positional and connectivity relationships within the neural network, the process further includes: Based on the heat value distribution of the region of the connection density heatmap corresponding to each macrogroup and the load information of each sub-operation unit of the operation unit corresponding to the macrogroup, the region of the connection density heatmap corresponding to the macrogroup is divided into multiple sub-regions, and all neurons corresponding to each sub-region are determined to be at least one neuron included in a subdivision group; the heat value of the connection density heatmap at any position represents the connection density between neurons at that position.

7. The neural network decomposition method according to claim 6, wherein, The load information includes computing power and / or memory; When the load information includes computing power, the product of the first preset coefficient, the area of ​​each sub-region, and the average thermal value of the sub-region is not greater than the total computing power of the sub-computing unit corresponding to the sub-region, wherein the first preset coefficient is an adjustment coefficient and the first preset coefficient is a constant. When the load information includes memory, the product of the second preset coefficient, the area of ​​each sub-region, and the average thermal value of the sub-region is not greater than the total memory of the sub-operation unit corresponding to the sub-region, wherein the second preset coefficient is an adjustment coefficient and is a constant.

8. The neural network decomposition method according to claim 6, wherein, The load information includes computing power and / or memory; When the load information includes computing power, the product of the two-dimensional integral of the thermal value in each sub-region and the third preset coefficient is not greater than the total computing power of the sub-operation unit corresponding to that sub-region, wherein the third preset coefficient is an adjustment coefficient and the third preset coefficient is a constant; When the load information includes memory, the product of the two-dimensional integral of the thermal value in each sub-region and the fourth preset coefficient is not greater than the total memory of the sub-operation unit corresponding to that sub-region, wherein the fourth preset coefficient is an adjustment coefficient and is a constant.

9. A neural network decomposition device, comprising: The splitting module is used to divide multiple neurons into multiple macrogroups based on their positional and connectivity relationships. Each macrogroup includes multiple neurons, and neurons in the same macrogroup are mapped to one processing unit for processing, while neurons in different macrogroups are mapped to different processing units for processing. The step of dividing multiple neurons into multiple macrogroups based on their positional and connectivity relationships within the neural network includes: Based on the positional relationship of multiple neurons in the neural network, a connection density heatmap of multiple neurons in the neural network is constructed according to the connection relationship of multiple neurons in the neural network. The heat value of the connection density heatmap at any position represents the connection density of the neuron corresponding to that position. Based on the thermal value distribution of the connection density heatmap, the connection density heatmap is divided into multiple regions, and all neurons corresponding to each region are determined to be multiple neurons included in a macrogroup.

10. An electronic device, comprising: Multiple processing cores; as well as The on-chip network is configured to interact with data between the multiple processing cores and external data; One or more processing cores store one or more instructions, and the one or more instructions are executed by one or more processing cores to enable the one or more processing cores to perform the neural network splitting method of any one of claims 1-8.

11. A computer-readable medium having a computer program stored thereon, wherein, The computer program, when executed by the processing kernel, implements the neural network splitting method as described in any one of claims 1-8.

Images